Table of Contents
A virtual lab is a set of machine-learning models, trained on your own past formulations and test results, that predicts how a new recipe will perform before anyone weighs a gram. It ranks candidates against several targets and proposes the next few experiments worth running, so the physical lab spends its time confirming strong ideas instead of searching blindly.
Key takeaways
- A virtual lab combines three things: a model that predicts test results from a recipe, an optimiser that chooses the next experiments, and a structured record of every trial.
- It is not a digital twin. A twin mirrors one physical asset in operation. A virtual lab explores recipes and conditions that do not exist yet.
- It does not replace design of experiments. It adds a sequential loop in which each batch of tests is chosen using everything learned so far.
- The models are only reliable inside the range of the data they were trained on. Truly new chemistry still needs the lab first.
- Benefit should be measured against a baseline fixed before the project starts, such as experiments and days needed to reach a specification.
What is a virtual lab in industrial R&D?
Most formulation work follows the same pattern. A chemist or process engineer starts from a reference recipe, changes a few components or dosages, runs the standard tests, reads the results and decides what to try next. In drilling fluids the tests are rheology, fluid loss and stability under temperature. In detergents they are cleaning performance, foam and stability. In cement and concrete, compressive strength at several ages.
A virtual lab puts a model in the middle of that loop. The model has learned, from the company's own records, how the inputs of a recipe relate to the measured outputs. Given a candidate recipe, it returns a predicted result and, in most methods, an estimate of how uncertain that prediction is. An optimiser then uses those predictions to propose which recipes to test next.
These relationships are rarely simple. The public concrete data set donated by I-Cheng Yeh to the UC Irvine Machine Learning Repository, with 1,030 mixes and 8 inputs such as cement, water, slag, fly ash, superplasticiser and age, describes compressive strength as a highly nonlinear function of age and ingredients. That is exactly the kind of relationship a statistical model can learn from data and a person struggles to hold in their head across eight variables at once.
At KABAS AI, Virtual Lab is the product we build for this job, configured for each client's formulation domain.
How is a virtual lab different from a digital twin?
The two terms are often mixed up, and vendors do not always help. The definitions are clear, though.
NIST describes digital twin technology as the creation of electronic representations of real-world entities, with the ability to view their states and the transitions between them. The US National Academies, in a 2024 report, say a digital twin couples computational models with a physical counterpart and is dynamically updated through bidirectional data flows as conditions change.
The key words are "physical counterpart". A digital twin follows one reactor, one compressor or one production line while it runs. It is fed by sensors and is used to monitor, predict and sometimes control that asset. We cover that use in our article on digital twins in manufacturing.
A virtual lab has no single physical counterpart. Its subject is a design space: every recipe you could make from your raw materials, under every condition you might test. Most of those recipes have never existed. Its data comes from the lab bench, not from plant sensors, and it is updated each time a new candidate is tested, not every few seconds.
The two can meet. A formulation chosen in a virtual lab eventually goes into production, where a process twin or a soft sensor may take over. But they answer different questions: "which recipe should we make?" versus "what is happening in this unit right now?"
How is it different from classic design of experiments?
Design of experiments (DoE) is the established statistical method for planning lab work. The NIST/SEMATECH e-Handbook describes it as an efficient procedure for planning experiments so that the data obtained can be analysed to give valid conclusions. Factorial, response-surface and mixture designs are standard tools, and they work.
DoE has two practical limits in formulation work.
First, the number of runs grows fast. A full two-level factorial with six factors requires 64 runs, which is why fractional designs exist. Real formulations often have more than six adjustable components, plus process conditions.
Second, the plan is fixed before the first test. Results from run 5 do not change what run 6 will be. Formulations also carry a constraint DoE handles with special designs: in a mixture design the components are proportions that must sum to one, so changing one changes the others.
A virtual lab keeps the statistical discipline of DoE but works sequentially. It starts from existing history, or from a small DoE if history is thin, then proposes a few experiments, learns from their results and proposes the next few. It also reuses data from past projects that were never part of a designed experiment, which DoE ignores.
How do the three approaches compare?
| Classic design of experiments | Digital twin | Virtual lab | |
|---|---|---|---|
| Main question | Which factors matter, and how? | What is this asset doing now, and next? | Which recipe should we test next? |
| Subject | A planned set of experiments | One physical asset or process in operation | A design space of recipes and conditions |
| Data source | The designed runs only | Live sensor and process data | Historical lab records plus each new test |
| How the plan evolves | Fixed in advance | Continuous update from the asset | Batch by batch, after each round of tests |
| Uses past projects | Rarely | Uses the asset's own history | Yes, as the starting point |
| Typical user | Statistician, R&D engineer | Operations, maintenance, process control | Formulator, lab head, R&D manager |
What methods sit behind a virtual lab?
Three ideas do most of the work. None of them is new, and all are well documented.
Surrogate models: predicting the test result
A surrogate model is a fast statistical stand-in for a slow or expensive experiment. It takes a recipe and conditions as input and predicts the measured properties.
Common choices are Gaussian processes, random forests and neural networks. Gaussian processes, covered in the standard reference by Rasmussen and Williams, return both a prediction and an uncertainty estimate, which is exactly what the search step described below needs.
Drilling fluids give a concrete example of how much a modest data set can carry. A 2020 study by Gowida and colleagues at King Fahd University of Petroleum and Minerals used 200 field data points of mud density and Marsh funnel viscosity to train neural networks that predict plastic viscosity, yield point and apparent viscosity of a high-bentonite mud. They reported a correlation coefficient above 0.90 and a maximum average absolute percentage error of 6%. That study predicts properties from field measurements rather than designing new recipes, but it shows the scale of data at which useful models start to appear in this domain.
Bayesian optimisation: choosing the next experiment
A surrogate model on its own only answers "what would happen if?". Bayesian optimisation turns it into a search. At each step it looks at the model's predictions and uncertainties and picks the candidates with the best balance between "likely to be good" and "we know little about this region". Testing those candidates improves the model, and the loop repeats.
Peter Frazier's tutorial describes the method as designed for objectives that take minutes or hours to evaluate, as tolerant of noisy results, and as best suited to continuous domains of less than 20 dimensions. That matches many lab formulation problems: tests are slow and costly, results vary between repeats, and the number of variables a team actually adjusts is often modest. Problems with many more variables need to be narrowed first.
The method has been tested against people. In a study published in Nature in 2021, Shields and colleagues benchmarked Bayesian optimisation against expert chemists and engineers on a reaction optimisation task, and found that it outperformed human decision-making in both average efficiency, measured as the number of experiments, and consistency. Freely available libraries such as BoTorch, built on PyTorch, make the method accessible to any team with Python skills.
Active learning: improving the model where it is weakest
Active learning is a close relative. Burr Settles' survey states the key idea: a learning algorithm can reach greater accuracy with fewer training labels if it is allowed to choose the data from which it learns, which matters most when labels are expensive. In a lab, the "label" is a test result.
The difference from Bayesian optimisation is the goal. Bayesian optimisation looks for the best recipe. Active learning looks for the experiments that will make the model most accurate across the whole space, which is useful when the team wants a reliable "what-if" tool for future projects, not one optimum.
Several targets at once
Real specifications are never a single number. A drilling fluid must meet rheology, fluid loss and stability targets at an acceptable cost. A detergent must clean, foam correctly, stay stable and meet a price.
Multi-objective methods handle this by finding the set of recipes where no target can be improved without worsening another, and letting the formulator choose among them. Cao and colleagues applied this to a formulated liquid product of commercial interest, coupling a machine-learning classifier with a multi-objective optimiser so that discrete and continuous targets were handled together. Their approach found nine suitable recipes within 15 working days and, according to the authors, outperformed human intuition on the target performance.
One of the best-known demonstrations of the full loop is the mobile robotic chemist built by Burger and colleagues. Guided by a batched Bayesian search, it ran 688 experiments in a ten-variable space over eight days and found photocatalyst mixtures six times more active than the starting formulations.
Both of those studies used robots to run the experiments. Many industrial labs have no such equipment, and that is fine: the same loop works with technicians running the proposed tests. Each round simply takes longer.
What data does a virtual lab need?
The model can only learn what was recorded. In practice the useful records are:
- The recipe: every component, its quantity and unit, and the order of addition when it matters.
- The raw materials: supplier, grade and lot, because the same nominal component can behave differently from batch to batch.
- The conditions: temperature, ageing time, mixing speed and duration, pressure where relevant.
- The results: each measured property, the test method used and the conditions of the test.
- The failures: recipes that separated, gelled, failed a spec or were abandoned. A model trained only on successes cannot learn where the boundaries are.
Most R&D departments have this information, but spread across lab notebooks, spreadsheets, PDF reports and the memories of senior staff. The FAIR principles, published by Wilkinson and colleagues in 2016, are a good checklist for fixing this: data should be findable, accessible, interoperable and reusable, with machine-readable metadata so software as well as people can use it.
How much data is enough? There is no honest universal answer. It depends on the number of variables you change, the noise in your test methods and how far from known recipes you want to go. The examples above range from 200 field points to about 1,000 concrete mixes, but those are reference points, not thresholds. A short audit of existing records, of the kind described in our guide to assessing AI readiness, is the fastest way to find out. Our data science team usually starts there.
Where does a virtual lab work, and where does it not?
A virtual lab is a strong tool for a specific kind of problem. It is not a general replacement for chemistry knowledge.
The most important limit is the training range. In the field of chemical property models, the OECD principles for validating such models require a defined domain of applicability: predictions are generally reliable only for inputs similar to the training data, and anything outside that range is an extrapolation. The same rule applies to any formulation model. If your history covers polymer dosages in one range, the model's opinion about a dosage three times higher is a guess.
| Works well when | Works poorly when |
|---|---|
| Tests are slow, costly or use scarce materials | Tests are cheap and fast enough to screen everything |
| There are several interacting variables | There are one or two variables a simple DoE can map |
| A history of comparable recipes and tests exists | The chemistry is new, with no relevant history |
| Test methods are standardised and repeatable | Results depend heavily on who ran the test |
| The goal is to adapt or optimise within a known family | The goal is a discovery far outside known recipes |
| Specifications combine several targets and cost | Success depends on a property nobody measures |
Two further points matter in industry. First, a model prediction is never a qualification result. Any formulation that goes to a well, a customer or a regulator must still pass the physical tests the specification requires. Second, the model knows nothing about safety unless safety-relevant outputs are measured and modelled. Hazard reviews stay with chemists.
How do you measure the benefit honestly?
This is where many AI projects in R&D become vague. A few rules keep the evaluation honest.
Fix the baseline before you start. Record how many experiments and how many calendar days your team typically needs to bring a comparable formulation to specification. Without this, any later claim is a feeling.
Compare against a fair alternative. Academic benchmarks do this explicitly. Liang and colleagues compared Bayesian optimisation strategies across five experimental materials systems using metrics they call acceleration and enhancement factors, measured against a random sampling baseline. They also found that random forests performed comparably to Gaussian processes with anisotropic kernels, a useful reminder that the model type matters less than the evaluation. In industry, the fairer baseline is your own team's usual method, as in the human comparison by Shields and colleagues.
Check every prediction. Each time the lab tests a recommended recipe, log the predicted value next to the measured one. Over a project, this gives a real error rate on new data, not on data the model has already seen.
Count everything. Include failed experiments, repeated tests and the time spent preparing data. A virtual lab that saves lab runs but needs weeks of data cleaning per project has a different cost profile from one that does not.
| Metric | What it tells you | How to measure it |
|---|---|---|
| Experiments to target | Lab effort saved | Count all tests until a recipe meets the full specification |
| Calendar days to target | Speed of response to a request | Days from specification to validated recipe |
| Prediction error on new tests | Whether the model can be trusted | Predicted vs measured value for each recommended recipe |
| Share of recommended recipes in spec | Quality of suggestions | Recipes meeting spec divided by recipes tested |
| Material and consumables used | Direct cost | Lab consumption per project, before and after |
| Reuse of past work | Knowledge capitalisation | How often new projects start from stored recipes and models |
Our guide to building an industrial AI business case explains how to turn these measures into a financial case.
What we have seen at KABAS AI
We have deployed Virtual Lab in two formulation environments that we can describe publicly.
Drilling fluids at ADFC (Sonatrach Group). ADFC develops and supplies drilling fluid systems for wells across Algeria's geological formations. As described in the ADFC case study, its R&D team faced multi-section wells with distinct pressure and temperature profiles, raw material variability between batches and suppliers, high testing costs, and formulation knowledge dispersed across formats and locations. Virtual Lab was configured for drilling fluids, with multi-criteria optimisation across rheological performance, fluid loss control, wellbore stability, environmental compliance and cost, and "what-if" scenarios for component substitutions and dosage changes. The workflow runs in four phases: define targets and simulate from historical reference recipes, generate a focused laboratory test playbook, compare simulated and actual results, then store validated formulations as reusable data. The case study reports a significant reduction in physical testing cycles, faster responses to technical specifications and complete traceability of formulation decisions, with the platform deployed inside Sonatrach's IT environment.
Detergents at ENAD Shymeca. ENAD Shymeca, a subsidiary of ACS Holding, is the national leader in detergents and cleaning products. The ENAD Shymeca case study describes time-intensive formulation cycles, cost pressure, documentation demands for compliance and the need to capture knowledge from experienced researchers approaching retirement. The project went through discovery, configuration for detergent-specific parameters and test protocols, migration of historical formulation data, training of R&D staff, and continuing support. The optimisation balances objectives such as cleaning efficacy against cost and foam characteristics against environmental impact. The partnership is the first Open Innovation contract between an AI startup and a major national industrial company in Algeria.
The lesson common to both: the first piece of work was not the model. It was turning years of scattered formulation records into a structured, reusable base. That step is what makes the predictions possible and what keeps the knowledge in the company when people leave.
How do you start a virtual lab project, step by step?
- Choose one product family and one decision. For example, "adapt our water-based mud to a new clay inhibitor supplier" or "reformulate a liquid detergent to a lower cost". A narrow first scope gives a clear answer.
- Write the specification. List every target property, its acceptable range, the test method, and the hard constraints such as banned components or maximum cost.
- Audit and collect the history. Gather recipes, raw material details, conditions and results, including failures, for that family. Note gaps and inconsistent test methods.
- Fix the baseline. Record how many experiments and days similar projects needed in the past.
- Build and check a first model. Train a surrogate on part of the history and test it on recipes it has not seen. If the error is too large for decisions, collect a small designed set of experiments before going further.
- Run a few optimisation rounds. Let the optimiser propose small batches, have the lab run them, feed the results back. Keep the formulators in charge of what is actually tested.
- Review against the baseline. Compare experiments, days and prediction error with the figures from step 4, and decide whether to extend to other families.
- Capitalise. Store validated recipes, models and test results so the next project starts from them.
Because formulations are core intellectual property, decide early where the data and models will live. Our guide to sovereign, on-premise AI covers the options. For a structured first assessment, our AI consulting team can scope a pilot with your R&D department, and the chemistry and oil and gas solution pages show how Virtual Lab fits each sector. Related reading: AI process control in chemical manufacturing.
Sources
- UCI Machine Learning Repository: Concrete Compressive Strength data set (Yeh)
- NIST IR 8356: Considerations for Managing the Security of Digital Twin Technology
- National Academies: Foundational Research Gaps and Future Directions for Digital Twins (2024)
- NIST/SEMATECH e-Handbook: What is design of experiments?
- NIST/SEMATECH e-Handbook: Fractional factorial designs
- NIST/SEMATECH e-Handbook: Mixture designs
- Rasmussen and Williams: Gaussian Processes for Machine Learning (MIT Press, 2006)
- Gowida et al.: Newly Developed Correlations to Predict the Rheological Parameters of High-Bentonite Drilling Fluid Using Neural Networks, Sensors (2020)
- Frazier: A Tutorial on Bayesian Optimization (2018)
- Shields et al.: Bayesian reaction optimization as a tool for chemical synthesis, Nature (2021)
- BoTorch: Bayesian Optimization in PyTorch
- Settles: Active Learning Literature Survey, University of Wisconsin–Madison
- Cao et al.: Optimization of Formulations Using Robotic Experiments Driven by Machine Learning DoE, Cell Reports Physical Science
- Burger et al.: A mobile robotic chemist, Nature (2020)
- GO FAIR Foundation: FAIR Principles
- Sahigara et al.: Comparison of Different Approaches to Define the Applicability Domain of QSAR Models, Molecules (2012)
- Liang et al.: Benchmarking the performance of Bayesian optimization across multiple experimental materials science domains (2021)
Frequently asked questions
What is a virtual lab in industrial R&D?
A virtual lab is a set of machine-learning models trained on a company's past recipes and test results. It predicts how a new formulation will perform, ranks candidates against several targets at once and proposes the next experiments worth running. The physical lab is still used, but to confirm the most promising candidates rather than to search blindly.
Is a virtual lab the same as a digital twin?
No. A digital twin is coupled to a specific physical asset or process and is updated with data from it as conditions change. A virtual lab models a design space of recipes and conditions, most of which have never been made, and is updated each time the lab tests a new candidate.
Does a virtual lab replace design of experiments?
No. Classic design of experiments is still a sound way to collect the first data set, especially when little history exists. The difference is that a virtual lab chooses experiments one batch at a time, using everything learned so far, instead of fixing the whole plan in advance.
How much data do we need to start?
There is no universal minimum, because it depends on how many variables you change and how noisy your test methods are. What matters most is that recipes, raw material details, conditions and results, including failed trials, are recorded consistently. A short audit of existing lab records usually shows whether a first model is realistic.
How do we know the virtual lab is actually helping?
Fix a baseline before the project starts, such as the number of experiments and calendar days your team usually needs to reach a target specification. Then track the same measures with the virtual lab, check every model prediction against the real lab result, and count all experiments, including the ones that failed.
Can a virtual lab run on our own servers?
Yes. The models used in a virtual lab are small compared with large language models and can run on-premise. For companies whose formulations are core intellectual property, keeping recipes and test data inside their own infrastructure is often a requirement rather than an option.



