Table of Contents
A soft sensor is a software model that estimates a process variable you cannot measure online, such as product purity or melt index, from variables you already measure continuously, such as temperatures, pressures and flows. It runs in the control system and gives a value every few seconds or minutes, instead of waiting hours for a lab result.
Key takeaways
- A soft sensor turns easy measurements into a continuous estimate of a quality variable that is hard or too costly to measure online (Lawrence et al., 2024).
- Lab results arrive late. Polypropylene melt flow index is measured only every 2 to 4 hours (Farsang et al., 2015), and many quality samples are taken once per 8-hour shift or once a day (Lawrence et al., 2024).
- There are three families: model-based (first principles), data-driven, and hybrid. Data-driven models dominate research, but practitioners usually start with a linear model.
- The lab stays in the loop as the reference for validation and for bias updates.
- Soft sensors fail quietly through input sensor faults, process drift and operation outside the training range.
What is a soft sensor?
In a refinery, chemical plant, cement plant or concentrator, the variables that matter most commercially are often the hardest to measure: product purity, polymer melt index, clinker free lime, concentrate grade. A recent survey describes them as variables that are "difficult to measure online due to technological limitations or the high cost of sensors" (Lawrence et al., 2024).
A soft sensor is a mathematical model that estimates such a variable continuously from measurements that are cheap and reliable. "Soft" means software: a calculation in the DCS, the advanced process control layer or a server connected to the historian, not a new field instrument. The same idea is also called an inferential sensor or virtual sensor. According to the same survey, refineries, steel plants, polymer producers and cement plants remain the dominant users.
Why are lab measurements too slow for process control?
The lab is accurate but slow: a sample is drawn, carried, analysed and entered while the unit keeps producing. Published examples show the gap.
| Quality variable | Process | Reference measurement | Delay or interval |
|---|---|---|---|
| Butane content in debutanizer bottoms | Refinery naphtha fractionation | Gas chromatograph on the overhead of the deisopentanizer column | 30 to 75 minutes delay (Pani et al., 2016) |
| Melt flow index | Polypropylene reactor | Lab test to ASTM D1238 or ISO 1133 | Every 2 to 4 hours (Farsang et al., 2015) |
| Free lime in clinker | Cement kiln | Lab sample | More than 2 hours delay (Fuller Technologies) |
| Concentrate grade | Froth flotation | On-stream analyser | About every 10 to 15 minutes (Duchesne et al., 2003) |
| Typical quality variables | Process industries in general | Lab sample | Once per 8-hour shift or once a day (Lawrence et al., 2024) |
Even the flotation analyser is described by its authors as too slow for the process dynamics (Duchesne et al., 2003).
The cost of the delay is margin. A disturbance that comes and goes between two samples is never seen, so operators keep a safety distance from the specification. A cement equipment supplier describes kilns being over- and under-burned to compensate for the lag in free lime results, and finish mills run conservatively between fineness samples (Fuller Technologies). A soft sensor narrows that blind spot.
Model-based or data-driven: which kind of soft sensor?
Soft sensors fall into three families (Lawrence et al., 2024):
| Type | Built from | Strengths | Weaknesses |
|---|---|---|---|
| Model-based (white box) | Mass and energy balances, thermodynamics, kinetics; often a Kalman filter | Physically meaningful; obeys conservation laws | Needs deep process knowledge; hard to calibrate for complex nonlinear processes |
| Data-driven (black box) | Historical process and lab data: PLS, neural networks and similar | Captures complex behaviour without a full model | Knows only what is in the data; degrades when the process moves to a new operating region |
| Hybrid (gray box) | A first-principles structure whose parameters or residuals are learned from data | Combines physical structure with a data fit | More effort; the data part may add little far from the training data |
In soft sensor papers from 2015 to 2023, research focused mainly on data-driven models: neural networks accounted for about 48% of them and PLS for about 11%, while hybrid soft sensors received the least attention (Lawrence et al., 2024). Industrial practice is more conservative: the usual practice is to develop a linear model first and move to complex methods only if it is not accurate enough. A PLS model's coefficients can also be checked against process knowledge by the engineers who will own it.
Where are soft sensors used in process plants?
Distillation product quality. In a debutanizer column, butane in the bottoms is measured by a gas chromatograph located on the overhead of the deisopentanizer column, which introduces a 30 to 75 minute delay that a soft sensor on the bottom flow can bridge (Pani et al., 2016). In the public dataset for this column, the seven inputs are top, tray and bottom temperatures, top and bottom pressures, reflux flow and top product flow (Wang and Dang, 2024).
Polymer melt index. At a Hungarian polypropylene plant using two loop reactors, a soft sensor estimates melt flow index between lab tests by combining PCA and PLS with balance equations to reconcile noisy measurements (Farsang et al., 2015).
Cement free lime. A cement process control supplier trains deep neural networks on a plant's operating and lab history to predict free lime and fineness every two to three minutes; lab results validate the model and recalculate its bias (Fuller Technologies).
Flotation grade. Camera images of the froth surface, condensed with PCA and PLS, are used to predict concentrate grade between on-stream analyser readings (Duchesne et al., 2003).
These examples match the four sectors we work with: oil and gas, chemistry, manufacturing and mining.
What data do you need to build a soft sensor?
- Process history from the historian for every candidate input, at a resolution finer than the process dynamics, covering all grades, feeds and operating modes the soft sensor will meet.
- Lab results with the sampling time. The timestamp must be when the sample was drawn, not when it was analysed or entered in the LIMS; otherwise the model learns from misaligned data.
- Event data: grade changes, shutdowns, upsets, instrument calibrations and analyser maintenance, so those periods can be excluded or labelled.
- Process knowledge: residence times and dead times between each input and the sampling point, used to shift inputs in time.
Labelled data is the bottleneck, and the survey calls it the main challenge in soft sensor development (Lawrence et al., 2024). A year of once-per-shift sampling yields about a thousand lab points.
How do you build a soft sensor, step by step?
The steps below follow the industrial workflow described by Lawrence et al. (2024), with practical notes added.
- Define the target and the business case. Involve the panel operator, process engineer and control engineer, and weigh costs against the expected benefit, such as less off-spec product or quality giveaway.
- Select inputs from process knowledge. Where knowledge is thin, methods such as LASSO or ridge regression help remove inputs with negligible influence.
- Collect and clean the data. Remove outliers, missing values, shutdowns and upsets. Industrial data is abundant but poor in information.
- Align in time. Inputs are sampled far more often than the lab. Keep the input values that correspond to each lab sample, shifted by the relevant dead time.
- Train and validate offline. Start with a linear model. Split the data by time: train on older data, test on a later period.
- Deploy in open loop. Run it in the DCS or on a connected server, show it to operators and compare it with lab results before relying on it. Integrating models with SCADA and OT systems covers the connection side.
- Hand over. Once the estimate consistently follows the lab, it can be used in a control loop and handed to the panel operator.
How do you validate a soft sensor against the lab?
Validation asks whether the estimate beats what operators already have between lab samples.
- Test on a later period that includes a grade change or disturbance, not only steady operation.
- Report standard metrics. Root mean squared error and the correlation with lab values are the usual measures (Lawrence et al., 2024). Add the mean error, which reveals bias.
- Plot it. Put estimates and lab values on one trend and check that the estimate follows the lab.
- Beat a naive baseline: the last lab value held until the next one. Otherwise the soft sensor adds nothing.
- The lab has its own error. Once the soft sensor's error approaches the lab method's repeatability, further gains cannot be measured.
Why do soft sensors fail, and how do you maintain them?
| Failure mode | What it looks like | Response |
|---|---|---|
| Input sensor fault (stuck, drifting or failed transmitter, lost data link) | Sudden jump or flat line in the estimate | Range and rate checks on every input; flag or freeze the estimate and fall back to the lab |
| Slow process drift (catalyst ageing, fouling, raw material change) | Growing offset from lab values | Bias update with each lab result; retrain on recent data when the offset keeps returning |
| Operation outside the training range (new grade, feed or throughput) | Large errors, estimates outside the normal operating range | Check whether inputs fall inside the training envelope; build separate models per mode; retrain |
| Bad lab results | Bias corrections that chase noise | Screen lab values before using them for updates |
When estimates degrade, practitioners check the inputs first (sensor failures, data transmission problems, outliers, shutdowns, upsets) before touching the model (Lawrence et al., 2024). Static soft sensors lose accuracy as the process shifts to a new operating region. Adaptive soft sensors update their parameters from new samples, yet fewer than one-third of those in the survey were adaptive. Bias updating, which pulls the estimate toward the latest lab result, is the more popular maintenance approach. It corrects an offset, not a wrong slope, which needs retraining. An earlier review covering chemical, bioprocess and steel plants also treats maintenance as an open issue (Kadlec et al., 2009).
Give each soft sensor a named owner who reviews its error against the lab on a fixed schedule.
Where should you start?
Pick one quality variable where the lab delay costs money, with at least a year of lab history and reliable input instruments. A short AI readiness assessment helps check the data before any modelling. Related: AI process control in chemical manufacturing.
Our data science team builds predictive models on industrial data and deploys them on your own infrastructure. The discipline of predicting first and confirming with the lab also underlies our Virtual Lab work in formulation R&D, a different use case from an online soft sensor: at ENAD Shymeca it reduced physical testing cycles by 40%, and at ADFC comparisons between simulated and actual results guide refinements. More on that approach in Virtual Lab for industrial R&D.
Sources
- Lawrence et al., Machine learning for industrial sensing and control: A survey and practical perspective, Control Engineering Practice, 2024 (arXiv 2401.13836)
- Farsang et al., PCA Based Data Reconciliation in Soft Sensor Development: Application for Melt Flow Index Estimation, Chemical Engineering Transactions 43, 2015
- Pani, Amin and Mohanta, Soft sensing of product quality in the debutanizer column with principal component analysis and feed-forward artificial neural network, Alexandria Engineering Journal 55, 2016
- Wang and Dang, Fuzzy Recurrent Stochastic Configuration Networks for Industrial Data Analytics, arXiv 2407.11038, 2024
- Duchesne et al., Application of Multivariate Image Analysis (MIA) to Predict Concentrate Grade in Froth Flotation Processes, CIM, 2003
- Fuller Technologies, AI-Powered Soft Sensors for Cement Process Optimization
- Kadlec, Gabrys and Strandt, Data-driven soft sensors in the process industry, Computers and Chemical Engineering 33, 2009
Frequently asked questions
What is the difference between a soft sensor and an online analyzer?
An online analyzer, such as a gas chromatograph or a flotation on-stream analyzer, physically measures a sample of the process stream. A soft sensor is a calculation that estimates the same property from other measurements such as temperatures, pressures and flows. The two are often combined: the soft sensor fills the gaps between analyzer or lab results and is corrected by them.
Is a soft sensor the same as an inferential sensor or a virtual sensor?
Yes, in practice the terms describe the same thing. Soft sensor, virtual sensor, inferential sensor and inferential model all refer to a model that infers a hard-to-measure variable from easy-to-measure ones. Vendors and textbooks simply use different names.
How much data do I need to build a soft sensor?
The limit is usually the number of lab results, not process data. If the quality variable is sampled once per shift, a full year gives about a thousand labelled points, and once a day gives about 365. The history should also cover all the grades, feeds and operating modes the soft sensor will see in service.
Can a soft sensor be used in closed-loop control?
Yes, but only after a period of online validation in open loop, where operators and engineers compare its estimates with lab results. Once it follows the lab consistently, it can be used as the measurement in a control loop or an advanced process control application. It should still be monitored, with a fallback when its inputs fail.
How often should a soft sensor be retrained?
There is no fixed interval. A common approach is to apply a bias correction when a new lab result arrives and to retrain the model only when errors grow, after a process modification, or when a new product grade or feed appears. Tracking the error between estimate and lab over time tells you when retraining is due.
Does a soft sensor replace laboratory analysis?
No. The lab remains the reference measurement. It is used to train the soft sensor, to validate it before use, and to correct it once it is live, so reducing lab sampling too far removes the data the soft sensor depends on.



