Table of Contents
Model drift is the loss of accuracy of a model in service because the process, raw materials or instruments have changed since its training data was collected. In a plant, you catch it by monitoring four things: the health of incoming data, the input distributions, the predictions, and the error against lab or inspection results as they arrive.
Key takeaways
- Drift comes from a change in the inputs, in the relationship between inputs and target, or both; only a change in the relationship moves the decision boundary (Lu et al., 2018).
- Plant labels arrive late, often once per shift or once a day (Lawrence et al., 2024). Input and prediction monitoring gives the early warning; the error against the lab confirms it.
- Many alerts are data problems, not drift: a replaced transmitter, a re-ranged tag, a stalled feed. Check the data first.
- Distribution tests on large historian windows raise false alarms unless thresholds are tuned (Breck et al., 2017).
- Every model needs a named owner, a written response to each alert, a tested rollback and a fallback to traditional automation or manual operation (CISA et al., 2025).
What is model drift in an industrial plant?
Lu et al. define concept drift as "unforeseeable changes in the underlying distribution of streaming data over time" (Lu et al., 2018). The input distribution can change while the input–target relationship stays the same; they call this virtual drift and note that it does not affect the decision boundary. The relationship itself can change, which they call actual drift and which reduces accuracy. In many real applications, both happen together.
| Type | What changes | Plant example | Visible without labels? |
|---|---|---|---|
| Data drift (virtual drift) | The distribution of the inputs | A new crude blend, ore from another pit, higher throughput | Yes, from input statistics |
| Concept drift (actual drift) | The relationship between inputs and target | Catalyst ageing or fouling, when they are not model inputs: same temperatures, different product | Only against lab, inspection or failure records |
| Data pipeline change | What a tag means, not the process | A transmitter replaced or re-ranged, a historian setting changed, a frozen value | Usually, with data quality checks |
The last row is not drift in Lu et al.'s sense, but it looks the same to the model. Breck et al. warn that outages, version upgrades and other changes in a source system can "radically change the feature's meaning" without producing values strange enough to trigger other monitoring (Breck et al., 2017). Our guide to industrial data quality problems covers those checks.
Lu et al. also distinguish sudden, gradual, incremental and reoccurring drift: a feed switch is sudden, catalyst deactivation incremental, a returning product grade reoccurring.
Why do industrial models drift?
Guidance on AI in operational technology, published in December 2025 by CISA and eight partner agencies, lists model drift as a risk: models "may become less accurate over time" when they receive data not represented in their initial training data, and "alterations to production processes can affect model performance" (CISA et al., 2025). A survey of industrial soft sensors adds that operating conditions change with product demand and raw material prices, and that a model built in one operating region degrades as the process moves to another (Lawrence et al., 2024).
Typical causes are catalyst deactivation, fouling and wear; a new supplier, crude or ore blend; a new grade or recipe; instrument replacement or a DCS or historian upgrade; and seasonal ambient and cooling water temperatures.
Why is drift hard to see in a plant?
Labels arrive late. Quality variables are often sampled once per 8-hour shift or once a day (Lawrence et al., 2024), and failure labels can take months to accumulate, as our post on labelling failures in time-series data explains.
Models fail silently. Rabanser et al. note that machine learning systems "tend to fail silently" (Rabanser et al., 2019). A drifting soft sensor still shows a plausible number on the operator's screen.
Label-free monitoring has a limit. NannyML's methods for estimating performance without targets assume that "there is no concept drift" (NannyML). Input monitoring tells you the plant has moved; only lab, inspection or failure results tell you whether the model is still right. Do not cut lab sampling because a model is in service.
What should you monitor?
| Layer | What to compute | How often | Needs labels? |
|---|---|---|---|
| Data health | Freshness, share of missing or bad-status values, frozen runs, values outside physical limits | Continuously | No |
| Input drift | Distance between recent and reference distributions, per operating mode | Hourly to daily | No |
| Prediction drift | Mean and distribution of predictions; action limits | Hourly to daily | No |
| Performance | Error against lab or inspection: mean error, RMSE, or defects caught and false rejects | At each new label | Yes |
| Staleness and dependencies | Model age; changes to source tags, instruments and systems | At each change | No |
Data health comes first. Evidently's documentation notes that its drift tests do not react to an increase in empty values and recommends separate tests on the share of nulls (Evidently, data drift explainer). A failed transmitter can pass a drift test.
Predictions are a cheap signal. Sculley et al. suggest monitoring prediction bias, whether the distribution of predictions matches that of observed labels, and action limits that trigger alerts and manual investigation (Sculley et al., 2015). The CISA guidance gives the example of statistical process control detecting outliers in a predictive maintenance model's outputs.
Performance is the confirmation. Join each lab or inspection result to the prediction made at the sample time; track mean error, which reveals bias, and RMSE, or defects caught and false rejects for predictive quality classifiers.
Staleness and dependencies. Breck et al. recommend alerting on model age and notifying the team when upstream sources change; in a plant, read the instrument maintenance log and DCS change records.
Which drift tests suit process data?
| Method | Compares | Typical use | Default or guidance |
|---|---|---|---|
| Kolmogorov–Smirnov test | Two samples of one numerical variable | Input drift, small samples | Evidently default for numerical columns up to 1,000 objects; drift when p < 0.05 |
| Wasserstein distance (normed) | Two samples of one numerical variable | Input drift, larger samples | Evidently default above 1,000 objects; drift when ≥ 0.1 |
| PSI or Jensen–Shannon distance | Binned or categorical distributions | Grades, modes, binned inputs | Evidently default threshold 0.1 |
| EWMA chart | One series against its own history | Model error, one key input | λ usually 0.2 to 0.3 |
| Page-Hinkley test | Mean of a stream | Model error or prediction stream | Detects increases, decreases or both |
| ADWIN | Mean of a stream, with an adaptive window | Model error stream | Drops the older sub-window after a change |
At the time of writing, Evidently picks a test by column type and number of observations, and by default flags dataset drift when half the columns drift (Evidently, customise data drift). Treat these defaults as starting points.
For the model's error against the lab, control charts fit well. An EWMA chart "can be made sensitive to a small or gradual drift" (NIST/SEMATECH), and Lu et al. describe a detector that tracks error rate with one. River's Page-Hinkley and ADWIN detectors take one value at a time, which suits an error series that grows by one point per lab sample.
Three cautions for historian data:
- Large samples flag tiny differences. With months of minute data, almost any difference is significant, and consecutive values are not independent. Breck et al. stress that alert thresholds need careful tuning to balance false positives and false negatives. Aggregate to hourly or shift values and tune thresholds on past periods.
- Compare like with like. Use a reference per grade or operating mode, or every grade change will fire an alert; our anomaly detection guide explains why modes need separate treatment.
- Not every drift matters. Virtual drift does not move the decision boundary, so prioritise the inputs the model relies on most, and watch the predictions.
How do you set up monitoring, step by step?
- Freeze the reference: the training window per operating mode, with feature definitions and code versions.
- Log every prediction with timestamp, inputs, outputs and model version, the audit trail the CISA guidance recommends.
- Join labels at the sample time, not the LIMS entry time.
- Choose metrics and windows for each layer above.
- Calibrate thresholds on history. Replay the past year through the monitor, count the alerts, and adjust until each one is worth a person's time.
- Route alerts to a named owner and show key indicators where operators already look; CISA suggests integrating monitoring into existing HMI views.
- Write the response to each alert and rehearse the rollback.
The monitor should read data exactly as the model does; see historian data for machine learning.
What do you do when a drift alert fires?
| Alert | Check first | Typical response |
|---|---|---|
| Data health | Instrument, network link, historian | Fix the source; flag or freeze predictions; use the fallback |
| Input drift, error still stable | New mode, feed or grade? Inside the training range? | Watch; collect labels; plan retraining if the region is new |
| Prediction shift, no new labels | Inputs and process events | Ask operators; request an extra lab sample |
| Constant offset from lab | Lab method, sample timing | Bias update |
| Error growing or varying with conditions | Process or equipment change | Retrain on recent data, validate, redeploy |
| Unsafe or unexplained behaviour | Act first, investigate after | Switch the model off; revert to the previous method |
The industrial survey describes bias updating, which brings outputs closer to lab data, as the more popular way to maintain a soft sensor, with retraining on recent data as the alternative (Lawrence et al., 2024). Lu et al. add that severity should guide the choice: for mild drift, updating the current model may be enough; for severe drift, retraining a new one can be better.
Google Cloud's MLOps guidance lists retraining triggers: a schedule, new data, performance degradation and significant changes in data distributions (Google Cloud). In plants, add process change records.
The CISA guidance asks for failsafe mechanisms that "revert to traditional automation or manual" operation. Breck et al. add that rolling back to a previous model should be quick, safe and practised outside emergencies.
How do you retrain without disrupting production?
- Train on a window that includes the new regime; keep older data if old conditions may return.
- Validate on the latest labelled period, not a random split.
- Compare the candidate with the current model on that period, then run it in shadow mode; see moving from AI pilot to plant rollout.
- Version data, code and models, and keep the previous model ready.
- Script the pipeline. Breck et al. warn that a model retrained by hand once or twice a year is hard to retrain once its engineer leaves.
Who owns model monitoring?
The NIST AI Risk Management Framework playbook says AI systems should be monitored in production, since drift means they "no longer meet the assumptions and limitations of the original design" (NIST AI RMF Playbook). The CISA guidance recommends KPIs and regular reviews with vendors, governance boards and operators.
In practice, pair a process engineer who knows what changed in the plant with the data scientist who owns the pipeline, and review each model monthly. Models nobody monitors are one reason industrial AI projects fail; see also building a first industrial data team.
What we have seen at KABAS AI
Our data science service runs from data collection and preparation to deployment (MLOps), with pipelines for continuous model operation and monitoring in production. On our manufacturing page, DALILE provides quality trend analysis and anomaly detection on production data.
Our practical advice: design monitoring together with the model, not after go-live. Agree beforehand which alert leads to which action, and who decides to switch a model off. Run the monitoring on the same on-premise servers as the model, so historian and lab data stay inside your network.
Sources
- Lu et al., Learning under Concept Drift: A Review, IEEE Transactions on Knowledge and Data Engineering, 2018 (arXiv 2004.05785)
- Lawrence et al., Machine learning for industrial sensing and control: A survey and practical perspective, Control Engineering Practice, 2024 (arXiv 2401.13836)
- Sculley et al., Hidden Technical Debt in Machine Learning Systems, NeurIPS 2015
- Breck et al., The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction, IEEE Big Data 2017
- Rabanser, Günnemann and Lipton, Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift, NeurIPS 2019 (arXiv 1810.11953)
- CISA et al., Principles for the Secure Integration of Artificial Intelligence in Operational Technology, December 2025
- NIST, AI RMF Playbook, Measure function
- Google Cloud, MLOps: Continuous delivery and automation pipelines in machine learning
- NIST/SEMATECH e-Handbook of Statistical Methods, EWMA Control Charts
- Evidently, Data drift (how the default algorithm works)
- Evidently, Customize data drift
- NannyML, Estimation of Performance of the Monitored Model
- River, ADWIN
- River, PageHinkley
Frequently asked questions
What is the difference between data drift and concept drift?
Data drift is a change in the inputs the model receives, such as a new feed or a higher throughput, while the relationship between inputs and target stays the same. Concept drift is a change in that relationship, for example when catalyst ageing changes product quality at the same temperatures and flows. Data drift can be seen from input statistics alone; concept drift can only be confirmed against lab, inspection or failure records.
How often should an industrial model be retrained?
There is no fixed interval. Retrain when the error against lab or inspection results keeps growing, after a process, feed, product or instrument change, or when the inputs move into a region the model has not seen. Some teams also retrain on a schedule when new labelled data arrives regularly. Monitoring tells you when retraining is due.
Can we detect drift without lab results or other labels?
Partly. Data health checks and input and prediction distributions show that the plant or the data has changed, without any labels. They cannot show whether the model is still right: label-free performance estimation methods assume there is no concept drift. Lab, inspection or failure results remain the confirmation.
Which tools can monitor drift on our own servers?
Python libraries such as Evidently for comparing distributions, River for streaming change detectors such as ADWIN and Page-Hinkley, and NannyML for estimating performance without labels can run on servers inside your own network. Simple statistics written to the historian and shown on existing dashboards are often enough to start.
Is a bias update enough to fix a drifting model?
Only for a constant offset. A bias update pulls the estimate toward the latest lab result, which corrects a shift but not a changed slope or a new operating region. If the error keeps returning after bias updates, or varies with operating conditions, the model needs retraining on recent data.
What should happen when a model is switched off?
Operators should fall back to the method they used before the model: lab results, the previous calculation, traditional automation or manual operation. The fallback, and who decides to use it, should be written down and tested before go-live, together with a way to roll back to the previous model version.



