Table of Contents
Predictive quality uses data recorded during production, such as machine settings, sensor signals and station test results, to estimate whether each part or batch will pass its final quality check. Parts likely to fail are flagged while they can still be inspected, reworked or diverted, instead of being found at the end of the line or by the customer.
Key takeaways
- A 2022 review of 81 studies defines predictive quality as using machine learning to estimate product quality from process and product data, but found that in most cases the approaches were not integrated into a running manufacturing process (Tercan and Meisen, 2022).
- The first requirement is part-level traceability: each unit must be linked to the data recorded for it at every station and to its final inspection result.
- Defects are rare, so accuracy misleads. In the Bosch production line data, fewer than 1% of parts failed (Mangal and Kumar, 2016), so a model that calls every part good is more than 99% accurate and catches nothing.
- The alert threshold is a business decision that trades extra inspections against escaped defects. The default probability of 0.5 is rarely the right one (scikit-learn).
- Validate on a later period than the training data, and watch the model for drift once it is live.
What is predictive quality?
Tercan and Meisen define predictive quality as the use of machine learning and deep learning in production to "estimate product-related quality based on process and product data" (Tercan and Meisen, 2022). The estimate can be a number, such as a part weight or a surface roughness, or a class, such as pass or fail. Of the 81 studies they reviewed, published from 2012 to 2021, 51 were regression tasks and 30 classification tasks, and cutting and joining (mainly welding) were the most studied processes.
The aim is to move the quality decision upstream. A defect created at station 3 is usually found at the end-of-line test, after later stations have added cost to a part that was already scrap. A model reading station 3's data can flag it straight away.
How does it differ from SPC, inspection and soft sensors?
NIST separates two older tools. Statistical process control is based on a "comparison of what is happening today with what happened previously", while statistical quality control tools inspect the product "for defects or defective pieces after all processing is complete" (NIST/SEMATECH). Predictive quality sits between them: it uses process data, like SPC, but its output is a statement about each unit, like inspection.
| Approach | Input | Output | When the answer arrives |
|---|---|---|---|
| End-of-line inspection or test | The finished product | Pass or fail, measured | After all processing |
| Statistical process control | One or a few process variables | The process has changed | During production, per variable |
| Anomaly detection | Many process variables | This does not look like normal operation | During production |
| Predictive quality | Process and product data linked to each unit | Predicted quality, or probability of failure, for that unit | As soon as the relevant stations have run |
| Soft sensor | Continuous process measurements | Continuous estimate of a quality variable | Continuously, in continuous processes |
Our guide to anomaly detection methods covers the third row; unlike predictive quality, it needs no defect labels. In continuous processes the same idea is called a soft sensor, and in batch plants it is batch quality prediction. This guide focuses on discrete parts and multi-station lines.
What does the published evidence show?
Injection moulding in a pilot cell. Aminabadi et al. moulded a high-gloss ABS part with in-mould pressure and temperature sensors, machine signals collected over OPC UA, ambient sensors and in-line measurement of every part. With about 2,500 samples, models predicted part weight and one of the part's dimensions with an R² of about 0.99, but surface quality stayed at about 0.70 (Aminabadi et al., 2022). Cosmetic defects are harder to predict than weight. The data came from a designed experiment with 56 settings, not from months of production.
An assembly line with rare failures. Bosch released anonymised measurements and test results for parts on its lines in a Kaggle competition. Mangal and Kumar describe 1,184,687 training parts, 51 stations on 4 production lines, more than 4,000 features, and fewer than 1% failed parts (Mangal and Kumar, 2016). Their gradient boosting model reached an AUC of 0.718: useful for ranking parts by risk, far from a perfect sort. They also found more than 7,000 distinct paths through the stations, a sign of several product families that would each need their own model.
A semiconductor process. The SECOM dataset holds 1,567 production records with 591 features each, including 104 failures, with missing values. Its description notes that the signals contain "a combination of useful information, irrelevant information as well as noise" (UCI Machine Learning Repository).
The review sees the same gap across the field: most studies used around 100 samples, and "in most cases, however, the approaches are not integrated into the manufacturing process" (Tercan and Meisen, 2022).
What data do you need?
- A unique identifier per unit: serial number, data matrix code or batch number, read at every station.
- Process data per unit and station: setpoints, actual values, sensor curves such as pressure or torque, cycle times, and the tool or mould used.
- Material data: supplier lot and incoming inspection results.
- The quality outcome: test results, defect codes, rework and customer returns, linked to the same identifier.
- Event data: tool changes, maintenance, recipe changes.
The hard part is the join. Process data usually sits in PLCs, SCADA and the historian, and part genealogy in the MES, which ISA-95 places at level 3, between control systems and business systems (ISA). If the line only records scrap totals per shift, a per-part model cannot be trained until traceability is in place. Our posts on where AI fits in the Purdue model and integrating AI with SCADA and OT systems cover access to these layers.
Keep the timestamps: in the Bosch study, the time a part spent on the line was an important feature, positively correlated with failure (Mangal and Kumar, 2016).
Why does accuracy mislead when defects are rare?
If 0.6% of parts fail, a model that predicts "good" for every part is 99.4% accurate and catches nothing. Chicco and Jurman show that accuracy and the F1 score can be overoptimistic on imbalanced data. In one example, a classifier scored an accuracy of 0.90 and an F1 score of 0.95 while its Matthews correlation coefficient (MCC) was −0.03 (Chicco and Jurman, 2020). They recommend MCC, which is high only when both classes are predicted well. The Bosch competition leaderboard was scored with it.
For the plant, translate the results into three counts per week:
- Defects caught: the share of defective units the model flagged (recall).
- False rejects: good units flagged, each costing an extra inspection.
- Escapes: defective units the model missed.
How do you set the alert threshold?
A classifier gives a probability; the threshold turns it into an action. The scikit-learn documentation says its default rule, predicting the positive class above 0.5, is "most certainly not ideal for most use cases". It recommends tuning the threshold for a metric that fits the use case, and never on the same data used to train the classifier (scikit-learn).
In the Bosch study, MCC peaked at a threshold of 0.11, where only 3,235 of the more than one million training parts were flagged. Inspecting the top two deciles of risk instead would have meant checking more than 200,000 parts (Mangal and Kumar, 2016).
Set the threshold from costs you can state: an extra inspection, scrapping a part after more value has been added, and a defect reaching the customer. When escapes are very costly, accept a low threshold and more inspection. When inspection capacity is fixed, flag only what the inspection station can absorb.
How do you validate a predictive quality model?
Split by time. Ordinary cross-validation on time-ordered data "would lead to training on future data and evaluating on past data", which is why scikit-learn offers a time-based split (scikit-learn). A random split also puts parts from the same shift, tool and material lot in both training and test sets, which makes the test easier than production. Both studies above used random or k-fold splits, which suit comparing methods, not deciding whether to trust a model on your line.
Look for leakage. scikit-learn defines data leakage as using "information that would not be available at prediction time" to build the model, which gives overly optimistic estimates (scikit-learn). Typical plant examples are a rework flag, a downstream test result or the end-of-line inspection timestamp left among the inputs.
What do you do with a prediction?
Actions are listed from lowest to highest risk:
- Inspect more closely. Send flagged units to a fuller test or a manual check.
- Stop adding value. Divert a flagged unit before an expensive operation such as machining, coating or final assembly.
- Hold the lot. If many units from one tool or material lot are flagged, hold it for review.
- Find the cause. Feature importance points to the stations and variables behind the risk. In the Bosch study, date features and features from production line 3 were among the most important.
- Adjust the process. The injection moulding cell above used an AI controller that adjusted machine settings and held the average of a dimension at 120 mm within ±0.025 mm (Aminabadi et al., 2022). Closed-loop control comes last, after the model has proven itself in open loop.
How do you keep the model accurate?
Lines change. Lu et al. describe concept drift as "unforeseeable changes in the underlying distribution of streaming data over time" and distinguish sudden, gradual, incremental and reoccurring drift. For mild drift, updating the current model may be enough; for severe drift, retraining a new model can be better (Lu et al., 2018).
In practice, track defects caught and false rejects every week, retrain after tool, material, recipe or product changes, and keep end-of-line inspection: it provides the labels that show whether the model still works.
How do you start, step by step?
- Pick one product and one defect type with a cost you can state and enough failures in the history.
- Check traceability. Take ten failed units and try to retrieve their data from every station.
- Assemble a history long enough to include tool changes, material lots and product variants.
- Build a simple baseline, such as logistic regression or gradient boosted trees on summary features per station, before trying deep learning.
- Validate on a later period and report defects caught, false rejects and MCC.
- Choose the threshold with production and quality managers, from stated costs.
- Run in shadow mode for several weeks, scoring every unit without acting, then act on flags through existing quality procedures.
- Name an owner who reviews performance every month.
What we have seen at KABAS AI
Our manufacturing page lists quality assurance and defect detection among the use cases we address, with DALILE providing quality trend analysis and anomaly detection on production data, and Virtual Lab supporting parameter optimisation for quality and efficiency. Our data science team covers data collection and preparation, predictive models, and the MLOps pipelines that keep models monitored in production.
Our practical advice: the first weeks of a predictive quality project go to traceability, not modelling. If a failed part cannot be traced back to its station data, fix that first; it pays off even without a model. Agree on the costs of a false reject and an escape before training anything. Keep data and models on your own servers, since process parameters and defect history describe how your product is made. Before you start, run an AI readiness assessment, read why industrial AI projects fail, and see building a first industrial data team for who will own the model.
Sources
- Tercan and Meisen, Machine learning and deep learning based predictive quality in manufacturing: a systematic review, Journal of Intelligent Manufacturing 33, 2022
- Mangal and Kumar, Using Big Data to Enhance the Bosch Production Line Performance: A Kaggle Challenge, IEEE Big Data 2016 (arXiv 1701.00705)
- Aminabadi et al., Industry 4.0 In-Line AI Quality Control of Plastic Injection Molded Parts, Polymers 14(17), 2022
- UCI Machine Learning Repository, SECOM dataset
- NIST/SEMATECH e-Handbook of Statistical Methods, Process or Product Monitoring and Control
- ISA, ISA-95 Standard
- Chicco and Jurman, The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation, BMC Genomics 21, 2020
- scikit-learn, Tuning the decision threshold for class prediction
- scikit-learn, TimeSeriesSplit
- scikit-learn, Common pitfalls and recommended practices
- Lu et al., Learning under Concept Drift: A Review, IEEE Transactions on Knowledge and Data Engineering (arXiv 2004.05785)
Frequently asked questions
Is predictive quality the same as statistical process control?
No. Statistical process control watches one or a few process variables and signals when the process changes. Predictive quality uses many process and product variables linked to each part or batch and estimates whether that unit will pass its final check. The two work well together: SPC keeps the process stable, and predictive quality flags individual units at risk.
Can predictive quality replace end-of-line inspection?
Not at first, and usually not entirely. The model decides which parts deserve extra attention, while inspection still decides pass or fail. Inspection results are also the labels the model needs for validation and retraining, so cutting inspection too far removes the data that keeps the model accurate.
How many defects do we need in the history to build a model?
There is no universal number, and the limit is usually the number of defective units, not the total number of parts. A line may produce millions of parts but only a few hundred failures of a given type. The history should also cover the tools, material lots and product variants the model will see in service.
What is the difference between predictive quality and camera-based visual inspection?
Visual inspection looks at the part itself, usually with cameras and image models, and finds defects that are already visible. Predictive quality looks at how the part was made, such as pressures, temperatures, torques and cycle times, and estimates the result before it can be measured. Some plants use both: process data to flag risky parts early, and cameras to confirm.
Can predictive quality models run on-premise?
Yes. The usual models, such as logistic regression and gradient boosted trees on features extracted from process data, are small and run on ordinary servers inside the plant network. Keeping them on-premise also keeps process parameters and defect history, which describe how your product is made, inside the company.



