Table of Contents
An LLM can support root cause analysis by searching years of work orders, shift logs and inspection reports for similar past failures, building a cited timeline and proposing candidate causes to test. It cannot confirm a cause. Engineers still check each hypothesis against data, the failed part and the people involved, then record the confirmed cause.
Key takeaways
- A root cause is the initiating failing from which the other causes spring, and it is generally a management, planning or organisational failing (HSE, HSG245).
- Retrieving similar past cases made the difference in Microsoft's RCACopilot study: a micro F1-score of 0.766, against 0.026 for the same GPT-4 setup without past incidents in the prompt (Chen et al., 2024).
- Maintenance text needs adaptation. On failure mode classification, off-the-shelf GPT-3.5 scored an F1 of 0.46, against 0.80 after fine-tuning (Stewart et al., 2023).
- Most maintenance histories record what failed, not why. In ISO 14224 data collection, failure cause and failure mechanism are not part of the minimum data (Selvik, 2014).
- Treat model output as hypotheses with citations. LLMs often reason well about causes but "exhibit unpredictable failure modes" (Kıcıman et al., 2023).
What does root cause analysis mean in maintenance?
IEC 62740, the international standard on root cause analysis, describes the principles of RCA and the steps an RCA process should include. It covers only analysis after the event, and its techniques "are not designed to assign responsibility or liability" (IEC 62740:2015).
The UK Health and Safety Executive's investigation guide separates three levels of cause (HSE, HSG245):
- Immediate cause: the most obvious reason the event happened.
- Underlying cause: the less obvious system or organisational reason, such as checks that were not carried out.
- Root cause: "an initiating event or failing from which all other causes or failings spring".
HSE writes about incidents that harm people, but the logic carries over to equipment. It adds that investigations concluding operator error was the sole cause "are rarely acceptable".
ISO 14224, the reliability and maintenance data standard for the petroleum, petrochemical and natural gas industries, gives the equipment vocabulary. As presented by the technical editor of the ISO working group, it distinguishes failure mode, failure mechanism and failure cause (Selvik, 2014):
| Term | Source | Definition | Illustrative example: pump seal leak |
|---|---|---|---|
| Failure mode | ISO 14224 | The effect by which a failure is observed | External leak at the mechanical seal |
| Failure mechanism | ISO 14224 | The physical, chemical or other process that leads to a failure | Wear of the seal faces |
| Failure cause (root cause) | ISO 14224 | The circumstances during design, manufacture, installation, use or maintenance that led to the failure | Pump and motor misaligned after a repair |
| Root cause | HSE HSG245 | The initiating failing from which the other causes spring; generally organisational | No alignment check in the repair procedure |
The pump example is ours. A cause chain has several levels, and the records an LLM reads usually stop at the first.
Why is maintenance history hard to analyse?
Causes are rarely written down. In the ISO 14224 failure data fields presented by Selvik, the minimum data are the failure date, failure mode and failure impact on equipment function. Failure mechanism and failure cause are not in that minimum set (Selvik, 2014). A record can meet the minimum and still say nothing about why the equipment failed.
Free text and jargon. NIST built its Nestor toolkit because maintenance work order data was "too unstructured or filled with jargon to analyze" (NIST, Nestor). NIST researchers and colleagues argue that out-of-the-box language processing pipelines "need re-imagining" for engineering text, an approach they call technical language processing (Brundage et al., 2021). In Algerian plants one note can mix French, Arabic and abbreviations; see handling documents that mix Arabic, French and English.
Dates are detection dates. A work order is dated when someone noticed the problem, not when the degradation began; labelling failures in time-series data explains the difference.
Evidence is spread out. Shift logs, alarm journals, inspection reports and change records sit in different systems.
What can an LLM do in a root cause analysis?
Most of the value is in reading and organising, not in deciding.
| Task | What the LLM does | What stays with people |
|---|---|---|
| Find similar past events | Searches work orders and reports across wording and languages, and ranks the closest cases with references | Judging whether the cases are truly comparable |
| Build the timeline | Pulls dated entries from work orders, shift logs and alarms into one sequence, each with its source | Checking the order and the gaps |
| Normalise free text | Proposes failure mode codes from your own list | Reviewing a sample before trusting the codes |
| Propose hypotheses | Lists candidate causes with the records that support and contradict each one | Deciding which to test, and how |
| Count and trend | Calls a database query instead of counting from text | Checking the query |
| Draft the report | Writes the report in the site template, in French, Arabic or English | Approving and signing it |
| Confirm the cause | Nothing | Inspecting the failed part, analysing data, interviewing people |
"How many seal failures on this pump class since 2020" is a database question, not a reading task; see text-to-SQL on production databases.
What does the published evidence say?
Few published studies cover LLM-assisted RCA in plants. The larger ones come from incident management at Microsoft, on cloud incident tickets rather than plant work orders, so treat their numbers as indications, not plant benchmarks.
Retrieval of similar cases matters most. RCACopilot, presented at EuroSys 2024, was evaluated on a year of 653 incidents from one Microsoft service. It finds the nearest historical incidents, puts their summaries and labelled root cause categories into the prompt as examples, and asks the model to pick the incident most likely to share the same root cause, or "Unseen incident". With GPT-4 it reached a micro F1-score of 0.766; the same variant with no historical incidents in the prompt scored 0.026. Adding more diagnostic data to the prompt lowered the score, which the authors read as a sign that excess information can hurt predictions. Incidents with a new root cause category made up 24.96% of the dataset. Faced with one such case, the system proposed a new category, "I/O Bottleneck"; engineers later classified it as "DiskFull" (Chen et al., 2024).
Engineers found the suggestions useful, not reliable. An ICSE 2023 study evaluated models on more than 40,000 incidents, then asked incident owners to score root cause suggestions for correctness from 1 to 5. The best single model averaged 2.88, and just over 70% of owners rated the models' overall usefulness 3 or higher. One owner warned that "if the suggestion is incorrect, on-call engineers may take longer time to investigate the problem"; another noted that "not all data are recorded in the database" (Ahmed et al., 2023).
Maintenance codes need examples. Stewart, Hodkiewicz and Li asked models to assign failure mode codes to maintenance observations. Fine-tuned GPT-3.5 reached an F1-score of 0.80, a conventional text classifier 0.60, and GPT-3.5 out of the box 0.46. The authors conclude that such domain tasks need high-quality fine-tuning data (Stewart et al., 2023).
Causal reasoning is strong but uneven. Kıcıman and colleagues report that GPT-3.5 and GPT-4 based methods reached 97% accuracy on a pairwise causal discovery benchmark. They also found unpredictable failure modes, and because LLMs ignore the actual data, they suggest combining them with data-based causal methods (Kıcıman et al., 2023).
Industrial work is at proof-of-concept stage. Trilla and colleagues describe a "causality-aware retrieval augmented generation system" for troubleshooting in an industrial environment. It works on return-on-experience records and combines an LLM's learned representations with causal links between failure modes and mechanisms. The authors tested it on one predictive maintenance use case (Trilla et al., 2024).
How do you set up LLM-assisted RCA, step by step?
- Frame the event. Record the asset tag, equipment class, failure mode, dates and consequence. One investigation, one question.
- Collect the evidence read-only. Pull work orders on the asset and its class, shift logs, alarm and trip journals, inspection reports, procedures and recent changes. Connecting an AI agent to a CMMS covers read-only tools and the agent's own account.
- Retrieve similar cases with hybrid search. Combine keyword and vector search so tags and part numbers match exactly, and filter by equipment class, as described in RAG for maintenance manuals. Consistent tag-to-equipment mapping helps; see data contextualisation for tags and equipment.
- Send a few relevant records, not the whole history. Models use information at the start or end of a long input more reliably than information in the middle (Liu et al., 2023), and in RCACopilot more input data lowered accuracy.
- Build a cited timeline. Every line points to a record, and gaps are shown, not filled.
- Ask for hypotheses with evidence for and against, plus an explicit "no similar past case" option, as RCACopilot does with "Unseen incident".
- Test the hypotheses outside the model. Use trends, inspection of the failed part, interviews and the repeated "why" that HSE describes (HSE, HSG245).
- Write the confirmed cause back into the CMMS, with mode, mechanism and cause codes, so the next investigation starts from better history.
Where does LLM-assisted RCA go wrong?
| Trap | What it looks like | Guard |
|---|---|---|
| Plausible but unsupported cause | A fluent explanation that no record mentions | A citation for every claim; the records shown next to the answer |
| Counting from text | "This pump failed five times" taken from a summary | Counts come from database queries |
| False similarity | The same symptom wording on a different equipment type | Filter by equipment class and tag; an engineer confirms each match |
| Anchoring on the past | A new failure forced into an old category | An explicit "no similar case" option |
| Stopping at operator error | Human error offered as the root cause | Ask what allowed the error to happen |
| Missing evidence | Shift notes on paper, reports never scanned | The report lists which sources were not searched |
For how to test an assistant against the first trap, see measuring hallucination in an industrial assistant.
Where should the records and the model run?
Work orders and shift logs name technicians and contractors and describe your equipment's weak points. Keep documents, search index, prompts and logs on your own servers; securing an on-premise LLM covers access control and logging.
What we have seen at KABAS AI
DALILE is our on-premise multi-agent assistant. It searches documents and gives answers with exact sources, queries databases in plain language, connects to ERP and document management systems, and runs on the client's own servers. Our manufacturing page lists equipment maintenance prediction among its use cases, and our agentic AI team builds custom agents and workflows on DALILE.
We have not published a root cause analysis project. Our two case studies, at ADFC and ENAD Shymeca, concern R&D with Virtual Lab. So this is the approach we recommend, not a reported result:
- Start with one equipment class that fails repeatedly, and a reliability engineer who owns its investigations.
- Begin with retrieval and timelines. Add hypothesis generation once engineers trust the citations.
- Measure on real investigations: time to a first evidence pack, and how often the confirmed cause was among the candidates.
- Bring in shift reports early. They often hold what happened in the hours before a failure; see automating shift reports and handover.
Sources
- IEC 62740:2015, Root cause analysis (RCA), IEC Webstore
- UK Health and Safety Executive, HSG245 Investigating accidents and incidents, 2004
- Selvik, Collection and exchange of reliability and maintenance data for equipment (ISO 14224), ISO/TC 67/WG4, Rio de Janeiro, 2014
- NIST, Nestor: toolkit for maintenance work order text
- Brundage et al., Technical language processing: Unlocking maintenance knowledge, Manufacturing Letters 27, 2021
- Chen et al., Automatic Root Cause Analysis via Large Language Models for Cloud Incidents, EuroSys 2024 (arXiv 2305.15778)
- Ahmed et al., Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models, ICSE 2023 (arXiv 2301.03797)
- Stewart, Hodkiewicz and Li, Large Language Models for Failure Mode Classification: An Investigation, arXiv 2309.08181, 2023
- Kıcıman et al., Causal Reasoning and Large Language Models: Opening a New Frontier for Causality, arXiv 2305.00050, 2023
- Trilla et al., Industrial-Grade Smart Troubleshooting through Causal Technical Language Processing: a Proof of Concept, arXiv 2407.20700, 2024
- Liu et al., Lost in the Middle: How Language Models Use Long Contexts, arXiv 2307.03172, 2023
Frequently asked questions
Can an LLM find the root cause of an equipment failure on its own?
No. It can find similar past failures, build a timeline from the records and propose candidate causes with the evidence for each. Confirming a cause needs evidence the records do not hold, such as inspection of the failed part, trend analysis and interviews, and an engineer who signs the conclusion.
Do we need to fine-tune a model on our work orders?
Not as a first step. Retrieving similar past cases and giving the model your own failure code list does most of the work. Fine-tuning can help later for tasks such as assigning failure codes: in one study, fine-tuning raised GPT-3.5's F1-score on failure mode classification from 0.46 to 0.80, but it needed a set of annotated work orders.
Which records should we connect first?
Start with CMMS work orders on the asset and its equipment class, then shift logs, the alarm and trip journal, and inspection or condition monitoring reports. Procedures and change records come next, because many causes sit in what changed shortly before the failure.
Our work orders mix French, Arabic and abbreviations. Will this work?
It can, with a search setup that handles both scripts, a glossary of site abbreviations and a check of a sample of results by engineers. Keep the original text next to every summary or proposed code, so readers can see what the technician actually wrote.
How do we measure whether the assistant helps?
Measure the time to a first evidence pack for an investigation, how often the confirmed cause was among the assistant's candidates, and how many of its citations an engineer finds correct on review. Track these on real investigations, not demonstration questions.
Does this replace structured RCA methods?
No. The assistant prepares and organises evidence; the analysis still follows a structured method, such as the process and techniques described in IEC 62740 or the repeated 'why' used in the UK HSE investigation guide. The model makes the evidence-gathering step faster.



