Table of Contents
For a private industrial assistant that answers from manuals, procedures and reports, start with retrieval-augmented generation (RAG). It gives the model your documents at question time, with citations, and updates by re-indexing. Fine-tuning is better at changing how a model behaves than what it knows. Use it later, for format, style or a narrow task, usually on top of RAG.
Key takeaways
- In controlled tests, RAG outperformed unsupervised fine-tuning for adding knowledge to a model, both for facts seen in pre-training and for entirely new ones (Ovadia et al., 2023).
- Models learn new facts from fine-tuning slowly, and once they do, their tendency to hallucinate increases (Gekhman et al., 2024).
- Fine-tuning is the right tool when the problem is behaviour: inconsistent format, tone or reasoning on a defined task (OpenAI).
- The two combine. In one study, fine-tuning added over 6 percentage points of accuracy and RAG a further 5 (Balaguer et al., 2024).
- What goes into the weights cannot be filtered per user and is hard to remove, which matters for confidential plant data.
What is the difference between fine-tuning and RAG?
RAG leaves the model as it is. A search layer finds the passages in your documents that are relevant to the question, and the model is instructed to answer from them. The approach was introduced by Lewis et al. in 2020, whose paper notes that, for knowledge stored only in model weights, providing provenance for answers and updating that knowledge remain open problems. RAG addresses both: the answer can cite its passage, and a new revision is indexed rather than trained.
Fine-tuning continues training the model on your own examples, so its weights change. Full fine-tuning updates every parameter. Parameter-efficient methods train only a small number of extra parameters, which, according to the Hugging Face PEFT documentation, significantly decreases compute and storage costs while giving performance comparable to full fine-tuning. The best known is LoRA, which freezes the original weights and adds small trainable low-rank matrices to each layer (Hu et al., 2021).
In short, RAG changes what the model reads; fine-tuning changes how it responds. For the full retrieval pipeline over technical documents, see our guide to RAG for maintenance manuals.
Can fine-tuning teach a model your plant's documents?
This is the most common expectation, and the research does not support it.
Ovadia et al. (2023) compared unsupervised fine-tuning with RAG on knowledge-intensive tasks. Fine-tuning gave some improvement, but RAG consistently performed better, both for knowledge the model had met during pre-training and for entirely new knowledge. The authors conclude that models struggle to learn new facts through unsupervised fine-tuning, and suggest that exposing them to many variations of the same fact may help.
Gekhman et al. (2024) went further. In controlled experiments, models learned fine-tuning examples that contained new knowledge slowly, and once they had learned them, their tendency to hallucinate increased linearly. The authors' reading is that models acquire factual knowledge mostly during pre-training, and that fine-tuning mainly teaches them to use what they already know more efficiently.
For a plant, new knowledge is exactly what matters: equipment tags, torque values, alarm codes, procedure revisions. And a value absorbed during training comes with no page reference for the engineer to check.
How do RAG and fine-tuning compare?
| Criterion | RAG | Fine-tuning |
|---|---|---|
| What it changes | The context the model reads | The model's weights |
| Best at | Proprietary or up-to-date knowledge | Consistent format, tone or reasoning on a defined task |
| New document revision | Re-index the document | Rebuild training data and retrain |
| Citations to source pages | Yes, from the retrieved passages | No |
| Per-user access control | Filter passages at retrieval time | Not possible inside the weights |
| Removing a document | Delete it from the index | Retrain without it |
| Data you need | The documents and a test set of questions | Curated input and output examples |
| Extra on-premise compute | Embedding model and search index | Training runs on GPUs |
| Typical failure | Retrieval misses the right passage | Confident answers with no source; loss of general skills |
The "best at" row follows OpenAI's guide to optimizing accuracy, which points to retrieval for out-of-date or proprietary knowledge and to fine-tuning for inconsistent behaviour, and stresses that the two are additive, not exclusive.
When does fine-tuning make sense for an industrial assistant?
Fine-tuning earns its place when you can show the target behaviour with examples and a good prompt still cannot produce it reliably:
- A fixed output format. Work-order drafts, shift handover notes or inspection findings that must follow the template your CMMS or reporting tool expects.
- Classification. Sorting free-text maintenance reports by failure mode, equipment class or criticality.
- Answering style. Always citing document number, revision and page the same way, or refusing in set wording when the documents are silent.
- Efficiency. OpenAI lists reaching the same accuracy with fewer tokens or a smaller model as one reason to fine-tune (OpenAI). On your own GPUs, a smaller model means more users per server.
- Better use of retrieved passages. The RAFT method trains a model to ignore retrieved documents that do not help answer the question and to quote the passage that does. It improved domain-specific RAG results on the PubMed, HotpotQA and Gorilla datasets (Zhang et al., 2024).
You need fewer examples than many teams expect. OpenAI's fine-tuning guide sets a minimum of 10 and recommends starting with 50 well-crafted demonstrations, then evaluating; if 50 examples have no impact, it advises rethinking the task or prompt before adding more data (OpenAI). Those figures are for OpenAI's hosted service, but the principle carries over to open-weight models.
What does fine-tuning cost on your own GPUs?
Less than it used to. Compared with full fine-tuning of GPT-3 175B, LoRA cut the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times (Hu et al., 2021). QLoRA, which trains LoRA adapters on top of a 4-bit quantized model, made it possible to fine-tune a 65-billion-parameter model on a single 48 GB GPU (Dettmers et al., 2023).
There is a trade-off. One study found that, in standard low-rank settings, LoRA substantially underperforms full fine-tuning on programming and mathematics, but better preserves the base model's performance outside the target domain (Biderman et al., 2024). For an assistant that must still read French and Arabic documents and reason generally, forgetting less is often worth learning less.
Serving the result is straightforward. vLLM can serve LoRA adapters through its OpenAI-compatible server, several adapters can share one base model, and each request selects its adapter through the model parameter (vLLM documentation). The same documentation warns that loading adapters dynamically at runtime carries security risks and should not be used in production outside an isolated, fully trusted environment.
The larger cost is people: building the examples, running evaluations, and repeating both when the base model changes. For hardware, see on-premise LLM hardware and GPU sizing; for the full budget, what an industrial AI project costs.
What are the risks of fine-tuning on plant data?
Training data can be extracted. Carlini et al. (2020) recovered hundreds of verbatim sequences from GPT-2's training data, including names, phone numbers and email addresses, even when a sequence appeared in only one training document, and found larger models more vulnerable than smaller ones. Treat whatever goes into the weights as readable by every user of the model.
No per-user permissions. With RAG, a technician who cannot open an HSE investigation file never receives passages from it, because retrieval filters by user. A fine-tuned model has no such boundary. Fine-tune only on data that every user of that model is allowed to see.
Poisoned or wrong examples. OWASP lists data and model poisoning as LLM04 in its 2025 Top 10 and names fine-tuning as one of the stages where manipulated data can introduce vulnerabilities or backdoors. It recommends tracking data origins, version control for datasets and red-team testing (OWASP LLM04:2025). A superseded procedure left in the training set is a quieter version of the same problem.
Personal data. Maintenance logs and shift reports name people. In Algeria, Law 18-07, as amended by Law 25-11 of 24 July 2025, makes transfers of personal data abroad subject to authorisation by the national authority and requires an impact assessment for processing likely to result in a high risk, according to the CMS guide. Sending such data to a foreign fine-tuning service raises the transfer question directly. Confirm how these rules apply to your case with legal counsel.
Training on-premise, or in an air-gapped environment, removes the transfer issue but not the other risks.
Can you combine RAG and fine-tuning?
Yes, and the gains can stack. In a study on agricultural question answering, fine-tuning raised accuracy by over 6 percentage points and RAG added a further 5 on top (Balaguer et al., 2024). The usual division of labour: retrieval supplies the facts and the citations; a light fine-tune, if one is needed at all, makes the model dependable at the format and the task.
When the assistant drafts work orders or queries the ERP rather than only answering, a person should approve each action. See chatbot vs copilot vs AI agent and human-in-the-loop AI agents.
How do you decide, step by step?
- Write the evaluation set first. Collect real questions and tasks from technicians and engineers, with correct answers and source passages. OpenAI's guide works from a baseline of 20 or more questions with known answers and advises investing in fine-tuning only after evaluations are in place (accuracy guide; fine-tuning guide). For a plant, include questions in every language your teams use and questions the documents cannot answer.
- Start with a good prompt on a capable open-weight model. In OpenAI's words, prompt engineering is "typically the best place to start".
- Add RAG over a pilot document set, with hybrid search, citations and permission filtering.
- Sort each remaining failure. Wrong or missing facts are a context problem: fix the OCR, chunking, retrieval or the documents themselves. Right facts in the wrong form are a behaviour problem.
- Fine-tune only for behaviour problems that persist. Start with about 50 clean examples and a LoRA adapter, and compare the result against the evaluation set.
- Put the adapter under change control. Version the training data, re-run the evaluations whenever the base model or adapter changes, and log which adapter produced which answer.
What we have seen at KABAS AI
DALILE, our on-premise assistant for industrial knowledge, is built as a multi-agent system that analyses technical documents and connects to ERP, CRM, DMS and databases, with deployment fully on-premise and data under the client's control. That design follows the logic above: plant knowledge lives in documents and systems that keep changing, so the assistant reads them when asked instead of carrying a frozen copy in its weights.
Our advice when scoping a private assistant:
- Spend the first weeks on documents, retrieval and the test set, not on training.
- Do not expect fine-tuning to compensate for missing or messy documentation. It will not fix a manual that was never scanned or a procedure with three revisions in circulation.
- When a specific output is required, such as a report template, try instructions and examples in the prompt first, and train an adapter only if the evaluation shows a gap that persists.
- Keep any fine-tuned weights, training data and evaluation results on the client's infrastructure and under the client's ownership.
If you are building assistants that act on plant systems, our agentic AI services cover design and deployment. The sovereign AI on-premise guide covers infrastructure and governance.
Sources
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv, 2020)
- Ovadia et al., Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs (arXiv, 2023)
- Gekhman et al., Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (arXiv, 2024)
- Balaguer et al., RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture (arXiv, 2024)
- Zhang et al., RAFT: Adapting Language Model to Domain Specific RAG (arXiv, 2024)
- Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (arXiv, 2021)
- Dettmers et al., QLoRA: Efficient Finetuning of Quantized LLMs (arXiv, 2023)
- Biderman et al., LoRA Learns Less and Forgets Less (arXiv, 2024)
- Carlini et al., Extracting Training Data from Large Language Models (arXiv, 2020)
- Hugging Face, PEFT documentation
- OpenAI, Optimizing LLM accuracy
- OpenAI, Supervised fine-tuning
- vLLM documentation, LoRA adapters
- OWASP, LLM04:2025 Data and Model Poisoning
- CMS Expert Guide to data protection and cyber security laws: Algeria
Frequently asked questions
Can fine-tuning teach a language model the content of our technical documents?
Not reliably. In published comparisons, retrieval-augmented generation (RAG) outperformed unsupervised fine-tuning for adding knowledge, and fine-tuning on facts the model did not already know increased its tendency to hallucinate. RAG also lets every answer cite the page it came from, which fine-tuning cannot.
When is fine-tuning worth it for an industrial assistant?
When the problem is behaviour rather than knowledge: a fixed output format such as a work-order template, classification of free-text reports, a consistent answering style, or getting a smaller model to do one narrow task well. Try prompting and RAG first, and fine-tune only for failures that remain on your evaluation set.
Can we use RAG and fine-tuning together?
Yes. Retrieval supplies the facts and citations, and a light fine-tune can make the model more consistent at the format or task. In one published study, the accuracy gains from fine-tuning and from RAG added up.
Do we need large GPU servers to fine-tune on-premise?
Less than before. Parameter-efficient methods such as LoRA train only small added matrices, and QLoRA was shown to fine-tune a 65-billion-parameter model on a single 48 GB GPU. The larger cost is usually the people who build the training examples and run the evaluations.
Is it safe to fine-tune a model on confidential or personal data?
Treat anything in the training data as readable by every user of the model. Research has shown that verbatim training text, including personal details, can be extracted from language models, and a fine-tuned model cannot apply per-user document permissions. Keep personal and restricted data in a permission-filtered RAG index instead, and check data protection obligations with counsel.



