Table of Contents
Keep an on-premise LLM up to date by treating it as four parts that age at different speeds: the documents it answers from, the software that serves it, the model weights and the embedding model behind search. Re-index documents as they change, patch software on a fixed cycle, and change models only when your test set shows a gain.
Key takeaways
- A model's built-in knowledge stops at its training cutoff. Llama 3.3 70B was released on 6 December 2024 with pretraining data that ends in December 2023 (Meta).
- Plant knowledge belongs in the document index, not in the weights. Keeping it current means re-indexing, not retraining.
- The serving stack needs its own patch cycle. NVIDIA's May 2026 bulletin for its GPU display drivers covered data-centre GPUs on Linux, with a highest severity score of 8.8 (NVIDIA).
- On your own servers, the model changes only when you change it. Cloud models are retired on the provider's schedule (OpenAI).
- Every change goes through the same gate: test set, staging, limited rollout, and a way back.
What actually goes out of date in an on-premise LLM?
"Updating the AI" usually means one of five different jobs. They have different triggers, owners and risks, so plan them separately.
| Component | What goes stale | Typical trigger | How to update | What to test |
|---|---|---|---|---|
| Documents and index | New revisions, reports, work orders | A document is approved or withdrawn | Incremental re-index; mark superseded revisions | Questions on the changed documents |
| Inference server, GPU driver, operating system | Security fixes, support for new models | Security advisory or vendor bulletin | Patch on a fixed cycle, urgent path for severe issues | Full test set, response time, memory |
| LLM weights | Knowledge cutoff; better models appear | A candidate passes your test set | Staged swap with rollback | Full test set in every language you use |
| Embedding model and reranker | Retrieval quality, language coverage | Retrieval tests show a gain | Full re-index in parallel, then cut over | Retrieval recall on your questions |
| Prompts and fine-tuned adapters | Tied to one model version | Any model change | Rewrite prompt templates; retrain adapters | Task-specific tests |
For the deployment options and governance around these components, see our guide to sovereign, on-premise AI.
Why not retrain the model on new documents?
Because that is not where current knowledge should live. Every model has a training cutoff, and it is often well before release: Llama 3.3 70B shipped a year after its pretraining data ended (Meta). Your procedures change every week.
The paper that introduced retrieval-augmented generation (RAG) named "updating their world knowledge" as an open problem for models that store knowledge only in their weights (Lewis et al., 2020). RAG solves it by giving the model the relevant passages at question time. Later studies found that retrieval consistently outperformed unsupervised fine-tuning for adding knowledge (Ovadia et al., 2023), and that models learn new facts from fine-tuning slowly, then hallucinate more once they have learned them (Gekhman et al., 2024). We compare the two approaches in fine-tuning vs RAG.
In practice, a new revision of a lockout procedure should be searchable the day it is approved, with no GPU training run and no model change.
How do you keep the document index current?
Tie indexing to your document approval process, not to a calendar.
- One source of truth. Connect to the document management system or file share where approved revisions live, not to copies on someone's laptop.
- Index on approval. When a revision is approved, index it and mark the previous one superseded in the same step. Our RAG guide for maintenance manuals explains why an obsolete revision in the index is a safety issue.
- Handle deletions. Withdrawn documents must leave the index, not just stop being updated.
- Re-sync permissions. Access rights change at the source; the index must follow, as described in securing an on-premise LLM.
- Version the index. Record which document revisions each index build contains, so an answer can be traced to what the system knew at the time.
How often should you patch the serving stack?
More often than the model. The inference server, GPU driver, CUDA libraries, container images and operating system all receive security fixes.
The vLLM project, an open-source inference server, publishes security advisories on GitHub, rated from low to high severity; many concern requests that can crash or hang the server. Our build vs buy analysis counts how many appeared in one year.
GPU drivers matter too. NVIDIA's GPU display driver bulletin of 19 May 2026 listed affected Linux driver branches R595, R580 and R535 for Tesla data-centre GPUs, among other product lines. NVIDIA states that the vulnerabilities might lead to denial of service, escalation of privileges, information disclosure, data tampering and code execution, and the highest-rated issue scored 8.8 (NVIDIA).
A workable routine:
- Pin exact versions of every component, and keep a list of them.
- Review new advisories on a fixed cycle, and decide for each one whether it affects you.
- Keep an urgent path for high-severity issues that affect your configuration.
- Apply every patch on a staging server first, then run the test set, because a driver or server upgrade can change speed, memory use and even answers.
On an isolated network, each update crosses the gap through a controlled transfer; air-gapped AI deployment covers that process.
When should you upgrade the model itself?
Less often than you might think. An on-premise model keeps behaving the same way until you replace it, which is a real advantage for validated industrial workflows.
Cloud models do not work that way. OpenAI's deprecation page, for example, shows gpt-4.5-preview announced for deprecation on 14 April 2025 and shut down on 14 July 2025, and o3 announced on 11 June 2026 for shutdown on 11 December 2026 (OpenAI). Behaviour can also shift between versions of a hosted model: researchers measured GPT-4 telling prime from composite numbers with 84% accuracy in its March 2023 version and 51% in its June 2023 version (Chen, Zaharia and Zou, 2023).
Good reasons to upgrade:
- A candidate scores clearly better on your own test set, including Arabic and French questions.
- You need a capability the current model lacks, such as a language or a longer usable context.
- The current model or its serving support is no longer maintained.
- A smaller model now passes the same tests and frees GPU memory.
Weak reasons: a leaderboard position or a release announcement. Our guide to choosing an open-weight LLM covers licence, size and language checks, which you must repeat for each new version.
Three practical details:
- Pin by commit. Hugging Face lets you download a model at a specific branch, tag or commit hash, and requires the full-length hash when you use one (Hugging Face). Record that hash with the file checksums.
- Check the prompt format. A new model can expect a different chat template, so prompts need adapting before you compare answers.
- Retrain adapters. LoRA "freezes the pre-trained model weights" and adds small trainable matrices to them (Hu et al., 2021). An adapter therefore belongs to the base model it was trained on, and a new base model means training it again.
What happens when you change the embedding model?
You rebuild the whole index. A new embedding model produces vectors in a different space, and similarity between old and new vectors is meaningless, so the two cannot share an index (Zilliz).
The safe pattern is to build the new index alongside the old one, re-embed every document in the background while the old index keeps serving, validate the new one against a held-out set of queries, then switch over in one step (Zilliz). Keep the parsed text of every document, so a re-index does not mean running OCR on thousands of scans again.
How do you roll out an update safely, step by step?
Machine learning systems are tightly coupled. Google engineers called this the CACE principle, "Changing Anything Changes Everything", and applied it to hyper-parameters, data selection and "essentially every other possible tweak" (Sculley et al., 2015). Treat every update the same way, however small.
- Register the change. Record what changes, why, the exact versions and hashes, and who approved it. A model registry such as MLflow gives each new registration of a model the next version number and lets production refer to a named alias, such as "champion", instead of a fixed version (MLflow).
- Install on staging. Use a server that mirrors production: same GPU type, drivers and configuration.
- Run the full test set. Compare old and new on the same questions, in Arabic, French and English. Treat any new failure on a critical question as blocking. Our guide to evaluating an LLM on your documents sets out the method.
- Check speed and memory at your expected number of simultaneous users.
- Run a canary. Google's SRE workbook defines canarying as "a partial and time-limited deployment of a change in a service and its evaluation". Give the new version to a small group, compare it with everyone else, and also check absolute thresholds (Google SRE).
- Switch, and keep the way back. Move the alias or configuration entry to the new version. Keep the previous model, images and index on disk.
- Monitor in production. The NIST AI Risk Management Framework calls for monitoring the system's behaviour in production and for post-deployment plans that include incident response, recovery and change management (NIST AI RMF 1.0). Collect user reports of wrong answers and add them to the test set.
Who should own each update, and how often?
The cadences below are a starting point to adapt, not a standard.
| Update | Suggested rhythm | Owner |
|---|---|---|
| Documents and permissions | Continuous, on approval or change at the source | Document owners, platform administrator |
| Security advisories | Reviewed on a fixed cycle, urgent path for severe issues | IT security and infrastructure |
| Driver and operating system | On your existing patch cycle | IT infrastructure |
| Model and embedding candidates | Reviewed once or twice a year, or when a need appears | AI system owner with the engineers who use it |
| Test set | After every incident or reported wrong answer | Engineers who use the assistant |
Without named owners, updates stop after the project team leaves, and quality degrades quietly.
What we have seen at KABAS AI
DALILE runs on the client's own servers, works without an internet connection, and connects to systems such as SAP, Oracle, Microsoft 365, LIMS, document management systems and SQL databases. We handle the setup and train the client's team. Because the system sits inside the client's network, every update follows the client's own change and security process, and we plan for that from the start.
The guidance we give follows from that:
- Agree the update process before go-live: who approves model changes, how files reach the servers, and how quickly a severe advisory is handled.
- Keep the application independent of any one model, so a better open-weight release can replace the current one after testing, without rewriting integrations.
- Keep the test set, model files and update records on the client's infrastructure, under the client's ownership.
- Separate document updates from model updates. The first is daily work for document owners; the second is an occasional, tested project.
If you want help designing the update process or reviewing an existing deployment, our consulting team can work through it with your IT, OT and security teams.
Sources
- Meta, Llama 3.3 70B Instruct model card, Hugging Face
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, 2020
- Ovadia et al., Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs, 2023
- Gekhman et al., Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?, 2024
- vLLM, GitHub security advisories
- NVIDIA, Security Bulletin: NVIDIA GPU Display Drivers, May 2026
- OpenAI, Deprecations
- Chen, Zaharia and Zou, How is ChatGPT's behavior changing over time?, 2023
- Hugging Face, Download files from the Hub
- Hu et al., LoRA: Low-Rank Adaptation of Large Language Models, 2021
- Zilliz, How do you change your embedding model without re-indexing everything?
- Sculley et al., Hidden Technical Debt in Machine Learning Systems, NeurIPS 2015
- MLflow, Model Registry
- Google SRE Workbook, Canarying Releases
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023
Frequently asked questions
How often should an on-premise LLM be updated?
There is no single interval, because the parts age at different speeds. Documents should be re-indexed as soon as a new revision is approved. Security advisories for the inference server, drivers and operating system should be reviewed on a fixed cycle, with an urgent path for high-severity issues. The model itself should change only when a candidate passes your own test set with a clear gain.
Do we need to retrain the model when our documents change?
No. In a retrieval-augmented (RAG) setup the model reads your documents at question time, so a new procedure revision is indexed, not trained. Research comparing the two approaches found that retrieval outperformed fine-tuning for adding knowledge, and that models learn new facts from fine-tuning slowly.
Will a new model version work with our existing prompts and fine-tuning?
Not automatically. A new model may expect a different chat template or prompt format, may handle Arabic and French differently, and may come under a different licence. A LoRA adapter is trained on top of one specific base model, so it has to be retrained for a new one. Re-run the full test set before switching.
What happens if we never upgrade the model?
An on-premise model keeps behaving the same way until you change it, which is one of its advantages. Its built-in knowledge stays fixed at its training cutoff, but in a RAG system current knowledge comes from the documents. What you cannot skip is patching the software around it: the inference server, GPU drivers and operating system receive security fixes.
How do we roll back a bad update?
Keep the previous model files, container images and document index on disk until the new version has run in production for an agreed period. Point the application at versions through a named reference, such as a registry alias or a configuration entry, so rolling back means switching that reference back rather than reinstalling.



