Table of Contents
An air-gapped AI system runs on servers with no network path to the internet. To make it work, you download the model files, inference software, container images and packages on a connected machine, check them, carry them across through a controlled transfer process, and configure every component so it never tries to call home. Updates follow the same path.
Key takeaways
- A large language model does not need the internet to run. It needs its weight files, an inference server and their dependencies on local disk.
- Everything that crosses the gap should be hashed, scanned and recorded. Prefer safetensors files: pickle-based model files can execute code when loaded (Hugging Face).
- Libraries must be told they are offline. Hugging Face tools have an
HF_HUB_OFFLINEsetting, and vLLM collects usage statistics unless you opt out (vLLM). - An air gap reduces risk but does not remove it. Attackers have reached air-gapped government systems through USB drives (ESET).
- The hard part is the update process, not the first install. Write it down, test it on a staging system and give it an owner.
What does "air-gapped" mean for an AI system?
An air-gapped system has no network connection to the internet or to less trusted networks. Data moves in and out only through a deliberate process: removable media, a one-way transfer device or a dedicated transfer station.
For an AI system, the model, the inference engine and the document index all sit inside the isolated zone, and users reach them from inside the same zone. Nothing reaches out: no licence check, no telemetry, no automatic model download.
This is the strictest of the deployment options compared in our guide to sovereign, on-premise AI. It suits workloads close to control systems, classified engineering data, or remote sites without a reliable link. For an on-premise server with a controlled internet connection, much of this article is optional.
What has to cross the air gap?
A working LLM assistant is more than a model file. Everything below must be brought in for the first install, and again whenever it changes.
| Component | Examples | How it is packaged | When it changes |
|---|---|---|---|
| Model weights | LLM, embedding model, reranker | safetensors files plus configuration and tokenizer files | When you decide to upgrade |
| Inference server | vLLM, NVIDIA NIM | Container image saved as a tar archive | When a security fix or needed feature is released |
| Python dependencies | Document parsing and retrieval libraries | Wheel files collected with pip download | With each application release |
| Operating system and GPU stack | Linux packages, NVIDIA driver, CUDA | Local package mirror or vendor bundle | On your operating system and vendor patch cycle |
| Application | Retrieval pipeline, user interface, connectors | Container images | With each release |
| Content | Manuals, procedures, reports | Documents, indexed inside the zone | As your documents change |
Two standard tools cover most of the packaging. docker image save writes one or more images to a tar archive, which can then be restored with docker load (Docker). pip download resolves and downloads packages exactly as pip install would, and saves them to a directory that can later be used for offline installation (pip).
Do not forget the document side. A retrieval assistant needs a parser and an embedding model that also run locally. Docling, an MIT-licensed document converter, lists local execution for sensitive data and air-gapped environments among its features (Docling). Our guide to RAG on maintenance manuals covers that pipeline.
How do you make the software run without internet?
Many AI tools assume they are online. They check for newer model versions, download missing files on first use, or send usage statistics. In an air gap those calls fail, sometimes with a delay, sometimes with an error at startup. Configure each component explicitly.
Download on the connected side, load from disk on the isolated side. The Transformers documentation describes this workflow: download the model repository ahead of time with snapshot_download, then set HF_HUB_OFFLINE=1 to prevent HTTP calls to the Hub, or load with local_files_only=True (Hugging Face Transformers). In offline mode only cached files are used, and an error is raised if a file is missing (huggingface_hub). That error is useful: during testing, it tells you something was not brought across.
Disable telemetry. vLLM collects anonymous usage statistics by default, including hardware details and model architecture. Setting VLLM_NO_USAGE_STATS=1 or DO_NOT_TRACK=1 turns this off (vLLM). Hugging Face libraries respect HF_HUB_DISABLE_TELEMETRY=1 (huggingface_hub). The data could not leave anyway, but switching it off avoids failed connection attempts and shows auditors the intent.
Use vendor air-gap procedures where they exist. NVIDIA documents an air-gap deployment for its NIM containers for LLMs: download the model profile into a cache on a connected machine, copy the cache to the air-gapped system, and start the container without an NGC API key (NVIDIA NIM).
Test the cold start with the network unplugged. Before going live, start the full stack on a staging server with no network connection at all. Every hidden download shows up as an error.
How do you check model files before they enter?
Once a file is inside, it tends to be trusted. Checks belong before the transfer.
Prefer safetensors. Pickle, the default format for PyTorch weights, can execute arbitrary code when a file is loaded, and Hugging Face advises loading models from users and organisations you trust (Hugging Face). Safetensors is a format built to store tensors safely, as opposed to pickle (safetensors). If a model is published only as pickle files (.bin, .pt), look for another source or convert it inside an isolated sandbox.
Verify hashes on both sides. Compute SHA-256 checksums on the connected machine right after download, and again inside the air gap after transfer. Any difference means the file changed on the way.
Sign what you approve. The OWASP Top 10 for LLM applications notes that published models carry no strong provenance assurances, and recommends integrity checks using signing and file hashes, plus a software bill of materials (OWASP LLM03:2025). Sigstore's open-source model signing tool can sign a model directory with your own private key and verify it against a public key file (Sigstore model-transparency).
Keep a register. For each item: name, version, source, hash, who approved it and the date it entered.
Is an air gap enough to keep an AI system safe?
No. It removes the network path but creates a physical one. ESET researchers documented a group called GoldenJackal that used custom tools to compromise air-gapped systems through USB drives: at a South Asian embassy in Belarus from at least August 2019, and at a government organisation in a European Union country from May 2022 to March 2024 (ESET).
The UK National Cyber Security Centre's pattern for safely importing data into a high-security system lists the controls for the inbound path: transform content to remove anything potentially active, verify it, break the network protocol, and enforce one-way flow, for example with a data diode (NCSC). The NCSC notes that one-way flow does not stop an exploit hidden in the content, but it makes remote control and data exfiltration harder.
| Transfer method | How it works | Main risk | Controls |
|---|---|---|---|
| Removable media | USB drive or disk carried between zones | Malware on the media; lost media | Dedicated, inventoried media; scanning station; hash check on arrival |
| Transfer station | A dedicated machine scans and copies content between zones | The station itself is compromised | Hardened and patched station; every transfer logged |
| Data diode | Hardware that lets data flow in one direction only | Malicious content still passes | Transform and verify content before it reaches the diode |
Software inside the gap still has vulnerabilities. In October 2026 alone, the vLLM project published several security advisories, one rated high severity (vLLM security advisories, checked November 2026). An air gap buys time to patch; it does not remove the need to.
If the AI system reads plant data, keep that path read-only and one-way, as described in integrating AI with SCADA and OT systems.
How do you update an air-gapped AI system, step by step?
- Decide what to update. Review security fixes for the inference server, operating system and drivers on a fixed cycle, with an urgent path for high-severity issues. Upgrade the model only when an evaluation shows a gain.
- Assemble the bundle on a connected build machine. Download model files, save container images, collect Python wheels and pin exact versions.
- Check and record. Scan for malware, confirm the safetensors format, compute hashes, sign the bundle and update the register.
- Transfer through the controlled path. Use dedicated media or a one-way transfer device. Verify hashes and signatures on arrival.
- Install on a staging system inside the gap. It should mirror production: same GPU type, drivers and configuration.
- Evaluate before switching. Run your test set of real questions, in Arabic, French and English, against the old and new versions. Compare answer quality, response time and memory use.
- Switch with a way back. Keep the previous model and images on disk so you can roll back quickly.
- Rebuild the index if the embedding model changed. Vectors from different embedding models are not compatible, so a new embedding model means re-indexing every document.
How large are the transfers?
Model weights dominate. Hugging Face's rule of thumb is roughly 2 GB per billion parameters at bfloat16 precision, so a 70-billion-parameter model is about 140 GB of weights (Hugging Face). 8-bit and 4-bit quantized versions are smaller. Choose media and procedures that handle hundreds of gigabytes, and allow time for hashing and scanning files of that size. For the GPU side, see on-premise LLM hardware and GPU sizing.
What we have seen at KABAS AI
DALILE, our platform for industrial documents, runs completely offline on your own servers, with no internet access required. Designing for that constraint leads to three practical recommendations.
Agree the update process with security before the first install. The transfer path, approvers and media are usually set by the site's security team. The AI system becomes one more user of that process, so define its bundle format, checks and frequency with them from the start.
Plan for documents, not only models. Model upgrades are occasional. The manuals, procedures and reports an assistant answers from change continually, and they need their own routine for parsing and indexing inside the zone.
Keep the staging system after go-live. It is where every update is tested with the network unplugged, and where you rehearse a rollback.
When you evaluate suppliers, ask how their product behaves offline and which features stop working; our guide to choosing an industrial AI vendor lists the questions. Our consulting team can help design the architecture and update process for an isolated site.
Sources
- Hugging Face: Pickle scanning and the risks of pickle files
- Hugging Face: Safetensors documentation
- Hugging Face Transformers: Installation and offline mode
- Hugging Face Hub: Environment variables
- Hugging Face: Optimizing LLMs for speed and memory
- vLLM: Usage stats collection
- vLLM: GitHub security advisories
- NVIDIA: Air gap deployment for NVIDIA NIM for LLMs
- Docker: docker image save
- pip: pip download
- Docling on GitHub
- OWASP Top 10 for LLM Applications 2025: LLM03 Supply Chain
- Sigstore: Model transparency and model signing
- ESET Research: Mind the (air) gap, GoldenJackal gooses government guardrails, 2024
- NCSC: Pattern for safely importing data
Frequently asked questions
Can a large language model run with no internet connection?
Yes. Once the model weights, the inference server and their dependencies are on local disk, nothing in the inference itself needs a network. Some libraries still try to check for updates or send usage statistics, so each one must be configured for offline use and tested with the network unplugged.
How often should an air-gapped AI system be updated?
There is no single right interval. Security fixes for the inference server, operating system and GPU drivers should be reviewed on a fixed cycle, with an urgent path for high-severity issues. Model upgrades should happen only when an evaluation on your own questions shows that the new model is better.
Is it safe to bring models in on a USB drive?
It can be, with controls. Removable media is a known attack route into air-gapped networks, so use dedicated, inventoried media, scan content on a separate station, check file hashes and signatures on both sides, and prefer the safetensors format over pickle-based model files.
What is a data diode and do I need one?
A data diode is a device that lets data flow in one direction only. The UK NCSC describes one-way flow control as one part of a safe import path, alongside content transformation and verification. It makes remote control and data theft harder, but it does not stop malicious content from passing, so the files still need checking.
Can we add new documents to an offline AI assistant?
Yes. In a retrieval-augmented (RAG) setup, new documents are parsed and indexed inside the isolated zone, so they never leave it. The document parser and the embedding model must also run locally. If you change the embedding model, the whole document index has to be rebuilt.



