Table of Contents
Buy, unless you already run a capable platform team and see the LLM stack as a long-term internal product. Assembling one from open-source parts is feasible, but the model is the small part. The larger, permanent work is integration, security patching, licence review, evaluation and support, and it stays with you after the project ends.
Key takeaways
- A private LLM stack has about ten layers, from GPU drivers to access control and answer evaluation. The model is only one of them.
- Building means owning the upkeep. vLLM alone shipped 33 releases and published 96 security advisories in the twelve months to October 2026.
- "Free" components can carry conditions: Open WebUI's licence restricts removing its branding above 50 users, and Llama's licence requires "Built with Llama" attribution on products that contain it.
- "Buy" has three forms: supported open-source components, a packaged on-premise product, or an integrator's custom build. Each moves a different part of the work off your team.
- Decide on your data, your staff and the three-year cost, and write exit terms into any purchase.
What is actually in an on-premise LLM stack?
A private assistant over your manuals, procedures and reports is not one piece of software. It is a chain of layers, and each one needs an owner.
| Layer | What it does | What you own if you build it |
|---|---|---|
| GPU servers, drivers, container runtime | Runs everything else | Driver and firmware updates, monitoring, spare parts |
| Inference server (for example vLLM) | Loads the model and serves requests | Version upgrades, security patches, performance tuning |
| Model weights (for example gpt-oss-120b) | Generates the answers | Licence review, choice of version, regression testing on upgrade |
| Document ingestion and OCR | Turns PDFs, scans and spreadsheets into text | Parsers for your formats, Arabic and French text quality |
| Embedding model and vector index | Finds the relevant passages | Re-indexing when documents or the embedding model change |
| Retrieval and orchestration code | Builds the prompt from passages and the question | Your own code: bugs, tests, prompt changes |
| User interface | Where engineers ask questions | Licence terms, language support, right-to-left display |
| Identity and permissions | Limits who sees which documents | Mapping to Active Directory or your document system |
| Logging, monitoring, backups | Traceability and recovery | Retention rules, who may read the logs |
| Evaluation | Measures whether answers are right | A test set from your engineers, rerun after every change |
Our guide to sovereign, on-premise AI covers the deployment options and governance around this stack, and GPU sizing for a plant covers the bottom layer.
Why is the model the smallest part of the work?
This is not new to LLMs. In a widely cited 2015 paper, Google engineers argued that "only a small fraction of real-world ML systems is composed of the ML code", and that a mature system "might end up being (at most) 5% machine learning code and (at least) 95% glue code" (Sculley et al., 2015). They also warned that "it is common to incur massive ongoing maintenance costs in real-world ML systems."
Retrieval systems follow the same pattern. An experience report on three RAG systems, in research, education and biomedicine, concluded that "validation of a RAG system is only feasible during operation" and that "the robustness of a RAG system evolves rather than designed in at the start" (Barnett et al., 2024). In practice, the questions your engineers ask in month six will expose problems no pilot found. Someone has to be there to fix them.
What does building commit you to after go-live?
Keeping up with releases. The components move fast. Between 1 November 2025 and 31 October 2026, vLLM, an open-source inference server under the Apache 2.0 licence, published 33 releases, from v0.11.1 to v0.32.0: roughly one every 11 days. New models often need a recent version, and each upgrade should be tested before it reaches users.
Tracking security advisories. Over the same twelve months, the project published 96 security advisories, 2 rated critical and 21 high (our count via the GitHub API, 10 November 2026). Many concern denial of service; some concern remote code execution. Open disclosure is a good sign, but each advisory is a decision for your team: does it affect us, and when do we patch?
Hardening defaults. The vLLM documentation states that "all communications between nodes in a multi-node vLLM deployment are insecure by default", recommends a dedicated, isolated network, and notes that its --api-key option protects only some endpoints, so a reverse proxy should allow only the ones you intend to expose (vLLM security guide). Skipping this happens in practice: in September 2025, Cisco researchers found 1,139 Ollama servers reachable from the internet, 214 of them answering prompts without authentication (Cisco).
Evaluating after every change. A new model version, embedding model or parser can improve English answers and degrade Arabic ones. Without a test set rerun after each change, you find out from users. Our guide to evaluating an LLM on your own documents explains how to build one, and securing an on-premise LLM covers the controls.
Which licences hide inside a self-built stack?
Open-source does not always mean unconditional. Three examples a legal team should see before go-live:
- User interface. From version 0.6.6 (April 2025), the Open WebUI licence forbids altering or removing its branding unless the deployment has 50 or fewer users in a 30-day period, you are a contributor with specific written permission, or you hold an enterprise licence that allows it. Code up to v0.6.5 remains BSD-3.
- Model. If you distribute or make available the model, or a product or service that contains it, the Llama 3.1 Community License requires you to prominently display "Built with Llama". If you use it to build or fine-tune a model that you distribute or make available, that model's name must begin with "Llama". By contrast, gpt-oss-120b is published under Apache 2.0.
- Everything else. OWASP's supply chain entry for LLM applications notes that AI development mixes software and dataset licences, and recommends keeping a signed software bill of materials (SBOM) and a licence inventory, auditing them regularly, and checking model files with hashes or signatures (OWASP LLM03:2025).
None of this rules out open source. It makes licence review part of the build.
What are the ways to buy?
"Buy" is not one option. There are three, and they move different parts of the work off your team.
1. Supported open-source components. You assemble the stack, but buy support for its core. Red Hat AI Inference Server, announced in May 2025, is a hardened, supported version of vLLM that runs standalone in a container or inside RHEL AI and OpenShift AI (Red Hat). NVIDIA AI Enterprise is licensed per GPU, at a list price of $4,500 per GPU for one year including Business Standard support, according to NVIDIA's licensing guide (last updated September 2026, checked November 2026) (NVIDIA). You still build ingestion, retrieval, the interface and permissions.
2. A packaged on-premise product. The vendor delivers the whole chain, from ingestion to interface, installed on your servers, and maintains it. You configure it and own the data. DALILE is our product in this category.
3. An integrator's custom build. A partner assembles open-source components to your specification. You get a stack that fits, but check who maintains it after handover, and at what cost.
How do building and buying compare?
| Criterion | Build from open source | Supported components | Packaged on-premise product |
|---|---|---|---|
| Software licence cost | Usually none | Per GPU or per server subscription | Licence or subscription |
| Integration work by your team | All of it | Everything above the supported layer | Configuration and connectors |
| Security patching | Your team, for every layer | Shared: vendor patches the supported layer | The vendor, if the contract says so |
| Skills needed after go-live | GPU operations, containers, Python, security, evaluation | Most of the same | Administration and evaluation |
| Time to first pilot | Depends on your team's experience | Shorter for the core, same for the rest | Shortest if the product fits your use case |
| Control over the code | Complete | High | Limited to what the vendor exposes |
| Main risk | Key people leave; upkeep slips | Gaps between supported and unsupported layers | Vendor lock-in; product does not fit |
| Best fit | Groups with a platform team and several AI projects planned | Groups with IT operations but no AI developers | One or two well-defined use cases, small IT team |
Hardware is similar in all three. To compare the server with cloud APIs, see on-premise LLM vs cloud API cost.
How should you decide?
- Write down the use cases and the data. Which documents, which languages, which users, and whether any of it may leave the site. If nothing may leave, every option must run fully offline; see air-gapped AI deployment.
- Count the people you will still have in three years. Name who will patch, upgrade and evaluate the stack. If those names are one person, or contractors on a fixed project, building is fragile.
- List the layers you must customise. If your needs differ from a standard product in only one or two layers, such as a connector to your maintenance system, buy and extend. If they differ in most layers, building may be justified.
- Build a test set first. Fifty to a hundred real questions with reference answers, in Arabic, French and English as your engineers write them. Use it to compare every option on equal terms.
- Price three years, not year one. Include your own staff time for operations and upgrades on the build side, and licences, support and renewals on the buy side.
- Run a pilot on your own documents. Agree the pass mark before it starts.
- Negotiate the exit before you sign. Data, indexes, configurations and any fine-tuned weights should be exportable in open formats.
Our 15 questions for choosing an industrial AI vendor turns steps 6 and 7 into a score sheet.
Does Algerian procurement law affect the choice?
It shapes how you buy, for both routes, since building still means purchasing hardware and often services. This summarises the published text of Law 23-12 of 5 August 2023 on public procurement; confirm your case with your procurement department and counsel.
- Article 13: public economic enterprises are not bound by the law's award procedures, but must adopt their own, based on its principles.
- Article 41: direct negotiated procedure is allowed in listed cases, including labelled start-ups providing digital and innovation services, provided the solution is unique and innovative.
- Article 62: a preference margin applies to products of Algerian origin and to companies under Algerian law majority-owned by resident nationals.
What we have seen at KABAS AI
We sell a packaged product, so read this section with that in mind, and ask us the same questions you would ask anyone.
DALILE is built for the case where buying makes sense: it runs on your own servers, works without an internet connection, and connects to systems such as SAP, Oracle, Microsoft 365, LIMS, document management systems and SQL databases, as described on the DALILE page. We handle the setup, build the connections and train your team. That takes the integration work and the interface off your plate, but not everything: you still need to own the use cases, the documents, the access rules and the test set.
If you have a strong platform team and want to build, keep the model replaceable behind an OpenAI-compatible API, write the test set before choosing components, put licence review and advisory tracking in someone's job description, and budget the upkeep from day one.
If you are weighing the two routes, our consulting team can run the decision with your data, people and budget. To see DALILE on your own documents, contact us.
Sources
Figures for vLLM releases and advisories were counted on 10 November 2026. Prices change; recheck them before you decide.
- Sculley et al., Hidden Technical Debt in Machine Learning Systems, NeurIPS 2015
- Barnett et al., Seven Failure Points When Engineering a Retrieval Augmented Generation System, 2024 (arXiv 2401.05856)
- vLLM releases, GitHub
- vLLM security advisories, GitHub
- vLLM documentation: Security
- Cisco: Detecting Exposed LLM Servers, a Shodan case study on Ollama, 1 September 2025
- Open WebUI: License
- Meta: Llama 3.1 Community License Agreement
- Hugging Face: openai/gpt-oss-120b model card
- OWASP: LLM03:2025 Supply Chain
- Red Hat: Red Hat AI Inference Server press release, 20 May 2025
- NVIDIA AI Enterprise licensing guide: Pricing
- Law 23-12 of 5 August 2023 on public procurement, Journal officiel no. 51 (FAOLEX copy)
Frequently asked questions
Is it cheaper to build an on-premise LLM stack from open-source components?
The software licences are often free, but the stack is not. You pay in engineering time to integrate the components, and then every year to patch, upgrade, monitor and evaluate them. Compare a three-year cost that includes your own staff time against the price of a supported platform or product, not against a licence fee of zero.
What skills does a company need to run its own LLM stack?
At minimum: Linux and GPU server administration, container operations, Python, network security, identity and access management, and someone who can evaluate answer quality on your own documents. Those skills must be available after go-live, not only during the project, because the components change frequently.
Can we start with a packaged product and move to our own stack later?
Yes, if the contract allows it. Ask for an export of your documents, indexes, configurations and any fine-tuned weights in open formats, and prefer products that use standard interfaces such as an OpenAI-compatible API, so the model or the application can be replaced later.
Is open-source LLM software safe to run on an industrial network?
It can be, with the same discipline as any other server software. The vLLM project, for example, published 96 security advisories on GitHub between November 2025 and October 2026, and its documentation warns that communication between nodes is insecure by default. Someone must track advisories, apply updates and keep the servers on an isolated network segment.
Does the choice between build and buy change the hardware we need?
Not much. Both routes need GPU servers sized for the model, the document lengths and the number of requests in flight. What changes is who installs, tunes and maintains the software running on them, and who answers when it fails.



