Table of Contents
Choose an open-weight model by checking five things in order: a licence your legal team accepts, a size your GPUs can serve, stated and tested support for Arabic and French, a context length that fits your documents, and support in your serving software. Then shortlist two or three and test them on your own questions before deciding.
Key takeaways
- "Open-weight" is not the same as "open source". The Open Source AI Definition also requires training code and detailed information about the training data.
- Licences differ more than people expect. Qwen3's dense models, gpt-oss and Mistral Small 3.1 use Apache 2.0, while Llama and Gemma come with their own terms and use policies.
- Language claims vary. The Llama 3.3 model card lists eight supported languages without Arabic, while Llama 4 Scout lists Arabic and French.
- A claimed context window is not an effective one: in one benchmark, only about half of the models claiming 32,000 tokens or more performed satisfactorily at that length (Hsieh et al., 2024).
- The decision belongs to your own test set, not to public leaderboards.
What is an open-weight model, and is it open source?
An open-weight model is one whose trained parameters you can download and run on your own hardware. That is what makes an on-premise or air-gapped deployment possible: the model works without calling anyone's API.
Open source is a stricter idea. The Open Source Initiative's Open Source AI Definition 1.0 requires three things: the model parameters, the complete code used to train and run the system, and enough information about the training data for a skilled person to build a substantially equivalent system. It also requires the freedom to use the system "for any purpose and without having to ask for permission".
For a buyer, the consequence is simple: do not infer what you may do with a model from the word "open". Read its licence. For the wider picture of data, model, infrastructure and operations control, see our guide to sovereign, on-premise AI.
Which licence terms should you check first?
The licence decides whether a model is even a candidate, so check it before any testing. Three common licence types:
| Licence | Examples | What to check |
|---|---|---|
| Apache 2.0 | Qwen3 dense models, gpt-oss, Mistral Small 3.1 | Permissive: Mistral's card describes it as allowing use and modification for commercial and non-commercial purposes (Mistral AI). Keep the licence and notices when you redistribute. |
| Llama Community Licences | Llama 3.3, Llama 4 | A licence from Meta is needed above 700 million monthly active users; "Built with Llama" must be displayed; derived models must start their name with "Llama" (Llama 3.3 card). |
| Gemma Terms of Use | Gemma 3 | A prohibited use policy applies and must be passed on to anyone you redistribute to; Google "reserves the right to restrict (remotely or otherwise) usage" it believes breaches the terms (Gemma Terms of Use). |
Two details matter in practice. First, the licence can differ inside one family: Qwen's announcement states that the six dense Qwen3 models are released under Apache 2.0 but does not state a licence for the two mixture-of-experts models (Qwen), so check each model's own page. Second, gated models such as Llama and Gemma require you to accept terms on Hugging Face before downloading, which your procurement file should record.
This is not legal advice. Have counsel read the full licence text for the exact model version you plan to deploy.
How do well-known open-weight models compare?
The table compares examples, not a ranking. New versions appear every few months, so treat it as a template for your own shortlist. Details were checked on the model pages in November 2026.
| Model | Licence | Size | Context length | Arabic and French |
|---|---|---|---|---|
| Qwen3-32B | Apache 2.0 | 32.8B, dense | 32,768 native; 131,072 with YaRN | Qwen3 lists 119 languages and dialects, including French and Standard, Moroccan and Tunisian Arabic; Algerian is not listed (Qwen) |
| gpt-oss-120b | Apache 2.0 | 117B total, 5.1B active (MoE) | 131,072 in its configuration | Not stated on the model card |
| Mistral Small 3.1 | Apache 2.0 | 24B, dense | 128k | Both listed |
| Gemma 3 27B | Gemma Terms of Use | 27B, dense | 128K input, 8,192 output | "Over 140 languages"; the card does not name them |
| Llama 3.3 70B | Llama 3.3 Community License | 70B, dense | 128k | French listed; Arabic not among the eight supported languages |
| Llama 4 Scout | Llama 4 Community License | 109B total, 17B active (MoE) | 10M | Both listed |
How big a model can your hardware run?
Model size drives GPU memory, and GPU memory drives cost. As a rule, weights need about 2 GB per billion parameters at 16-bit precision and less after quantization. Our GPU sizing guide works through weights, KV cache and concurrent users in detail.
Mixture-of-experts (MoE) models change the calculation. All parameters must sit in memory, but only some are used for each token, which generally makes generation faster than a dense model of the same total size. gpt-oss-120b, with 5.1 billion active parameters, fits on a single 80 GB GPU, and gpt-oss-20b runs within 16 GB (OpenAI on Hugging Face). Llama 4 Scout fits on a single H100 with on-the-fly int4 quantization, according to its model card.
Note the precision behind each vendor claim. gpt-oss was post-trained with MXFP4 quantization of its MoE weights, and Llama 4 Scout's single-GPU figure assumes int4. Compare candidates at the precision you will actually run, and check answer quality at that precision too.
Does the model handle Arabic and French?
Algerian industrial documents mix Modern Standard Arabic (MSA), French and English. Start with what each model claims, then test.
The claims vary, as the table shows. Some families list Arabic explicitly; Llama 3.3 does not; gpt-oss and Gemma 3 do not name individual languages on their cards. A dialect list such as Qwen3's is useful, but if your technical documents are written in MSA and French, dialect coverage matters mainly for free-text fields such as shift notes.
Public benchmarks help build a shortlist. The Open Arabic LLM Leaderboard v2 removed machine-translated tasks and added native Arabic benchmarks, including a retrieval-augmented generation task called ALRAGE. Its authors hope future benchmarks will cover domain-specific areas, and most of its tasks are in MSA. No public leaderboard tests your maintenance reports.
Check tokenization as well. The same text can produce very different numbers of tokens in different languages, with differences of up to 15 times in some cases (Petrov et al., 2023). More tokens per page means less document fits in the context window and each answer takes longer. Run a sample of your Arabic and French documents through each candidate's tokenizer. Our article on Arabic, French and English documents covers the document side.
How much context length do you really need?
Context length is the number of tokens the model can read at once: your question, the retrieved passages and the answer. The worked example in our GPU sizing guide assumes a document question of about 6,500 tokens, retrieved passages and answer included. Every model in the table above offers far more.
Advertised maximums deserve caution. The RULER benchmark tested 17 long-context models, all claiming at least 32,000 tokens; most scored almost perfectly on a simple needle-in-a-haystack test, yet only about half kept satisfactory performance at 32,000 tokens on harder tasks (Hsieh et al., 2024). Long contexts also consume GPU memory for the KV cache.
The practical rule: decide the context you need from your own prompts, then test each candidate at that length, not at its maximum. If you are weighing retrieval against training the model on your documents, read fine-tuning vs RAG first.
Will your serving software run it?
A model is only useful if your inference server supports its architecture. vLLM, an open-source inference server, publishes a list of supported architectures and tells you to compare it with the architectures field in the model's config.json (vLLM). Qwen3, Gemma 3, Mistral, gpt-oss and Llama 4 all appear on it.
Some models also expect a specific prompt format. The gpt-oss card states that the models "should only be used with the harmony format as it will not work correctly otherwise" (OpenAI on Hugging Face). Check this before you compare answers, or you will be measuring a misconfigured model.
How do you download and verify model files safely?
Model files are software you run inside your network. Treat them that way.
- Prefer safetensors over pickle files. Hugging Face warns that loading a pickle file can execute arbitrary code, and recommends loading models only from users and organisations you trust (Hugging Face).
- Download from the publisher's official repository. OWASP notes that there are currently "no strong provenance assurances in published models" and recommends using only verifiable sources, with integrity checks based on signing and file hashes (OWASP LLM03:2025).
- Record hashes and versions in an inventory, so you know exactly which files run in production and can roll back.
- Bring files into isolated networks through your normal transfer and scanning process, not by a laptop download.
Our article on securing an on-premise LLM covers the rest of the stack.
How do you choose, step by step?
- Write down the use case and constraints. Languages, typical document length, expected simultaneous users, and whether the system must run air-gapped.
- Filter by licence. Remove any model whose terms your legal team will not accept for this use.
- Filter by hardware. Keep models whose weights, at a precision you accept, leave room for the KV cache on the GPUs you have or plan to buy.
- Filter by stated languages and serving support. Check the model card and your inference server's supported list.
- Shortlist two or three models, including one smaller than you think you need.
- Test on your own questions. Use real questions from engineers, in every language they use, with reference answers and source passages. Our guide to evaluating an LLM on your documents sets out the method.
- Compare quality, speed and memory together, quantized and unquantized, then choose the smallest model that passes with margin.
- Plan the next change. Record the version, licence and test results, and re-run the same test set whenever a new candidate appears.
If you are also deciding whether to assemble this stack yourself or buy a platform, see build vs buy for an on-premise LLM.
What we have seen at KABAS AI
DALILE runs on the client's own servers and answers questions from technical documents, spreadsheets and databases, so model choice is part of planning every deployment. This article describes how we approach that choice, not a benchmark we ran: the figures above all come from the public sources cited.
Three practices follow from it. Keep the application independent of any one model, so a better open-weight release can replace the current one after testing. Keep the test set, the model files and the results on the client's infrastructure. And let the client's own questions, in Arabic, French and English, decide between candidates rather than a leaderboard position.
If you want help building the shortlist and the test set, our AI consulting team can run the selection with your documents.
Sources
Model details checked in November 2026. They change with new releases; recheck before you decide.
- Open Source Initiative, The Open Source AI Definition 1.0
- Qwen, Qwen3: Think Deeper, Act Faster
- Qwen3-32B model card, Hugging Face
- openai/gpt-oss-120b model card, Hugging Face
- openai/gpt-oss-120b config.json, Hugging Face
- Mistral Small 3.1 model card, Hugging Face
- Gemma 3 27B model card, Hugging Face
- Google, Gemma Terms of Use
- Llama 3.3 70B Instruct model card, Hugging Face
- Llama 4 Scout model card, Hugging Face
- Hugging Face, The Open Arabic LLM Leaderboard 2
- Petrov et al., Language Model Tokenizers Introduce Unfairness Between Languages, 2023
- Hsieh et al., RULER: What's the Real Context Size of Your Long-Context Language Models?, 2024
- vLLM, Supported models
- Hugging Face, Pickle scanning
- OWASP, LLM03:2025 Supply Chain
Frequently asked questions
Is an open-weight model the same as an open-source model?
Not necessarily. Open-weight means you can download the model's parameters and run them yourself. The Open Source Initiative's definition of open-source AI also requires the training and inference code and detailed information about the training data, under terms that let anyone use, study, modify and share the system. Many popular open-weight models do not meet that definition.
Can we use Llama or Gemma models in an industrial company?
Their licences allow broad use but attach conditions. The Llama 3.3 licence requires a "Built with Llama" notice and a separate licence above 700 million monthly active users, and the Gemma Terms of Use include a prohibited use policy that you must pass on to anyone you redistribute to. Have your legal team read the full licence before you commit to a model.
Should we choose the largest model our budget allows?
No. Choose the smallest model that passes your own test set with margin. A smaller model leaves more GPU memory for long documents and simultaneous users, answers faster and is cheaper to replace when a better version appears. Mixture-of-experts models also change the trade-off, since only part of their parameters is used for each token.
Do we need an Arabic-specialised model for Arabic documents?
Not always. Several general models list Arabic among their supported languages, while some well-known ones do not. Public Arabic benchmarks help build a shortlist, but they mostly test Modern Standard Arabic, so test the candidates on your own Arabic, French and mixed-language documents before deciding.
How often should we replace the model once it is in production?
There is no fixed schedule. New open-weight versions appear every few months, but each change needs the full test set re-run, a check of the new licence and a controlled rollout. A practical approach is to review candidates at a fixed interval, for example once or twice a year, and switch only when the gain on your own test set justifies the work.



