Table of Contents
It depends on the product and the contract, not the vendor's name. Major providers say they do not train on API or business-tier data by default, while free and consumer tiers may. But "no training" is only one question: ask in writing about retention, human review, logs, embeddings, feedback and where data is processed.
Key takeaways
- The same vendor can apply different rules to different tiers. Google uses content sent to its unpaid Gemini API services to improve its products, but not prompts sent to its paid services (Google).
- Terms change. In 2025 Anthropic asked users of its consumer plans to choose whether their chats may train its models, with retention extended to five years for those who agree (Anthropic).
- Models can memorise training data, and logs, abuse-monitoring stores and embeddings hold copies of your content even when no training happens.
- On-premise deployment removes most of these questions, but check what the software still sends out.
- Write the answers into the contract. If personal data is involved, check Algeria's Law 18-07 with counsel.
Why does it matter whether a vendor trains on your data?
Because a model can repeat what it was trained on. Carlini and colleagues extracted verbatim training text from GPT-2, including names, phone numbers, email addresses and code, "even though each of the above sequences are included in just one document in the training data" (Carlini et al.). A later study extracted gigabytes of training data from open models and showed that ChatGPT, under a "divergence attack", emitted training data at a rate 150 times higher than in normal use (Nasr et al.).
Joint guidance from CISA and partner agencies, published in December 2025, applies this to plants. If process measurements are used to train or update a model, "the data may become accessible or stored (statistically) for a longer period of time in the model." The same guidance notes that operators "may not want vendors to train AI on operational data as it could include IP or be sensitive", and recommends a data usage policy covering data residency, communication paths, encryption and storage (CISA et al.).
For an industrial company, the content at stake is formulations, process recipes, equipment drawings and failure histories. Deleting it later does not undo training that has already happened: Anthropic's notice, for example, says deleted conversations will not be used for future training. OWASP lists "data leak via training data" among its scenarios for sensitive information disclosure, and recommends letting users opt out of training and keeping clear policies on retention, usage and deletion (OWASP LLM02:2025).
What do the major providers' terms say?
The table summarises published terms, checked in November 2026. They change, so read the current version for the exact product you use.
| Service | Used to train or improve models by default? | What is kept |
|---|---|---|
| OpenAI API | No, unless you opt in (OpenAI) | Abuse monitoring logs for up to 30 days by default; files, vector stores and some other features keep data until you delete it; zero data retention needs OpenAI's approval |
| Anthropic Claude Free, Pro and Max | The user chooses (Anthropic) | Five years if the user allows training, otherwise 30 days |
| Anthropic commercial services, including the API | No: "Anthropic may not train models on Customer Content from Services" (Anthropic) | Not stated in the clause quoted |
| Google Gemini API, unpaid | Yes, and "human reviewers may read, annotate, and process" inputs and outputs (Google) | Google asks users not to submit sensitive, confidential or personal information |
| Google Gemini API, paid | No (Google) | Prompts and responses logged for a limited period to detect policy violations |
| Microsoft Foundry models sold by Azure, including Azure OpenAI | No, and not available to OpenAI (Microsoft) | Flagged prompts and outputs may be stored for human review, unless modified abuse monitoring is approved |
| Slack | Customer data is analysed to build non-generative models, such as emoji and channel recommendations; no generative AI training without opt-in (Slack) | Opting out of global models takes an email from the workspace owner |
Two patterns stand out. Free and consumer tiers are where training is allowed, or left to the user. And the business tiers that exclude training still keep some data for a period, for abuse monitoring or for features you switch on.
Is "we don't train on your data" enough?
No. It closes one route. Four others stay open.
Retention and human review. Data kept for abuse monitoring can be read by people. Microsoft describes authorised employees reviewing flagged prompts and completions; customers approved for modified abuse monitoring avoid this storage (Microsoft).
Legal orders. On 13 May 2025, in the New York Times lawsuit, a US court ordered OpenAI to "preserve and segregate all output log data that would otherwise be deleted", including API outputs normally kept for 30 days. Customers with zero data retention agreements were excluded, and the order was lifted, with exceptions, in October 2025 (Simon Willison). A retention promise holds only as long as the courts in the provider's country allow. Our guide to sovereign, on-premise AI covers the jurisdiction question in more depth.
Derived data. Embeddings, the vectors a RAG system stores for search, are not anonymous. Researchers recovered 92% of 32-token texts exactly from their embeddings, and full names from a dataset of clinical notes (Morris et al.). A vendor-hosted vector index is a copy of your documents.
Location of processing. Storage location and processing location can differ. On Microsoft's "Global" deployments, prompts and responses may be processed in any geography where the model is deployed, while stored data stays in the chosen geography (Microsoft).
Also watch the wording. "Not used to train generative models" is narrower than "not used to improve any product", as Slack's distinction shows.
What should you ask any AI vendor?
Send these in writing and ask for the contract clause or document behind each answer. They go deeper on data use than question 2 of our 15 questions for choosing an industrial AI vendor.
- Which of our data do you receive? Prompts, uploaded files, document passages retrieved for each answer, outputs, feedback ratings, logs, embeddings and telemetry.
- Is any of it used to train, fine-tune, evaluate or improve any model or product, for us or for others? Ask about non-generative models too, and whether the default is opt-in or opt-out.
- Which terms apply to our accounts? Free trials, pilot accounts and employees' personal accounts often fall under different terms from the enterprise contract.
- What is retained, where, for how long, and who can read it? Include abuse monitoring stores, backups and human reviewers.
- Can we have zero retention, and which features do we lose? OpenAI, for example, lists features that keep data until deleted.
- Where is the data processed, not only stored? Ask for the list of subprocessors and their countries.
- What happens if a court or government demands our data? Under which jurisdiction, and will you tell us when the law allows?
- Who owns the fine-tuned models and indexes built from our data? Microsoft, for instance, states that fine-tuned models are exclusively available to the customer whose data was used.
- How will you tell us about changes to these terms, and can we refuse them?
- How can we verify your answers? Settings we can inspect, audit reports, and written confirmation of deletion at the end of the contract.
Does running the model on-premise settle the question?
Mostly. If an open-weight model runs on your own servers, no outside provider sees your prompts or documents. But software can still talk to the outside.
vLLM, an open-source inference server, "collects anonymous usage data by default", such as hardware and model architecture; setting VLLM_NO_USAGE_STATS or DO_NOT_TRACK turns it off (vLLM). Ollama states that it does not see prompts when models run locally, but it also offers cloud models and web search, which can be disabled with a local-only mode, and it downloads models from the internet (Ollama). Neither is a training risk, but both show why "on-premise" needs checking.
For a packaged on-premise product, ask: does it phone home for licence checks, telemetry, crash reports or updates? Does support need remote access? Does the vendor ever take logs or samples back to improve the product? Ask for a list of outbound connections and check it against your firewall logs. Our article on air-gapped AI deployment covers sites where nothing may leave, and securing an on-premise LLM covers the logs you keep yourself.
What should the contract say?
- A definition of customer data that covers inputs, outputs, files, embeddings, logs and feedback.
- A ban on using customer data to train or improve any model or product, with exceptions only by written opt-in.
- Retention periods for each type of data, and deletion with written confirmation.
- The list of subprocessors and processing locations, with notice before changes.
- Notice of legal demands for your data, where the law allows it.
- Ownership, or exclusive use, of fine-tuned models, configurations and indexes.
Algerian law. According to the CMS guide, Law 18-07, as amended by Law 25-11, allows transfers of personal data to a foreign state only with the authorisation of the national authority and if that state ensures a sufficient level of protection, with listed exceptions. The guide also states that a subcontractor must work under a written contract and "acts only under the sole instruction of the controller." Maintenance reports and shift logs often name employees, so ask counsel how a vendor that reuses such data for its own models fits these rules. This is a summary of a law-firm guide, not legal advice; confirm with counsel against the official texts. Our AI project RFP guide shows how to put these terms into a tender.
What we have seen at KABAS AI
We sell an on-premise product, so we have a stake in this question. Ask us the same ten questions.
DALILE runs completely offline on the client's own servers, and its product page states that nothing is sent to external servers, with encryption, access controls and audit logging. That removes the training question for the documents and databases it reads. It does not remove the questions about your own logs, who on our side can access the system during support, and how updates arrive; ask us for those answers in writing. On the R&D side, the published ADFC case study describes a sovereign deployment that keeps full data control within Sonatrach's IT environment.
Two practical steps before any vendor discussion. First, classify your data: what may leave the site, what may not, and what contains personal data. Second, check which AI tools your staff already use with personal or free accounts, since those tiers are where training is most likely to be allowed, as the table above shows. For a comparison of hosting options, see sovereign cloud vs private cloud vs on-premise.
Our consulting team can review a vendor's terms against these questions with you. To see DALILE on your own documents, contact us.
Sources
- OpenAI, Data controls in the OpenAI platform
- Anthropic, Updates to Consumer Terms and Privacy Policy, 28 August 2025
- Anthropic, Commercial Terms of Service
- Google, Gemini API Additional Terms of Service
- Microsoft, Data, privacy, and security for Foundry Models sold by Azure
- Slack, Privacy principles: search, learning and artificial intelligence
- Carlini et al., Extracting Training Data from Large Language Models, arXiv 2012.07805
- Nasr et al., Scalable Extraction of Training Data from (Production) Language Models, arXiv 2311.17035
- Morris et al., Text Embeddings Reveal (Almost) As Much As Text, arXiv 2310.06816
- CISA, ASD's ACSC et al., Principles for the Secure Integration of Artificial Intelligence in Operational Technology, 3 December 2025
- OWASP, LLM02:2025 Sensitive Information Disclosure
- Simon Willison, OpenAI slams court order to save all ChatGPT logs, including deleted chats, 5 June 2025
- vLLM documentation: Usage stats collection
- Ollama FAQ
- CMS Expert Guide to data protection and cyber security laws: Algeria
Frequently asked questions
Do AI providers train their models on data sent through their API?
The large providers generally say no for paid and business use. OpenAI states that API data is not used to train its models unless you opt in, Anthropic's commercial terms say it may not train models on customer content, and Google does not use prompts on paid Gemini API services to improve its products. Free tiers can differ, so check the terms for the exact product and tier you use.
Is it enough if the vendor anonymises our data before training?
Ask what anonymised means and how it was tested. Research has shown that language models can memorise sequences that appear only once in their training data, and that text can be largely recovered from its embeddings. For industrial data, removing names does not protect the know-how itself, such as a formulation or a process window.
If we delete our data, is it removed from the model?
Deletion removes files and conversations from the vendor's storage under its retention rules, but it does not undo training that has already happened. Anthropic's 2025 notice, for example, says deleted conversations will not be used for future model training. That is why the training question has to be settled before any data is sent.
Does an on-premise AI system send any data to the vendor?
It should not send your content, but check everything that leaves the network: telemetry, licence checks, updates and remote support. Some open-source components send anonymous usage statistics by default; vLLM does, and documents how to switch it off. Ask the vendor for a list of every outbound connection and confirm it with your firewall logs.
What does Algerian law say about sending company data to an AI vendor abroad?
Law 18-07 covers personal data. According to the CMS legal guide, transfers of personal data to a foreign state need the authorisation of the national authority, the ANPDP, and an adequate level of protection, with listed exceptions. Non-personal data such as formulations is protected mainly by your contract and confidentiality terms. Confirm your case with legal counsel.



