Table of Contents
Choose an industrial AI vendor by putting the same 15 written questions to every candidate and scoring the evidence, not the promises. The questions cover where your data is processed, who owns the model and its outputs, integration with your systems, proof on your own data, security, and how you leave. Documents and contract clauses outweigh demos.
Key takeaways
- Ask every vendor the same questions in writing and score what they can prove, not what they say in a demo.
- Data location, training on your data, and offline operation are the first filters for sensitive industrial data.
- Agree the pilot baseline, metric and pass mark before the pilot starts.
- Certificates such as ISO/IEC 27001 or ISO/IEC 42001 only help if their scope covers what you are buying.
- Negotiate the exit before you sign.
Why does an industrial AI purchase need its own questions?
Industrial AI sits close to systems that control physical processes. The joint guidance Principles for the Secure Integration of Artificial Intelligence in Operational Technology, published on 3 December 2025 by CISA and Australia's ACSC with agencies including Germany's BSI and the UK's NCSC, puts it plainly: "Ultimately, humans are responsible for functional safety." A standard IT checklist does not cover that.
The 15 questions below fall into six themes. Send them in writing, ask for evidence, and score the answers with the table further down. They apply equally to a global platform, a regional integrator or a local start-up, including us. If you have not yet assessed your own data, start with our AI readiness assessment guide.
What should you ask about your data and where it is processed?
1. Where exactly will our data be stored and processed?
Ask where documents, sensor data, prompts, outputs, logs and backups sit, and from where support staff connect. The CISA-led guidance advises operators to understand how AI vendors access and use OT data, "especially if it involves remote, cloud, or offshore access", and warns that businesses based in foreign countries "are subject to foreign government control and laws."
- Good answer: a diagram naming the location of each data type, written into the contract.
- Weak answer: "your data is secure in the cloud", with no country, provider or log policy.
2. Will you use our data to train or improve models for other customers?
The same guidance notes that operational data can contain intellectual property and recommends a data usage policy covering residency, communication paths, encryption and storage.
- Good answer: no training on your data by default, with any exception opt-in and in writing.
- Weak answer: a general clause allowing customer content to be used for "service improvement".
3. Can the system run on our premises or without a permanent internet connection?
CISA and its partners recommend asking whether vendors can operate on premises or without continuous access to the internet or the vendor cloud. A cloud API can suit non-sensitive uses; drilling data or formulations may need to stay on site. Our comparison of on-premise LLMs and cloud APIs covers the cost side.
- Good answer: a documented deployment option, its hardware requirements, and the features lost when offline.
- Weak answer: "on-premise is on the roadmap."
Who owns the model and what you build with it?
4. Which model is underneath, and under which licence?
Many AI products are built on third-party or open-weight models with their own licences. The Llama 3.1 Community License, for example, requires companies whose products had more than 700 million monthly active users to request a separate licence from Meta. Such terms rarely affect an industrial buyer, but you should know what you depend on.
- Good answer: the base model, version and licence, and how upgrades are handled.
- Weak answer: "it is our proprietary AI", with no detail even under NDA.
5. Who owns the fine-tuned models, configurations and outputs built with our data?
A model adapted on your maintenance history or formulations carries your know-how.
- Good answer: a clause giving you ownership or exclusive use of fine-tuned weights, configurations and outputs, barring reuse for other clients.
- Weak answer: silence, or a clause assigning "all improvements" to the vendor.
6. What documentation comes with the model?
Model cards, proposed by Mitchell and colleagues, are "short documents accompanying trained machine learning models" that state the intended use and how the model was evaluated. Ask for the equivalent, whatever the vendor calls it.
- Good answer: intended use, known limits, evaluation data and results.
- Weak answer: a marketing brochure.
How will it fit into your existing systems?
7. What will you connect to, how, and who does the work?
List your ERP, historian, SCADA and document systems and ask for a connection plan. The CISA-led guidance encourages push-based architectures, where data is pushed out of the OT network without persistent access into it, with new connection paths going through the existing DMZ or jump host. See our guide to integrating AI with SCADA and OT systems.
- Good answer: named interfaces, read-only by default, with the integration effort estimated in person-days.
- Weak answer: "we integrate with everything", or permanent inbound access to the control network.
8. Does it work with our languages and documents?
Plant documentation in Algeria often mixes Arabic, French and English, and some of it is scanned. Ask the vendor to process a sample of your own documents.
- Good answer: results on your sample, including the failures, with an explanation.
- Weak answer: a demo on clean English documents only.
9. What happens when the AI is wrong or unavailable?
The guidance recommends failsafe mechanisms that revert to traditional automation or manual operation, limits on AI control of OT without a human in the loop, and operator control over enabling or disabling AI features.
- Good answer: a described fallback, human approval for any action, and a switch your team controls.
- Weak answer: "the model is accurate enough that this is not needed."
How can a vendor prove results?
10. Will you run a pilot on our data with success criteria agreed in advance?
The NIST AI Risk Management Framework, released in January 2023 for voluntary use, organises AI risk work into four functions: Govern, Map, Measure and Manage. Apply Measure to the pilot: fix the baseline, metric, test data and pass mark before it starts, and hold back data the vendor has never seen. Our guide to building an industrial AI business case turns pilot metrics into value.
- Good answer: a written pilot protocol with baseline, metric, holdout data, duration and cost.
- Weak answer: success defined after the pilot, or "accuracy" with no definition.
11. Can we speak to a customer using it in production?
- Good answer: a production reference in a comparable industry that you can call without the vendor present.
- Weak answer: logos without contacts, or references from pilots only.
How do you check security claims?
12. Which standards do you follow, and what evidence can you show?
Three families matter. ISO/IEC 27001:2022 specifies requirements for an information security management system. ISO/IEC 42001:2023 does the same for an AI management system, for any organisation that provides or uses AI-based products or services. The ISA/IEC 62443 series covers industrial automation and control systems, including 62443-4-1 on the secure product development lifecycle and 62443-2-4 on security programmes for service providers. A young vendor may not be certified; that is not disqualifying if it can show the practices.
- Good answer: certificates with their scope (entity, sites, services), or a dated gap analysis against the standard.
- Weak answer: a logo on a slide, or a certificate covering a different entity.
13. How do you test against AI-specific attacks, and will you provide an SBOM?
The OWASP Top 10 for LLM Applications 2025 lists risks including prompt injection, data and model poisoning, and excessive agency. The CISA-led guidance recommends requesting a software bill of materials (SBOM), clarity on where the AI models are hosted, and disclosure when a vendor finds its AI can give improper advice.
- Good answer: a test report on the relevant OWASP categories, an SBOM, and a vulnerability disclosure process.
- Weak answer: "our model cannot be hacked."
What about support, long-term cost and leaving?
14. Who supports us, where, in which language, and at what cost after year one?
- Good answer: response times, on-site capability, support language, and a three-to-five-year price schedule covering licences, hardware and model updates.
- Weak answer: a first-year price only.
15. If we leave, what do we take with us?
- Good answer: export of your data, fine-tuned weights, configurations and documentation in open formats at a stated cost, plus a plan if the vendor stops trading, such as source code escrow.
- Weak answer: proprietary formats, no export clause, or "no customer has ever left us."
How do you score the answers?
Score each answer from 0 to 3. 0: no answer. 1: verbal claim. 2: written answer or document. 3: verified by a contract clause, a certificate with matching scope, or a result reproduced on your data. Weighted score = weight × score ÷ 3; the maximum total is 100.
| # | Question | Weight | Score (0–3) | Weighted | Evidence seen |
|---|---|---|---|---|---|
| 1 | Data storage and processing location | 8 | |||
| 2 | No training on our data | 6 | |||
| 3 | On-premise or offline operation | 6 | |||
| 4 | Base model and licence | 5 | |||
| 5 | Ownership of fine-tuned models and outputs | 6 | |||
| 6 | Model documentation | 4 | |||
| 7 | Integration plan and effort | 8 | |||
| 8 | Languages and our documents | 6 | |||
| 9 | Fallback and human control | 6 | |||
| 10 | Pilot with agreed criteria | 12 | |||
| 11 | Production references | 8 | |||
| 12 | Standards and evidence | 8 | |||
| 13 | AI-specific testing and SBOM | 7 | |||
| 14 | Support and multi-year cost | 5 | |||
| 15 | Exit terms | 5 | |||
| Total | 100 |
Adjust the weights to the use case: a document assistant may weight question 8 more, an OT project questions 7 and 9. For sensitive data or OT-connected systems, treat questions 1, 2 and 9 as pass or fail: a score of 0 or 1 on any of them should stop the evaluation, whatever the total.
What are the red flags?
- The answer on data location changes between the sales presentation and the draft contract.
- The default terms allow the vendor to train on your data.
- Accuracy figures come with no test set, no baseline and no definition.
- Pilot success criteria are proposed only after the results are in.
- The system needs persistent inbound access to your control network.
- Certification is claimed, but no certificate or scope is provided.
- Exporting your own data is missing from the contract or priced as a new project.
Does Algerian law change the evaluation?
Two texts are worth raising with your legal team early. This is not legal advice; verify the current texts with counsel.
- Personal data. Law 18-07 of 10 June 2018, amended by Law 25-11 of 24 July 2025, governs personal data. According to an analysis by the law firm Gide, transfers of personal data abroad are subject in particular to prior authorisation from the ANPDP, the national authority, save for exceptions set out in the law. An AI service that sends shift logs or HR files to servers abroad may fall under this.
- Public procurement. Law 23-12 of 5 August 2023 sets the general rules. Under Article 13, public economic enterprises are not bound by its award procedures but must adopt their own, based on its principles. Article 41 allows direct negotiated procedure for labelled start-ups in digital and innovation services when the solution is unique and innovative. Article 62 grants a preference margin to products of Algerian origin and to companies under Algerian law majority-owned by resident nationals.
These rules shape how you can buy. They do not replace the technical evaluation.
What does this mean if the vendor is KABAS AI?
We sell DALILE and Virtual Lab, so we have a stake in this question. Apply the same 15 questions to us. The ENAD Shymeca case study reports a 40% reduction in physical testing cycles, and the ADFC case study describes a sovereign deployment within Sonatrach's IT environment. Ask us, as you would any vendor, how that 40% was measured.
For help turning these questions into a tender specification, our consulting team does this work; an independent adviser is also a reasonable choice.
Sources
- CISA, ASD's ACSC et al., Principles for the Secure Integration of Artificial Intelligence in Operational Technology, 3 December 2025
- NIST, AI Risk Management Framework
- NIST AI RMF 1.0, Section 5: AI RMF Core
- ISO/IEC 42001:2023, Artificial intelligence management system (SIS catalogue entry)
- ISO/IEC 27001:2022, Information security management systems (SIS catalogue entry)
- ISA, ISA/IEC 62443 series of standards
- OWASP, Top 10 for LLM Applications 2025
- Mitchell et al., Model Cards for Model Reporting
- Meta, Llama 3.1 Community License Agreement
- Gide, Protection of personal data: focus on Algerian regulations
- Law 23-12 of 5 August 2023 on public procurement, Journal officiel no. 51 (FAOLEX copy)
Frequently asked questions
What is the most important question to ask an AI vendor?
Start with where your data will be stored and processed, including logs, backups and remote support access, and whether the vendor may use it to train models for others. If the answers are vague or cannot be written into the contract, the rest of the evaluation matters less. For data tied to production or safety, also ask whether the system can run without a permanent internet connection.
Should an industrial company choose a local or a foreign AI vendor?
Neither is automatically better. A foreign vendor may bring a mature product but is subject to its home country's laws, which joint guidance from CISA and partner agencies flags as a data sovereignty risk. A local vendor may offer on-site support and local data hosting but may have a shorter track record. Ask both the same questions and score the evidence.
Does an AI vendor need ISO/IEC 42001 certification?
ISO/IEC 42001 certification is a useful signal that a vendor runs a structured AI management system, but it is not a guarantee about any specific product. Many capable vendors are not yet certified. What matters is the scope of any certificate and whether the vendor can show the underlying practices, such as risk assessment, testing and change control.
How should we run an AI pilot with a vendor?
Agree in writing, before the pilot starts, on the baseline, the metric, the test data, the pass mark, the duration and the cost. Keep part of your data back so the vendor has never seen it, and evaluate on that. The pilot should run long enough to cover the operating conditions that matter to you, such as a full production campaign or several maintenance cycles.
Can we use a cloud AI service for industrial data?
Sometimes. A cloud service can be a sensible choice for non-sensitive tasks such as searching public standards. For formulations, process data or anything connected to operational technology, the CISA-led guidance advises minimising the sensitive data shared with models hosted by external parties. In Algeria, transfers of personal data abroad are regulated by Law 18-07, so check with counsel before sending documents that identify people.



