Table of Contents
OPC UA is the platform-independent standard, published as IEC 62541, that lets software read live and historical data from control systems. For a data scientist, think of it as a typed graph of nodes, not a tag list: every value arrives with a quality status and two timestamps, and you either read it on demand or subscribe to changes.
Key takeaways
- OPC UA is the successor to OPC Classic, released in 2008, and identifies every client and server with X.509 certificates (OPC Foundation).
- Data lives in an address space of nodes linked by references. A node's ID contains a namespace index that may change between sessions, so store the namespace URI (OPC 10000-5).
- Each value comes as a DataValue: the value, a status code, a source timestamp and a server timestamp (OPC 10000-4). Keep all four.
- Subscriptions report changes, not a regular time series. Sampling is best effort, and deadbands drop small changes.
- Insecure configurations were found on 92% of OPC UA deployments reachable from the internet (Dahlmanns et al., 2020). Connect only through a path the OT team approves.
What is OPC UA, and why should a data scientist care?
OPC UA (Unified Architecture) is how most modern control systems share data with other software. The OPC Foundation released it in 2008 to bring the functions of the older OPC Classic specifications into one framework that runs on Windows, Linux and embedded devices (OPC Foundation). It is also published by the IEC as the IEC 62541 series (IEC).
In a plant, OPC UA servers sit in SCADA and DCS systems, PLCs, historians and protocol gateways. Your code is the client. In the Purdue model these servers are at Levels 1 to 3, so your access almost always goes through a DMZ, as explained in the Purdue model and where AI fits.
It is not a REST API that returns a clean table. It returns typed nodes, statuses and change notifications, and those details decide what your features mean.
How is data organised in an OPC UA server?
The server's address space is "the set of Objects and related information that the OPC UA Server makes available to Clients". Objects are represented as nodes, described by attributes and connected by references (OPC 10000-3). It is a graph, not a table.
| OPC UA term | What it is | Closest data science idea |
|---|---|---|
| Address space | The graph of nodes a server exposes | A data catalogue |
| Object | A node that groups others, such as a pump or a unit | An entity |
| Variable | A node that holds a value, such as a flow measurement | One column or time series |
| Reference | A typed link between two nodes | A foreign key or graph edge |
| ObjectType, VariableType | Type definitions that instances follow | A class or schema |
| NodeId | The identifier of a node | A primary key |
| Namespace URI | The naming authority behind a NodeId | A schema name |
A NodeId has three parts: a namespace index, an identifier type and the identifier. The type can be numeric, string, GUID or opaque, and string identifiers are case sensitive (OPC 10000-3, NodeId). The index points into the server's namespace array, where index 0 is the OPC UA namespace itself.
The trap is the index. The server must not change the namespace table while a client has an open session, but clients that cache indexes "should always check when starting a session" that they have not changed (OPC 10000-5). A config file holding ns=2;s=FIC101.PV can break after a server upgrade. Store the namespace URI and resolve the index at each connection.
What comes back when you read a value?
Not just a number. A read returns a DataValue with six fields: the value, a status code, a source timestamp, a server timestamp and two picosecond fields (OPC 10000-4).
- Status code. Good means the result "may be used", Uncertain that it "might not be suitable for some purposes", and Bad that it "cannot be used" (OPC 10000-4, StatusCode). If the status is an error, the value is to be ignored.
- Source timestamp. The time the data source applied to the value, usually the device or controller.
- Server timestamp. The time the server received the value or knew it to be accurate.
For features, use the source timestamp and keep the server timestamp for diagnosing delays. Keep the status column through the whole pipeline and treat Bad as missing, never as zero. Our guide to industrial data quality problems covers what else hides behind a Good status.
Read the metadata once per tag. Analog values are often of AnalogItemType, which must carry an EURange, "the value range likely to be obtained in normal operation", and may carry EngineeringUnits and an InstrumentRange (OPC 10000-8). That gives you units and plausible ranges for validation without asking for a spreadsheet.
Polling or subscribing: how do you get a stream of values?
You can call Read in a loop, but OPC UA is designed around subscriptions: you create one with a publishing interval and add a monitored item per variable. The server samples each item and sends the changes at each publishing cycle.
Three settings change the data you receive:
- Sampling interval. It is "the fastest rate at which the Server should sample its underlying source for data changes", on a best-effort basis. The server may assign a different interval than you asked for and returns the revised value; 0 means the fastest practical rate (OPC 10000-4). Log the revised value.
- Trigger and deadband. By default a notification is sent when the status or the value changes. An absolute deadband suppresses changes smaller than a fixed amount (OPC 10000-4, DataChangeFilter). A percent deadband uses a percentage of the EURange (OPC 10000-8).
- Change-based delivery. A steady signal produces few notifications and a noisy one many. Like compressed historian data, the stream is irregular and needs care when resampled.
| Mechanism | How it works | Typical use | Watch out for |
|---|---|---|---|
| Read | Request and response for the current value | Snapshots, metadata, tests | You set the timing; frequent polling loads the server |
| Subscription | Server samples monitored items and sends changes each publishing cycle | Live features for inference, dashboards | Revised sampling intervals, deadbands, irregular timestamps |
| HistoryRead | Query the server's stored history | Training sets, backfills | Only where the server keeps history |
| PubSub | Publishers send messages over UDP or through a broker such as MQTT | Many consumers, one-way flows | Different configuration and encodings from client-server |
What about historical data?
OPC UA defines HistoryRead with several modes: raw or modified values over a time range, processed aggregates per interval, values at specific timestamps, events, and annotations (OPC 10000-11). Whether a server supports them depends on the product.
Which mode and which aggregate to use for training data is covered in extracting process historian data for machine learning. The short version: pull raw values with status and do the resampling yourself.
What is OPC UA PubSub, and how does it relate to MQTT?
Client-server needs a session between your code and the server. PubSub removes it: "Publishers send messages to a Message Oriented Middleware, without knowledge of what, if any, Subscribers there may be" (OPC 10000-14). The broker-less variant uses datagram protocols such as UDP; the broker-based variant uses messaging protocols such as MQTT, for example with JSON encoding.
This is where OPC UA meets the unified namespace pattern, explained in unified namespace and MQTT. The decoupling also suits one-way architectures, covered in data diodes and one-way transfer.
What are companion specifications?
OPC UA lets industries agree on types. The OPC Foundation lists more than 150 companion specifications (OPC Foundation). In oil and gas, MDIS defines the information model for the interface between a subsea production control system and the topside DCS (OPC 30020), with object types for items such as valves and chokes.
For a data scientist, a companion specification is a shared schema: if every valve follows the same type, one feature pipeline can serve them all. Ask whether your equipment implements one before mapping tags by hand.
What does a first connection look like in Python?
The open-source asyncua library provides an asyncio-based OPC UA client and server for Python, under the LGPL-3.0 licence. Its maintainers state that it "comes with absolutely no warranty" (opcua-asyncio), so test it against your server before relying on it. A minimal read of one value, against a test server:
import asyncio
from asyncua import Client
URL = "opc.tcp://test-gateway.example:4840"
NS_URI = "urn:example:plant" # ask the OT team for the real URI
async def main():
async with Client(url=URL) as client:
idx = await client.get_namespace_index(NS_URI)
node = client.get_node(f"ns={idx};s=FIC101.PV")
dv = await node.read_data_value(raise_on_bad_status=False)
print(dv.Value.Value, dv.StatusCode, dv.SourceTimestamp, dv.ServerTimestamp)
asyncio.run(main())Two details matter. The index is resolved from the URI at each connection. And read_data_value raises an exception on a Bad status by default; turning that off lets you record the status instead of losing the row. Against a real server, call set_security_string before connecting to use signing and encryption.
How do you connect without creating a security problem?
OPC UA has strong security, but it must be switched on. The MessageSecurityMode can be None ("No security is applied"), Sign, or SignAndEncrypt (OPC 10000-4). Clients and servers identify each other with X.509 certificates, and users can be required to authenticate (OPC Foundation).
In practice the settings are often weak. Scanning the IPv4 address space, Dahlmanns and colleagues found problems such as missing access control on 24% of hosts, disabled security functionality on 24% and deprecated cryptographic primitives on 25%, in total on 92% of the reachable deployments (Dahlmanns et al., 2020).
The rules for a data science client are simple: SignAndEncrypt, a dedicated read-only account, a secondary server or gateway rather than the primary SCADA server, and a path through the DMZ. Integrating AI with SCADA and OT systems and OT cybersecurity when you add AI explain why.
Checklist before your first OPC UA project
- Agree the access path with the OT team: which server, which endpoint, which network route through the DMZ.
- Exchange certificates and get a read-only account. Use SignAndEncrypt; never fall back to None to save time.
- Get namespace URIs, not indexes, and resolve the index at each connection.
- Browse and export the tag list with data types, engineering units and EURange.
- Choose the mechanism: HistoryRead or the historian for training, subscriptions for live inference.
- Record the sampling interval the server granted and any deadband, and store them with the training data.
- Keep the value, status and both timestamps, in UTC, through the whole pipeline.
- Test against a simulation or test server before touching production.
What we have seen at KABAS AI
Our information systems service integrates existing systems, including SCADA, MES, LIMS and ERP, and our data science service builds predictive models on industrial data. The guidance above comes from that kind of work, not from a published benchmark.
Three points are worth settling before any modelling. First, the tag list and its metadata usually take longer than the code, so start there. Second, a model is only as reproducible as its data path: if training used historian averages and inference uses a deadbanded subscription, the features differ even when the tag names match. Third, give the connection a named owner on the OT side, because certificates expire and servers get upgraded.
Sources
- OPC Foundation, Unified Architecture
- IEC TR 62541-1:2020, OPC Unified Architecture, Part 1: Overview and concepts
- OPC 10000-3, Part 3: Address Space Model, 4 AddressSpace concepts
- OPC 10000-3, Part 3: Address Space Model, 8.2 NodeId
- OPC 10000-5, Part 5: Information Model, 6.3 ObjectTypes for the Server Object
- OPC 10000-4, Part 4: Services, 7.11 DataValue
- OPC 10000-4, Part 4: Services, 7.38.1 StatusCode
- OPC 10000-4, Part 4: Services, 5.13.1.2 Sampling interval
- OPC 10000-4, Part 4: Services, 7.22.2 DataChangeFilter
- OPC 10000-4, Part 4: Services, 7.20 MessageSecurityMode
- OPC 10000-8, Part 8: Data Access, 5.3.2.3 AnalogItemType
- OPC 10000-8, Part 8: Data Access, 7.2 PercentDeadband
- OPC 10000-11, Part 11: Historical Access, 6.5 HistoryReadDetails parameters
- OPC 10000-14, Part 14: PubSub, 4 Overview
- OPC 30020, MDIS OPC UA Companion Specification, Scope
- FreeOpcUa, opcua-asyncio on GitHub
- Dahlmanns et al., Easing the Conscience with OPC UA: An Internet-Wide Study on Insecure Deployments, ACM IMC 2020 (arXiv 2010.13539)
Frequently asked questions
Is OPC UA a database?
No. OPC UA is a communication standard with an information model: it defines how a client finds, reads and subscribes to data that a server exposes. Some servers also keep history and answer history queries, but that depends on the product and its configuration. For long training sets, the plant historian is usually the main source.
Should I train models on OPC UA subscriptions or on historian data?
Use history for training and subscriptions for live inference. A subscription only delivers what arrives while you are connected, filtered by the sampling interval and any deadband. What matters is that both paths compute the same features in the same way, so record the settings used in each.
Why did my NodeIds stop working after the server restarted?
A NodeId contains a namespace index, and a server may change that index between sessions. The namespace URI is the stable part. Store the URI in your configuration, look up its current index each time you connect, and build the NodeIds from it.
Does an OPC UA subscription give me every value the sensor produced?
Not necessarily. The server samples at a best-effort rate that it may revise, it reports only changes, and a deadband suppresses changes smaller than a set amount. The result is irregular, change-based data, so resample it with care before using it as features.
Is it safe to connect a Python script to the plant's OPC UA server?
Only through a path the OT team approves. That normally means a read-only account on a secondary server or gateway outside the control network, the SignAndEncrypt security mode, and certificates exchanged on purpose. An internet-wide study found insecure configurations on 92% of reachable OPC UA deployments, so do not assume the defaults are safe.
What is the difference between OPC UA and MQTT?
MQTT is a lightweight publish-subscribe messaging protocol that carries any payload. OPC UA defines an information model, services and security for industrial data. The two meet in OPC UA PubSub, which can publish OPC UA data through an MQTT broker, for example with JSON encoding.



