Can the agent report its own breaches?

It can help, but it is no control: a hijacked or misconfigured agent may not recognise or report what it did. Detection runs outside the agent, at the gateway, the platform and the connected systems.

How long should traces be kept?

Long enough for reviews and reports, and in line with data protection. For high-risk AI systems, deployers keep logs at least six months under Art. 26(6) AI Act; other rules may require more.

Are canary tokens allowed in OT?

Yes, on the IT side of OT — in engineering, historian or MES systems — but never as a signal that a controller or a safety function reads.

All context cards

Context card

Detecting an AI agent that crosses its boundary

How do you notice that an AI agent has done something it wasn’t allowed to do — before a customer, an operator or the month-end report does?

By knowledge.aitechdoc.world

Reviewed 21 October 2026

Review log and changes

The short answer

A boundary breach shows in what the agent did, not in how good its results look. The signals are refused tool calls, unusual volumes, targets or timing, actions that don’t fit the declared task and touched canary tokens — all visible only if every run is traced. In OT and on machines the process itself adds signals, but agent alerts never replace process alarms.

For: Documentation systems developers, technical project managers and operations teams running AI agents

Key points

Good results prove nothing: an agent can reach a good result through a forbidden action.
The agent trace with every tool call is the basis of detection.
Refused calls, volume spikes, new targets and actions outside the declared task are the strongest signals.
Canary tokens give alerts with almost no false alarms.
In OT and on machines, process alarms stay the operators’ main signal; agent alerts are added, never substituted.

The context

Results hide the path

An agent that delivers what was asked looks healthy. Whether it read a forbidden folder, mailed a new recipient or changed a record outside its task shows only in its agent trace — the record of every tool call, parameter, refusal and approval. Detection starts with tracing; without it, a breach is found by its damage.

Signals that matter

Signal	What it can mean	Where it shows
Refused tool calls	The agent tries to leave its allowlist	Gateway, platform log
Volume spikes	Bulk changes, mass messages, fleet-wide commands	Trace, transaction logs, device cloud
New targets	New recipients, systems, device groups or tags	Trace, mail and API logs
Actions outside the task	A hijacked goal or a misread instruction	Trace compared with the declared task
Touched canary tokens	Reaching beyond the task	Decoy records, keys or documents
Slow drift of patterns	A rogue agent or a changed configuration	Baselines, anomaly detection

Per system type

IT systems: Watch transaction counts per agent identity in ERP and CRM, outgoing mail, deletions in repositories and cloud consoles, and status changes in the CCMS.
IoT fleets: Watch the size of command batches, rollout waves and settings outside the product’s usual range; a canary device entry in the registry catches enumeration.
OT: Watch writes at the conduit and changes in the historian and MES; process alarms and alarm management stay the operators’ primary signal.
Machines and robots: Watch programme and parameter changes and orders to fleet managers; the machine’s own monitoring and safety functions act independently of any agent alert.

Standards and maturity

Agent platforms bring their own traces. OpenTelemetry is developing semantic conventions for generative AI so traces can be read across tools; their status is “Development” as of October 2026. OWASP’s agentic Top 10 recommends behavioural monitoring against rogue agents. NIST SP 800-61 Rev. 3 places detection in the CSF 2.0 function Detect.

The whole picture is on the pathway map on AI agents crossing the line.

Questions readers ask next

Can the agent report its own breaches?: It can help, but it is no control: a hijacked or misconfigured agent may not recognise or report what it did. Detection runs outside the agent, at the gateway, the platform and the connected systems.
How long should traces be kept?: Long enough for reviews and reports, and in line with data protection. For high-risk AI systems, deployers keep logs at least six months under Art. 26(6) AI Act; other rules may require more.
Are canary tokens allowed in OT?: Yes, on the IT side of OT — in engineering, historian or MES systems — but never as a signal that a controller or a safety function reads.

Sources

OWASP Top 10 for Agentic Applications for 2026 — OWASP GenAI Security Project, 9 December 2025
Semantic conventions for generative AI systems — OpenTelemetry
NIST SP 800-61 Rev. 3: Incident Response Recommendations and Considerations for Cybersecurity Risk Management — NIST, 3 April 2025

Review log and changes

Every context card is checked against its sources before it is published, and again whenever it changes; the date under the byline is the last review. Corrections (something was wrong) and additions (something was missing) are logged below with date and time (Berlin time). Typos, formatting and link fixes are not listed.

Reviewed 21 October 2026

No corrections or additions since publication.

Glossary terms on this card

Agent trace
An agent trace is the structured, time-stamped record of one AI agent run: the task and its origin, the model and configuration, each reasoning step the platform exposes, every tool call with its parameters and result, the approvals requested and given, and the final output — linked so that the run can be reconstructed afterwards.
Canary token
A canary token is a decoy — a fake credential, document, record, URL or device entry that no legitimate process ever uses — placed so that any access to it raises an alert, revealing an intruder or an AI agent that is reaching beyond its task.
Anomaly detection
In industrial data analytics, anomaly detection is the identification of data points, patterns or time periods that deviate significantly from expected or learned normal behavior. Methods range from fixed limits and statistical tests to unsupervised machine learning models.
Security monitoring
In cybersecurity, security monitoring is the continuous collection and analysis of logs, network traffic and system states to detect attacks, policy violations and anomalies in time to respond. In OT, it covers industrial networks, controllers, HMIs and remote access paths.
Rogue agent (AI)
A rogue agent is an AI agent that keeps acting outside its intended purpose or authority over time — because it was compromised, misconfigured or has drifted — while still appearing legitimate to the systems it uses.
Agent goal hijack
Agent goal hijack is an attack in which content or instructions an AI agent processes change what the agent is trying to achieve — its objective, its plan or its choice of tasks — so that it pursues the attacker’s goal with its own legitimate tools and permissions.
Tool allowlist (AI agent)
A tool allowlist is the explicit list of tools, functions, MCP servers and parameter ranges an AI agent is permitted to call; every call that is not on the list is refused and logged, instead of being allowed until someone forbids it.
AI interaction log
In AI governance, an AI interaction log is a record of exchanges with an AI system — the prompt or input, the system, model and version identifiers, the returned output, and what a human then did with it — kept so that a later reader can reconstruct how an output came about. It is the evidence layer beneath review and sign-off: without it, a claim that a model version was checked or an output was approved cannot be substantiated. Scope, retention and access have to be defined deliberately, because such logs often contain confidential input, customer data and personal data. In EU law the artifact appears not under this name but as the "logs" that high-risk AI systems must be able to record automatically, and that providers and deployers must keep.
OpenTelemetry
In software operations, OpenTelemetry (OTel) is a vendor-neutral open-source observability framework of the Cloud Native Computing Foundation (CNCF) that defines APIs, SDKs, semantic conventions, a data protocol (OTLP) and a Collector for generating, processing and exporting telemetry such as traces, metrics and logs.