Context card
Why a copied interface is not a test boundary
What makes a test setup for an AI agent isolated — a copied interface or an enforced boundary?
The short answer
A test boundary for an AI agent is the set of enforced limits on everything a run can reach: tools, network destinations, credentials, file paths and redirects. A copied or simulated web interface changes only what the model sees and is one element of that boundary, not the boundary itself. If the run can still reach the live service when the copy fails, it is not isolated.
For: Engineers and technical writers documenting AI agent tests
Key points
- A test boundary is enforced outside the model, not by wording in the task.
- An instruction such as "fill in but do not send" is an expectation, not a limit.
- If a practice page is unavailable, the tool should fail rather than fall back to the live service.
- Access that looks like reading can still cause effects on the other side.
- What the boundary blocked belongs in the test record.
The context
What the boundary has to cover
An evaluation run gives a model tasks and tools and records what happens. A copied form page changes the screen the model works on. The boundary is something else: the list of destinations the run may reach in the network, the tools it may call, the credentials it holds, the file paths it may write, and the handling of redirects and shortened links, which can move a request to a destination nobody put on the list. Wording in the task is not part of this boundary. "Fill in the form but stop before sending" describes an expectation; without an enforcing layer, nothing prevents the send.
The failure case matters most. If a practice copy is gone or a session has been closed, the tool should return an error. A fallback that searches for an equivalent page is how a test task ends up on a real service.
"Read-only" is not a boundary
Blocking write methods narrows the surface but guarantees nothing. A request that looks like a read can still trigger an effect on a service with a software defect or an unusual design: a counter, a notification mail, a state change behind a link. In a browser tool, the fact that a click was performed says little about what the service did with it. The evidence is the server's response, not the screenshot.
Substitutes and the test record
Useful substitutes are stubbed target services, mail recipients inside the test domain, allow-listed write targets and offline copies of websites that a run would otherwise open. Each of them has to carry the same identifiers and error behaviour as the real target, or the test measures a different task.
A test record should show which destinations were requested, which were blocked and which credentials were in force. Such a record describes the behaviour of one configuration under the cases that were tried; it is not evidence that future runs are contained, which is why the boundary itself stays a subject of security review and change control.
Questions readers ask next
- Can the model be told to stay inside the test environment?
- It can be told, and that may help. But an instruction is interpreted together with the page the model sees, so it cannot be the only thing standing between a test and a live service.
- Does a separate browser profile isolate a run?
- It separates cookies and sessions, which helps. Destinations, outbound connections, mail, payment and file access remain to be limited separately.
- What belongs in the documentation of an agent test?
- The task, the tools and versions, the destinations allowed, the credentials used, the blocked attempts and the responses received. Without the responses, the record shows intentions rather than effects.
Sources
- Agentic AI – Threats and Mitigations — OWASP GenAI Security Project, 13 October 2026
- Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 — NIST, 26 January 2023
Review log and changes
Every context card is checked against its sources before it is published, and again whenever it changes; the date under the byline is the last review. Corrections (something was wrong) and additions (something was missing) are logged below with date and time (Berlin time). Typos, formatting and link fixes are not listed.
Reviewed
No corrections or additions since publication.