The most dangerous enterprise AI agent is not necessarily the one that makes a mistake. It is the one the organization does not know how to stop. In a chatbot, “stop” usually means the text stops streaming. In an agentic system, that definition collapses. The model may stop talking while a queued task keeps running. A server may be disconnected while a cached capability remains visible. A token may be revoked after a downstream action has already occurred. Logs may be lost at exactly the moment the organization needs them most.
That is the Agent Kill Switch Problem. The question is not whether an agent has a stop button. The question is whether the organization can stop the layers where the agent can continue acting: runtime loops, tool calls, MCP servers, credentials, queues, handoffs, downstream systems, and evidence pipelines. A real kill switch is not a button. It is a control chain.
This post proposes a practical framework for that control chain. It defines eight kill-switch layers, identifies common failure scenarios, maps the problem to current agent and MCP architectures, and introduces a three-tier solution model that separates practical controls from advanced platform patterns and future research.
A note on posture before we start: this is a proposed framework and evaluation method, not a completed benchmark. Where I describe patterns like stop epochs or handoff passports, treat them as design directions to validate, not shipped standards. I flag the maturity of each idea explicitly, because the most common failure mode in this space is selling a research-grade concept as a near-term checklist item.
Stopping output vs. stopping action
The framing deliberately separates stopping output from stopping action. In conventional chatbots, stopping output may be adequate because the primary effect is a response. In tool-using agents, the primary risk is the action performed through tools, the authority used to perform it, and the persistence of work beyond the user-visible exchange.
Modern agent systems are increasingly built around managed loops, tool execution, guardrails, handoffs, sessions, and traces. The OpenAI Agents SDK describes an agent loop where tool calls and handoffs can cause the loop to continue rather than terminate. MCP makes the problem more acute because tools may sit behind standardized client/server boundaries, authorization flows, cached tool discovery, and long-running state — and the 2026-07-28 release candidate pushes toward a stateless core, cacheable responses, trace-context propagation, and a formal deprecation policy, all of which change what “stop” has to mean.
Where an agent can keep acting
A useful reference architecture for the kill-switch problem is intentionally simple: User → Agent Host → Agent Runtime → MCP Client → MCP Server → Tool/API → External System. In real deployments this chain may also include identity providers, queues, observability pipelines, policy engines, secret stores, vector databases, SaaS APIs, and human approval systems. Each boundary is a place where an agent can keep acting after a user thinks they hit “stop.”
| Boundary | What can continue after a user-visible stop? | Likely owner | Kill-switch implication |
|---|---|---|---|
| Agent host / app shell | Session state, UI output, user identity, approval screens | Application / product owner | The UI stop must be linked to the runtime/run ID, not just the text stream |
| Agent runtime / run loop | Further model turns, retries, handoffs, tool selection | Agent platform owner or vendor SDK | Runtime cancellation must prevent later turns and mark the run as cancelled |
| Tool selection layer | Selected tool call may be pending or already authorized | Application / security / policy engine | Tool calls must be interruptible before side effects |
| MCP client/server | Cached tool metadata, available server tools, long-running tasks | Integration / platform team | Server disablement must invalidate stale capabilities and fail closed |
| Identity provider / credential broker | Tokens, grants, scopes, service accounts | IAM / security team | Authority must be revocable independently of the process |
| Queue / workflow layer | Retries, scheduled tasks, long-running workflows | Platform / SRE / workflow owner | Stop state must propagate to pending work |
| External business system | Tickets, messages, transactions, deployments, records | Business system owner | Agent-created side effects must be containable and trace-linked |
| Observability / evidence layer | Traces, logs, approvals, hashes, chain-of-custody | Security / IR / compliance | Emergency stop must preserve evidence, not destroy it |
Read as a control chain, the layers look like this:
Eight kill-switch layers
Breaking the chain into named layers gives each one a control and a research question — the question being “does stopping here actually stop anything downstream?”
| # | Kill switch | What it stops | Research question |
|---|---|---|---|
| 1 | Conversation stop | User-facing chat or generation | Did the model stop talking, or did the workflow stop acting? |
| 2 | Agent runtime stop | Planning, tool selection, retries, handoffs, chained loop steps | Can the active agent run be cancelled mid-execution? |
| 3 | Tool-call stop | Specific tool calls before, during, or after execution | Can risk be interrupted at the moment a tool action appears? |
| 4 | MCP server stop | Server access and cached capability discovery | If a server is disabled, does the agent still believe its tools exist? |
| 5 | Credential stop | OAuth grants, access tokens, service accounts, delegated authority | Can authority be revoked even if the process is still alive? |
| 6 | Task/queue stop | Queued, scheduled, retrying, or long-running work | Can work be stopped after it leaves the active runtime? |
| 7 | Downstream stop | External-system side effects and business records | Can the business system contain an agent action after it occurs? |
| 8 | Evidence-preserving stop | Trace / log / approval / credential / task / downstream evidence | Can the agent be stopped without destroying the incident record? |
Briefly, each layer does the following:
- Conversation stop — useful as a UX feature, but weak as a control. It may only stop visible output.
- Agent runtime stop — cancels the active run loop and should prevent more model turns, retries, tool selection, or handoffs.
- Tool-call stop — blocks high-risk tool calls before execution and prevents chained actions after a partial stop.
- MCP server stop — disables a server and invalidates stale tool discovery or cached capabilities.
- Credential stop — revokes agent authority, including tokens, grants, scopes, and service accounts.
- Task/queue stop — cancels pending tasks, retry schedules, long-running work, and resumable jobs.
- Downstream system stop — freezes, reverses, or marks agent-created business-system side effects for review.
- Evidence-preserving stop — creates a security-grade stop record that preserves enough context for incident response and audit review.
How kill switches fail
The layers matter because stopping one is routinely mistaken for stopping the system. These are the failure scenarios the framework is designed to surface.
| Scenario | What goes wrong | Root cause | Relevant controls |
|---|---|---|---|
| Chat stopped, tool continues | User thinks the agent stopped, but an API action completes | UI stop is mistaken for action stop | Tool-call stop; runtime stop; evidence freeze |
| Tool removed, cache persists | Agent still sees stale tool metadata or allowed capability list | Capability cache is not invalidated | MCP server stop; signed snapshots |
| Runtime stopped, queue continues | Background task keeps running after active-run cancellation | Queued work is outside runtime control | Task/queue stop; stop epochs |
| Token revoked too late | Agent completes an external action before revocation propagates | Credential revocation is not atomic | Credential stop; capability leases |
| Handoff occurs before stop | Child agent continues after the parent is cancelled | Stop state does not propagate across agents | Runtime stop; handoff passports |
| Guardrail blocks output only | Unsafe action already happened before the response was blocked | Output guardrail is mistaken for action guardrail | Tool-call stop; approval gates |
| Logs lost during shutdown | Incident cannot be reconstructed | Stop action destroys context | Forensic stop capsule |
| Downstream SaaS remains active | Agent-created record must be manually found and reversed | No linkage between run and external record | Downstream stop; shadow commit |
| Approval replay | Old approval is reused after tool scope changes | Approval is not bound to version/action | Approval binding; capability leases |
| Retry after revoke | Failed action is retried using an alternate still-valid credential | Retry policy ignores emergency stop state | Stop epochs; queue stop; policy gate |
A three-tier solution model
The most important design decision in this framework is to separate controls by maturity and implementation burden. Without that separation, the research sounds unrealistic: a small team cannot implement formal verification of all stop paths, consensus kill, and cross-vendor handoff passports overnight. But the same small team can start with forensic stop records, scoped credentials, approval gates, and circuit breakers. A mature platform team can go further with stop epochs, capability leases, MCP gateway enforcement, and signed capability snapshots.
Tier 1 — Practical Now
Forensic stop record · short-lived scoped credentials · tool allowlists · logging and approvals · circuit breakers
Tier 2 — Buildable by Advanced Teams
Stop epochs · capability leases · MCP gateway enforcement · signed capability snapshots
Tier 3 — Experimental / Future
Handoff passports · shadow commit everywhere · consensus kill · formal stop-path verification
| Tier | Controls | Realism | Best use |
|---|---|---|---|
| Tier 1: Practical Now | Forensic stop record, short-lived scoped credentials, tool allowlists, logging, approval gates, circuit breakers | High — extensions of existing IAM, logging, approval, and SRE practices | Immediate enterprise governance requirements and launch review checklists |
| Tier 2: Buildable by Advanced Teams | Stop epochs, capability leases, MCP gateway enforcement, signed capability snapshots | Medium to high for mature internal platforms; harder for ad hoc SaaS deployments | Advanced agent platforms, MCP gateways, and regulated workflows |
| Tier 3: Experimental / Future | Handoff passports, shadow commit everywhere, consensus kill, formal verification of stop paths | Low to medium today; valuable as research direction and design pressure | High-assurance environments, future standards, and platform/vendor roadmaps |
Tier 1: Practical Now
| Control | What it solves | Main weakness | Implementation path |
|---|---|---|---|
| Forensic stop record | Creates a minimal stop artifact that links run, tool, credential, queue, downstream, and trace evidence | May collect too much sensitive data if not designed carefully | Define a stop-record schema; capture IDs and hashes where possible; avoid storing full sensitive payloads |
| Short-lived scoped credentials | Limits residual authority after a stop and reduces risk from stale tokens | Legacy apps often rely on long-lived service accounts or broad scopes | Use short TTLs, least-privilege scopes, user/run attribution, and revocation procedures |
| Tool allowlists | Prevents agents from discovering or invoking unapproved tools | Can become stale and may block useful changes if too rigid | Maintain an approved tool registry with owner, risk level, schema version, and review date |
| Logging and trace linkage | Connects user action, run ID, tool call, approval, and downstream record | Developer traces may not satisfy audit or incident-response needs | Standardize run IDs, trace IDs, downstream record IDs, and retention classes |
| Approval gates | Blocks high-impact actions until a human or policy service approves them | Approval fatigue; approvals may be too broad or replayable | Bind approval to tool, argument set, user, run, schema version, and time window |
| Circuit breakers | Trips when abnormal tool chains, retries, or handoffs occur | False positives can interrupt useful work | Start with simple thresholds and manual review; tune with operational evidence |
The evidence-preserving layer needs a concrete artifact. A stop event should create a minimal, tamper-evident evidence bundle instead of scattering evidence across logs, queues, and vendor traces. I call this the forensic stop capsule.
Tier 1 is realistic for many organizations because it reuses concepts they already understand: least privilege, approval gates, audit records, logging, incident response, and circuit breakers. The hard part is not inventing new technology. The hard part is linking these controls to agent-specific state — run IDs, tool schemas, prompt context, handoff events, MCP servers, and downstream side effects. OWASP’s AI Agent Security guidance supports the security logic here (least privilege, explicit authorization, human approval, monitoring, audit trails, validation evidence), and the NIST AI RMF and Generative AI Profile support the broader governance logic.
Tier 2: Buildable by Advanced Teams
| Pattern | Core idea | Prerequisites | Main weakness |
|---|---|---|---|
| Stop epochs | Treat stop as a distributed version/state number. Every run, tool call, queue item, handoff, and downstream action checks whether its epoch is still valid | Shared run identity, policy service, and enforcement at tool/gateway boundaries | Breaks if any component ignores the epoch; requires consistent propagation across systems |
| Capability leases | Replace long-lived authority with short-lived, run-bound leases tied to tool, scope, approval, risk level, and expiry | Credential broker, scoped permissions, lease validation in gateway/tool layer | Hard with legacy APIs, broad service accounts, or tools that cannot validate leases |
| MCP gateway enforcement | Place a policy-enforcing gateway between agent hosts and MCP/tool servers | Central registry, identity integration, policy decisions, logging, fail-closed behavior | Gateway can become a bottleneck or single point of failure; needs strong operations |
| Signed capability snapshots | Pin approved tool names, schemas, server identity, risk level, and metadata version. Changed capabilities require re-review | Signing infrastructure, tool registry, cache invalidation, schema diffing | Operational friction; tool updates become more formal and may slow iteration |
The two most useful Tier 2 patterns work together: every action carries a current authorization epoch and a short-lived lease, and a stop event invalidates future checks across tools, queues, and handoffs.
Tier 2 is realistic for advanced internal platforms but probably unrealistic for scattered one-off agent deployments. The reason is simple: stop epochs, leases, gateway enforcement, and signed snapshots only work when the organization controls the places where actions are authorized and executed. If an agent calls a random SaaS connector directly, there may be nowhere to enforce the epoch or validate the lease. MCP makes Tier 2 more plausible because it creates a more standardized tool-access boundary, and the release candidate’s move toward cacheable responses, trace-context propagation, and Tasks makes cache invalidation, task cancellation, and trace continuity first-class concerns.
Deep dive: stop epochs
A stop epoch is a distributed stop-state version. Each agent run begins with an epoch value. Tool calls, leases, queue entries, handoffs, and downstream actions carry that value. When an authorized stop occurs, the policy service increments or invalidates the epoch. Any later component that sees an old epoch refuses to continue.
| Component | Epoch responsibility | Failure if missing |
|---|---|---|
| Agent runtime | Attach current epoch to each planned action and stop when the epoch changes | Runtime continues after external stop |
| MCP gateway | Reject tool calls with a stale epoch | Cached tools or old actions remain callable |
| Queue | Check epoch before executing delayed/retry work | Queued task runs after stop |
| Credential broker | Bind lease/token validity to the current epoch | Credential remains valid after stop |
| Child agent | Inherit parent epoch and stop when the parent epoch is invalidated | Handoff continues after parent cancellation |
| Evidence store | Record epoch at run start, stop time, and failed stale-epoch attempts | Incident cannot prove what was stopped |
Deep dive: capability leases
A capability lease is a short-lived grant that authorizes a specific agent run to use a specific capability under defined conditions. It is narrower than a generic token. A strong lease binds together run ID, user, tool, MCP server, schema version, arguments (or argument class), scope, risk level, approval state, expiry, and stop epoch.
| Lease field | Purpose |
|---|---|
run_id | Prevents reuse by unrelated runs |
tool_name and schema_version | Prevents approval from silently applying to changed tools |
scope and risk_level | Limits what the tool call can do |
approved_arguments or argument_class | Prevents broad approval from becoming replayable authority |
expiry | Limits residual authority |
stop_epoch | Invalidates the lease after a stop |
trace_id | Links use to evidence |
Tier 3: Experimental / Future
| Concept | Why it is interesting | Why it is hard | Possible future use |
|---|---|---|---|
| Handoff passports | A signed, limited-scope packet travels from parent agent to child agent with stop epoch, trace context, max autonomy, expiry, and allowed tools | Requires cross-agent and possibly cross-vendor agreement; easy to bypass if not universally enforced | Multi-agent enterprise workflows and cross-domain delegation |
| Shadow commit everywhere | High-risk actions enter a pending/reversible state before becoming final | Many real systems do not support pending states or clean rollback | Finance, access management, deployments, outbound communications |
| Consensus kill | Multiple independent authorities or monitors agree to trigger emergency stop for high-impact agents | Slow, complex, and overkill for most workflows | Critical infrastructure, defense, high-value financial actions |
| Formal verification of stop paths | Prove that stop-state transitions prevent continuation under defined assumptions | Hard to model LLM-driven behavior and messy integrations; assumptions may be too narrow | High-assurance runtimes, gateways, and regulated environments |
Tier 3 should be included because it gives the framework intellectual ambition, but it should be framed carefully. These concepts are not impossible; they are simply not ready to become ordinary governance checklist items. They are best used as design pressure: if agent ecosystems mature, what properties should future protocols and platforms support? The most promising Tier 3 idea is not consensus kill; it is handoff passports, because they attack a real gap — stop state and authority often do not naturally propagate across delegated work. But handoff passports need shared semantics across agent runtimes, gateways, and tool providers. Until that exists, they should be treated as an experimental pattern.
Cross-vendor kill-switch surface
Different agent stacks expose different parts of the kill-switch problem. The important question is not whether a platform has “a stop button,” but which layer of action it can interrupt, revoke, cancel, or preserve as evidence. This comparison is deliberately conservative: it identifies research questions and control surfaces rather than declaring any vendor safe or unsafe.
| Stack / ecosystem | What it exposes well | Kill-switch relevance | Gap to investigate |
|---|---|---|---|
| OpenAI Agents SDK | Agent runs, tool calls, handoffs, guardrails, sessions, tracing, span-level observability | Useful for runtime stop, tool-call visibility, handoff tracking, and evidence collection | Test whether cancellation, tracing, tool blocking, and downstream containment are governance-grade or developer-debug-only |
| Anthropic Claude tool use | Client tools, server tools, Anthropic-defined tools, and MCP connectors | Useful for analyzing where tool authority actually lives: model, app, provider, or connector | Test stop/revoke differences for client tools, server tools, MCP connectors, and persistent tool access |
| Google ADK / agent ecosystem | Agent-building framework with tools, orchestration, and deployment concepts | Useful for comparing enterprise deployment and lifecycle controls | Identify whether stop semantics cover active runs, long-running tasks, retries, tool calls, and external side effects |
| Microsoft AutoGen | Multi-agent orchestration and termination conditions | Useful for studying conversation termination and workflow boundaries | Termination is not necessarily credential revocation, queue cancellation, or downstream containment |
| LangGraph | Durable execution, interrupts, persistence, and human-in-the-loop workflow controls | Strong candidate for testing workflow stop, pause/resume, approval gates, and recovery | Test whether persisted state and resumed execution create stale-approval or post-stop continuation risks |
| MCP | Standardized client/server tool access, capability discovery, tool schemas, authorization hardening, Tasks, cacheable discovery, trace-context propagation | Central to tool, server, task, authorization, and capability-cache kill switches | Test stale tool lists, disabled servers, task cancellation, token revocation, and trace survival |
| OWASP / NIST guidance | Least privilege, approval, auditability, monitoring, risk management, documented controls | Useful for translating engineering into governance and audit questions | Existing guidance supports controls but does not define an agent-specific maturity model |
Governance and standards alignment
OWASP and NIST support the governance logic behind the kill-switch framework even though neither fully defines an agent kill-switch maturity model. The value of this framework is to turn those governance instincts into agent-specific stop semantics: what exactly gets stopped, who owns the stop path, what evidence survives, and what residual action remains.
| AI RMF function | Kill-switch interpretation |
|---|---|
| Govern | Assign kill-switch owners, define risk acceptance, document stop-path accountability, and require evidence-preserving stop procedures before deployment |
| Map | Map agent actions, tools, MCP servers, identities, data sources, queues, handoffs, and downstream systems where the agent can continue acting |
| Measure | Test stop behavior at each layer, measure residual work after cancellation, and validate evidence completeness |
| Manage | Operate kill switches, review incidents, revoke credentials, update baselines, tune circuit breakers, and improve controls after failed tests |
A local evaluation lab
This framework should be validated, not asserted. The right next step is a harmless, self-contained evaluation lab that simulates an agent runtime, MCP-style tool access, queued work, fake credentials, handoffs, and downstream records — without touching real email, real SaaS systems, secrets, or production APIs.
| Component | Purpose |
|---|---|
| Agent runner | A local script or lightweight framework that simulates an agent loop |
| Mock MCP server | Exposes fake tools and tool metadata |
| Fake tools | read_mock_policy, create_mock_ticket, update_mock_ticket, queue_mock_task, send_mock_email, transfer_to_mock_agent |
| State store | Local JSON files for runs, tool calls, fake credentials, queued tasks, and downstream records |
| Evidence store | Append-only local stop records for audit reconstruction |
| Control plane | Local flags for runtime stop, disabled tools, server disablement, credential revocation, queue cancellation, downstream freeze, and evidence freeze |
The evaluation answers one question: after a stop is triggered, what still continues? Each test records the stop trigger, active layer, residual action, and whether evidence survived. The tiered model should be tested by implementing Tier 1 first, then adding Tier 2 patterns, and finally modeling Tier 3 concepts without implying they are production-ready.
| ID | Test case | Tier focus | Expected evidence |
|---|---|---|---|
| T1 | Stop before first tool call | Tier 1 | No tool call executed; run cancelled; stop record created |
| T2 | Stop after tool selection before execution | Tier 1 | Selected tool recorded; tool denied; no downstream side effect |
| T3 | Stop during write action | Tier 1 / Tier 3 shadow commit | Partial action explicit; record frozen, rolled back, or staged pending review |
| T4 | Stop after write before chained step | Tier 1 | Chained step blocked; downstream record linked to stop record |
| T5 | Stop after handoff | Tier 2 / Tier 3 passport | Parent and child run status captured; handoff state tested |
| T6 | Stop with queued task pending | Tier 2 stop epoch | Queued task cancelled or blocked by stale-epoch check |
| T7 | Stop after fake credential issued | Tier 2 lease | Capability lease revoked or expired; future tool calls fail closed |
| T8 | Stop with stale MCP tool list | Tier 2 signed snapshot | Cached capability invalidated or call blocked |
| T9 | Guardrail-only block | Tier 1 negative test | Demonstrates why output-only control is insufficient |
| T10 | Forensic stop all layers | Tier 1 target + Tier 2 extension | Action stopped, authority revoked, queues cancelled, downstream frozen, evidence preserved |
A kill-switch maturity model
Putting the layers and tiers together yields a maturity model. It ranges from Level 0, uncontrolled, to Level 6, forensic stop — the target state for high-impact enterprise agents.
| Level | Name | Capability | Governance interpretation |
|---|---|---|---|
| 0 | Uncontrolled | No meaningful stop mechanism beyond process termination or user abandonment | Not acceptable for tool-using enterprise agents |
| 1 | Cosmetic Stop | Visible chat/output can be stopped | Useful UX feature, not a governance control |
| 2 | Runtime Stop | Active agent loop can be cancelled | Minimum for agent runtimes, but incomplete if tools, credentials, and queues continue |
| 3 | Tool Stop | Specific tools or tool calls can be blocked | Useful for high-risk action gates and incident response |
| 4 | Authority Stop | Credentials, grants, tokens, scopes, and leases can be revoked | Critical when agent authority can persist outside runtime |
| 5 | Workflow Stop | Queued work, retries, tasks, handoffs, and downstream effects can be cancelled or contained | Required for complex enterprise workflows |
| 6 | Forensic Stop | Action stops and evidence is preserved for audit, investigation, and governance review | Target state for high-trust / high-impact enterprise agents |
The forensic stop capsule, in detail
The evidence-preserving layer needs a concrete artifact. The Emergency Agent Stop Record is the practical implementation of the broader forensic stop capsule. It should capture enough metadata to reconstruct action, authority, continuity, and evidence — without becoming a sensitive-data dumping ground.
| Section | Fields |
|---|---|
| Stop metadata | stop_id, timestamp, stop initiator, stop reason, emergency/non-emergency flag |
| Agent state | agent_id, run_id, session_id, model/provider/version, runtime version, current step |
| Tool state | Selected tool, attempted tool calls, completed tool calls, blocked tool calls, tool versions |
| MCP state | Connected servers, disabled servers, cached tool-list version, discovery timestamp |
| Authority state | Credential type, lease/token/grant ID, scopes, revocation timestamp, revocation result |
| Task state | Queued tasks, retry schedules, long-running tasks, cancellation status |
| Downstream state | External systems touched, record IDs, freeze/rollback status, owner notified |
| Control state | Guardrails triggered, approvals required, approvals granted, approvals invalidated |
| Evidence state | Trace ID, log bundle, retention class, hash/signature, evidence owner |
Putting it into practice
For an enterprise review, each approved agent should satisfy a set of concrete control objectives:
- Every approved agent has a documented stop path that identifies what can be stopped at the conversation, runtime, tool, server, credential, queue, downstream-system, and evidence layers.
- High-risk tool calls can be blocked before execution and cannot be retried through alternate tools without re-approval.
- MCP server disablement invalidates cached capabilities and fails closed when tools are no longer authorized or discoverable.
- Agent credentials are scoped, short-lived, revocable, attributable, and bound to the approved use case and runtime context.
- Long-running tasks, retries, scheduled jobs, and handoff workflows have cancellation semantics linked to the originating run.
- Downstream systems can identify and contain agent-created records, transactions, messages, tickets, or changes.
- Emergency stop events preserve trace, log, approval, credential, tool, task, and downstream evidence.
- Kill-switch tests are performed before launch and after material changes to tools, credentials, MCP servers, autonomy level, or capability leases.
That translates into a practical approval checklist reviewers can actually run:
| Review area | Question |
|---|---|
| Ownership | Who owns the kill switch? Who can trigger it? Who reviews the evidence afterward? |
| Scope | What exactly does the stop path stop: conversation, runtime, tools, server, credential, queue, downstream action, or all of them? |
| Runtime | Can the active run be cancelled mid-loop, after a tool call, and after a handoff? |
| Tools | Can high-risk tool calls be blocked before execution? Are failed or blocked tool calls visible to the model and operator? |
| MCP | Can MCP servers be disabled? Are cached tool lists invalidated? Does the client fail closed? |
| Identity | Can delegated authority, tokens, service accounts, leases, and refresh tokens be revoked quickly and visibly? |
| Tasks | Can queued, retrying, scheduled, and long-running work be cancelled? |
| Downstream | Can external records or actions be frozen, rolled back, or linked to the originating run? |
| Evidence | Are traces, logs, approvals, tool metadata, credentials, stop epochs, and stop events preserved without storing unnecessary sensitive data? |
| Testing | Has the stop path been tested before launch and after material changes? |
A realistic organization should not start with formal verification or consensus kill. It should start with Tier 1 controls, then select a small number of Tier 2 patterns if the agent platform justifies the added complexity:
- Start with inventory — identify agent runs, tools, MCP servers, credentials, queues, handoffs, downstream systems, and evidence sources.
- Implement the forensic stop record — define the minimum fields needed for incident response and audit reconstruction.
- Reduce authority — use short-lived scoped credentials and bind approvals to tool, argument, run, schema version, and time window.
- Add tool and workflow gates — use allowlists, approval gates, circuit breakers, and high-risk tool deny rules.
- Build the local lab — validate whether stop actions prevent residual tool calls, queued work, stale credentials, and lost evidence.
- For advanced teams, add stop epochs and capability leases — enforce them at the MCP gateway, queue, and credential-broker boundaries.
- Use signed capability snapshots — for MCP servers and tool schemas that can materially affect risk.
- Treat handoff passports, shadow commit, consensus kill, and formal verification as research/future design goals — unless the workflow is genuinely high-assurance.
Kill-switch ownership
A stop path with no owner is not a control. Each layer needs a named owner, a trigger authority, and defined verification evidence.
| Layer | Likely owner | Trigger authority | Verification evidence |
|---|---|---|---|
| Conversation stop | Application / product owner | User or support operator | Session status, UI stop event, timestamp |
| Runtime stop | Agent platform owner | App operator, platform operator, incident commander | Run state cancelled, no further turns, trace closed |
| Tool-call stop | Application / security owner | Policy engine, approval service, operator | Denied tool call, blocked arguments, policy reason |
| MCP server stop | Integration / platform owner | Platform operator, security, MCP registry owner | Server disabled, discovery cache invalidated, calls fail closed |
| Credential stop | Identity / security owner | IAM admin, incident commander, automated policy | Token/grant/lease/refresh revoked, access denied |
| Task/queue stop | Workflow / platform owner | Workflow operator, incident commander | Queue item cancelled, retry disabled, task state stopped |
| Downstream stop | Business system owner | System owner, incident commander, workflow owner | Record frozen, transaction cancelled, rollback ID, owner sign-off |
| Evidence freeze | Security / IR / compliance owner | Incident commander, security operator | Trace bundle, log hash, retention class, evidence owner |
Limitations and careful wording
This is a proposed framework, not an industry standard. The local lab is planned, not completed. Vendor documentation shows exposed concepts and control surfaces, not complete security guarantees. Experimental concepts such as handoff passports, consensus kill, and formal verification are future-facing design ideas, not settled consensus. A few phrasings I hold myself to:
- Not “no agents are stoppable,” but “many systems may stop the visible interaction without stopping every layer where work can continue.”
- Not “Vendor X fails,” but “Vendor X exposes these control surfaces; the open question is whether they satisfy forensic stop requirements in a given deployment.”
- Not “this proves,” but “this proposed lab is designed to test.”
- Not “every org should implement consensus kill,” but “consensus kill is a high-assurance future pattern, not a normal enterprise baseline.”
The new risk is not that agents cannot be stopped. The risk is that organizations may not know which layer they stopped, which layers kept acting, and what evidence was lost in the process.
The next step for this work is to build the harmless local lab described above — fake tools, fake credentials, fake queues, fake handoffs, and local JSON evidence records — run the ten test cases against Tier 1 controls first, then add stop epochs and capability leases for Tier 2, and publish a follow-up with the repo, test logs, and a revised maturity model measured against a real agent stack. If you’re working on agent governance and want to compare notes, get in touch.
Sources
- OpenAI Agents SDK — Running agents: https://openai.github.io/openai-agents-python/running_agents/
- OpenAI Agents SDK — Tracing: https://openai.github.io/openai-agents-python/tracing/
- OpenAI Agents SDK — Handoffs: https://openai.github.io/openai-agents-python/handoffs/
- Model Context Protocol — Security Best Practices: https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices
- MCP Blog — The 2026-07-28 MCP Specification Release Candidate: https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/
- OWASP Cheat Sheet Series — AI Agent Security: https://cheatsheetseries.owasp.org/cheatsheets/AI_Agent_Security_Cheat_Sheet.html
- Anthropic Claude Platform Docs — Tool use with Claude: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview
- LangGraph Docs — Interrupts: https://docs.langchain.com/oss/python/langgraph/interrupts
- NIST — AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
- NIST AI 600-1 — Generative AI Profile: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
- Microsoft AutoGen — Termination: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/termination.html
- Google Agent Development Kit: https://adk.dev/