65 questions to ask before an AI agent goes to production

The questions a mature enterprise asks before an AI agent touches real systems.

• For the enterprise: what to ask a vendor, or its own team.
• For a founder selling AI into an enterprise: what to answer to pass the security review.

The answers assume Swiftward is the control layer in use.

Part 1. What you control

1. Boundary

1.1 How is the AI controlled, where does that control sit, and who owns it?

A control inside the product’s own code is written and shipped by the same engineers it is meant to check. Separation of duties puts the control outside the product: a different team, a different release.

The control is a control layer, and it runs as its own deployment, outside the product. The product reaches it over the network, or routes through a gateway that reaches it. Rules live outside the product’s code, so a rule changes without a product release and without a product engineer.

Rights are granted per kind of object, so a risk or compliance team can hold authoring and deployment while the product team holds neither.

One control layer serves several products. Rules differ per product; the audit, the counters and the labels are one set across them. Governed apart, reviewed together.

1.2 What does the control layer do at runtime, and where does it stop?

A control layer covers some things and leaves others outside it. Where it stops has to be stated, because anything outside that line needs a different control.

It takes an event from the application, checks it against rules in a declarative engine, and returns a decision: allowed, denied, or human review. During the check it reads per-entity state — counters, labels, metadata — and can call helper functions for data the rules need. After the check it updates that state and fires actions. Rulesets are versioned, backtested over past events, and combined into deployments with traffic split and shadow execution, swappable instantly.

It does not guarantee the rules are right. That belongs to whoever writes and tests them.

1.3 What is covered out of the box, what by configuration, what needs custom work, and what is out of scope?

One capability list hides four different things: what ships working, what is a setting, what has to be built for this deployment, and what the product never does. Anything in the custom-work column is a project with a budget, an owner and a date.

2. Where the check sits and what it costs

2.1 Are synchronous policy checks supported in the critical runtime path? Where exactly is the decision point?

An asynchronous check is monitoring. Enforcement is synchronous: the application needs the verdict before it proceeds, and every call the agent makes goes through the check.

Yes, three stages in order:

• enrichment — the gateway adds what it knows about the request, and a rule can fetch more before deciding;
• decision — the engine, which evaluates the rules and returns the verdict;
• enforcement — the gateway, and every call to an LLM, an MCP server or the outside network goes through it.

The application calls the engine directly over REST or gRPC, or routes through the gateway and calls nothing itself — the shape that also closes the bypass (18.1).

2.2 What latency does the check add?

Latency depends on what the rule does, so one number answers nothing. A deterministic rule and a rule that calls an LLM are orders of magnitude apart.

Deterministic rules — thresholds, lists, counters, state lookups — run at about 2 ms. A rule that uses an LLM as a judge takes what that LLM takes: tens of seconds. A rule that calls an outside service takes what that service takes.

Only the signals a condition reads are computed, and independent ones run in parallel, so new rules add no latency where they are not used.

To do

• Measure your own ruleset at p50, p95 and p99, under production-like load on production-like hardware.

2.3 What throughput is expected, where are the bottlenecks, and how does it scale?

On the synchronous path, the control layer takes every event of every product that depends on it. A ceiling it reaches is an outage in all of them.

The synchronous path, where the application waits for the verdict, scales horizontally. Up to tens of millions of events a day on ordinary hardware, hundreds of millions on hardware sized for it, and higher where someone needs it.

The bottleneck is the writes: every event leaves a decision record, and a stateful rule changes state as well. A rule that calls an LLM is bounded by that LLM.

It runs on PostgreSQL, which is enough for most deployments. Where the volume needs more, three things move without the rules changing:

• the event stream is partitioned by a key you declare, so events that need no common ordering are processed in parallel;
• the queue moves to Kafka;
• history archives into ClickHouse, so a long horizon stays queryable at analytical speed.

3. Identity and permissions

3.1 Is the agent’s identity separate from the user’s?

Two identities act on every request: the agent, and the person on whose behalf it acts. An identity that the calling application fills in is self-asserted, so each record has to say which values were verified and which were merely claimed.

Yes, and both travel with the request: which agent acted, and on whose behalf. So a prompt can be attributed to a person or an organization without reading the prompt. Permission is the intersection of the two: the action has to be allowed for the agent and for the person. An agent working for a user can never exceed either. Both are in the record.

Each of the two has its own source, set independently: authentication, or a header. The source is configured; the trust label follows from it. A credential the control layer resolved gives verified; a bare identifier in a header gives declared. Every record says which of the two it holds, and no setting turns one into the other.

Declared identity is a way to start controlling something quickly. Where grants, counters and spend ceilings depend on who is calling, the endpoint runs in authenticated mode and the caller arrives as a user with its own permissions, limits and audit trail. One configuration value.

What that distinction carries:

• the header is set by whatever runs the agent, so an instruction inside the data cannot change it;
• a declared identity may only raise the risk level the request requires, so the strictest level wins (7.5).

3.2 Does the agent start with no privileges, and who grants them?

Deny by default is the starting position. Whoever can grant a privilege is inside the audit scope, and the engineer who ships the agent holding that right means no separation at all.

Tools and LLMs are reached through a gateway that asks the engine first. Permissions live in a versioned ruleset outside the application code, so granting one is a policy change.

On the tool path the gateway applies three layers before the engine is asked:

• a deny entry on the endpoint blocks a tool outright;
• an allow entry is necessary and not sufficient — the caller must also hold the right to call that one tool on that one endpoint, and a call with no matching allow entry is refused;
• with guardrails on, the call is then evaluated as an event like any other.

The tool list the agent sees can be filtered by the same grants, so a tool it may not call is not offered to it.

Granting is two acts by two people: the steward who registers an agent and declares what it is accountable for, and the endpoint administrator who decides what that endpoint may reach. Registering an agent opens no access by itself.

3.3 Which tools, which data and which LLM may one agent reach?

Per tool is the minimum. “May use the database tool” and “may run this statement against this table” are different permissions, and only a rule that sees the arguments can tell one from the other.

Every tool call is its own decision, and the rule sees the arguments as well as the name, so a tool can be allowed and this call denied. Which data it may touch and which LLM serves it are conditions in the same rule. Counters per agent and per end customer make “third call this minute” a condition.

Beside the rule, on the same path:

• arguments the agent never sees — the gateway can hide a parameter from the tool catalog and inject the value itself, so a credential or an account identifier is never something the LLM can choose;
• which LLM serves — the rule states the risk level the content requires, the gateway picks from what qualifies, and refuses when nothing does (7.5).

3.4 How fast can the AI be stopped, by whom, and what keeps working?

There must be a way to stop the AI without stopping the product. A switch that waits for a vendor call or a deployment arrives after the incident.

Point the active deployment at another ruleset version. One change, effective on the next request, done by the enterprise’s own administrator because the permission is a role. Nothing restarts, and rollback is the same change back.

It stops at three sizes, so an incident in one place leaves the rest running:

• a tenant — rules are bound to one, so one organization stops while the others run;
• an agent — each carries its own off switch, read while the call is handled, so it takes effect on that agent’s very next call;
• an endpoint — point it at a different ruleset version and only what passes through it changes.

What keeps running: other deployments. The off switch is read on every call, so even an approved sequence stops at its next step.

4. Prompt injection

4.1 What protects the LLM from instructions that arrive inside data?

Any text the LLM reads can carry an instruction: a support ticket, a web page, a tool response, a PDF a customer uploaded. By design an LLM cannot distinguish data from instructions, so the defense is a stack of detectors with a declared response when one fires.

The defense is layered, and every layer is declared and swappable. A typical stack:

• Normalization and decoding — an instruction hidden by look-alike characters, invisible characters, leetspeak, ROT13 or base64 is decoded to plain text before anything judges it, so the detectors see what the LLM will see.
• Fast classifiers — open classifiers for injection and jailbreak run in parallel inside the enterprise’s own perimeter, single-language or multilingual, each with its own threshold.
• A third-party detector — where you already pay for one, it plugs in as one more signal.
• An LLM as judge — on what the fast layers flag, going in or coming back, with the policy given to it as text.
• A canary in the system prompt, scanned on the way out.

A detector firing on the way in says someone is trying. On the way out, a canary token or the judge finding a breach says they succeeded. Different response, both recorded.

The control still sits on the action. Reading a bad instruction is not the damage; sending the file is, and the send is a tool call the same engine decides.

4.2 Is the whole path ready for prompt-engineering testing, and under what scope?

No defense stops every injection, and without one everything gets through. The test runs through the product end to end, because a way around the check can sit in the integration.

Yes, end to end through the product. Agree the scope in writing first: which environment, what data, how much notice.

Every attack that got through becomes an eval case with the verdict it should have produced, and eval cases run before any version is promoted, so a hole closes once. Re-run the same attack against the fix and see it fail. Tightening a threshold is a policy change and needs no release, and a stricter version can run on live traffic before it enforces anything (13.1).

5. Hallucination and role drift

5.1 What happens when the LLM makes something up?

No control prevents an LLM from inventing. The enforceable part is the check that the answer is supported by its sources, and the record that it ran.

No control layer judges truth. It enforces that the check happened: a reply failing it is sent back to the LLM to be rewritten or denied.

The material to check against is already on the path: tool and MCP responses cross the same gateway as the reply. The check is a rule, cheapest first:

• a source was called before the reply;
• the identifiers and numbers in the reply appear in what came back — plain conditions (7.2);
• an LLM as judge with the policy given to it as text, or an outbound call to the operator’s own service, for what those cannot cover.

The control layer supplies the material, the verdict and the record: every rule that matched, the one that won, and each signal with its inputs.

5.2 What stops the agent answering outside its role?

A system prompt is a request, and anything that edits the conversation can edit it. The role holds only where it is checked outside the conversation, on the reply before it leaves.

The same rules check the reply on the way out, layered like the check on the way in (4.1): classifiers score the answer against declared labels, each with its own threshold, and an LLM as judge reads the policy as text where a label is not enough. The role is written in ordinary words, so “this is investment advice” is a label.

A failing answer is sent back to the LLM to be rewritten or denied.

6. Data leaks, both directions

6.1 What stops sensitive data reaching an LLM or a tool?

Sensitive data means secrets as well as personal data, each kind recognized its own way, so the list of kinds is the whole of the claim. The rule that acts on a finding has to be readable by the enterprise and changeable without the vendor.

Once something is recognized, two reactions are usual: redact the finding and let the call through, or raise the risk level the request requires, so only providers trusted that far can serve it (7.5). Denying the call is the third.

What the strip is made of:

• Secrets — around thirty-five kinds recognized by their own shape: provider API keys, cloud access keys, personal access tokens, webhook URLs, private keys, session tokens, and a keyword rule for a password or key written into a value.
• Personal data — email address, phone number, payment card, public IP address, URL. Names, locations, national identifiers, bank account numbers and medical identifiers are available too, multilingual and switched on per policy, each with its own threshold.
• Reading through encodings — the scan decodes URL-encoding, the escapes inside a tool call’s arguments, and base64 that decodes to text.
• Replacement — a finding becomes a placeholder from a keyed hash, the same value giving the same placeholder throughout a request and nothing reversible without the key. The original is put back in the reply, including a streamed one.

6.2 What stops proprietary source code reaching an untrusted LLM?

Where code is the asset — chips, defense, quant, game engines — the leak is a function body nobody named, and a keyword list decides nothing.

Source code is fingerprinted, so a fragment of a protected repository on its way out is recognized. Matching is structural, and the reaction is a rule: stop the call, or raise the risk level so only an LLM inside the perimeter can serve it (7.5).

What that gives an operator:

• a non-reversible index built from the repositories the operator approves, signed, and verified by each role before it is used;
• a hit that names the repository, file, line range, symbol and category, with a probability, so the finding can be argued about;
• a dashboard over the traffic that was checked, the detections, and the split between stopped and delivered — and a window where the detector was not ready is marked as such, so zero detections is never shown as a clean result.

6.3 What stops secrets, personal data or source code coming back out in the answer?

The reply is the second path out. A streamed answer leaves in fragments, so a check that needs the whole text arrives too late.

The same check runs on the reply, recorded the same way: what a detector finds is redacted, or the reply denied, before it reaches a person. What the LLM itself produced is judged on its own, separately from putting back what was masked on the way in.

6.4 What can be done about data that already reached an LLM?

Some leak gets through. What was sent and when has to be a query over the records.

The check runs before the call, so the data usually never leaves, and the refusal is recorded. When a rule missed it, the decision record holds the event as it was sent, so what went out and when is a query. A detector added later runs over that history to find the rest (13.1). Removal at the far end is the LLM provider’s process under their contract with you.

7. Business rules

7.1 Can an enterprise’s own thresholds, limits and regulatory checks be expressed as rules?

Business thresholds, transaction limits and regulatory conditions are controls too, and they belong where the security ones already are. A threshold only the vendor can change puts the enterprise’s compliance on the vendor’s roadmap.

Yes, without a release: rules are declarative and live outside the application code, so a threshold is a policy change. A business limit and a regulator’s condition are the same kind of rule.

A number can move without touching the ruleset at all. A rule may read a declared parameter, resolved for the tenant of the event being judged, so one business unit points a threshold, an address or an LLM at its own value while every unit runs the same version.

7.2 What conditions and attributes are supported in evaluation?

A policy language reaches as far as what a condition can read and call, and the list of available functions is that reach.

Any field of the event; entity state — labels, counters, counters bucketed by time window, metadata; constants and environment; results of helper functions. Action type, actor, resource, amount, geography, device, network, time of day, allow-lists are each a payload field or a helper function away.

More than fifty helper functions ship built in, and they set what a condition can express:

• a filesystem path inside an allow-list;
• an SQL statement checked against a permitted set of operations;
• an email domain outside your own;
• a tool allowed for a role;
• regular expressions and glob matching;
• membership in a list;
• a number classified into named ranges;
• arithmetic and time arithmetic;
• an outbound HTTP call to a service of yours.

7.3 Can stateless and stateful rules be combined in one flow?

One decision needs both: this request is fine, the fourth one this hour is not. Split across two systems, that rule cannot be stated at all.

Yes, in one ruleset and inside a single rule. State for an entity is loaded only if a condition references it, so a stateful rule costs nothing on the events that never reach it.

7.4 How are counters, thresholds and time windows stored, and what is the consistency model?

A limit is a limit only if it holds while two events are in flight. The consistency model is what decides whether it does.

Strong consistency per entity, eventual between entities.

Every label, counter and metadata entry can carry an expiry. Windows are calendar-aligned buckets, declared per counter at the granularity a rule needs.

State is never updated apart from the decision that produced it: the state change and the decision record either both land or neither does. An event that already finished cannot be processed a second time, so no counter moves twice.

7.5 Can use be restricted by country or region?

A region restriction has to hold at the moment of the call. Where nothing qualifies to serve the request, it has to be refused.

Geography is a condition like any other, so a rule can refuse a request from a region without a release.

A rule can decide anything a condition can express. For which provider may see the content, risk level is the ready-made abstraction — one number from 0 to 100:

• a provider carries the most sensitive content it may be trusted with;
• an agent carries the minimum any provider must meet to serve it;
• a rule decides, per request, the minimum the content requires.

Four properties make it hold up:

• A rule has no way to name a provider, so it cannot be written to steer traffic at a chosen vendor.
• The strictest wins — the higher of what the rule requires and what the calling agent demands.
• Every comparison is “at least”, so a new level slots in between two already in use without invalidating a rule.
• Absence is not zero — a provider nobody classified meets no requirement, so one registered and forgotten never serves sensitive traffic.

If nothing qualifies, the request is refused — not served by the endpoint’s configured provider, not by a key below the requirement, not by the caller’s own credential. The refusal names nothing internal: no category, no detector, no level. It is recorded as the request’s own terminal event, so last quarter’s refusals are a query.

8. Token spend

8.1 What does the AI cost, broken down?

Finance needs a budget owner for AI spend, broken down to where it is spent. A budget is a figure that reconciles against the provider invoice.

Spend is accounted automatically per request, so cost lands per agent, per end customer and per LLM without anyone instrumenting the application. Because spend is a counter like any other, a rule can stop the work when a budget is reached, before the invoice.

A call the provider did not price is counted on its own, so a flat total next to a busy agent is visible. Totals are kept far below the precision of a cent: an ordinary LLM call costs a fraction of a cent, and rounding it is how a ledger stops matching an invoice.

8.2 Are there limits on LLM use per request, per step and per action?

An agent in a loop spends until something refuses it. A dashboard reports the spend after the fact; a ceiling stops it at the time.

A counter with a ceiling is an ordinary rule — per agent, per end customer, per hour.

A ceiling can also be declared on the agent itself, and applies wherever that agent calls:

• two measures — the money it may spend and the number of requests it may make. The count matters on its own: a provider that stops reporting usage stops moving the money total while the money is being spent, and on the tool gateway nothing is priced at all;
• three windows — a rolling hour as the brake on a runaway, and a calendar day and month as the budget someone reconciles against an invoice;
• a tenant-wide default, so an agent nobody configured still carries your organization’s limits;
• absence is the only way to say “no ceiling” — a ceiling of zero is refused, because the agent’s own off switch already says that (3.4).

A refusal on a ceiling is a policy decision: it names the rule and the ruleset version, says which measure and which window was crossed, appears in the audit trail, and can be run against past traffic before you switch it on.

8.3 Can a request be sent to a cheaper or safer LLM without changing application code?

The LLM list changes during the contract: a provider gets dropped, a cheaper one appears, one gets banned for a class of data. Choice of LLM has to be a policy change.

Yes. A rule can reroute a request that arrived for one LLM to another one, decided per request on whatever the rule can see — the tenant, the data class, the budget already spent.

How the choice is made:

• the rule states the risk level the content requires, and the strictest of that and the calling agent’s own level wins (7.5);
• candidates are filtered before any strategy runs: the provider must speak the dialect the request arrived in and must be trusted at least that far. Where nothing qualifies the request is refused;
• the LLM the caller named becomes a preference. Where a qualifying provider offers it, it is used untouched; otherwise the substitute answers, the reply names the LLM that actually served, and both the requested and the serving LLM are in the record;
• re-leveling a provider or declaring an agent’s level is configuration. No new ruleset version, nothing to re-approve.

9. Human review

9.1 Are multi-step and step-up flows supported — approve, block, challenge, second approval, escalate, log only?

Existing enterprise controls have more than two verdicts, so a control layer with two verdicts does not map onto them. In a step-up flow a person confirms and the agent waits, which outlives the request that started it.

Yes, through events.

• approve, block, log-only — covered by the three verdicts and declared actions;
• second approval and escalation — send the decision to human review, with a case behind it;
• challenge or re-authentication — the verdict and its tags go back to the application, which runs its own flow and emits a new event.

State between steps lives on the entity.

The tool gateway holds the wait. A call that needs an answer outlives the HTTP request: the agent’s connection ends and the agent asks for the outcome later. A hold that names a question is answered by the user, a hold that names a case by an operator, and each hold carries its own deadline and what to do when it passes. The send that follows an approval is authorized at most once, so one approval cannot move money twice. Whether the send completes is the product’s own path.

9.2 What does human review produce that can be audited?

An approval that is not recorded did not happen. An auditor a year later needs the decision, the person, the time, and proof that nobody with database access changed it since.

The case carries a queue, an assignment and a priority. Cases are typed: the type decides the queue, the screen the operator sees and the actions open to them. The decision lands in the audit trail and can re-enter the pipeline as an event of its own, to be judged by rules in turn. A deadline per queue is optional and its behavior is declared. A decision made by a person and a decision made by the deadline are recorded alike, each named as what it was.

An approval is sealed like any other decision (11.3).

Part 2. How you operate the control

10. Audit of every decision

10.1 What does one decision record hold?

An investigation six weeks later either reads one record or reconstructs the event from four systems. The record of a blocked leak holds the thing that leaked, so what is kept of the prompt is itself a decision.

One record per active version, plus one per shadow version beside it. Each holds:

• the event as received and the state before it;
• which rules matched and which errored;
• every signal and every helper function, with its parameters and its result;
• the verdict, the state changes applied, the actions fired with their parameters;
• the response returned to the caller, and how long the evaluation took;
• the version and deployment that produced it.

What makes it evidence:

• One record is the whole decision. Nothing has to be assembled from a second source to answer why, so the record survives a retention policy that drops everything around it.
• The version is named by the digest of its own content, so the record says which text judged the event.
• Fields that carry secrets or personal data are encrypted in the database, and who reads the records at all is a grant (15.1).

To do

• Attach one anonymized record. One record answers this faster than any description.

10.2 Are security events logged as well as decisions?

A refusal is two things at once: a policy outcome and a security event. Recorded only on the policy side, it never reaches the security operations center, where every other attack signal already lands.

It is recorded like any other decision: what was attempted, which rule stopped it, on whose behalf.

Two checks a security team makes:

• A refusal is recorded as a decision to refuse, kept apart from a fault, so refusals are never counted among faults.
• Where a gateway made the refusal, everything the gateway states about the refused party is kept separate and marked as asserted by the gateway. The chain proves the record was not altered; it does not make an assertion true.

Under flood the volume is set by the attacker, so record writing is sampled per endpoint, at a rate the enterprise sets, and the rest folds into a per-minute summary: how many, by reason, by gate, and how many times the credential changed. Enforcement is untouched, and the full count stays in OpenTelemetry metrics.

10.3 How does the record get out, and how fast?

The SIEM is already chosen and will not change for the control layer. Either the records fit it, or the fit becomes a project with a budget.

Three ways out:

• Push as it happens, over syslog or HTTPS, to any endpoint. The message is formatted by the rule itself, so the shape the enterprise’s SIEM wants is a few lines of policy.
• Pull through the API, with filters over any recorded field.
• Follow a change feed, for a streaming consumer that wants every record as it lands.

A SIEM outage loses nothing: the records stay in the enterprise’s own database (11.2), and the other two ways out still reach them.

10.4 What retention is available, and can it differ by data type?

One horizon for everything is either too short for the auditor or too long for the regulator. Retention has to be declared per kind of record and narrowed by condition.

Retention is declared, per entity, and narrowed by condition. A decision record, a human review case and a raw event each keep their own horizon, and a shorter horizon for anything carrying personal data is a condition.

The horizon itself can be set per tenant, so one tenant’s regulator is answered without changing anyone else’s, and each tenant’s deletion records what it removed inside that tenant.

Old records archive automatically into the analytical tier (2.3). Deletion can never overtake the archive: both horizons are resolved again immediately before anything is removed, so a policy change cannot delete records the archive had not yet copied.

11. Evidence

11.1 Is a decision reproducible, and what is pinned?

An incident gets reconstructed six weeks later, after the rules have moved on three versions. Everything that decided the outcome has to be pinned to the record.

The input, the ruleset version, the state snapshot and the time anchor are recorded, and the anchor is the ingestion time, so the same evaluation an hour later gives the same result. Rules that call nothing outside are exactly reproducible.

The version is pinned by a content digest: the event is admitted under it, and the evaluation is refused when the rules in force do not match, naming both.

A rule calling an LLM or an outside service is not exactly reproducible: the call is made again and the answer may differ. Input, version, state and time are still pinned, and the original call and its result stay in the record.

11.2 Who owns the records?

The evidence has to survive the end of the contract and the end of the vendor. Ownership means the records sit in the enterprise’s own database and leave in a standard export.

The enterprise does. The records live in its own database inside its own perimeter, the vendor has no production access unless granted, and export is a database dump. There is no copy on our side.

11.3 How is it known that a record was not edited afterwards?

Non-repudiation: a record the vendor or an administrator can quietly change settles no dispute. The proof has to survive a restore of the database and its log together, which puts it outside both.

Every covered write is sealed as it happens by a proof record holding hashes and positions only. Those records are hash-chained, and the chain is folded into one RFC 6962 hash tree per tenant with signed checkpoints. The tree, the tile layout, the checkpoint format and both proof types are the standard ones used by the Go checksum database.

Checkpoints can be witnessed to a write-once store outside the database that holds the records — S3 Object Lock in compliance mode, the enterprise’s or its auditor’s, a place the vendor cannot write to. The maximum age a checkpoint may reach unwitnessed is a setting the enterprise chooses, and the witnessed mode is refused unless the target, a signing key and that age are all in place.

Coverage is declared the way retention is: per entity and by condition, so the records with legal weight are sealed without paying to seal everything. An entity can also be marked so that a covered write fails while coverage is switched off — for an enterprise whose regulator does not accept a promise to keep it on.

11.4 Can the evidence be verified outside the deployment?

An auditor, a regulator or an end customer verifies without the vendor’s software and without its secrets. A published key is the one thing they take from the operator.

Yes, two ways.

A signed checkpoint is enough. Anyone holding one checks that a record is in the tree, with the standard tooling for the format (11.3).

TRACE trust records are the portable per-session form: a signed record of the identity that acted, the LLM, the policy in force, the data class touched and the tool calls made. The signature is Ed25519, checked against the operator’s published key, which is what makes the record the operator’s. The specification’s own conformance suite verifies the record and the transcript it commits to, and the control layer passes it.

The portable record shows that a call was held and a case opened. Who decided and when stays on your side, in full (9.2).

11.5 A person’s data has to be erased. Does that break the proof?

Deletion rights and tamper-evidence look like opposites. A lawful removal and an edited record both leave an absence, and verification has to tell one from the other.

No. Verification tells a lawful removal from tampering by reading the sealed record of what was deleted (10.4). An absence covered by a completed deletion is reported as lawfully removed, an absence nothing accounts for as tampering, and the two are counted separately, so routine deletions never bury the one that matters.

Personal fields can also be masked at write time, and a value never written needs no erasing.

12. Versions and who ships them

12.1 How is a policy version managed?

A version in production has to be immutable by every route, including one that bypasses the application.

Only a frozen version deploys, and a frozen version cannot be edited: its text, its code and the ruleset it belongs to are immutable to every administrator and every API writer.

The running deployment is immutable too: a change creates a new deployment, and the previous one keeps its own activation and deactivation times. “What was live at 14:20 on Tuesday” is a record.

A change to a ruleset can itself be judged by a ruleset (maker-checker): the change is raised as an event, the same engine evaluates it against a policy written in the same language, and the version carries its own review state. Any save that changes the content returns the version to unreviewed and clears what decided, so an approval never survives the text it approved.

12.2 Can one person write a rule, freeze it and ship it?

Writing a rule and putting it into production are two different rights. One person holding both is a path out of the network.

Permissions are granular, so authoring a version and deploying it are separate rights held by different people. Activating a deployment is its own right, and it can carry its own approval step, the way a version change does (12.1).

Whoever did it is recorded either way: the version write, the status change and the deployment each land in the administrative trail with the actor on them (12.5).

To do

• Split authoring from deploying before the call, and attach the role matrix. Two named people is what an enterprise accepts here.

12.3 Who reviews an extension before it can run?

An agent has a software supply chain: the code a rule calls, and anything a third party can publish into that path. Code reaching the path needs a version, an owner and an approval step, the same as the rules it runs beside.

A helper function that a rule calls is part of the deployment: it is declared, it ships with a version, and it goes through the same review and the same audit as the rules that use it. A rule author cannot add one. There is no marketplace where a third party publishes something the agent can reach. More than fifty ship built in (7.2), so the ordinary rule needs nothing new.

12.4 How does a change that affects the decisions become visible?

A change in behavior that nobody announced surfaces weeks later, in the enterprise’s own numbers. The difference between a candidate version and the one judging live traffic now is computable, so the enterprise can see it without an announcement.

Every version is dated, named and attributable, and the difference against what live traffic is judged by now is computed and kept.

Who tells you, within what time and through which channel, belongs in the contract beside it.

12.5 Is there an audit log of administrative actions?

Who changed the control and what the control decided are two separate records. The administrative one has to resist the administrator, which means it cannot be written by the same hand that can edit it.

Two trails. Operations records who did what and when, as a tree, so a nested action groups under the request that caused it. Changes records what a value was before and after, rendered as a real before-and-after difference. Both cover rulesets, versions, deployments, users, roles, permissions and keys, and any object and operation can be added, reading included.

What answers the insider question:

• Both are written as part of the change itself, so work that was rolled back leaves no trail entry claiming it happened.
• Both are covered by the evidence chain automatically once it is enabled, on their own chains, separate from the one carrying decisions, so an administrator cannot edit the record of their own administration.
• Both outlive the tenant they describe. Who deleted an organization, and when, stays answerable after the organization is gone. Where someone acted on behalf of another user, the real actor is recorded beside the assumed one.

13. Testing a change before it ships

13.1 How is a new rule version tested against reality before it goes live?

A rule change that looks safe can be wrong, and the only evidence that settles it is the enterprise’s own past traffic. A candidate judged over that history produces a list of the events it would have decided differently.

Backtest runs a candidate over a window of real past events and diffs its verdicts against the live version. Each run carries a reason, required by default, because the “why” belongs in the audit next to the result.

Two choices set what “would have” means:

• State — read each event’s captured state, or let the candidate’s own effects accumulate over the window;
• Outside calls — reuse the result recorded at the time, which keeps the run deterministic and free, or call live and test against the world as it is now.

Shadow runs the candidate on live traffic beside the live version: evaluated and recorded, changing no state and firing no action.

Traffic split gives a version a percentage of live traffic, the arm picked by a consistent hash of a value on the event, so the same end customer or account always lands in the same arm. Rollback is one change back.

13.2 How is divergence between versions analyzed?

A divergence rate means nothing until the denominator is named. An error on one side is a third outcome and belongs apart from a changed verdict.

Per event, in the console, as a difference of live against candidate, down to the rule and the signal that made them differ, and exportable as a file for the enterprise’s own auditor.

In aggregate, through the API: the divergence share, a breakdown by rule, the rule most often responsible, and samples of mismatches.

What makes the aggregate defensible:

• The denominator is stated: the events both versions actually decided;
• three axes — outcome (verdict, matched rules, state changes, actions, response), verdict only as the conventional headline, and rule, for when a different rule reached the same answer;
• an error is its own dimension, counted apart from a changed verdict, so a fault cannot inflate the number a ship decision rests on.

Direction is reported too: a candidate that refuses more than live is stricter, one that refuses less is laxer.

13.3 Are staged rollouts supported, and what does rollout governance look like?

A change advisory board needs a control it can name in a ticket. A rollout step and a rollback are each a named change with an actor and a time on it.

Deploy at five percent, watch, raise to twenty-five, fifty, a hundred — the percentages are the arms’ own, and the control layer refuses a split that does not add up.

Every change goes through the API, so the enterprise’s own pipeline drives it and nothing depends on a person using a screen. Every change lands in the administrative trail with the actor and the before-and-after values. The active deployment cannot be edited in place: a rollout step is a new deployment, and a rollback is the previous one, still intact.

Part 3. Enterprise foundation

14. Where it runs

14.1 Which deployment models are supported?

A hosted control layer means every prompt and tool call in the company crosses to a third party. A deployment model earns the answer once someone runs it in production.

In the enterprise’s cloud, its data center, or bare metal. Every deployment runs inside the enterprise’s own environment. It installs as one image with nothing else to orchestrate. Detectors that run locally ship in the same release, from the same registry.

14.2 Can it run in an isolated network with no outbound internet? What stops working?

An isolated network gets tested by pulling the cable and watching what breaks. A license check, a telemetry call or a hosted detector shows up on day one.

Yes. Images are pulled once outside and mirrored into the enterprise’s registry. The engine makes no outbound call of its own — only the ones a rule makes, under limits you configure. Witnessing checkpoints to a write-once store (11.3) needs that store reachable.

14.3 Which LLM does the control layer call, and does data cross a border?

Residency is a legal duty with a regulator attached, signed off by a data protection officer. It has to hold at every step the data takes, including a call a rule makes while deciding.

It may use none. Thresholds, limits, lists and counters call nothing, and then no data crosses a border.

Where a rule does call an LLM or a classifier, its location is configured — inside the enterprise’s perimeter, or a provider the enterprise chose and contracted. That provider is its subprocessor under its own contract, and it belongs on the list its data protection officer keeps.

15. Access and isolation

15.1 Is there role-based access control, and how does it differ between runtime and administration?

Administration is who may read, change or delete a rule. Runtime is what the evaluation itself may see while it decides, and the two are separate grants.

Two layers.

• Administration — read, write and delete are granted per kind of object, and the roles are configured, so the enterprise’s own role names apply.
• Runtime — the tenant is the security boundary: evaluation runs in the tenant of the event, and a rule reads only that tenant’s data (15.3).

15.2 Which single sign-on protocols are live in production, and how does an account come into existence?

Single sign-on counts where it runs in production against the enterprise’s own issuer, because that issuer is what joiners and leavers already flow through. An account created by hand on first login is an account nobody removes on the last day.

OIDC in production, against the enterprise’s own issuer, with just-in-time provisioning on first login, so no account is created by hand. SAML through a bridge, a separate component.

15.3 How is tenant isolation enforced?

Isolation holds where one filter covers every path to the data. A rule each developer has to remember fails on the first one who forgets.

A tenant is one organization: a customer of the product, or a business unit inside one. The tenant comes from the token, and a caller cannot set it or ask for another one. The filter is applied in one place for every read and every write, with tests behind it.

Because the deployment is the enterprise’s, it can satisfy itself: take a token for one tenant and try to reach another tenant’s rules, decisions and state through the API.

A separate database per tenant is on the roadmap and gets built when an enterprise needs it.

15.4 How is the administration plane protected?

A product with its own login is a second identity system, and nobody remembers to offboard from it. The console belongs behind the directory that already handles joiners and leavers.

Sign-in delegates to the enterprise’s own issuer over OIDC (15.2), so the console inherits the multi-factor and conditional access already in force there, and a leaver loses it with the directory account. What the control layer adds: rights per kind of object under the enterprise’s own role names (15.1), and every administrative action in two trails (12.5).

16. What you install and update

16.1 What is in the shipped image, and how is it verified before it runs?

The vendor’s own software supply chain arrives with the image. An enterprise registry has to verify the contents and the builder on its own, before anything starts.

Every image is built with a bill of materials and build provenance attached to the image itself.

• The build scans for known vulnerabilities and fails on a high or critical finding that has a fix.
• Dependencies are scanned on every commit.
• The published manifest is signed, and the signature is tied to the build pipeline, so the enterprise’s registry can verify who built it before it runs.
• Images are published for amd64 and arm64, each built natively.

16.2 How are updates performed in an enterprise-hosted install, and can they be rolled back?

A platform team runs the update on its own cluster. The software version and the policy version have to move independently, so an upgrade never changes a verdict.

A rolling update of the image, the ordinary way for the enterprise’s own cluster. Policy versions are independent of the software version, so an update does not change a decision. A detector a rule calls is a versioned helper function (12.3), and the version it runs against is a parameter of the rule, pinned like any other.

You do not have to take that on trust. Backtest your live ruleset against its own history on the new build before you accept the upgrade: that reads zero divergence (13.1). Anything else is the upgrade changing a verdict, named down to the rule.

16.3 How are secrets and keys handled?

Secrets already live in the enterprise’s own manager, with its rotation and its access records. A product that insists on a key store of its own becomes a second place keys are held, outside all of that.

Read from the environment, so any secret manager that populates environment variables works. Sensitive values stored as data are encrypted with a rotatable key.

16.4 What protects data in transit and at rest?

Encryption is answered by naming who holds the key, because that is who can read the data.

Everything runs inside the enterprise’s own environment (14.1) and the records sit in its own database (11.2), so transport and encryption at rest are the standards already in force there, configured at deployment. What the control layer adds: fields carrying secrets or personal data are encrypted by it (10.1).

17. Staying up

17.1 What uptime is committed?

On the synchronous path the control layer is a new single point of failure, and a vendor outage gets priced as an enterprise outage. Either a number goes into the contract, or availability belongs to whoever runs the software.

It runs inside the enterprise’s environment, so uptime belongs to its own operations. Nothing about that availability depends on a company it cannot audit.

It holds no state between requests and runs in as many copies as are wanted. State and records live in the enterprise’s own database, under whatever high availability they already run.

17.2 What has to be restored after a failure, and how fast?

Recovery targets already exist for everything else in the estate. The control layer adds policy versions and decision records, and restoring both must not depend on a vendor being reachable.

Two things: the policy versions and the records. Both are rows in the enterprise’s own database. They fall under the backup policy it already has, recovery targets are the enterprise’s to set, and there is no vendor step in the path.

17.3 Is fail-open and fail-closed supported per rule or per category, and where does it apply?

Fail-open is a legitimate choice where it is declared. Each failure on the path is its own behavior to declare, and a health check on the engine alone misses most of them.

Five levels, each declared separately:

17.4 What happens to events that fail before evaluation?

An event that nobody decided is a hole in the audit, and a hole nobody can see is the expensive kind. Those failures have to be kept, countable and searchable.

They land in a dead-letter queue with the payload and the error, searchable through the API. No decision record, because no decision was made.

18. Getting in and out

18.1 How much of the application has to change, and can the check be bypassed?

A control an engineer can route around by changing one URL is a suggestion. Enforcement has to come from outside the application.

Pass-through needs no change. The product’s existing URL for LLMs, MCP servers or outbound traffic points at the gateway instead. Human review, step-up and challenge add application work: asking for the outcome and emitting the follow-up event (9.1).

Enforcement is a network endpoint. Close outbound traffic in the enterprise’s cluster to everything except the gateway and it becomes the only path a call can take — a control the enterprise’s own infrastructure team owns and can verify.

Configuring the policy is still work.

18.2 Is there a published input and output schema?

A verdict that carries only allowed or denied sends the application back for a second lookup before it can act. What the response carries decides how much application code sits around it.

Yes. The response carries whatever the rule puts in it — matched rule, reason, tags, version — so the application acts on the verdict without a second lookup.

18.3 What is achievable in two weeks, and what usually causes delay?

Two weeks is a date somebody has already promised. A named list of what causes delay says what has to be arranged now, before the clock starts.

Two weeks buys one event type, a working ruleset, decisions recorded, one gateway in place.

Delay comes from three places, in this order:

• agreeing what an event contains
• getting the deployment approved
• connecting the SIEM

18.4 If a component is replaced later, what can be taken along?

Lock-in ends up in the contract, written by procurement. The policies written and the history accumulated belong to the enterprise, in a form another tool can read.

Policy definitions are declarative text. Decisions and state are rows in the enterprise’s own database. Both leave in a standard export.

19. What stays open

19.1 What stays open once everything above is in place?

Every control layer has a limit, and a deployment plan has to work around it. A gap named at the start is one nobody meets at go-live.

The control layer runs inside the enterprise’s own environment, so it falls inside the scope already audited there. It carries no attestation of its own. Latency under load is a number the enterprise measures in its own environment (2.2). Reports come from complete audit data, formatted by the enterprise to whatever standard it files under. Sub-millisecond decisions on a trading hot path, and very large numbers of tenants in one shared pool, sit outside what this was built for.

Built for agent control, payments, trust and safety, internal operations, audit-heavy work, and on-premise deployment in regulated environments.

What this document does not answer

Everything above is the control layer. The rest of the review sits outside it.

• The LLM and its data. Which LLM, what it was trained on, whether customer data is trained on, accuracy, bias testing.
• Staff training and internal AI policies. What the team has had, and how AI use is written down. A document is expected for each.
• Certifications, or what goes in the file when there are none.
• The reviewed product’s own supply chain. Bill of materials for what it ships, how fast a fix lands by severity, and what happens to a version no longer supported.
• Contract terms. Liability, indemnification, incident notification in hours, continuity if a small company stops existing.
• Territory. The EU AI Act, for one, sets training and documentation duties that do not exist elsewhere.
• Expectations beyond these questions. Every enterprise adds questions of its own.

Stuck on some of these, or have others? Send them over — happy to help.