Change a rule. See what it does before it goes live.
Write the candidate, see what it does to traffic you already recorded and to traffic arriving now, then promote it.
Rules only, so changing one needs no release.
A version has four states
- draftyou are editing it
- candidateunder test, on real traffic
- frozenlive and immutable, once you promote it
- archivedreplaced by a later version
A frozen version cannot be changed. That is what makes a decision record from six months ago worth anything: the rules it names are still the rules that ran. Rolling back is naming the previous version in the deployment, and it takes effect immediately.
Backtest: what would it have done to traffic you already have?
Run the candidate over real past events and compare its verdicts against the version that was live at the time. Same engine, same events, different ruleset.
backtest candidate v8 baseline v7 window 1 - 31 Jul events 184,203 verdict changed 1,847 1.0% unchanged 182,356 99.0%
The percentage tells you how much moved. Below it, every event whose verdict changed is listed: open them one at a time and read what the new rules did, and that is how you judge whether the change was right.
| Event | v7 live | v8 candidate | Rule that fired |
|---|---|---|---|
| transfer_intent 4f2a | approved | rejected | daily_volume_exceeded |
| transfer_intent 71c8 | approved | rejected | daily_volume_exceeded |
| transfer_intent 9be1 | approved | flagged | first_time_destination |
| transfer_intent 22d0 | rejected | approved | cooldown_active |
Filter to the events that changed, to one rule, to a single wallet, or to the errors. Then open one and read all of it: the event, the state the engine saw, which rule matched, and why each version reached the verdict it did.
Watch for a row where the candidate approves what the live version refused. In the table above that is transfer_intent 22d0: v7 rejected it on cooldown_active and v8 lets it through. Tightening a rule shows up in complaints; loosening one shows up in losses, so this is the direction to be sure about before you promote.
Shadow: run it on live traffic, deciding nothing
A backtest answers what the candidate would have done to traffic you already recorded. Shadow answers what it is doing to traffic arriving now. The candidate runs beside the live version on every event and records the verdict it would have reached, and nothing it decides reaches a user, a payment or a takedown.
A/B: let it decide, for part of your traffic
Shadow tells you what the candidate would do. A/B lets it do it, for a share of traffic you choose, while the live version keeps the rest. The split is deterministic, so the same wallet always lands on the same version and a customer never meets one rule on Monday and another on Tuesday. Both versions write the same decision records, so you compare outcomes on real events.