The verdict lattice
A rule is not true or false while a run is happening. It is true so far, false already, or still owed something — and when the run ends, whatever is still owed has to resolve into one of those or admit that it cannot.
| Status | Claims |
|---|---|
satisfied |
Nothing so far has broken this, and nothing is outstanding. |
violated |
Something broke it. This is terminal — a violated rule does not recover. |
pending |
An obligation is open. The run has not yet done what the rule requires, and has not yet run out of time. |
unresolved |
The run ended with an obligation still open. |
Why pending is not violated
Section titled “Why pending is not violated”respondWithin('pageOncall', 'pageOncall', { within: 4 }) opens an obligation the moment the call is made. Two events
later the obligation is still open and nothing is wrong — the run has two events left to discharge it. Calling that a
violation would make every liveness property fail at the moment it starts mattering.
leaning appears only on pending and says which way the obligation would fall if the run ended here. That is what
makes a live dashboard readable: a rule leaning violate with one event of slack left is the thing to look at, and it is
distinguishable from one that just opened.
Why unresolved is not violated either
Section titled “Why unresolved is not violated either”A trace that simply stops is not evidence that a property broke. A run that was killed, a process that was deployed over, a recording that was truncated — none of those tell you the agent did anything wrong, and a build that failed on them would be failing on infrastructure.
So unresolved passes by default, appears in the summary and the counts, and is promoted to a failure by
--strict when every run in your corpus is supposed to have completed.
Why violated is terminal
Section titled “Why violated is terminal”Because “this rule was broken during this run” is the useful claim, and a rule that recovered would let a run launder a violation by doing the right thing afterwards. The verdict stream still records everything that happened — verdicts are emitted on transition — so a consumer that wants “when did it break” reads the stream, and one that wants “did it break” reads the last value.
Where each one appears
Section titled “Where each one appears”satisfied, violated and pending come from the transition, per event. unresolved comes only from end-of-run
resolution: finish on the monitor, observeRun’s onFinish, or the end
of a batch check. A verdict stream that contains unresolved therefore always contains it at the last event’s position,
alongside whatever else resolved there.