Skip to content

The verdict lattice

A rule is not true or false while a run is happening. It is true so far, false already, or still owed something — and when the run ends, whatever is still owed has to resolve into one of those or admit that it cannot.

Status Claims
satisfied Nothing so far has broken this, and nothing is outstanding.
violated Something broke it. This is terminal — a violated rule does not recover.
pending An obligation is open. The run has not yet done what the rule requires, and has not yet run out of time.
unresolved The run ended with an obligation still open.

respondWithin('pageOncall', 'pageOncall', { within: 4 }) opens an obligation the moment the call is made. Two events later the obligation is still open and nothing is wrong — the run has two events left to discharge it. Calling that a violation would make every liveness property fail at the moment it starts mattering.

leaning appears only on pending and says which way the obligation would fall if the run ended here. That is what makes a live dashboard readable: a rule leaning violate with one event of slack left is the thing to look at, and it is distinguishable from one that just opened.

A trace that simply stops is not evidence that a property broke. A run that was killed, a process that was deployed over, a recording that was truncated — none of those tell you the agent did anything wrong, and a build that failed on them would be failing on infrastructure.

So unresolved passes by default, appears in the summary and the counts, and is promoted to a failure by --strict when every run in your corpus is supposed to have completed.

Because “this rule was broken during this run” is the useful claim, and a rule that recovered would let a run launder a violation by doing the right thing afterwards. The verdict stream still records everything that happened — verdicts are emitted on transition — so a consumer that wants “when did it break” reads the stream, and one that wants “did it break” reads the last value.

satisfied, violated and pending come from the transition, per event. unresolved comes only from end-of-run resolution: finish on the monitor, observeRun’s onFinish, or the end of a batch check. A verdict stream that contains unresolved therefore always contains it at the last event’s position, alongside whatever else resolved there.