Stochastic Macro
Request access
Engineering notes · Convergence

A bounded, provably-terminating state machine for AI review loops.

Stochastic Macro ·

An AI review bot without a state machine will cheerfully re-raise the same finding every cycle. The developer pushes a fix, the bot re-reads the diff, decides the fix is "only partial," re-raises the original finding. The developer pushes again. The bot re-raises. This is a live failure mode for every PR review pipeline we've seen that does not formally bound convergence.

This post describes a small state machine that gives you a provable upper bound on how many cycles a single review thread can remain open. The design has three parts: a four-action taxonomy, a per-cycle one-action-per-thread invariant, and a round-count escalation trigger on same-stance repetition.

The four-action taxonomy

Every time the AI reviewer looks at a previously-opened thread, it must pick exactly one of four actions:

  • resolve — "this is now fixed; close the thread." The thread is marked resolved and removed from future cycles.
  • veto — "the author's response does not address my concern, but I am not going to re-argue; escalate to a human." The thread is handed to a human reviewer and removed from the AI's future cycles.
  • escalate — "this is higher-severity than I thought; raise it and hand to human." Same effect as veto for termination purposes.
  • reply — "the author's response is interesting but not dispositive; here's one more round of dialogue." The thread remains open for the next cycle.

Crucially, there is no "re-raise the same finding as if it were new" action. Raising a finding that duplicates an already-open thread is a protocol violation — detect it and fail the cycle rather than allow it.

The one-action-per-thread invariant

Per cycle, each thread receives exactly one action. Not zero (if the thread is open, the reviewer must decide something) and not more than one (a thread cannot both reply and resolve in the same cycle). This invariant makes the state machine well-defined and lets you compute the exact number of cycles analytically.

Implementation: before the reviewer runs, enumerate the set of open threads it owns. After the reviewer runs, require each thread in that set to map to exactly one of the four actions above. If the mapping is missing or multi-valued, fail the cycle.

The round-count escalation trigger

The remaining risk is the reply-loop: reviewer replies, author replies, reviewer replies again, indefinitely. The termination trick is to bound same-stance repetition.

Define "stance" as the reviewer's current position on a thread: either it wants the author to change something (seeks_change) or it does not (accepts). Maintain a counter per thread: round_count. When the reviewer's stance this cycle matches its stance in the previous cycle, increment round_count. When the stance flips, reset to zero.

The escalation rule: when round_count >= 2 — i.e., the reviewer has held the same stance across three consecutive cycles — the reviewer is required to pick veto or escalate on the next action. reply is no longer a legal action on that thread.

Why this terminates

A thread's lifecycle has a bounded number of cycles:

  1. Cycle 1 — reviewer opens the thread with its initial stance.
  2. Cycle 2 — reviewer can reply, resolve, veto, or escalate. If the reviewer replies, round_count becomes 1 (same stance) or resets to 0 (stance flipped).
  3. Cycle 3 — if round_count reached 2 by this point, reply is disallowed and the reviewer must veto/escalate or resolve.

In the worst case — the reviewer and author disagree from the start and never converge — the thread lasts about three cycles before the round-count rule forces an escalation. The exact bound depends on how you count cycle 1, but the important property is that it is finite and small, not "eventually with high probability."

Pseudocode

for thread in open_threads:
    stance = reviewer.current_stance(thread)
    prev   = thread.previous_stance

    if stance == prev:
        thread.round_count += 1
    else:
        thread.round_count = 0

    if thread.round_count >= 2:
        # reply is no longer a legal action; reviewer must escalate.
        action = reviewer.pick_action(
            thread,
            legal_actions=["resolve", "veto", "escalate"],
        )
    else:
        action = reviewer.pick_action(
            thread,
            legal_actions=["resolve", "veto", "escalate", "reply"],
        )

    apply_action(thread, action)

The legal_actions parameter is important: do not rely on the model to self-police the round-count rule. Enforce it structurally by constraining the action set the model is allowed to choose from.

Handling the "duplicate finding" attack on the model itself

Models will sometimes raise "new" findings that are semantically the same as an already-open thread. This undoes the termination guarantee because each duplicate is a fresh thread with its own counter.

Defense: before accepting a reviewer's newly-raised findings, compare each against the set of open threads the reviewer already owns. If there's a substantive overlap — same file, same location, semantically similar title/body — reject the finding as a duplicate and force the reviewer to pick an action on the original thread instead. This can be a small embedding-similarity check or, at the simple end, a file+line overlap check with a title-keyword threshold.

Why per-thread, not per-PR

It is tempting to bound convergence at the PR level — "after N cycles, give up." The problem is that PR-level bounds are too coarse: a PR might have 30 threads in various states of disagreement, and you want the ones that are converging to keep converging while the ones that are stuck get escalated. Per-thread bounds give you that; per-PR bounds do not.

Why we're publishing this

We have not seen this particular combination — four-action taxonomy, one-action-per-thread invariant, and round-count escalation trigger with a legal-action constraint — specified anywhere in public writing on LLM review agents. It is the kind of small, boring state-machine work that is worth having in the public record rather than rediscovered privately by every team.

← All engineering notes