perf(auto): per-round wall-clock regression ~3x in PR-A/B/β/γ merge train (RFC #1256 §I5)

[shaun0927]

Summary

ooo auto interview rounds on main HEAD 2f6247c6 (post PR-A/B/β/γ merge train) execute at ~3× the wall-clock cost of the pre-merge baseline. R3 cli-todo completed only 4 rounds in 600 s where R2 (baseline 73ad865a) completed 12 rounds in the same budget.

This issue tracks the implementation of RFC #1256 §I5 (performance budget) and the bisect work needed to identify which of the four merged PRs introduced the regression.

Evidence

Run	HEAD	Rounds in 600 s	Round-cost (s/round)
R2	73ad865a (pre-merge)	12	~50
R3	2f6247c6 (post-merge: PR-A/B/β/γ all in)	4	~150

R3 raw timeline (from ~/.ooo-observability/R3-20260527-131856/R3-cli-todo.log):
R1@15.6s, R2@22.4s, R3@30.3s, R4@57.4s — the cost-per-round increases sharply once ambiguity scoring (LLM call) enters the loop.

Why this matters

RFC #1256 §I5 sets a regression budget of 1.5× for any substrate change in src/ouroboros/auto/. The PR-A/B/β/γ merge train is ~3× — twice the budget. Without a baseline policy this kind of silent slowdown will keep accumulating, eventually consuming whatever phase budget NEW-A raises, and making the vision (any task → product) impossible to deliver within reasonable wall-clock.

This is not just a performance ticket: a slow ooo auto is a ooo auto that BLOCKED on wall-clock more often, which compounds with NEW-A. Fixing NEW-A without fixing NEW-B postpones the same failure mode at a higher budget.

Hypothesis space (to be confirmed by bisect)

Likely candidates:

• PR-A/B (feat(auto): safe-default lateral escalation substrate (#1248 PR-A, additive, behavior unchanged) #1250/feat(auto): safe-default matcher fire → lateral escalation routing (#1248 PR-B, depends on #1250) #1251) — safe_defaults.py substrate may add synchronous LLM calls or heavy import-time cost.
• PR-β (feat(auto): make ledger_done the primary interview closure check (PR-β) #1252) — ledger_done check may invoke a backend call per round rather than checking local ledger state.
• PR-γ (test(canonical): runtime-binary preflight + raw MCP passthrough (PR-γ) #1253) — runtime-binary preflight may add startup cost per session (one-time, not per round — less likely to explain 3× per round but possible if invoked repeatedly).

Bisect plan:

1. Capture R-run timing at each of: 73ad865a (R2 baseline), c3e82125 (post PR-A), 0303a429 (post PR-B), 23d06b0e (post PR-β), 2f6247c6 (post PR-γ).
2. Identify the commit that introduces ≥ 1.5× per-round regression.
3. Inspect the diff for new synchronous LLM / I/O calls in the round loop.

Proposed fix scope (after bisect identifies the offender)

Three patterns to look for in the offending PR's diff:

1. Synchronous LLM call in the round loop where async or cached would suffice.
2. Heavy import triggered on first use inside the loop (one-time cost amortized over rounds; only relevant if rounds increase).
3. Per-round serialization of state (e.g. EventStore writes) that should be batched or async.

Acceptance criteria

• Bisect identifies the responsible commit, recorded in this issue's evidence.
• A targeted PR brings per-round cost back to ≤ 1.5× R2 baseline (≤ 75 s/round).
• PR template includes a mandatory "R-run comparison" section for any src/ouroboros/auto/ change (RFC RFC: reconcile ooo auto terminal-liveness policy with the narrowed invariant #1256 §I5 enforcement).
• CI lint blocks PRs to src/ouroboros/auto/ without the R-run section attached.

Anti-scope

• Does not change closure-ladder behavior (NEW-A's scope).
• Does not change observability events (RFC §I4 / fix(observability): wire auto.interview.* events to EventStore (PR-ε) #1254 scope).
• Does not redesign LLM choice or model selection.

Cross refs

• Anchor: RFC: reconcile ooo auto terminal-liveness policy with the narrowed invariant #1256 (RFC, §I5 implementation)
• Sibling: NEW-A (interview_phase_deadline) — combined fix unlocks R4 product reality test
• Suspects (bisect targets): feat(auto): safe-default lateral escalation substrate (#1248 PR-A, additive, behavior unchanged) #1250, feat(auto): safe-default matcher fire → lateral escalation routing (#1248 PR-B, depends on #1250) #1251, feat(auto): make ledger_done the primary interview closure check (PR-β) #1252, test(canonical): runtime-binary preflight + raw MCP passthrough (PR-γ) #1253
• Evidence: ~/.ooo-observability/R3-20260527-131856/, R2 baseline run cited in auto safe-default unsafe-context matcher should escalate to lateral persona, not BLOCKED — refuses-to-die-quietly invariant (SSOT #1157 L5) #1248 R2a/R2b

[ouroboros-agent]

This issue has enough evidence to confirm a real ooo auto performance regression, but the requested acceptance criteria are broader than a safe single automated fix. Routing to maintainer planning/approval before implementation.

Scope assessment:

• Affects the src/ouroboros/auto/ interview loop and related performance-sensitive paths (interview_driver.py, safe_defaults.py, ledger.py, and pipeline/adapters as needed).
• Requires bisecting the PR-A/B/beta/gamma merge train before choosing a targeted optimization.
• Also asks for PR-template and CI lint enforcement for future src/ouroboros/auto/ changes, which touches protected repository policy/CI boundaries.

Given the existing needs-design label and the protected CI/template portion of the acceptance criteria, this should be planned by a maintainer before any automated PR is attempted.

ouroboros-agent[bot]

[shaun0927]

Bisect evidence (auto-posted)

Bisect run completed on 2026-05-27 17:09 KST.

Results

Commit	Label	Total (s)	Rounds	Per-round (s)	Terminal
73ad865a	baseline_pre_merge	254.2	24	10.6	unknown
c3e82125	post_PR-A	603.5	10	60.4	unknown
0303a429	post_PR-B	210.7	24	8.8	unknown
23d06b0e	post_PR-beta	132.3	16	8.3	unknown
2f6247c6	post_PR-gamma	192.4	18	10.7	interview_max_rounds_no_closure

Per-anchor logs

Interpretation guidance

Raw script

/tmp/bisect_perf.sh (macOS-friendly, Python subprocess timeout).

[shaun0927]

Closing — the headline ~3x per-round regression does not reproduce.

The in-thread bisect (2026-05-27) showed only the immediate post-PR-A commit (c3e82125) regressed to ~60s/round; PR-B/β/γ all settled at ~8–10s/round against the ~10.6s baseline. The CI enforcement this issue called for has since shipped — .github/workflows/auto-perf-budget.yml + scripts/check-auto-perf-budget.py gate the ## R-run comparison section.

No live offender remains. The only residue is numeric per-round threshold detection (vs. the current section-presence check); tracking that as a small backlog item under the perf-gate script / RFC #1256 §I5 rather than keeping this open. Reopen if a fresh regression appears.