irvingpop · Oct 28, 2026 · Oct 28, 2026
@@ -168,10 +168,11 @@ Run the spec's workflow acceptance tests under a test org ruleset that requires
 - [x] The rest of the acceptance list in the spec's "Testing" section that needs no account X, except the four unit-test-only cases listed in the next item (approver removed, edited override, artifact removed, gate cancel after inline comments), including the outage process (spec "Phase 3 results"). Design change (spec P3-1): the invalid federation rule case ran live with two denied exchanges (runs 40561015054 and 40583290922); the exhausted budget case is covered by V9 (phase 0) and the unit tests, with no Console limit edit (ruling Q7).
 - [ ] Acceptance cases that needed account X: V6 f as a Write user is done (residual CONFIRMED, PR 41). The web UI "Re-run all jobs" button on a ruleset-started run passed (run 40628870689, PR 43: new attempt on the same run ID, live state re-read, summary comment updated; merge box not observed because the PR was merged); the override cases ran with the REST re-run. Also not run, covered by unit tests: gate cancelled after inline comments with a refused summary write, approver role removed, edited override, artifact upload removed.
 - [x] Sonnet 5.5 on every pass (spec P3-4). Release evals 13 of 13 at M3 (run 40592861233); live run 40593729924 passed with `claude-sonnet-5-5` (spec P3-6). Eval at 302882f: runs 40589761270 and 40590079545 each matched 13 of 13 with no failed pass; the Opus reference run 40528717338 also matched 13 of 13. Total tokens 0.97x and 0.98x of Opus, output tokens 0.71x and 0.76x, eval wall time 84 s and 82 s against 138 s. The switch takes effect at the next release: the production rule pins a SHA, so merging does not change production.
+- [x] Comment redesign release M4 `9e2795b` (2026-10-28, spec P3-7): unit tests run 40710532524 success; release evals run 40713029193, 13 of 13 matched; R1 edited in place to M4 (no new rule); TR pin moved from M3 to M4 and read back; live runs 40713506554 (PR 28, pass layout) and 40713856249 (PR 40, fail layout). Production now runs M4.
 
 Owner:
 
-- [x] Owner: set `job_workflow_ref` of the production federation rule to the candidate release SHA. Done 2026-10-27 (S1, then M1 `590a387`; the run at M1 got a token; no read-back recorded). The candidate's code must name that rule ID (spec "Auth and cost", "Each release").
+- [x] Owner: set `job_workflow_ref` of the production federation rule to the candidate release SHA. Done 2026-10-27 (S1, then M1 `590a387`; the run at M1 got a token; no read-back recorded). Later moved to M3 `2c63f49`, then to M4 `9e2795b` on 2026-10-28 (spec P3-7; the rule detail value was not read back, and the denied exchange before the edit was NOT RUN). The candidate's code must name that rule ID (spec "Auth and cost", "Each release").
 - [ ] Owner: an account outside the org opens the fork PRs.
 - [x] Owner: test users with Write, Maintain and Admin on claude-review-test for V25. Done with account X `irvingpop2` (2026-10-28). Remove its access and empty the TR bypass list after phase 3 (owner's choice pending).
 

@@ -984,3 +984,4 @@ P2: phase 2 rulings, 2026-10-27 (UTC). Where a P2- line changes an earlier line,
 - P3-4. Model per pass, Sonnet 5.5 (2026-10-27). The owner proposed `claude-sonnet-5-5` for general use. The model is now a per-pass setting, and every pass uses Sonnet 5.5. The switch needed a full match of the 13 cases (P2-4 bar); Opus 5.5 would stay on any pass where Sonnet missed. Result: two eval runs at 302882f, [40589761270](https://github.com/OperationCode/.github/actions/runs/40589761270) and [40590079545](https://github.com/OperationCode/.github/actions/runs/40590079545), each matched 13 of 13 cases with no failed pass (Opus reference [run 40528717338](https://github.com/OperationCode/.github/actions/runs/40528717338), also 13 of 13). Against Opus, total tokens were 0.97x and 0.98x, output tokens 0.71x and 0.76x, and eval wall time 84 s and 82 s against 138 s. No currency figures were recorded. The switch takes effect at the next release: the production rule pins a SHA, so merging does not change production.
 - P3-5. Summary comment redesigned for fast human judgment (owner and coordinator, 2026-10-27). The comment now opens with a result heading and one colored callout, then a findings table, a checks strip and a provenance line, with the waiver and the full record collapsed ("Summary comment" under "Publishing (code)"). The owner chose the Fable design with these rulings: every fail uses the red `[!CAUTION]` callout and the red heading, waivable or not (a waivable fail on a fake check was misread as a mild warning under `[!WARNING]`, so `[!WARNING]` is not used for a fail); the waiver line stays in the collapsed Waiver block; model summaries and token counts stay in the collapsed record; duplicate findings merge only at the same file, side, line and gate effect; a medium finding in a blocking category reads "Does not block"; the visible line shows the 7-character head SHA and the record shows the full SHAs; emoji are allowed, each next to its word. Machine markers, the override line format, the size limit and the overflow path are unchanged.
 - P3-6. Account X results and Sonnet release (owner and coordinator, 2026-10-28). Account X is `irvingpop2`, a separate GitHub account outside the org that the owner made as a test identity; it became an outside collaborator when it got repo access. Steps ran from an isolated container. V8 b pass (PR 38): the central run started on a first-time-contributor fork PR with no approval click, and marking the draft ready started no new run. V8 a pass (PR 39): from a first-time fork, GitHub holds the fake `pull_request` job (`action_required`), so it cannot start before the central run; after a maintainer approved only that run it passed, and the box stayed BLOCKED and followed the central run; the fake-first case stays covered by PR 36. V23 pass (PR 40): a same-path `claude-review.yml` in the fork PR did not replace the central run, which failed on the bidi hard block; the fork run stayed `action_required` and never ran, so the running case is covered by PR 35. V6 f as a Write user: CONFIRMED residual (PR 41); deleting the newer run reopened the box on an older pass, and it stays in the residuals the owner must accept before org-wide enforcement. A Write user's plain merge of a failed PR was refused; this does not test the bypass, and the `--admin` attempt as Write was not run. V25 pass for Maintain and Admin, Write bypass NOT RUN: with the TR bypass list, a Maintain user (PR 42) and an Admin user (PR 43) merged a failed PR; GitHub records rule suite result `bypass` with the user ID as actor and no role ID, and the org rule-suites endpoint returns 403 on this plan, so use the repo endpoint; a direct push to main by Maintain is refused (GH013). The TR bypass list is a test-only setting and goes back to empty after phase 3 (owner's choice pending). Process: the owner ran the bypass merges and the direct push by hand, because the automation's safety classifier blocks bypass merges; the evidence comes from the GitHub API. Sonnet release: release evals 13 of 13 at M3 ([run 40592861233](https://github.com/OperationCode/.github/actions/runs/40592861233)); the live run [40593729924](https://github.com/OperationCode/claude-review-test/actions/runs/40593729924) passed with `claude-sonnet-5-5`. Console read-back: the owner read back R1 at 590a387 and R2 at 1cefe42 on 2026-10-27, before the M3 edit; R1 was then moved to M3; the service account and the workspace were not read back. V26 a is done for `job_workflow_ref`. NOT RUN, stays open: the Write `--admin` bypass attempt and V26 c deny at M2 before the R2 edit. Web UI "Re-run all jobs": pass (run 40628870689, PR 43, 2026-10-28): the click made attempt 2 of the same run ID, which re-read live state and updated the one summary comment; the merge box was not observed because the PR was already merged.
+- P3-7. Release M4, the comment redesign (owner and coordinator, 2026-10-28). M4 is `9e2795b97b22869b9e5c7a777e71a2afa8366eeb` on main (code from PR 12 `a410ddd`, docs PR 13). There is no separate candidate branch: the merge SHA was validated directly. Unit tests: `test.yml` at M4, [run 40710532524](https://github.com/OperationCode/.github/actions/runs/40710532524), success. Release evals: `eval.yml` at M4, [run 40713029193](https://github.com/OperationCode/.github/actions/runs/40713029193), 13 of 13 matched, success; the coordinator approved the run under the owner's delegation through phase 4. Console: the owner edited R1 `claude-review-prod-r1` (`fdrl_01N2ZktTiVKqUvvXwywBFfys`) so that `job_workflow_ref` is `OperationCode/.github/.github/workflows/claude-review.yml@9e2795b97b22869b9e5c7a777e71a2afa8366eeb`. The owner reported "rule updated" and shared a screenshot of the rules list; the rule detail value was not read back. TR (org ruleset 25835887) moved from M3 `2c63f49b461feac74c2c70c4f86ca90040df6a0a` to M4; read back: `repository_id` 1432342572, path `.github/workflows/claude-review.yml`, `sha` `9e2795b`, no `ref`, bypass list empty, enforcement active. Live checks in claude-review-test: PR 28 reopened, [run 40713506554](https://github.com/OperationCode/claude-review-test/actions/runs/40713506554) at M4, success, all four jobs `ok`, summary comment in the new layout (green PASS heading, TIP callout, checks strip). PR 40 reopened, [run 40713856249](https://github.com/OperationCode/claude-review-test/actions/runs/40713856249), failure, red FAIL heading, CAUTION callout, the hard block row (bidi) first, merged duplicate rows with "Found by" listing both passes. Both PRs were closed again. Console rules in the owner's screenshot: `claude-review-p3-r2`, `claude-review-prod-r1`, `eval-federation-rule`, `claude-reviewer-test`; the coordinator advised archiving `claude-review-p3-r2`. Deviation 1: no new federation rule for this release; R1 was edited in place, because the merged code names R1 (same as the phase 3 release, where R1 moved from S1 to M3). Cost: M3 can no longer get a token, so a rollback to M3 needs another Console edit. Deviation 2: the denied exchange at M4 before the Console edit (release step 5a) was NOT RUN, because the owner made the edit before the check. M4 is the current production SHA and the current pin.