Skip to content

Phase 4: pilot fixes and records (combines #15-#19) - #20

Merged
irvingpop merged 14 commits into
mainfrom
irvingpop/phase4-combined
Oct 29, 2026
Merged

irvingpop merged 14 commits into
mainfrom
irvingpop/phase4-combined

Conversation

@irvingpop

Copy link
Copy Markdown
Member

One PR for all phase 4 repo work so far. It merges five reviewed branches with signed --no-ff merges; the only conflicts were decision-log appends in the two docs (both sides kept). No code conflicts.

  1. Eval artifact + comment fix (was Phase 4: upload fixture eval results; show the never-approves line once #15): eval.yml uploads fixture eval results (always, 14 days, existing upload-artifact pin; eval egress allowlist gains *.blob.core.windows.net:443, documented). The pass comment shows "This bot never approves a PR" once.
  2. Continuation-comments plan (was Phase 4: plan for bounded continuation comments (planned, not built) #16, plan only): bounded continuation comments for summary-only findings; per-run part ownership; fail-closed kept. Open minors: Q1 decision wording says the owner accepted, but the coordinator ruled no cross-run reuse; a failed final-check read leaves publication "complete"; stale snapshot during cleanup undefined.
  3. Pilot record (was Phase 4: record the pilot start and the OIDC subject prerequisite #17): four-repo pilot, ruleset ids, approval protection, CODEOWNERS, V25, 2FA, decisions P4-1..P4-6; new prerequisite: use_immutable_subject per target repo (older repos send the legacy OIDC sub, which the federation rule rejects); list open PRs right after enabling the ruleset.
  4. pnpm 12 lockfile parser (was Parse pnpm 12 two-document pnpm-lock.yaml (pilot false positive) #18, K-16): two-document pnpm-lock.yaml parses with the same safe unique-key loader; strict shape; packages from both docs get every check; fail closed otherwise. Fixes the hard block on every front-end lockfile PR.
  5. Anchored diff fallback (was Anchored diff fallback so lockfile-heavy PRs get model review (pilot false skip) #19, K-17): lockfile-heavy PRs no longer skip model review as "too large" (front-end#2027: about 451k -> 94k tokens). No path-based special case.

Each part was reviewed before this merge (items 4 and 5 by an adversarial reviewer from another model family, each with a fix round or clean pass, then a scoped re-review). Unit tests 2678 passed; release tests 2 passed. Eval not run.

Items 1, 4, 5 change reviewer behavior and go live only with a release (release evals, Console job_workflow_ref, both rulesets' sha).

🤖 Generated with Claude Code

irvingpop and others added 14 commits October 28, 2026 08:35
…ine once

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add docs/continuation-comments-plan.md (planned, not built): current
fail-closed behavior with file:line citations, limits (6 parts of 64,000
characters, severity first, fail closed past the bound), idempotent slot
reuse, security, gate interaction, tests, rollout and open questions.
Point build-plan phase 4 at it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per-run part ownership: a run POSTs its own parts and retires another
run's parts only when that run attempt is done and no bot sticky links
them, so concurrent runs cannot remove a linked set. Add final-check
verification, a Publication result kept through the final update,
defined read/write outcomes, cleanup caps, a time budget, a total
per-finding order, the over-limit pass case, the first-run backlink
and interleaving tests. Move the open questions to Decisions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uisite (P4-1 to P4-6)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…xt; add open-PR timing lesson (P4 fix round 1)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
pnpm 12 writes the env lockfile (pnpm itself) as a first YAML document
before the main lockfile. The single-document load raised ComposerError,
so every front-end PR that changed the lockfile got the false hard block
"incomplete dependency data".

Read every document with the same unique-key safe loader. Accept one
document, or exactly two with an env document of the known shape. Each
document needs a supported lockfileVersion, and every env dependency
needs a packages entry. Packages from both documents get every check.
Anything else stays a parse failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of 01d3cb3 found that an env package entry could take another
package's identity through its own name and version, so pnpm could pass
the env check and then skip every check. The env check now refuses an
entry whose name or version differs from its key, and checks each env
dependency against the packages the walker emits.

The main document of a two-document file had no shape check. It may now
hold only the top-level keys of pnpm 12's main lockfile. The
one-document form keeps its old check.

The shared YAML loader expands merge keys before the duplicate-key
check, so a merged key that repeats another key is a parse failure.

Tests cover both scenarios, cross-document aliases, unsafe tags in each
document, merge-key collisions, end markers, a BOM, a document marker in
a block scalar, and a malformed two-document base in collect (K-10).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ate (K-17)

PR 2027 in OperationCode/front-end (a 16k-line pnpm-lock.yaml changed near
both ends) got "too large for automatic review": its middle was past
DIFF_BUDGET, so the patch was one hunk of every base and head line and the
bundle estimated 451,544 tokens against the 150,000 limit.

In collect only, a middle past the size budget is now split at lines that
occur once on each side (patience anchors), and each gap gets the bounded
diff with the same file and run work budgets. The patch stays a correct
edit script, reads no path, and a full rewrite still has no anchors, so the
size gate stays fail-closed. PR 2027 now estimates 93,786 tokens (115,961
with the K-16 parser and full registry data). The publisher is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…op/phase4-combined

# Conflicts:
#	docs/build-plan.md
…op/phase4-combined

# Conflicts:
#	docs/claude-review-design.md
…op/phase4-combined

# Conflicts:
#	docs/claude-review-design.md
@github-actions

Copy link
Copy Markdown

🔴 FAIL: OC Claude review did not complete

Caution

Incomplete file coverage blocks this PR. No waiver can clear it.
Next step for the author: fix the cause named in the row, then push. The review runs again on the new commit.

Severity Effect Where Finding Found by
🚫 Hard block Blocks, no waiver review/tests/unit/test_collect.py Incomplete file coverage: file content not scanned: unterminated-private-key, in review/tests/unit/test_collect.py File checks (code, no model)
⚠️ Not run Blocks, no waiver Gate A required job did not succeed: dependency-review: failure Workflow (code, no model)
⚠️ Not run Blocks, no waiver Gate Dependency review did not succeed: job result failure Workflow (code, no model)
⚠️ Not run Blocks, no waiver Gate Dependency review did not succeed: step outcome failure Workflow (code, no model)
⚠️ Not run Blocks, waivable Model passes Too large for automatic review. File checks ran, model checks did not. Workflow (code, no model)
File checks Review Security Supply chain Dependency review Coverage
🚫 1 hard block, 5 signals ⚠️ skipped, too large ⚠️ skipped, too large ⚠️ skipped, too large ❌ failed 🚫 incomplete

Run 40924927063, attempt 1 for head 4a82819 into main. The required workflow in the merge box is the result that counts. This bot never approves a PR.

📋 Full review record: reasons, checks, signals, finding details, model summaries, token usage

Reviewed snapshot

Head SHA Merge base SHA Base branch Run
4a8281911d5c279c57e3e81eb6b15b4072545498 03e1b8d09c322bf215d1699da0e793f0064f2974 main 40924927063, attempt 1

Reasons the gate failed

Reason Waivable? Detail
A required job did not succeed 🚫 no dependency-review: failure
Dependency review did not succeed 🚫 no job result failure
Dependency review did not succeed 🚫 no step outcome failure
Change too large for the model review ✅ yes too large for automatic review
Hard block from the file checks 🚫 no incomplete-coverage in review/tests/unit/test_collect.py
Hard block from the file checks 🚫 no incomplete-coverage (coverage record is not complete)

No waiver line is shown. At least one reason cannot be waived, so a waiver would not change the result.

Checks (code, no model)

Check Result Detail
Hard block 🚫 blocks Incomplete file coverage: file content not scanned: unterminated-private-key, in review/tests/unit/test_collect.py
Coverage 🚫 NOT complete files in comparison 14, files collected 14, trees complete
Unavailable file ⚠️ unavailable review/tests/unit/test_collect.py: unterminated-private-key
Dependency changes ✅ none
Dependency review job ❌ failed

Signals (context only, never block)

  • ⚠️ Sensitive path changed: change to workflow, in .github/workflows/eval.yml
  • ℹ️ Very long line: lines longer than 500 characters, in docs/build-plan.md, lines 189 193 194 202
  • ℹ️ Very long line: lines longer than 500 characters, in docs/claude-review-design.md, lines 197 200 242 280 564 576 811 991 992 997 and more
  • ℹ️ Very long line: lines longer than 500 characters, in docs/continuation-comments-plan.md, lines 22 26 28 30 36 157 159 246 299 306 and more
  • ℹ️ Hidden zero-width character: zero-width characters U+FEFF, in review/tests/unit/test_deps.py, lines 496

Dependency changes

None.

Finding details

Model output from untrusted input. Check it before you act on it. Verified means only that the quoted text is at that place. It does not mean the claim is true.

Blocking is decided by code from category and severity, never by the model. A finding in a blocking category blocks only at high or critical severity.

No findings from the passes that ran. The Review, Security, Supply chain pass did not run, so this is not a clean result.

Model summaries

Model output from untrusted input. Check it before you act on it. Verified means only that the quoted text is at that place. It does not mean the claim is true.

Pass Status Model Summary
Review ❌ skipped-size No summary. Reason: bundle status skipped-size
Security ❌ skipped-size No summary. Reason: bundle status skipped-size
Supply chain ❌ skipped-size No summary. Reason: bundle status skipped-size

Token usage

Pass Input Output Cache write Cache read Attempts
Review not recorded not recorded not recorded not recorded not recorded
Security not recorded not recorded not recorded not recorded not recorded
Supply chain not recorded not recorded not recorded not recorded not recorded
Recorded total not recorded not recorded not recorded not recorded not recorded

@irvingpop
irvingpop merged commit cc710fb into main Oct 29, 2026
3 of 5 checks passed
@irvingpop
irvingpop deleted the irvingpop/phase4-combined branch October 29, 2026 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant