Skip to content

Split pr_review.py's Check and Round Readers Into Leaf Modules - #3462

Merged
ptr727 merged 4 commits into
developfrom
feature/pr-review-checks-split
Oct 25, 2026
Merged

ptr727 merged 4 commits into
developfrom
feature/pr-review-checks-split

Conversation

@ptr727

@ptr727 ptr727 commented Oct 25, 2026

Copy link
Copy Markdown
Owner

Summary

scripts/pr_review.py sat at 2,999 lines against its 3,000-line ceiling, so the #3451 fix had no room. The concern isn't the number. Each fix added a few lines to one file until it needed a full refactor. This change splits the file along the two kinds of reading the gate does, rather than trimming it to fit:

  • scripts/pr_review_checks.py reads the head's CI rollup. It holds check_nodes, check_shape, the stuck and settling readers, held_checks_open, required_unmet, the check constants (CHECK_GRACE, CHECK_STALL, CHECKS_WINDOW), and the digest's check lines.
  • scripts/pr_review_rounds.py reads the review rounds. It holds coverage and the file table, refusals and quota history, request state, threads and finding matching, the local attestation and its hold, and the reviewer-login vocabulary.
  • scripts/pr_review.py keeps the GitHub calls, digest, the verdict, the writes, and main. It is now 2,105 lines.

Design

The cut follows purity. Both new modules are leaves. Each reads a payload the gate already fetched, makes no network or subprocess call, and never imports pr_review. Importing the gate from a module it imports would load a second copy whenever the gate runs as __main__. A leaf can also be tested without a mock. The rule that picks what moved: every function that does not, even indirectly, call the network, a subprocess, print, or time. liveness is the one exception, since a test spies on it through the gate.

Patched names stay in the gate. Every name a test patches (gh_graphql, gh_rest, gql, _git, digest, and the rest) stays defined in pr_review.py. A patch on a name the moved code calls through its own module would silently stop applying, and a test meant to block a real gh call would let one through. The tests now call each reader in the module that defines it. The parse-once spies count both bindings of check_nodes.

Data instead of a branch chain. digest's if/elif chain over stuck shapes is now the STUCK_LINES table beside check_shape, rendered by check_lines. A new shape is one entry in the module that defines shapes. Its output is byte-identical to the chain it replaces. I compared both versions over constructed payloads covering every shape, a superseded failure, an unreadable rollup, missing workflow keys, a truncated window, and an age-unknown stamp.

What did not move, and why. A survey of the rest found where the gate keeps growing, and each is filed as its own issue rather than mixed into a move:

These all edit pr_review.py, so they land one at a time.

Ceilings as tripwires, not targets. The size test now lists each module with modest headroom over its current length: 2,200 for the gate, 450 for checks, and 700 for rounds. Its docstring says a ceiling reached is a prompt to redesign along the module's boundaries, never to raise it. New guards keep the boundary true:

  • The whole-source write-verb, suppression, and node-id scans read all three modules.
  • A new case holds each reader's imports to the standard library and the reviewer adapter, with a floor so an empty match fails.
  • The same case asks a fresh interpreter, through import_delta, whether importing the module loads the gate, the transport, or subprocess.

The comments label. Most comment lines in the two new files moved verbatim. These are newly written:

  • each module's docstring
  • the STUCK_LINES comment
  • the check_lines docstring
  • the comments on the new test constants

This unblocks #3451, whose fix lands on top in its own pull request.

Remaining Local-Review Findings

  • pre-existing: eight sentences over 25 words, all moved verbatim or older text on a changed line: scripts/README.md lines 9 and 190, scripts/pr_review.py line 16, scripts/pr_review_checks.py lines 88 and 94, and scripts/pr_review_rounds.py lines 367, 553, and 615.
  • introduced, raised by the third pass after the edit budget ran out: tests/test_pr_review.py test_the_readers_import_nothing_that_reaches_out compares before with after and never asserts that before is empty. A watched module already loaded at interpreter start would therefore pass vacuously. Only subprocess could be, and neither CI leg loads it today. The fix is self.assertEqual([], before), which this pull request's review loop will dispose of.

Verification

  • python -m unittest discover -s tests (under uvx --with-requirements tests/requirements.txt): 2,901 tests OK, 7 skipped, on the head.
  • The digest output was compared byte for byte, old against new, over the constructed payloads above.
  • Each new guard was mutation-tested and fails when its target breaks:
    • a second check_nodes parse inside the checks module
    • import subprocess in the rounds module
    • import gh_graphql in the Copilot adapter, caught through the rounds module
  • ruff check, ruff format --check, mypy ., repo_gate.py --check eol, and prose_lint.py --diff with --allow-comments: clean.
  • local-strict-review: three recorded frontier-tier passes. The first raised five findings and the second three, all fixed. The edit budget then ran out, and anything the third raised is listed above.

🤖 Generated with Claude Code

ptr727 and others added 3 commits October 24, 2026 20:25
The gate held its own readers beside its plumbing and writes, so every
fix grew one file. The pure readers now live in two leaf modules that
take a fetched payload and make no call of their own:

- pr_review_checks.py reads the head's CI rollup, and renders the
  digest's check lines from a table keyed by stuck shape.
- pr_review_rounds.py reads the review rounds: coverage, refusals,
  quota history, threads, and the local attestation.

The code moved verbatim apart from check_lines, whose output is
byte-identical to the chain it replaces. Every name a test patches
stays defined in pr_review.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The parse-once spies counted only calls from the gate, so a second
parse inside the checks module would have passed. They now count both
bindings. The whole-source write and suppression guards read the two
reader modules too, and a new case holds their imports to the standard
library and the reviewer adapter. The size ceilings sit just above each
module's current length, so growth reaches a redesign prompt sooner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The import guard read each reader's own import statements and asserted
no floor, so a collector matching nothing passed, and the adapter's
imports went unchecked. It now asserts one known import per module and
asks a fresh interpreter, through import_delta, whether importing the
module loads the gate, the transport, or subprocess. The node-id
literal scan reads the reader modules as well, since the thread readers
moved there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ptr727 ptr727 added script A defect in hub tooling comments Permits the comment lines the pull request adds or edits, which the prose gate otherwise refuses labels Oct 25, 2026
Copilot AI lite review requested due to automatic review settings October 25, 2026 03:59
@ptr727

ptr727 commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head bf4dd0c38598f021112cd1eb3e006720709bbef2, the content this pull request carries at that commit against develop, and it recorded 9 findings.

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The import-boundary test can pass vacuously unless it asserts that the probe starts with no watched modules already loaded.

1 open issue
What changed in this PR

This pull request splits pr_review.py’s check and review-round readers into dedicated leaf modules while preserving the gate’s network, write, and command-line responsibilities.

Changes:

  • Adds pr_review_checks.py for check normalization, stuck-state evaluation, and digest check rendering.
  • Adds pr_review_rounds.py for review-round, coverage, thread, quota, request, and attestation readers.
  • Updates tests, size ceilings, import-boundary checks, import probing, and documentation.
  • Leaves one unresolved test issue: the import-boundary test does not assert that the probe starts with an empty set of watched modules.
File Summary
tests/​test_pr_review.py Retargets reader tests and adds parsing and import-boundary coverage; its import probe should reject a non-empty initial module set.
tests/​test_pr_review_size.py Adds independent size ceilings for the gate and reader modules.
tests/​import_probe.py Generalizes fresh-interpreter import probing to configurable watched modules.
scripts/​README.md Documents the separation between the gate and reader modules.
scripts/​pr_review.py Retains gate-level orchestration and delegates check and review-round processing to the new readers.
scripts/​pr_review_rounds.py Provides pure readers for review rounds, coverage, threads, quotas, requests, and attestations.
scripts/​pr_review_checks.py Provides pure check readers, state evaluation, constants, and digest check rendering.

Review effort: Lite

Comment thread tests/test_pr_review.py Resolved
@codecov

codecov Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.24242% with 3 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@636ddff). Learn more about missing BASE report.

Files with missing lines Patch % Lines
scripts/pr_review_rounds.py 98.84% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #3462   +/-   ##
==========================================
  Coverage           ?   80.90%           
==========================================
  Files              ?       25           
  Lines              ?     8850           
  Branches           ?        0           
==========================================
  Hits               ?     7160           
  Misses             ?     1690           
  Partials           ?        0           
Flag Coverage Δ
python-3.13 80.90% <99.24%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

A watched module loaded at interpreter start sat in both lists, so the
before-and-after comparison passed without testing that module. Both
lists are now asserted empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ptr727

ptr727 commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 236eceab8f02d2c3f2ee0a38d5da0abb34b920c9, the content this pull request carries at that commit against develop, and it recorded 8 findings.

@ptr727
ptr727 merged commit c75a730 into develop Oct 25, 2026
13 checks passed
@ptr727
ptr727 deleted the feature/pr-review-checks-split branch October 25, 2026 13:58
ptr727 added a commit that referenced this pull request Oct 25, 2026
## Summary

A pull request into a non-default branch is "held": an attested local
pass covers its head, and `pr_review.py wait` polls the head's checks
instead of a Copilot round. Run right after a push, that poll exited `0`
while CI was still running. A job behind `needs:` enters the rollup only
once its dependencies finish, so the merge read `CLEAN` and every
registered check was green while most of a run's jobs had not been
created yet. A caller trusting the exit judged the pull request ready.

The held poll now holds while any workflow run on the head has not
completed:

- `runs_open(pr)` in `scripts/pr_review_checks.py` names every workflow
run whose suite is not `COMPLETED`. It reads the head commit's own
suites and each rollup check run's suite, so a fresh head's run counts
before most of its jobs exist. A suite with no workflow run, such as
another app's that never reports, holds nothing.
- `held_checks_open` reads its conditions in this order:
  - a pull request no longer open releases
  - a merge reading `UNKNOWN` holds
- a `BLOCKED` merge with a required check stuck in a shape waiting does
not clear ends the poll at `44`
  - an open run holds
  - only then does a `CLEAN`, `UNSTABLE`, or `HAS_HOOKS` merge release
- A check run `WAITING` past `--check-grace` is its own `GATED` shape.
It gets a `CHECK WAITING ON ITS ENVIRONMENT` line saying to approve the
job or let its timer pass, since a re-run meets the same rule. `WAITING`
leaves `NOT_STARTED`.
- Where `workflowRun` is unreadable, an Actions suite stands in, named
`suite <id>`, through `unkeyed_actions_suite`, which `check_nodes`
shares. The head's suites query now selects `app{ slug }` for this.
- The `status=CHECKS_PENDING` line names the open runs. The `wait` help
and the `scripts/README.md` held-wait and exit-44 paragraphs describe
the new reading, in the code's order.

Closes on promotion: #3451

## Decisions

- **Every open run holds, with no exception.** Earlier revisions
released a run waiting on an environment, then one with a starved job.
Review showed each release reopened the early exit `0`: a job pending
behind a concurrency group, a busy runner pool, or a dependent not yet
registered. The cost is that a run waiting on an approval that never
comes, or on a job no runner takes, polls to the timeout. It then exits
`30`, with the digest's `GATED` or `CHECK NOT PICKED UP` line naming
what holds it.
- **A gated or long-running required check does not end a `BLOCKED` poll
at `44`.** A timer clears by itself and an approval may land, so on a
held head these hold to `30`. On the Copilot path a required gated check
still exits `44`, as a long-running one already did.
- **A required starved check under `BLOCKED` still exits `44` at once,**
as before.
- **No shared walker with `check_nodes`.** Its suite walk is threaded
through per-node normalization, so `runs_open` reads the suites itself.
The one condition the two share, whether a suite is an Actions suite
left unkeyed, is one predicate.
- **A job-less Actions suite may hold in degraded mode.** With
`workflowRun` unreadable, an unfinished Actions suite holds even when it
has no job yet, since right after a push it looks the same as one that
never gets a job. I measured the check suites on the 8 most recent pull
request heads here: every Actions suite carried jobs, and none sat
unfinished. The suites left queued with no jobs belonged to other apps
(`azure-pipelines`, `claude`, `coderabbitai`), which the predicate
excludes and a test pins.

## Remaining Local-Review Findings

Each push had recorded frontier-tier passes, and every `introduced`
finding they raised is fixed in the commits above, apart from these:

- `introduced`, fixed in `8acf0511`: the `wait` help's exit-30 line read
as a complete list of holds. It now points to the holds
`status=CHECKS_PENDING` names.
- `introduced`, deferred to #3471: the exit-0 help's completed-runs
clause now names coverage, the condition the code reads (`3545ba46`).
One path still falls outside it: when the rollup's commit is not the
head, `wait` deliberately exits `0` with `CHECKS UNREADABLE`, and no run
is read. That predates this change and a test pins it, but the help's
new clause now overclaims there. It joins #3471, which tracks exit `0`
where the required checks were never read.
- `pre-existing`: the help's claim that exit `0` means the head's
required checks settled is false under a `DIRTY` or `BEHIND` merge whose
required checks never ran. GitHub runs no `pull_request` workflow on a
conflicted pull request, and `held_checks_open` holds an unregistered
required context only under `BLOCKED`. These paths predate this change,
so they are left to an issue of their own.
- `introduced`, conditional: the degraded-mode decision above. If GitHub
ever leaves a job-less Actions suite queued on a head, the held wait
would poll to `30` in that mode alone. The measurement above found none.
- `style`: one help string and two help-text lines are wrapped short.
Declined, since no gate reads them and the file already carries longer
lines.
- `pre-existing`: a head read before any check suite exists, on a base
requiring no check, can still exit `0` under `CLEAN`. Nothing on the
head exists yet for any reader to see. Every develop ruleset in this
fleet requires the aggregator, so there the merge reads `BLOCKED` and
`required_unmet` holds.
- `pre-existing`: sentences over 25 words that this change only
rewrapped or left in place: three at `scripts/README.md` line 233,
`scripts/pr_review.py` lines 120 and 123, and the `49 =` help sentence.

## Verification

- `python -m unittest discover -s tests` (under `uvx --with-requirements
tests/requirements.txt`): 2,940 tests OK, 7 skipped, at `e4295145`. The
commits after it change only the `wait` help text, and the
`test_pr_review` and `test_cli_usage` suites pass on each.
- Each rule was mutation-tested, and dropping it fails a named test:
  - the run hold, and its place ahead of `CLEAN`
  - the closed-PR release
  - the stuck-required release ahead of the hold
  - the gated exemption from `44`
  - the `COMPLETED` skip
  - the no-workflow-run exclusion
  - the rollup-suite read
- the Actions fallback for a rollup suite, for a job-less head suite,
and its other-app exclusion
  - the `GATED` shape
- On a live pull request, #3462, a wait using an earlier revision of
this change (`f6837bd9`) held for 781 seconds until every run on the
head completed, then exited `0`. The same wait without it had returned
while two checks were still running.
- The extended suites query was read live with `status` against this
repository.
- `ruff check`, `ruff format --check`, `mypy .`, and `prose_lint.py
--diff`: clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ptr727 added a commit that referenced this pull request Oct 25, 2026
…nd Nine Fixes to Main (#3472)

## Summary

Promotes the pull requests merged into develop since release 2.0.1166.

- [#3444](#3444): keeps
untracked files out of the pre-commit snippet's prose gate.
- [#3446](#3446): exempts
the BuildKit `check=` parser directive in the prose gate.
- [#3448](#3448): compares
intent staleness against the hub's own copy and names the moved
canonical.
- [#3450](#3450): says
only the prose gate is diff-scoped in the commit hook.
- [#3453](#3453):
reclassifies HolidayLights as an operational config repository in the
registry.
- [#3455](#3455):
classifies the remaining subprocess failures by status rather than by
prose.
- [#3464](#3464): splits
the semicolon series in WORKFLOW.md's filesystem output-seam bullet.
- [#3466](#3466): audits a
caller's `workflow-model` input against the registry.
- [#3469](#3469): emits
one snapshot shape from the skills report's cannot-read paths.
- [#3462](#3462): splits
`pr_review.py`'s pure check and round readers into the leaf modules
`pr_review_checks.py` and `pr_review_rounds.py`.
- [#3470](#3470): holds a
covered held `wait` while any workflow run on the head has not
completed.
- [#3473](#3473): answers
this promotion's review. The `wait` help and `scripts/README.md` now say
what a held head's exit `0` does not prove: a `CHECKS UNREADABLE`
payload, the conditions the unregistered-required-check hold reads, and
the reach of the completed-run read. The exit code itself stays for
#3471.

Closes #3370
Closes #3372
Closes #3371
Closes #3369
Closes #3378
Closes #3454
Closes #3332
Closes #1759
Closes #3451

## Constituent coverage

Each change this promotion carries had coverage on its own pull request
into develop, at its merged head:

| Pull request | Merged head | Coverage |
| --- | --- | --- |
| [#3444](#3444) |
`a61d1e2a` | attested local pass at the merged head, base `develop` |
| [#3446](#3446) |
`0a0efea1` | attested local pass at the merged head, base `develop` |
| [#3448](#3448) |
`c2c71ea7` | attested local pass at the merged head, base `develop` |
| [#3450](#3450) |
`4ec37c34` | attested local pass at the merged head, base `develop` |
| [#3453](#3453) |
`f20a543c` | a Copilot round on the merged head, its one changed file,
no open findings |
| [#3455](#3455) |
`85f0ce60` | attested local pass at the merged head, base `develop` |
| [#3464](#3464) |
`fcd48e2d` | attested local pass at the merged head, base `develop` |
| [#3466](#3466) |
`3426a19c` | attested local pass at the merged head, base `develop` |
| [#3469](#3469) |
`ea4b0bb7` | attested local pass at the merged head, base `develop` |
| [#3462](#3462) |
`236eceab` | attested local pass at the merged head, base `develop` |
| [#3470](#3470) |
`3545ba46` | attested local pass at the merged head, base `develop` |
| [#3473](#3473) |
`46010530` | attested local pass at the merged head, base `develop`, and
a Copilot table on an earlier head naming exactly the changed files |

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comments Permits the comment lines the pull request adds or edits, which the prose gate otherwise refuses script A defect in hub tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants