Skip to content

Block PRs that use AI without disclosing it - #5

Merged
fzipi merged 1 commit into
mainfrom
feat/ai-disclosure-blocker
Oct 27, 2026
Merged

fzipi merged 1 commit into
mainfrom
feat/ai-disclosure-blocker

Conversation

@fzipi

@fzipi fzipi commented Oct 27, 2026

Copy link
Copy Markdown
Member

what

A new pre-merge check, "AI Use Without Disclosure", runs in mode: error and flags a ❌ blocker when a PR shows AI assistance without a real disclosure. A direct signal is enough: an AI tool or agent named in the body, a commit or a comment, an AI Co-Authored-By or "Generated with" line, or an AI bot as commit author. Otherwise it needs two indirect signals, and a thin case passes.

The PR template already ships ## AI Disclosure with **AI usage for this contribution:** None / Used, so the heading alone counts for nothing. The check treats the disclosure as missing when the section is deleted, the answer is unedited or blank, "Used" is left with placeholder rows, or "None" contradicts a direct signal.

The existing "AI Contribution Disclosure" check stays a warning and now only reviews disclosure quality. It accepts the template's table as well as the bullet form; before, every PR that used the template would have been warned. README and AGENTS.md now document the new check and the extra ❌ blocker case.

why

Accounts are opening agent-generated PRs in bulk and dropping the disclosure section. kosoyyagent opened 21 PRs in two days and named Devin on its first four (coreruleset/coreruleset#4903 and #4906–#4908), then removed the section from the rest. One of those, coreruleset/coreruleset#4921, has a commit authored by devin-ai-integration[bot]. The old check only raised a warning for this case.

mode: error blocks merges only when reviews.auto_approvals (request_changes_workflow) is enabled. That setting is still false, because turning it on makes CodeRabbit approve or request changes on every PR in the org, which needs its own decision. Until then the blocker appears in the review but does not stop a merge. The check also runs one PR at a time, so it cannot use an author's disclosures on other PRs.

refs

ai disclosure

Add an "AI Use Without Disclosure" pre-merge check in error mode that
flags a blocker when a PR shows AI assistance (a named AI tool or agent,
an AI trailer or bot commit author, or two indirect signals) but the PR
template's AI Disclosure section is deleted, unedited, still placeholders,
or answers None against direct evidence.

The existing "AI Contribution Disclosure" check stays a warning and now
only reviews disclosure quality, accepting the PR template's table as well
as the bullet form.

@EsadCetiner EsadCetiner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not an expert in code rabbit, but this looks fine to me.

Do we want to auto-close PRs that don't disclose AI usage? imo those PRs are not worth our attention if AI usage isn't being disclosed.

@fzipi

fzipi commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

If they have a label already, I can autoclose them using my AI 😄 . Which is funny.

@fzipi

fzipi commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

@EsadCetiner For now I'll test how it works, and if it is accurate we can autoclose.

@fzipi
fzipi merged commit e90e479 into main Oct 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants