Skip to content

Add flakewhy to Tools - #148

Open
hamarshehmhmd wants to merge 1 commit into
cleder:mainfrom
hamarshehmhmd:add-flakewhy
Open

hamarshehmhmd wants to merge 1 commit into
cleder:mainfrom
hamarshehmhmd:add-flakewhy

Conversation

@hamarshehmhmd

@hamarshehmhmd hamarshehmhmd commented Nov 2, 2026 •

Copy link
Copy Markdown

Adds flakewhy to Tools, in alphabetical order.

It runs a pytest suite under repeat / shuffle / TZ / PYTHONHASHSEED variations, confirms the cause of each flaky test with re-runs, bisects order-dependent failures down to the polluting test, and prints a one-line repro. Zero dependencies, pytest 7+, AGPL-3.0.

I'm the author. rumdl check README.md passes locally.

Summary by Sourcery

Add flakewhy to the README’s collection of Python testing tools.

New Features:

  • Add flakewhy to the testing tools directory, highlighting its support for diagnosing flaky and order-dependent pytest failures.

Documentation:

  • Document flakewhy in README.md in alphabetical order.

@semanticdiff-com

Copy link
Copy Markdown

Review changes with  SemanticDiff

@sourcery-ai

sourcery-ai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

The README now catalogs flakewhy as a zero-dependency pytest 7+ tool for reproducing, confirming, and diagnosing flaky tests across execution-order, environment, timezone, and hash-seed variations.

Flow diagram for flakewhy flaky-test diagnosis

flowchart LR
    Suite[Pytest suite] --> Variations[Repeat, shuffle, timezone and hash-seed variations]
    Variations --> Confirm[Confirm flaky tests with reruns]
    Confirm --> Diagnose[Bisect order-dependent failures]
    Diagnose --> Repro[One-line reproduction]
Loading

File-Level Changes

Change Details Files
Adds flakewhy to the testing-tools catalog in alphabetical order.
  • Adds a README entry linking to the project repository.
  • Summarizes its pytest flakiness diagnosis, variation testing, confirmation, order-dependency bisection, and repro output capabilities.
README.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@qodo-code-review

qodo-code-review Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

Grey Divider

Sorry, something went wrong

We weren't able to complete the code review on our side. Please try again manually by commenting /agentic_review on this PR.

Grey Divider

Qodo Logo

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Approved.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Add flakewhy to the Python testing tools list

📝 Documentation 🕐 Less than 5 minutes

Grey Divider

AI Description

• List flakewhy in the README’s Tools section in alphabetical order.
• Describe its flaky-test diagnosis, order-dependency bisection, and reproducible failure commands.
High-Level Assessment

A concise entry in the existing alphabetical Tools list is the appropriate approach; no architectural alternative is warranted.

Files changed (1) +1 / -0

Documentation (1) +1 / -0
README.mdList flakewhy among Python testing tools +1/-0

List flakewhy among Python testing tools

• Adds a link to flakewhy between csv-quality-gate and ghostwriter. The entry describes how it diagnoses flaky pytest runs and produces a reproduction command.

README.md

@llamapreview llamapreview Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LlamaPReview — No blocking issues found

The entry is alphabetized and describes the capabilities claimed in the PR; no PR-caused defect is established, so it is clear to merge.

Review details and evidence

LlamaPReview checks

  • Reviewed changed regions in README.md.

Still coding alone? Open CarbonChat, a chat room right inside Codex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant