Repository navigation
test(conformance): judge the Network tab in three BiDi modes, and fix the Python race it found - #440
Merged
Merged
Conversation
The suite ran each scenario with and without BiDi and served no requests in either, so an adapter that withheld the page's network rows whenever BiDi was attached passed. The mode Appium's XCUITest shim presents, a subscription that delivers nothing, was not modelled, and Safari's was modelled with nothing to list. The fake browser now loads two subresources with every document and records them as ground truth, drains them as Resource Timing rows the way the page collector reports them, and runs every scenario in three modes: live (events delivered, as Chrome), silent (subscribed, nothing delivered, as the Appium shim) and off (no socket, as Safari). Three rules hold each adapter to listing every load, listing it once, and stating a row's source where nothing but the page could have served it; the first two are separate so the declared no-BiDi gaps can cover a withheld load without excusing a doubled one. The cross-adapter agreement compares the requests listed as well, and a gap explains a difference there only where the holder's own run shows the load withheld and the other adapter's run is clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selenium delivers every BiDi event callback on a thread of its own, so a request's beforeRequestSent and responseCompleted handlers ran concurrently. When the response handler won, the completed row reached the dashboard first and the late pending row replaced it by id, leaving a served request with no status or headers. The conformance rule request-row-carries-what-the-browser-served caught it about one run in five. One lock now covers the three network handlers. A response or fetch error that arrives before its request is held, keyed by request id and redirect count so a held hop cannot close the next one, and replayed right after the request's own frame instead of being dropped. At most BIDI_EARLY_EVENT_LIMIT are held, and the teardown summary counts the ones whose request never arrived. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
[Medium impact] The PR appears safe to merge; the previous summary-labeling issue is fixed. SummaryThe PR checks Network tab rows across live, silent, and off BiDi modes and orders Python network events before sending them.
Diagram%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Response arrives] --> B{Request already seen?}
B -->|Yes| C[Send completed row]
B -->|No| D[Hold response]
D --> E{Hold limit exceeded?}
E -->|Yes| F[Evict oldest and count drop]
E -->|No| G[Wait for matching request]
G --> H[Send pending row]
H --> C
Reviews (2) · Last reviewed commit: "fix(selenium-devtools-py): count dropped..." · Reviewed by Greptile |
A response evicted past the hold limit was reported as one whose request was never seen, and if that request arrived later it was also counted as awaiting a response, so one request was reported twice and the summary blamed a missing request for what was a dropped response. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The conformance suite now checks the Network tab, and doing so exposed a race in the Python adapter's BiDi network capture. This PR carries both.
Conformance: three BiDi modes
Before this change, every scenario ran with and without BiDi but served no requests in either. An adapter that withheld the page's network rows whenever BiDi was attached therefore passed.
The fake browser now loads two subresources with every document and records them as ground truth. It reports them as Resource Timing rows, the way the page collector does. Every scenario runs in three modes:
Three new rules hold each adapter to the following:
The first two are separate rules. That way a declared no-BiDi gap can excuse a withheld load without excusing a doubled one. The cross-adapter agreement now compares the requests listed too.
Fix: Python BiDi events out of order
Selenium runs each BiDi event callback on a new thread. A request's
beforeRequestSentandresponseCompletedhandlers could therefore run concurrently. When the response won, the completed row reached the dashboard first. The late pending row then replaced it by id, and the request was left with no status or headers.With real Chrome this shows up as a served request stuck as pending in the Network tab. The new rule
request-row-carries-what-the-browser-servedcaught it in about one run in five.The fix:
A
changes/fragment records it as a patch.Testing
pnpm test: green in 6 of 6 runs.pnpm lintandpnpm buildboth pass.Not verified against a real browser. The race was reproduced and fixed against the conformance suite's fake BiDi transport, which delivers events on separate threads the same way selenium does.
🤖 Generated with Claude Code