Skip to content

Fix Parallel URL filtering and extract timeouts - #38

Merged
shriyatheunicorn merged 1 commit into
browserbase:shriyatheunicorn/parallel-researchfrom
georgeatparallel:george/fix-parallel-research
Oct 30, 2026
Merged

shriyatheunicorn merged 1 commit into
browserbase:shriyatheunicorn/parallel-researchfrom
georgeatparallel:george/fix-parallel-research

Conversation

@georgeatparallel

@georgeatparallel georgeatparallel commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Search and Extract could lose usable results when another URL was excluded. The auth check also matched harmless author query params. Slow live fetches need to leave time for partial results.

Solution

  • Stop matching auth inside author query params.
  • Skip blocked URLs one at a time, keep usable results and errors, and set truncated when anything gets dropped.
  • Give Extract a 20-second live-fetch timeout inside the existing 30-second client timeout. That leaves room for partial results, but it doesn't guarantee every request finishes.

Both fixes apply whenever Parallel is enabled. This targets #37's branch.

Testing

I ran three regression tests on the old code and the fix. They cover author URLs, mixed allowed and blocked results, and partial extraction. All three failed before and pass now. I didn't rerun the live Parallel/Browserbase checks.

Monitoring

I haven't verified runtime monitoring for omitted sources or Extract timeouts.

@georgeatparallel
georgeatparallel requested a review from a team as a code owner October 30, 2026 04:12
@shriyatheunicorn
shriyatheunicorn merged commit f777d37 into browserbase:shriyatheunicorn/parallel-research Oct 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants