Skip to content

feat(search): Phase 3 text-first docs routing + honest docs - #85

Open
ManSio wants to merge 12 commits into
fix/gates-in-pre-commitfrom
feat/phase3-search-docs
Open

ManSio wants to merge 12 commits into
fix/gates-in-pre-commitfrom
feat/phase3-search-docs

Conversation

@ManSio

@ManSio ManSio commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Add a docs-only FTS channel and automatic BM25-margin routing (MSCODEBASE_DOCS_MARGIN, default 4.0); explicit intent still wins.
  • Exclude experiments/** from the index, prune stale rows on full text reindex, and add scripts/rebuild_graph.py for graph-only rebuilds.
  • Rewrite README and ARCHITECTURE/SEARCH_PIPELINE in EN/RU/ZH for the actual Phase 3 text-first runtime; mark historical deep-dive docs.
  • Align stale tests with Phase 3 defaults, bump vulnerable lockfile entries, and fix the microtask acceptance fixture.

Measurements (internal 48-query panel, clean index)

  • code quality: 0.825 hit@1 / 0.900 hit@5
  • overall quality: 0.688 / 0.750
  • auto docs route: 0.792 / 0.875 (same-basename gold)
  • fast mode: 0.542 / 0.729

These are panel results, not a universal benchmark; the README states the limits.

Verification

  • CI: all checks green — test (ubuntu-latest), test (windows-latest), clean-state, manifest-parity, docker-build, transport-equiv-e2e.
  • pip-audit -r requirements-lock.txt --no-deps → no known vulnerabilities.
  • ruff check src/ tests/ → clean.
  • Focused search tests → 28 passed; dense/cache alignment → 29 passed; auto-doc/next-step → 21 passed; microtask → 5 passed.
  • Local docs links checked: 0 missing.

Not included

  • Untracked experiments/e30..e57 run artifacts and llama_msvc.bak/ are not staged. They need a separate experiments-data PR/cleanup.
  • This PR is stacked on fix/gates-in-pre-commit to keep the review diff focused.

MSCodeBase Agent added 2 commits October 25, 2026 15:10
Use the SQLite/FTS5 chunk store as the default candidate source, keep dense/reranker layers optional, and add a docs-only channel selected by a BM25 score margin. Exclude experiments from the index, prune stale rows on full text reindex, and provide a graph-only rebuild utility. Tests cover the docs channel, query plan, index guard and chunk-store pruning.
Align EN/RU/ZH main pages with the text-first runtime, measured quality and its limits. Rewrite ARCHITECTURE and SEARCH_PIPELINE for the default SQLite/FTS5 path and mark the historical deep-dive explicitly. Correct tool counts, Python requirement and the docs routing contract; remove unverified 95% and stale LanceDB-first claims.
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5ad6f2f1-3245-4870-99d5-165043723ebc

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

MSCodeBase Agent added 10 commits October 25, 2026 21:21
P3 anchor degradation must not assume every Searcher instance has _chunk_store. The unit-test doubles exercise the degraded path directly; use getattr and keep the deterministic fallback.
Address pip-audit findings: certifi 2026.10.19, filelock 4.2.1, mcp 1.30.0. MCP stays on the 1.x transport; 2.x changes the protocol and is intentionally not adopted here. pip-audit on the lockfile now reports no known vulnerabilities.
The default runtime is text-first: dense_off and chunk augmentation are on, and on-demand rerank defaults on. Tests that exercise dense/cache paths now opt back into them explicitly instead of asserting the old defaults.
Mocked searchers return a truthy MagicMock from prefers_docs, which routed every test and every non-Searcher double into the docs channel. Require is True so only the real BM25-margin decision can switch channels.
The registered set now has 33 core classes + 16 intel + 13 inline + 4 dev = 66; execute_script is the 67th optional tool. Update the EN/RU/ZH contract and the real-project guard instead of publishing 64/65. The exact default-visible set is described as a curated allowlist rather than a stale hardcoded count.
The file imports tools/factory/ops.py, which is not tracked in this branch. Keep the PR focused on the Phase 3 search and docs changes; the factory work needs its own commit with its implementation.
The temporary project used a namespace-package src/ that can be shadowed by the installed package on CI. Add an explicit src/__init__.py so the acceptance command tests the edit, not an import accident.
The acceptance command runs inside a temporary project whose src/ may be shadowed by the installed package. Put the project directory first on PYTHONPATH so the edited file is the one under test.
A conftest in the fixture keeps the local src package ahead of the installed one on CI, so the acceptance command validates the edited file rather than an import accident.
pytest in the clean-state job resolves the installed src package before the temporary fixture. The acceptance command now imports the edited module directly from the project root, which is what the test is about.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant