You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reduce memory-maintenance latency with checkpoint-aware short-job priority - #3241
Reuse evidence-scoped Wiki deferrals across unrelated changes and write assessment/family receipts in one atomic history commit.
Reuse versioned, content-addressed history payloads instead of repeatedly serializing large unchanged topic/build inputs.
Add work-conserving short-build priority, bounded native fairness, and cancellation-safe native yields at committed, drained checkpoints. Preserve per-corpus ordering and shutdown ownership.
Share maintenance model admission across default models, the shipped settings factory and derived-view extraction; support guarded, ordered Wiki assessment preparation.
Use strict source-synopsis output schemas and independently observe required knowledge checkpoints without weakening readiness or resetting semantic failures.
Reviewer notes
Build/model concurrency defaults stay 1/24. Short priority is enabled by default, with an explicit FIFO opt-out. Fixed reservation remains opt-in; enabling both policies fails. Wiki preparation defaults to one lane.
Legacy history stays readable and migrates on a later write. Do not downgrade a migrated store. Payloads remain under the existing atomic Git ref and privacy-purge boundary; corruption is explicit.
Synopsis recipe advances to v7; schema-unsupported providers fail rather than falling back or truncating.
This is a scoped port onto current main, excluding unrelated local native/replay experiments. Existing whole-curation budgets, understanding options and committed-reader behavior remain intact.
Validation
All 29 applicable CI checks pass on 9fd0bd0e2; the fork-only formatting check is intentionally skipped.
Full workspace build; 235 focused maintenance/Wiki/native/cancellation regressions, 15 browser client regressions, and paired large-history checks.
193 CI memory-quality tests, including 50 checkpoint-observer/runner tests; all four PR code-quality ratchets and pinned formatting.
Five paired history comparisons at each of 27/52 MiB: over 99.99% fewer serialized bytes, 69-71% lower median read-update-commit time, matching state and improved p95.
Synthetic measurements are not production throughput or semantic-quality qualification.
Reuse scoped Wiki admission receipts and immutable history payloads, add guarded parallel preparation, share maintenance model admission, and prioritize short builds with durable native checkpoint yields. Observe required knowledge checkpoints independently without changing readiness requirements.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The first CI failure was the repository-wide package-metadata ordering gate in the existing memory MCP package. I reproduced and validated the metadata-only fix locally (all 12,096 policy checks pass). The formatter bot published the identical fix first in 90eec64, so I integrated that commit rather than pushing a duplicate. No dependency versions or runtime behavior changed.
Addressed the CI complexity failures in target deadline selection and history publication without changing behavior. Also bounded the new 97-subject paging fixture to descriptor discovery/deferrals, avoiding unnecessary article generation and the full-suite timeout. All 147 tests in the six affected suites, the full workspace build, and the actual committed-file PR ratchets pass locally.
This PR reduces latency in the memory service by reusing unchanged history payloads and Wiki assessments, and by allowing shorter builds to run at safe native checkpoints.
Changes:
Adds short-build priority, shared model admission, configurable concurrency, and checkpoint-safe yielding.
Introduces content-addressed history payloads, atomic Wiki receipts, and strict synopsis schemas.
Adds independent checkpoint observation, retained-corpus preflight checks, and regression coverage.
Address review findings in durable candidate progress, bounded preflight and snapshot publication, model cancellation and configured extraction, callback continuation admission, strict synopsis providers, and preparation ownership. Preserve final authoritative checkpoint refresh and benchmark attempt identity.
Also fix atomic retry admission and align the browser test fixture with shipped stylesheet and capabilities imports. Validate with 193 focused product tests, 192 memory-quality tests, the browser regression, full workspace build and fresh 105-run scheduling qualification.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Final source commit: 9fd0bd0. All 29 applicable current-head CI checks pass; the fork-only formatting check is intentionally skipped. This includes all nine Node/OS TypeScript combinations, all three shell packages, both .NET builds, CodeQL, policy and the Linux/Windows Azure smoke tests.
All 23 findings across three review rounds are addressed: 18 inline threads resolved, plus five overview-only findings explicitly answered. No review requests or unresolved threads remain. The PR is mergeable, targets main and remains unmerged; branch policy still requires human approval.
Local validation: full workspace build; 235 distinct affected product regressions across ten suites, including the final 49-case synopsis suite; 193 CI memory-quality tests; 15 browser tests; all four genuine PR ratchets; pinned formatting; 12,096 policy checks.
Fresh 105-run scheduling qualification passes: late-short median queue 92.9% lower, native-only throughput 0.3% lower, mixed completion 10.6% longer, identical outputs/calls/checkpoints. Paired history median time is 69.3%/70.6% lower at 27/52 MiB, with over 99.99% fewer serialized bytes, equal state and improved p95. The history test also passed against the full-history object database fetched from a fresh shallow clone. All 18 recorded identities match the clean published commit and the frozen original patch is unchanged. Synthetic results are not production throughput or semantic-quality qualification.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
A failed observer write can prevent service shutdown and leave maintenance running, alongside unresolved cursor, cancellation, and regression-test issues.
The responses wire API also ignores request.jsonSchema in ResponsesWireApiProvider.buildRequestBody, so these endpoints still perform synopsis generation without the required strict schema. Reject that wire API too until its adapter sends and enforces the schema.
A short-line document can produce hundreds of 64-character evidence IDs even when the request fits the model context. Putting all of them in one enum can exceed structured-output provider enum limits, so valid source inputs fail at inference instead of being sectioned. Include enum count and string-size limits in selection/composition packing while retaining strict ID enforcement.
Preserve global FIFO ordering when priority is disabled
The FIFO opt-out no longer preserves the previous global ordering: with one slot, enqueue A1 and A2 for corpus A, then B1 for corpus B. A2 stays behind A1's corpus tail while B1 enters the scheduler, so B1 runs before the earlier A2 even when priority is disabled. Preserve legacy FIFO ordering in the opt-out mode while retaining per-corpus exclusion.
Make regression baseline available in shallow checkouts
The scheduled memory-regression workflow uses a shallow checkout, so this fixed historical commit is unavailable and the new regression cases fail at git show on all three platforms. Make the baseline available independently of local history, or have that workflow fetch the required history.
Classify observer failures separately from cancellation
The combined signal also aborts when the observer's status read or publication fails, so those failures are always recorded as an interrupted wait even when the caller did not cancel. Classify interruption using the external control signal; an observer failure should remain a failure rather than look like a bounded cancellation.
Track visited candidate identities across ranking, discovery and publication changes, and commit selected visits only with build admission. Bind speculative preparation to target deadlines while retaining joined cleanup. Always close benchmark services and restore observers after publication failure.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The shallow regression checkout lacks the historical baseline, causing both new tests to fail, while schema, scan, scheduling, and cache-invalidation defects remain unresolved.
The memory regression workflow uses the default shallow checkout, so this fixed historical commit is not present when test:regression runs. Both new regression cases fail at git show on every scheduled/manual run instead of exercising the comparison. Fetch the baseline commit in that workflow or use a checked-in baseline fixture so the suite works with a shallow checkout.
Reject responses API when strict schema enforcement is unavailable
The responses wire API also ignores request.jsonSchema and emits no text.format (see ResponsesWireApiProvider.buildRequestBody). A synopsis configured for that API therefore runs without the required strict schema instead of failing, just like the rejected messages path. Reject responses here as well until its adapter enforces the schema.
Bound selection and composition by provider enum-size limits
Each evidence ID is a 64-character hash, so 251 IDs exceed OpenAI's 15,000-character limit for string enums with more than 250 members. selectionFits checks only token capacity, allowing short-line sources or composition batches to reach the provider with an invalid schema and fail instead of being split. Include this enum-size bound in selection and composition packing while retaining exact-ID validation.
Preserve FIFO ordering for same-corpus jobs with priority disabled
Deferring enqueue until the previous same-corpus job settles breaks the documented FIFO opt-out even at concurrency 1. Submitting A1, A2 (same corpus), then B1 runs A1, B1, A2: releasing A1 admits the already-queued B1 before A2 reaches the limiter. Preserve submission order among runnable jobs when priority is disabled, while still preventing simultaneous same-corpus execution.
Fingerprint effective model configuration in assessment policy
The default adapter identity records only the endpoint name, so changing the configured model behind that name leaves this dependency key unchanged. Scoped deferrals can now survive that model change even after an index-generation change, which previously invalidated them. Include a fingerprint of the effective, non-secret model configuration in the assessment policy.
Revisit completed deferrals when nominated canonical context changes without rereading unaffected candidates. Preserve corpus ownership across native yields while enqueuing FIFO tickets at submission. Fingerprint effective non-secret model settings, reject schema-unsupported synopsis adapters, pack within provider enum limits, and fetch history for regression comparisons.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Addressed the five additional findings in the last review overview in 9fd0bd0:
Regression baseline: the scheduled/manual memory regression workflow now fetches full history. A fresh depth-one local clone reproduced the missing baseline; fetching full history made the exact pinned commit available. The paired regression will also be rerun using that clone's Git object database.
Schema-unsupported adapters: synopsis configuration now explicitly rejects Responses as well as Messages, native Copilot and native Ollama adapters before inference. OpenAI-compatible adapters still use the schema-capable request path. Provider tests cover each rejected path.
Evidence enum limits: selection and composition packing cap distinct evidence choices at 250; direct schema construction also rejects overflow. Boundary, duplicate-ID and multi-stage composition tests prove requests stay within the limit without dropping exact-ID validation.
FIFO/corpus ordering: jobs now enter admission at submission, with synchronous corpus ownership determining eligibility. Blocked same-corpus work consumes no execution slot; native checkpoint yields retain corpus ownership. Tests cover capacities one/two, checkpoint resumption and actual ViewBuildRunner A1/A2/B1 ordering.
Effective model identity: default synthesis identity fingerprints non-secret settings for every configured pool member, including endpoint/model/wire API/request constraints, rather than only the endpoint alias. A configuration-change regression verifies alias remapping invalidates identity while credential-only rotation does not.
The full workspace build, ten-suite 233-test run, final 49-case synopsis suite, 193 CI memory-quality tests and policy checks pass locally. Fresh scheduling/history qualification and the new-head CI matrix are being verified. The one new inline thread is also fixed, replied to and resolved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reviewer notes
Validation
9fd0bd0e2; the fork-only formatting check is intentionally skipped.Synthetic measurements are not production throughput or semantic-quality qualification.