Skip to content

feat(rivetkit-core): run sqlite synchronously with commit tickets - #5883

Draft
eersnington wants to merge 1 commit into
feat/sqlite-deferred-commitsfrom
feat/sqlite-sync
Draft

eersnington wants to merge 1 commit into
feat/sqlite-deferred-commitsfrom
feat/sqlite-sync

Conversation

@eersnington

@eersnington eersnington commented Oct 17, 2026 •

Copy link
Copy Markdown
Member
  • Adds execute_sync, exec_sync, and begin_transaction_sync to SqliteDb.
  • Adds commit_ticket and when_durable, so an embedder can hold output until the writes it depends on are durable.
  • Adds abort_generation, which restarts the actor from durable state.
  • Adds sqlite_deferred_commits to the actor config. It is off by default and is not user-facing.
  • A sync call runs only on local native SQLite with deferred commits on, and outside a runtime task. Otherwise it fails with sqlite.sync_unsupported.
  • A sync call fails with sqlite.writer_busy while a host-thread transaction is open. It waits for internal transactions, such as state saves.
  • A cold page read, or a write while the backlog is full, can still wait on depot.
  • A generation with a SQLite failure skips its final state save, so SQLite cleanup still runs.
  • New errors: sqlite.statement_failed, sqlite.sync_unsupported, sqlite.deferred_commits_unsupported, sqlite.durability_failed, sqlite.invalid_commit_ticket.

Rebased onto main.

Stack: #5882, #5883 (this PR), #5911.

@claude

claude Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Review (updated)

Overall this is a well-scoped, well-documented change. The fail-fast design (sync_unsupported, storage_behind) is sound, and the tests cover the main paths. Notes below, mostly about risk and edge cases.

Potential issues

  1. Read-miss deadline applies to every deferred-mode read, not just sync callers (depot-client/src/vfs.rs). Any page read that waits over 2 s on depot breaks the generation, including async callers and databases over 64 MiB. A transient depot stall then restarts the actor. Consider scoping it to host-thread sync reads, or document that all deferred-mode actors get this behavior.
  2. Per-actor 64 MiB eager load has no process-wide bound (load_small_database). Many actors with near-cap databases can use a lot of memory, and peak usage is roughly doubled during open because initial.pages is held alongside the cache. Consider a global budget or inserting pages per chunk. Pages with bytes: None are dropped silently, so a debug log would help.
  3. take_backlog_full() is a swap-on-read flag (backlog_error). A stale flag plus a genuine SQLITE_FULL can be misreported as storage_behind, which tells callers to retry forever. Tie the flag to the specific commit or clear it when the backlog drains.
  4. abort_generation silently ignores failures (sqlite/mod.rs, around line 1156). I rechecked the current code: if let Ok(config) = self.runtime_config() and if let Ok(db) = self.native_db_handle() still skip the abort with no signal. This conflicts with the fail-by-default rule in CLAUDE.md. Return a Result or at least tracing::error!.
  5. Ticket validation (resolved). commit_ticket_from_js already rejects NaN, infinite, negative, fractional and over-2^53 values.

Convention and parity

  • Confirm the native-only cache policy and eager load are out of scope for the WASM VFS (docs-internal/engine/sqlite-vfs.md parity rule) and note it there.
  • New public error codes: confirm pnpm docs:gen-api was run and docs/api/error-registry.json is updated. The artifacts are under rivetkit-rust/engine/artifacts/errors/, but CLAUDE.md names engine/artifacts/errors, so please verify the directory.
  • Mention the new internal env flags (RIVETKIT_SQLITE_OPT_READ_CACHE_MAX_BYTES, ..._READ_MISS_DEADLINE_MS) in the optimization-flags docs.

Concurrency

  • wait_on_runtime uses Handle::block_on guarded by tokio::task::try_id(). try_id returns None inside spawn_blocking, so that case passes the guard. Probably unreachable from NAPI, but worth a comment.
  • Add a test that a cancelled or timed-out begin_transaction_sync returns the host-thread lease counter to 0.

Tests to add

(a) storage_behind versus real SQLITE_FULL, (b) the spawn_blocking case, (c) async reads over the cache cap with respect to the deadline.

No security concerns.

🤖 Generated with Claude Code

@eersnington
eersnington force-pushed the feat/sqlite-deferred-commits branch from 7261035 to 1d1d6c8 Compare October 17, 2026 05:55
@eersnington
eersnington force-pushed the feat/sqlite-deferred-commits branch from 1d1d6c8 to 64ca955 Compare November 3, 2026 16:54
@eersnington eersnington changed the title feat(rivetkit): run sqlite synchronously with commit tickets for native database providers feat(rivetkit-core): run sqlite synchronously with commit tickets Nov 3, 2026
@RivetAgent

RivetAgent commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Changes look good: no high-confidence bugs or CLAUDE.md compliance issues found in the supplied commit range.

No Blocking, Should-fix, or Nit findings.

Review limited to local code inspection; tests were not run. GitHub authentication was unavailable, so PR status and prior comments could not be verified.

Copy prompt to fix all
Fix all actionable findings in the latest Ironworks code review on PR rivet-dev/rivet#5883. Inspect every referenced location, address each finding without changing unrelated behavior, run the relevant tests, and summarize what changed.

@eersnington
eersnington force-pushed the feat/sqlite-deferred-commits branch from 64ca955 to bc14557 Compare November 4, 2026 14:46
@eersnington
eersnington force-pushed the feat/sqlite-deferred-commits branch from bc14557 to 7a32f03 Compare November 4, 2026 15:20
@eersnington
eersnington force-pushed the feat/sqlite-deferred-commits branch from 7a32f03 to 08d0868 Compare November 8, 2026 14:29
@NathanFlurry
NathanFlurry force-pushed the feat/sqlite-deferred-commits branch from 08d0868 to 4cf6492 Compare November 12, 2026 00:57
@eersnington
eersnington force-pushed the feat/sqlite-deferred-commits branch 2 times, most recently from ae39d9b to 88c2d79 Compare November 20, 2026 19:20
@eersnington
eersnington force-pushed the feat/sqlite-sync branch 2 times, most recently from 341419b to 335d27e Compare November 20, 2026 19:22

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants