Skip to content

Reduce duplicate R2 cache refresh writes - #1448

Closed
tannerlinsley wants to merge 19 commits into
mainfrom
taren/reduce-r2-cache-writes
Closed

tannerlinsley wants to merge 19 commits into
mainfrom
taren/reduce-r2-cache-writes

Conversation

@tannerlinsley

@tannerlinsley tannerlinsley commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Delivered to main through #1446, commit 8bd5925377facea34de26ce268c41f22f0ee3a97. The retained R2 implementation is unchanged in that combined release.

Expired edge entries could fetch GitHub content and rewrite R2 even when another location had already refreshed the shared object. Refreshes now check R2 before fetching content or rebuilding artifacts.

Publishing and invalidation use conditional writes and content revisions so an older refresh cannot overwrite a push invalidation. Whole-ref invalidation covers inherited docs and raw cache paths. Existing freshness windows, outage fallback, and repair of corrupt entries remain covered by regression tests.

Adds sampled operation counts by cache action and repository, plus a read-only script for comparing R2 operations over matching time windows.

Validation: type checks, lint, 661 unit tests, 2,270 chat tests, 20 desktop tests, and the production build passed. The exact Cloudflare preview serves the homepage, Query/Router/Start docs, Markdown, and the admin shell, and rejects unsigned webhooks and unauthenticated chat requests. GitHub's PostgreSQL runtime checks also passed.

@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Walkthrough

The PR adds ETag-based cache revalidation and conditional writes for GitHub content and documentation artifacts. It also measures blob-storage operations across cache and invalidation paths, adds cache race and edge-refresh tests, and provides a script to query Cloudflare R2 operation analytics.

Changes: Blob Cache Reliability

Layer / File(s) Summary
Cache reads and edge revalidation
src/utils/github-content-cache.server.ts, tests/github-cache-edge-refresh.test.ts, tests/github-cache-corruption.test.ts, tests/github-content-cache.test.ts
Cache reads can revalidate against R2, preserve ETags, and return placeholders for malformed stored values. Tests cover edge-cache reuse, stale refreshes, failed reads, and corrupted entries.
Conditional refresh and invalidation
src/utils/github-content-cache.server.ts, tests/github-cache-invalidation-operations.test.ts, tests/github-cache-r2-runtime.test.ts, tests/github-cache-refresh-race.test.ts, tests/github-cache-test-utils.ts
Refreshes and invalidations use ETag conditions. Tests cover invalidation scope, concurrent refreshes, and preservation of stale markers during races.

Blob Storage Operation Metrics

Layer / File(s) Summary
Operation recording and storage wrapper
src/server/runtime/blob-storage-metrics.server.ts, src/server/runtime/blob-storage.server.ts, tests/blob-storage-metrics.test.ts
Measurement contexts count named blob-storage operations. Nested measurements share counts, and the test checks that counts are logged when an inner operation fails.
Cache and invalidation measurement
src/utils/github-content-cache.server.ts, src/utils/npm-download-cache.server.ts, src/routes/api/github/webhook.ts, src/utils/docs-admin.server.ts
Cache reads, writes, pruning, and invalidation calls run within measurement contexts. Webhook and admin stale-marking calls are also measured.
Cloudflare R2 operation report
scripts/measure-r2-operations.mjs
The script queries Cloudflare analytics for a supplied or default time window, filters to three buckets, and prints formatted JSON.

Priority: Normal

Estimated code review effort: 4 (Complex) | ~50 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant CacheCaller
  participant GithubContentCache
  participant EdgeCache
  participant R2Bucket
  participant ContentOrigin
  CacheCaller->>GithubContentCache: Request cached content
  GithubContentCache->>EdgeCache: Read cached entry
  GithubContentCache->>R2Bucket: Revalidate stored entry
  GithubContentCache->>ContentOrigin: Fetch refreshed content
  GithubContentCache->>R2Bucket: Write with ETag condition
  GithubContentCache->>EdgeCache: Update after successful publication
Loading

Suggested reviewers: kevinvandy

Merge Risk: Minimal · previous review d1f59

No issue identified in this review currently blocks merging. The investigated cache-repair and Cloudflare compatibility concerns do not establish a failure.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 5.88% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "Reduce duplicate R2 cache refresh writes" matches the main change. The objectives say cache refreshes now check R2 before fetching GitHub content or rebuilding artifacts, and the conditiona…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "Reduce duplicate R2 cache refresh writes" matches the main change. The objectives say cache refreshes now check R2 before fetching GitHub content or rebuilding artifacts, and the conditional writes and ETag checks in github-content-cache.server.ts and the test utilities prevent stale overwrites. The title is specific and short. The PR also adds metrics and a measurement script, but those are supporting work and need not appear in the title.

Full details: Docstring Coverage

Explanation

Docstring coverage is 5.88% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 17 functions across 15 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@changeset-bot

changeset-bot Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 138a23d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@tannerlinsley
tannerlinsley marked this pull request as ready for review October 29, 2026 22:10
@tannerlinsley
tannerlinsley requested a review from a team October 29, 2026 22:10
@tannerlinsley
tannerlinsley marked this pull request as draft October 29, 2026 22:20
@tannerlinsley
tannerlinsley changed the base branch from main to taren/visitor-chat October 29, 2026 22:20
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
tanstack-com 138a23d Commit Preview URL

Branch Preview URL
Oct 29 2026, 11:39 PM

@tannerlinsley
tannerlinsley marked this pull request as ready for review October 29, 2026 22:54
Base automatically changed from taren/visitor-chat to main October 30, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant