Skip to content

fix(files): stream served files with byte ranges instead of buffering them under 100 MB - #9542

Merged
waleedlatif1 merged 3 commits into
stagingfrom
fix/file-serve-streaming
Oct 31, 2026
Merged

waleedlatif1 merged 3 commits into
stagingfrom
fix/file-serve-streaming

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • /api/files/serve read every file fully into memory and refused anything over 100 MB (413), while workspace files are admitted up to 5 GB, so large files (e.g. agent-rendered videos) could never be downloaded or previewed
  • Every request answered with the stored object byte-for-byte (raw, or not a Sim page / generated document / preview=1, which may be a HEIC transcode) now streams from storage instead; transformed representations stay buffered and bounded exactly as before
  • HTTP Range support end to end: resolveByteRange (single range, suffix, clamp, 416, If-Range strong-ETag only), ranged downloadFileStream for S3 / Blob / GCS / local, and createStoredFileStreamResponse (200 / 206 / 304 / 416 with Accept-Ranges, Content-Range, a strong ETag). Seeking works, and Safari can play at all
  • Workspace files keep the open-then-recheck delivery guarantee through a shared streamAuthorizedWorkspaceFile helper (access + read receipt + provenance rechecked after storage opens, stream disposed on refusal); the inline-image use case now uses the same helper instead of its own copy
  • Legacy contexts (execution outputs, chat/copilot attachments, knowledge docs) stream the same way, sized and versioned by a storage HEAD
  • Removed the buffered readWorkspaceFileContentByKey use case (raw reads now stream) and renamed its module to read-workspace-file-record-by-key
  • Client: downloads over 100 MB are HEAD-probed then handed to the browser's download manager instead of being read into a tab Blob; smaller downloads are unchanged
  • Client: audio/video previews play straight from the serve URL (progressive playback + seeking) on sources that declare servesByteRanges; project and public-share sources keep the blob path. A failed load is diagnosed with a HEAD probe (replaced key → record re-resolved; unplayable format → "can't be played" message), and a load that finishes before hydration is picked up on mount so the spinner never sticks

Type of Change

  • Bug fix

Testing

  • New lib/uploads/server/byte-range.test.ts (29 cases, written failure-modes-first), stream-authorized-workspace-file.test.ts, streaming cases in app/api/files/serve/[...path]/route.test.ts (200/206/416/304/If-Range, legacy streaming, buffered paths unchanged), large-download cases in lib/uploads/client/download.test.ts; each guard red-checked by reverting it
  • Ran the touched suites plus storage-service, provider client, inline/download use case and files-utils tests locally (all green); check:api-validation:strict, check:comment-hygiene, check:source-conventions, check:client-boundary, check:test-patterns, check:file-names, check:utils, check:boundaries, block registry check, lint
  • Local E2E against next dev with local storage and a real session: 14/14 HTTP checks on a 151 MB non-faststart MP4 (byte-exact full download by SHA-256, exact 206 slices incl. suffix/tail, 416, 304, stale/matching If-Range, HEAD, immutable v=, unauthenticated 401); WebKit + Chromium: preview plays, seeks to near the end, Download button saves the exact file; a 5 MB clip behaves the same; server RSS flat while a 1 MB/s client streamed the whole file
  • Full bun run test, check:audits and type-check left to CI

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 31, 2026 11:28pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Files served without changes now stream from storage with byte-range support. Rendered pages, generated documents, and image previews keep their bounded, buffered paths.

  • Stored files stream in full or by the byte range a caller requests.
  • Workspace audio and video play directly from range-enabled file URLs.
  • Large raw downloads go straight to the browser download manager.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Authorized file request] --> B{Stored bytes unchanged?}
  B -->|Yes| C[Check version and requested range]
  C --> D[Stream full file or byte range]
  B -->|No| E[Read under size limit]
  E --> F[Render or compile]
  F --> G[Return bounded response]
Loading

Reviews (3) · Last reviewed commit: "fix(files): probe served bytes before ha..." · Reviewed by Greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 23 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/lib/uploads/client/download.ts Resolved
Comment thread apps/sim/lib/uploads/client/download.ts Resolved
…uments in-page, version local objects by inode
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 24 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 5815964 into staging Oct 31, 2026
69 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/file-serve-streaming branch November 1, 2026 04:52

This branch was previously deployed

1 inactive deployment
Preview — 75dacb4f Deployed Oct 31, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant