Skip to content

[3/3] [core] Resume a parked VM on the hook_received this process wrote - #5573

Draft
benpankow wants to merge 2 commits into
ben/park-resume-stalefrom
ben/park-takes-wake-event
Draft

benpankow wants to merge 2 commits into
ben/park-resume-stalefrom
ben/park-takes-wake-event

Conversation

@benpankow

@benpankow benpankow commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Description

A resumeHook() wake makes the orchestrator read the log for the hook_received that the same process just wrote. With a parked VM (#5467) that read is incremental, but it plus runs.get is still ~80ms on Vercel, and it sits between the write and the turn.

This builds on #5468 (merged), in which the wake names its hook_received. resumeHook() keeps the event it committed, payload included, with the run's parked entry (noteParkedRunWrite). When the kept events continue the parked log, the resuming delivery takes them as its read and reads neither the log nor the run:

  • the first kept event holds the slot after tailSlot;
  • the rest follow it;
  • the wake names one of them.

It adopts { seq: last slot, seqInBand: parked count } and uses the run the parking delivery held. The parked entry also keeps the event ceiling its reads reported, so a delivery that reads nothing still enforces maxEvents.

Kept events count against WORKFLOW_PARKED_VM_MAX_BYTES as they're kept: one that takes its entry over half the budget drops the entry, the oldest other entries are evicted to stay within it, and an entry keeps at most 16 such events.

The kept events don't prove the log ends there. Another process may have cancelled or completed the run after them, or handled the same wake first. Then the reused run says running and the adopted count is stale. The turn's first in-band write settles it (the World refuses work on an ended run, and the fence refuses the count), but with run-ahead a body started ahead of its creation would run anyway. So until a write of the delivery commits, nothing runs ahead. eve doesn't run ahead, so for it this costs nothing; with run-ahead on, the turn's first step waits for its start. Reading the run instead wouldn't help with the duplicated wake, so the run is still reused.

This only applies when the orchestrator runs in the process that wrote the event, which in practice means inline delivery (#5365), and it shouldn't land before #5365 does. On a spike stacked with #5365 and eve's W4 workload (follow-ups after 15s idle, Vercel Lambda), it applied on about half the turns. The misses were the previous turn running on another instance. With a 400ms mock TTFT, client p50 went from 936 to 876ms. Without inline delivery it never applies, and W4 was unchanged (668 → 667ms).

Stack: #5467 → #5470 → this.

How did you test your changes?

New tests in parked-session.runtime.test.ts: a resume on the hook_received that resumeHook() wrote reads nothing before the turn, including when a step body sets an attribute. A write landing in between makes it read once and catch up. With a 50ms World round trip on each step start, a run cancelled or completed elsewhere after the kept event runs no step body, and a wake another process already handled is refused by the fence with the turn's body run once; both ran the body without the run-ahead gate. Unit tests in parked-session.test.ts cover the budget, eviction and cap for kept events. The max-events ceiling tests pass against both parked paths.

PR Checklist - Required to merge

  • 📦 pnpm changeset was run to create a changelog for this PR
    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  • 🔒 DCO sign-off passes (run git commit --signoff on your commits)
  • 📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete

🤖

@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
workflow-docs Ready Ready Preview, v0 Nov 21, 2026 9:50pm UTC

@changeset-bot

changeset-bot Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 6e6005c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
Name Type
@workflow/core Patch
@workflow/builders Patch
@workflow/cli Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/web Patch
workflow Patch
@workflow/world-cloudflare Patch
@workflow/world-testing Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch
@workflow/world-vercel Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

You must have Developer access to commit code to Vercel Labs on Vercel. If you contact an administrator and receive Developer access, commit again to see your changes.

Learn more: https://vercel.com/docs/accounts/team-members-and-roles/access-roles#team-level-roles

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

✅ All tests passed

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

  • webhookWorkflow (nextjs-turbopack · local-prod / local / node / stable)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • run-pickup-stall · runs app-generated source against a registered step (tanstack-start) · at 21:54:03Z · abandoned wrun_01M801KTJ34RMW2PTP51NTD1D2
  • cold-start-warmup · suite warmup (tanstack-start) · at 21:54:10Z · abandoned wrun_01M801KTJB5Z8XASE9SQHHYV44

E2E Test Summary

Summary
Passed Failed Skipped Total
✅ 💻 Local Development 5272 0 731 6003
✅ 📦 Local Production 5272 0 731 6003
✅ 🐘 Local Postgres 5417 0 586 6003
✅ 🪟 Windows 401 0 13 414
✅ 🌐 Cross-language Conformance 68 0 114 182
✅ dynamic-runs 0 0 0 0
✅ local-postgres-spec9 0 0 364 364
✅ local-spec9 409 0 5 414
✅ vercel-http-transport 1054 0 188 1242
✅ vercel-spec9 395 0 19 414
✅ vercel-ws-transport 712 0 116 828
Total 19000 0 2867 21867
Details by Category

✅ 💻 Local Development

App Passed Failed Skipped
✅ astro-stable-node 172 0 35
✅ astro-stable-quickjs 171 0 36
✅ express-stable-node 172 0 35
✅ express-stable-quickjs 171 0 36
✅ fastify-stable-node 172 0 35
✅ fastify-stable-quickjs 171 0 36
✅ hono-stable-node 172 0 35
✅ hono-stable-quickjs 171 0 36
✅ nest-stable-node 172 0 35
✅ nest-stable-quickjs 171 0 36
✅ nextjs-turbopack-canary-node 201 0 6
✅ nextjs-turbopack-canary-quickjs 200 0 7
✅ nextjs-turbopack-quickjs-snapshot 200 0 7
✅ nextjs-turbopack-stable-node 201 0 6
✅ nextjs-turbopack-stable-quickjs 200 0 7
✅ nextjs-webpack-canary-node 201 0 6
✅ nextjs-webpack-canary-quickjs 200 0 7
✅ nextjs-webpack-stable-node 201 0 6
✅ nextjs-webpack-stable-quickjs 200 0 7
✅ nitro-stable-node 172 0 35
✅ nitro-stable-quickjs 171 0 36
✅ nuxt-stable-node 172 0 35
✅ nuxt-stable-quickjs 171 0 36
✅ sveltekit-stable-node 191 0 16
✅ sveltekit-stable-quickjs 190 0 17
✅ tanstack-start-node 172 0 35
✅ tanstack-start-quickjs 171 0 36
✅ vite-stable-node 172 0 35
✅ vite-stable-quickjs 171 0 36

✅ 📦 Local Production

App Passed Failed Skipped
✅ astro-stable-node 172 0 35
✅ astro-stable-quickjs 171 0 36
✅ express-stable-node 172 0 35
✅ express-stable-quickjs 171 0 36
✅ fastify-stable-node 172 0 35
✅ fastify-stable-quickjs 171 0 36
✅ hono-stable-node 172 0 35
✅ hono-stable-quickjs 171 0 36
✅ nest-stable-node 172 0 35
✅ nest-stable-quickjs 171 0 36
✅ nextjs-turbopack-canary-node 201 0 6
✅ nextjs-turbopack-canary-quickjs 200 0 7
✅ nextjs-turbopack-quickjs-snapshot 200 0 7
✅ nextjs-turbopack-stable-node 201 0 6
✅ nextjs-turbopack-stable-quickjs 200 0 7
✅ nextjs-webpack-canary-node 201 0 6
✅ nextjs-webpack-canary-quickjs 200 0 7
✅ nextjs-webpack-stable-node 201 0 6
✅ nextjs-webpack-stable-quickjs 200 0 7
✅ nitro-stable-node 172 0 35
✅ nitro-stable-quickjs 171 0 36
✅ nuxt-stable-node 172 0 35
✅ nuxt-stable-quickjs 171 0 36
✅ sveltekit-stable-node 191 0 16
✅ sveltekit-stable-quickjs 190 0 17
✅ tanstack-start-node 172 0 35
✅ tanstack-start-quickjs 171 0 36
✅ vite-stable-node 172 0 35
✅ vite-stable-quickjs 171 0 36

✅ 🐘 Local Postgres

App Passed Failed Skipped
✅ astro-stable-node 177 0 30
✅ astro-stable-quickjs 176 0 31
✅ express-stable-node 177 0 30
✅ express-stable-quickjs 176 0 31
✅ fastify-stable-node 177 0 30
✅ fastify-stable-quickjs 176 0 31
✅ hono-stable-node 177 0 30
✅ hono-stable-quickjs 176 0 31
✅ nest-stable-node 177 0 30
✅ nest-stable-quickjs 176 0 31
✅ nextjs-turbopack-canary-node 206 0 1
✅ nextjs-turbopack-canary-quickjs 205 0 2
✅ nextjs-turbopack-quickjs-snapshot 205 0 2
✅ nextjs-turbopack-stable-node 206 0 1
✅ nextjs-turbopack-stable-quickjs 205 0 2
✅ nextjs-webpack-canary-node 206 0 1
✅ nextjs-webpack-canary-quickjs 205 0 2
✅ nextjs-webpack-stable-node 206 0 1
✅ nextjs-webpack-stable-quickjs 205 0 2
✅ nitro-stable-node 177 0 30
✅ nitro-stable-quickjs 176 0 31
✅ nuxt-stable-node 177 0 30
✅ nuxt-stable-quickjs 176 0 31
✅ sveltekit-stable-node 196 0 11
✅ sveltekit-stable-quickjs 195 0 12
✅ tanstack-start-node 177 0 30
✅ tanstack-start-quickjs 176 0 31
✅ vite-stable-node 177 0 30
✅ vite-stable-quickjs 176 0 31

✅ 🪟 Windows

App Passed Failed Skipped
✅ nextjs-turbopack-node 201 0 6
✅ nextjs-turbopack-quickjs 200 0 7

✅ 🌐 Cross-language Conformance

App Passed Failed Skipped
✅ python 68 0 114

✅ dynamic-runs

App Passed Failed Skipped
✅ example-vercel 0 0 0
✅ express-vercel 0 0 0
✅ hono-vercel 0 0 0
✅ nextjs-turbopack-vercel 0 0 0
✅ nitro-vercel 0 0 0
✅ vite-vercel 0 0 0

✅ local-postgres-spec9

App Passed Failed Skipped
✅ nextjs-turbopack-stable-node 0 0 182
✅ nextjs-turbopack-stable-quickjs 0 0 182

✅ local-spec9

App Passed Failed Skipped
✅ nextjs-turbopack-node 205 0 2
✅ nextjs-turbopack-quickjs 204 0 3

✅ vercel-http-transport

App Passed Failed Skipped
✅ example 171 0 36
✅ express 171 0 36
✅ hono 171 0 36
✅ nextjs-turbopack 199 0 8
✅ nitro 171 0 36
✅ vite 171 0 36

✅ vercel-spec9

App Passed Failed Skipped
✅ nextjs-turbopack-node 198 0 9
✅ nextjs-turbopack-quickjs 197 0 10

✅ vercel-ws-transport

App Passed Failed Skipped
✅ example 171 0 36
✅ express 171 0 36
✅ nextjs-turbopack 199 0 8
✅ vite 171 0 36

📋 View full workflow run

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟢 world-sim scenario book — 0 fail of 45 total
scenario outcome events virt replay violations
✅ smoke-no-steps completed 3 0ms ok 0
✅ smoke-one-step completed 5 0ms ok 0
✅ hook-at-step-started completed 9 0ms ok 0
✅ hook-at-step-completed completed 9 0ms ok 0
✅ hook-at-hook-created completed 9 0ms ok 0
✅ claim-hook-at-registration completed 9 0ms ok 0
✅ claim-hook-at-step-started completed 9 0ms ok 0
✅ deadline-hook-wins completed 6 1.0h ok 0
✅ deadline-expires completed 6 1.0h ok 0
✅ step-vs-timer-early-settlement completed 6 1.0h ok 0
✅ long-sleep completed 9 30.0d ok 0
✅ hook-never-arrives stalled 3 0ms skipped 0
✅ step-retries-twice completed 9 2.0s ok 0
✅ parallel-steps completed 7 0ms ok 0
✅ hook-on-execution-state completed 9 0ms ok 0
✅ peek-hook-before-branch completed 9 0ms ok 0
✅ peek-hook-after-branch completed 9 0ms ok 0
✅ peek-hook-at-registration completed 9 0ms ok 0
✅ race-hook-before-probe completed 9 0ms ok 0
✅ race-hook-after-probe completed 9 0ms ok 0
✅ race-duplicate-delivery completed 10 0ms ok 0
✅ attr-hook-before-step completed 9 0ms ok 0
✅ attr-hook-after-step completed 9 0ms ok 0
✅ attr-from-step-body completed 10 0ms ok 0
✅ fork-hook-after-timeout completed 11 1.0m ok 0
✅ fork-hook-before-timeout completed 11 1.0m ok 0
✅ count-hook-after-timeout completed 13 1.0m ok 0
✅ count-hook-before-timeout completed 15 1.0m ok 0
✅ stale-read-step-count-fork completed 15 1.0m ok 0
✅ stale-read-equal-step-counts completed 11 1.0m ok 0
✅ step-vs-step-fork completed 9 0ms ok 0
✅ step-vs-step-fork-fenced completed 9 1.0s ok 0
✅ fence-catches-benign-direction completed 9 1.0s ok 0
✅ run-ahead-fenced-overlap completed 10 1.0s ok 0
✅ in-flight-before-decision completed 15 1.0m ok 0
✅ in-flight-before-decision-counted completed 13 1.0m ok 0
✅ in-flight-after-decision completed 15 2.0m ok 0
✅ stale-read-step-count-fork-fenced completed 15 1.0m ok 0
✅ fork-hook-wins completed 10 1.0m ok 0
✅ fork-timeout-wins completed 10 1.0m ok 0
✅ unclaimed-payload-under-fork completed 14 1.0m ok 0
✅ claimed-payload-under-fork completed 14 1.0m ok 0
✅ writers-independent-step-bodies completed 9 0ms ok 0
✅ writers-scripted-tempo completed 9 0ms ok 0
✅ cancel-mid-step cancelled 6 0ms skipped 0

Full trace: world-sim.txt

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

❌ The benchmark run for 6e6005c failed. See the run logs for details.

No benchmark results were produced.

ℹ️ Metric definitions & methodology

Metrics —

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies. Streaming delivery performance is measured in durabench.

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor
Framework Flow route Step reg. Framework output
hono 306.6 KiB (±0) 126.2 KiB (+100 B) 2.43 MiB (+6.0 KiB)
nextjs-turbopack 398.2 KiB (±0) 491 B (±0) 1.14 MiB (+1.0 KiB)
About these numbers

Sizes are gzip; parentheses show the change against main.
Flow route and Step reg. gate this job, on raw bytes rather than the gzip shown, at max(2%, 50.0 KiB). Framework output is informational.

6e6005c · run

@vercel-internal-security-reviewer

Copy link
Copy Markdown

Security review details

@benpankow
benpankow force-pushed the ben/park-takes-wake-event branch from 429fca3 to 8aef15a Compare November 21, 2026 01:08
@benpankow benpankow changed the title [core] Resume a parked VM on the hook_received this process wrote [3/3] [core] Resume a parked VM on the hook_received this process wrote Nov 21, 2026
Signed-off-by: benpankow <ben.pankow@vercel.com>
…n them until a write confirms the fence

Kept events count against the byte budget when they are kept, and an entry keeps at most 16. A resume on them adopts an in-band count no read confirmed, so another process that ended the run or handled the same wake would let a body started ahead run anyway; run-ahead waits for the delivery's first committed write.

Signed-off-by: benpankow <ben.pankow@vercel.com>

This branch was successfully deployed

1 active deployment
Preview – workflow-docs — 6e6005ce Deployed Nov 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant