Repository navigation
[core][world] Include hook_received event ID in resumeHook() wake - #5468
Conversation
|
You must have Developer access to commit code to Vercel Labs on Vercel. If you contact an administrator and receive Developer access, commit again to see your changes. Learn more: https://vercel.com/docs/accounts/team-members-and-roles/access-roles#team-level-roles |
🦋 Changeset detectedLatest commit: ea47270 The changes in this PR will be included in the next version bump. This PR includes changesets to release 22 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results✅ All tests passed
|
| Passed | Failed | Skipped | Total | |
|---|---|---|---|---|
| ✅ ▲ Vercel Production | 4649 | 0 | 940 | 5589 |
| ✅ 💻 Local Development | 5272 | 0 | 731 | 6003 |
| ✅ 📦 Local Production | 5272 | 0 | 731 | 6003 |
| ✅ 🐘 Local Postgres | 5417 | 0 | 586 | 6003 |
| ✅ 🪟 Windows | 401 | 0 | 13 | 414 |
| ✅ 🌐 Cross-language Conformance | 68 | 0 | 114 | 182 |
| ✅ dynamic-runs | 0 | 0 | 0 | 0 |
| ✅ local-postgres-spec9 | 0 | 0 | 364 | 364 |
| ✅ local-spec9 | 409 | 0 | 5 | 414 |
| ✅ vercel-http-transport | 1054 | 0 | 188 | 1242 |
| ✅ vercel-multi-region | 27 | 0 | 0 | 27 |
| ✅ vercel-queue-hold | 4 | 0 | 0 | 4 |
| ✅ vercel-spec9 | 395 | 0 | 19 | 414 |
| ✅ vercel-ws-transport | 712 | 0 | 116 | 828 |
| Total | 23680 | 0 | 3807 | 27487 |
Details by Category
✅ ▲ Vercel Production
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-node | 171 | 0 | 36 |
| ✅ astro-quickjs | 170 | 0 | 37 |
| ✅ example-node | 171 | 0 | 36 |
| ✅ example-quickjs | 170 | 0 | 37 |
| ✅ express-node | 171 | 0 | 36 |
| ✅ express-quickjs | 170 | 0 | 37 |
| ✅ fastify-node | 171 | 0 | 36 |
| ✅ fastify-quickjs | 170 | 0 | 37 |
| ✅ hono-node | 171 | 0 | 36 |
| ✅ hono-quickjs | 170 | 0 | 37 |
| ✅ nest-node | 171 | 0 | 36 |
| ✅ nest-quickjs | 170 | 0 | 37 |
| ✅ nextjs-turbopack-node | 199 | 0 | 8 |
| ✅ nextjs-turbopack-quickjs | 198 | 0 | 9 |
| ✅ nextjs-webpack-node | 199 | 0 | 8 |
| ✅ nextjs-webpack-quickjs | 198 | 0 | 9 |
| ✅ nitro-node | 171 | 0 | 36 |
| ✅ nitro-quickjs | 170 | 0 | 37 |
| ✅ nuxt-node | 171 | 0 | 36 |
| ✅ nuxt-quickjs | 170 | 0 | 37 |
| ✅ python-node | 66 | 0 | 141 |
| ✅ sveltekit-node | 190 | 0 | 17 |
| ✅ sveltekit-quickjs | 189 | 0 | 18 |
| ✅ tanstack-start-node | 171 | 0 | 36 |
| ✅ tanstack-start-quickjs | 170 | 0 | 37 |
| ✅ vite-node | 171 | 0 | 36 |
| ✅ vite-quickjs | 170 | 0 | 37 |
✅ 💻 Local Development
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 172 | 0 | 35 |
| ✅ astro-stable-quickjs | 171 | 0 | 36 |
| ✅ express-stable-node | 172 | 0 | 35 |
| ✅ express-stable-quickjs | 171 | 0 | 36 |
| ✅ fastify-stable-node | 172 | 0 | 35 |
| ✅ fastify-stable-quickjs | 171 | 0 | 36 |
| ✅ hono-stable-node | 172 | 0 | 35 |
| ✅ hono-stable-quickjs | 171 | 0 | 36 |
| ✅ nest-stable-node | 172 | 0 | 35 |
| ✅ nest-stable-quickjs | 171 | 0 | 36 |
| ✅ nextjs-turbopack-canary-node | 201 | 0 | 6 |
| ✅ nextjs-turbopack-canary-quickjs | 200 | 0 | 7 |
| ✅ nextjs-turbopack-quickjs-snapshot | 200 | 0 | 7 |
| ✅ nextjs-turbopack-stable-node | 201 | 0 | 6 |
| ✅ nextjs-turbopack-stable-quickjs | 200 | 0 | 7 |
| ✅ nextjs-webpack-canary-node | 201 | 0 | 6 |
| ✅ nextjs-webpack-canary-quickjs | 200 | 0 | 7 |
| ✅ nextjs-webpack-stable-node | 201 | 0 | 6 |
| ✅ nextjs-webpack-stable-quickjs | 200 | 0 | 7 |
| ✅ nitro-stable-node | 172 | 0 | 35 |
| ✅ nitro-stable-quickjs | 171 | 0 | 36 |
| ✅ nuxt-stable-node | 172 | 0 | 35 |
| ✅ nuxt-stable-quickjs | 171 | 0 | 36 |
| ✅ sveltekit-stable-node | 191 | 0 | 16 |
| ✅ sveltekit-stable-quickjs | 190 | 0 | 17 |
| ✅ tanstack-start-node | 172 | 0 | 35 |
| ✅ tanstack-start-quickjs | 171 | 0 | 36 |
| ✅ vite-stable-node | 172 | 0 | 35 |
| ✅ vite-stable-quickjs | 171 | 0 | 36 |
✅ 📦 Local Production
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 172 | 0 | 35 |
| ✅ astro-stable-quickjs | 171 | 0 | 36 |
| ✅ express-stable-node | 172 | 0 | 35 |
| ✅ express-stable-quickjs | 171 | 0 | 36 |
| ✅ fastify-stable-node | 172 | 0 | 35 |
| ✅ fastify-stable-quickjs | 171 | 0 | 36 |
| ✅ hono-stable-node | 172 | 0 | 35 |
| ✅ hono-stable-quickjs | 171 | 0 | 36 |
| ✅ nest-stable-node | 172 | 0 | 35 |
| ✅ nest-stable-quickjs | 171 | 0 | 36 |
| ✅ nextjs-turbopack-canary-node | 201 | 0 | 6 |
| ✅ nextjs-turbopack-canary-quickjs | 200 | 0 | 7 |
| ✅ nextjs-turbopack-quickjs-snapshot | 200 | 0 | 7 |
| ✅ nextjs-turbopack-stable-node | 201 | 0 | 6 |
| ✅ nextjs-turbopack-stable-quickjs | 200 | 0 | 7 |
| ✅ nextjs-webpack-canary-node | 201 | 0 | 6 |
| ✅ nextjs-webpack-canary-quickjs | 200 | 0 | 7 |
| ✅ nextjs-webpack-stable-node | 201 | 0 | 6 |
| ✅ nextjs-webpack-stable-quickjs | 200 | 0 | 7 |
| ✅ nitro-stable-node | 172 | 0 | 35 |
| ✅ nitro-stable-quickjs | 171 | 0 | 36 |
| ✅ nuxt-stable-node | 172 | 0 | 35 |
| ✅ nuxt-stable-quickjs | 171 | 0 | 36 |
| ✅ sveltekit-stable-node | 191 | 0 | 16 |
| ✅ sveltekit-stable-quickjs | 190 | 0 | 17 |
| ✅ tanstack-start-node | 172 | 0 | 35 |
| ✅ tanstack-start-quickjs | 171 | 0 | 36 |
| ✅ vite-stable-node | 172 | 0 | 35 |
| ✅ vite-stable-quickjs | 171 | 0 | 36 |
✅ 🐘 Local Postgres
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 177 | 0 | 30 |
| ✅ astro-stable-quickjs | 176 | 0 | 31 |
| ✅ express-stable-node | 177 | 0 | 30 |
| ✅ express-stable-quickjs | 176 | 0 | 31 |
| ✅ fastify-stable-node | 177 | 0 | 30 |
| ✅ fastify-stable-quickjs | 176 | 0 | 31 |
| ✅ hono-stable-node | 177 | 0 | 30 |
| ✅ hono-stable-quickjs | 176 | 0 | 31 |
| ✅ nest-stable-node | 177 | 0 | 30 |
| ✅ nest-stable-quickjs | 176 | 0 | 31 |
| ✅ nextjs-turbopack-canary-node | 206 | 0 | 1 |
| ✅ nextjs-turbopack-canary-quickjs | 205 | 0 | 2 |
| ✅ nextjs-turbopack-quickjs-snapshot | 205 | 0 | 2 |
| ✅ nextjs-turbopack-stable-node | 206 | 0 | 1 |
| ✅ nextjs-turbopack-stable-quickjs | 205 | 0 | 2 |
| ✅ nextjs-webpack-canary-node | 206 | 0 | 1 |
| ✅ nextjs-webpack-canary-quickjs | 205 | 0 | 2 |
| ✅ nextjs-webpack-stable-node | 206 | 0 | 1 |
| ✅ nextjs-webpack-stable-quickjs | 205 | 0 | 2 |
| ✅ nitro-stable-node | 177 | 0 | 30 |
| ✅ nitro-stable-quickjs | 176 | 0 | 31 |
| ✅ nuxt-stable-node | 177 | 0 | 30 |
| ✅ nuxt-stable-quickjs | 176 | 0 | 31 |
| ✅ sveltekit-stable-node | 196 | 0 | 11 |
| ✅ sveltekit-stable-quickjs | 195 | 0 | 12 |
| ✅ tanstack-start-node | 177 | 0 | 30 |
| ✅ tanstack-start-quickjs | 176 | 0 | 31 |
| ✅ vite-stable-node | 177 | 0 | 30 |
| ✅ vite-stable-quickjs | 176 | 0 | 31 |
✅ 🪟 Windows
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack-node | 201 | 0 | 6 |
| ✅ nextjs-turbopack-quickjs | 200 | 0 | 7 |
✅ 🌐 Cross-language Conformance
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ python | 68 | 0 | 114 |
✅ dynamic-runs
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-vercel | 0 | 0 | 0 |
| ✅ example-vercel | 0 | 0 | 0 |
| ✅ express-vercel | 0 | 0 | 0 |
| ✅ fastify-vercel | 0 | 0 | 0 |
| ✅ hono-vercel | 0 | 0 | 0 |
| ✅ nest-vercel | 0 | 0 | 0 |
| ✅ nextjs-turbopack-vercel | 0 | 0 | 0 |
| ✅ nextjs-webpack-vercel | 0 | 0 | 0 |
| ✅ nitro-vercel | 0 | 0 | 0 |
| ✅ nuxt-vercel | 0 | 0 | 0 |
| ✅ sveltekit-vercel | 0 | 0 | 0 |
| ✅ tanstack-start-vercel | 0 | 0 | 0 |
| ✅ vite-vercel | 0 | 0 | 0 |
✅ local-postgres-spec9
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack-stable-node | 0 | 0 | 182 |
| ✅ nextjs-turbopack-stable-quickjs | 0 | 0 | 182 |
✅ local-spec9
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack-node | 205 | 0 | 2 |
| ✅ nextjs-turbopack-quickjs | 204 | 0 | 3 |
✅ vercel-http-transport
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ example | 171 | 0 | 36 |
| ✅ express | 171 | 0 | 36 |
| ✅ hono | 171 | 0 | 36 |
| ✅ nextjs-turbopack | 199 | 0 | 8 |
| ✅ nitro | 171 | 0 | 36 |
| ✅ vite | 171 | 0 | 36 |
✅ vercel-multi-region
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack | 27 | 0 | 0 |
✅ vercel-queue-hold
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack | 4 | 0 | 0 |
✅ vercel-spec9
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack-node | 198 | 0 | 9 |
| ✅ nextjs-turbopack-quickjs | 197 | 0 | 10 |
✅ vercel-ws-transport
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ example | 171 | 0 | 36 |
| ✅ express | 171 | 0 | 36 |
| ✅ nextjs-turbopack | 199 | 0 | 8 |
| ✅ vite | 171 | 0 | 36 |
📊 Workflow Benchmarkscommit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 136400ms → this run 91756ms (Δ -44644ms, -33%) ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor. |
Sim WorldSimulated world deterministic testing for races. Traces 🟢 world-sim scenario book — 0 fail of 45 total
Full trace: |
About these numbersSizes are gzip; parentheses show the change against
|
|
@vercel/workflow another eve improvement, this one for resume hooks for suspended sessions |
Signed-off-by: benpankow <ben.pankow@vercel.com>
QuickJS records no consumed position and never reads the tail, so its assertions held whatever the wake named. Signed-off-by: benpankow <ben.pankow@vercel.com>
Signed-off-by: benpankow <ben.pankow@vercel.com> Co-authored-by: vercel-v[bot] <353008837+vercel-v[bot]@users.noreply.github.com>
8b9b845 carries wakeEventId through coalesced re-runs on world-local and world-postgres (the first run's already-consumed value is stripped, the absorbed wake's is carried with its resume timing), records workflow.noop_check_skipped on the span when the hint skips the tail read, and covers a malformed or non-slot hint, the replay the hinted wake runs, and the resolved-fallback write's event ID. Its message lost everything but a mismatched sign-off. I, Ben Pankow <pankow.ben@gmail.com>, hereby add my Signed-off-by to this commit: 8b9b845 Signed-off-by: Ben Pankow <pankow.ben@gmail.com> Co-authored-by: vercel-v[bot] <353008837+vercel-v[bot]@users.noreply.github.com>
7202924 to
b261f87
Compare
VaguelySerious
left a comment
There was a problem hiding this comment.
(AI) Preliminary review:
AI review: no blocking issues
| // Node only: the QuickJS engine records no consumed position, so it never | ||
| // reads the tail and there is nothing for the wake to skip. | ||
| it.runIf(engine === 'node')( | ||
| 'skips the tail read for a wake that names an event it has not consumed', |
There was a problem hiding this comment.
AI Review: Note
The two halves are tested separately. resume-hook.durable.test.ts mocks createEvent to return { event: { eventId } }, and this test enqueues a hand-built wake. No test runs a real resumeHook() wake through a delivery. If a World's events.create response stopped carrying event.eventId for an out-of-band hook_received, the hint would go away without any test failing.
I ran a throwaway test in the style of hook-wait.runtime.test.ts (real resumeHook() against AppendOnlyWorld, node engine). It's a multi-turn chat loop (for await (const p of hook) await tick(p.value)):
- Every wake carried the
hook_receivedid that the World wrote. - Each turn's delivery did 0 tail reads, the run completed, and a cold replay gave the same result.
- With
!wakesForNewEvent()removed, it fails on the first turn (tailReads: 1). - Two resumes before any delivery: the first wake skips the tail read and consumes both events. The second wake then names an event at or below the consumed position, so it does exactly one tail read and acknowledges as a no-op (no new events, no extra step run).
Consider adding a version of the first case to hook-wait.runtime.test.ts. It would catch a break anywhere between producer and consumer:
await resumeHook(TOKEN, { value });
expect((world.held.at(-1)!.message as { wakeEventId?: string }).wakeEventId)
.toBe(eventsOf(world, 'hook_received').at(-1)!.eventId);
const before = world.listCalls.filter(isTailRead).length;
await world.runUntilIdle();
expect(world.listCalls.filter(isTailRead).length).toBe(before);| const wakesForNewEvent = (): boolean => { | ||
| const slot = wakeEventId ? eventIdToSlot(wakeEventId) : null; | ||
| const isNew = slot !== null && isAboveConsumedPosition(world, runId, slot); | ||
| if (isNew) span?.setAttributes(Attribute.WorkflowNoopCheckSkipped(true)); |
There was a problem hiding this comment.
AI Review: Nit
wakesForNewEvent() sets the span attribute inside the if condition. workflow.noop_check_skipped only means "skipped the tail read" because this call is the last conjunct and only runs once every other condition holds. If someone reorders the conjuncts, the attribute would be set on deliveries that never reach the tail read. Two options: say so in a comment, or compute skip before the if and set the attribute next to where the read would have happened.
Separately, a wake that names an event the hint can't use (already consumed, or not slot-numbered) leaves no trace. Recording a false (or a reason) when wakeEventId is present but unused would show how often the hint actually fires in traces. It's optional, but it's the number you'd want when deciding whether the hint pays off.
| string, | ||
| unknown | ||
| >; | ||
| return hookResumeTiming === undefined |
There was a problem hiding this comment.
AI Review: Nit
resumeOf here and in world-local/src/queue.ts:459 only carries wakeEventId to the re-run when the absorbed wake also has hookResumeTiming. That's fine for resumeHook(), which always sends both. But the schema doc in world/src/queue.ts describes wakeEventId generally ("the event an out-of-band writer committed before it sent this wake"). A future producer that sets only wakeEventId would lose it silently on a coalesced re-run. It's harmless, since the field is only a hint. Either narrow the doc to hook resumes, or carry wakeEventId on its own.
| "@workflow/world-postgres": patch | ||
| --- | ||
|
|
||
| A `resumeHook()` wake names the `hook_received` it was sent for, so a spec 9 orchestrator delivery in a process that already consumed the run's log below that event skips the tail read that checks for a no-op wake. |
There was a problem hiding this comment.
AI Review: Note
The last comment says this was folded into #5573, but this PR is still the base of #5467, which #5470 and #5573 stack on, and all three contain these commits. Is the plan to land #5468 first and then the stack, or to close it and land everything through #5573? If it's closed, #5467 has to be retargeted to main. I reviewed this PR on its own and only skimmed how #5467 (the test change in b7129b6) and #5573 (matching kept events by wakeEventId) use the field.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Signed-off-by: Karthik Kalyanaraman <karthik.kalyanaraman@vercel.com>
|
AI note: I merged Why I think it is safe by construction. The new check sits in front of the "nothing new, acknowledge without a replay" exit. When it fires, the delivery skips that exit's tail read and takes the normal load and replay. It adds no new way to acknowledge early, so a wrong or stale What I ran
Not covered. A real multi-region deployment (the hint is a position in the run's own log, so it compares the same anywhere, and a stale read there would cost an extra load); a 4.x receiver; and the latency saving itself, which I did not measure. Two observations, neither blocking
|
|
No backport to This is a latency optimization for the opt-in spec 9 orchestrator, not a fix for a defect: it adds a new To override, re-run the Backport to stable workflow manually via |
Description
When operating in spec 9, a
resumeHook()wake carries only the run ID. Sometimes the wake reaches a process that has already consumed the run's log. In this case, the delivery must first read the log tail to decide whether the wake is a no-op, and loads the log if not. For a hook resume, that read is wasteful, and it delays the step the payload starts. In traces of an eve chat session it took 40–104ms on Vercel.This adds
wakeEventIdtoWorkflowInvokePayload, set to thehook_receivedevent thatresumeHook()just wrote. If that event is past what this process has already consumed, the delivery skips the tail read and loads the log directly. In every other case nothing changes. It's only a hint: a malformed value parses toundefined, and continuation messages drop it.How did you test your changes?
New unit tests in
resume-hook.durable.test.tsandorchestrator.runtime.test.ts. The orchestrator test fails with the check disabled.PR Checklist - Required to merge
pnpm changesetwas run to create a changelog for this PRpatchfor bug fixes,minorfor new features,majorfor breaking changes.pnpm changeset --emptyif you are changing documentation or workbench appsgit commit --signoffon your commits)@vercel/workflowin a comment once the PR is ready, and the above checklist is complete