Skip to content

feat(validator): propose builder blocks (fulu) - #705

Open
MegaRedHand wants to merge 10 commits into
feat/beacon-builder-blocksfrom
feat/beacon-builder-vc
Open

MegaRedHand wants to merge 10 commits into
feat/beacon-builder-blocksfrom
feat/beacon-builder-vc

Conversation

@MegaRedHand

Copy link
Copy Markdown
Collaborator

Fifth step of MEV-Boost builder support: the validator client side. With --builder-proposals, the client registers its validators with the beacon node's builder and accepts a blinded block when the builder's bid wins. It signs the blinded block and publishes it back to the node that produced it. Stacked on #704.

Scope:

  • Covered: fulu and earlier (electra blinded blocks are accepted for third-party beacon nodes).
  • Gloas is unchanged: produceBlockV4 keeps its self-build configuration, and registrations stop at the gloas fork. The gloas builder API comes in phase B.

Flags

Flag Behavior
--builder-proposals Off by default. Registers validators and sends builder_boost_factor with every pre-gloas block request
--builder-boost-factor <u64> Default 100. Requires --builder-proposals
--prefer-builder-proposals Sends u64::MAX, so the builder wins whenever there is a valid bid, unless the execution client asks to override the builder. Requires --builder-proposals; conflicts with --builder-boost-factor

Without --builder-proposals, nothing changes: the client sends builder_boost_factor=0 and refuses a blinded answer, as before.

What it adds

Piece Behavior
Registrations One ValidatorRegistrationV1 per validator the node resolved, signed under the builder domain (genesis fork version, zero validators root). The signature and timestamp are cached until the validator's fee recipient or gas limit changes. The full set is sent after each epoch refresh and changed keys every slot, in batches of 500, to every configured node, on spawned tasks with signing off the async runtime. Validators with no fee recipient are skipped with one warning
Blinded answer ProducedBlock gains a blinded variant. Its root is the full block's, so the proposal guard and the signing path are unchanged. Slot and proposer are checked as for a full block. The fee-recipient check is skipped, since the header names the builder's address
Blinded publish POST /eth/v2/beacon/blinded_blocks, SSZ, only on the node that produced the block, with no failover: only that node's builder holds the payload, and only that node claimed the slot. The request runs on a spawned task, so the proposal budget cannot cut it off
Unconfirmed publish A 5xx, a timeout, or a publish that outlives the proposal budget is reported as "unconfirmed", not as a missed proposal: the relay may still publish the block. Nothing retries, since the slot is signed and a second block would be an equivocation. A 4xx, a 503 or a refused connection is a definite failure, counted as a failed proposal even when it arrives after the budget, by the publish task itself
Metrics ethlambda_validator_builder_registrations_sent_total, ..._builder_registration_failures_total, ..._blinded_blocks_proposed_total, ..._blinded_blocks_unconfirmed_total

Known limitations

  • No slashing-protection record, as for every block this client signs. The in-process guard covers blinded blocks the same way.
  • Per-validator builder toggles (Lighthouse's PATCH route) are out of scope; the flags apply to the whole process.
  • A registration whose send fails is not resent until the next epoch's full send, even though it is cached right away.

Tests

  • Unit: the blinded decode and its root, the HTTP query and status mapping, producer-only publish in the fallback, the registration signature domain, the cache and batching, the gloas stop, the proposal path's outcomes, and the flags.
  • End to end through ethlambda's own beacon node and the mock relay:
    • registration → blinded produce → sign → publish → relay;
    • a full block when builder proposals are off;
    • a full block when the relay is down.

make lint passes, as do the validator lib, rpc lib and binary tests.

@github-actions

Copy link
Copy Markdown

🤖 Kimi Code Review

I'll review this PR for the builder proposals feature in ethlambda. This is a substantial change adding MEV-Boost style builder block proposals to the validator client.

Security & Correctness

Critical: Blinded block publish isolation is correct but fragile

crates/validator/src/beacon_node/fallback.rs:294-317 — The publish_blinded_block implementation correctly pins blinded publishes to the producing node. However, the blinded_producer lock uses unwrap_or_else(std::sync::PoisonError::into_inner) which recovers from poisoned locks by clearing the error. This is dangerous: if a thread panicked while holding the lock, the Option<(Slot, usize)> could be in an inconsistent state. Given this is consensus-critical (prevents equivocation), consider whether panicking is safer.

Critical: u64::MAX as "prefer builder" sentinel

bin/ethlambda/src/validator.rs:95-100 — Using u64::MAX as the boost factor for --prefer-builder-proposals is correct per the spec, but the arithmetic local >= factor * bid / 100 could overflow if implemented naively on the node side. The PR doesn't show the node-side implementation, but verify that multiplication is checked or uses uint256.

Medium: Race between register_changed and register_all

crates/validator/src/registration.rs:335-341 — register_all stores signed_generation after current_blocking completes, but register_changed reads it before checking. If register_changed fires during register_all's current_blocking, it will see stale generation, re-sign everything, and send duplicates. The register_changed spawn in lib.rs:304 happens after register_all completes, so this is unlikely in practice, but the ordering is subtle.

Performance

Good: Chunked signing off async runtime

crates/validator/src/registration.rs:280-302 — SIGN_CHUNK = 64 with spawn_blocking properly avoids blocking the async runtime. The per-chunk store read lock is correct.

Concern: register_changed every slot even with no changes

crates/validator/src/lib.rs:368-370 — The spawn happens unconditionally off the slot loop. The generation check is fast (atomic read), but this still allocates a task per slot. Consider whether this is necessary when builder is None or when no validators are registered.

Rust Idioms & Code Quality

Good pattern: then_some for conditional construction

bin/ethlambda/src/validator.rs:93 — self.builder_proposals.then_some(...) is idiomatic.

Issue: std::sync::Mutex in async code

crates/validator/src/beacon_node/fallback.rs:46 — blinded_producer: std::sync::Mutex<Option<(Slot, usize)>> is held across await points in produce_block (line 137) and publish_blinded_block (line 311). This is technically fine since the critical sections are brief, but tokio::sync::Mutex would be more idiomatic for async contexts. The std::sync::Mutex is used elsewhere in this file for consistency with existing code.

Issue: unwrap_or_else(PoisonError::into_inner) pattern repeated

Multiple files (fallback.rs, proposal.rs, registration.rs) use this pattern. Consider a helper or parking_lot which doesn't poison.

Error Handling

Good: BlindedUnconfirmed distinction

crates/validator/src/error.rs:72-80 — Properly distinguishes "relay may have published" from definite failures. The comment correctly notes this is never retried.

Issue: task.await.unwrap_or_else in publish_blinded

crates/validator/src/proposal.rs:589-593 — If the spawned task panics, this returns Ok(BuilderUnconfirmed), which is correct (unknown state), but the panic info is lost. Consider logging the panic payload if available.

Consensus & Spec Compliance

Good: Fork-aware blinded block handling

crates/validator/src/beacon_node/block_contents.rs:348-369 — Explicitly rejects blinded blocks for forks that don't support them (Phase0 through Deneb, Gloas, Lean). The exhaustive match forces future fork additions to make a decision.

Question: Gloas builder market transition

crates/validator/src/registration.rs:257-260 — Registrations stop at Gloas, but the PR description says "from gloas the builder market works through proposer preferences instead." Is there a corresponding proposer_preferences implementation, or is this a TODO? The produceBlockV4 path in http.rs doesn't show preference handling.

Testing

Good: Comprehensive test coverage

The validator_client_tests.rs builder flow module tests end-to-end: registration, blinded production, publish, and failure modes. The fallback.rs tests verify no failover on blinded publish.

Missing test: Clock observation for blinded publish delay

crates/validator/src/proposal.rs:550-558 — The publication delay metric is only observed when slot_start is Some. No test verifies this path with a real SlotClock.

Documentation

Minor: Typo in CLAUDE.md

CLAUDE.md:855 — "pre-gloas slots register validators with the node's builder and may sign a blinded block, published to the producing node only" — "published" should be "publishable" or rephrase for clarity.

Minor: docs/cli.md inconsistency

docs/cli.md:296 — --builder-boost-factor default is 100, but bin/ethlambda/src/validator.rs:84 shows default_value = "100". The description says "0 prefers the local payload unless it cannot be built" but the CLI help says "0 prefers the local payload unless it cannot be built" — verify this matches node behavior.

Suggestions

  1. Line 137 of fallback.rs: Consider tokio::sync::Mutex for blinded_producer to avoid blocking the async runtime, or document why std::sync::Mutex is sufficient (hold time is brief).

  2. Line 589 of proposal.rs: Log the panic payload if the publish task panicked, not just the JoinError.

  3. Line 335-341 of registration.rs: Move signed_generation.store before self.send to reduce the window for the race with register_changed, or document why the current ordering is safe.

  4. General: The PoisonError::into_inner recovery pattern appears 10+ times. Consider whether parking_lot::Mutex (no poisoning) would simplify and be safer.

Summary

The implementation is correct and well-tested for the core security property: blinded blocks don't fail over to wrong nodes. The ProposalGuard integration prevents double-signing. The main concerns are around lock poisoning recovery in consensus-critical paths and a subtle race in registration caching. The code is idiomatic Rust with good async patterns for the signing workload.


Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Codex Code Review

Looks good overall. The validator-side builder flow is careful about proposer-safety: the slot is recorded at signing time, blinded publishes are isolated onto a background task, and the timeout path no longer misclassifies a slow relay as a missed proposal.

  • No correctness or security bug jumped out in the changed paths around crates/validator/src/proposal.rs:515 and crates/validator/src/lib.rs:677.
  • The most important consensus property here—never re-signing a second block for the same slot even if publication overruns—is preserved by the guard plus signed_blinded_at, and the new tests cover that well in crates/validator/src/proposal.rs:978 and crates/validator/src/proposal.rs:889.

Minor nits only:

  • crates/validator/src/proposal.rs:544 logs Published::Imported / Published::BuilderUnconfirmed as unexpected for blinded publish, but still increments success counters for any non-BuilderUnconfirmed outcome at crates/validator/src/proposal.rs:552. If Imported is truly impossible on this path, consider treating it as an error instead of counting it as a successful blinded proposal; otherwise metrics can hide a beacon-node/API contract regression.
  • crates/validator/src/lib.rs:683 / crates/validator/src/lib.rs:692 key off signed_blinded_at(slot) after the proposal future resolves. That is fine with the current one-duty-per-slot model, but it couples loop behavior to shared mutable state. A slightly cleaner pattern would be to return an explicit “signed blinded” marker from ProposalService::propose so the caller does not have to re-infer path selection from side state.

If you want, I can also do a second pass on the beacon-node / RPC-side builder changes from the same PR for spec conformance and edge cases.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review

Review of PR 705 (validator client: builder proposals and blinded publish)

I read the diff through the end of the HTTP client changes, including the CLI flags, the fallback node, http.rs, block_contents.rs, the rpc test refactor and the end-to-end tests. I did not read the rest of the diff, which likely holds the validator-service wiring, registration signing and caching, and metrics. I did not build the code or run the tests. The review below covers what I read.

The design is sound. The main safety properties hold:

  • No failover on blinded publish. publish_blinded_block goes only to the producing node, recorded by (slot, position) in FallbackBeaconNode. A second node cannot be handed a signed block it never produced.
  • No double-sign. Nothing retries after the slot is signed.
  • Stale claims are cleared. The record is overwritten on every successful produce_block, so a full block clears an earlier blinded claim.
  • Unconfirmed vs. definite failures. The split in publish_blinded_block (crates/validator/src/beacon_node/http.rs) is sensible. Connect errors and 503 are definite because the node cannot have acted on the block. Other 5xx errors, timeouts and resets are unconfirmed.
  • Blinded answers are only decoded when requested. A blinded answer is refused when builder_boost_factor is None, so a client with builder proposals off never signs a block it cannot publish.
  • Gloas and the other forks are refused by name. from_blinded_ssz has no _ arm, so a new fork forces a decision.
  • CLI rules are tested. The requires and conflicts_with rules are exercised by parsing real argument lists.

Minor points and questions

  1. 503 as "definite" (http.rs, publish_blinded_block). This holds for ethlambda and Lighthouse, where 503 means syncing before the builder is touched. A reverse proxy or load balancer can also return 503 after forwarding the request, and then the relay may still publish. The doc comment says "the node cannot have acted", which is only true for that assumption. Consider saying so in the doc, or treating a 503 as definite only when the body or headers match a sync refusal. The cost of the current choice is only a mislabelled metric, since nothing retries either way.

  2. Fallback record with produce_block failure (fallback.rs, produce_block). If every node fails, the early return via ? leaves the previous record in place. This is harmless because publish checks the slot. Writing None on failure as well would make the invariant "record reflects the latest produce" literal, but I do not think it is needed.

  3. register_validators timeout on the fallback path. REGISTRATION_TIMEOUT is 12 s per node, and try_all is sequential. With N unreachable nodes a registration round can take up to N × 12 s. The PR says this runs on spawned tasks, so it does not block duties. Please confirm that no per-slot "changed keys" send can queue behind a previous round indefinitely.

  4. Unused parameter. publish_blinded_block in http.rs takes _slot, which only the fallback layer uses. That is fine for the trait, but a one-line comment would stop someone from "fixing" it.

  5. Copy of the body. post_ssz_body calls body.to_vec(). A blinded block is small, so this does not matter. For full blocks with blobs it was already the existing behaviour, and the refactor does not make it worse.

  6. Boost factor semantics. The doc on --builder-boost-factor says the local payload wins iff local >= factor * bid / 100. With u64::MAX the node side must use widened arithmetic or special-case the value. That is in the prior PR, but it is worth confirming that feat(rpc): serve and publish builder blocks (fulu) #704 handles it without overflow.

Tests. Coverage is good. It includes status mapping, the connect-failure case, fallback routing and no-failover, a registration JSON round trip, and end-to-end tests against the real router with a mock relay. Moving the builder test world into proposal::test_support is a clean reuse.

Overall: LGTM on what I read. The points above are minor, with Point 1 the most worth a doc tweak.


Automated review by Claude (Anthropic) · sonnet · custom prompt

When the beacon node's builder wins a slot, produceBlockV3 answers with a
blinded block. The validator client needs a ProducedBlock variant for it
that keeps the full block's root, so the signing path and the proposal
guard stay unchanged, and that encodes as a SignedBlindedBeaconBlock for
the blinded_blocks endpoint.
Builder proposals need the client to tell the node how to weigh a bid, to
accept a blinded answer only when it will publish one, and to hand the
signed block to the node's builder. BlockRequest carries an optional
builder_boost_factor (None keeps today's behaviour: factor 0, blinded
refused), and the trait gains register_validators and
publish_blinded_block.

A blinded publish that fails with a 5xx or no answer is its own error,
BlindedUnconfirmed, because the relay may still publish the block, unlike
a 4xx which is a definite refusal. Registrations go to every node, since
they install state on each node's builder.
…ed it

Only the producing node's builder holds the payload a blinded header
commits to, and only that node claimed the slot. Failing over to another
node would hand a signed block to a builder that never saw the bid, so
the fallback remembers which node produced the latest blinded block and
publishes it there alone, returning that node's answer unchanged so the
caller can tell a refusal from an unconfirmed send.
A builder only builds for a validator that registered a fee recipient and
gas limit, signed under the builder domain (genesis fork version, zero
validators root) so it verifies on any fork. RegistrationService caches
each signature until the settings change, since builders keep the newest
timestamp and a fresh signature per epoch would cost a BLS signature per
validator for nothing, re-sends the full set every epoch because nodes
forget registrations, and stops at gloas where proposer preferences
replace them. Failures are logged and counted, never propagated: a missed
registration costs at most a builder block, not a duty.
The pieces for the builder flow (blinded blocks, registrations, publishing
on the producing node) had no caller. This wires them: the proposal service
sends a builder_boost_factor, signs a blinded block under the same root and
guard as a full one, and skips the fee-recipient comparison since the header
names the builder's address. A publish the node fails to confirm is reported
as unconfirmed rather than a missed proposal, because the relay may still
publish the block and a retry would be an equivocation.

Registrations are sent from a spawned task at each epoch refresh and for
changed keys every slot, so a slow node never delays duties. Three flags
(--builder-proposals, --builder-boost-factor, --prefer-builder-proposals)
configure it, the latter two requiring the first.
Each half of the builder flow was tested alone: the node's routes against
the mock relay, and the client against a mock node. Nothing checked that
the two agree on the wire. These tests run the client's own HttpBeaconNode
and SigningContext against this node's routes with a Builder layered over
the mock relay: a registration the relay verifies, a blinded block produced
under a boost factor and published, and the two cases that must stay on a
full block (builder proposals off, relay down).

The builder world helpers from proposal.rs are exposed to the test tree
instead of copied.
Operators need to know the three new flags and their constraints, the four
counters that show whether registrations and blinded blocks are landing, and
the three places the client departs from the builder flow (no failover for a
blinded publish, no slashing record beyond the in-process guard, no
registrations from gloas), since the first one decides what an unconfirmed
publish means for a proposal.
The beacon node waits up to 5 s on the relay, so the duty loop's budget
could fire first and count a block that was signed and handed to the
builder as a failed proposal. The blinded publish now runs on its own
task, which logs and counts its real outcome, and the loop treats an
overrun after a blinded signing as neither a failure nor a delay sample.
An unconfirmed publish no longer feeds blocks_proposed or the publication
delay either.

Builder registrations are signed on the blocking pool in chunks that take
the store lock per chunk, are spawned only after a refresh that succeeded
(so a failing refresh cannot respawn a full send every slot), and the
per-slot check returns on one atomic read of a settings generation counter
unless a keymanager write moved it. Removed keys are pruned from the cache.

Also tightens the blinded publish's error contract (a connect error and a
503 are definite refusals), gives register_validators its own timeout,
makes the tests sensitive to the boost factor and the builder domain, and
moves the builder test world into a test_support module.
…ish task

Once the duty loop's budget fires, nothing awaits the blinded publish, so a
4xx that arrived afterwards was silent and a late success never reached the
publication delay histogram. The task now logs and counts a definite refusal
as a failed proposal and observes the delay itself (the service takes the
slot clock for this), and the loop skips both for a block it knows was
signed blinded so nothing is counted twice.

A test drops a proposal mid-publish and checks the block is still sent.
The keymanager table still called `gas_limit` "stored and served only"
because "there is no builder flow"; with `--builder-proposals` a changed
value goes out in the next slot's builder registration, and so does a
changed fee recipient, which an ethlambda node then applies. The README's
validator flag table, rewritten on main by #716, gains
`--builder-proposals`.
@MegaRedHand
MegaRedHand force-pushed the feat/beacon-builder-vc branch from a940907 to 353fb90 Compare October 29, 2026 20:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant