diff --git a/CLAUDE.md b/CLAUDE.md index a775bae50..aa67c4bca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -588,6 +588,17 @@ default** on its two SSZ endpoints and bare integers in JSON, because `checkpoint_sync.rs` and other clients' lean sync read those bytes and may send no `Accept` at all; `/lean/v0/blocks/finalized` now answers JSON when asked for it by name. +With `--builder`, the beacon surface also runs the **fulu builder flow +(MEV-Boost)**. `register_validator` forwards registrations to the builder +(`beacon/registrations.rs`, client in `beacon/builder_client/`). +`produceBlockV3` asks for the builder's bid alongside the local build and +serves a blinded block when the bid passes its checks and wins on value +(`beacon/builder_flow/`: bid checks, circuit breaker, boost-factor choice). +`blinded_blocks` hands the signed block back for the builder to publish, with +no local fallback, since a second block for the slot would be an equivocation. Gloas +has no blinded blocks; its builder market is separate (the **Gloas** bullet +under Beacon Chain STF). + See [`docs/rpc.md`](docs/rpc.md) for the full reference: CLI flags and defaults, the lean API endpoints (health, finalized state/block, justified checkpoint, blocks by root/slot, fork-choice tree + D3.js UI, runtime aggregator toggle), the Beacon API endpoints and the three ids they refuse, the metrics/debug endpoints (Prometheus `/metrics`, jemalloc heap profiling), the Hive test-driver endpoints, plus request/response shapes, status codes, and content types. ## Beacon Chain types (`crates/common/types/src/beacon/`) diff --git a/README.md b/README.md index 116373eaf..d9cc433a6 100644 --- a/README.md +++ b/README.md @@ -312,6 +312,7 @@ what they sign. | `--checkpoint-sync-url` | Where the anchor comes from. Required on a fresh data directory for a built-in network, since those never start from genesis. A resumable data directory is used before it; a network loaded from a directory anchors at its own `genesis.ssz` when no URL is given | | `--node-key` | Optional, but worth persisting: the columns this node custodies are a function of its node id, so without a key file it changes identity, and custody set, on every restart | | `--execution-endpoint`, `--execution-jwt-secret` | Optional Engine API pairing, given together. Without them, blocks import without payload validation | +| `--builder` | Optional builder API endpoint, normally MEV-Boost. With it, a fulu proposal from a validator registered through this node gets a blinded block when the builder's bid wins; without it every block is built locally | A built-in network needs nothing else on disk. Its `eth-clients` `config.yaml` and bootnode list, and the two genesis values the fork digest is computed from, diff --git a/bin/ethlambda/src/cli.rs b/bin/ethlambda/src/cli.rs index 4101c56a5..5fc1a57f1 100644 --- a/bin/ethlambda/src/cli.rs +++ b/bin/ethlambda/src/cli.rs @@ -325,6 +325,21 @@ pub(crate) struct MainnetOptions { value_parser = clap::value_parser!(u64).range(1..), )] pub(crate) builder_header_timeout: u64, + + /// Build locally, skipping the builder, after more than this many + /// consecutive empty slots right before the proposal (the circuit breaker). + #[arg(long, default_value_t = ethlambda_rpc::FallbackThresholds::DEFAULT.skips, requires = "builder")] + pub(crate) builder_fallback_skips: u64, + + /// Build locally after more than this many empty slots in the epoch's + /// worth of slots before the proposal. + #[arg(long, default_value_t = ethlambda_rpc::FallbackThresholds::DEFAULT.skips_per_epoch, requires = "builder")] + pub(crate) builder_fallback_skips_per_epoch: u64, + + /// Build locally when the proposal is more than this many epochs past the + /// finalized checkpoint. + #[arg(long, default_value_t = ethlambda_rpc::FallbackThresholds::DEFAULT.epochs_since_finalization, requires = "builder")] + pub(crate) builder_fallback_epochs_since_finalization: u64, } /// Flags only the lean chain takes. @@ -1067,6 +1082,58 @@ mod tests { } } + #[test] + fn the_fallback_flags_parse_and_default_to_lighthouses() { + let beacon = parse_beacon(beacon_args()); + assert_eq!(beacon.mainnet.builder_fallback_skips, 3); + assert_eq!(beacon.mainnet.builder_fallback_skips_per_epoch, 8); + assert_eq!(beacon.mainnet.builder_fallback_epochs_since_finalization, 3); + + let mut args = beacon_args(); + args.extend([ + "--builder", + "http://127.0.0.1:18550", + "--builder-fallback-skips", + "5", + "--builder-fallback-skips-per-epoch", + "12", + "--builder-fallback-epochs-since-finalization", + "2", + ]); + let beacon = parse_beacon(args); + assert_eq!(beacon.mainnet.builder_fallback_skips, 5); + assert_eq!(beacon.mainnet.builder_fallback_skips_per_epoch, 12); + assert_eq!(beacon.mainnet.builder_fallback_epochs_since_finalization, 2); + } + + #[test] + fn each_fallback_flag_needs_a_builder() { + for flag in [ + "--builder-fallback-skips", + "--builder-fallback-skips-per-epoch", + "--builder-fallback-epochs-since-finalization", + ] { + let mut args = beacon_args(); + args.extend([flag, "5"]); + try_parse_from(args).expect_err(flag); + } + } + + #[test] + fn a_bad_fallback_value_is_refused() { + for flag in [ + "--builder-fallback-skips", + "--builder-fallback-skips-per-epoch", + "--builder-fallback-epochs-since-finalization", + ] { + for value in ["-1", "many"] { + let mut args = beacon_args(); + args.extend(["--builder", "http://127.0.0.1:18550", flag, value]); + try_parse_from(args).expect_err(flag); + } + } + } + #[test] fn a_zero_header_timeout_is_refused() { let mut args = beacon_args(); diff --git a/bin/ethlambda/src/main.rs b/bin/ethlambda/src/main.rs index 813055209..5f76f9c52 100644 --- a/bin/ethlambda/src/main.rs +++ b/bin/ethlambda/src/main.rs @@ -465,6 +465,14 @@ async fn run_node(options: Options) -> eyre::Result<()> { }) .transpose()?, }; + let builder_fallback = match &network { + Network::Lean(_) => ethlambda_rpc::FallbackThresholds::DEFAULT, + Network::Mainnet { mainnet, .. } => ethlambda_rpc::FallbackThresholds { + skips: mainnet.builder_fallback_skips, + skips_per_epoch: mainnet.builder_fallback_skips_per_epoch, + epochs_since_finalization: mainnet.builder_fallback_epochs_since_finalization, + }, + }; if let Some(builder) = builder.clone() { tokio::spawn(async move { let _ = builder @@ -872,6 +880,7 @@ async fn run_node(options: Options) -> eyre::Result<()> { client_version: version::engine_client_version(), identity: rpc_identity, builder, + builder_fallback, }, local_peer_id, rpc_shutdown, diff --git a/crates/blockchain/state_transition/src/beacon/stf/mod.rs b/crates/blockchain/state_transition/src/beacon/stf/mod.rs index 3dba6eef8..17672afd1 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/mod.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/mod.rs @@ -71,7 +71,7 @@ use crate::beacon::helpers::accessors::{ActiveBalanceCache, CommitteeCache, get_ use crate::beacon::helpers::misc::compute_signing_root; use crate::beacon::lean_state_unreachable; use crate::beacon::preset; -use crate::beacon::primitives::{HashTreeRoot as _, Slot}; +use crate::beacon::primitives::{BlsSignature, HashTreeRoot as _, Root, Slot, ValidatorIndex}; use crate::beacon::{bls, config::Config, constants}; /// What the execution layer would answer for a block's payload. @@ -212,12 +212,31 @@ pub fn verify_block_signature( state: &BeaconState, signed_block: &containers::SignedBeaconBlock, ) -> bool { - let Ok(proposer) = state.validator(signed_block.proposer_index()) else { + verify_proposer_signature( + state, + signed_block.proposer_index(), + signed_block.message_hash_tree_root(), + &signed_block.signature(), + ) +} + +/// Whether `signature` is `proposer_index`'s over the block whose root is +/// `block_root`, under `state`'s proposer domain. +/// +/// A blinded block has the root of its full block, so one check serves +/// both. +pub fn verify_proposer_signature( + state: &BeaconState, + proposer_index: ValidatorIndex, + block_root: Root, + signature: &BlsSignature, +) -> bool { + let Ok(proposer) = state.validator(proposer_index) else { return false; }; let domain = get_domain(state, constants::DOMAIN_BEACON_PROPOSER, None); - let signing_root = compute_signing_root(signed_block.message_hash_tree_root(), domain); - bls::verify(&proposer.pubkey, signing_root, &signed_block.signature()) + let signing_root = compute_signing_root(block_root, domain); + bls::verify(&proposer.pubkey, signing_root, signature) } /// Advances the state to `slot`, running epoch processing at each boundary and diff --git a/crates/net/engine/src/building.rs b/crates/net/engine/src/building.rs index 6ff109248..e7ac9559f 100644 --- a/crates/net/engine/src/building.rs +++ b/crates/net/engine/src/building.rs @@ -118,6 +118,9 @@ pub struct BuiltPayload { pub blobs_bundle: BlobsBundle, /// The EIP-7685 request list, each entry its type byte then its data. pub execution_requests: Vec>, + /// The engine's `shouldOverrideBuilder`: it wants this payload built + /// locally whatever a builder bid pays. Absent from an answer means false. + pub should_override_builder: bool, } /// `engine_getPayloadV6`'s answer, decoded: [`BuiltPayload`] with Amsterdam's @@ -151,6 +154,8 @@ pub(crate) struct GetPayloadV5Response { block_value: String, blobs_bundle: BlobsBundleJson, #[serde(default)] + should_override_builder: bool, + #[serde(default)] execution_requests: Vec, } @@ -259,6 +264,7 @@ impl TryFrom for BuiltPayload { block_value: parse_uint256(&response.block_value)?, blobs_bundle, execution_requests: decode_requests(&response.execution_requests)?, + should_override_builder: response.should_override_builder, }) } } @@ -454,6 +460,29 @@ mod tests { assert_eq!(built.execution_requests, vec![vec![0x00, 0xaa]]); } + #[test] + fn a_v5_answer_carries_should_override_builder_and_defaults_it_to_false() { + let mut json = response_json(); + json["shouldOverrideBuilder"] = true.into(); + let response: GetPayloadV5Response = serde_json::from_value(json).unwrap(); + assert!( + BuiltPayload::try_from(response) + .unwrap() + .should_override_builder + ); + + let mut json = response_json(); + json.as_object_mut() + .unwrap() + .remove("shouldOverrideBuilder"); + let response: GetPayloadV5Response = serde_json::from_value(json).unwrap(); + assert!( + !BuiltPayload::try_from(response) + .unwrap() + .should_override_builder + ); + } + #[test] fn a_wrong_width_hash_is_a_decode_error() { let mut json = response_json(); diff --git a/crates/net/rpc/src/beacon/blinded_blocks.rs b/crates/net/rpc/src/beacon/blinded_blocks.rs new file mode 100644 index 000000000..97262ebd2 --- /dev/null +++ b/crates/net/rpc/src/beacon/blinded_blocks.rs @@ -0,0 +1,251 @@ +//! `POST /eth/v2/beacon/blinded_blocks` (fulu), and the guard both publish +//! endpoints share against signing two blocks for one slot. +//! +//! A blinded block goes to the builder, which publishes the full block and +//! its data columns itself (`submitBlindedBlockV2`). The block then comes +//! back over gossip and imports like any other. Nothing is gossiped or +//! imported here. + +use std::collections::BTreeMap; +use std::sync::{Arc, Mutex}; + +use axum::Extension; +use axum::body::Bytes; +use axum::extract::State; +use axum::http::{HeaderMap, StatusCode}; +use axum::response::{IntoResponse, Response}; +use ethlambda_state_transition::beacon::block_production::advance_to_slot; +use ethlambda_state_transition::beacon::helpers::accessors::get_beacon_proposer_index; +use ethlambda_state_transition::beacon::stf::verify_proposer_signature; +use ethlambda_storage::Store; +use ethlambda_types::ShortRoot; +use ethlambda_types::beacon::builder::SignedBlindedBeaconBlock; +use ethlambda_types::beacon::fork::ForkName; +use ethlambda_types::beacon::primitives::{HashTreeRoot as _, Root, Slot}; +use ethlambda_types::beacon::signing::compute_start_slot_at_epoch; +use tracing::{error, info, warn}; + +use super::builder_client::BuilderError; +use super::builder_flow::Builder; +use super::proposal::require_fulu_slot; +use super::{ApiError, BodyEncoding}; + +/// The block root this node published or handed to the builder for each +/// slot, so a second, different block for a slot is refused instead of +/// broadcast. Pruned below the finalized slot. +#[derive(Debug, Clone, Default)] +pub(crate) struct PublishedProposals(Arc>>); + +/// Why a slot could not be claimed. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum ClaimError { + /// The slot is below the finalized one, where claims are forgotten, so a + /// second block could not be told from the first. + Finalized, + /// This node already published another block for the slot. + Taken(Root), +} + +impl ClaimError { + /// Logs the refusal, naming the root that holds the slot. + pub(crate) fn log(&self, slot: Slot) { + match self { + Self::Finalized => warn!(slot, "Refusing a block for a finalized slot"), + Self::Taken(root) => warn!( + slot, + existing = %ShortRoot(&root.0), + "Refusing a second, different block for the slot" + ), + } + } + + /// The 400 a publish endpoint answers with. + pub(crate) fn into_api_error(self) -> ApiError { + match self { + Self::Finalized => ApiError::BadRequest("the slot is already finalized"), + Self::Taken(_) => { + ApiError::BadRequest("this node already published a different block for the slot") + } + } + } +} + +impl PublishedProposals { + /// Claims `slot` for `root`, or refuses: below `finalized_slot` (a slot + /// equal to it is still held, as `split_off` keeps it), or with the root + /// already claimed. The same root again is a retry, not an equivocation. + pub(crate) fn claim( + &self, + slot: Slot, + root: Root, + finalized_slot: Slot, + ) -> Result<(), ClaimError> { + if slot < finalized_slot { + return Err(ClaimError::Finalized); + } + let mut published = self.0.lock().expect("published proposals lock poisoned"); + *published = published.split_off(&finalized_slot); + match published.get(&slot) { + Some(&claimed) if claimed != root => Err(ClaimError::Taken(claimed)), + Some(_) => Ok(()), + None => { + published.insert(slot, root); + Ok(()) + } + } + } +} + +/// The finalized checkpoint's start slot, below which no claim matters. +pub(crate) fn finalized_slot(store: &Store) -> Slot { + compute_start_slot_at_epoch(store.beacon_finalized_checkpoint().epoch) +} + +/// Checked like `post_block`'s fulu path, before anything leaves this node: +/// the parent is held, the block is after it, its proposer is the one the +/// parent's state names for the slot, and the proposer's signature verifies. +/// Then the slot is claimed, and the block goes to the builder. +/// +/// The submission runs on its own task. Once the proposer has signed it is +/// committed, so a validator client that hangs up must not cancel the +/// reveal. The answer is the builder's: 202 becomes 200 and a 400 is passed +/// through. Anything else is a 500, with no local fallback, since a second +/// block would be an equivocation. +pub(crate) async fn post_blinded_block( + State(store): State, + builder: Option>>, + Extension(published): Extension, + headers: HeaderMap, + body: Bytes, +) -> Response { + let Some(Extension(builder)) = builder else { + return ApiError::Internal("no builder is configured (--builder)").into_response(); + }; + let fork = headers + .get("eth-consensus-version") + .and_then(|value| value.to_str().ok()) + .and_then(ForkName::parse); + if fork != Some(ForkName::Fulu) { + return ApiError::BadRequest("Eth-Consensus-Version must be fulu").into_response(); + } + let encoding = match BodyEncoding::from_headers(&headers) { + Ok(encoding) => encoding, + Err(err) => return err.into_response(), + }; + let Some(signed) = encoding.decode::(&body) else { + return ApiError::BadRequest("the body is not a fulu SignedBlindedBeaconBlock") + .into_response(); + }; + let slot = signed.message.slot; + if let Err(err) = require_fulu_slot( + &store.config(), + slot, + "blinded blocks are accepted for fulu slots only", + ) { + return err.into_response(); + } + let Some(parent_state) = store.get_state(&signed.message.parent_root).ok().flatten() else { + return ApiError::BadRequest("the block's parent is not known to this node") + .into_response(); + }; + let config = store.config(); + let checked = tokio::task::spawn_blocking(move || { + if slot <= parent_state.slot() { + return Err("the block is not after its parent"); + } + let state = advance_to_slot(&parent_state, slot, &config) + .map_err(|_| "advancing the parent state failed")?; + let proposer = get_beacon_proposer_index(&state).map_err(|_| "no proposer for the slot")?; + if signed.message.proposer_index != proposer { + return Err("wrong proposer for the slot"); + } + let root = signed.message.hash_tree_root(); + if !verify_proposer_signature(&state, proposer, root, &signed.signature) { + return Err("invalid block signature"); + } + Ok((signed, root)) + }) + .await; + let (signed, root) = match checked { + Ok(Ok(checked)) => checked, + Ok(Err(message)) => return ApiError::BadRequest(message).into_response(), + Err(err) => { + error!(%err, "Checking a published blinded block failed"); + return ApiError::Internal("checking the blinded block failed").into_response(); + } + }; + if let Err(err) = published.claim(slot, root, finalized_slot(&store)) { + err.log(slot); + return err.into_api_error().into_response(); + } + + // The outcome is logged inside the task: once the proposer has signed, a + // validator client that hangs up drops this handler, and the reveal's + // result would otherwise go unrecorded. A block the builder does not take + // is a missed slot, so those are errors. + let client = builder.client.clone(); + let submitted = tokio::spawn(async move { + let result = client.submit_blinded_block(&signed, ForkName::Fulu).await; + match &result { + Ok(()) => info!(slot, "The builder took the blinded block"), + Err(BuilderError::BadRequest(message)) => { + error!(slot, %message, "The builder refused the signed blinded block") + } + Err(err) => error!(slot, %err, "The builder did not take the signed blinded block"), + } + result + }) + .await; + match submitted { + Ok(Ok(())) => StatusCode::OK.into_response(), + Ok(Err(BuilderError::BadRequest(message))) => { + ApiError::BadRequestDetail(message).into_response() + } + Ok(Err(_)) => ApiError::Internal("the builder did not take the block").into_response(), + Err(err) => { + error!(slot, %err, "Submitting the blinded block failed"); + ApiError::Internal("submitting the block failed").into_response() + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_slot_is_claimed_once_and_a_retry_of_the_same_root_passes() { + let published = PublishedProposals::default(); + let first = Root::repeat_byte(1); + assert_eq!(published.claim(10, first, 0), Ok(())); + assert_eq!(published.claim(10, first, 0), Ok(())); + assert_eq!( + published.claim(10, Root::repeat_byte(2), 0), + Err(ClaimError::Taken(first)) + ); + } + + #[test] + fn a_claim_below_the_finalized_slot_is_refused() { + let published = PublishedProposals::default(); + published.claim(10, Root::repeat_byte(1), 0).unwrap(); + assert_eq!( + published.claim(10, Root::repeat_byte(2), 11), + Err(ClaimError::Finalized) + ); + assert_eq!( + published.claim(10, Root::repeat_byte(1), 11), + Err(ClaimError::Finalized) + ); + } + + #[test] + fn the_finalized_slot_itself_can_still_be_claimed() { + let published = PublishedProposals::default(); + assert_eq!(published.claim(11, Root::repeat_byte(1), 11), Ok(())); + assert_eq!( + published.claim(11, Root::repeat_byte(2), 11), + Err(ClaimError::Taken(Root::repeat_byte(1))) + ); + } +} diff --git a/crates/net/rpc/src/beacon/builder_client/mock_relay.rs b/crates/net/rpc/src/beacon/builder_client/mock_relay.rs index 92c6d2281..219ce3f89 100644 --- a/crates/net/rpc/src/beacon/builder_client/mock_relay.rs +++ b/crates/net/rpc/src/beacon/builder_client/mock_relay.rs @@ -187,15 +187,25 @@ impl MockRelay { /// `message` as this relay sends it: naming the relay's key and signed /// with it under the builder domain. - pub(crate) fn sign_bid(&self, mut message: BuilderBid) -> SignedBuilderBid { - message.pubkey = self.pubkey(); + pub(crate) fn sign_bid(&self, message: BuilderBid) -> SignedBuilderBid { let genesis_fork_version = self.state.lock().unwrap().genesis_fork_version; - let domain = compute_builder_domain(genesis_fork_version); - let root = compute_signing_root(message.hash_tree_root(), domain); - SignedBuilderBid { - message, - signature: sign_for(RELAY_KEY, root), - } + signed_bid(message, genesis_fork_version) + } +} + +/// `message` signed by the mock relay's key under the builder domain of the +/// chain whose genesis fork version is `genesis_fork_version`, naming that +/// key. +pub(crate) fn signed_bid( + mut message: BuilderBid, + genesis_fork_version: Version, +) -> SignedBuilderBid { + message.pubkey = BlsPubkey(secret_key_for(RELAY_KEY).sk_to_pk().to_bytes()); + let domain = compute_builder_domain(genesis_fork_version); + let root = compute_signing_root(message.hash_tree_root(), domain); + SignedBuilderBid { + message, + signature: sign_for(RELAY_KEY, root), } } diff --git a/crates/net/rpc/src/beacon/builder_flow/bid.rs b/crates/net/rpc/src/beacon/builder_flow/bid.rs new file mode 100644 index 000000000..88686a98f --- /dev/null +++ b/crates/net/rpc/src/beacon/builder_flow/bid.rs @@ -0,0 +1,290 @@ +//! Checks on a builder's bid before it may stand against the local payload. +//! +//! Everything that can be checked without the payload itself. The header +//! fields the state transition checks (parent hash, prev_randao, timestamp, +//! withdrawals, blob count) are checked here too, before +//! `assemble_blinded_block` repeats them as a backstop, so a refusal has a +//! reason to count and costs no state transition. What is left unchecked +//! (the payload's execution validity, its block hash and the fields only an +//! execution client could verify) is trusted to the relay, as every client +//! does: the payload is revealed only after the proposer signs. + +use ethlambda_state_transition::beacon::block_production::PayloadInputs; +use ethlambda_state_transition::beacon::bls; +use ethlambda_state_transition::beacon::gossip::execution_payload_bid::is_gas_limit_target_compatible; +use ethlambda_types::beacon::builder::SignedBuilderBid; +use ethlambda_types::beacon::containers::capella; +use ethlambda_types::beacon::primitives::{HashTreeRoot as _, Version}; +use ethlambda_types::beacon::signing::{compute_builder_domain, compute_signing_root}; + +use crate::beacon::bid_selection::wei_u128; + +/// EIP-4844's blob gas per blob. +pub(crate) const GAS_PER_BLOB: u64 = 1 << 17; + +/// What a bid is checked against, all from the proposal state and the +/// proposer's registration. +pub(crate) struct BidContext<'a> { + pub(crate) genesis_fork_version: Version, + pub(crate) inputs: &'a PayloadInputs, + /// The block number of the payload the block builds on. + pub(crate) parent_block_number: u64, + /// The gas limit of the payload the block builds on. + pub(crate) parent_gas_limit: u64, + /// The gas limit the proposer registered. + pub(crate) target_gas_limit: u64, + pub(crate) max_blobs: u64, +} + +/// Why a bid was refused. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Rejection { + ZeroValue, + ParentHash, + PrevRandao, + Timestamp, + BlockNumber, + Withdrawals, + TooManyBlobs, + BlobGas, + GasLimit, + Signature, +} + +impl Rejection { + /// The metric label. + pub(crate) fn label(self) -> &'static str { + match self { + Self::ZeroValue => "zero_value", + Self::ParentHash => "parent_hash", + Self::PrevRandao => "prev_randao", + Self::Timestamp => "timestamp", + Self::BlockNumber => "block_number", + Self::Withdrawals => "withdrawals", + Self::TooManyBlobs => "too_many_blobs", + Self::BlobGas => "blob_gas", + Self::GasLimit => "gas_limit", + Self::Signature => "signature", + } + } +} + +/// Whether `bid` is acceptable for the proposal `context` describes. +/// Cheap checks first; the signature, the one that costs a pairing, last. +/// +/// The bid's fork label is not checked here: an electra bid at a fulu slot is +/// accepted on purpose (the containers are identical, so a lagging relay's bid +/// stays usable), and `BuilderClient` already refuses every other fork. +pub(crate) fn validate(bid: &SignedBuilderBid, context: &BidContext) -> Result<(), Rejection> { + let message = &bid.message; + let header = &message.header; + if wei_u128(&message.value) == 0 { + return Err(Rejection::ZeroValue); + } + if header.parent_hash != context.inputs.parent_hash { + return Err(Rejection::ParentHash); + } + if header.prev_randao != context.inputs.prev_randao { + return Err(Rejection::PrevRandao); + } + if header.timestamp != context.inputs.timestamp { + return Err(Rejection::Timestamp); + } + // Against the parent's number rather than the local payload's, unlike + // Lighthouse: the parent is always known, even when the local build failed. + if context.parent_block_number.checked_add(1) != Some(header.block_number) { + return Err(Rejection::BlockNumber); + } + let withdrawals: capella::Withdrawals = context + .inputs + .withdrawals + .clone() + .try_into() + .map_err(|_| Rejection::Withdrawals)?; + if header.withdrawals_root != withdrawals.hash_tree_root() { + return Err(Rejection::Withdrawals); + } + let blobs = message.blob_kzg_commitments.len() as u64; + if blobs > context.max_blobs { + return Err(Rejection::TooManyBlobs); + } + if header.blob_gas_used != blobs * GAS_PER_BLOB { + return Err(Rejection::BlobGas); + } + if !is_gas_limit_target_compatible( + context.parent_gas_limit, + header.gas_limit, + context.target_gas_limit, + ) { + return Err(Rejection::GasLimit); + } + let domain = compute_builder_domain(context.genesis_fork_version); + let root = compute_signing_root(message.hash_tree_root(), domain); + if !bls::verify(&message.pubkey, root, &bid.signature) { + return Err(Rejection::Signature); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use ethlambda_types::beacon::builder::BuilderBid; + use ethlambda_types::beacon::containers::capella::Withdrawal; + use ethlambda_types::beacon::containers::deneb::ExecutionPayloadHeader; + use ethlambda_types::beacon::containers::electra::{self, ExecutionRequests}; + use ethlambda_types::beacon::primitives::{ + BlsPubkey, Bytes32, ExecutionAddress, ExecutionBlockHash, KzgCommitment, Uint256, + }; + + use ethlambda_state_transition::beacon::helpers::test_state::secret_key_for; + + use super::*; + use crate::beacon::builder_client::mock_relay::signed_bid; + + const GENESIS_FORK_VERSION: Version = [0x10, 0x00, 0x00, 0x38]; + const PARENT_BLOCK_NUMBER: u64 = 99; + const PARENT_GAS_LIMIT: u64 = 30_000_000; + const TARGET_GAS_LIMIT: u64 = 40_000_000; + /// Lighthouse's `expected_gas_limit(30M, 40M)`: one adjustment step up. + const NEXT_GAS_LIMIT: u64 = 30_029_295; + + fn inputs() -> PayloadInputs { + PayloadInputs { + timestamp: 1_700_000_012, + prev_randao: Bytes32::repeat_byte(2), + withdrawals: vec![Withdrawal { + index: 1, + validator_index: 2, + address: ExecutionAddress::repeat_byte(4), + amount: 5, + }], + parent_hash: ExecutionBlockHash::repeat_byte(3), + } + } + + fn context(inputs: &PayloadInputs) -> BidContext<'_> { + BidContext { + genesis_fork_version: GENESIS_FORK_VERSION, + inputs, + parent_block_number: PARENT_BLOCK_NUMBER, + parent_gas_limit: PARENT_GAS_LIMIT, + target_gas_limit: TARGET_GAS_LIMIT, + max_blobs: 9, + } + } + + /// A one-blob bid that passes every check against `inputs()`, with `edit` + /// applied before it is signed. + fn bid(edit: impl FnOnce(&mut BuilderBid)) -> SignedBuilderBid { + let inputs = inputs(); + let withdrawals: capella::Withdrawals = inputs.withdrawals.clone().try_into().unwrap(); + let mut header = + ExecutionPayloadHeader::from(&electra::BeaconBlockBody::empty().execution_payload); + header.parent_hash = inputs.parent_hash; + header.prev_randao = inputs.prev_randao; + header.timestamp = inputs.timestamp; + header.block_number = PARENT_BLOCK_NUMBER + 1; + header.withdrawals_root = withdrawals.hash_tree_root(); + header.gas_limit = NEXT_GAS_LIMIT; + header.blob_gas_used = GAS_PER_BLOB; + let mut message = BuilderBid { + header, + blob_kzg_commitments: vec![KzgCommitment::default()].try_into().unwrap(), + execution_requests: ExecutionRequests::default(), + value: Uint256::from(1_000_000_000u64), + pubkey: BlsPubkey([0; 48]), + }; + edit(&mut message); + signed_bid(message, GENESIS_FORK_VERSION) + } + + fn rejection(bid: &SignedBuilderBid) -> Option { + validate(bid, &context(&inputs())).err() + } + + #[test] + fn a_bid_that_matches_the_proposal_is_accepted() { + assert_eq!(rejection(&bid(|_| {})), None); + } + + #[test] + fn each_check_refuses_its_own_field() { + type Edit = fn(&mut BuilderBid); + let cases: [(Rejection, Edit); 9] = [ + (Rejection::ZeroValue, |bid| bid.value = Uint256::from(0u64)), + (Rejection::ParentHash, |bid| { + bid.header.parent_hash = ExecutionBlockHash::repeat_byte(9) + }), + (Rejection::PrevRandao, |bid| { + bid.header.prev_randao = Bytes32::repeat_byte(9) + }), + (Rejection::Timestamp, |bid| bid.header.timestamp += 1), + (Rejection::BlockNumber, |bid| bid.header.block_number += 1), + (Rejection::Withdrawals, |bid| { + bid.header.withdrawals_root = Bytes32::repeat_byte(9) + }), + (Rejection::TooManyBlobs, |bid| { + bid.blob_kzg_commitments = vec![KzgCommitment::default(); 10].try_into().unwrap(); + bid.header.blob_gas_used = 10 * GAS_PER_BLOB; + }), + (Rejection::BlobGas, |bid| bid.header.blob_gas_used = 0), + (Rejection::GasLimit, |bid| { + bid.header.gas_limit = TARGET_GAS_LIMIT + }), + ]; + for (expected, edit) in cases { + assert_eq!(rejection(&bid(edit)), Some(expected), "{expected:?}"); + } + } + + #[test] + fn a_block_number_that_overflows_is_refused() { + let inputs = inputs(); + let mut context = context(&inputs); + context.parent_block_number = u64::MAX; + let bid = bid(|bid| bid.header.block_number = 0); + assert_eq!(validate(&bid, &context), Err(Rejection::BlockNumber)); + } + + #[test] + fn a_bid_changed_after_signing_is_refused() { + let mut forged = bid(|_| {}); + forged.message.value = Uint256::from(2_000_000_000u64); + assert_eq!(rejection(&forged), Some(Rejection::Signature)); + } + + #[test] + fn a_bid_signed_by_a_key_other_than_the_one_it_names_is_refused() { + let mut forged = bid(|_| {}); + // The signature is the relay's, the message names another key. + forged.message.pubkey = BlsPubkey(secret_key_for(7).sk_to_pk().to_bytes()); + assert_eq!(rejection(&forged), Some(Rejection::Signature)); + } + + #[test] + fn a_bid_signed_under_another_genesis_fork_version_is_refused() { + let message = bid(|_| {}).message; + let wrong = signed_bid(message, [0x99, 0x00, 0x00, 0x99]); + assert_eq!(rejection(&wrong), Some(Rejection::Signature)); + } + + /// The gloas gossip predicate this reuses agrees with Lighthouse's + /// `expected_gas_limit` on its own test vectors. + #[test] + fn the_gas_limit_rule_matches_lighthouses_vectors() { + let vectors = [ + (30_000_000, 30_000_000, 30_000_000), + (30_000_000, 40_000_000, 30_029_295), + (30_029_295, 40_000_000, 30_058_619), + (30_058_619, 30_000_000, 30_029_266), + ]; + for (parent, target, expected) in vectors { + assert!(is_gas_limit_target_compatible(parent, expected, target)); + assert!(!is_gas_limit_target_compatible( + parent, + expected + 1, + target + )); + } + } +} diff --git a/crates/net/rpc/src/beacon/builder_flow/breaker.rs b/crates/net/rpc/src/beacon/builder_flow/breaker.rs new file mode 100644 index 000000000..07f5129b9 --- /dev/null +++ b/crates/net/rpc/src/beacon/builder_flow/breaker.rs @@ -0,0 +1,236 @@ +//! The circuit breaker: when the chain looks unhealthy, propose a local +//! block rather than a builder's. +//! +//! builder-specs leaves the conditions to the client; these follow +//! Lighthouse's. They read the proposal state alone, so two nodes holding +//! the same state agree: a slot `i` before the proposal slot was empty iff +//! `block_roots[i] == block_roots[i - 1]`, since `process_slots` carries the +//! previous root forward over an empty slot. + +use ethlambda_state_transition::beacon::helpers::accessors::get_block_root_at_slot; +use ethlambda_types::beacon::containers::BeaconState; +use ethlambda_types::beacon::preset::SLOTS_PER_EPOCH; +use ethlambda_types::beacon::primitives::Slot; +use ethlambda_types::beacon::signing::compute_epoch_at_slot; + +/// How unhealthy the chain may look before block production skips the +/// builder. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct FallbackThresholds { + /// The most consecutive empty slots allowed right before the proposal. + pub skips: u64, + /// The most empty slots allowed in the epoch's worth of slots before the + /// proposal. + pub skips_per_epoch: u64, + /// The most epochs the proposal may be past the finalized checkpoint. + pub epochs_since_finalization: u64, +} + +impl FallbackThresholds { + /// Lighthouse's defaults. + pub const DEFAULT: Self = Self { + skips: 3, + skips_per_epoch: 8, + epochs_since_finalization: 3, + }; +} + +impl Default for FallbackThresholds { + fn default() -> Self { + Self::DEFAULT + } +} + +/// Why the breaker kept the builder out. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Trip { + Syncing, + OptimisticHead, + /// Reads the state's realized `finalized_checkpoint`, not fork choice's. + NoFinality, + ConsecutiveSkips, + SkipsPerEpoch, +} + +impl Trip { + /// The metric label. + pub(crate) fn label(self) -> &'static str { + match self { + Self::Syncing => "syncing", + Self::OptimisticHead => "optimistic_head", + Self::NoFinality => "no_finality", + Self::ConsecutiveSkips => "consecutive_skips", + Self::SkipsPerEpoch => "skips_per_epoch", + } + } +} + +/// Whether the builder may be used for a block at `state`'s slot. +/// +/// `state` is the head state advanced to the proposal slot. `optimistic` and +/// `syncing` are the node's own view, which the state cannot know. +pub(crate) fn check( + state: &BeaconState, + thresholds: &FallbackThresholds, + optimistic: bool, + syncing: bool, +) -> Result<(), Trip> { + if syncing { + return Err(Trip::Syncing); + } + if optimistic { + return Err(Trip::OptimisticHead); + } + let slot = state.slot(); + let since_finalization = + compute_epoch_at_slot(slot).saturating_sub(state.finalized_checkpoint().epoch); + if since_finalization > thresholds.epochs_since_finalization { + return Err(Trip::NoFinality); + } + let consecutive = (1..slot).rev().take_while(|&i| is_empty(state, i)).count() as u64; + if consecutive > thresholds.skips { + return Err(Trip::ConsecutiveSkips); + } + let window_start = slot.saturating_sub(SLOTS_PER_EPOCH).max(1); + let in_window = (window_start..slot).filter(|&i| is_empty(state, i)).count() as u64; + if in_window > thresholds.skips_per_epoch { + return Err(Trip::SkipsPerEpoch); + } + Ok(()) +} + +/// Whether slot `i` (with `0 < i < state.slot()`) had no block. +fn is_empty(state: &BeaconState, i: Slot) -> bool { + match ( + get_block_root_at_slot(state, i), + get_block_root_at_slot(state, i - 1), + ) { + (Ok(root), Ok(previous)) => root == previous, + // Older than the state remembers: count it as filled, so a short + // history never trips the breaker on its own. + _ => false, + } +} + +#[cfg(test)] +mod tests { + use ethlambda_state_transition::beacon::helpers::test_state::with_signing_validators_at; + use ethlambda_types::beacon::fork::ForkName; + use ethlambda_types::beacon::primitives::{Epoch, H256}; + + use super::*; + + /// A fulu state at `slot` whose every earlier slot had a block except + /// those in `empty`, finalized at `finalized_epoch`. + fn state_at(slot: Slot, empty: &[Slot], finalized_epoch: Epoch) -> BeaconState { + let mut state = with_signing_validators_at(ForkName::Fulu, 8); + let BeaconState::Fulu(fulu) = &mut state else { + unreachable!("built as fulu") + }; + fulu.slot = slot; + fulu.finalized_checkpoint.epoch = finalized_epoch; + let history = fulu.block_roots.len() as u64; + let mut root = H256([0; 32]); + for i in 0..slot { + if !empty.contains(&i) { + let mut bytes = [0u8; 32]; + bytes[..8].copy_from_slice(&(i + 1).to_le_bytes()); + root = H256(bytes); + } + fulu.block_roots[(i % history) as usize] = root; + } + state + } + + const LENIENT_SYNC: (bool, bool) = (false, false); + + fn check_at(state: &BeaconState) -> Result<(), Trip> { + let (optimistic, syncing) = LENIENT_SYNC; + check(state, &FallbackThresholds::DEFAULT, optimistic, syncing) + } + + #[test] + fn a_healthy_chain_uses_the_builder() { + assert_eq!(check_at(&state_at(100, &[], 2)), Ok(())); + } + + #[test] + fn three_empty_slots_in_a_row_are_tolerated_and_four_are_not() { + assert_eq!(check_at(&state_at(100, &[97, 98, 99], 2)), Ok(())); + assert_eq!( + check_at(&state_at(100, &[96, 97, 98, 99], 2)), + Err(Trip::ConsecutiveSkips) + ); + } + + #[test] + fn eight_scattered_empty_slots_in_an_epoch_are_tolerated_and_nine_are_not() { + let eight = [70, 72, 74, 76, 78, 80, 82, 84]; + assert_eq!(check_at(&state_at(100, &eight, 2)), Ok(())); + let nine = [70, 72, 74, 76, 78, 80, 82, 84, 86]; + assert_eq!(check_at(&state_at(100, &nine, 2)), Err(Trip::SkipsPerEpoch)); + } + + #[test] + fn empty_slots_outside_the_epoch_window_do_not_count() { + let old = [10, 12, 14, 16, 18, 20, 22, 24, 26, 28]; + assert_eq!(check_at(&state_at(100, &old, 2)), Ok(())); + } + + #[test] + fn three_epochs_without_finality_are_tolerated_and_four_are_not() { + // Slot 200 is in epoch 6. + assert_eq!(check_at(&state_at(200, &[], 3)), Ok(())); + assert_eq!(check_at(&state_at(200, &[], 2)), Err(Trip::NoFinality)); + } + + #[test] + fn a_proposal_near_genesis_counts_no_slot_before_slot_one() { + assert_eq!(check_at(&state_at(1, &[], 0)), Ok(())); + assert_eq!(check_at(&state_at(5, &[], 0)), Ok(())); + // Slots 1 to 4 are all empty: four in a row, and slot 0 is never judged. + assert_eq!( + check_at(&state_at(5, &[1, 2, 3, 4], 0)), + Err(Trip::ConsecutiveSkips) + ); + assert_eq!(check_at(&state_at(4, &[1, 2, 3], 0)), Ok(())); + } + + #[test] + fn a_history_that_wraps_is_read_at_its_wrapped_index() { + let wrapped = 8192 + 100; + assert_eq!(check_at(&state_at(wrapped, &[], 300)), Ok(())); + let run = [wrapped - 4, wrapped - 3, wrapped - 2, wrapped - 1]; + assert_eq!( + check_at(&state_at(wrapped, &run, 300)), + Err(Trip::ConsecutiveSkips) + ); + } + + #[test] + fn a_slot_older_than_the_history_counts_as_filled() { + let slot = 8192 + 100; + let all_empty: Vec = (1..slot).collect(); + let state = state_at(slot, &all_empty, 300); + // Slot 1 is out of the state's reach, so its roots are not compared. + assert!(!is_empty(&state, 1)); + assert!(is_empty(&state, slot - 1)); + } + + #[test] + fn an_empty_run_that_does_not_end_right_before_the_proposal_is_not_consecutive() { + // Five empty slots, but slot 99 had a block. + assert_eq!(check_at(&state_at(100, &[90, 91, 92, 93, 94], 2)), Ok(())); + } + + #[test] + fn an_optimistic_head_or_a_syncing_node_keeps_the_builder_out() { + let state = state_at(100, &[], 2); + let thresholds = FallbackThresholds::DEFAULT; + assert_eq!( + check(&state, &thresholds, true, false), + Err(Trip::OptimisticHead) + ); + assert_eq!(check(&state, &thresholds, false, true), Err(Trip::Syncing)); + } +} diff --git a/crates/net/rpc/src/beacon/builder_flow/cache.rs b/crates/net/rpc/src/beacon/builder_flow/cache.rs new file mode 100644 index 000000000..8ccbb7ecb --- /dev/null +++ b/crates/net/rpc/src/beacon/builder_flow/cache.rs @@ -0,0 +1,139 @@ +//! The registrations this node forwarded to its builder, so block +//! production knows which proposers use the builder, and with what gas +//! limit. + +use std::collections::HashMap; +use std::sync::{Arc, Mutex}; + +use ethlambda_types::beacon::builder::ValidatorRegistrationV1; +use ethlambda_types::beacon::primitives::{BlsPubkey, Epoch}; + +/// How many epochs a registration lasts without being sent again. Validator +/// clients resend theirs every epoch, so two missed epochs mean the client +/// stopped using the builder, or stopped. +pub(crate) const REGISTRATION_LIFETIME_EPOCHS: Epoch = 2; + +/// Each validator's latest registration, keyed by its pubkey. +#[derive(Debug, Clone, Default)] +pub(crate) struct BuilderRegistrations(Arc>>); + +#[derive(Debug, Clone)] +struct Entry { + registration: ValidatorRegistrationV1, + /// The epoch it was last sent in. + seen: Epoch, +} + +impl BuilderRegistrations { + /// Records `registrations`, sent in `epoch`. + /// + /// One with an older timestamp than the registration held is ignored, as + /// builders ignore it; any other replaces it and restarts its lifetime. + /// Expired entries are dropped on the way, which bounds the map by the + /// registrations still being sent. + pub(crate) fn record<'a>( + &self, + registrations: impl IntoIterator, + epoch: Epoch, + ) { + let mut entries = self.0.lock().expect("builder registrations lock poisoned"); + entries.retain(|_, entry| !expired(entry, epoch)); + for registration in registrations { + match entries.get_mut(®istration.pubkey) { + Some(entry) if registration.timestamp < entry.registration.timestamp => {} + Some(entry) => { + entry.registration = registration.clone(); + entry.seen = epoch; + } + None => { + let entry = Entry { + registration: registration.clone(), + seen: epoch, + }; + entries.insert(registration.pubkey, entry); + } + } + } + } + + /// `pubkey`'s registration, if it is still current at `epoch`. + pub(crate) fn get(&self, pubkey: &BlsPubkey, epoch: Epoch) -> Option { + let entries = self.0.lock().expect("builder registrations lock poisoned"); + entries + .get(pubkey) + .filter(|entry| !expired(entry, epoch)) + .map(|entry| entry.registration.clone()) + } + + #[cfg(test)] + fn len(&self) -> usize { + self.0.lock().unwrap().len() + } +} + +fn expired(entry: &Entry, epoch: Epoch) -> bool { + epoch > entry.seen.saturating_add(REGISTRATION_LIFETIME_EPOCHS) +} + +#[cfg(test)] +mod tests { + use ethlambda_types::beacon::primitives::ExecutionAddress; + + use super::*; + + fn registration(key: u8, timestamp: u64, gas_limit: u64) -> ValidatorRegistrationV1 { + ValidatorRegistrationV1 { + fee_recipient: ExecutionAddress::repeat_byte(key), + gas_limit, + timestamp, + pubkey: BlsPubkey([key; 48]), + } + } + + #[test] + fn a_registration_lasts_two_epochs_without_a_resend() { + let cache = BuilderRegistrations::default(); + let held = registration(1, 100, 30_000_000); + cache.record([&held], 10); + assert_eq!(cache.get(&held.pubkey, 12), Some(held.clone())); + assert_eq!(cache.get(&held.pubkey, 13), None); + } + + #[test] + fn a_resend_restarts_the_lifetime() { + let cache = BuilderRegistrations::default(); + let held = registration(1, 100, 30_000_000); + cache.record([&held], 10); + cache.record([&held], 12); + assert_eq!(cache.get(&held.pubkey, 14), Some(held)); + } + + #[test] + fn a_newer_registration_replaces_the_held_one() { + let cache = BuilderRegistrations::default(); + let old = registration(1, 100, 30_000_000); + let new = registration(1, 200, 60_000_000); + cache.record([&old], 10); + cache.record([&new], 10); + assert_eq!(cache.get(&old.pubkey, 10), Some(new)); + } + + #[test] + fn an_older_registration_is_ignored_and_does_not_refresh() { + let cache = BuilderRegistrations::default(); + let held = registration(1, 200, 60_000_000); + let stale = registration(1, 100, 30_000_000); + cache.record([&held], 10); + cache.record([&stale], 12); + assert_eq!(cache.get(&held.pubkey, 12), Some(held.clone())); + assert_eq!(cache.get(&held.pubkey, 13), None); + } + + #[test] + fn expired_entries_are_dropped_on_the_next_record() { + let cache = BuilderRegistrations::default(); + cache.record([®istration(1, 100, 1)], 10); + cache.record([®istration(2, 100, 1)], 20); + assert_eq!(cache.len(), 1); + } +} diff --git a/crates/net/rpc/src/beacon/builder_flow/choice.rs b/crates/net/rpc/src/beacon/builder_flow/choice.rs new file mode 100644 index 000000000..fd5712b10 --- /dev/null +++ b/crates/net/rpc/src/beacon/builder_flow/choice.rs @@ -0,0 +1,146 @@ +//! Choosing between the local payload and a builder's. + +/// beacon-APIs' default `builder_boost_factor`: compare the two as they are. +pub(crate) const DEFAULT_BOOST_FACTOR: u64 = 100; + +/// Where the block's payload comes from. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Source { + Local, + Builder, +} + +impl Source { + /// The metric label. + pub(crate) fn label(self) -> &'static str { + match self { + Self::Local => "local", + Self::Builder => "builder", + } + } +} + +/// The local payload's side of the comparison. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct LocalOffer { + /// What it pays, in wei. + pub(crate) value: u128, + /// The execution client wants it built locally whatever a bid pays. + pub(crate) should_override_builder: bool, +} + +/// Which payload the block carries, or `None` when neither is usable. +/// +/// beacon-APIs' `builder_boost_factor`, taken literally: the local payload +/// wins iff `local >= factor * (bid / 100)`. So 100 compares the two as they +/// are, 0 always keeps the local payload, and a tie goes local. +/// A factor of `u64::MAX` always takes the builder, except that the execution +/// client's `shouldOverrideBuilder` still wins. When one side failed, the +/// other stands. +pub(crate) fn choose( + local: Option, + bid_value: Option, + boost_factor: u64, +) -> Option { + match (local, bid_value) { + (None, None) => None, + (Some(_), None) => Some(Source::Local), + (None, Some(_)) => Some(Source::Builder), + (Some(local), Some(_)) if local.should_override_builder => Some(Source::Local), + (Some(_), Some(_)) if boost_factor == u64::MAX => Some(Source::Builder), + (Some(local), Some(bid)) => { + let boosted = u128::from(boost_factor).saturating_mul(bid / 100); + Some(if local.value >= boosted { + Source::Local + } else { + Source::Builder + }) + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const ETH: u128 = 1_000_000_000_000_000_000; + + fn local(value: u128) -> Option { + Some(LocalOffer { + value, + should_override_builder: false, + }) + } + + #[test] + fn the_selection_table() { + let overriding = Some(LocalOffer { + value: 1, + should_override_builder: true, + }); + let cases = [ + ("neither", None, None, 100, None), + ("local only", local(1), None, 100, Some(Source::Local)), + ("bid only", None, Some(ETH), 100, Some(Source::Builder)), + ( + "bid pays more", + local(ETH), + Some(2 * ETH), + 100, + Some(Source::Builder), + ), + ( + "local pays more", + local(2 * ETH), + Some(ETH), + 100, + Some(Source::Local), + ), + ( + "a tie goes local", + local(ETH), + Some(ETH), + 100, + Some(Source::Local), + ), + ( + "factor 0 keeps local", + local(1), + Some(100 * ETH), + 0, + Some(Source::Local), + ), + ( + "factor 50 halves the bid", + local(ETH), + Some(2 * ETH), + 50, + Some(Source::Local), + ), + ( + "factor 200 doubles it", + local(ETH + 1), + Some(ETH), + 200, + Some(Source::Builder), + ), + ( + "u64::MAX takes the bid", + local(100 * ETH), + Some(1), + u64::MAX, + Some(Source::Builder), + ), + ( + "the override keeps local", + overriding, + Some(100 * ETH), + u64::MAX, + Some(Source::Local), + ), + ]; + for (name, local, bid, factor, expected) in cases { + assert_eq!(choose(local, bid, factor), expected, "{name}"); + } + } +} diff --git a/crates/net/rpc/src/beacon/builder_flow/mod.rs b/crates/net/rpc/src/beacon/builder_flow/mod.rs new file mode 100644 index 000000000..6fc76ae4e --- /dev/null +++ b/crates/net/rpc/src/beacon/builder_flow/mod.rs @@ -0,0 +1,41 @@ +//! The fulu builder flow's decisions: which proposers use the builder, when +//! the chain is too unhealthy to, whether a bid is acceptable, and which +//! payload goes into the block. + +pub(crate) mod bid; +pub(crate) mod breaker; +pub(crate) mod cache; +pub(crate) mod choice; + +use ethlambda_blockchain::SyncStatusController; + +use super::builder_client::BuilderClient; +use breaker::FallbackThresholds; +use cache::BuilderRegistrations; + +/// What the builder flow's handlers share. It is layered only when +/// `--builder` is set, so a handler reads it as +/// `Option>>`. A node without a builder, or a test +/// router that never layers it, takes the local path. +pub(crate) struct Builder { + pub(crate) client: BuilderClient, + pub(crate) registrations: BuilderRegistrations, + pub(crate) fallback: FallbackThresholds, + /// The node's sync status, which the circuit breaker reads. + pub(crate) sync_status: SyncStatusController, +} + +impl Builder { + pub(crate) fn new( + client: BuilderClient, + fallback: FallbackThresholds, + sync_status: SyncStatusController, + ) -> Self { + Self { + client, + registrations: BuilderRegistrations::default(), + fallback, + sync_status, + } + } +} diff --git a/crates/net/rpc/src/beacon/mod.rs b/crates/net/rpc/src/beacon/mod.rs index dc6af10c5..b1f9837f7 100644 --- a/crates/net/rpc/src/beacon/mod.rs +++ b/crates/net/rpc/src/beacon/mod.rs @@ -17,9 +17,11 @@ use crate::shared::block_id::IdError; pub(crate) mod bid_selection; pub(crate) mod bids; +pub(crate) mod blinded_blocks; pub(crate) mod blocks; pub(crate) mod builder_client; pub(crate) mod builder_config; +pub(crate) mod builder_flow; pub(crate) mod builders; pub(crate) mod config; pub(crate) mod envelopes; diff --git a/crates/net/rpc/src/beacon/proposal.rs b/crates/net/rpc/src/beacon/proposal.rs index 8dcd6115d..c5d0e66f8 100644 --- a/crates/net/rpc/src/beacon/proposal.rs +++ b/crates/net/rpc/src/beacon/proposal.rs @@ -3,9 +3,10 @@ //! execution client, and `POST /eth/v2/beacon/blocks` (`publishBlockV2`), the //! signed block back to gossip and import. //! -//! Only unblinded, locally built blocks: this node has no builder flow, so -//! `builder_boost_factor` is accepted and ignored, and every answer carries -//! `Eth-Execution-Payload-Blinded: false`. +//! Locally built blocks, or, with `--builder` set and a builder's bid that +//! validates and beats the local payload under `builder_boost_factor`, the +//! blinded block around that bid. Every answer says which in +//! `Eth-Execution-Payload-Blinded`. //! //! A payload's blobs travel with the block: production answers fulu's //! `BlockContents` (cell proofs and blobs beside the block), and publication @@ -25,6 +26,7 @@ use axum::{ response::{IntoResponse, Response}, routing::{get, post}, }; +use ethlambda_blockchain::metrics::SyncStatus; use ethlambda_engine::{ EngineClient, ForkchoiceStateV1, building::{BuiltPayload, PayloadAttributesV3}, @@ -33,9 +35,9 @@ use ethlambda_engine::{ use ethlambda_network_api::RpcToP2PRef; use ethlambda_state_transition::beacon::{ block_production::{ - BlockInputs, Operations, advance_to_slot, assemble_block, empty_sync_aggregate, - pack_attestations, pack_operations, parse_execution_requests, payload_inputs, - verified_sync_aggregate, + BlindedBlockInputs, BlockInputs, Operations, advance_to_slot, assemble_blinded_block, + assemble_block, empty_sync_aggregate, pack_attestations, pack_operations, + parse_execution_requests, payload_inputs, verified_sync_aggregate, }, bls, data_columns::{self, SidecarError}, @@ -49,15 +51,20 @@ use ethlambda_state_transition::beacon::{ use ethlambda_storage::Store; use ethlambda_types::{ beacon::{ + builder::{BlindedBeaconBlock, BuilderBid, ValidatorRegistrationV1}, config::Config, containers::{ self, BeaconState, + altair::SyncAggregate, deneb::Blob, - electra::{BeaconBlock, SignedBeaconBlock}, + electra::{Attestation, BeaconBlock, ExecutionRequests, SignedBeaconBlock}, }, fork::ForkName, preset, - primitives::{BlsSignature, Bytes32, ExecutionAddress, KzgProof, Slot}, + primitives::{ + BlsPubkey, BlsSignature, Bytes32, ExecutionAddress, ExecutionBlockHash, + HashTreeRoot as _, KzgProof, Slot, Uint256, ValidatorIndex, + }, signing::compute_epoch_at_slot, }, primitives::H256, @@ -66,11 +73,23 @@ use libssz::SszEncode as _; use libssz_derive::{SszDecode, SszEncode}; use libssz_types::SszList; use serde::Deserialize; -use tracing::{error, info, warn}; +use std::sync::Arc; +use tracing::{debug, error, info, warn}; +use crate::beacon::bid_selection::wei_u128; +use crate::beacon::blinded_blocks::{self, PublishedProposals, finalized_slot}; +use crate::beacon::builder_client::VersionedBid; +use crate::beacon::builder_flow::{ + Builder, + bid::{self, BidContext}, + breaker, + choice::{self, DEFAULT_BOOST_FACTOR, LocalOffer, Source}, +}; use crate::beacon::graffiti::{self, OwnVersion}; use crate::beacon::{ApiError, BodyEncoding, validator::FeeRecipients, validator::head}; +use crate::metrics::{inc_bid_rejection, inc_builder_fallback, inc_payload_source}; use crate::shared::content::{Encoding, ssz_response, with_consensus_version}; +use crate::shared::optimistic::block_is_optimistic; /// One KZG proof per cell of every blob, fulu's `kzg_proofs` bound. pub(crate) type CellKzgProofs = SszList< @@ -108,6 +127,12 @@ pub(crate) fn routes() -> Router { "/eth/v2/beacon/blocks", post(post_block).layer(DefaultBodyLimit::max(super::MAX_PUBLISH_BODY_BYTES)), ) + .route( + "/eth/v2/beacon/blinded_blocks", + post(blinded_blocks::post_blinded_block) + .layer(DefaultBodyLimit::max(super::MAX_PUBLISH_BODY_BYTES)), + ) + .layer(Extension(PublishedProposals::default())) } /// `POST /eth/v2/beacon/blocks`, `SignedBlockContents` (fulu) or a bare @@ -126,6 +151,7 @@ pub(crate) fn routes() -> Router { async fn post_block( State(store): State, Extension(p2p): Extension, + Extension(published): Extension, headers: HeaderMap, body: Bytes, ) -> Response { @@ -141,7 +167,7 @@ async fn post_block( Err(err) => return err.into_response(), }; if fork == Some(ForkName::Gloas) { - return post_gloas_block(&store, &p2p, encoding, &body).await; + return post_gloas_block(&store, &p2p, &published, encoding, &body).await; } let Some(contents) = encoding.decode::(&body) else { return ApiError::BadRequest("the body is not fulu SignedBlockContents").into_response(); @@ -227,6 +253,12 @@ async fn post_block( return ApiError::Internal("building the data column sidecars failed").into_response(); } }; + let root = signed_block.message.hash_tree_root(); + let slot = signed_block.message.slot; + if let Err(err) = published.claim(slot, root, finalized_slot(&store)) { + err.log(slot); + return err.into_api_error().into_response(); + } let block = containers::SignedBeaconBlock::Fulu(signed_block); let sidecars = sidecars .into_iter() @@ -250,6 +282,7 @@ async fn post_block( async fn post_gloas_block( store: &Store, p2p: &RpcToP2PRef, + published: &PublishedProposals, encoding: BodyEncoding, body: &[u8], ) -> Response { @@ -300,6 +333,12 @@ async fn post_gloas_block( Ok(Err(err)) => return err.into_response(), Err(_) => return ApiError::Internal("verifying the block failed").into_response(), } + let slot = block.slot(); + let root = block.message_hash_tree_root(); + if let Err(err) = published.claim(slot, root, finalized_slot(store)) { + err.log(slot); + return err.into_api_error().into_response(); + } // A gloas block carries no blobs of its own: its data columns travel with // the execution payload envelope, published separately, so there are no // sidecars to gossip with the block. @@ -336,6 +375,9 @@ struct ProduceQuery { /// A flag with no value, validated by [`randao_verification`]. #[serde(default)] skip_randao_verification: Option, + /// beacon-APIs' weight on the builder's bid, in percent; absent is 100. + #[serde(default)] + builder_boost_factor: Option, } /// Whether the block's RANDAO reveal is verified, from the request's @@ -372,7 +414,11 @@ pub(super) fn randao_verification( /// advances a state to it: advancing a fulu head into a gloas slot would run /// `upgrade_to_gloas` only to refuse the result. Every other fork is refused /// too, since the containers here are fulu's. -fn require_fulu_slot(config: &Config, slot: Slot, message: &'static str) -> Result<(), ApiError> { +pub(super) fn require_fulu_slot( + config: &Config, + slot: Slot, + message: &'static str, +) -> Result<(), ApiError> { match config.fork_at_epoch(compute_epoch_at_slot(slot)) { ForkName::Fulu => Ok(()), ForkName::Phase0 @@ -395,6 +441,7 @@ async fn get_block( Extension(sync_pool): Extension, Extension(fee_recipients): Extension, Extension(OwnVersion(own_version)): Extension, + builder: Option>>, headers: HeaderMap, ) -> Response { let Ok(slot) = slot.parse::() else { @@ -431,44 +478,111 @@ async fn get_block( &sync_pool, &fee_recipients, &own_version, + builder.as_ref().map(|Extension(builder)| builder.as_ref()), + query.builder_boost_factor.unwrap_or(DEFAULT_BOOST_FACTOR), slot, query.randao_reveal, BlockProcessingOptions { randao }, graffiti, ) .await; - let (contents, payload_value, fork) = match produced { + let (produced, payload_value, fork) = match produced { Ok(produced) => produced, Err(err) => return err.into_response(), }; + let blinded = matches!(produced, Produced::Blinded(_)); let accept = headers.get(header::ACCEPT).and_then(|v| v.to_str().ok()); - let mut response = match Encoding::from_accept(accept) { - Encoding::Ssz => ssz_response(contents.to_ssz()), - Encoding::Json => crate::json_response(serde_json::json!({ + let mut response = match (Encoding::from_accept(accept), produced) { + (Encoding::Ssz, Produced::Full(contents)) => ssz_response(contents.to_ssz()), + (Encoding::Ssz, Produced::Blinded(block)) => ssz_response(block.to_ssz()), + (Encoding::Json, Produced::Full(contents)) => crate::json_response(serde_json::json!({ "version": fork.as_str(), "execution_payload_blinded": false, "execution_payload_value": payload_value, "consensus_block_value": "0", "data": contents, })), + (Encoding::Json, Produced::Blinded(block)) => crate::json_response(serde_json::json!({ + "version": fork.as_str(), + "execution_payload_blinded": true, + "execution_payload_value": payload_value, + "consensus_block_value": "0", + "data": block, + })), }; let headers = response.headers_mut(); headers.insert( "eth-execution-payload-blinded", - HeaderValue::from_static("false"), + HeaderValue::from_static(if blinded { "true" } else { "false" }), ); if let Ok(value) = HeaderValue::from_str(&payload_value) { headers.insert("eth-execution-payload-value", value); } - // Not computed: nothing here reads it, and the builder comparison it - // exists for does not happen on this node. + // Not computed: the builder comparison here weighs the execution + // payloads' values alone, and nothing else on this node reads it. headers.insert("eth-consensus-block-value", HeaderValue::from_static("0")); with_consensus_version(response, fork) } -/// The block for `slot` with its blobs and cell proofs, the payload's value in -/// wei as a decimal string, and the block's fork. +/// What `produceBlockV3` answers with. +enum Produced { + /// The local block, with its blobs and cell proofs. + Full(FuluBlockContents), + /// The blinded block around a builder's bid. + Blinded(BlindedBeaconBlock), +} + +/// The block's consensus contents, packed once against the proposal state +/// and offered to both payloads. +struct Packed { + attestations: Vec, + operations: Operations, + sync_aggregate: SyncAggregate, +} + +impl Packed { + /// Whether dropping the packed contents might save a block that failed to + /// build. + fn droppable(&self) -> bool { + !self.attestations.is_empty() + || !self.operations.is_empty() + || self.sync_aggregate != empty_sync_aggregate() + } +} + +/// The local payload, checked and converted for a block but not yet built +/// into one, so choosing between it and a bid costs no state transition. +struct LocalParts { + built: BuiltPayload, + blobs: Blobs, + kzg_proofs: CellKzgProofs, + execution_requests: ExecutionRequests, +} + +/// The blinded block around a validated bid, and what the bid pays. +struct BlindedCandidate { + block: BlindedBeaconBlock, + value: Uint256, +} + +/// What to ask the builder for this slot. +struct BuilderRequest<'a> { + builder: &'a Builder, + pubkey: BlsPubkey, + registration: ValidatorRegistrationV1, + parent_hash: ExecutionBlockHash, +} + +/// The block for `slot`, its payload's value in wei as a decimal string, and +/// the block's fork. +/// +/// With a builder layered, the proposer registered through this node and the +/// circuit breaker quiet, the builder's bid is fetched alongside the local +/// payload. Both are checked without a state transition, [`choice::choose`] +/// picks one, and only that one is built into a block; the other is built only +/// if the first fails. Without a builder the block is built exactly as it is +/// without one. #[allow(clippy::too_many_arguments)] async fn produce( store: &Store, @@ -476,11 +590,13 @@ async fn produce( sync_pool: &SharedSyncCommitteePool, fee_recipients: &FeeRecipients, own_version: &ClientVersionV1, + builder: Option<&Builder>, + boost_factor: u64, slot: Slot, randao_reveal: BlsSignature, options: BlockProcessingOptions, graffiti: Bytes32, -) -> Result<(FuluBlockContents, String, ForkName), ApiError> { +) -> Result<(Produced, String, ForkName), ApiError> { let config = store.config(); let (head_root, head_state) = head(store)?; if slot <= head_state.slot() { @@ -492,16 +608,157 @@ async fn produce( let proposer = ethlambda_state_transition::beacon::helpers::accessors::get_beacon_proposer_index(&state) .map_err(|_| ApiError::Internal("no proposer for the slot"))?; + let request = + builder.and_then(|builder| builder_request(store, builder, &state, head_root, proposer)); // Asked alongside the build rather than before it, so a healthy execution // client costs the block nothing for its version and a silent one costs it - // at most the version's own timeout. - let (built, el_version) = tokio::join!( + // at most the version's own timeout. The builder's bid is asked alongside + // as well, bounded by its own header timeout. + let (built, el_version, bid) = tokio::join!( build_payload(store, engine, fee_recipients, &state, head_root, proposer), graffiti::execution_client_version(engine, own_version), + fetch_bid(request.as_ref(), slot), ); - let mut built = built?; let graffiti = graffiti::with_client_versions(graffiti, el_version.as_ref(), own_version); + let local = built.and_then(|built| local_parts(built, slot)); + let bid = request + .as_ref() + .zip(bid) + .and_then(|(request, bid)| accepted_bid(&state, &config, request, bid)); + // Nothing to build a block from: skip the packing, whose sync aggregate + // costs a BLS verification. + let (local, mut bid) = match (local, bid) { + (Err(err), None) => return Err(err), + pair => pair, + }; + let packed = pack(store, sync_pool, &state, head_root, slot, &config); + + let local_offer = local.as_ref().ok().map(|parts| LocalOffer { + value: wei_u128(&parts.built.block_value), + should_override_builder: parts.built.should_override_builder, + }); + let bid_value = bid.as_ref().map(|bid| wei_u128(&bid.value)); + let order = if choice::choose(local_offer, bid_value, boost_factor) == Some(Source::Builder) { + [Source::Builder, Source::Local] + } else { + [Source::Local, Source::Builder] + }; + + let (mut local, mut local_error) = match local { + Ok(parts) => (Some(parts), None), + Err(err) => (None, Some(err)), + }; + for source in order { + match source { + Source::Builder => { + let Some(message) = bid.take() else { + continue; + }; + let blinded = blinded_block( + &state, + &config, + message, + &packed, + randao_reveal, + graffiti, + options, + ); + if let Some(blinded) = blinded { + inc_payload_source(Source::Builder.label()); + let value = decimal(&blinded.value); + info!(%slot, proposer, %value, "Produced a blinded block around the builder's bid"); + return Ok((Produced::Blinded(blinded.block), value, fork)); + } + } + Source::Local => { + let Some(parts) = local.take() else { + continue; + }; + let assembled = local_block( + &state, + parts, + &packed, + randao_reveal, + graffiti, + &config, + options, + slot, + ); + match assembled { + Ok((contents, value)) => { + if builder.is_some() { + inc_payload_source(Source::Local.label()); + } + let block = &contents.block; + info!( + %slot, + proposer, + attestations = block.body.attestations.len(), + slashings = block.body.proposer_slashings.len() + + block.body.attester_slashings.len(), + exits = block.body.voluntary_exits.len(), + bls_changes = block.body.bls_to_execution_changes.len(), + transactions = block.body.execution_payload.transactions.len(), + blobs = block.body.blob_kzg_commitments.len(), + graffiti = %graffiti::display(&block.body.graffiti), + "Produced block" + ); + return Ok((Produced::Full(contents), decimal(&value), fork)); + } + Err(err) => local_error = Some(err), + } + } + } + } + // The local error surfaces only when the builder did not step in. + Err(local_error.unwrap_or(ApiError::Internal("the block failed to build"))) +} + +/// The block's consensus contents for `state`: attestations, operations and +/// the sync aggregate, each checked against `state`. +fn pack( + store: &Store, + sync_pool: &SharedSyncCommitteePool, + state: &BeaconState, + head_root: H256, + slot: Slot, + config: &Config, +) -> Packed { + let candidates = store.attestation_pool().block_candidates(); + let attestations = pack_attestations(state, candidates); + let operation_candidates = { + let pool = store.operation_pool(); + Operations { + proposer_slashings: pool.proposer_slashings(), + attester_slashings: pool.attester_slashings(), + voluntary_exits: pool.voluntary_exits(), + bls_to_execution_changes: pool.bls_to_execution_changes(), + } + }; + let operations = pack_operations(state, operation_candidates, config); + // What the committee signed at the previous slot over this block's parent, + // verified exactly as `process_sync_aggregate` will check it against the + // state the block is built on; a candidate that fails becomes the empty + // aggregate rather than a block the chain would refuse. + let candidate = sync_pool + .lock() + .expect("sync committee pool lock poisoned") + .sync_aggregate(slot - 1, head_root); + let sync_aggregate = candidate.map_or_else(empty_sync_aggregate, |candidate| { + verified_sync_aggregate(state, candidate) + }); + Packed { + attestations, + operations, + sync_aggregate, + } +} + +/// The execution client's payload checked and converted for a block: its +/// blobs bundle against the commitments, the blobs and cell proofs into their +/// containers, and its execution requests parsed. No state transition runs. +fn local_parts(mut built: BuiltPayload, slot: Slot) -> Result { let bundle = &built.blobs_bundle; let malformed = bundle.commitments.len() != bundle.blobs.len() || bundle.proofs.len() != bundle.blobs.len() * preset::CELLS_PER_EXT_BLOB; @@ -534,32 +791,33 @@ async fn produce( }; let execution_requests = parse_execution_requests(&built.execution_requests) .map_err(|_| ApiError::Internal("the execution client's request list is malformed"))?; - let payload_value = decimal(&built.block_value); + Ok(LocalParts { + built, + blobs, + kzg_proofs, + execution_requests, + }) +} - let candidates = store.attestation_pool().block_candidates(); - let attestations = pack_attestations(&state, candidates); - let operation_candidates = { - let pool = store.operation_pool(); - Operations { - proposer_slashings: pool.proposer_slashings(), - attester_slashings: pool.attester_slashings(), - voluntary_exits: pool.voluntary_exits(), - bls_to_execution_changes: pool.bls_to_execution_changes(), - } - }; - let operations = pack_operations(&state, operation_candidates, &config); - // What the committee signed at the previous slot over this block's parent, - // verified exactly as `process_sync_aggregate` will check it against the - // state the block is built on; a candidate that fails becomes the empty - // aggregate rather than a block the chain would refuse. - let candidate = sync_pool - .lock() - .expect("sync committee pool lock poisoned") - .sync_aggregate(slot - 1, head_root); - let sync_aggregate = candidate.map_or_else(empty_sync_aggregate, |candidate| { - verified_sync_aggregate(&state, candidate) - }); - let has_sync_aggregate = sync_aggregate != empty_sync_aggregate(); +/// The local block: the execution client's payload around `packed`, with +/// its blobs and cell proofs, and the payload's value. +#[allow(clippy::too_many_arguments)] +fn local_block( + state: &BeaconState, + parts: LocalParts, + packed: &Packed, + randao_reveal: BlsSignature, + graffiti: Bytes32, + config: &Config, + options: BlockProcessingOptions, + slot: Slot, +) -> Result<(FuluBlockContents, Uint256), ApiError> { + let LocalParts { + built, + blobs, + kzg_proofs, + execution_requests, + } = parts; let inputs = |attestations, operations, sync_aggregate| BlockInputs { sync_aggregate, randao_reveal, @@ -570,49 +828,179 @@ async fn produce( blob_kzg_commitments: built.blobs_bundle.commitments.clone(), execution_requests: execution_requests.clone(), }; - let attestation_count = attestations.len(); - let has_operations = !operations.is_empty(); let block = match assemble_block( - &state, - inputs(attestations, operations, sync_aggregate), - &config, + state, + inputs( + packed.attestations.clone(), + packed.operations.clone(), + packed.sync_aggregate.clone(), + ), + config, options, ) { Ok(block) => block, - // `pack_attestations` and `pack_operations` check every candidate - // against this state, and the sync aggregate was verified above, so this - // should not happen; but a block without them still earns the proposal, - // and one that fails to build earns nothing. - Err(err) if attestation_count > 0 || has_operations || has_sync_aggregate => { + // `pack` checked every attestation and operation against this state + // and verified the sync aggregate against it, so this should not + // happen; but a block without them still earns the proposal, and one + // that fails to build earns nothing. + Err(err) if packed.droppable() => { warn!(%slot, %err, "Block with attestations, operations or a sync aggregate failed to build; retrying without"); assemble_block( - &state, + state, inputs(Vec::new(), Operations::default(), empty_sync_aggregate()), - &config, + config, options, ) .map_err(|_| ApiError::Internal("the block failed to build"))? } Err(_) => return Err(ApiError::Internal("the block failed to build")), }; - info!( - %slot, - proposer, - attestations = block.body.attestations.len(), - slashings = block.body.proposer_slashings.len() + block.body.attester_slashings.len(), - exits = block.body.voluntary_exits.len(), - bls_changes = block.body.bls_to_execution_changes.len(), - transactions = block.body.execution_payload.transactions.len(), - blobs = block.body.blob_kzg_commitments.len(), - graffiti = %graffiti::display(&block.body.graffiti), - "Produced block" - ); let contents = FuluBlockContents { block, kzg_proofs, blobs, }; - Ok((contents, payload_value, fork)) + Ok((contents, built.block_value)) +} + +/// What to ask the builder for, if anything. +/// +/// The builder is asked only when the proposer registered through this node +/// and the circuit breaker is quiet. A validator client that does not use +/// builders sends no registrations, so its validators never get here. +/// `get_block` has already refused every fork but fulu. +fn builder_request<'a>( + store: &Store, + builder: &'a Builder, + state: &BeaconState, + head_root: H256, + proposer: ValidatorIndex, +) -> Option> { + let BeaconState::Fulu(fulu) = state else { + return None; + }; + let pubkey = state.validator(proposer).ok()?.pubkey; + let registration = builder + .registrations + .get(&pubkey, compute_epoch_at_slot(state.slot()))?; + let optimistic = block_is_optimistic(store, head_root); + let syncing = builder.sync_status.get() == SyncStatus::Syncing; + if let Err(trip) = breaker::check(state, &builder.fallback, optimistic, syncing) { + inc_builder_fallback(trip.label()); + info!( + slot = state.slot(), + reason = trip.label(), + "The circuit breaker keeps the builder out; building locally" + ); + return None; + } + Some(BuilderRequest { + builder, + pubkey, + registration, + parent_hash: fulu.latest_execution_payload_header.block_hash, + }) +} + +/// The builder's bid, or `None` when it was not asked, had none, or failed. +async fn fetch_bid(request: Option<&BuilderRequest<'_>>, slot: Slot) -> Option { + let request = request?; + request + .builder + .client + .get_header(slot, request.parent_hash, &request.pubkey) + .await + .inspect_err(|err| warn!(%slot, %err, "The builder gave no bid; building locally")) + .ok() + .flatten() +} + +/// `bid`'s message, if the bid is acceptable. No state transition runs. +fn accepted_bid( + state: &BeaconState, + config: &Config, + request: &BuilderRequest, + bid: VersionedBid, +) -> Option { + let slot = state.slot(); + let BeaconState::Fulu(fulu) = state else { + return None; + }; + let inputs = payload_inputs(state, config).ok()?; + let context = BidContext { + genesis_fork_version: config.genesis_fork_version, + inputs: &inputs, + parent_block_number: fulu.latest_execution_payload_header.block_number, + parent_gas_limit: fulu.latest_execution_payload_header.gas_limit, + target_gas_limit: request.registration.gas_limit, + max_blobs: config.max_blobs_per_block(compute_epoch_at_slot(slot)), + }; + if let Err(rejection) = bid::validate(&bid.bid, &context) { + inc_bid_rejection(rejection.label()); + warn!(%slot, reason = rejection.label(), "Refusing the builder's bid"); + return None; + } + let message = bid.bid.message; + if message.header.fee_recipient != request.registration.fee_recipient { + // Expected: relays pay the proposer with a transaction in the payload + // and name the builder as its fee recipient (docs/spec_deviations.md). + debug!(%slot, "The bid's fee recipient is not the registered one"); + } + Some(message) +} + +/// The blinded block around an accepted bid, if it passes the state +/// transition. +#[allow(clippy::too_many_arguments)] +fn blinded_block( + state: &BeaconState, + config: &Config, + message: BuilderBid, + packed: &Packed, + randao_reveal: BlsSignature, + graffiti: Bytes32, + options: BlockProcessingOptions, +) -> Option { + let slot = state.slot(); + let assemble = |attestations, operations, sync_aggregate| { + assemble_blinded_block( + state, + BlindedBlockInputs { + randao_reveal, + graffiti, + attestations, + operations, + execution_payload_header: message.header.clone(), + blob_kzg_commitments: message.blob_kzg_commitments.to_vec(), + execution_requests: message.execution_requests.clone(), + sync_aggregate, + }, + config, + options, + ) + }; + let assembled = match assemble( + packed.attestations.clone(), + packed.operations.clone(), + packed.sync_aggregate.clone(), + ) { + Err(err) if packed.droppable() => { + warn!(%slot, %err, "Blinded block with attestations, operations or a sync aggregate failed to build; retrying without"); + assemble(Vec::new(), Operations::default(), empty_sync_aggregate()) + } + other => other, + }; + match assembled { + Ok(block) => Some(BlindedCandidate { + block, + value: message.value, + }), + Err(err) => { + inc_bid_rejection("state_transition"); + warn!(%slot, %err, "The builder's bid fails the state transition"); + None + } + } } /// Ask the execution client to build on the head for `state`'s slot, then @@ -1025,4 +1413,613 @@ mod tests { assert_eq!(status, StatusCode::BAD_REQUEST); assert!(rejected.blocks.lock().unwrap().is_empty()); } + + use crate::beacon::builder_client::mock_relay::MockRelay; + use crate::beacon::builder_client::{BuilderClient, VersionedBid}; + use crate::beacon::builder_flow::breaker::FallbackThresholds; + use ethlambda_blockchain::SyncStatusController; + use ethlambda_types::beacon::builder::{BuilderBid, ValidatorRegistrationV1}; + use ethlambda_types::beacon::containers::capella; + use ethlambda_types::beacon::containers::deneb::{ExecutionPayloadHeader, KzgCommitments}; + use ethlambda_types::beacon::containers::electra::{self, ExecutionRequests}; + use libssz::SszDecode as _; + use std::time::Duration; + + const ETH: u64 = 1_000_000_000_000_000_000; + /// The registered gas limit the test proposer asks for. + const TARGET_GAS_LIMIT: u64 = 60_000_000; + + /// Thresholds that never trip, so a test state's history does not decide + /// whether the builder is asked. + const NEVER_TRIP: FallbackThresholds = FallbackThresholds { + skips: u64::MAX, + skips_per_epoch: u64::MAX, + epochs_since_finalization: u64::MAX, + }; + + /// A builder world for the next slot on `fulu_store()`: a mock relay, a + /// `Builder` pointing at it with the proposer registered, and the + /// advanced proposal state. + struct BuilderWorld { + store: Store, + slot: Slot, + state: BeaconState, + relay: MockRelay, + builder: Arc, + } + + async fn builder_world(registered: bool, sync: SyncStatus) -> BuilderWorld { + let (store, head_slot) = fulu_store(); + let slot = head_slot + 1; + let config = store.config(); + let (_, head_state) = super::super::validator::head(&store).unwrap(); + let state = advance_to_slot(&head_state, slot, &config).unwrap(); + let proposer = get_beacon_proposer_index(&state).unwrap(); + let pubkey = state.validator(proposer).unwrap().pubkey; + let relay = MockRelay::start(config.genesis_fork_version).await; + let client = + BuilderClient::new(relay.url.clone(), Duration::from_secs(1), "ethlambda/test") + .unwrap(); + let builder = Arc::new(Builder::new( + client, + NEVER_TRIP, + SyncStatusController::new(sync), + )); + if registered { + let registration = ValidatorRegistrationV1 { + fee_recipient: ExecutionAddress::repeat_byte(0x11), + gas_limit: TARGET_GAS_LIMIT, + timestamp: 1, + pubkey, + }; + builder + .registrations + .record([®istration], compute_epoch_at_slot(slot)); + } + BuilderWorld { + store, + slot, + state, + relay, + builder, + } + } + + impl BuilderWorld { + /// A bid of `value` wei that passes every check for this slot, with + /// `edit` applied before the relay signs it. + fn offer(&self, value: u64, edit: impl FnOnce(&mut BuilderBid)) { + let config = self.store.config(); + let inputs = payload_inputs(&self.state, &config).unwrap(); + let BeaconState::Fulu(fulu) = &self.state else { + unreachable!("built as fulu") + }; + let parent_gas_limit = fulu.latest_execution_payload_header.gas_limit; + let step = (parent_gas_limit / 1024).saturating_sub(1); + let withdrawals: capella::Withdrawals = inputs.withdrawals.clone().try_into().unwrap(); + let mut header = + ExecutionPayloadHeader::from(&electra::BeaconBlockBody::empty().execution_payload); + header.parent_hash = inputs.parent_hash; + header.prev_randao = inputs.prev_randao; + header.timestamp = inputs.timestamp; + header.block_number = fulu.latest_execution_payload_header.block_number + 1; + header.withdrawals_root = withdrawals.hash_tree_root(); + header.gas_limit = TARGET_GAS_LIMIT.clamp( + parent_gas_limit - step, + parent_gas_limit.saturating_add(step), + ); + header.block_hash = ExecutionBlockHash::repeat_byte(0xbb); + let mut message = BuilderBid { + header, + blob_kzg_commitments: KzgCommitments::default(), + execution_requests: ExecutionRequests::default(), + value: Uint256::from(value), + pubkey: self.relay.pubkey(), + }; + edit(&mut message); + let bid = self.relay.sign_bid(message); + self.relay.offer(VersionedBid { + fork: ForkName::Fulu, + bid, + }); + } + + fn app(&self, engine: Option) -> Router { + produce_app(self.store.clone(), engine).layer(Extension(self.builder.clone())) + } + + fn uri(&self, query: &str) -> String { + format!( + "{}{query}", + produce_uri(self.slot, &infinity_reveal(), "&skip_randao_verification") + ) + } + } + + /// What a `produceBlockV3` answer says about the payload it carries. + struct Answer { + status: StatusCode, + blinded: Option, + value: Option, + json: serde_json::Value, + } + + impl Answer { + /// The answer was a 200 whose headers and body say `blinded` and + /// `value` (wei, decimal), and the relay was asked `header_requests` + /// times. + fn assert(&self, world: &BuilderWorld, blinded: bool, value: &str, header_requests: usize) { + assert_eq!(self.status, StatusCode::OK, "{}", self.json); + assert_eq!( + self.blinded.as_deref(), + Some(if blinded { "true" } else { "false" }), + "{}", + self.json + ); + assert_eq!(self.value.as_deref(), Some(value), "{}", self.json); + assert_eq!(served_blinded(&self.json), blinded, "{}", self.json); + assert_eq!( + world.relay.header_requests().len(), + header_requests, + "{}", + self.json + ); + } + } + + async fn produce_answer( + world: &BuilderWorld, + engine: Option, + accept: Option<&str>, + query: &str, + ) -> Answer { + let mut request = Request::get(world.uri(query)); + if let Some(accept) = accept { + request = request.header("accept", accept); + } + let response = world + .app(engine) + .oneshot(request.body(Body::empty()).unwrap()) + .await + .unwrap(); + let header = |name: &str| { + response + .headers() + .get(name) + .map(|value| value.to_str().unwrap().to_owned()) + }; + let (status, blinded, value) = ( + response.status(), + header("eth-execution-payload-blinded"), + header("eth-execution-payload-value"), + ); + let body = response.into_body().collect().await.unwrap().to_bytes(); + Answer { + status, + blinded, + value, + json: serde_json::from_slice(&body).unwrap_or_default(), + } + } + + async fn produce( + world: &BuilderWorld, + engine_value: &'static str, + override_builder: bool, + query: &str, + ) -> Answer { + let engine = super::super::validator_client_tests::fake_execution_client_paying( + engine_value, + override_builder, + ) + .await; + produce_answer(world, Some(engine), None, query).await + } + + fn served_blinded(json: &serde_json::Value) -> bool { + json["execution_payload_blinded"] == serde_json::json!(true) + } + + /// An execution client that is not there. + fn dead_engine() -> EngineClient { + EngineClient::new( + "http://127.0.0.1:1".to_owned(), + ethlambda_engine::JwtSecret::new([0x0f; 32]), + ) + .unwrap() + } + + #[tokio::test] + async fn a_bid_worth_more_than_the_local_payload_is_served_blinded() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(ETH, |_| {}); + let answer = produce(&world, "0x2a", false, "").await; + answer.assert(&world, true, Ð.to_string(), 1); + assert_eq!( + answer.json["data"]["slot"], + serde_json::json!(world.slot.to_string()) + ); + assert_eq!( + answer.json["data"]["body"]["execution_payload_header"]["block_hash"], + serde_json::json!(format!("0x{}", "bb".repeat(32))) + ); + } + + #[tokio::test] + async fn a_tie_keeps_the_local_payload() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(ETH, |_| {}); + // 1 ETH in hex wei. + let answer = produce(&world, "0xde0b6b3a7640000", false, "").await; + answer.assert(&world, false, Ð.to_string(), 1); + } + + #[tokio::test] + async fn a_boost_factor_of_zero_keeps_the_local_payload() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(ETH, |_| {}); + let answer = produce(&world, "0x2a", false, "&builder_boost_factor=0").await; + answer.assert(&world, false, "42", 1); + } + + #[tokio::test] + async fn the_maximum_boost_factor_takes_the_bid() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(1_000_000_000, |_| {}); + let query = format!("&builder_boost_factor={}", u64::MAX); + let answer = produce(&world, "0xde0b6b3a7640000", false, &query).await; + answer.assert(&world, true, "1000000000", 1); + } + + #[tokio::test] + async fn the_execution_clients_override_keeps_the_local_payload() { + let world = builder_world(true, SyncStatus::Synced).await; + // 10 ETH: the largest round value a u64 of wei holds. + world.offer(10 * ETH, |_| {}); + let answer = produce(&world, "0x2a", true, "").await; + answer.assert(&world, false, "42", 1); + } + + #[tokio::test] + async fn an_unregistered_proposer_never_reaches_the_builder() { + let world = builder_world(false, SyncStatus::Synced).await; + world.offer(ETH, |_| {}); + let answer = produce(&world, "0x2a", false, "").await; + answer.assert(&world, false, "42", 0); + } + + #[tokio::test] + async fn a_syncing_node_keeps_the_builder_out() { + let world = builder_world(true, SyncStatus::Syncing).await; + world.offer(ETH, |_| {}); + let answer = produce(&world, "0x2a", false, "").await; + answer.assert(&world, false, "42", 0); + } + + #[tokio::test] + async fn a_bid_on_the_wrong_parent_is_dropped() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(ETH, |bid| { + bid.header.parent_hash = ExecutionBlockHash::repeat_byte(9) + }); + let answer = produce(&world, "0x2a", false, "").await; + answer.assert(&world, false, "42", 1); + } + + #[tokio::test] + async fn a_bid_on_the_wrong_block_number_is_dropped() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(ETH, |bid| bid.header.block_number += 5); + let answer = produce(&world, "0x2a", false, "").await; + answer.assert(&world, false, "42", 1); + } + + #[tokio::test] + async fn without_a_local_payload_the_bid_is_served() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(ETH, |_| {}); + let answer = produce_answer(&world, Some(dead_engine()), None, "").await; + answer.assert(&world, true, Ð.to_string(), 1); + } + + #[tokio::test] + async fn without_a_bid_or_a_local_payload_there_is_no_block() { + let world = builder_world(true, SyncStatus::Synced).await; + let answer = produce_answer(&world, Some(dead_engine()), None, "").await; + // The local error: the execution client did not start building. + assert_eq!( + answer.status, + StatusCode::SERVICE_UNAVAILABLE, + "{}", + answer.json + ); + assert_eq!(world.relay.header_requests().len(), 1); + } + + #[tokio::test] + async fn a_bid_whose_block_fails_to_build_falls_back_to_a_local_block_that_fails_too() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(ETH, |_| {}); + let engine = + super::super::validator_client_tests::fake_execution_client_paying("0x2a", false).await; + // The infinity reveal without the skip flag fails the randao check in + // both blocks' state transitions. + let uri = produce_uri(world.slot, &infinity_reveal(), ""); + let request = Request::get(uri).body(Body::empty()).unwrap(); + let (status, json) = respond(world.app(Some(engine)), request).await; + assert_eq!(status, StatusCode::INTERNAL_SERVER_ERROR, "{json}"); + assert_eq!(json["message"], "the block failed to build"); + assert_eq!(world.relay.header_requests().len(), 1); + } + + #[tokio::test] + async fn an_unreachable_builder_leaves_the_local_payload() { + let mut world = builder_world(true, SyncStatus::Synced).await; + // The same registered proposer, with a builder nothing listens at. + let client = BuilderClient::new( + "http://127.0.0.1:1".to_owned(), + Duration::from_secs(1), + "ethlambda/test", + ) + .unwrap(); + let dead = Arc::new(Builder::new( + client, + NEVER_TRIP, + SyncStatusController::new(SyncStatus::Synced), + )); + let proposer = get_beacon_proposer_index(&world.state).unwrap(); + let registration = ValidatorRegistrationV1 { + fee_recipient: ExecutionAddress::repeat_byte(0x11), + gas_limit: TARGET_GAS_LIMIT, + timestamp: 1, + pubkey: world.state.validator(proposer).unwrap().pubkey, + }; + dead.registrations + .record([®istration], compute_epoch_at_slot(world.slot)); + world.builder = dead; + let answer = produce(&world, "0x2a", false, "").await; + assert_eq!(answer.status, StatusCode::OK, "{}", answer.json); + assert_eq!(answer.blinded.as_deref(), Some("false"), "{}", answer.json); + assert_eq!(answer.value.as_deref(), Some("42"), "{}", answer.json); + } + + #[tokio::test] + async fn a_blinded_block_is_served_in_ssz_with_its_headers() { + let world = builder_world(true, SyncStatus::Synced).await; + world.offer(ETH, |_| {}); + let engine = + super::super::validator_client_tests::fake_execution_client_paying("0x2a", false).await; + let request = Request::get(world.uri("")) + .header("accept", "application/octet-stream") + .body(Body::empty()) + .unwrap(); + let response = world.app(Some(engine)).oneshot(request).await.unwrap(); + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(response.headers()["eth-execution-payload-blinded"], "true"); + assert_eq!( + response.headers()["eth-execution-payload-value"], + ETH.to_string().as_str() + ); + assert_eq!(world.relay.header_requests().len(), 1); + let body = response.into_body().collect().await.unwrap().to_bytes(); + let block = BlindedBeaconBlock::from_ssz_bytes(&body).unwrap(); + assert_eq!(block.slot, world.slot); + } + + use ethlambda_state_transition::beacon::helpers::accessors::get_domain; + use ethlambda_state_transition::beacon::helpers::misc::compute_signing_root; + use ethlambda_state_transition::beacon::helpers::test_state::sign_for; + use ethlambda_types::beacon::builder::SignedBlindedBeaconBlock; + use ethlambda_types::beacon::constants::DOMAIN_BEACON_PROPOSER; + use ethlambda_types::beacon::primitives::Root; + + /// A blinded block for `world`'s slot on its head, signed by the slot's + /// proposer (or by `signer`, to forge it), with `graffiti` to make + /// distinct blocks. + fn signed_blinded( + world: &BuilderWorld, + graffiti: u8, + signer: Option, + ) -> SignedBlindedBeaconBlock { + let (head_root, _) = super::super::validator::head(&world.store).unwrap(); + let proposer = get_beacon_proposer_index(&world.state).unwrap(); + let mut body = electra::BeaconBlockBody::empty(); + body.graffiti = Bytes32::repeat_byte(graffiti); + let block = electra::BeaconBlock { + slot: world.slot, + proposer_index: proposer, + parent_root: head_root, + state_root: Root::ZERO, + body, + }; + let message = BlindedBeaconBlock::from(block); + let domain = get_domain(&world.state, DOMAIN_BEACON_PROPOSER, None); + let root = compute_signing_root(message.hash_tree_root(), domain); + let signature = sign_for(signer.unwrap_or(proposer as usize), root); + SignedBlindedBeaconBlock { message, signature } + } + + async fn post_blinded( + app: Router, + block: &SignedBlindedBeaconBlock, + ) -> (StatusCode, serde_json::Value) { + let request = Request::post("/eth/v2/beacon/blinded_blocks") + .header("content-type", "application/octet-stream") + .header("eth-consensus-version", "fulu") + .body(Body::from(block.to_ssz())) + .unwrap(); + respond(app, request).await + } + + fn publish_app(world: &BuilderWorld) -> Router { + let network: RpcToP2PRef = Arc::new(RecordingNetwork::default()); + routes() + .with_state(world.store.clone()) + .layer(Extension(network)) + .layer(Extension(world.builder.clone())) + } + + #[tokio::test] + async fn a_signed_blinded_block_goes_to_the_builder() { + let world = builder_world(true, SyncStatus::Synced).await; + let block = signed_blinded(&world, 1, None); + let (status, json) = post_blinded(publish_app(&world), &block).await; + assert_eq!(status, StatusCode::OK, "{json}"); + assert_eq!(world.relay.blinded_blocks(), [block]); + } + + #[tokio::test] + async fn a_forged_blinded_block_never_reaches_the_builder() { + let world = builder_world(true, SyncStatus::Synced).await; + let proposer = get_beacon_proposer_index(&world.state).unwrap() as usize; + let forged = signed_blinded(&world, 1, Some(proposer + 1)); + let (status, _) = post_blinded(publish_app(&world), &forged).await; + assert_eq!(status, StatusCode::BAD_REQUEST); + assert!(world.relay.blinded_blocks().is_empty()); + } + + #[tokio::test] + async fn a_second_different_block_for_the_slot_is_refused() { + let world = builder_world(true, SyncStatus::Synced).await; + let app = publish_app(&world); + let first = signed_blinded(&world, 1, None); + let second = signed_blinded(&world, 2, None); + assert_eq!(post_blinded(app.clone(), &first).await.0, StatusCode::OK); + assert_eq!(post_blinded(app.clone(), &first).await.0, StatusCode::OK); + let (status, json) = post_blinded(app, &second).await; + assert_eq!(status, StatusCode::BAD_REQUEST, "{json}"); + assert_eq!(world.relay.blinded_blocks(), [first.clone(), first]); + } + + #[tokio::test] + async fn the_builders_answer_decides_the_status() { + for (answer, expected) in [ + (StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST), + ( + StatusCode::INTERNAL_SERVER_ERROR, + StatusCode::INTERNAL_SERVER_ERROR, + ), + ] { + let world = builder_world(true, SyncStatus::Synced).await; + world.relay.answer_submissions_with(answer); + let block = signed_blinded(&world, 1, None); + let (status, json) = post_blinded(publish_app(&world), &block).await; + assert_eq!(status, expected, "{answer}: {json}"); + } + } + + #[tokio::test] + async fn without_a_builder_a_blinded_block_is_a_server_error() { + let world = builder_world(true, SyncStatus::Synced).await; + let network: RpcToP2PRef = Arc::new(RecordingNetwork::default()); + let app = routes() + .with_state(world.store.clone()) + .layer(Extension(network)); + let block = signed_blinded(&world, 1, None); + let (status, _) = post_blinded(app, &block).await; + assert_eq!(status, StatusCode::INTERNAL_SERVER_ERROR); + } + + #[tokio::test] + async fn a_claim_survives_a_failed_submission() { + let world = builder_world(true, SyncStatus::Synced).await; + world + .relay + .answer_submissions_with(StatusCode::INTERNAL_SERVER_ERROR); + let app = publish_app(&world); + let first = signed_blinded(&world, 1, None); + let second = signed_blinded(&world, 2, None); + assert_eq!( + post_blinded(app.clone(), &first).await.0, + StatusCode::INTERNAL_SERVER_ERROR + ); + // The slot stays claimed, so a different block is refused outright. + assert_eq!( + post_blinded(app.clone(), &second).await.0, + StatusCode::BAD_REQUEST + ); + // The same block is a retry, and goes to the builder again. + assert_eq!( + post_blinded(app, &first).await.0, + StatusCode::INTERNAL_SERVER_ERROR + ); + assert_eq!(world.relay.blinded_blocks(), [first.clone(), first]); + } + + #[tokio::test] + async fn a_block_published_in_full_blocks_a_different_blinded_one() { + let world = builder_world(true, SyncStatus::Synced).await; + // A full block for the slot from the local path, signed by its proposer. + let answer = produce(&world, "0x2a", false, "").await; + answer.assert(&world, false, "42", 1); + let mut contents: FuluBlockContents = + serde_json::from_value(answer.json["data"].clone()).unwrap(); + // The test store keys its head by a root that is not its header's, so + // the block names the stored key as its parent, as `signed_blinded` does. + let (head_root, _) = super::super::validator::head(&world.store).unwrap(); + contents.block.parent_root = head_root; + let proposer = get_beacon_proposer_index(&world.state).unwrap(); + let domain = get_domain(&world.state, DOMAIN_BEACON_PROPOSER, None); + let signing_root = compute_signing_root(contents.block.hash_tree_root(), domain); + let signed = FuluSignedBlockContents { + signed_block: electra::SignedBeaconBlock { + message: contents.block, + signature: sign_for(proposer as usize, signing_root), + }, + kzg_proofs: contents.kzg_proofs, + blobs: contents.blobs, + }; + + let network = Arc::new(RecordingNetwork::default()); + let p2p: RpcToP2PRef = network.clone(); + let app = routes() + .with_state(world.store.clone()) + .layer(Extension(p2p)) + .layer(Extension(world.builder.clone())); + let request = Request::post("/eth/v2/beacon/blocks") + .header("content-type", "application/octet-stream") + .header("eth-consensus-version", "fulu") + .body(Body::from(signed.to_ssz())) + .unwrap(); + let (status, json) = respond(app.clone(), request).await; + assert_eq!(status, StatusCode::OK, "{json}"); + assert_eq!(network.blocks.lock().unwrap().len(), 1); + + let blinded = signed_blinded(&world, 1, None); + let (status, json) = post_blinded(app, &blinded).await; + assert_eq!(status, StatusCode::BAD_REQUEST, "{json}"); + assert_eq!( + json["message"], + "this node already published a different block for the slot" + ); + assert!(world.relay.blinded_blocks().is_empty()); + } + + /// The blinded route takes the publish limit, not axum's 2 MiB default. + #[tokio::test] + async fn a_blinded_block_over_the_default_body_limit_is_not_a_413() { + let world = builder_world(true, SyncStatus::Synced).await; + let request = Request::post("/eth/v2/beacon/blinded_blocks") + .header("content-type", "application/octet-stream") + .header("eth-consensus-version", "fulu") + .body(Body::from(vec![0u8; 3 * 1024 * 1024])) + .unwrap(); + let (status, _) = respond(publish_app(&world), request).await; + assert_ne!(status, StatusCode::PAYLOAD_TOO_LARGE); + } + + #[tokio::test] + async fn an_oversized_blinded_block_is_a_413() { + let world = builder_world(true, SyncStatus::Synced).await; + let request = Request::post("/eth/v2/beacon/blinded_blocks") + .header("content-type", "application/octet-stream") + .header("eth-consensus-version", "fulu") + .body(Body::from(vec![ + 0u8; + super::super::MAX_PUBLISH_BODY_BYTES + 1 + ])) + .unwrap(); + let (status, _) = respond(publish_app(&world), request).await; + assert_eq!(status, StatusCode::PAYLOAD_TOO_LARGE); + } } diff --git a/crates/net/rpc/src/beacon/registrations.rs b/crates/net/rpc/src/beacon/registrations.rs index bca74807e..f90124571 100644 --- a/crates/net/rpc/src/beacon/registrations.rs +++ b/crates/net/rpc/src/beacon/registrations.rs @@ -9,9 +9,12 @@ //! not reach the builder, not that the request fails. Each kept //! registration also becomes its validator's fee recipient, as //! `prepare_beacon_proposer` would make it, so a block built locally after -//! all still pays the address the validator registered. +//! all still pays the address the validator registered. The kept +//! registrations are also remembered for block production, which asks the +//! builder only for proposers that registered. use std::collections::{HashMap, HashSet}; +use std::sync::Arc; use axum::body::Bytes; use axum::extract::{DefaultBodyLimit, State}; @@ -29,7 +32,8 @@ use ethlambda_types::beacon::signing::compute_epoch_at_slot; use libssz::SszDecode as _; use tracing::warn; -use super::builder_client::{BuilderClient, BuilderError}; +use super::builder_client::BuilderError; +use super::builder_flow::Builder; use super::validator::{FeeRecipients, head}; use super::{ApiError, BodyEncoding}; use crate::metrics::inc_builder_registrations; @@ -45,11 +49,12 @@ pub(crate) fn routes() -> Router { async fn post_register_validator( State(store): State, - Extension(builder): Extension>, + builder: Option>>, Extension(fee_recipients): Extension, headers: HeaderMap, body: Bytes, ) -> Response { + let builder = builder.map(|Extension(builder)| builder); register(store, builder, fee_recipients, &headers, &body) .await .map_or_else(IntoResponse::into_response, |()| { @@ -59,7 +64,7 @@ async fn post_register_validator( async fn register( store: Store, - builder: Option, + builder: Option>, fee_recipients: FeeRecipients, headers: &HeaderMap, body: &[u8], @@ -72,12 +77,13 @@ async fn register( // The head-state read and the walk over the whole registry are blocking // work, as the duty endpoints' walks are. let submitted = registrations.len(); - let kept = tokio::task::spawn_blocking(move || { + let (kept, epoch) = tokio::task::spawn_blocking(move || { let (_, state) = head(&store)?; // Judged at the later of the head's epoch and the wall clock's, so a // lagging head cannot let an exited validator through. let wall_epoch = compute_epoch_at_slot(super::node::wall_slot(&store)); - Ok::<_, ApiError>(active_or_pending(&state, registrations, wall_epoch)) + let epoch = compute_epoch_at_slot(state.slot()).max(wall_epoch); + Ok::<_, ApiError>((active_or_pending(&state, registrations, epoch), epoch)) }) .await .map_err(|_| ApiError::Internal("checking the registrations failed"))??; @@ -92,6 +98,12 @@ async fn register( recipients.insert(*index, registration.message.fee_recipient); } } + // Remembered at the epoch the filter judged them at, so block production + // sees them for as long as the validator client keeps sending them. + builder.registrations.record( + kept.iter().map(|(_, registration)| ®istration.message), + epoch, + ); let kept: Vec = kept .into_iter() .map(|(_, registration)| registration) @@ -100,7 +112,7 @@ async fn register( // and the forward, its count and its log must complete regardless. let forwarded = tokio::spawn(async move { let count = kept.len() as u64; - let result = builder.register_validators(&kept).await; + let result = builder.client.register_validators(&kept).await; match &result { Ok(()) => inc_builder_registrations("forwarded", count), Err(err) => { @@ -133,14 +145,13 @@ fn decode( } /// The registrations whose validator `state` holds as active or pending at -/// the later of its own epoch and `wall_epoch`, each with its validator -/// index. One pass over the registry, whatever the number of registrations. +/// `epoch`, each with its validator index. One pass over the registry, +/// whatever the number of registrations. fn active_or_pending( state: &BeaconState, registrations: Vec, - wall_epoch: Epoch, + epoch: Epoch, ) -> Vec<(ValidatorIndex, SignedValidatorRegistrationV1)> { - let epoch = compute_epoch_at_slot(state.slot()).max(wall_epoch); let wanted: HashSet = registrations .iter() .map(|registration| registration.message.pubkey) @@ -168,6 +179,8 @@ mod tests { use axum::body::Body; use axum::http::{Request, StatusCode}; + use ethlambda_blockchain::SyncStatusController; + use ethlambda_blockchain::metrics::SyncStatus; use ethlambda_state_transition::beacon::helpers::test_state::with_signing_validators_at; use ethlambda_types::beacon::constants::FAR_FUTURE_EPOCH; use ethlambda_types::beacon::fork::ForkName; @@ -177,6 +190,8 @@ mod tests { use tower::ServiceExt as _; use super::*; + use crate::FallbackThresholds; + use crate::beacon::builder_client::BuilderClient; use crate::beacon::builder_client::mock_relay::{MockRelay, signed_registration}; use crate::test_utils::beacon_store_at; @@ -202,14 +217,15 @@ mod tests { async fn post( store: Store, - builder: Option, + builder: Option>, fee_recipients: FeeRecipients, request: Request, ) -> (StatusCode, String) { - let app = routes() - .with_state(store) - .layer(Extension(builder)) - .layer(Extension(fee_recipients)); + let app = routes().with_state(store).layer(Extension(fee_recipients)); + let app = match builder { + Some(builder) => app.layer(Extension(builder)), + None => app, + }; let response = app.oneshot(request).await.unwrap(); let status = response.status(); let body = response.into_body().collect().await.unwrap().to_bytes(); @@ -223,11 +239,19 @@ mod tests { .unwrap() } - fn client_for(relay: &MockRelay) -> Option { - Some( + fn builder_for(client: BuilderClient) -> Arc { + Arc::new(Builder::new( + client, + FallbackThresholds::DEFAULT, + SyncStatusController::new(SyncStatus::Synced), + )) + } + + fn client_for(relay: &MockRelay) -> Option> { + let client = BuilderClient::new(relay.url.clone(), Duration::from_secs(1), "ethlambda/test") - .unwrap(), - ) + .unwrap(); + Some(builder_for(client)) } #[tokio::test] @@ -338,12 +362,14 @@ mod tests { async fn an_unreachable_builder_is_a_server_error() { let store = store_with_exited(&[]); let registration = signed_registration(0, genesis_fork_version(&store)); - let builder = BuilderClient::new( - "http://127.0.0.1:1", - Duration::from_secs(1), - "ethlambda/test", - ) - .unwrap(); + let builder = builder_for( + BuilderClient::new( + "http://127.0.0.1:1", + Duration::from_secs(1), + "ethlambda/test", + ) + .unwrap(), + ); let (status, _) = post( store, Some(builder), @@ -428,12 +454,14 @@ mod tests { async fn fee_recipients_are_set_even_when_the_builder_is_unreachable() { let store = store_with_exited(&[]); let registration = signed_registration(0, genesis_fork_version(&store)); - let builder = BuilderClient::new( - "http://127.0.0.1:1", - Duration::from_secs(1), - "ethlambda/test", - ) - .unwrap(); + let builder = builder_for( + BuilderClient::new( + "http://127.0.0.1:1", + Duration::from_secs(1), + "ethlambda/test", + ) + .unwrap(), + ); let fee_recipients = FeeRecipients::default(); let (status, _) = post( @@ -451,6 +479,30 @@ mod tests { ); } + #[tokio::test] + async fn kept_registrations_are_remembered_for_block_production() { + let store = store_with_exited(&[]); + let relay = MockRelay::start(genesis_fork_version(&store)).await; + let builder = client_for(&relay).unwrap(); + let kept = signed_registration(0, genesis_fork_version(&store)); + let unknown = signed_registration(VALIDATORS + 10, genesis_fork_version(&store)); + + let (status, body) = post( + store, + Some(builder.clone()), + FeeRecipients::default(), + json_request(&[kept.clone(), unknown.clone()]), + ) + .await; + + assert_eq!(status, StatusCode::OK, "{body}"); + assert_eq!( + builder.registrations.get(&kept.message.pubkey, 0), + Some(kept.message) + ); + assert_eq!(builder.registrations.get(&unknown.message.pubkey, 0), None); + } + #[tokio::test] async fn a_malformed_body_is_a_400() { let store = store_with_exited(&[]); diff --git a/crates/net/rpc/src/beacon/validator_client_tests.rs b/crates/net/rpc/src/beacon/validator_client_tests.rs index 693450ebf..654fb6d5d 100644 --- a/crates/net/rpc/src/beacon/validator_client_tests.rs +++ b/crates/net/rpc/src/beacon/validator_client_tests.rs @@ -420,18 +420,29 @@ fn valid_blob(seed: u8) -> Vec { /// The fulu bundle carries `blob_count` valid blobs with their commitments and /// cell proofs (flattened blob by blob, as `BlobsBundleV2` does). pub(super) async fn fake_execution_client(blob_count: usize) -> ethlambda_engine::EngineClient { - fake_execution_client_with(blob_count, None).await + fake_execution_client_with(blob_count, None, "0x2a", false).await +} + +/// [`fake_execution_client`] with no blobs, whose fulu payload pays +/// `block_value` (hex wei) and carries `should_override_builder`. +pub(super) async fn fake_execution_client_paying( + block_value: &'static str, + should_override_builder: bool, +) -> ethlambda_engine::EngineClient { + fake_execution_client_with(0, None, block_value, should_override_builder).await } /// [`fake_execution_client`] whose gloas payloads carry `blob`, with its real /// commitment and cell proofs, in the blobs bundle. async fn fake_execution_client_with_blob(blob: Option>) -> ethlambda_engine::EngineClient { - fake_execution_client_with(0, blob).await + fake_execution_client_with(0, blob, "0x2a", false).await } async fn fake_execution_client_with( blob_count: usize, blob: Option>, + block_value: &'static str, + should_override_builder: bool, ) -> ethlambda_engine::EngineClient { use axum::{Json, routing::post}; use ethlambda_state_transition::beacon::kzg; @@ -513,9 +524,9 @@ async fn fake_execution_client_with( v5_payload["blobGasUsed"] = blob_gas_used.clone().into(); serde_json::json!({ "executionPayload": v5_payload, - "blockValue": "0x2a", + "blockValue": block_value, "blobsBundle": { "commitments": commitments, "proofs": proofs, "blobs": blobs }, - "shouldOverrideBuilder": false, + "shouldOverrideBuilder": should_override_builder, "executionRequests": [], }) } diff --git a/crates/net/rpc/src/lib.rs b/crates/net/rpc/src/lib.rs index af3f5f9cc..42dc58a1f 100644 --- a/crates/net/rpc/src/lib.rs +++ b/crates/net/rpc/src/lib.rs @@ -30,6 +30,7 @@ mod spec; pub mod test_driver; pub use beacon::builder_client::{BuilderClient, BuilderError, VersionedBid}; +pub use beacon::builder_flow::breaker::FallbackThresholds; pub(crate) use base::json_response; @@ -245,6 +246,9 @@ pub struct BeaconApiHandles { /// The builder `--builder` names, for the builder flow's endpoints. /// `None` when the node runs without one. pub builder: Option, + /// When block production skips the builder for a local block + /// (`--builder-fallback-*`). Unused without a builder. + pub builder_fallback: FallbackThresholds, } /// Start the HTTP servers for a beacon node. @@ -261,6 +265,13 @@ pub async fn start_beacon_rpc_server( peer_id: String, shutdown: CancellationToken, ) -> Result<(), std::io::Error> { + let builder = handles.builder.map(|client| { + Arc::new(beacon::builder_flow::Builder::new( + client, + handles.builder_fallback, + sync_status.clone(), + )) + }); let api_router = build_beacon_api_router(store, config.version, peer_id) .layer(Extension(sync_status)) .layer(Extension(handles.p2p)) @@ -270,12 +281,17 @@ pub async fn start_beacon_rpc_server( .layer(Extension(handles.custody_columns)) .layer(Extension(beacon::validator::FeeRecipients::default())) .layer(Extension(handles.engine)) - .layer(Extension(handles.builder)) .layer(Extension(handles.events)) .layer(Extension(handles.identity)) .layer(Extension(beacon::graffiti::OwnVersion(Arc::new( handles.client_version, )))); + // Layered only when `--builder` is set, so every handler reads it as an + // `Option>` and a node without one keeps the local path. + let api_router = match builder { + Some(builder) => api_router.layer(Extension(builder)), + None => api_router, + }; start_http_servers(config, Some(api_router), shutdown).await } diff --git a/crates/net/rpc/src/metrics.rs b/crates/net/rpc/src/metrics.rs index 14baad83e..913f7d7d8 100644 --- a/crates/net/rpc/src/metrics.rs +++ b/crates/net/rpc/src/metrics.rs @@ -84,3 +84,51 @@ pub(crate) fn inc_builder_registrations(outcome: &str, count: u64) { .with_label_values(&[outcome]) .inc_by(count); } + +/// Count builder bids refused before selection, by reason. +pub(crate) fn inc_bid_rejection(reason: &str) { + static LEAN_BEACON_BUILDER_BID_REJECTIONS_TOTAL: LazyLock = + LazyLock::new(|| { + register_int_counter_vec!( + "lean_beacon_builder_bid_rejections_total", + "Builder bids refused before selection, by reason", + &["reason"] + ) + .unwrap() + }); + LEAN_BEACON_BUILDER_BID_REJECTIONS_TOTAL + .with_label_values(&[reason]) + .inc(); +} + +/// Count blocks produced with a builder configured, by where the payload +/// came from. +pub(crate) fn inc_payload_source(source: &str) { + static LEAN_BEACON_BUILDER_PAYLOAD_SOURCE_TOTAL: LazyLock = + LazyLock::new(|| { + register_int_counter_vec!( + "lean_beacon_builder_payload_source_total", + "Blocks produced with a builder configured, by where the payload came from", + &["source"] + ) + .unwrap() + }); + LEAN_BEACON_BUILDER_PAYLOAD_SOURCE_TOTAL + .with_label_values(&[source]) + .inc(); +} + +/// Count proposals the circuit breaker kept from the builder, by reason. +pub(crate) fn inc_builder_fallback(reason: &str) { + static LEAN_BEACON_BUILDER_FALLBACK_TOTAL: LazyLock = LazyLock::new(|| { + register_int_counter_vec!( + "lean_beacon_builder_fallback_total", + "Proposals the circuit breaker kept from the builder, by reason", + &["reason"] + ) + .unwrap() + }); + LEAN_BEACON_BUILDER_FALLBACK_TOTAL + .with_label_values(&[reason]) + .inc(); +} diff --git a/docs/cli.md b/docs/cli.md index 489b24f9e..b9a65aa5d 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -116,6 +116,9 @@ from a normal build. | `--safe-slots-to-import-optimistically` | the specification's own value | How far behind the wall clock a block must be before it may be imported optimistically on age alone | | `--builder` | none | Base URL of a builder API endpoint, normally MEV-Boost (e.g. `http://127.0.0.1:18550`). Without it every block is built locally | | `--builder-header-timeout` | `1000` | Milliseconds block production waits for the builder's bid; also sent to the builder as `X-Timeout-Ms`. Requires `--builder` | +| `--builder-fallback-skips` | `3` | Build locally when more than this many consecutive slots right before the proposal had no block (the circuit breaker). Requires `--builder` | +| `--builder-fallback-skips-per-epoch` | `8` | Build locally when more than this many of the slots in the epoch's worth before the proposal had no block. Requires `--builder` | +| `--builder-fallback-epochs-since-finalization` | `3` | Build locally when the proposal is more than this many epochs past the finalized checkpoint. Requires `--builder` | Accepted range is `CUSTODY_REQUIREMENT` to `NUMBER_OF_CUSTODY_GROUPS` (4 to 128), enforced at parse time rather than clamped: serving a different set than diff --git a/docs/metrics.md b/docs/metrics.md index 5e3edce48..087d990f0 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -469,6 +469,9 @@ ethlambda-specific, not part of the leanMetrics spec. | `lean_beacon_publish_data_columns_seconds` | Histogram | Time to compute and verify a published block's cells and build its data column sidecars | On every published block that carries blobs and whose cells verify | | 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5 | | `lean_beacon_builder_request_seconds` | Histogram | Time a builder API call took | On every `getHeader`, `registerValidator`, `submitBlindedBlockV2` and `status` call to the configured builder | call=get_header,register_validators,submit_blinded_block,status
outcome=ok,no_bid,timeout,unreachable,bad_request,unexpected_status,invalid_response,unsupported_fork,encode | 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 0.75, 1, 2, 5, 10 | | `lean_beacon_builder_registrations_total` | Counter | Validator registrations received, by what became of them | On every `register_validator` request | outcome=forwarded,failed,dropped | | +| `lean_beacon_builder_bid_rejections_total` | Counter | Builder bids refused before selection, by reason | On every bid that fails validation or the state transition | reason=zero_value,parent_hash,prev_randao,timestamp,block_number,withdrawals,too_many_blobs,blob_gas,gas_limit,signature,state_transition | | +| `lean_beacon_builder_payload_source_total` | Counter | Blocks produced with a builder configured, by where the payload came from | On every produced block when `--builder` is set | source=local,builder | | +| `lean_beacon_builder_fallback_total` | Counter | Proposals the circuit breaker kept from the builder, by reason | On every proposal by a registered proposer that the breaker trips for | reason=syncing,optimistic_head,no_finality,consecutive_skips,skips_per_epoch | | ### Beacon Committee Cache diff --git a/docs/rpc.md b/docs/rpc.md index e63a27332..4772c3027 100644 --- a/docs/rpc.md +++ b/docs/rpc.md @@ -265,10 +265,11 @@ surface rather than sitting beside it; a `/lean/v0` path on a beacon node is a | `GET` | `/eth/v1/validator/sync_committee_contribution` | JSON | The pooled messages for `(slot, subcommittee_index, beacon_block_root)`, aggregated | | `POST` | `/eth/v1/validator/contribution_and_proofs` | *(status only)* | Validate, pool and gossip `SignedContributionAndProof`s | | `POST` | `/eth/v1/validator/sync_committee_subscriptions` | *(status only)* | Join sync committee subnets until an epoch | -| `GET` | `/eth/v3/validator/blocks/{slot}` | SSZ or JSON | An unsigned fulu block built on the head (`produceBlockV3`) | +| `GET` | `/eth/v3/validator/blocks/{slot}` | SSZ or JSON | An unsigned fulu block built on the head, or, with `--builder`, the blinded block around a winning bid (`produceBlockV3`) | | `POST` | `/eth/v4/validator/blocks/{slot}` | SSZ or JSON | An unsigned gloas block: self-built, with its envelope and blobs when asked, or on a pooled builder bid (`produceBlockV4`) | | `GET` | `/eth/v1/validator/execution_payload_envelopes/{slot}/{beacon_block_root}` | SSZ or JSON | The unsigned envelope `produceBlockV4` built (gloas) | | `POST` | `/eth/v2/beacon/blocks` | *(status only)* | Gossip and import a signed fulu or gloas block (`publishBlockV2`; JSON or SSZ body) | +| `POST` | `/eth/v2/beacon/blinded_blocks` | *(status only)* | Hand a signed fulu blinded block to the builder to reveal (`publishBlindedBlockV2`; JSON or SSZ body) | | `POST` | `/eth/v1/beacon/execution_payload_envelopes` | *(status only)* | Gossip a signed envelope and its data columns (gloas; JSON or SSZ body) | | `POST` | `/eth/v1/beacon/execution_payload_bids` | *(status only)* | Validate, pool and gossip a builder's `SignedExecutionPayloadBid` (gloas; JSON or SSZ body) | | `POST` | `/eth/v1/validator/proposer_preferences` | *(status only)* | Validate, cache and gossip `SignedProposerPreferences` (gloas; JSON or SSZ body) | @@ -437,12 +438,35 @@ the chain actor writes, so no request waits on the actor. nothing verifiable is pooled). The payload bundle's KZG commitments go in the body. The state root comes from running the block through `process_block`. The answer is fulu `BlockContents`: the block, the bundle's cell proofs and its blobs, with - `Eth-Execution-Payload-Blinded: false`; there is no builder flow. It is a + `Eth-Execution-Payload-Blinded: false`. It is a **`503`** without a configured execution client, or when the bundle is malformed (its commitments, proofs and blobs disagree in count). A slot the schedule does not place at fulu (gloas included) is a `400`, checked before the execution client; `produceBlockV3` is the fulu endpoint, and a gloas slot goes to `produceBlockV4`. + + **The builder path** is taken only when `--builder` is set, the proposer + registered through this node (`register_validator`, remembered for two + epochs after it was last sent), and the circuit breaker is quiet: it keeps + the builder out while the node is syncing, the head is optimistic, finality + is too far behind, or too many slots were skipped (the `--builder-fallback-*` + flags set the thresholds). The builder's `getHeader` is then asked + concurrently with the local build, bounded by its own timeout, and an error + or a timeout means no bid. A bid is checked before it competes: a non-zero + value, the head's execution block as its parent, the proposal's `prev_randao` + and timestamp, the parent's block number plus one, the withdrawals root, + the blob commitment and blob gas bounds, a gas limit within one step of the parent's toward the registered target, + and the builder's signature. Its fee recipient is not checked (see + `spec_deviations.md`). A bid that passes goes through `assemble_blinded_block` + with the packed attestations, operations and sync aggregate, so the blinded + block's state root comes from running the block through `process_block`. + `builder_boost_factor` (percent, default 100) then weighs the two: the local + payload wins iff its value is at least `factor * bid / 100`, so a tie and a + factor of 0 keep it and `u64::MAX` takes the bid; the execution client's + `shouldOverrideBuilder` keeps the local payload whatever the bid pays. If one + side fails the other is served. A blinded answer is the blinded block as SSZ + or JSON, with `Eth-Execution-Payload-Blinded: true` and + `Eth-Execution-Payload-Value` set to the bid's value in wei. - **Graffiti** is the validator client's, with both clients' codes and commits appended as far as they fit: `hello RH1a2bLA3c4d` for reth at `1a2b…` under ethlambda at `3c4d…`. The execution client's half comes from @@ -474,7 +498,27 @@ the chain actor writes, so no request waits on the actor. slot must be scheduled at gloas, the parent must be held, the slot must be after the parent's, and the proposer and signature are checked against the parent's state advanced to the slot (all `400` otherwise), before the block is - gossiped and handed to the chain actor. + gossiped and handed to the chain actor. This node publishes one block per + slot: a second block for a slot whose root differs from the first, through + this endpoint or `blinded_blocks`, is a `400`, since a validator that signed + two would be slashed. The same block again is a retry and passes. A slot + below the finalized one is refused too, since claims there are forgotten. +- **`POST beacon/blinded_blocks`** takes a fulu `SignedBlindedBeaconBlock` as + JSON or SSZ, by `Content-Type` as above, with `Eth-Consensus-Version: fulu`. + It is checked as a fulu `beacon/blocks` is, before anything leaves this node: + the slot is scheduled at fulu, the parent is held and before the block, the + proposer is the one the parent's state names for the slot, and the proposer's + signature verifies (a blinded block has its full block's root, so the check + is the same). Then the slot is claimed, shared with `beacon/blocks`, and the + block goes to the builder (`submitBlindedBlockV2`) on its own task, so a + validator client that hangs up cannot cancel the reveal. The builder + publishes the full block and its data columns; it returns over gossip and + imports like any block, and nothing is gossiped here. The body limit is + the one `beacon/blocks` has (`413` over it). The builder's answer is logged + from the submission task, an error when it did not take a signed block. + The status is the builder's: 202 becomes `200`, a builder `400` stays `400`, and anything else + is a `500`. There is no local fallback, since a second block would be an + equivocation. With no `--builder` the answer is `500`. - **The operation pool** holds, beside the attestation pool and in memory in the same `Store` (`Store::operation_pool`): proposer slashings (one per proposer), attester slashings (one per `hash_tree_root`), voluntary exits and diff --git a/docs/spec_deviations.md b/docs/spec_deviations.md index 191be46cd..8e04fb262 100644 --- a/docs/spec_deviations.md +++ b/docs/spec_deviations.md @@ -125,7 +125,18 @@ from the Beacon API, and no other way. the bid's. - **Why:** gossip and API bids need only a pool and the gossip rules. A builder API client is a separate crate with its own failure modes and timeouts, and - is left for a later phase. + is left for a later phase for gloas. Fulu's builder API (MEV-Boost) is now + served (`produceBlockV3` and `blinded_blocks`); gloas's is not. + +## Builder bids: the fee recipient is not checked + +- **The specification:** builder-specs' `process_bid` asserts + `header.fee_recipient == fee_recipient`. +- **ethlambda:** a mismatch is logged at `debug` only. +- **Why:** relays pay the proposer with a transaction in the payload and name + the builder's address as the coinbase, so the assertion would refuse nearly + every real bid. Every client reviewed (Lighthouse, Teku, Prysm, Nimbus, + Lodestar, Grandine) skips it. ## Fee recipient and gas target come from proposer preferences, with fallbacks