Remove the legacy heap region from snapshots

CHANGELOG.md

@@ -11,7 +11,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 * Core dumps no longer record the guest binary path. The process name and command line are always `<unknown>`, as they already were for sandboxes created from snapshots by @ludfjig in https://github.com/hyperlight-dev/hyperlight/pull/1975
 
 * **Breaking:** Generated libc bindings use `usize` for `size_t` arguments and return values, including `strlen` and `memcpy`. Rust callers that use `u64` or `c_ulong` need explicit conversions. The C ABI is unchanged.
-* **Breaking:** Snapshots use ABI 8 and config schema v4. Existing snapshots
+* **Breaking:** Snapshots use ABI 9 and config schema v5. Existing snapshots
   must be regenerated.
 * **Breaking:** `PtRootFinder` receives a `SnapshotMemoryReader` and returns a
   `Result<Vec<u64>>`. Read snapshot bytes with `snapshot.read_exact(gpa, buffer)?`
@@ -33,10 +33,17 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   creation.
 * Use `talc` as the guest heap allocator. Guest init writes only the start and
   end of the heap, which speeds up sandbox creation.
+* **Breaking:** Snapshots no longer contain a zero-filled heap region, which
+  makes them smaller and faster to create, save, and load. The guest heap is
+  backed by scratch and has no fixed size. Running out of scratch aborts the
+  guest with `MallocFailed` instead of failing the allocation.
 
 * **Breaking:** The transport arena sits one unmapped guard page below the reserved scratch-top pages, at an address independent of the scratch size, and the page tables at the scratch base. The guest's page allocator stops at the arena GPA the host publishes in the scratch-top metadata, so guests must be rebuilt. `hyperlight_common::layout::scratch_allocator_limit_gpa` is renamed `scratch_reserved_top_gpa`, the start of the reserved pages. An allocator that stops there runs into the arena. The minimum scratch size grows by that guard page. Snapshot ABI 7: existing snapshots must be regenerated.
 
 ### Removed
+* **Breaking:** `SandboxConfiguration::set_heap_size`,
+  `SandboxConfiguration::DEFAULT_HEAP_SIZE`, `SandboxBuilder::heap_size`, and
+  `SandboxBuilder::get_heap_size`.
 
 ### Fixed
 

docs/hyperlight-execution-details.md

@@ -27,7 +27,7 @@ With this background in mind, it's well worth focusing on the "lifecycle" of a V
 At the highest level, Hyperlight takes roughly the following steps to create and run arbitrary code inside a VM:
 
 1. Loads a specially built, statically linked binary (currently, the [PE](https://en.wikipedia.org/wiki/Portable_Executable) and [ELF](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) executable formats are supported) into memory. This is the code that is executed inside a virtual machine.
-2. Allocates additional memory regions, for example stack and heap for the guest, as well as some regions used for communication between the host and the guest.
+2. Allocates a scratch memory region. The guest maps its stack and heap from scratch on first touch. Scratch also holds the regions used for communication between the host and the guest.
 3. Creates a Virtual Machine and maps shared memory into it
 5. Create one virtual CPU (vCPU) within the newly created VM
 6. Write appropriate values to the new vCPUs registers.

docs/snapshot-oci-format.md

@@ -32,7 +32,7 @@ Four blob kinds per tag:
 * **manifest** (`application/vnd.oci.image.manifest.v1+json`). Tiny JSON
   pointer record selected via `index.json`. References one config,
   the snapshot blobs, and one transport layer by digest.
-* **config** (`application/vnd.hyperlight.snapshot.config.v4+json`). The
+* **config** (`application/vnd.hyperlight.snapshot.config.v5+json`). The
   snapshot descriptor: arch, hypervisor, CPU vendor, ABI version,
   resume address and captured registers, memory and transport layout,
   registered host functions, the start address, data length, and page-table

docs/snapshot-versioning.md

@@ -27,16 +27,17 @@ A snapshot carries four independently evolvable version markers:
   (`application/vnd.hyperlight.snapshot.transport.v1`), aliased as
   `MT_TRANSPORT_CURRENT`. This is the binary encoding of canonical
   virtqueue state stored outside the memory layer.
-* **Config schema**, `MT_CONFIG_V4`
-  (`application/vnd.hyperlight.snapshot.config.v4+json`), aliased as
+* **Config schema**, `MT_CONFIG_V5`
+  (`application/vnd.hyperlight.snapshot.config.v5+json`), aliased as
   `MT_CONFIG_CURRENT`. This is the JSON shape of the config blob:
   field names, types, required vs optional, the descriptors the loader
   needs in order to reconstruct the sandbox (memory sizes, buffer
   sizes, `abi_version`, `hyperlight_version`, etc.). Renaming a field,
   changing its type, or adding a required field is a schema change and
   bumps this constant. Version 4 describes snapshot blobs, the blob with
   the page tables, and transport geometry, and requires a transport
-  layer. Config v1, v2, and v3 are incompatible with the current ABI.
+  layer. Version 5 drops the heap size. Config v1 to v4 are incompatible
+  with the current ABI.
 
 ABI 6 requires checkpoint readiness after guest initialization and permits
 H2G prefill to omit retained slots. Earlier snapshots must be regenerated.
@@ -48,6 +49,9 @@ base. Earlier snapshots must be regenerated.
 ABI 8 stores snapshot memory as layers. Earlier snapshots must be
 regenerated.
 
+ABI 9 removes the heap region from snapshot memory. Earlier snapshots must
+be regenerated.
+
 The `OCI_LAYOUT_VERSION` constant is pinned by the OCI image-layout
 spec at `1.0.0`.
 

fuzz/fuzz_targets/host_call.rs

@@ -19,7 +19,6 @@ static SANDBOX: OnceLock<Mutex<Sandbox>> = OnceLock::new();
 fuzz_target!(
     init: {
         let mut cfg = SandboxConfiguration::default();
-        cfg.set_heap_size(512 * 1024);
         cfg.set_g2h_pool_pages(16);
         cfg.set_h2g_pool_pages(16);
         cfg.set_scratch_size(512 * 1024);

src/hyperlight_common/src/layout.rs

@@ -25,8 +25,8 @@ pub const SCRATCH_TOP_RESERVED_PAGES: usize = 12;
 // down from the top of scratch memory.
 #[repr(C)]
 struct ScratchTopMetadata {
-    /// Padding that keeps the exception stack 16-byte aligned.
-    _alignment_padding: [u8; 8],
+    /// Nonzero if guest heap pages are mapped executable.
+    heap_executable: u64,
     /// Host-published capacity of each H2G buffer.
     h2g_buffer_size: u64,
     /// Number of pages reserved for the H2G pool.
@@ -85,10 +85,15 @@ pub const SCRATCH_TOP_LIBC_RNG_SEED_OFFSET: u64 =
     scratch_top_offset(offset_of!(ScratchTopMetadata, libc_rng_seed));
 pub const SCRATCH_TOP_GUEST_LOG_LEVEL_OFFSET: u64 =
     scratch_top_offset(offset_of!(ScratchTopMetadata, guest_log_level));
+pub const SCRATCH_TOP_HEAP_EXECUTABLE_OFFSET: u64 =
+    scratch_top_offset(offset_of!(ScratchTopMetadata, heap_executable));
 pub const SCRATCH_TOP_EXN_STACK_OFFSET: u64 = size_of::<ScratchTopMetadata>() as u64;
 
 const _: () = {
-    assert!(size_of::<ScratchTopMetadata>().is_multiple_of(EXN_STACK_ALIGNMENT));
+    assert!(
+        size_of::<ScratchTopMetadata>().is_multiple_of(EXN_STACK_ALIGNMENT),
+        "pad ScratchTopMetadata at its low end to keep the exception stack 16-byte aligned"
+    );
     assert!(SCRATCH_TOP_SIZE_OFFSET == 0x08);
     assert!(SCRATCH_TOP_ALLOCATOR_OFFSET == 0x10);
     assert!(SCRATCH_TOP_SNAPSHOT_PT_GPA_BASE_OFFSET == 0x18);
@@ -102,6 +107,7 @@ const _: () = {
     assert!(SCRATCH_TOP_H2G_QUEUE_SIZE_OFFSET == 0x58);
     assert!(SCRATCH_TOP_H2G_POOL_PAGES_OFFSET == 0x60);
     assert!(SCRATCH_TOP_H2G_BUFFER_SIZE_OFFSET == 0x68);
+    assert!(SCRATCH_TOP_HEAP_EXECUTABLE_OFFSET == 0x70);
     assert!(SCRATCH_TOP_EXN_STACK_OFFSET == 0x70);
 
     assert!(VIRTQ_BUFFER_GVA_START < VIRTQ_BUFFER_GVA_END);

src/hyperlight_common/src/mem.rs

@@ -15,7 +15,6 @@ pub struct GuestMemoryRegion {
 #[repr(C)]
 pub struct HyperlightPEB {
     pub init_data: GuestMemoryRegion,
-    pub guest_heap: GuestMemoryRegion,
 }
 
 #[cfg(test)]
@@ -29,10 +28,6 @@ mod tests {
                 size: 0x1111,
                 ptr: 0x2222,
             },
-            guest_heap: GuestMemoryRegion {
-                size: 0x3333,
-                ptr: 0x4444,
-            },
         };
         let bytes = bytemuck::bytes_of(&peb);
         let peb2 = *bytemuck::from_bytes::<HyperlightPEB>(bytes);

src/hyperlight_guest/src/arch/aarch64/layout.rs

@@ -5,6 +5,11 @@
 pub const MAIN_STACK_TOP_GVA: u64 = 0x0000_ff00_0000_0000;
 pub const MAIN_STACK_LIMIT_GVA: u64 = 0x0000_fe00_0000_0000;
 
+/// Guest heap. Pages are mapped from scratch on first touch. The range
+/// is 512 GiB aligned so it sits under one level 0 table entry.
+pub const HEAP_GVA_START: u64 = 0x0000_fd80_0000_0000;
+pub const HEAP_GVA_END: u64 = 0x0000_fe00_0000_0000;
+
 pub fn scratch_size() -> u64 {
     let addr = crate::layout::scratch_size_gva();
     unsafe { addr.read_volatile() }

src/hyperlight_guest/src/arch/amd64/layout.rs

@@ -13,6 +13,11 @@
 pub const MAIN_STACK_TOP_GVA: u64 = 0xffff_ff00_0000_0000;
 pub const MAIN_STACK_LIMIT_GVA: u64 = 0xffff_fe00_0000_0000;
 
+/// Guest heap. Pages are mapped from scratch on first touch. The range
+/// is 512 GiB aligned so it sits under one PML4 entry.
+pub const HEAP_GVA_START: u64 = 0xffff_fd80_0000_0000;
+pub const HEAP_GVA_END: u64 = 0xffff_fe00_0000_0000;
+
 pub fn scratch_size() -> u64 {
     let addr = crate::layout::scratch_size_gva();
     let x: u64;

src/hyperlight_guest/src/layout.rs

@@ -5,7 +5,14 @@
 #[cfg_attr(target_arch = "aarch64", path = "arch/aarch64/layout.rs")]
 mod arch;
 
-pub use arch::{MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA};
+pub use arch::{HEAP_GVA_END, HEAP_GVA_START, MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA};
+
+const _: () = assert!(
+    hyperlight_common::layout::VIRTQ_BUFFER_GVA_END <= HEAP_GVA_START
+        && HEAP_GVA_START < HEAP_GVA_END
+        && HEAP_GVA_END <= MAIN_STACK_LIMIT_GVA,
+    "heap overlaps the transport aliases or the stack"
+);
 
 fn scratch_top_gva(offset: u64) -> *mut u64 {
     (hyperlight_common::layout::SCRATCH_TOP_GVA as u64 - offset + 1) as *mut u64
@@ -51,6 +58,9 @@ pub fn guest_log_level_gva() -> *mut u64 {
     use hyperlight_common::layout::{SCRATCH_TOP_GUEST_LOG_LEVEL_OFFSET, SCRATCH_TOP_GVA};
     (SCRATCH_TOP_GVA as u64 - SCRATCH_TOP_GUEST_LOG_LEVEL_OFFSET + 1) as *mut u64
 }
+pub fn heap_executable_gva() -> *mut u64 {
+    scratch_top_gva(hyperlight_common::layout::SCRATCH_TOP_HEAP_EXECUTABLE_OFFSET)
+}
 pub use arch::{scratch_base_gpa, scratch_base_gva};
 
 /// Exclusive upper GPA for the scratch page allocator: the transport arena,

src/hyperlight_guest_bin/src/arch/aarch64/exception/handle.rs

@@ -9,16 +9,23 @@ use hyperlight_common::arch::exn::{
 use hyperlight_common::vmem::{BasicMapping, Mapping, MappingKind, PAGE_SIZE, PhysAddr, VirtAddr};
 use hyperlight_guest::error::ErrorCode;
 use hyperlight_guest::exit::write_abort;
-use hyperlight_guest::layout::{MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA};
+use hyperlight_guest::layout::{
+    HEAP_GVA_END, HEAP_GVA_START, MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA,
+};
 
 use super::super::{mrs, msr};
 use super::types::*;
 use crate::HyperlightAbortWriter;
 
-fn handle_stack_fault(far: u64) {
-    // TODO: perhaps we should have a sanity check that the
-    // stack grows only one page at a time, which should be
-    // ensured by our stack probing discipline?
+/// Maps a fresh scratch page at `far` in the stack or heap range.
+///
+/// # Safety
+/// `far` must be in the stack or heap range and must not be mapped. No
+/// other page table operation may run concurrently.
+unsafe fn handle_stack_or_heap_fault(far: u64, executable: bool) {
+    // SAFETY: the scratch allocator is set up before any exception can
+    // occur, and the masked address is page aligned. The caller upholds
+    // the rest of the `map_region` contract.
     unsafe {
         let new_page = hyperlight_guest::prim_alloc::alloc_phys_pages(1);
         crate::paging::map_region(
@@ -28,7 +35,7 @@ fn handle_stack_fault(far: u64) {
             MappingKind::Basic(BasicMapping {
                 readable: true,
                 writable: true,
-                executable: false,
+                executable,
             }),
         );
         // We don't use crate::barrier::first_valid_same_ctx, because
@@ -90,7 +97,22 @@ fn handle_internal_fault(exn: Exception, far: u64, orig_mapping: Option<Mapping>
             ..
         }) => {
             if (MAIN_STACK_LIMIT_GVA..MAIN_STACK_TOP_GVA).contains(&far) {
-                handle_stack_fault(far);
+                // TODO: perhaps we should have a sanity check that the
+                // stack grows only one page at a time, which should be
+                // ensured by our stack probing discipline?
+                // SAFETY: a translation fault in the stack region means
+                // the page is unmapped, and exception handling is the
+                // only page table writer.
+                unsafe { handle_stack_or_heap_fault(far, false) };
+                true
+            } else if (HEAP_GVA_START..HEAP_GVA_END).contains(&far) {
+                // SAFETY: the host writes the scratch-top metadata before entry.
+                let executable =
+                    unsafe { hyperlight_guest::layout::heap_executable_gva().read_volatile() } != 0;
+                // SAFETY: a translation fault in the heap region means
+                // the page is unmapped, and exception handling is the
+                // only page table writer.
+                unsafe { handle_stack_or_heap_fault(far, executable) };
                 true
             } else if let Some(
                 m @ Mapping {

src/hyperlight_guest_bin/src/arch/amd64/exception/handle.rs

@@ -7,7 +7,9 @@ use core::sync::atomic::Ordering;
 use hyperlight_common::outb::Exception;
 use hyperlight_common::vmem::{BasicMapping, MappingKind, PAGE_SIZE, PhysAddr, VirtAddr};
 use hyperlight_guest::exit::write_abort;
-use hyperlight_guest::layout::{MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA};
+use hyperlight_guest::layout::{
+    HEAP_GVA_END, HEAP_GVA_START, MAIN_STACK_LIMIT_GVA, MAIN_STACK_TOP_GVA,
+};
 
 use super::super::context::Context;
 use super::super::machine::ExceptionInfo;
@@ -46,10 +48,15 @@ pub type ExceptionHandler = fn(
     page_fault_address: u64,
 ) -> bool;
 
-fn handle_stack_pagefault(gva: u64) {
-    // TODO: perhaps we should have a sanity check that the
-    // stack grows only one page at a time, which should be
-    // ensured by our stack probing discipline?
+/// Maps a fresh scratch page at `gva` in the stack or heap range.
+///
+/// # Safety
+/// `gva` must be in the stack or heap range and must not be mapped. No
+/// other page table operation may run concurrently.
+unsafe fn handle_stack_or_heap_pagefault(gva: u64, executable: bool) {
+    // SAFETY: the scratch allocator is set up before any exception can
+    // occur, and the masked address is page aligned. The caller upholds
+    // the rest of the `map_region` contract.
     unsafe {
         let new_page = hyperlight_guest::prim_alloc::alloc_phys_pages(1);
         crate::paging::map_region(
@@ -59,7 +66,7 @@ fn handle_stack_pagefault(gva: u64) {
             MappingKind::Basic(BasicMapping {
                 readable: true,
                 writable: true,
-                executable: false,
+                executable,
             }),
         );
         // This is mapping an entry for the first time, so we don't
@@ -113,18 +120,34 @@ fn try_handle_internal_pagefault(
 ) -> bool {
     let error_code = unsafe { (&raw const (*exn_info).error_code).read_volatile() };
 
-    let mut orig_mappings = crate::paging::virt_to_phys(gva);
-    let orig_mapping = orig_mappings.next();
-
     let present = (error_code & (1 << 0)) != 0; // bit 0 is P
     if !present {
-        // If the fault was caused by a not-present page, check if we
-        // should populate it with a stack page
+        // The stack and heap ranges need no page table walk.
         if (MAIN_STACK_LIMIT_GVA..MAIN_STACK_TOP_GVA).contains(&gva) {
-            handle_stack_pagefault(gva);
+            // TODO: perhaps we should have a sanity check that the
+            // stack grows only one page at a time, which should be
+            // ensured by our stack probing discipline?
+            // SAFETY: the fault is not-present in the stack region,
+            // and exception handling is the only page table writer.
+            unsafe { handle_stack_or_heap_pagefault(gva, false) };
+            return true;
+        }
+        if (HEAP_GVA_START..HEAP_GVA_END).contains(&gva) {
+            // SAFETY: the host writes the scratch-top metadata before entry.
+            let executable =
+                unsafe { hyperlight_guest::layout::heap_executable_gva().read_volatile() } != 0;
+            // SAFETY: the fault is not-present in the heap region, and
+            // exception handling is the only page table writer.
+            unsafe { handle_stack_or_heap_pagefault(gva, executable) };
             return true;
         }
-        // Or, check if it is in any zero-init region
+    }
+
+    let mut orig_mappings = crate::paging::virt_to_phys(gva);
+    let orig_mapping = orig_mappings.next();
+
+    if !present {
+        // Check if it is in any zero-init region
         if let Some(mapping) = orig_mapping
             && let MappingKind::ZeroInit(bm) = mapping.kind
         {

src/hyperlight_guest_bin/src/arch/amd64/layout.rs

@@ -15,3 +15,8 @@ const _: () = assert!(
     hyperlight_common::layout::VIRTQ_BUFFER_GVA_END <= PROC_CONTROL_GVA,
     "transport aliases overlap processor control"
 );
+const _: () = assert!(
+    PROC_CONTROL_GVA + 2 * hyperlight_common::vmem::PAGE_SIZE as u64
+        <= hyperlight_guest::layout::HEAP_GVA_START,
+    "processor control overlaps the heap"
+);

src/hyperlight_guest_bin/src/lib.rs

@@ -17,6 +17,7 @@ use hyperlight_common::mem::HyperlightPEB;
 use hyperlight_common::outb::OutBAction;
 use hyperlight_guest::exit::write_abort;
 use hyperlight_guest::guest_handle::handle::GuestHandle;
+use hyperlight_guest::layout::{HEAP_GVA_END, HEAP_GVA_START};
 use talc::TalcLock;
 use talc::source::Manual;
 
@@ -264,18 +265,20 @@ pub(crate) extern "C" fn generic_init(
         #[allow(static_mut_refs)]
         let peb_ptr = GUEST_HANDLE.peb().unwrap();
 
-        let heap_start = (*peb_ptr).guest_heap.ptr as usize;
-        let heap_size = (*peb_ptr).guest_heap.size as usize;
         #[cfg(not(all(feature = "mem_profile", target_arch = "x86_64")))]
         let heap_allocator = &HEAP_ALLOCATOR;
         #[cfg(all(feature = "mem_profile", target_arch = "x86_64"))]
         let heap_allocator = &HEAP_ALLOCATOR.0;
-        // SAFETY: the host reserves the PEB heap range for the guest heap, and
-        // init claims it once. Only this allocator writes to it.
+        // SAFETY: nothing else uses the heap range, and init claims it
+        // once. The fault handlers, installed before this point, map
+        // each page from scratch on first touch.
         heap_allocator
             .try_lock()
             .expect("Failed to access HEAP_ALLOCATOR")
-            .claim(heap_start as *mut u8, heap_size)
+            .claim(
+                HEAP_GVA_START as *mut u8,
+                (HEAP_GVA_END - HEAP_GVA_START) as usize,
+            )
             .expect("Failed to claim guest heap");
         peb_ptr
     };