Repository navigation
Conversation
…registry
With fjmethodid=lazy (HotSpot, cstack=vm), no jmethodIDs are preloaded at
ClassPrepare. Instead every class is recorded in KlassRegistry, a Klass* to
weak-mirror table. walkVM keeps storing raw Method* frames for classes
without jmethodIDs; at dump time resolve() reads only the holder Klass* and
idnum from the Method*, pins the registered mirror (null if the class was
unloaded), checks that the Method* is among the pinned class's current
methods, and only then allocates jmethodIDs for that one class.
Falls back to the partial mode when the Klass*/methods vmStructs are
missing. Adds method_resolve_lazy_{unloaded,not_current,class_loaded}
counters, gtests for the registry, containsMethod, argument parsing and the
load-state selection, and a Java test covering custom-loader resolution and
frames of an unloaded class.
Klass* address reuse after class unloading is not handled yet; see
doc/LazyJmethodIdResolution.md (investigation notes, to be removed before
landing).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ass* addresses A class unloaded and replaced by a new class at the same Klass* (and Method*) addresses made frames sampled from the old class resolve as methods of the new one, and made both produce identical raw frames. KlassRegistry now keeps an epoch, bumped when a known Klass* gets a different mirror and when a rehash purges dead entries; each entry records the epoch it was registered at. walkVM stores the current epoch in every raw frame as a 26-bit tag (10 spare bci bits plus the top 16 bits of the Method*), and pin() refuses frames older than the entry. A Method* that uses the tag's address bits is stored as not walkable. If registering a class fails, its jmethodIDs are preloaded so it never yields raw frames. resolve() now takes the frame's bci to recover the tag. Adds method_resolve_lazy_stale, klass_registry_add_failed and walkvm_raw_frame_untaggable counters, an end-to-end gtest for the Klass*-reuse cases, epoch tests for the registry and tag round-trip tests for FrameType. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
KlassTraceId::calibrate() makes the JVM create six array klasses of java.lang.Void back to back and accepts the single 8-byte-aligned offset within sizeof(Klass) at which all of them hold distinct JFR trace IDs (raw >> 16) no further apart than consecutive creation allows; anything else, including an ambiguous match, leaves the offset unknown. Runs when fjmethodid=lazy is selected and publishes the result in the klass_trace_id_offset gauge. Not used by the sampler yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When KlassTraceId found Klass::_trace_id, walkVM now stores the low 26 bits of the holder's trace ID (read with SafeAccess via the new VMMethod::holderOrNull()) as the raw frame's tag instead of the registry epoch, and resolveViaRegistry() compares it with the pinned class's trace ID. The ID is unique per load of a class, so frames of an earlier class at a reused Klass* are refused without the epoch, and unrelated registry bumps no longer change a stack's frames. Frames whose holder trace ID is unreadable are stored as not walkable. The tag source is chosen once per process: calibration now runs only on the first lazy start, and the klass_trace_id_offset gauge is republished on every start since counters are reset then. The end-to-end lazy-resolve gtest runs every case under both tag sources. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…st JFR in CI The top bit of a raw frame's tag now says whether the rest is a trace ID or a registry epoch (25 bits each), and resolve() goes by the frame's own kind rather than the current tag source. In debug builds, resolveViaRegistry() logs the trace ID it read for each class it allocates jmethodIDs for. LazyTraceIdJfrValidationTest runs a child JVM (new ExternalLauncher mode lazy-trace-id) with the profiler in lazy mode and JFR's own sampler, and requires JFR's RecordedClass.getId() for the probe class to equal that ID. The spin class generator moves to a shared SpinClasses test helper. The profiler does not call into JFR at runtime: JVM.getClassId() reaches JFR state that only exists once its recorder has been created, and the profiler itself initializes jdk.jfr.internal.JVM (tsc.cpp), so that class being initialized is no sign the recorder exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes for fjmethodid=lazy: - VMKlass::containsMethod() reads the methods array only through SafeAccess. Pinning the class does not keep the array alive: a redefinition swaps in a new one at a safepoint and frees the old one at a later safepoint, neither of which waits for the native dump thread. - Redefinition: the RedefineClasses/RetransformClasses hooks mark the class in KlassRegistry before calling the original function and preload its jmethodIDs afterwards. pin() refuses raw frames of a marked class, and resolveViaRegistry() checks the mark again after its metadata reads, since a redefinition can free a Method* and reuse its address within the same class without changing its mirror, trace ID or epoch. New counter method_resolve_lazy_redefined; pin() now reports a PinStatus. - Every registration bumps the registry epoch. Previously only replacements and purges did, so a dead class's Method* reused by a class loaded at a different Klass* could match an epoch-tagged frame. - klass_trace_id_beyond_tag_range counts registrations of classes whose trace ID no longer fits in the 25-bit tag, the first point at which two classes' trace-ID tags can collide. - klassRegistry_ut includes vmStructs.inline.h so it links in release. The doc gets a "Guarantees and limits" section stating the exact condition under which a tag collision can still misattribute a frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion samples LazyTraceIdJfrValidationTest now records only jdk.ClassDefine in the child JVM (settings written by the test) and compares the probe class's RecordedClass.getId() from that event with the trace ID the profiler logged. JFR emits the event once per class on every JFR-enabled release, whereas JFR's pre-25 sampler recorded almost no jdk.ExecutionSample events for the spinning probe, even without the profiler running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lookup now keeps a per-dump map from (untagged Method*, raw-frame tag) to the resolved jmethodID, or nullptr for a refused frame, so a raw method that recurs across traces is resolved once per dump instead of once per occurrence. Each resolution sets up crash protection, takes the KlassRegistry mutex and pins the class through JNI. New counter raw_frame_resolve_cache_hits. Tests cover repeated occurrences with different bci and frame type, a fresh cache per dump, and frames from two loads of a class at the same addresses staying apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RedefineClassesHook and RetransformClassesHook now forward a call with a positive count and a null array straight to the original function, which returns JVMTI_ERROR_NULL_POINTER; before, the hooks' own loops dereferenced the array first and crashed, in every jmethodID mode. In fjmethodid=lazy, the per-class work before and after the call only touches elements that JVMTI's GetClassStatus accepts as instance classes (HotspotSupport::isInstanceClass, also used by registerClass), so a non-class object or an array class is never looked up as a Klass* or preloaded. redefineHooks_ut calls both hooks with stubbed originals in lazy and full-preload mode: null arrays, non-class, array-class and null elements. Without the fix its first case crashes with SIGSEGV. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LazyRedefinitionStressTest loads a small agent by self-attach to get Instrumentation, then redefines a generated class with its own bytes about every 2 ms while another thread spins in it, fjmethodid=lazy samples it and the profiler dumps every 100 ms. Raw frames of the class are collected for 300 ms before the first redefinition, so the first dumps afterwards have stale raw frames of a marked class to refuse. The test requires the JVM to survive, every redefinition to be marked, a stale raw frame to be refused, and frames sampled after the redefinitions to resolve by name. New counter klass_registry_redefinitions_marked counts the marks the redefinition hooks set. SpinClasses can hand out a class's bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raw frames of fjmethodid=lazy are now always tagged with the holder's JFR trace ID. When the trace-ID offset can't be calibrated (8u before 8u262, builds without JFR), lazy mode falls back to partial mode instead of tagging frames with the registry epoch. The 25-bit epoch wrapped after 2^24 registrations, after which every frame of a long-lived class was refused, and bumping it on every registration split identical stacks into separate traces. KlassRegistry no longer keeps an epoch. - Only lazy mode tags frames. Partial-mode frames are plain Method*s again, and resolve() picks the lazy or partial path from the frame's marker bit rather than the current mode, so frames that outlive a restart in the other mode still resolve the way they were sampled. - Redefinition: the hooks call KlassRegistry::beginRedefinition() before and endRedefinition() after the original function. A class is refused while a call runs and for good once one succeeds; a call JVMTI rejects leaves it as before and preloads nothing. Redefinitions are tracked in every mode once the registry has entries. - Mode selection uses the stack walker mode the profiler resolved, not the cstack argument: cstack=default resolves to dwarf where the VM walker is unavailable, and then all jmethodIDs must be preloaded. This also applied to fjmethodid=false. - VMKlass::containsMethod() scans at most 256 methods after the idnum fast path. - readMethodHolder() uses VMMethod::holderOrNull(). - The rehash() comment now matches what the code does. The prototype design notes (doc/LazyJmethodIdResolution.md) are removed; their content moves to the PR description. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The parse runs in the thread that reads the child JVM's output, so an exception there would stop that thread instead of failing the test. A value that does not parse now leaves the logged ID unset, which the existing assertion reports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Working notes for the branch, to be removed before the PR lands. Co-Authored-By: Zhengyu Gu <zhengyu.gu@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HotspotSupport::resolve() now reads a raw Method* frame's class name, method name, signature, Thread.run status and line number table straight from HotSpot metadata (readMethod(), under crash protection) and returns them in a ResolvedMethod. In fjmethodid=lazy mode this runs while the class is pinned through KlassRegistry, after the trace-ID and current-method checks, so the dump no longer allocates jmethodIDs for sampled classes. Partial mode reads the same metadata instead of looking the method up through JNI FindClass/GetMethodID, which also covers <clinit> without a special case. - vmStructs: ConstMethod::_flags and _code_size, Klass::_super, the version-dependent has-line-number-table constant, and a decoder for the compressed line number table (VMLineNumberStream). - Lookup: raw frames get MethodMap rows keyed by the untagged Method* and tag bits (RAW_METHOD_MARK); refused frames never touch a row, and rows are refilled at their first use in every chunk. The JVMTI and VM-metadata paths share fillJavaMethodNames(); line tables decoded from VM metadata are malloc'd and freed with free(). - Counter method_resolve_lazy_class_loaded becomes method_resolve_lazy_resolved. - Tests: the resolve() crash-protection gtests and the line table, Thread.run and bci-bound cases; the lazy-resolution gtests now run on a class without jmethodIDs and check none are allocated; LazyLineNumberConsistencyTest compares decoded line numbers with JVMTI's; SpinClasses' generated method has a line number table. Co-Authored-By: Zhengyu Gu <zhengyu.gu@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Zhengyu Gu <zhengyu.gu@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
🔗 Commit SHA: 1d1a3d7 | Docs | View more details | Give us feedback! |
Contributor
CI Test ResultsRun: #39307521270 | Commit:
Status Overview
Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled Summary: Total: 32 | Passed: 32 | Failed: 0 Updated: 2026-10-19 19:24:01 UTC |
Contributor
The test attaches its agent through com.sun.tools.attach.VirtualMachine, which on JDK 8 lives in the JDK's lib/tools.jar and is not on the test classpath, so the test was always skipped there. Load the attach API from tools.jar through a child class loader when it isn't on the classpath. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A lazy raw frame whose Method* no longer leads to a holder Klass* and ConstMethod (its metadata freed after class unloading, or the read faulted) was refused without being counted by any method_resolve_lazy_* counter. Under class-loader churn this is most of the lost frames, so the loss was largely invisible. Count it as method_resolve_lazy_unreadable. Adds a gtest that clears a sampled method's ConstMethod and checks that only the new counter moves, and prints the counter in LazyJmethodIdTest's unload test. Environment: Datadog workspace Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration probed Void arrays of dimensions 1-6 once per process and kept a failure for good. If some of those dimensions already existed, their trace IDs were far apart from the new ones, no offset qualified, and fjmethodid=lazy stayed off for every later start in that JVM. calibrate() now makes up to four attempts, each on the six dimensions above the previous attempt's, so every attempt probes klasses it has just created. Only a found offset is kept; after a failure the next start tries again. Debug builds log which early exit was taken and dump the probes' words when no offset qualifies. LazyTraceIdCalibrationTest creates Void[] and Void[][] with a few hundred class definitions in between before starting lazy mode in a child JVM, and requires calibration to succeed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LazyLineNumberConsistencyTest and LazyTraceIdCalibrationTest parse a counter from the child JVM's output in the line consumer. A malformed line threw NumberFormatException from the stdout reader and left the value unset, which the tests' assumptions then turned into a skip. Keep the line and fail on it after the child exits. The line-number workload no longer parses the regex's digits; it merges their length, which is the same regex and map work. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 of 3 tasks
HotSpot names a hidden class "<name>+0x<hex>", and JVMTI GetClassSignature() reports it with that last '+' turned into '.'. Raw frames copied Klass::_name as is, so a lambda's frames were named differently with and without jmethodIDs, and one class could appear under both names in a chunk. readMethod() now applies the same rename; the hidden flag is not in vmStructs on JDK 21+, so the "+0x<hex>" suffix identifies the class. Also correct which frames reach the partial-mode path in resolve(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Zhengyu Gu <zhengyu.gu@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NMT does not see the registry's table, which the profiler allocates with calloc. klass_registry_capacity and klass_registry_entries (gauges) give its slots (24 B each) and occupied entries, live or dead, and klass_registry_purged counts dead entries dropped when the table is rebuilt. Lazy mode's start republishes the gauges after counters are reset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lazy mode kept a JNI weak reference and a 24-byte table entry for every loaded class, registered at ClassPrepare. Dead entries were only purged when the table had to grow, so after heavy class churn they could stay for good: on JDK 26, where the JVM frees jmethodIDs, that left lazy mode using more memory than preloading. Every dump already pins all loaded classes: Recording::finishChunk() holds the local references GetLoadedClasses() returns until it ends. The dump now hands that list to HotspotSupport::beginDump(), and the first lazy resolution of the dump collects the classes' Klass*s into a temporary set, freed by endDump(). A raw frame is read only if its holder is in that set; the trace-ID, current-method and redefinition checks are unchanged. A dump without lazy frames builds no set. ClassPrepare no longer does anything in lazy mode. KlassRegistry now only holds the redefinition marks: the hooks register the class they mark, and a mark is no longer reported once that class has been unloaded, so a new class at the same address starts unmarked. New gauges lazy_dump_classes and lazy_dump_pin_ns report the classes a dump pinned and the time it took to collect them; klass_trace_id_beyond_tag_range becomes a gauge computed from them. klass_registry_add_failed is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- finishChunk() always calls beginDump(), with no list if GetLoadedClasses() failed, and ends the dump through a scope guard, so an exception out of writeCpool() can't leave a later dump reading a stale class list whose local references are gone. - A redefinition whose mark can't be recorded (no memory for the registry entry) is counted in klass_registry_add_failed, and every lazy frame is refused from then on instead of being trusted unmarked. - _lazy_used is read by redefining threads, so it is published with release/acquire atomics. - The per-dump Klass* table is mmap'd (OS::safeAlloc): at hundreds of thousands of classes it is megabytes, allocated and freed every dump, which would fragment the malloc arenas. - Lazy mode skips the start-up GetLoadedClasses() pass, which only registered classes. - Stale comments describing the class registry are updated. New gtests cover a dump without a class list, a lookup among 3000 pinned classes, an unmarkable redefinition, and a redefinition before lazy mode started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Effect (opt-in via
fjmethodid=lazy; HotSpot withcstack=vm; J9, Zing and the default mode are unchanged):Method*frames are named, with line numbers, straight from VM metadata. The only classes that still get jmethodIDs are those redefined during the session.GetClassMethodsor take the JVM's jmethodID-creation lock. The profiler keeps no per-class state between dumps.finagle-http.What does this PR do?:
It adds
fjmethodid=lazy, which profiles without jmethodIDs. Todayfjmethodid=falsealready stores rawMethod*frames, but only for classes of the boot, platform and application loaders. Custom-loader, hidden and lambda classes, the ones with high churn, are still preloaded. Lazy mode preloads nothing and resolves every raw frame at dump time:This combines the class-liveness design of #864 with the VM-metadata method resolution of #867 (Zhengyu Gu, co-author), and replaces both.
Motivation:
Preloading jmethodIDs costs memory, about 16 bytes per method for every method of every loaded class, and class-loading time. Most of it goes to classes that are never sampled.
How it works:
Flow
HotspotSupport::fillJavaFrame()stores the rawMethod*with a tag (below). The method is on the stack, so it is valid at that moment.Recording::finishChunk()callsGetLoadedClassesat the start of every dump and holds the returned local references until the dump ends, so no loaded class can be unloaded meanwhile. It hands the list toHotspotSupport::beginDump(). The dump's first lazy resolution collects these classes'Klass*s into a temporary hash set, freed byendDump(); a dump without lazy frames builds none. Then, for each raw frame, inHotspotSupport::resolve():Klass*and the idnum from the possibly staleMethod*(readMethodHolder()). Both serve as lookup keys and are never trusted. If they can't be read, the frame is unknown (method_resolve_lazy_unreadable).Method*is garbage, and the frame is unknown (method_resolve_lazy_unloaded). A pinned class keeps its class loader, and so the class's metadata, alive.method_resolve_lazy_stale).VMKlass::containsMethod(M, idnum)compares pointers against the class's current methods array, throughSafeAccess. A miss means theMethod*was freed or replaced (method_resolve_lazy_not_current).Mread:readMethod()copies the method name, signature and holder name, detectsThread.runoverrides via theKlass::_superchain, and decodes the compressed line number table. All of this runs under crash protection, with the class pinned (method_resolve_lazy_resolved).Lookup::fillRawMethod()resolves each distinct (Method*, tag) at most once per dump. Refused frames all go to the shared unknown row and never touch a method row. Resolved frames get a row keyed by the untaggedMethod*plus tag bits, kept apart from jmethodID keys. A row is refilled at its first use in every chunk, so a row from an earlier chunk is never trusted without resolving again.Lazy mode requires the stack walker to be
vm, as resolved by the profiler (cstack=defaultbecomesdwarfwhere the VM walker is unavailable); otherwise everything is preloaded as in the default mode. It also needsVMStructs::hasKlassMethods()and a calibrated trace-ID offset; without them it uses partial mode. HotSpot only.Reading names and line numbers from VM metadata
Method → ConstMethod → ConstantPool → {name, signature}Symbols, andConstantPool → holder Klass → name. Every pointer step goes throughSafeAccessor runs under asigsetjmplanding pad. Names longer than the inline buffers (512 B for method names, 1 KiB for signatures and class names) fall back tomalloc, up to 64 KiB.ConstMethod::code_end()), whenConstMethod::_flagshas the "has line number table" bit. That bit isConstMethod::_has_linenumber_tableon JDK 8–20 andConstMethodFlags::_misc_has_linenumber_tableon 21+.VMLineNumberStreammirrorsCompressedLineNumberReadStream, including the UNSIGNED5 change in JDK 20. A table whose bcis go pastcode_size(the boundConstMethod::verify_on()checks) is rejected as stale metadata.<name>+0x<hex>, and JVMTI'sGetClassSignature()turns that last+into.. The raw path does the same, so lambda frames are named alike in every mode.ConstMethod::_flags/_code_size,Klass::_super, and the version-dependent flag constant above.Raw frame tags (
Klass*/Method*reuse)Freed class metadata is reused: a new class can get the same
Klass*and the sameMethod*addresses as an unloaded one. So every lazy-mode raw frame carries a 26-bit tag, stored in 10 spare bci bits plus the top 16 bits of theMethod*. The tag is a marker bit plus the low 25 bits of the holder's JFR class trace ID (Klass::_trace_id). JFR assigns that ID at class creation from one global counter, whether or not JFR is recording, so it is unique per load of a class. Frames from different loads of a class therefore never merge into one trace, and stale frames are refused at dump time.Partial-mode (
fjmethodid=false) frames stay untagged, as before.resolve()goes by the marker bit of each frame, not by the current mode. A lazy-modeMethod*that uses the tag's address bits, or whose holder's trace ID can't be read, is stored as not walkable (walkvm_raw_frame_untaggable), never untagged.Finding
Klass::_trace_idThe field isn't in
gHotSpotVMStructs, and its position moves between releases; on mainline it even depends on a build option. SoKlassTraceId::calibrate()finds it by probing:java.lang.Voidarray klasses of six consecutive dimensions, back to back.sizeof(Klass)at which all six hold distinct trace IDs, no further apart than consecutive creation allows. Pointers, small integers and mark words fail this test, and more than one match counts as a failure.klass_trace_id_offsetgauge.What calibration relies on, checked against 107 OpenJDK GA tags
_trace_idis declared inclass Klass(so array klasses carry it);arrayKlass.cppassigns it (INIT_ID(this)); the ID iscounter << 16. All three hold on every JFR-enabled tag:_trace_id(pre-JFR): calibration fails, partial mode_biased_lock_revocation_count_access_flags_access_flags_access_flags_access_flags_archived_mirror_indexCalibrated offsets at runtime: 8u462 → 184, 11.0.28 → 168, 17.0.16 → 168, 21.0.8 → 184, 25 → 192, 26 → 192, 27 → 192.
Pinning per dump instead of a class registry
GetLoadedClassesreturns a JNI local reference for every loaded class, including hidden classes and arrays, after marking each class loader alive. Lazy mode adds oneKlass*read per loaded class (GetLongFieldonjava.lang.Class.klass), only in dumps that have lazy frames.Klass*with at least twice as many slots as classes, allocated for the dump and freed after it. Nothing is kept between dumps.GetLoadedClassescall isn't pinned, so a frame of it in that dump resolves as unknown.lazy_dump_classesandlazy_dump_pin_nsreport how many classes a dump pinned and how long collecting theirKlass*s took.Redefinition
A redefined class stays alive, but its old methods array and old
Method*s are freed once they're off every stack, and a later redefinition can reuse those addresses within the same class. So:RedefineClasses/RetransformClasseshooks register the class inKlassRegistryand mark it before calling the original function. The registry holds only redefined classes:Klass*→ weak global ref to the mirror plus the mark, 24 B per entry. The mark lasts while the call runs, and for good if it succeeds. After a successful call in lazy mode, the class's jmethodIDs are preloaded, so its later frames aren't raw. A call that JVMTI rejects changes nothing.method_resolve_lazy_redefined).resolve()checks the mark again after reading the metadata, under the registry mutex. If the mark is still absent then, every read happened before the methods could be swapped.Klass*address starts unmarked.To check that a method is one of its class's current methods, the profiler looks it up in the methods array. Normally a method sits at the index given by its idnum, so that's a single read. Methods added after parsing can shift entries: bridge methods the JVM generates for default-method conflicts at link time, or a redefinition. Then the array is scanned, but only in classes of at most 256 methods. In a larger class such a frame is reported as unknown.
Partial mode
fjmethodid=falsenow reads raw frames' names from VM metadata too, instead of a JNIFindClass/GetMethodIDlookup. This also covers<clinit>, which JNI can't look up, without a special case. As before, partial mode keeps no redefinition marks. AMethod*freed by a redefinition and then reused resolves to whichever method lives at that address now.Overhead compared with preloading:
fjmethodid=lazyGetClassMethodsplus the jmethodID-creation lock at every ClassPrepareMethod*plus its tag: fourSafeAccessloads (holder chain and trace ID)Klass*read per loaded class (typically 10–70 ns each). Once per distinct raw method per dump: crash-protection setup, pinned-set lookup, redefinition mark (registry mutex), tag and methods checks, metadata copy and line-table decodeMeasurements:
All on x64 Linux (AWS VM, 16 vCPU, not bare metal), release build,
cpu=10ms,cstack=vm,-Xms4g -Xmx4g. Each run is 40 s with a dump every 10 s; 5 fresh JVMs per mode, with modes interleaved. Every lazy run confirmed lazy mode was active.Memory, NMT committed Class + Internal Δ over the no-profiler run, in MiB. These Renaissance runs used the earlier version of this PR, which kept a registry entry and a weak ref for every loaded class; the lazy numbers include those weak refs, and the registry table (not visible to NMT) came on top. Both are gone now, so lazy's cost here can only be lower.
true/lazytrue/lazytrue/lazydottyfinagle-httpals(Spark)About 16,000 raw frames per run are resolved in these lazy runs. Preloading puts the per-class jmethodID cache under Class (
NEW_C_HEAP_ARRAY(jmethodID, …, mtClass)) and the method-ID slots under Internal (JNIMethodBlockNode), 8 B per method each.Synthetic: 1M methods (macOS arm64, JDK 25)
NMT committed Δ in KB, 20,000 classes × 50 methods in custom class loaders, median of 3. "sampled" also calls every class's methods for 8 s at
cpu=1msand dumps before measuring; ~9,300 raw frames are resolved in lazy mode.fjmethodid=truefjmethodid=falsefjmethodid=lazyThe extra ~280 KB of Internal in the sampled runs appears with
trueandfalsetoo, so it comes from sampling and dumping, not from jmethodIDs.Lambda churn: ~1.1M short-lived 1-method classes (macOS arm64, release)
4 threads keep defining hidden classes (JDK 15+) with one method each, run it briefly so it gets sampled, and drop it, at ~18,000 classes/s for 60 s; then
System.gc()and 20 s without new classes, so every churned class is unloaded. Dump every 10 s. Profiler cost over the no-profiler run once everything is unloaded: NMT Class + Internal, plus the table NMT can't see where there is one.true)Dump latency, median ms:
truelazytruelazyfinagle-httpdottyalsfinagle-http, lazy is about 15% faster for both the first and later dumps, with min–max ranges that don't overlap. Elsewhere the modes are within noise; JDK 21's firstdottydump overlaps (lazy 46–70,true49–77).Names of classes unloaded before the dump are lost: such frames serialize as unknown. With jmethodIDs they are lost too, as
<unloaded>. In a survey of all 25 Renaissance benchmarks (lazy, JDK 21, 60 s each), 6,191 classes were unloaded and 269,803 raw frames resolved, and only 6 frames were lost (0.002%). Heavy synthetic churn (~800 classes dying per second) with 60 s dumps loses a large share. Keeping those names is a possible follow-up, but it needs work during GC pauses, so it's left out here.Additional Notes:
cstackargument. Before,fjmethodid=falsewithcstack=defaulton a JVM where that resolves todwarfskipped preloading, althoughdwarfgets its Java frames from AsyncGetCallTrace, which needs jmethodIDs.method_resolve_lazy_class_loadedfrom earlier iterations is gone;method_resolve_lazy_resolvedcounts raw frames resolved per dump.Guarantees and limits
Method*or its metadata goes throughSafeAccessor runs under a landing pad, and reads past the holder only happen while the class is pinned. The holder itself is only a lookup key.Method*points into is a different load with a different ID. Classes loaded after the dump'sGetLoadedClassescall, and classes whoseKlass*can't be read, aren't pinned; their frames resolve as unknown.klass_registry_add_failed).Method*address; and the IDs differ by an exact multiple of 2^25.klass_trace_id_beyond_tag_rangecounts registrations past the tag range./<identity hash>to VM-anonymous class names (lambdas there); lazy mode reports them without it.dottyon JDK 11 counted 14unloadedframes over 5 runs although the JVM unloaded no classes. Those frames resolved as unknown, not as a wrong method, and the cause wasn't found. JDK 21 and 25 showed 0. Not re-measured with per-dump pinning yet.ClassLoaderDatawith those loaders plus the trace-ID check would do, and only other classes would go into the set. Collecting the dump's distinct holders first would bound the set by the sampled classes. Neither avoids the fullGetLoadedClassescall, which the dump also makes for the jmethodID path.Alternatives considered
Klass*→ weak ref, filled at ClassPrepare), pinned per frame throughNewLocalRef: an earlier version of this PR. It cost a hash insert and a weak ref at every class load, and after heavy class churn its dead entries could stay until the table had to grow, which made it worse than preloading on JDK 26 (+16.9 MiB vs +1.9 MiB in the churn benchmark). The dump already pins every loaded class, so that is used instead.Symbols can be freed and reused independently, giving wrong names that no check after the read can catch. The fingerprint only encodes the signature's shape, so many methods share it.JVM.getClassId(): crashed the JVM, because it needs JFR's recorder to exist. The offset is validated against JFR in CI instead (LazyTraceIdJfrValidationTest).New counters
method_resolve_lazy_{resolved,unloaded,unreadable,not_current,stale,redefined},lazy_dump_classesandlazy_dump_pin_ns(gauges),klass_registry_redefinitions_marked,klass_registry_add_failed,klass_registry_{capacity,entries}(gauges) andklass_registry_purged,walkvm_raw_frame_untaggable,klass_trace_id_offsetandklass_trace_id_beyond_tag_range(gauges),raw_frame_resolve_cache_hits.How to test the change?:
klassRegistry_ut: redefinition marks (in progress, failed, overlapping, not carried over to a new class at the same address), table growth and its counters,containsMethodincluding the scan cap.klassTraceId_ut: offset calibration against decoy layouts.hotspotLazyResolve_ut: end to end on a class without jmethodIDs, checking none are allocated; frames resolve only against the classes a dump pins, the set is only built for lazy frames, a dump without a class list pins nothing, and lookups work among thousands of pinned classes; every reuse and redefinition case; partial-mode frames; frames resolved after a mode switch; method rows and the per-dump cache; hidden-class names; the tag-range gauge.hotspotMethodId_ut: crash protection ofresolve(), the malloc fallback for long names, line-table decoding for both UNSIGNED5 variants, the bci bound, a fault in the middle of a table,Thread.rundetection.redefineHooks_ut: invalid arguments to the hooks.frame_ut,arguments_utandjvmSupport_ut.LazyJmethodIdTest: a custom-loader class's frames resolve with the right line numbers; frames of an unloaded class are dropped; calibration finds the offset.LazyLineNumberConsistencyTest: in a fresh JVM, the same workload runs in lazy mode and withfjmethodid=true. Every (method, bci) in both recordings must have the same line, and no class may keep HotSpot's internal hidden-class name.LazyTraceIdJfrValidationTest: the calibrated trace ID equals JFR's class ID fromjdk.ClassDefine(debug builds).LazyTraceIdCalibrationTest: calibration succeeds when some probe dimensions already exist.LazyRedefinitionStressTest: ~1000 live redefinitions racing ~47 dumps.ClinitResolutionTest:<clinit>frames resolve in partial mode.*Lazy*testsklass_trace_id_offsetLine-table comparisons covered 249–389 frames per run. The x64 matrix ran before the calibration retry, the hidden-class naming and per-dump pinning were added. The aarch64 matrix was rerun with per-dump pinning (
d0f4041fe): glibc JDK 17 / 21 / 25 debug, JDK 21 ASan, and musl JDK 8 / 11 / 21 debug all pass every gtest and*Lazy*test, with 0 line-table mismatches and matching JFR trace IDs.For Datadog employees:
credentials of any kind, I've requested a security review (run the
dd:platform-security-reviewskill, or file a request via the PSEC review form).
bewairealso runs automatically on every PR.🤖 Generated with Claude Code