Repository navigation
Skip scanning string and bytes data during garbage collection - #11708
Merged
Merged
Conversation
String and bytes buffers never hold pointers on firmware, so allocate them without the collect flag and collections get cheaper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Garbage collection no longer looks inside string and bytes data for pointers.
Why
Part of #10491.
That data never holds pointers on firmware, so skipping it makes collections faster.
On a real RP2350,
gc.collect()with strings live dropped from 22.8 ms to 10.8 ms.PSRAM boards, the case that motivated #10264, were not tested.
Hardware tested
Metro RP2350 (no PSRAM, about 390 KB free) and Metro M0 Express (about 15 KB free), base and patched from the same commit. Workloads are scaled to fit.
gc.collect()time in µs: mean, with min and max, of 15 trials over 5 alternating flashes. Each trial is the median of 7 batches of collections.jsonon M0)A build with the same call sites that still scans these blocks matched base within 0.04% on every row, so these changes come from scanning, not moved code.
String building with collections as needed, change after subtracting that build's change: RP2350 strs and bytes about -10.5%, dict -2.6%, JSON -1%, control 0%. M0 strs -1.8%, bytes -1.3%, dict -1%, control -0.2%.
struct.pack('O', x),'S',array('O')ValueError: bad typecodeon both boards and buildsio,json)gc.mem_free()Other testing
run-tests.py, base and patchedbasicsandmicropython, collecting on every 3rd allocation, counter build looking for heap pointers in unscanned blocksvstr_tdeliberately not scannedDesktop lab
Desktop lab (cp-desktop-lab, x86_64, 16 MB heap), not a board.
gc.collect()time in µs: median, min and max of 10 process runs, each run the median of 7 samples of 400 collections. The second base column is the same binary run again, as a control.Blocks scanned per collection, from a throwaway counter build:
A run of string building and JSON with a mostly empty heap showed little change: 43.8 to 42.9 ms with strs held, 43.5 to 43.5 ms with JSON held.
Scope
standardvariantstruct.pack('O', obj)crashes with this patch, because its typecodes allow object pointers. Firmware andcoveragereject them. That variant does not build on main today; tested with ringio turned offobjint.c:265,objint_mpz.c:99,persistentcode.c:204, mpz digitsringbuf_alloc, parser chunks, emergency exception bufferSelective collect is CircuitPython only, so this can't go to MicroPython first.
@lynt-smitka your cp-desktop-lab gave the first numbers here, before any board run.
AI assistance
Claude wrote the patch and ran the lab and board tests. I directed the work and reviewed the results.
🤖 Generated with Claude Code