Skip to content

fix(criterion): avoid duplicate setup in simulation iter_batched - #198

Merged
not-matthias merged 1 commit into
mainfrom
cod-3941-fix-duplicate-setup-in-criterion-simulation-iter_batched
Oct 20, 2026
Merged

not-matthias merged 1 commit into
mainfrom
cod-3941-fix-duplicate-setup-in-criterion-simulation-iter_batched

Conversation

@not-matthias

Copy link
Copy Markdown
Member

TLDR: Reuse the prepared input in simulation iter_batched instead of running setup twice for the measured iteration.

  • Avoid keeping an unused input alive during measurement; setup and output destruction remain outside measurement.
  • Add regression coverage for one setup per routine call and input lifetime.
  • Verified both library tests with RUSTFLAGS="--cfg codspeed"; the new test fails before the fix. A native lifecycle probe reports 6 setups and 6 routine calls. Native instrumentation was unavailable, so no performance claim is made.

Fixes COD-3941

@codspeed

codspeed Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 4 benchmarks

⚠️ 59 benchmarks measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. These results are not comparable, so they count as unchanged.

Preventing compiler optimizations

⚠️ 9 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

ℹ️ 49 benchmarks have no flame graph
  • The flamegraph could not be generated because too much time was spent in runtime internals, making the visualization not useful for performance analysis. (1 benchmark)
  • The flamegraph could not be generated because no execution data was recorded when the benchmark was run. Ensure that the benchmark has not been optimized out. (48 benchmarks)

⚡ 9 improved benchmarks
❌ 4 regressed benchmarks
✅ 581 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ WallTime hamiltonian_cycle[5] 935 ns 1,571 ns -40.48%
❌ WallTime add 2 ns 3 ns -33.33%
❌ WallTime rem 5 ns 6 ns -16.67%
❌ WallTime graph_coloring[6] 1.1 µs 1.2 µs -3.35%
⚡ WallTime iter_batched_ref_large_input 13 ns 6 ns ×2.2
⚡ WallTime bench_array1[4] 2 ns 1 ns ×2
⚡ WallTime graph_coloring[4] 1,331 ns 989 ns +34.58%
⚡ WallTime hamiltonian_cycle[6] 1,145 ns 923 ns +24.05%
⚡ WallTime iter_batched_large_input 9 ns 8 ns +12.5%
⚡ WallTime iter_with_setup 47 ns 44 ns +6.82%
⚡ WallTime add_two_integers[(255, 255)] 19 ns 18 ns +5.56%
⚡ WallTime add_two_integers[(42, 13)] 19 ns 18 ns +5.56%
⚡ WallTime iter_batched_per_iteration 58 ns 56 ns +3.57%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing cod-3941-fix-duplicate-setup-in-criterion-simulation-iter_batched (76685ee) with main (5aaf570)

Open in CodSpeed

@not-matthias
not-matthias marked this pull request as ready for review October 20, 2026 12:33
@greptile-apps

greptile-apps Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

Fixes synchronous iter_batched so the measured call uses the input already prepared for it.

  • Each batched routine call uses its prepared input.

No actionable issues found. Tests were not run during this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A["setup() once"] --> B{"Warmup?"}
    B -->|Yes| C["routine(input)"]
    B -->|No| D["start_benchmark()"]
    D --> E["routine(input)"]
    E --> F["end_benchmark()"]
    C --> G["Drop output"]
    F --> G
Loading

Reviews (1) · Last reviewed commit: "fix(criterion): avoid duplicate setup in..." · Reviewed by Greptile

@GuillaumeLagrange GuillaumeLagrange left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@not-matthias
not-matthias force-pushed the cod-3941-fix-duplicate-setup-in-criterion-simulation-iter_batched branch from 9c372b6 to bae9321 Compare October 20, 2026 13:31
Use the input already prepared outside measurement instead of calling setup again for the measured iteration. This prevents an unused input from remaining alive during the routine. Add regression coverage for setup counts and input lifetime.
@not-matthias
not-matthias force-pushed the cod-3941-fix-duplicate-setup-in-criterion-simulation-iter_batched branch from bae9321 to 76685ee Compare October 20, 2026 13:31
@not-matthias
not-matthias merged commit 76685ee into main Oct 20, 2026
46 of 47 checks passed
@not-matthias
not-matthias deleted the cod-3941-fix-duplicate-setup-in-criterion-simulation-iter_batched branch October 20, 2026 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants