bench: resize the microbenchmarks that flip on unrelated pull requests

[connortsui20]

Summary

CodSpeed has been flagging regressions and improvements on pull requests that change no Rust code, such as Renovate digest bumps and docs edits. I collected the CodSpeed report of every pull request merged since 2026-08-24 (195 reports, 1201 flagged rows, 271 distinct benchmarks) and cross-referenced each flagged benchmark with the crates the pull request touched, using the workspace dependency graph. The benchmarks in this PR flipped repeatedly on pull requests that could not have affected them, and the common cause is their size.

Two patterns:

• In the simulation shards CodSpeed runs a benchmark once and adds a fixed cost of roughly half a microsecond of reported time. A benchmark whose real work is tens of nanoseconds reports mostly that floor, and the floor differs between runs. allocate_drop_bytes[0] does about 3 ns of work locally, is reported as 0.5 µs, and flipped on 37 pull requests.
• On the walltime legs, iterations of a few microseconds flip by 10% to 90% between runs of identical code, while longer iterations in the same files do not.

This is the first of three stacked PRs. The second removes third-party and frozen baselines from CodSpeed. The third gives every benchmark binary the same allocator.

Changes

Benchmark	Flips on unrelated PRs (all PRs)	Change
allocation.rs: allocate_drop_*[0]	27 (46), 27 (37), 8 (11), 8 (11)	Drop the zero-byte size, which allocates nothing, and repeat each allocate-and-drop pair 256 times per iteration (the from_vec cases copy, so their batch shrinks with the size to about a mebibyte per iteration)
filter_fixed_width.rs: random_i8, random_i16, cached_indices_*	22 (44), 11 (23), 8 (23), 8 (9), 4 (9), 4 (6)	Size inputs by bytes (64 KiB) instead of 4096 elements. The i8 case did about 1 µs of real work and was reported at 67 to 91 µs
run_end_compress.rs: decompress[u64, (4000, 1024)]	27 (47)	Drop this grid point. (10000, 1024) and (10000, 4096) keep long runs covered
varbinview_compact.rs: compact*[(_, 90)]	0 (10, all vortex-array PRs)	Drop the 90% utilization cases: there is nothing to compact, so they did 14 ns of work
binary_ops.rs: mul_u32_nonnull, mul_i32_nonnull, *_shapes[(128, _)]	31 (57), 10 (19), 3 (9), 3 (8), 2 (6)	256 KiB per input instead of 96 KiB, and no 128-row shape case (2 µs per iteration, swings up to 95%)
collect_bool.rs: words_gather_dispatch[1024]	7 (9), 3 (5)	Gather sizes 64K and 1M instead of 1K and 64K. The 1K case ran in tens of nanoseconds
lane_kernels.rs: lanezip_checked_add_u32	1 (2)	65536 lanes on the walltime legs instead of 16384. The casts keep 16384 in simulation
take_primitive.rs: dict_canonicalize_gt_u8[16000000]	4 (15)	Drop the 16M case. It ran for 7 to 11 ms, over the 1 ms budget, and swung by up to 60%
zone_map_prune.rs: or_chain	1 (1)	4-term chain instead of 16. The 16-term chain ran for 1.4 ms at 1024 zones, over the budget, and longer at 8192 zones

The benchmarking guide gets a section on the floor: how small is too small, and what to do about it.

The CodSpeed report on this PR will show a large change for every resized benchmark, and the dropped cases will show as skipped until they are archived. Whether the flips stop can only be confirmed over the coming pull requests.

Local per-iteration medians after the change (4 vCPU VM, plain cargo bench)

Benchmark	Before	After
allocation: allocate_drop_*	3 to 70 ns	3 to 10 µs
allocation: from_vec_drop_*	30 ns to 1.4 µs	4 to 23 µs
filter_fixed_width: random_i8[0.5]	1.0 µs	7.0 µs
filter_fixed_width: random_i256[0.8]	(4096 elements)	4.1 µs
collect_bool: words_gather_dispatch	23 ns (1K), 1.7 µs (64K)	1.7 µs (64K), 26 µs (1M)
collect_bool: words_gather_scalar	0.4 µs (1K), 25 µs (64K)	24 µs (64K), 420 µs (1M)
lane_kernels: lanezip_checked_add_u32	12.9 µs	56 µs
binary_ops: mul_u32_nonnull	(96 KiB inputs)	22.6 µs
binary_ops: add_shapes[(32768, PerRowPerRow)]		13.5 µs
take_primitive: dict_canonicalize_gt_u8[1000000]	654 µs	unchanged
zone_map_prune: or_chain (16 / 1024 / 8192 zones)	133 / 171 / 360 µs	30 / 33 / 79 µs
run_end_compress: decompress[u64, _]	0.6 to 8 µs	unchanged (one point dropped)
varbinview_compact: compact*[(_, 10)]	10 to 33 µs	unchanged (90% cases dropped)

CodSpeed's simulation reports roughly ten times these numbers for medium-sized cases, so everything stays well under 1 ms.

Checks run:

• cargo clippy --locked --benches -p vortex-buffer -p vortex-compute -p vortex-runend -p vortex-layout -p vortex-array --features vortex-array/_test-harness -- -D warnings
• cargo +nightly-2026-09-10 fmt on the touched crates
• Every resized benchmark was run locally with --bench (table above)

Not run: the CodSpeed jobs themselves; they run on this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01B6ApyxEXpjiasZbQptsypn

---

Generated by Claude Code

[codspeed]

Merging this PR will degrade performance by 81.22%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 10 improved benchmarks
❌ 130 regressed benchmarks
✅ 2047 untouched benchmarks
🆕 30 new benchmarks
⏩ 284 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	from_vec_drop_arrow[256]	2.8 µs	133.9 µs	-97.9%
❌	Simulation	from_vec_drop_arrow[1024]	2.9 µs	136.1 µs	-97.89%
❌	Simulation	from_vec_drop_vortex[256]	3.1 µs	142.3 µs	-97.84%
❌	Simulation	from_vec_drop_vortex[1024]	3.1 µs	144.8 µs	-97.84%
❌	Simulation	from_vec_drop_bytes[1024]	2.2 µs	99.4 µs	-97.8%
❌	Simulation	from_vec_drop_bytes[256]	2.2 µs	96.5 µs	-97.74%
❌	Simulation	from_vec_drop_vortex[64]	3 µs	114.7 µs	-97.37%
❌	Simulation	from_vec_drop_arrow[64]	2.8 µs	105.3 µs	-97.33%
❌	Simulation	allocate_freeze_drop_vortex_minimal_alignment[64]	3.3 µs	109.1 µs	-96.95%
❌	Simulation	allocate_freeze_drop_vortex_custom[64]	4.2 µs	135.6 µs	-96.91%
❌	Simulation	allocate_freeze_drop_vortex[64]	3.4 µs	109.3 µs	-96.86%
❌	Simulation	from_vec_drop_bytes[64]	2.2 µs	68.6 µs	-96.82%
❌	Simulation	allocate_freeze_drop_vortex[256]	3.5 µs	109.4 µs	-96.81%
❌	Simulation	allocate_freeze_drop_bytes[64]	3.2 µs	97 µs	-96.74%
❌	Simulation	allocate_freeze_drop_bytes[256]	3.2 µs	97.1 µs	-96.68%
❌	Simulation	allocate_freeze_drop_bytes[1024]	3.3 µs	97.1 µs	-96.63%
❌	Simulation	allocate_freeze_drop_arrow[64]	3.8 µs	104.9 µs	-96.38%
❌	Simulation	allocate_freeze_drop_vortex_minimal_alignment[65536]	6 µs	148.5 µs	-95.99%
❌	Simulation	allocate_freeze_drop_vortex_minimal_alignment[16384]	6 µs	148.3 µs	-95.99%
❌	Simulation	allocate_freeze_drop_vortex[1024]	6 µs	148.6 µs	-95.96%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/beautiful-maxwell-7eyyvp (8442a96) with develop (c6678f3)}

Footnotes

1. 284 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩