Background
The bench harness (benches/lua_bench.lua) currently ships with only two fixtures:
benches/fixtures/small_api.json (~2 KB)
benches/fixtures/medium_resp.json (~100 KB)
The README roadmap (spec §9.3) calls out two additional fixtures that would expose scaling behavior not visible on the current ladder:
| Fixture |
Target size |
Intent |
large_dump.json |
~20 MB |
Bandwidth-bound behavior; separates L3-cached from memory-bandwidth-limited throughput |
deep_nest.json |
depth stress |
Worst-case for Phase 2 cursor walk and validate_brackets; exposes skip-cache misses |
Acceptance criteria
- Both files committed under
benches/fixtures/.
make bench includes them in the size ladder without special flags.
large_dump.json should be a realistic multimodal or API-response payload (not random bytes), so the bench reflects actual usage patterns.
deep_nest.json should exercise at least 200 levels of nesting.
Notes
- Synthetic generation scripts (Python / shell) are fine; commit the generated JSON, not the generator, so CI is deterministic.
deep_nest.json will also stress validate_brackets fusion (see related issue on AVX2/NEON fusion) and the memchr2 jump (very long string interiors). Having the fixture available is a prerequisite for measuring both.
- Check the
git lfs policy before committing a 20 MB file; if LFS is not configured, generate it at make bench time from a smaller seed instead.
Background
The bench harness (
benches/lua_bench.lua) currently ships with only two fixtures:benches/fixtures/small_api.json(~2 KB)benches/fixtures/medium_resp.json(~100 KB)The README roadmap (spec §9.3) calls out two additional fixtures that would expose scaling behavior not visible on the current ladder:
large_dump.jsondeep_nest.jsonvalidate_brackets; exposes skip-cache missesAcceptance criteria
benches/fixtures/.make benchincludes them in the size ladder without special flags.large_dump.jsonshould be a realistic multimodal or API-response payload (not random bytes), so the bench reflects actual usage patterns.deep_nest.jsonshould exercise at least 200 levels of nesting.Notes
deep_nest.jsonwill also stressvalidate_bracketsfusion (see related issue on AVX2/NEON fusion) and thememchr2jump (very long string interiors). Having the fixture available is a prerequisite for measuring both.git lfspolicy before committing a 20 MB file; if LFS is not configured, generate it atmake benchtime from a smaller seed instead.