Upgrade to Storm 1.14.1 and publish the 2026-09-13 run - #9
Merged
Merged
Conversation
Storm 1.14.1 states a scroll request as an ordering, a size and a position: the of(key, lastValue, size) factory is gone, so the keyset workload and the printSql dump build their request with Scrollable.of(Pet_.id, PAGE_SIZE).after(cursor). The workload is unchanged, one page of 20 rows seeking past a cursor. The statement it runs is not quite unchanged. Comparing the printSql output across the two versions, every workload emits the same SQL except the keyset query, whose select list now carries the key column a second time: 1.14.1 reads a window's position out of the row rather than from the request, so the sort and key columns join the selection. One extra column over a 20-row page, on the one workload that scrolls.
The full field on one dedicated instance, Storm built from v1.14.1 at 15f63804. Committed under results/2026-09-13 with its metadata, and mirrored into results/summary.md as the current run. The instance is not the one the 2026-09-03 table came from: the JDBC baseline measures 101.9 µs/op against 140.7 there, -28%, on a different CPU model, and the drift guard flagged it. The two tables are not cell-comparable, so each score reads against the JDBC baseline inside its own table, as the standing rule says. Within this run Storm is the fastest library on nine of the twelve workloads and second on the other three, behind Hibernate on the dynamic query by 1.3%, Jimmer on the ten-row join by 3.2%, and Ktorm on the graph insert by 1.3%. jOOQ keeps the object graph with its MULTISET aggregate, 36% ahead. The keyset query carries one extra column since 1.14.1 reads a window's position out of the row, which is the only SQL that changed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Storm 1.14.1 states a scroll request as an ordering, a size and a position, so the
Scrollable.of(key, lastValue, size)factory the keyset workload used is gone. The workload is unchanged — one page of 20 rows seeking past a cursor — and now builds its request the way 1.14.1 does:What the statements do
Comparing
printSqlacross the two versions, every workload emits the same SQL except the keyset query, whose select list now carries the key column a second time: 1.14.1 reads a window's position out of the row rather than from the request, so the sort and key columns join the selection. One extra column over a 20-row page, on the one workload that scrolls.The run
Full field on one dedicated instance (4 vCPU / 16 GB, Ubuntu 24.04.5, AMD EPYC 9V74), Storm built from
v1.14.1at15f63804: workflow run. Committed underresults/2026-09-13/with its metadata and mirrored intoresults/summary.md.The instance is not the one the 2026-09-03 table came from. The JDBC baseline measures 101.9 µs/op against 140.7 µs/op there, −28% on a different CPU model, and the drift guard flagged it:
So the two tables are not cell-comparable, and each score reads against the JDBC baseline inside its own table.
Where Storm lands within this run
Fastest library on nine of the twelve workloads: the primary-key lookup, the projection, keyset pagination, the hundred- and thousand-row joins, the batch insert, the single-row update, the create-then-amend transaction and the object graph's runner-up place are led or shared by Storm. Three go elsewhere, and they are worth separating by how solid the margin is within this run, comparing each gap against the two cells' combined JMH error:
graphInsertdynamicjoinWithMapping10jOOQ keeps the object graph outright, 845.5 against 1152.7, a 36% gap far beyond any error band, with its MULTISET JSON aggregate against repeated join rows.
Those orderings are within-run. The reproducibility repeat committed under
results/2026-09-03-repeat/measured individual scores moving about 1% between executions on equivalent hardware, worst case 6.5%, which is enough to reorder a group this close, so none of the three should be read as a stable ranking across runs.Data quality
Storm's twelve cells are the cleanest in the run: JMH error at most 2.2%, fork spread at most 7.6%. Nine of the 97 cells are noisier than the rest and should not be quoted as exact figures —
updateByIdon Exposed DAO (56% fork spread, ±14.4%),multiStatementon Hibernate (43%, ±14.3%),joinWithMapping100on jOOQ (17%, ±7.7%),batchInserton Ktorm (17%),objectGraphandupdateByIdon Hibernate (17% and 13%), and the JDBC baseline row itself (15%). The rankings around them are unaffected; the gaps there are far wider than the noise.Ratios are comparable across runs even when absolutes are not: Storm's score against the JDBC baseline moved at most 4.3% on any workload against 2026-09-03, most under 2%, so this run agrees with the previous one. The largest move is
keysetat +3.9%, which is also the one workload whose SQL changed — consistent with the extra key column, though within the repeat study's spread rather than proof of it.Nothing here is a before/after against 1.14.0: different hardware. The
baseline_refA/B mode on one instance is the instrument for that, and it is worth running for the per-statement stream guard and the read-only write check that 1.14.1 adds to the hot path.