Repository navigation
database: atomic batch writes, bounded temp tables, dead hiscore repos removed - #288
Merged
Merged
Conversation
added 5 commits
November 13, 2026 22:46
- prediction: remove repo-level commits; caller owns the tx. MUST land together with the worker_prediction begin() wrap (claimed by another branch) or retried batches roll back silently - report/player/hiscore: temp tables on InnoDB (MEMORY had a 16MB ceiling -> batch-full -> retry storm); report DROP TABLE -> DROP TEMPORARY (non-temp DDL implicitly commits the caller's tx) - hiscore: delete 4 dead single-row repos + 3 unused interfaces; they insert into STORED generated columns and are masked by INSERT IGNORE - api_public: get_report_score uses expanding bindparam (bare tuple bind only worked via asyncmy accident; empty tuple = invalid IN ()) - discord: set_primary_rsn bails before the clear when the target row is missing (was committing a wipe of all primary flags) - api: get_user limit(1); username has no unique constraint, a dup raised MultipleResultsFound -> auth 500 - core: POOL_RECYCLE 25s -> 3600s (constant reconnect churn) - player: select_player player_id=0 truthiness guard
- smoke: 7 tests against the compose mysql (skip unless SMOKE_DATABASE_URL is set): prediction batch atomicity, 20k-row player temp table (beyond the MEMORY ceiling), hiscore upserts across all 4 partitions, sighting dedupe, expanding IN list incl. empty, primary-rsn miss safety, engine connectivity. Idempotent across runs (autouse purge, disjoint id ranges) - smoke caught a real bug in the previous commit: after reordering set-first, the clear still reset the just-set row - the clear now excludes Player_id = :player_id
- the pytest file skipped in normal runs but became a footgun if SMOKE_DATABASE_URL was ever exported: every plain pytest run would write to and purge rows in that database - now a standalone script, same 7 checks, same idempotent purge: uv run python _infra/smoke/db_smoke.py [--url DSN] - defaults to the compose mysql DSN; exits non-zero on any failure
the database branch removed the prediction repos' self-commits; without this caller-owned begin() the runner's session close would roll back every batch. also fixes both test mocks to model begin() as an async CM. landed together with the database changes so PR #288 is self-contained.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Tiger-style review of
components/bot_detector/databasefound hazards in the write paths that the gates can't catch:predictiontable has no unique key). Worse, the report repo usedDROP TABLEinstead ofDROP TEMPORARY, whose implicit commit silently broke the rollback contract the worker relies on.ENGINE=MEMORYwith aVARCHAR(255)column allocates ~1 KB/row fixed-width; ~16k rows blows the 16 MB defaultmax_heap_table_size→ "table is full" → whole batch requeued → retry storm on the hottest write path (worker_hiscore).STOREDgenerated columns — guaranteed runtime error masked byINSERT IGNOREinto silent data loss for the first future caller.get_report_scorebound anINlist as a bare tuple (worked only via an asyncmy escaping accident; empty list = invalidIN ()→ 500),set_primary_rsncould permanently wipe all of a user's primary-RSN flags on a miss,get_usercould 500 auth on a duplicate username (column has no unique constraint), andPOOL_RECYCLE=25s recycled every connection every 25 seconds across every service.Solution
worker_predictionsession.begin()wrap (owned by a separate branch) — the commit message says so too.ENGINE=InnoDB(player, report, hiscore); report's drops now useDROP TEMPORARY.get_report_scoreusesbindparam(expanding=True);set_primary_rsnbails before the clear when the target row is missing and the clear now excludes the target row (the smoke tier caught that the reordered sequence was resetting the just-set primary);get_usertakesLIMIT 1;POOL_RECYCLE25s → 3600s;select_playerguardsplayer_id=0truthiness.Smoke tier
_infra/smoke/db_smoke.py— standalone script (not part of pytest, so normal test runs carry zero smoke noise and no DB-writing footgun), 7 checks that prove the raw-SQL surfaces against real MySQL: prediction batch atomicity under FK failure, 20k-row player update through the temp-table path (proves theMEMORY→InnoDBchange past the 16 MB ceiling), hiscore upserts across all 4 partitions, sighting dedupe, expandingINlist including empty tuple, primary-RSN miss safety, engine connectivity. Idempotent (purges its own rows before and after).Run against the compose
mysqlservice (port 3307):Exits non-zero on any failure. This tier caught the
set_primary_rsnregression described above.Verification
expandingbehavior verified empirically against SQLAlchemy 2.0.44 + asyncmy; generated-column/DDL semantics checked against the actual01_tables.sqlCreated with GLM-5.3-Flash in T3 Code.