Migrate source consumers to SourceRuntime (GTID failover 2/3)#452
Open
driv3r wants to merge 1 commit into
Open
Migrate source consumers to SourceRuntime (GTID failover 2/3)#452driv3r wants to merge 1 commit into
driv3r wants to merge 1 commit into
Conversation
Have the source-side components resolve the current source connection from the Ferry's SourceRuntime instead of holding a captured *sql.DB, so a future source master failover can repoint them all by swapping the runtime rather than editing each consumer. Consumers migrated (each keeps its static DB as a fallback when no runtime is configured, so behavior is unchanged today): - CursorConfig: NewCursor binds a cursor to the runtime's current handle at creation time. A cursor keeps that handle for its whole scan (a scan must be consistent against one server); only cursors created after a swap observe the new source. - DataIterator: resolves the source once at the start of Run. - InlineVerifier: currentSourceDB resolves the runtime handle and, on a detected swap, resets the source statement cache (cached statements are bound to the previous DB); the DB and its matching cache are returned together under a lock so concurrent workers stay consistent. - IterativeVerifier: currentSourceDB resolves the runtime handle for fingerprint/compressed-hash reads; its CursorConfig also carries the runtime for pagination scans. - ChecksumTableVerifier: currentSourceDB resolves the runtime handle for CHECKSUM TABLE reads. Ferry wires f.sourceRuntime into each constructor. No swap happens yet (that arrives with failover), so the runtime always resolves to the same initial handle and behavior is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Migrates Ghostferry's source-side consumers to resolve their connection from
SourceRuntime(introduced in the parent PR) instead of holding a captured*sql.DB.PR 2 of 3 of the GTID master failover rework:
SourceRuntime(Introduce SourceRuntime to own the source connection (GTID failover 1/3) #451).Why
With consumers holding their own
*sql.DB, a source master failover would have to hand-edit a field on each one — brittle and easy to miss. Having each consumer ask the runtime for the current source inverts that: a failover becomes a singleSourceRuntime.Replace, and every consumer that starts new work afterwards picks up the promoted writer automatically.How
Each consumer keeps its static
DBas a fallback when no runtime is configured, so today (no swap ever happens) behavior is unchanged.CursorConfig—NewCursorbinds a cursor to the runtime's current handle at creation. A cursor keeps that handle for its whole scan (a scan must be consistent against one server); only cursors created after a swap observe the new source.DataIterator— resolves the source once at the start ofRun.InlineVerifier—currentSourceDBresolves the runtime handle and, on a detected swap, resets the source statement cache (cached prepared statements are bound to the previous DB). The DB and its matching cache are returned together under a lock so concurrent verification workers stay consistent.IterativeVerifier—currentSourceDBresolves the runtime handle for fingerprint / compressed-hash reads; itsCursorConfigalso carries the runtime for pagination scans.ChecksumTableVerifier—currentSourceDBresolves the runtime handle forCHECKSUM TABLEreads.FerrywiressourceRuntimeinto each constructor.Testing
-raceclean.test/gointegration suite passes in bothfile_positionandgtidmodes (confirming behavior neutrality — the runtime resolves to the same initial handle when no failover occurs).go build+gofmtclean.