Skip to content

Fix multiOfaClient secondary send goroutine racing test teardown - #549

Merged
mostlyconsistent merged 2 commits into
developfrom
mkysel/fix-multiofa-flake
Sep 11, 2026
Merged

Fix multiOfaClient secondary send goroutine racing test teardown#549
mostlyconsistent merged 2 commits into
developfrom
mkysel/fix-multiofa-flake

Conversation

@mostlyconsistent

Copy link
Copy Markdown
Contributor

multiOfaClient.SendTransaction fans out to secondaries in fire-and-forget goroutines that outlive the call by up to secondarySendTimeout. Nothing could observe them, so on the error path they logged into a *testing.T that had already completed:

    panic: Log in goroutine after TestMultiOfaClient_SecondarySendRespectsTimeout
    has completed: ERROR Secondary backend send failed

CI surfaced this as a data race rather than the panic, because the race detector saw the cross-test-boundary access first. It has failed at least twice on unrelated branches (runs 32469028488, 34243124630). It is not a true flake: it reproduces on the second iteration of

    go test ./pkg/txm/clientwrappers/dualbroadcast/ -race \
      -run 'TestMultiOfaClient_SecondarySendRespectsTimeout$' -count=20

and is only hidden in CI because CI runs -count=1, where the panic needs a previous iteration's t to still be referenced.

multiOfaClient.SendTransaction fans out to secondaries in fire-and-forget
goroutines that outlive the call by up to secondarySendTimeout. Nothing could
observe them, so on the error path they logged into a *testing.T that had
already completed:

    panic: Log in goroutine after TestMultiOfaClient_SecondarySendRespectsTimeout
    has completed: ERROR Secondary backend send failed

CI surfaced this as a data race rather than the panic, because the race
detector saw the cross-test-boundary access first. It has failed at least
twice on unrelated branches (runs 32469028488, 34243124630). It is not a true
flake: it reproduces on the second iteration of

    go test ./pkg/txm/clientwrappers/dualbroadcast/ -race \
      -run 'TestMultiOfaClient_SecondarySendRespectsTimeout$' -count=20

and is only hidden in CI because CI runs -count=1, where the panic needs a
previous iteration's t to still be referenced.
@mostlyconsistent
mostlyconsistent marked this pull request as ready for review September 9, 2026 14:08
@mostlyconsistent
mostlyconsistent requested review from a team and dimriou as code owners September 9, 2026 14:08
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

📊 API Diff Results

No changes detected for module github.com/smartcontractkit/chainlink-evm

View full report

Comment thread pkg/txm/clientwrappers/dualbroadcast/multiofa_client.go Outdated
Comment thread pkg/txm/clientwrappers/dualbroadcast/multiofa_client.go Outdated
@mostlyconsistent
mostlyconsistent merged commit 9289784 into develop Sep 11, 2026
35 checks passed
@mostlyconsistent
mostlyconsistent deleted the mkysel/fix-multiofa-flake branch September 11, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants