Flaky test: TestPartitionCompactor_DeleteLocalSyncFiles (arm64) — data race in thanos extprom.TxGaugeVec via concurrent MetaFetcher.Fetch

[yeya24]

Describe the bug

TestPartitionCompactor_DeleteLocalSyncFiles failed on the test (arm64) CI job with a data race detected in vendored Thanos code. Observed on PR #7706 (a pkg/ring/kv/dynamodb-only change, unrelated to the compactor). test (amd64) passed on the same commit.

Failed job: https://github.com/cortexproject/cortex/actions/runs/29725428167/job/88297304228

Race details

Read side (goroutine 12424):

github.com/thanos-io/thanos/pkg/extprom.(*TxGaugeVec).WithLabelValues()
    vendor/github.com/thanos-io/thanos/pkg/extprom/tx_gauge.go:102
github.com/thanos-io/thanos/pkg/block.(*BaseFetcher).fetch()
    vendor/github.com/thanos-io/thanos/pkg/block/fetcher.go:674
github.com/thanos-io/thanos/pkg/block.(*MetaFetcher).Fetch()
    vendor/github.com/thanos-io/thanos/pkg/block/fetcher.go:725
github.com/thanos-io/thanos/pkg/compact.(*Syncer).SyncMetas.func2()
github.com/cortexproject/cortex/pkg/compactor.(*Compactor).compactUser()
    pkg/compactor/compactor.go:1111
...
pkg/compactor/compactor_paritioning_test.go:1633

Write side (goroutine 12500):

github.com/thanos-io/thanos/pkg/extprom.(*TxGaugeVec).ResetTx()
    vendor/github.com/thanos-io/thanos/pkg/extprom/tx_gauge.go:49
github.com/thanos-io/thanos/pkg/block.(*FetcherMetrics).ResetTx()
    vendor/github.com/thanos-io/thanos/pkg/block/fetcher.go:71
github.com/thanos-io/thanos/pkg/block.(*MetaFetcher).Fetch()
    vendor/github.com/thanos-io/thanos/pkg/block/fetcher.go:723
github.com/thanos-io/thanos/pkg/compact.(*Syncer).SyncMetas.func2()
...

Two concurrent MetaFetcher.Fetch() calls race on the shared TxGaugeVec: one resets the tx gauge (ResetTx) while the other is still reading/incrementing it (WithLabelValues). Syncer.SyncMetas uses singleflight per group, but concurrent per-user compactions in the test share fetcher metrics across different singleflight groups.

Context

This test family has a flaky history on arm64: #7565 (fixed by #7567), #7608 (fixed by #7619). Those fixes addressed timing/determinism of TestCompactor_DeleteLocalSyncFiles; this failure is the partitioning variant and a different failure mode (race detector rather than poll timeout).

To Reproduce

Flaky — observed on arm64 with -race. go test -race -count=N ./pkg/compactor/ -run TestPartitionCompactor_DeleteLocalSyncFiles on arm64 may reproduce.

Expected behavior

No data race; test passes reliably on all architectures.

Additional context

The race is in vendored Thanos code (extprom/tx_gauge.go), so the fix likely belongs upstream (guard ResetTx/WithLabelValues with the existing mutex consistently) or in how the cortex compactor/test shares FetcherMetrics across concurrent user compactions.