chore(ci): Run macOS workflows for on both Cirrus and Bitrise runners#6274
Conversation
Add runner_provider matrix dimension to all macOS CI jobs so they can run on both Cirrus and Bitrise. Bitrise jobs use continue-on-error so they won't block CI. Jobs won't actually run on Bitrise yet until the pool is provisioned — this prepares the ground to enable it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Semver Impact of This PR⚪ None (no version bump detected) 📋 Changelog PreviewThis is how your changes will appear in the changelog. This PR will not appear in the changelog. 🤖 This preview updates automatically when you update the PR. |
iOS (legacy) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| d038a14+dirty | 3845.71 ms | 1228.11 ms | -2617.59 ms |
| 41d6254+dirty | 3845.71 ms | 1224.51 ms | -2621.20 ms |
| 4966363+dirty | 3854.04 ms | 1231.55 ms | -2622.50 ms |
| c004dae+dirty | 3850.32 ms | 1227.79 ms | -2622.53 ms |
| eb93136+dirty | 3843.09 ms | 1220.11 ms | -2622.98 ms |
| 71abba0+dirty | 3821.93 ms | 1202.81 ms | -2619.12 ms |
| ad66da3+dirty | 3820.96 ms | 1214.43 ms | -2606.52 ms |
| ca9d079+dirty | 3835.63 ms | 1218.68 ms | -2616.95 ms |
| df5d108+dirty | 1225.90 ms | 1220.14 ms | -5.76 ms |
| 4b87b12+dirty | 1212.90 ms | 1222.09 ms | 9.19 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| d038a14+dirty | 5.15 MiB | 6.67 MiB | 1.51 MiB |
| 41d6254+dirty | 5.15 MiB | 6.70 MiB | 1.54 MiB |
| 4966363+dirty | 5.15 MiB | 6.68 MiB | 1.53 MiB |
| c004dae+dirty | 5.15 MiB | 6.67 MiB | 1.51 MiB |
| eb93136+dirty | 5.15 MiB | 6.69 MiB | 1.53 MiB |
| 71abba0+dirty | 5.15 MiB | 6.67 MiB | 1.52 MiB |
| ad66da3+dirty | 5.15 MiB | 6.67 MiB | 1.51 MiB |
| ca9d079+dirty | 5.15 MiB | 6.69 MiB | 1.53 MiB |
| df5d108+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
| 4b87b12+dirty | 3.38 MiB | 4.77 MiB | 1.39 MiB |
Previous results on branch: itay/ci-bitrise-dual-run
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 1799f9b+dirty | 3826.81 ms | 1211.84 ms | -2614.97 ms |
| ee60eb9+dirty | 3848.40 ms | 1221.72 ms | -2626.68 ms |
| b0e5f0d+dirty | 3844.02 ms | 1220.09 ms | -2623.93 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 1799f9b+dirty | 4.98 MiB | 6.45 MiB | 1.48 MiB |
| ee60eb9+dirty | 5.15 MiB | 6.68 MiB | 1.52 MiB |
| b0e5f0d+dirty | 4.98 MiB | 6.45 MiB | 1.48 MiB |
📲 Install BuildsAndroid
|
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
There was a problem hiding this comment.
Unprovisioned bitrise matrix entries in build jobs may delay or block downstream cirrus test jobs that needs them
The PR adds runner_provider: bitrise matrix entries to build-ios (sample-application.yml:64) and react-native-build (e2e-v2.yml:259), targeting runner labels (bitrise_pool_name:tahoe/{macos_version}) for pools the PR states are not provisioned in this repo. Downstream test-ios (needs: [..., build-ios], sample-application.yml:333) and react-native-test (needs: [react-native-build, ...], e2e-v2.yml:426) wait for the ENTIRE upstream job — all matrix combinations — to reach a terminal state before any downstream matrix entry starts. continue-on-error: true on the bitrise entries only takes effect after a runner picks up the job and it fails; it does nothing while a job sits in the queued/waiting-for-runner state. If the bitrise entries queue waiting for a runner that never appears (rather than failing fast on an unmatched label), the cirrus test-ios/react-native-test jobs are delayed until those entries hit their runner-acquisition timeout, potentially stalling iOS/e2e CI on every PR. The exact behavior depends on how GitHub/Cirrus/Bitrise runner-group label matching handles labels with no registered runner (fail-fast vs. queue-and-timeout), which cannot be confirmed from the repo.
Evidence
test-iosdeclaresneeds: [diff_check, detect-changes, build-ios](sample-application.yml:333); GitHub Actions waits for every matrix combination ofbuild-ios(including the newbitriseentries from line 64) before starting anytest-iosentry.react-native-testdeclaresneeds: [react-native-build, diff_check, detect-changes](e2e-v2.yml:426); same all-matrix wait applies to thebitriseentries added at line 259.continue-on-error: ${{ matrix.runner_provider == 'bitrise' }}(e.g. sample-application.yml:48, e2e-v2.yml:240) only suppresses failure after a runner runs the job; a job still waiting for a runner is not terminal and cannot satisfyneeds.- The PR description explicitly states the Bitrise runner pools are not provisioned for this repo, so no runner matches
bitrise_pool_name:*, meaning those entries cannot run normally and rely on a queue/timeout to become terminal. - Whether an unmatched runner label fails fast or queues until a timeout is infra-dependent and not determinable from the repo, so the magnitude of the delay (and whether downstream is meaningfully blocked) is uncertain.
New actions/cache@v4 steps use floating version tag instead of pinned commit SHA (.github/workflows/native-tests.yml:59)
The newly added Cache Ruby steps reference actions/cache@v4 without a commit SHA, while every other action reference in these workflows (e.g. actions/cache@27d5ce7f…, actions/checkout@df4cb1c…) uses a pinned SHA. A tag mutation or compromise of the v4 ref could execute arbitrary code on the runner with access to the environment, including secrets like SENTRY_AUTH_TOKEN and signing credentials (MATCH_PASSWORD, MATCH_GIT_PRIVATE_KEY).
Evidence
testflight.yml:28(a changed file) adds- uses: actions/cache@v4for the newCache Rubystep, while the adjacentactions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6andruby/setup-ruby@afeafc3d1ab54a631816aba4c914a0081c12ff2f # v1are pinned to SHAs.- The same unpinned
actions/cache@v4pattern is added across changed workflows:sample-application.yml:78,263,size-analysis.yml:116,e2e-v2.yml:121,379,sample-application-expo.yml:69— contrasting with the pre-existing pinnede2e-v2.yml:153(actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5). - These workflows handle signing/upload secrets (
testflight.ymlrunsbundle installand TestFlight upload), so a compromised floating tag runs in the same context as those secrets.
Both runner providers upload the same TestFlight build number, causing a duplicate-build conflict (.github/workflows/sample-application.yml:69)
In testflight.yml the upload_to_testflight job runs a runner_provider: ['cirrus', 'bitrise'] matrix. Both matrix legs execute yarn set-build-number ${{ github.run_number }} (identical across all matrix combinations in one workflow run) and then bundle exec fastlane ios upload_react_native_sample_to_testflight. Once Bitrise pools are provisioned, both legs will attempt to upload a build with the same build number; Apple App Store Connect rejects a build number that has already been processed, so the second upload will fail. Because continue-on-error is true only for bitrise, whichever provider uploads second determines the failure: if bitrise wins the race, the cirrus leg fails and breaks the workflow. There is no if: guard designating a single authoritative uploader.
Evidence
- testflight.yml line 24: matrix
runner_provider: ['cirrus', 'bitrise']produces two parallel jobs from one workflow run. - Line 69:
yarn set-build-number ${{ github.run_number }}usesgithub.run_number, which is identical across all matrix legs of a single run, so both jobs set the same build number. - Line 94:
bundle exec fastlane ios upload_react_native_sample_to_testflightruns unconditionally in both legs with noif: matrix.runner_provider == 'cirrus'guard to pick one uploader. continue-on-error: ${{ matrix.runner_provider == 'bitrise' }}masks only the bitrise failure; if bitrise uploads first, the cirrus leg's duplicate upload fails and surfaces as a workflow failure.- Currently latent: the PR notes Bitrise pools are not yet provisioned, so the conflict only manifests once both providers are active.
Both cirrus and bitrise TestFlight jobs upload with the same build number, risking duplicate-build conflicts and cirrus flakiness
The upload_to_testflight job now runs across a runner_provider: ['cirrus', 'bitrise'] matrix. Both matrix entries run yarn set-build-number ${{ github.run_number }} (identical value per run) and then unconditionally call bundle exec fastlane ios upload_react_native_sample_to_testflight. Once the Bitrise pools are provisioned, both jobs will attempt to upload a build with the same build number for the same app version, which Apple rejects as a duplicate. Because the two jobs run in parallel, whichever loses the race fails its upload. The bitrise failure is tolerated via continue-on-error: ${{ matrix.runner_provider == 'bitrise' }}, but the cirrus job has no such guard, so if cirrus loses the race the whole workflow fails. The actual upload (and arguably the build-number bump) should be restricted to a single provider via an if: matrix.runner_provider == 'cirrus' guard.
Evidence
testflight.ymladdsrunner_provider: ['cirrus', 'bitrise']to theupload_to_testflightmatrix with no per-providerif:guard on the build/upload steps.Set Build Numberrunsyarn set-build-number ${{ github.run_number }}(maps toreact-native-version --set-buildinsamples/react-native/package.json);github.run_numberis identical for both matrix entries in the same run.Run Fastlanerunsbundle exec fastlane ios upload_react_native_sample_to_testflightunconditionally for both providers.continue-on-erroris set only forbitrise, so a cirrus upload that loses the duplicate-build race would fail the workflow.- Impact is deferred: the PR states Bitrise pools are not yet provisioned, so today only cirrus runs and the conflict is dormant.
Identified by Warden find-bugs
Android (legacy) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| b9bebee+dirty | 438.86 ms | 452.21 ms | 13.35 ms |
| 3ce5254+dirty | 410.57 ms | 448.48 ms | 37.91 ms |
| ecf47a2+dirty | 420.40 ms | 458.02 ms | 37.62 ms |
| 3d377b5+dirty | 406.18 ms | 453.52 ms | 47.34 ms |
| a50b33d+dirty | 500.81 ms | 532.11 ms | 31.30 ms |
| 0d9949d+dirty | 403.57 ms | 437.00 ms | 33.43 ms |
| 1a2e7e0+dirty | 416.61 ms | 445.46 ms | 28.85 ms |
| 5125c43+dirty | 497.18 ms | 543.78 ms | 46.60 ms |
| 0b1b5e3+dirty | 416.42 ms | 470.58 ms | 54.16 ms |
| 1122a96+dirty | 422.22 ms | 464.33 ms | 42.10 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| b9bebee+dirty | 48.30 MiB | 53.58 MiB | 5.28 MiB |
| 3ce5254+dirty | 43.75 MiB | 48.12 MiB | 4.37 MiB |
| ecf47a2+dirty | 49.74 MiB | 54.82 MiB | 5.07 MiB |
| 3d377b5+dirty | 43.75 MiB | 48.14 MiB | 4.39 MiB |
| a50b33d+dirty | 43.75 MiB | 48.08 MiB | 4.33 MiB |
| 0d9949d+dirty | 43.75 MiB | 48.13 MiB | 4.37 MiB |
| 1a2e7e0+dirty | 49.74 MiB | 54.82 MiB | 5.07 MiB |
| 5125c43+dirty | 48.30 MiB | 53.54 MiB | 5.24 MiB |
| 0b1b5e3+dirty | 48.30 MiB | 53.60 MiB | 5.29 MiB |
| 1122a96+dirty | 48.30 MiB | 53.54 MiB | 5.24 MiB |
Previous results on branch: itay/ci-bitrise-dual-run
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| ee60eb9+dirty | 453.86 ms | 539.51 ms | 85.65 ms |
| b0e5f0d+dirty | 496.82 ms | 571.64 ms | 74.82 ms |
| 1799f9b+dirty | 424.78 ms | 467.62 ms | 42.84 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| ee60eb9+dirty | 48.30 MiB | 53.57 MiB | 5.26 MiB |
| b0e5f0d+dirty | 49.74 MiB | 54.82 MiB | 5.07 MiB |
| 1799f9b+dirty | 49.74 MiB | 54.82 MiB | 5.07 MiB |
iOS (new) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| d038a14+dirty | 3831.11 ms | 1216.30 ms | -2614.81 ms |
| 41d6254+dirty | 3849.78 ms | 1233.91 ms | -2615.86 ms |
| 4966363+dirty | 3863.07 ms | 1227.19 ms | -2635.88 ms |
| c004dae+dirty | 3857.82 ms | 1224.87 ms | -2632.95 ms |
| eb93136+dirty | 3846.51 ms | 1226.13 ms | -2620.39 ms |
| 71abba0+dirty | 3852.70 ms | 1224.53 ms | -2628.16 ms |
| ad66da3+dirty | 3855.02 ms | 1213.43 ms | -2641.59 ms |
| ca9d079+dirty | 3818.62 ms | 1216.72 ms | -2601.90 ms |
| df5d108+dirty | 1207.34 ms | 1210.50 ms | 3.16 ms |
| 4b87b12+dirty | 1199.49 ms | 1199.78 ms | 0.29 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| d038a14+dirty | 5.15 MiB | 6.67 MiB | 1.51 MiB |
| 41d6254+dirty | 5.15 MiB | 6.70 MiB | 1.54 MiB |
| 4966363+dirty | 5.15 MiB | 6.68 MiB | 1.53 MiB |
| c004dae+dirty | 5.15 MiB | 6.67 MiB | 1.51 MiB |
| eb93136+dirty | 5.15 MiB | 6.69 MiB | 1.53 MiB |
| 71abba0+dirty | 5.15 MiB | 6.67 MiB | 1.52 MiB |
| ad66da3+dirty | 5.15 MiB | 6.67 MiB | 1.51 MiB |
| ca9d079+dirty | 5.15 MiB | 6.69 MiB | 1.53 MiB |
| df5d108+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
| 4b87b12+dirty | 3.38 MiB | 4.77 MiB | 1.39 MiB |
Previous results on branch: itay/ci-bitrise-dual-run
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 1799f9b+dirty | 3835.45 ms | 1209.43 ms | -2626.02 ms |
| ee60eb9+dirty | 3845.13 ms | 1215.09 ms | -2630.05 ms |
| b0e5f0d+dirty | 3832.02 ms | 1214.46 ms | -2617.57 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 1799f9b+dirty | 4.98 MiB | 6.45 MiB | 1.48 MiB |
| ee60eb9+dirty | 5.15 MiB | 6.68 MiB | 1.52 MiB |
| b0e5f0d+dirty | 4.98 MiB | 6.45 MiB | 1.48 MiB |
Android (new) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| b9bebee+dirty | 500.50 ms | 536.42 ms | 35.92 ms |
| ecf47a2+dirty | 457.21 ms | 498.10 ms | 40.89 ms |
| 5fe1c6c+dirty | 365.84 ms | 408.62 ms | 42.78 ms |
| 4953e94+dirty | 398.80 ms | 431.81 ms | 33.01 ms |
| 1a2e7e0+dirty | 451.98 ms | 501.50 ms | 49.52 ms |
| 5125c43+dirty | 409.52 ms | 451.00 ms | 41.48 ms |
| 0b1b5e3+dirty | 425.58 ms | 476.02 ms | 50.44 ms |
| 1122a96+dirty | 510.16 ms | 542.00 ms | 31.84 ms |
| 7ff4d0f+dirty | 403.38 ms | 427.06 ms | 23.68 ms |
| 5a21b51+dirty | 505.16 ms | 539.20 ms | 34.04 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| b9bebee+dirty | 48.30 MiB | 53.58 MiB | 5.28 MiB |
| ecf47a2+dirty | 49.74 MiB | 54.82 MiB | 5.07 MiB |
| 5fe1c6c+dirty | 43.94 MiB | 49.00 MiB | 5.06 MiB |
| 4953e94+dirty | 43.94 MiB | 48.94 MiB | 5.00 MiB |
| 1a2e7e0+dirty | 49.74 MiB | 54.82 MiB | 5.07 MiB |
| 5125c43+dirty | 48.30 MiB | 53.54 MiB | 5.24 MiB |
| 0b1b5e3+dirty | 48.30 MiB | 53.60 MiB | 5.29 MiB |
| 1122a96+dirty | 48.30 MiB | 53.54 MiB | 5.24 MiB |
| 7ff4d0f+dirty | 48.30 MiB | 53.60 MiB | 5.30 MiB |
| 5a21b51+dirty | 48.30 MiB | 53.49 MiB | 5.19 MiB |
Previous results on branch: itay/ci-bitrise-dual-run
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| ee60eb9+dirty | 461.45 ms | 533.12 ms | 71.67 ms |
| b0e5f0d+dirty | 423.28 ms | 463.92 ms | 40.64 ms |
| 1799f9b+dirty | 460.00 ms | 526.64 ms | 66.64 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| ee60eb9+dirty | 48.30 MiB | 53.57 MiB | 5.26 MiB |
| b0e5f0d+dirty | 49.74 MiB | 54.82 MiB | 5.07 MiB |
| 1799f9b+dirty | 49.74 MiB | 54.82 MiB | 5.07 MiB |
…itay/ci-bitrise-dual-run
There was a problem hiding this comment.
Both Cirrus and Bitrise TestFlight jobs upload with the same build number
The upload_to_testflight job now runs twice (once per runner_provider), with both variants setting the build number to ${{ github.run_number }} and then calling upload_react_native_sample_to_testflight. Apple's App Store Connect API rejects uploads with duplicate build numbers, so whichever job uploads second will fail. Only the bitrise variant has continue-on-error: true; if Bitrise uploads first and the Cirrus upload is rejected, the workflow run fails non-deterministically. Note the PR states Bitrise jobs are not yet enabled, so this is currently a latent issue that becomes active once Bitrise runners are wired in. Fix by guarding the "Set Build Number"/"Run Fastlane" upload steps to a single provider (e.g. if: matrix.runner_provider != 'bitrise') or by deduplicating the upload.
Evidence
testflight.ymlmatrixrunner_provider: ["cirrus", "bitrise"]creates two instances ofupload_to_testflight.- Both instances run
yarn set-build-number ${{ github.run_number }};github.run_numberis constant across jobs in a run, so both stamp the same build number. - Both instances run
bundle exec fastlane ios upload_react_native_sample_to_testflight, which callstestflight(skip_waiting_for_build_processing: true)insamples/react-native/fastlane/Fastfilewith no provider guard. continue-on-error: trueis set only whenmatrix.runner_provider == 'bitrise', so a Cirrus upload rejected due to a duplicate build number fails the workflow.- PR body notes Bitrise jobs won't actually run yet, so the duplicate-upload collision is latent until Bitrise is enabled.
Identified by Warden find-bugs
- Added a step to remove asdf nodejs shims specifically for Bitrise runners in e2e-v2.yml. - Removed unnecessary bundle install command in sample-application-expo.yml. - Updated runner_provider matrix in testflight.yml to exclude Bitrise.
… Bitrise Removing asdf shims broke yarn resolution entirely. Instead, pin ASDF_NODEJS_VERSION to the actions/setup-node version so asdf shims pass through to the correct Node in all subdirectories. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Both cirrus and bitrise runners upload identical iOS size metrics to Sentry
When Bitrise becomes active, both matrix entries (cirrus and bitrise) in the ios job will independently run sentry-cli build upload against the same XCArchive with --build-configuration "Release", producing duplicate size-analysis data for every push to main. No guard differentiates the runner providers, and no runner suffix is added to the build configuration to distinguish the two uploads.
Evidence
- In
size-analysis.yml, theiosjob matrix isrunner_provider: ["cirrus", "bitrise"], so two jobs run per trigger once Bitrise runners exist. - The
Upload iOS XCArchive to Sentry Size Analysisstep is gated only byif: env.SENTRY_AUTH_TOKEN != '', with no condition onmatrix.runner_provider; both jobs upload the archive with the identical--build-configuration "Release"value. - By contrast,
testflight.yml'supload_to_testflightjob (which also uploads an artifact) deliberately sets its matrix torunner_provider: ["cirrus"]only, showing the maintainers intentionally avoid double-uploading from both providers — the size-analysis job omits that restriction. - The Android size job is a separate single (non-matrix) job and uploads once, so only the new iOS matrix introduces the duplicate upload.
Identified by Warden find-bugs
Added a step to set the global Ruby version to 3.3.0 using rbenv in the sample-application.yml workflow, ensuring consistency across the CI environment.
…ride ASDF_NODEJS_VERSION pointed to a Node version asdf didn't install, so it couldn't find yarn. Remove the asdf nodejs plugin entirely and re-run corepack enable so yarn resolves from actions/setup-node. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
rbenv install 3.3.0 missing -s flag causes cache-hit failures in build-macos Bitrise step
In sample-application.yml line 276 (build-macos job), the Bitrise Ruby install uses rbenv install 3.3.0 without the -s (skip-if-installed) flag — every other workflow file in this PR uses rbenv install -s. When the preceding actions/cache step restores ~/.rbenv/versions on a cache hit, the subsequent rbenv install 3.3.0 will abort with "already installed", failing the job. Change it to rbenv install -s (reads version from .ruby-version) to match the pattern used everywhere else.
Evidence
grep 'rbenv install' .github/workflows/*.ymlshows all eight occurrences userbenv install -sexceptsample-application.yml:276which usesrbenv install 3.3.0.- The
actions/cachestep immediately before (lines 263–270) restores~/.rbenv/versions, so on a cache hit Ruby 3.3.0 is already present when the install command runs. rbenv installwithout-sexits non-zero with "already installed" when the requested version exists, unlike-swhich silently skips.- The parallel
build-iosBitrise step in the same file (line 91) and all other workflows (e2e-v2.yml:411,native-tests.yml:71,size-analysis.yml:129,testflight.yml:40) correctly userbenv install -s.
Identified by Warden find-bugs
On Bitrise VMs, asdf's ruby shim resolves to Ruby 3.4.7 instead of the rbenv-installed 3.3.0. Add eval "$(rbenv init -)" for inline bundle install calls and prepend rbenv shims to GITHUB_PATH so subsequent steps also resolve the correct Ruby version. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Dual runner_provider matrix in size-analysis.yml ios job will produce duplicate Sentry size-analysis uploads
When Bitrise becomes operational, both cirrus and bitrise matrix legs will each run sentry-cli build upload, sending two XCArchive measurements to Sentry for every commit. Consider restricting the matrix to ["cirrus"] here, as testflight.yml does, to avoid duplicate or environment-skewed entries in the size trend data.
Evidence
size-analysis.ymliosjob setsrunner_provider: ["cirrus", "bitrise"]in the matrix.- The
Upload iOS XCArchive to Sentry Size Analysisstep is only gated byif: env.SENTRY_AUTH_TOKEN != ''— norunner_providerguard. testflight.ymlupload_to_testflightexplicitly usesrunner_provider: ["cirrus"]only, citing the same dual-runner pattern but intentionally avoiding duplicate uploads.- Once Bitrise is live, every main-branch push would yield two build entries in Sentry's size analysis dashboard, one per runner provider.
Identified by Warden code-review
Replace rbenv + PATH hacks with native asdf commands on Bitrise VMs: - Node: asdf install/set to match actions/setup-node version - Ruby: asdf install ruby (reads .ruby-version) instead of rbenv - Cache ~/.asdf/installs/ruby instead of ~/.rbenv/versions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bitrise VMs run asdf <0.15 which doesn't have the `asdf set` command. Use `asdf global` and `asdf local` instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The asdf nodejs plugin on Bitrise has outdated node-build definitions that don't know about the latest Node versions. Use `asdf global nodejs system` to make asdf's shim pass through to the actions/setup-node binary already on PATH. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Both matrix runners upload identical Sentry size analysis data, producing duplicate entries
In size-analysis.yml, the Upload iOS XCArchive to Sentry Size Analysis step (line 184) is only gated on SENTRY_AUTH_TOKEN != '', not on runner_provider == 'cirrus'. When both cirrus and bitrise matrix instances succeed, each uploads the same archive to the same org/project/build-configuration, creating duplicate size data points per commit. Guard the upload with if: env.SENTRY_AUTH_TOKEN != '' && matrix.runner_provider == 'cirrus'.
Evidence
size-analysis.ymliosjob now hasmatrix.runner_provider: ["cirrus", "bitrise"](newly added in this PR).- The upload step at line 184 checks only
if: env.SENTRY_AUTH_TOKEN != ''; there is norunner_providerguard. - Both jobs invoke
sentry-cli build uploadwith identical--org,--project, and--build-configuration "Release"arguments (lines 188–192). continue-on-error: trueon the Bitrise job does not prevent a successful upload; it only suppresses workflow failure.- The Android
sentry-cli build upload(line 86) is in a non-matrix job and is unaffected.
Identified by Warden find-bugs
asdf install ruby without a version reads .ruby-version (3.4.7) where kconv was moved to a separate gem. Pin to 3.3.0 to match Cirrus. Also remove Bitrise from size-analysis matrix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
asdf global writes to ~/.tool-versions (lowest priority), but the repo's .ruby-version (3.4.7) takes precedence when asdf traverses parent directories. Use asdf local to write .tool-versions in the repo root, which overrides .ruby-version. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Changed the runner provider from Cirrus to Bitrise in both size-analysis and testflight workflows to ensure consistency in CI execution.
There was a problem hiding this comment.
iOS size analysis job loses its Cirrus runner — job will never execute (.github/workflows/e2e-v2.yml:101)
The ios job matrix in size-analysis.yml contains only ["bitrise"] instead of ["cirrus", "bitrise"], so the previously-working Cirrus-based size analysis is silently dropped. Since Bitrise is not yet configured (per the PR description, the GitHub app is still pending), and continue-on-error evaluates to true for the sole bitrise entry, the job will always fail silently — meaning iOS size analysis never actually runs.
Evidence
size-analysis.ymlline 101:runner_provider: ["bitrise"]— only one value, missing the"cirrus"entry that every other macOS job uses.- Line 96:
continue-on-error: ${{ matrix.runner_provider == 'bitrise' }}evaluates totruefor the sole matrix entry, so failures are swallowed and the job effectively never produces real results. - The job still contains a Cirrus-only
Cache Rubystep gated onif: matrix.runner_provider != 'bitrise', which can never execute given the matrix — indicating the missing"cirrus"is an oversight rather than intent. - All other macOS jobs use
["cirrus", "bitrise"](e.g.native-tests.yml:43,sample-application.yml:64), matching the PR description that the matrix is added to all macOS jobs.
Testflight upload job loses its Cirrus runner — uploads to TestFlight will silently stop (.github/workflows/e2e-v2.yml:24)
The upload_to_testflight job matrix in testflight.yml contains only ["bitrise"] instead of ["cirrus", "bitrise"], removing the working Cirrus-based upload. The job's runs-on expression still defines a Cirrus fallback (fromJSON('["ghcr.io/cirruslabs/macos-tahoe-xcode:26.2.0", "runner_group_id:10"]')), but no matrix entry ever selects it. With Bitrise not yet configured and continue-on-error: ${{ matrix.runner_provider == 'bitrise' }} always evaluating to true for the sole bitrise entry, every push to main will appear to succeed while no build is actually uploaded to TestFlight.
Evidence
testflight.ymlline 24:runner_provider: ["bitrise"]— only one value, omitting"cirrus".- Line 17
runs-onretains the Cirrus branch (|| fromJSON('["ghcr.io/cirruslabs/macos-tahoe-xcode:26.2.0", "runner_group_id:10"]')) that is now unreachable because the matrix never yields a non-bitrise value. - Line 18
continue-on-error: ${{ matrix.runner_provider == 'bitrise' }}always resolves totruefor the only entry, suppressing all failures. - Workflow triggers on
push: branches: [main](lines 2-5), so every merge silently skips the TestFlight upload. - Sibling macOS jobs use
["cirrus", "bitrise"](sample-application.yml:64,263,371,native-tests.yml:43,sample-application-expo.yml:55), confirming"cirrus"was inadvertently dropped here.
Identified by Warden find-bugs
itaybre
changed the title
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
| uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 | ||
| with: | ||
| name: ${{ matrix.rn-version }}-${{ matrix.rn-architecture }}-${{ matrix.engine }}-${{ matrix.platform }}-${{ matrix.build-type }}-${{ matrix.ios-use-frameworks }}-app-package | ||
| name: ${{ matrix.rn-version }}-${{ matrix.rn-architecture }}-${{ matrix.engine }}-${{ matrix.platform }}-${{ matrix.build-type }}-${{ matrix.ios-use-frameworks }}-${{ matrix.runner_provider }}-app-package | ||
| path: dev-packages/e2e-tests |
There was a problem hiding this comment.
Bug: If the react-native-build job fails, it silently skips uploading the build artifact, causing the dependent react-native-test job to fail to download it and not run any tests.
Severity: MEDIUM
Suggested Fix
Add the if: always() condition to the "Upload App" step within the react-native-build job. This ensures the artifact upload step runs even if a preceding step in the job fails, preventing the downstream test job from failing due to a missing artifact.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: .github/workflows/e2e-v2.yml#L536-L539
Potential issue: The `react-native-build` job is configured with `continue-on-error:
true` at the job level. If a step within this job fails, GitHub Actions marks the job as
successful but skips all subsequent steps, including the one that uploads the build
artifact. Consequently, the downstream `react-native-test` job will fail when it tries
to download the non-existent artifact. Because the test job also has `continue-on-error:
true`, this failure is silently ignored, and the CI run appears to succeed without any
tests actually being executed.
Did we get this right? 👍 / 👎 to inform future reviews.
antonis
left a comment
antonis
left a comment
There was a problem hiding this comment.
LGTM 🚀
Thank you for adding the Bitrise runners Itay 🙇
2 participants
📢 Type of change
📜 Description
Adds a
runner_providermatrix dimension (cirrus/bitrise) to macOS CI jobs, replicating the approach from sentry-cocoa#7971.Changes per job
native-tests.ymltest-iossample-application.ymlbuild-ios,build-macos,test-iossample-application-expo.ymlbuild-iose2e-v2.ymlmetrics,react-native-build,react-native-testsize-analysis.ymliostestflight.ymlupload_to_testflightPattern applied
runner_providermatrix:["cirrus", "bitrise"]added to macOS jobs. For mixed-platform jobs (e2e), Android + Bitrise combinations are excluded.runs-on: Bitrise usesbitrise_pool_name:<macos_version>, Cirrus keeps the original runner image.continue-on-error: Bitrise jobs won't block CI.asdf global nodejs systemafteractions/setup-nodeso asdf's shim passes through to the GitHub Actions-installed Node.asdf install ruby 3.3.0+asdf local ruby 3.3.0to match the 3.3.0 version used byruby/setup-rubyon Cirrus. Usesasdf local(notglobal) because the repo's.ruby-version(3.4.7) would otherwise take precedence. Ruby installs are cached under~/.asdf/installs/ruby.runner_providerto avoid collisions.Bitrise-disabled workflows
testflight.yml: Both runners would upload the samegithub.run_numberbuild number to TestFlight, causing a duplicate rejection.size-analysis.yml: Not relevant for the Bitrise evaluation and would add noise.💡 Motivation and Context
Preparing to evaluate Bitrise as an alternative/additional macOS CI runner provider, matching the effort in sentry-cocoa.
💚 How did you test it?
Running workflows on this PR. Bitrise jobs use
continue-on-errorso failures don't block CI.📝 Checklist
sendDefaultPIIis enabled🔮 Next steps
testflight.ymlwith unique build numbers per provider#skip-changelog