chore: upgrade xunit from v2 to v3#4623
Open
dbrattli wants to merge 4 commits into
Open
Conversation
Replace the deprecated xunit v2 (`XUnit` 2.9.3) package with `xunit.v3` 3.2.2 in the Beam, Php, Python and Rust test projects. v3 test projects are self-executing, so each project now sets `OutputType=Exe` (added to Rust and Php; Python and Beam already had it) and the .NET-side `[<EntryPoint>]` is dropped in favour of the runner entry point generated by xunit.v3: - Rust `main.fs` is also transpiled to Rust's `fn main`, so its entry point is guarded with `#if FABLE_COMPILER` rather than removed. - Python/Beam/Php had the entry point only in the .NET (`#else`) branch of `Main.fs`, which is removed. `xunit.runner.visualstudio` 3.1.5 is already v3-compatible, so `dotnet test` continues to work. Test sources need no changes: `Xunit.FactAttribute`, `Assert.Equal` and `Assert.NotEqual` are unchanged in v3. Verified with `dotnet test -c Release`: Rust (2449), Python (2266) and Beam (2439) pass. The Php project has pre-existing compile errors on .NET unrelated to xunit and is not run by the build system. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Python Type Checking Results (Pyright)
Excluded files with errors (4 files)These files have known type errors and are excluded from CI. Remove from
|
CI revealed that the Fable transpilation of the Rust tests failed with
FS0433 ("EntryPointAttribute must be the last declaration in the last
file"). When Fable cracks a project it defines FABLE_COMPILER and runs
the CoreCompile target, which triggers xunit.v3's _XunitAttachSourceFiles
target. That appends a generated entry point and the built-in runner
reporters after the project's own source files. For Rust this collides
with the `[<EntryPoint>]` in tests/src/main.fs (and those sources also
reference runtime types Fable cannot transpile).
Disable both attached source files for the Fable build only, keyed off
the FABLE_COMPILER MSBuild property that Fable's MSBuildCrackerResolver
passes (`/p:FABLE_COMPILER=True`). On .NET the properties keep their
defaults, so xunit.v3 still generates the test runner entry point.
Verified `dotnet test -c Release` (Rust 2449 passing) and the Fable
transpile (Rust/Python/Beam) all succeed with no xunit sources leaking
into the transpiled output.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Several MailboxProcessor/async tests assert from inside a fire-and-forget `Async.StartImmediate`, whose continuation can run after its test collection has been torn down. xunit.v3 reports such background-thread exceptions as catastrophic failures and exits non-zero (xunit v2 silently ignored them). With parallel test collections this surfaces on Windows as "Catastrophic failure: Assert.Equal() ... Expected 42, Actual 0". Add tests/Python/xunit.runner.json disabling test-collection parallelization so each collection fully drains before the next runs. This is the only xunit suite that runs on Windows in CI (Rust/Beam/Php .NET tests run on Linux only; the JS suite uses Expecto). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…feedback - Add xunit.runner.json to Beam and Php: their tests use the same fire-and-forget Async.StartImmediate pattern as Python, so they are exposed to the same xunit.v3 catastrophic background-exception failures. - Collapse the now-identical #if FABLE_COMPILER branches in Beam/Php/Python Main.fs and add the missing trailing newlines. - Compress the .fsproj comments explaining the cracking workaround. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
xUnit v2 is deprecated. This upgrades the .NET-side test runner from xunit v2 (
XUnit2.9.3) toxunit.v33.2.2 across the Beam, Php, Python and Rust test projects.Changes
XUnit2.9.3 withxunit.v33.2.2 in all four test.fsprojfiles.OutputType=Exe(added to Rust and Php; Python and Beam already had it) and the .NET-side[<EntryPoint>]is dropped in favour of the entry point generated by xunit.v3:main.fsis also transpiled to Rust'sfn main, so its entry point is guarded with#if FABLE_COMPILERinstead of removed.Main.fscollapses to a plain emptymodule Program— both#ifbranches had become identical.FABLE_COMPILERset, which would pull in xunit.v3's generated entry point and runner-reporter sources (.NET-only, they break transpilation).XunitAutoGeneratedEntryPointandXunitRegisterBuiltInRunnerReportersare disabled for the Fable build only.xunit.runner.json(serial collections, single thread) to Python, Beam and Php: their MailboxProcessor/async tests assert from fire-and-forgetAsync.StartImmediatecontinuations that can outlive their test collection, and xunit.v3 reports such background exceptions as catastrophic (v2 ignored them). Observed failing on Windows; the same race exists in all three projects.xunit.runner.visualstudio3.1.5 is already v3-compatible, sodotnet testkeeps working via VSTest.Xunit.FactAttribute,Assert.EqualandAssert.NotEqualare unchanged in v3.Verification
dotnet test -c Release:CI timing impact of serial execution
Compared the CI run for this branch against the three most recent successful
mainruns (xunit v2, fully parallel) — no measurable degradation:Deltas are within run-to-run variance (baselines themselves spread by 16–43s). These steps are dominated by Fable transpilation and the target-runtime test runs, which are unaffected; the serialized .NET xunit pass takes only ~2–3s for ~2,300 tests locally.
Local timing on a multi-core machine
The relative cost of serialization is larger on a many-core dev machine, but still ~2s in absolute terms. Measured on a 24-core Linux box (
dotnet test -c Release --no-buildwall clock, xunit-reported duration in parentheses):Parallelism scales poorly here because xunit parallelizes per test collection (one per test module) and the runtime is dominated by a few slow async/MailboxProcessor collections with real waits — the run can't go below the slowest collection, while the other collections are sub-millisecond assertions either way.
🤖 Generated with Claude Code