Repository navigation
How to get the seed if process crashed? #724
Description
I'm trying to add fuzz tests to an existing code base that has a dependency on a native C DLL that I call via P/Invoke, and I'm getting an intermittent crash in that native library I'm trying to get to the bottom of.
However, because the test process crashes I'm not getting any indication of what the seed was to try and reliably reproduce the sequence of events/inputs that causes the crash because the code doesn't get as far as a failed assertion.
It doesn't seem to be quite as simple as "input A make function B crash" as from some manual logging it appears that which test/function is crashing the process varies, so I think I need to apply the seed to the whole process to get a repro I can reliably debug.
I couldn't see an obvious way to manually access the seed to log it for later re-use - is there a way to do this?
Activity
ploeh commentedon Dec 18, 2025
I can't say how to manually access the seed, but if that's not possible, a fallback strategy might be to use
Gen.sampleto pre-generate test values, so that you know which values you're about to try. Perhaps you can find some inspiration in this article of mine: https://blog.ploeh.dk/2021/01/04/dynamic-test-oracles-for-rho-problemskurtschelfthout commentedon Dec 23, 2025
If you use the Verbose config https://fscheck.github.io/FsCheck/reference/fscheck-config.html#Verbose it prints the arguments before each test...might help some.
Another not-exactly-what-you-want option is to pick a seed yourself and see if it crashes with that. Depending on how intermittent your crash is.
Would be a nice feature to have a config option to let FsCheck output the seed of each test in Verbose mode, and the initial seed in Quick mode.
martincostello commentedon Jan 2, 2026
Thanks - I'll see if I can get something useful out of verbose, though I did try that but didn't seem to get any useful output (but I might have just been holding it wrong).
martincostello commentedon Jan 5, 2026
Yeah, I don't seem to be getting anywhere with
[Property(Verbose = true)]. If I run my tests with the xunit v3 integration with the commanddotnet test --configuration Release --framework net10.0 --tl:off --logger "console;verbosity=detailed"I don't get any output as the tests run.I also added the below in case things were being written to the console directly, but that didn't help:
martincostello commentedon Jan 5, 2026
I can get the output if I refactor all the tests to use
[Fact]instead withProp.ForAll(...).VerboseCheck(outputHelper), but then yeah then I don't get the seed, just a list of all the inputs.I'll see if I can do a PR to get the seed to be output.
Prototype seed output
martincostello commentedon Jan 5, 2026
Had a quick play around, but I don't think I got towards a proper working prototype. I pushed the changes I made here: martincostello@5f0bf6b
Feels weird to me that it would be an extra option on top of verbose, rather than just being part of verbose itself. Also the current output implementation runs when a test finishes - if the process crashes, it would still fail to print the seed for the test that caused the crash, you'd just see all the seeds for tests prior to the crash.
For this scenario, it feels like instead it would be part of verbose, but a "before" header including the seed would need to be written, then the rest of the entry for a given test written afterwards (e.g. what all the inputs were), so the output would need to be sort of "streamed".
kurtschelfthout commentedon Jan 6, 2026
Hm, yeah I think maybe we lost the streaming output when we introduced concurrent tests - that tends to mess up things like that. I agree it should just be part of Verbose instead of a separate option...not sure how hard it is to make the output more incremental. xUnit has the TestOutputHelper which I believe should also work to neatly separate output from concurrently running tests, that's probably its main purpose.