Skip to content

Show remaining runs when using --repeat-until-failure#15270

Merged
josevalim merged 3 commits into
elixir-lang:mainfrom
Gazler:feat/show-remaining-runs
Apr 15, 2026
Merged

Show remaining runs when using --repeat-until-failure#15270
josevalim merged 3 commits into
elixir-lang:mainfrom
Gazler:feat/show-remaining-runs

Conversation

@Gazler
Copy link
Copy Markdown
Contributor

@Gazler Gazler commented Apr 14, 2026

The --repeat-until-failure can be very useful for finding flaky tests, however sometimes it is hard to gauge how long it has left to run before it can be considered successful.

This commit adds a small output showing the number of remaining runs when the --repeat-until-failure flag is used.

It looks like:

Running ExUnit with seed: 382462, max_cases: 32, remaining_runs: 3
Excluding tags: [windows: true]

.......
Finished in 0.02 seconds (0.00s async, 0.02s sync)

Result: 7 passed
Running ExUnit with seed: 422319, max_cases: 32, remaining_runs: 2
Excluding tags: [windows: true]

.......
Finished in 0.00 seconds (0.00s async, 0.00s sync)

@josevalim
Copy link
Copy Markdown
Member

@Gazler, nice call! Let's add it here:

Running ExUnit with seed: 49613, max_cases: 32, remaining_runs: 3

@Gazler
Copy link
Copy Markdown
Contributor Author

Gazler commented Apr 14, 2026

Gah - that's what I had originally and I opted to go against it because it's not really an option that you run with.

Running ExUnit with... feels more like configuration options than something that changes each run. Happy to go with it if you prefer though!

@josevalim
Copy link
Copy Markdown
Member

Yes, let’s go with that. :)

The --repeat-until-failure can be very useful for finding flaky tests,
however sometimes it is hard to gauge how long it has left to run before
it can be considered successful.

This commit adds a small output showing the number of remaining runs
when the `--repeat-until-failure` flag is used.

It looks like:

```
Running ExUnit with seed: 382462, max_cases: 32, remaining_runs: 3
Excluding tags: [windows: true]

.......
Finished in 0.02 seconds (0.00s async, 0.02s sync)

Result: 7 passed
Running ExUnit with seed: 422319, max_cases: 32, remaining_runs: 2
Excluding tags: [windows: true]

.......
Finished in 0.00 seconds (0.00s async, 0.00s sync)
```
@Gazler Gazler force-pushed the feat/show-remaining-runs branch from e22a59d to 734f5a4 Compare April 14, 2026 20:56
@Gazler
Copy link
Copy Markdown
Contributor Author

Gazler commented Apr 14, 2026

Yes, let’s go with that. :)

I've updated it.

Comment thread lib/ex_unit/lib/ex_unit.ex Outdated
Comment thread lib/ex_unit/lib/ex_unit/cli_formatter.ex Outdated
Co-authored-by: José Valim <jose.valim@gmail.com>
Comment thread lib/ex_unit/lib/ex_unit/cli_formatter.ex Outdated
@josevalim josevalim merged commit 5b97e9c into elixir-lang:main Apr 15, 2026
15 checks passed
@Gazler Gazler deleted the feat/show-remaining-runs branch April 15, 2026 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants