Skip to content

Add Elixir and OTP CI coverage and stabilize distributed tests - #5

Open
jeregrine wants to merge 2 commits into
mainfrom
add-ci-matrix
Open

Add Elixir and OTP CI coverage and stabilize distributed tests#5
jeregrine wants to merge 2 commits into
mainfrom
add-ci-matrix

Conversation

@jeregrine

@jeregrine jeregrine commented Sep 9, 2026

Copy link
Copy Markdown
Member

Problem

Group needs automated coverage across supported Elixir/OTP versions without making every PR run fault campaigns or performance benchmarks. The local suite was one module, preventing useful ExUnit concurrency, and started distribution even for local-only runs. Partition tests also raced shard discovery and cleanup, while failures lacked durable peer diagnostics.

Fix

  • Run latest-stable formatting, compiler checks and local examples/reference-model histories first, then a six-job runtime matrix with bounded real-peer coverage. Enforce warnings as errors throughout. Only push events on main run CI, avoiding duplicate branch/PR matrices.
  • Add nightly fault campaigns at 03:23 UTC and manual/release triggers: multiple seeds, 1/2/4 peer schedulers, larger histories, repeated partition/churn/restart scenarios, and three targeted behavioral mutations.
  • Add a separate performance workflow comparing a baseline and candidate on the same runner with a pinned Benchee harness and distributed load/recovery benchmarks.
  • Split local examples into independently scheduled subsystem modules, keeping global-tracing and environment-changing modules serial. Start distribution lazily and use configurable, smaller scheduler pools for fresh correctness peers.
  • Replace fairness synchronization sleeps with mailbox conditions, wait for shard state around partitions, acknowledge monitor installation, and prevent cross-VM node-name collisions.
  • Retain failure output, seeds, generated histories, topology, configuration, bounded shard state, queue/buffer counts and peer diagnostics before teardown.

Supporting information

  • Actions use current SHA-pinned releases with read-only repository permissions. PR failure artifacts and campaign reports are retained for 14 days; performance reports for 30 days.
  • Runtime ranges exclude prereleases. The supported matrix covers Elixir 1.19 / OTP 26–28, Elixir 1.20 / OTP 27–28, and the latest-stable pair.
  • Release publication reruns the workflows but is not automatically gated. Run them manually on the candidate ref and review performance against the previous release before publishing.
  • Generated histories are sequential reference-model checks, mutations are focused contract probes, and nightly repetitions are bounded fault/churn campaigns—not exhaustive mutation or long-duration soak coverage. Mixed Group-version wire compatibility is not established by a runtime-version matrix.
  • Runtime library behavior is unchanged; lane budgets, replay commands and limitations are documented in test/README.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant