From ef06f5d848a67aa06c4940fee01c5856e1c8efee Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 21:08:32 +0800 Subject: [PATCH 01/22] docs: implementation plan for test isolation/filter/json + musl deliverable --- .../2026-07-23-test-isolation-json-plan.md | 939 ++++++++++++++++++ 1 file changed, 939 insertions(+) create mode 100644 .agents/docs/2026-07-23-test-isolation-json-plan.md diff --git a/.agents/docs/2026-07-23-test-isolation-json-plan.md b/.agents/docs/2026-07-23-test-isolation-json-plan.md new file mode 100644 index 0000000..cf4fc66 --- /dev/null +++ b/.agents/docs/2026-07-23-test-isolation-json-plan.md @@ -0,0 +1,939 @@ +# mcpp test: per-test isolation, filter, JSON output — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** `mcpp test` becomes exercise-grade: one broken test no longer kills the run (per-test compile isolation), tests in subdirectories get path-based names, a positional pattern filters which tests run, and `--message-format json` emits machine-readable NDJSON — the four upstream capabilities required by the d2mcpp "exercises as tests" redesign (`d2mcpp/.agents/docs/2026-07-23-exercises-as-tests-design.md` §4), each shaped as a generic cargo/ctest-parity feature. + +**Architecture:** `run_tests` (src/build/execute.cppm) moves from "one monolithic backend build, then run everything" to a two-phase flow: Phase A builds package-level artifacts (every non-TestBinary link unit) — failure there is a *package error*; Phase B builds each test's own ninja target individually via a new `BuildOptions::ninjaTargets` knob and runs it on success — a compile failure is attributed to *that test only*. Test names become `tests/`-relative paths (`00-intro/0`, not stem), which is what makes chapter subdirectories usable at all (duplicate stems currently hard-error). The filter selects at Phase B only — the plan always contains all tests so `compile_commands.json` stays complete for clangd. + +**Tech Stack:** C++23 modules (mcpp builds itself with mcpp), ninja backend, bash e2e scripts under `tests/e2e/`. + +## Global Constraints + +- Repo: `/home/speak/workspace/github/mcpp-community/mcpp`, branch `feat/test-isolation-json` off `main`. +- Code comments and commit messages in English (repo convention; d2mcpp/d2x use Chinese, mcpp does not). +- No new dependencies. JSON emission via a local `json_escape` (same shape as the one in `src/cli/cmd_xpkg.cppm:28`). +- In `--message-format json` mode stdout carries NDJSON **only**; all human/status text must be absent (via `ui::set_quiet`) or on stderr. Consumers ignore unparseable lines, but we do not rely on that. +- JSON record schema is fixed by the d2mcpp design doc §4.3 (field names verbatim): `test`, `status` (`pass|compile_fail|run_fail`), `exit_code`, `signal`, `compile_output`, `run_output`; package-level record `{"error":"package",...}`. +- Every task ends with: mcpp still builds itself (`mcpp build` in repo root exits 0). +- Build/verify commands assume the locally installed `mcpp` bootstrap binary in PATH; e2e scripts take the freshly built binary via `MCPP=`. + +**Dev-loop commands used throughout:** + +```bash +cd /home/speak/workspace/github/mcpp-community/mcpp +mcpp build # self-build (glibc, fast) — dev loop +BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) # freshly built binary +``` + +--- + +### Task 0: Branch + +**Files:** none + +- [ ] **Step 1: Create branch** + +```bash +cd /home/speak/workspace/github/mcpp-community/mcpp +git checkout main && git pull --ff-only +git checkout -b feat/test-isolation-json +``` + +--- + +### Task 1: Path-based test names (subdirectory support) + +Currently `run_tests` names each test by file **stem** and hard-errors on duplicates, so `tests/00-a/0.cpp` + `tests/01-b/0.cpp` cannot coexist. Name them by `tests/`-relative path without extension instead. `target_output` (src/build/plan.cppm:192) maps name → `bin/`, so `00-a/0` yields nested `bin/00-a/0` — ninja creates output directories itself; no change needed there. + +**Files:** +- Modify: `src/build/execute.cppm` (run_tests, discovery block around line 705–722) +- Test: `tests/e2e/118_test_subdir_names.sh` (new) + +**Interfaces:** +- Produces: test names of the form `` with `/` separators (`generic_string()`), e.g. `00-intro/0`, `smoke` for a top-level `tests/smoke.cpp` (unchanged for flat layouts). Task 3/4/5 rely on `LinkUnit::targetName` carrying exactly this name. + +- [ ] **Step 1: Write the failing e2e test** + +Create `tests/e2e/118_test_subdir_names.sh` (executable): + +```bash +#!/usr/bin/env bash +# requires: +# tests/ subdirectories: same stem in two dirs must coexist, names are relative paths +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests/00-a pkg/tests/01-b +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "subdirs" +version = "0.1.0" +standard = "c++23" +EOF +cat > tests/00-a/0.cpp <<'EOF' +int main() { return 0; } +EOF +cat > tests/01-b/0.cpp <<'EOF' +int main() { return 0; } +EOF + +out=$("$MCPP" test 2>&1) || { echo "mcpp test failed: $out"; exit 1; } +[[ "$out" == *"00-a/0 ... ok"* ]] || { echo "missing path-based name 00-a/0: $out"; exit 1; } +[[ "$out" == *"01-b/0 ... ok"* ]] || { echo "missing path-based name 01-b/0: $out"; exit 1; } +[[ "$out" == *"2 passed"* ]] || { echo "expected 2 passed: $out"; exit 1; } +echo OK +``` + +- [ ] **Step 2: Run it to verify it fails** + +```bash +chmod +x tests/e2e/118_test_subdir_names.sh +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/118_test_subdir_names.sh +``` + +Expected: FAIL with `duplicate test name '0'`. + +- [ ] **Step 3: Implement path-based naming** + +In `src/build/execute.cppm` `run_tests`, replace the naming block (currently `auto name = f.stem().string();` plus the duplicate-stem error) with: + +```cpp + // 2. Synthesize a Target for each test file. + // Name = path relative to tests/, extension dropped, '/' separators — + // so tests/00-a/0.cpp and tests/01-b/0.cpp coexist as '00-a/0' and + // '01-b/0' (stems alone would collide). Flat layouts keep their old + // names ('tests/smoke.cpp' → 'smoke'). + std::vector testTargets; + std::set seenNames; + for (auto& f : testFiles) { + auto rel = std::filesystem::relative(f, testRoot / "tests"); + auto name = rel.replace_extension("").generic_string(); + if (!seenNames.insert(name).second) { + mcpp::ui::error(std::format( + "duplicate test name '{}' (two test files map to the same name)", name)); + return 2; + } + mcpp::manifest::Target t; + t.name = name; + t.kind = mcpp::manifest::Target::TestBinary; + // Relative to the member/package root prepare_build will operate on. + t.main = std::filesystem::relative(f, testRoot).string(); + testTargets.push_back(std::move(t)); + } +``` + +- [ ] **Step 4: Run the e2e test to verify it passes** + +```bash +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/118_test_subdir_names.sh +``` + +Expected: `OK`. Also confirm no regression on flat layout: `MCPP="$PWD/$BIN" bash tests/e2e/100_feature_sources_test_mode.sh` (existing test-mode e2e) still passes. + +- [ ] **Step 5: Commit** + +```bash +git add src/build/execute.cppm tests/e2e/118_test_subdir_names.sh +git commit -m "feat(test): name tests by tests/-relative path, enabling subdirectories" +``` + +--- + +### Task 2: `BuildOptions::ninjaTargets` — build a subset of the plan + +Generic backend knob: explicit ninja targets (LinkUnit `output` paths, relative to the plan's outputDir). Empty = current behavior (build everything). This is the mechanism Task 3 uses for both phases. + +**Files:** +- Modify: `src/build/backend.cppm:12-16` (BuildOptions) +- Modify: `src/build/ninja_backend.cppm:1104-1230` (NinjaBackend::build) + +**Interfaces:** +- Produces: `BuildOptions::ninjaTargets` (`std::vector`); when non-empty, `NinjaBackend::build` passes them as ninja goal targets and `producedArtifacts` lists only the matching link units. Diagnostics on failure are naturally scoped to what was built. + +- [ ] **Step 1: Extend BuildOptions** + +In `src/build/backend.cppm`: + +```cpp +struct BuildOptions { + bool verbose = false; + bool dryRun = false; + std::size_t parallelJobs = 0; + // Explicit ninja goal targets (LinkUnit::output paths, relative to the + // plan's outputDir). Empty = build the full plan (default behavior). + std::vector ninjaTargets; +}; +``` + +- [ ] **Step 2: Thread targets into the ninja invocation** + +In `src/build/ninja_backend.cppm` `NinjaBackend::build`, after the `-j` option block (line ~1196), append: + +```cpp + // Explicit goal targets: ninja builds only these outputs (and their + // prerequisites). Used by `mcpp test` to isolate per-test compiles. + for (auto& t : opts.ninjaTargets) + nargv.push_back(t); +``` + +And scope `producedArtifacts` in the success branch (replace the existing loop at line ~1217): + +```cpp + if (ok) { + if (opts.verbose && !out.empty()) + std::fputs(out.c_str(), stdout); + std::set want(opts.ninjaTargets.begin(), opts.ninjaTargets.end()); + for (auto& lu : plan.linkUnits) { + if (!want.empty() && !want.contains(lu.output.generic_string())) continue; + r.producedArtifacts.push_back(plan.outputDir / lu.output); + for (auto const& alias : lu.runtimeAliases) { + r.producedArtifacts.push_back(plan.outputDir / alias); + } + } + } +``` + +- [ ] **Step 3: Verify no behavior change with empty ninjaTargets** + +```bash +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/02_new_build_run.sh +MCPP="$PWD/$BIN" bash tests/e2e/118_test_subdir_names.sh +``` + +Expected: both `OK` / pass. (Dedicated behavior coverage for non-empty ninjaTargets lands with Task 3's e2e, which exercises it on both phases.) + +- [ ] **Step 4: Commit** + +```bash +git add src/build/backend.cppm src/build/ninja_backend.cppm +git commit -m "feat(build): BuildOptions::ninjaTargets — build an explicit subset of the plan" +``` + +--- + +### Task 3: Per-test compile isolation (two-phase run_tests) + +The acceptance test from the design doc: a project with one passing, one runtime-failing, one non-compiling test must produce **three results**, with the passing one actually executed. + +**Files:** +- Modify: `src/build/execute.cppm` (run_tests, build+run sections — replaces the single `backend->build` at line ~766 and the run loop at ~805) +- Test: `tests/e2e/119_test_isolation.sh` (new) + +**Interfaces:** +- Consumes: `BuildOptions::ninjaTargets` (Task 2), path-based names (Task 1). +- Produces: per-test result records in an internal `struct TestResult { std::string name; enum class St { Pass, CompileFail, RunFail } status; int exitCode; std::string compileOutput; std::string runOutput; };` collected in a `std::vector results` — Task 5 serializes exactly these fields. Human output per test: ` ... ok` / ` ... FAIL (exit N)` / ` ... FAIL (compile)`. Package-level (Phase A) failure keeps today's behavior: `ui::error` + diagnostics to stderr, return 1. + +- [ ] **Step 1: Write the failing e2e test** + +Create `tests/e2e/119_test_isolation.sh` (executable): + +```bash +#!/usr/bin/env bash +# requires: +# Per-test compile isolation: a non-compiling test fails alone; others build & run. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "isolation" +version = "0.1.0" +standard = "c++23" +EOF +cat > tests/ok.cpp <<'EOF' +int main() { return 0; } +EOF +cat > tests/runfail.cpp <<'EOF' +int main() { return 1; } +EOF +cat > tests/nocompile.cpp <<'EOF' +int main() { D2X_YOUR_ANSWER x = 1; return x; } +EOF + +set +e +out=$("$MCPP" test 2>&1) +code=$? +set -e +[[ $code -eq 1 ]] || { echo "expected exit 1, got $code: $out"; exit 1; } +[[ "$out" == *"ok ... ok"* ]] || { echo "passing test did not run: $out"; exit 1; } +[[ "$out" == *"runfail ... FAIL (exit 1)"* ]] || { echo "runtime failure not reported: $out"; exit 1; } +[[ "$out" == *"nocompile ... FAIL (compile)"* ]] || { echo "compile failure not isolated: $out"; exit 1; } +[[ "$out" == *"1 passed"* && "$out" == *"2 failed"* ]] || { echo "bad summary: $out"; exit 1; } + +# Package-level failure stays a hard error: break a src module, all-red is wrong, +# 'build failed' is right. +mkdir -p src +cat > src/isolation.cppm <<'EOF' +export module isolation; +this is not C++; +EOF +set +e +out2=$("$MCPP" test 2>&1) +code2=$? +set -e +[[ $code2 -ne 0 ]] || { echo "package error must fail: $out2"; exit 1; } +[[ "$out2" != *"nocompile ... FAIL (compile)"* ]] || { echo "package error misattributed to tests: $out2"; exit 1; } +echo OK +``` + +- [ ] **Step 2: Run it to verify it fails** + +```bash +chmod +x tests/e2e/119_test_isolation.sh +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/119_test_isolation.sh +``` + +Expected: FAIL — today the whole run dies with `error: build failed` and no test executes. + +- [ ] **Step 3: Implement the two-phase flow** + +In `src/build/execute.cppm` `run_tests`, replace step 5 (build everything) through step 6 (run loop) with: + +```cpp + struct TestResult { + std::string name; + enum class St { Pass, CompileFail, RunFail } status; + int exitCode = 0; + std::string compileOutput; + std::string runOutput; + }; + std::vector results; + + auto backend = mcpp::build::make_ninja_backend(); + + // Phase A: package-level artifacts (everything that is not a test + // binary — libs, deps). A failure here is the PACKAGE's fault, not any + // single test's: report it as a build error, never as 52 red tests. + std::vector pkgTargets; + for (auto& lu : ctx->plan.linkUnits) + if (lu.kind != mcpp::build::LinkUnit::TestBinary) + pkgTargets.push_back(lu.output.generic_string()); + if (!pkgTargets.empty()) { + mcpp::build::BuildOptions aOpts; + aOpts.ninjaTargets = pkgTargets; + auto a = backend->build(ctx->plan, aOpts); + if (!a) { + std::fflush(stdout); + mcpp::ui::error(a.error().message); + if (!a.error().diagnosticOutput.empty()) { + std::fputs(a.error().diagnosticOutput.c_str(), stderr); + if (a.error().diagnosticOutput.back() != '\n') + std::fputc('\n', stderr); + } + return 1; + } + } + + // Phase B: each test is built as its own ninja goal, so a compile + // failure is attributed to exactly that test; the rest still build+run. + auto t0 = std::chrono::steady_clock::now(); + + auto runtimeEnvKey = mcpp::platform::env::runtime_library_path_key(); + auto runtimeEnvValue = mcpp::platform::env::prepend_path_list( + runtimeEnvKey, ctx->plan.runtimeLibraryDirs); + + for (auto& lu : ctx->plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::TestBinary) continue; + + mcpp::build::BuildOptions bOpts; + bOpts.ninjaTargets = {lu.output.generic_string()}; + auto b = backend->build(ctx->plan, bOpts); + if (!b) { + std::println("{} ... FAIL (compile)", lu.targetName); + results.push_back({lu.targetName, TestResult::St::CompileFail, 0, + b.error().diagnosticOutput, {}}); + continue; + } + + auto exe = ctx->outputDir / lu.output; + mcpp::ui::status("Running", std::format("bin/{}", lu.targetName)); + + std::vector argv; + argv.push_back(exe.string()); + for (auto& a : passthrough) argv.push_back(a); + + std::vector> childEnv; + if (!runtimeEnvKey.empty() && !runtimeEnvValue.empty()) + childEnv.emplace_back(runtimeEnvKey, runtimeEnvValue); + + // Prepend the sandbox's subos/default/bin to the CHILD PATH so test + // binaries that shell out to bootstrapped tools (patchelf, ninja) find + // them — applied to the child only, not via a leaky shell prefix. + if constexpr (!mcpp::platform::is_windows) { + if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(ctx->tc.binaryPath)) { + auto registryDir = xpkgs->parent_path().parent_path(); + auto sandboxBin = registryDir / "subos" / "default" / "bin"; + if (std::filesystem::exists(sandboxBin)) { + std::array extra{sandboxBin}; + auto pathVal = mcpp::platform::env::prepend_path_list("PATH", extra); + if (!pathVal.empty()) childEnv.emplace_back("PATH", pathVal); + } + } + } + + int exitCode = mcpp::platform::process::run_exec(argv, childEnv); + if (exitCode == 0) { + std::println("{} ... ok", lu.targetName); + results.push_back({lu.targetName, TestResult::St::Pass, 0, {}, {}}); + } else { + std::println("{} ... FAIL (exit {})", lu.targetName, exitCode); + results.push_back({lu.targetName, TestResult::St::RunFail, exitCode, {}, {}}); + } + } +``` + +Then rewrite step 7 (summary) on top of `results`: + +```cpp + auto elapsed = std::chrono::duration_cast( + std::chrono::steady_clock::now() - t0); + + int passed = 0, failed = 0; + std::vector failures; + for (auto& r : results) { + if (r.status == TestResult::St::Pass) ++passed; + else { ++failed; failures.push_back(r.name); } + } + + std::println(""); + if (failed == 0) { + mcpp::ui::status("test result", + std::format("ok. {} passed; 0 failed; finished in {:.2f}s", + passed, static_cast(elapsed.count()) / 1000.0)); + return 0; + } + mcpp::ui::error(std::format( + "test result: FAILED. {} passed; {} failed; finished in {:.2f}s", + passed, failed, static_cast(elapsed.count()) / 1000.0)); + std::println(""); + std::println("failures:"); + for (auto& n : failures) std::println(" {}", n); + // Compile diagnostics per failed test, on stderr, after the summary — + // grouped so a learner scrolls to exactly their test's errors. + for (auto& r : results) { + if (r.status != TestResult::St::CompileFail || r.compileOutput.empty()) continue; + std::println(stderr, "\n--- {} (compile) ---", r.name); + std::fputs(r.compileOutput.c_str(), stderr); + } + return 1; +``` + +Keep the existing M3.2 BMI-populate loop (`ctx->depsToPopulate`) — move it to right after Phase A's successful build (deps are package-level artifacts). Keep `mcpp::ui::finished("test", ...)`: call it after Phase A with `a->elapsed` when `pkgTargets` is non-empty, otherwise drop that line for this run (the per-test `... ok/FAIL` lines plus the summary carry the timing story). + +- [ ] **Step 4: Run the e2e tests** + +```bash +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/119_test_isolation.sh +MCPP="$PWD/$BIN" bash tests/e2e/118_test_subdir_names.sh +MCPP="$PWD/$BIN" bash tests/e2e/100_feature_sources_test_mode.sh +``` + +Expected: all `OK`/pass. If the tests-only fixture (no `src/`) errors in `prepare_build`, that is a bug to fix in this task — the test-mode path (src/build/plan.cppm:780-789 `inTestMode`) already skips non-test targets; adjust whatever remains that assumes a lib/bin target exists. + +- [ ] **Step 5: Commit** + +```bash +git add src/build/execute.cppm tests/e2e/119_test_isolation.sh +git commit -m "feat(test): per-test compile isolation — a broken test no longer kills the run" +``` + +--- + +### Task 4: `mcpp test ` filter + +Positional pattern, substring match on the path-based test name, applied at **Phase B selection only** — the plan still contains every test so `compile_commands.json` stays complete (clangd depends on it; a filtered build must not clobber it down to one entry). + +**Files:** +- Modify: `src/cli.cppm:252-268` (test subcommand: add positional arg) +- Modify: `src/cli/cmd_build.cppm:114-152` (cmd_test: read positional, thread through) +- Modify: `src/build/execute.cppm` (run_tests signature + Phase B selection) +- Test: `tests/e2e/120_test_filter.sh` (new) + +**Interfaces:** +- Consumes: `TestResult`/two-phase flow (Task 3). +- Produces: `export struct TestOptions { std::string filter; };` in `mcpp::build` (execute.cppm), extended by Task 5 with a `format` member. New signature: `run_tests(std::span passthrough, BuildOverrides overrides = {}, TestOptions testOpts = {})`. + +- [ ] **Step 1: Write the failing e2e test** + +Create `tests/e2e/120_test_filter.sh` (executable): + +```bash +#!/usr/bin/env bash +# requires: +# `mcpp test `: substring filter on path-based test names +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests/00-a pkg/tests/01-b +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "filter" +version = "0.1.0" +standard = "c++23" +EOF +echo 'int main() { return 0; }' > tests/00-a/0.cpp +echo 'int main() { return 0; }' > tests/01-b/0.cpp + +out=$("$MCPP" test 00-a 2>&1) || { echo "filtered run failed: $out"; exit 1; } +[[ "$out" == *"00-a/0 ... ok"* ]] || { echo "filtered test did not run: $out"; exit 1; } +[[ "$out" != *"01-b/0"* ]] || { echo "filter leaked other tests: $out"; exit 1; } +[[ "$out" == *"1 passed"* ]] || { echo "bad summary: $out"; exit 1; } + +set +e +out2=$("$MCPP" test does-not-exist 2>&1) +code2=$? +set -e +[[ $code2 -eq 2 ]] || { echo "no-match should exit 2, got $code2: $out2"; exit 1; } +[[ "$out2" == *"no tests match"* ]] || { echo "missing no-match error: $out2"; exit 1; } +echo OK +``` + +- [ ] **Step 2: Run it to verify it fails** + +```bash +chmod +x tests/e2e/120_test_filter.sh +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/120_test_filter.sh +``` + +Expected: FAIL — the positional is not declared, so `mcpp test 00-a` errors or ignores the argument and runs both tests. + +- [ ] **Step 3: Declare the positional and thread it through** + +`src/cli.cppm`, test subcommand — add before the `.option(...)` chain: + +```cpp + .subcommand(cl::App("test") + .description("Build + run all tests/**/*.cpp (after `--`, args go to each test binary)") + .arg(cl::Arg("pattern") + .help("Run only tests whose name contains PATTERN (optional)")) +``` + +`src/cli/cmd_build.cppm` `cmd_test` — read it and pass through (both the fan-out loop and the single-package call): + +```cpp + mcpp::build::TestOptions to; + if (parsed.positional_count() > 0) to.filter = parsed.positional(0); +``` + +…and change both `mcpp::build::run_tests(passthrough, mo)` / `run_tests(passthrough, ov)` calls to `run_tests(passthrough, mo, to)` / `run_tests(passthrough, ov, to)`. + +`src/build/execute.cppm` — new options struct + signature: + +```cpp +export struct TestOptions { + std::string filter; // substring match on the path-based test name; empty = all +}; + +export int run_tests(std::span passthrough, + BuildOverrides overrides = {}, + TestOptions testOpts = {}) { +``` + +Phase B selection (inside the loop from Task 3, first lines): + +```cpp + for (auto& lu : ctx->plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::TestBinary) continue; + if (!testOpts.filter.empty() + && lu.targetName.find(testOpts.filter) == std::string::npos) continue; +``` + +And before Phase B, the no-match guard: + +```cpp + if (!testOpts.filter.empty()) { + bool any = false; + for (auto& lu : ctx->plan.linkUnits) + if (lu.kind == mcpp::build::LinkUnit::TestBinary + && lu.targetName.find(testOpts.filter) != std::string::npos) { any = true; break; } + if (!any) { + mcpp::ui::error(std::format("no tests match '{}'", testOpts.filter)); + return 2; + } + } +``` + +Also update the "Compiling X (test)" announcement loop (step 4 of run_tests) to apply the same filter, so announcements match what actually builds. + +- [ ] **Step 4: Run the e2e tests** + +```bash +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/120_test_filter.sh +MCPP="$PWD/$BIN" bash tests/e2e/119_test_isolation.sh +``` + +Expected: both `OK`. + +- [ ] **Step 5: Commit** + +```bash +git add src/cli.cppm src/cli/cmd_build.cppm src/build/execute.cppm tests/e2e/120_test_filter.sh +git commit -m "feat(test): positional pattern filters tests by path-based name" +``` + +--- + +### Task 5: `--message-format json` (NDJSON) + +One record per executed/attempted test + a package-level error record + a trailing summary record. stdout is pure NDJSON (`ui::set_quiet` silences status lines, which all go to stdout; errors already go to stderr). Run output is captured via `capture_exec` instead of streamed. + +**Files:** +- Modify: `src/cli.cppm` (test subcommand: `--message-format` option) +- Modify: `src/cli/cmd_build.cppm` (cmd_test: parse + validate the value) +- Modify: `src/build/execute.cppm` (TestOptions::format, JSON emission, capture path) +- Test: `tests/e2e/121_test_json.sh` (new) + +**Interfaces:** +- Consumes: `TestOptions` + `TestResult` (Tasks 3–4). +- Produces (the d2mcpp Provider consumes this verbatim): + - per test: `{"test":"","status":"pass|compile_fail|run_fail","exit_code":N,"signal":N|null,"compile_output":"...","run_output":"..."}` + - package failure: `{"error":"package","compile_output":"..."}` then exit 1 + - no-match: `{"error":"no-tests-matched","filter":"..."}` then exit 2 + - summary (last line): `{"summary":{"passed":N,"failed":N,"elapsed_ms":N}}` + +- [ ] **Step 1: Write the failing e2e test** + +Create `tests/e2e/121_test_json.sh` (executable): + +```bash +#!/usr/bin/env bash +# requires: +# --message-format json: NDJSON per test, package error record, pure stdout +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "jsonout" +version = "0.1.0" +standard = "c++23" +EOF +cat > tests/ok.cpp <<'EOF' +#include +int main() { std::puts("hello from ok"); return 0; } +EOF +echo 'int main() { return 1; }' > tests/runfail.cpp +echo 'int main() { D2X_YOUR_ANSWER x = 1; return x; }' > tests/nocompile.cpp + +set +e +out=$("$MCPP" test --message-format json 2>/dev/null) +code=$? +set -e +[[ $code -eq 1 ]] || { echo "expected exit 1, got $code"; exit 1; } + +# stdout must be pure NDJSON: every line parses as a JSON object. +while IFS= read -r line; do + [[ -z "$line" ]] && continue + [[ "$line" == "{"* ]] || { echo "non-JSON line on stdout: $line"; exit 1; } +done <<< "$out" + +echo "$out" | grep -q '"test":"ok","status":"pass"' || { echo "missing pass record"; exit 1; } +echo "$out" | grep -q '"test":"runfail","status":"run_fail"' || { echo "missing run_fail record"; exit 1; } +echo "$out" | grep -q '"exit_code":1' || { echo "missing exit_code"; exit 1; } +echo "$out" | grep -q '"test":"nocompile","status":"compile_fail"' || { echo "missing compile_fail record"; exit 1; } +echo "$out" | grep -q 'D2X_YOUR_ANSWER' || { echo "compile_output missing diagnostics"; exit 1; } +echo "$out" | grep -q 'hello from ok' || { echo "run_output not captured"; exit 1; } +echo "$out" | grep -q '"summary":{"passed":1,"failed":2' || { echo "missing summary"; exit 1; } + +# Invalid format value → usage error +set +e +"$MCPP" test --message-format yaml >/dev/null 2>&1 +[[ $? -eq 2 ]] || { echo "invalid format should exit 2"; exit 1; } +set -e +echo OK +``` + +- [ ] **Step 2: Run it to verify it fails** + +```bash +chmod +x tests/e2e/121_test_json.sh +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/121_test_json.sh +``` + +Expected: FAIL — `--message-format` is not a known option. + +- [ ] **Step 3: Implement** + +`src/cli.cppm`, test subcommand options: + +```cpp + .option(cl::Option("message-format").takes_value().value_name("FMT") + .help("Output format: human (default) | json (NDJSON, one record per test)")) +``` + +`src/cli/cmd_build.cppm` `cmd_test`: + +```cpp + if (auto mf = parsed.value("message-format")) { + if (*mf == "json") to.format = mcpp::build::TestMessageFormat::Json; + else if (*mf != "human") { + mcpp::ui::error(std::format("unknown --message-format '{}' (human|json)", *mf)); + return 2; + } + } +``` + +`src/build/execute.cppm`: + +```cpp +export enum class TestMessageFormat { Human, Json }; + +export struct TestOptions { + std::string filter; // substring match; empty = all + TestMessageFormat format = TestMessageFormat::Human; +}; +``` + +At the top of `run_tests` (right after the manifest-root check): + +```cpp + const bool json = (testOpts.format == TestMessageFormat::Json); + // JSON mode: stdout carries NDJSON only. All ui::status/info lines print + // to stdout, so silence them wholesale; errors already go to stderr. + if (json) mcpp::ui::set_quiet(true); +``` + +Local JSON helpers (file-scope, next to the other run_tests helpers; same escaping shape as `json_escape` in src/cli/cmd_xpkg.cppm:28 — keep both local, they are 15 lines and live on opposite sides of a module boundary): + +```cpp +namespace { +std::string test_json_escape(std::string_view s) { + std::string out; + out.reserve(s.size() + 8); + for (char c : s) { + switch (c) { + case '"': out += "\\\""; break; + case '\\': out += "\\\\"; break; + case '\n': out += "\\n"; break; + case '\r': out += "\\r"; break; + case '\t': out += "\\t"; break; + default: + if (static_cast(c) < 0x20) + out += std::format("\\u{:04x}", static_cast(c)); + else out += c; + } + } + return out; +} +} // namespace +``` + +Emission points: + +1. Package-level (Phase A) failure — before `return 1`: + +```cpp + if (json) + std::println("{{\"error\":\"package\",\"compile_output\":\"{}\"}}", + test_json_escape(a.error().diagnosticOutput)); +``` + +2. No-match (Task 4 guard) — before `return 2`: + +```cpp + if (json) + std::println("{{\"error\":\"no-tests-matched\",\"filter\":\"{}\"}}", + test_json_escape(testOpts.filter)); +``` + +3. Per-test — the run step uses `capture_exec` in JSON mode so output is captured (human mode keeps `run_exec` streaming): + +```cpp + int exitCode; + std::string runOutput; + if (json) { + auto rr = mcpp::platform::process::capture_exec(argv, childEnv); + exitCode = rr.exit_code; + runOutput = std::move(rr.output); + } else { + exitCode = mcpp::platform::process::run_exec(argv, childEnv); + } +``` + +…store `runOutput` in the `TestResult`, and after each result is recorded emit its record immediately (streaming NDJSON, one line per test as it finishes): + +```cpp + if (json) { + auto& r = results.back(); + const char* st = r.status == TestResult::St::Pass ? "pass" + : r.status == TestResult::St::CompileFail ? "compile_fail" + : "run_fail"; + std::string signal = (r.exitCode > 128 && r.exitCode < 128 + 65) + ? std::to_string(r.exitCode - 128) : "null"; + std::println("{{\"test\":\"{}\",\"status\":\"{}\",\"exit_code\":{},\"signal\":{}," + "\"compile_output\":\"{}\",\"run_output\":\"{}\"}}", + test_json_escape(r.name), st, r.exitCode, signal, + test_json_escape(r.compileOutput), test_json_escape(r.runOutput)); + std::fflush(stdout); + } +``` + +In JSON mode suppress the human per-test lines (`... ok` / `... FAIL`) — wrap each `std::println("{} ... ", ...)` in `if (!json)`. + +4. Summary — in the summary section, before returning (both branches): + +```cpp + if (json) + std::println("{{\"summary\":{{\"passed\":{},\"failed\":{},\"elapsed_ms\":{}}}}}", + passed, failed, elapsed.count()); +``` + +And in JSON mode skip the human summary/failures printing and the grouped stderr compile diagnostics (they are already inside the records). + +- [ ] **Step 4: Run the e2e tests** + +```bash +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/121_test_json.sh +MCPP="$PWD/$BIN" bash tests/e2e/119_test_isolation.sh +MCPP="$PWD/$BIN" bash tests/e2e/120_test_filter.sh +``` + +Expected: all `OK`. + +- [ ] **Step 5: Verify compile_commands completeness under filter (design-doc "4th item")** + +```bash +cd $(mktemp -d) +mkdir -p pkg/tests/00-a pkg/tests/01-b && cd pkg +printf '[package]\nname = "cc"\nversion = "0.1.0"\nstandard = "c++23"\n' > mcpp.toml +echo 'int main() { return 0; }' > tests/00-a/0.cpp +echo 'int main() { D2X_YOUR_ANSWER x = 1; return x; }' > tests/01-b/0.cpp +"$MCPP" test 00-a >/dev/null 2>&1 +grep -c '"file"' target/*/*/compile_commands.json +``` + +Expected: **2** entries (both tests present even though one was filtered out and the other doesn't compile). If fewer, fix in this task: compile_commands is written from the full plan in `NinjaBackend::build` (ninja_backend.cppm:1118-1120), which already runs before ninja — investigate what dropped an entry. + +- [ ] **Step 6: Commit** + +```bash +git add src/cli.cppm src/cli/cmd_build.cppm src/build/execute.cppm tests/e2e/121_test_json.sh +git commit -m "feat(test): --message-format json — NDJSON records for tooling (d2x provider et al.)" +``` + +--- + +### Task 6: musl static build + regression sweep + changelog + +The deliverable the d2mcpp work consumes locally: a statically linked `mcpp` with the new capabilities. Static musl also makes the nested-mcpp `LD_LIBRARY_PATH` glibc-poisoning crash structurally impossible for the mcpp binary itself (the design doc's "顺带修复" — the provider-side `unsetenv` workarounds get removed later in the d2mcpp repo, not here). + +**Files:** +- Modify: `CHANGELOG.md` (new Unreleased entries) +- Modify: any `docs/*.md` lines describing `mcpp test` behavior (found via grep below) + +- [ ] **Step 1: Full e2e regression with the glibc dev binary** + +```bash +cd /home/speak/workspace/github/mcpp-community/mcpp +mcpp build && BIN=$(ls -d target/x86_64-linux-gnu/*/bin/mcpp | head -1) +MCPP="$PWD/$BIN" bash tests/e2e/run_all.sh 2>&1 | tail -20 +``` + +Expected: same pass set as `main` (run once on `main` first if a baseline is needed). Investigate any new failure — especially e2e scripts that assert on `mcpp test` output wording. + +- [ ] **Step 2: musl static build** + +```bash +mcpp build --target x86_64-linux-musl +MUSL_BIN=$(ls -d target/x86_64-linux-musl/*/bin/mcpp | head -1) +file "$MUSL_BIN" +``` + +Expected: `statically linked` in the `file` output. + +- [ ] **Step 3: Run the four new e2e scripts against the musl binary** + +```bash +for t in 118_test_subdir_names 119_test_isolation 120_test_filter 121_test_json; do + MCPP="$PWD/$MUSL_BIN" bash tests/e2e/$t.sh || { echo "FAILED: $t"; break; } +done +``` + +Expected: four × `OK`. + +- [ ] **Step 4: Nested LD_LIBRARY_PATH immunity check** + +Simulates the d2x→provider→nested-mcpp chain: a poisoned `LD_LIBRARY_PATH` pointing at the sandbox glibc must not crash the static binary. + +```bash +MUSL_ABS=$(realpath "$MUSL_BIN") +GLIBC_LIB=$(dirname $(ls ~/.mcpp/registry/data/xpkgs/xim-x-glibc/*/lib64/libc.so.6 2>/dev/null | head -1) 2>/dev/null) +cd $(mktemp -d) && mkdir -p pkg/tests && cd pkg +printf '[package]\nname = "nest"\nversion = "0.1.0"\nstandard = "c++23"\n' > mcpp.toml +echo 'int main() { return 0; }' > tests/t.cpp +LD_LIBRARY_PATH="$GLIBC_LIB" "$MUSL_ABS" test +``` + +Expected: `t ... ok`, no segfault. (With a glibc-linked mcpp this environment reproduces the dynamic-linker crash the provider currently works around with `unsetenv`.) + +- [ ] **Step 5: Changelog + docs sync** + +Add to `CHANGELOG.md` under an Unreleased heading (create it if absent, matching the file's existing entry style): + +```markdown +- `mcpp test`: per-test compile isolation — a test that fails to compile is reported + as that test's failure (`FAIL (compile)`); remaining tests still build and run. + Package-level build failures (lib/deps) remain hard errors. +- `mcpp test`: tests are named by their `tests/`-relative path (`00-intro/0`), + so test files in subdirectories no longer collide on stem. +- `mcpp test `: run only tests whose name contains PATTERN. +- `mcpp test --message-format json`: NDJSON output for tooling — one record per + test (status/exit_code/signal/compile_output/run_output), package-error and + summary records. +``` + +Sync user docs: + +```bash +grep -rn "mcpp test" docs/*.md README.md README.zh-CN.md +``` + +Update every hit that describes test naming, failure behavior, or CLI usage to match the new behavior (at minimum the command list blurbs; keep edits to the lines the grep surfaces). + +- [ ] **Step 6: Commit** + +```bash +git add CHANGELOG.md docs/ README.md README.zh-CN.md +git commit -m "docs: changelog + doc sync for mcpp test isolation/filter/json" +``` + +- [ ] **Step 7: Report the musl binary path** + +Print the absolute `$MUSL_BIN` path in the final summary — the d2mcpp adaptation work (next plan) points `.d2x.json`/CI at this binary for local testing. + +--- + +## Out of scope (tracked in the d2mcpp design doc, not this plan) + +- Provider rewrite / `_current` manifest deletion in d2mcpp (separate plan, consumes this binary). +- Removing `unsetenv("LD_LIBRARY_PATH")` workarounds in d2mcpp `runner.cppm` / d2x `platform.cppm` (after the musl binary is adopted end-to-end). +- Per-test cxxflags carriage in mcpp.toml (design doc §7 open item — decide when the d2mcpp migration reaches the two affected exercises). +- Windows/macOS validation of the new test flow. From 4b0998440ea8a92db159e99d2dd3db1eaa13658e Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 21:10:42 +0800 Subject: [PATCH 02/22] feat(test): name tests by tests/-relative path, enabling subdirectories --- src/build/execute.cppm | 10 +++++++--- tests/e2e/118_test_subdir_names.sh | 29 +++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 3 deletions(-) create mode 100755 tests/e2e/118_test_subdir_names.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 4dbe49d..a1aa288 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -703,14 +703,18 @@ export int run_tests(std::span passthrough, } // 2. Synthesize a Target for each test file. - // Name = file stem; collisions → error. + // Name = path relative to tests/, extension dropped, '/' separators — + // so tests/00-a/0.cpp and tests/01-b/0.cpp coexist as '00-a/0' and + // '01-b/0' (stems alone would collide). Flat layouts keep their old + // names ('tests/smoke.cpp' → 'smoke'). std::vector testTargets; std::set seenNames; for (auto& f : testFiles) { - auto name = f.stem().string(); + auto rel = std::filesystem::relative(f, testRoot / "tests"); + auto name = rel.replace_extension("").generic_string(); if (!seenNames.insert(name).second) { mcpp::ui::error(std::format( - "duplicate test name '{}' (two .cpp files share the same stem)", name)); + "duplicate test name '{}' (two test files map to the same name)", name)); return 2; } mcpp::manifest::Target t; diff --git a/tests/e2e/118_test_subdir_names.sh b/tests/e2e/118_test_subdir_names.sh new file mode 100755 index 0000000..8e1fb34 --- /dev/null +++ b/tests/e2e/118_test_subdir_names.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +# requires: +# tests/ subdirectories: same stem in two dirs must coexist, names are relative paths +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests/00-a pkg/tests/01-b +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "subdirs" +version = "0.1.0" +standard = "c++23" +EOF +cat > tests/00-a/0.cpp <<'EOF' +int main() { return 0; } +EOF +cat > tests/01-b/0.cpp <<'EOF' +int main() { return 0; } +EOF + +out=$("$MCPP" test 2>&1) || { echo "mcpp test failed: $out"; exit 1; } +[[ "$out" == *"00-a/0 ... ok"* ]] || { echo "missing path-based name 00-a/0: $out"; exit 1; } +[[ "$out" == *"01-b/0 ... ok"* ]] || { echo "missing path-based name 01-b/0: $out"; exit 1; } +[[ "$out" == *"2 passed"* ]] || { echo "expected 2 passed: $out"; exit 1; } +echo OK From bce3e7d99ef20734dcdd1c69f220ad066361da0a Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 21:11:32 +0800 Subject: [PATCH 03/22] =?UTF-8?q?feat(build):=20BuildOptions::ninjaTargets?= =?UTF-8?q?=20=E2=80=94=20build=20an=20explicit=20subset=20of=20the=20plan?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/build/backend.cppm | 3 +++ src/build/ninja_backend.cppm | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/src/build/backend.cppm b/src/build/backend.cppm index 947a4fb..99b6227 100644 --- a/src/build/backend.cppm +++ b/src/build/backend.cppm @@ -13,6 +13,9 @@ struct BuildOptions { bool verbose = false; bool dryRun = false; std::size_t parallelJobs = 0; + // Explicit ninja goal targets (LinkUnit::output paths, relative to the + // plan's outputDir). Empty = build the full plan (default behavior). + std::vector ninjaTargets; }; struct BuildResult { diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 0f7c665..3778666 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -1195,6 +1195,11 @@ std::expected NinjaBackend::build(const BuildPlan& plan if (opts.parallelJobs) nargv.push_back(std::format("-j{}", opts.parallelJobs)); + // Explicit goal targets: ninja builds only these outputs (and their + // prerequisites). Used by `mcpp test` to isolate per-test compiles. + for (auto& t : opts.ninjaTargets) + nargv.push_back(t); + // Real env pairs for THIS run (the "@env" cache encoding above is only // for the fast path's later re-creation of the same environment). std::vector> nenv; @@ -1214,7 +1219,9 @@ std::expected NinjaBackend::build(const BuildPlan& plan if (ok) { if (opts.verbose && !out.empty()) std::fputs(out.c_str(), stdout); + std::set want(opts.ninjaTargets.begin(), opts.ninjaTargets.end()); for (auto& lu : plan.linkUnits) { + if (!want.empty() && !want.contains(lu.output.generic_string())) continue; r.producedArtifacts.push_back(plan.outputDir / lu.output); for (auto const& alias : lu.runtimeAliases) { r.producedArtifacts.push_back(plan.outputDir / alias); From ed557df5e1f2f2b711c9e2ca5b239f0de2245233 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 21:15:01 +0800 Subject: [PATCH 04/22] =?UTF-8?q?feat(test):=20per-test=20compile=20isolat?= =?UTF-8?q?ion=20=E2=80=94=20a=20broken=20test=20no=20longer=20kills=20the?= =?UTF-8?q?=20run?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/build/execute.cppm | 120 +++++++++++++++++++++++--------- tests/e2e/119_test_isolation.sh | 51 ++++++++++++++ 2 files changed, 140 insertions(+), 31 deletions(-) create mode 100755 tests/e2e/119_test_isolation.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index a1aa288..2ef7530 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -766,41 +766,74 @@ export int run_tests(std::span passthrough, } } - // 5. Build everything. + // 5. Two-phase build. Phase A: package-level artifacts (everything that + // is not a test binary — libs, deps). A failure here is the PACKAGE's + // fault, not any single test's: report it as a build error, never as + // N red tests. Phase B (below): each test is built as its own ninja + // goal, so a compile failure is attributed to exactly that test and + // the rest still build and run. + struct TestResult { + std::string name; + enum class St { Pass, CompileFail, RunFail } status; + int exitCode = 0; + std::string compileOutput; + std::string runOutput; + }; + std::vector results; + auto backend = mcpp::build::make_ninja_backend(); - mcpp::build::BuildOptions opts; - auto buildResult = backend->build(ctx->plan, opts); - if (!buildResult) { - std::fflush(stdout); - mcpp::ui::error(buildResult.error().message); - // Surface the compiler/linker stderr (parity with run_build_plan) — - // otherwise `mcpp test` failures show only "build failed" with no - // diagnostic, which is undebuggable (notably on CI). - if (!buildResult.error().diagnosticOutput.empty()) { - std::fputs(buildResult.error().diagnosticOutput.c_str(), stderr); - if (buildResult.error().diagnosticOutput.back() != '\n') - std::fputc('\n', stderr); + + // Phase A goal set: every shared prerequisite — all package/dep compile + // units EXCEPT the tests' own main TUs, plus any non-test link outputs. + // In test mode the lib link unit is skipped entirely (plan.cppm), so the + // package's module objects are the only place shared breakage can show + // up; building them here is what keeps a broken src/ module a PACKAGE + // error instead of N identical per-test compile failures. + std::set testMains; + for (auto& lu : ctx->plan.linkUnits) + if (lu.kind == mcpp::build::LinkUnit::TestBinary && lu.entryMain) + testMains.insert(*lu.entryMain); + std::vector pkgTargets; + for (auto& cu : ctx->plan.compileUnits) + if (!testMains.contains(cu.source)) + pkgTargets.push_back(cu.object.generic_string()); + for (auto& lu : ctx->plan.linkUnits) + if (lu.kind != mcpp::build::LinkUnit::TestBinary) + pkgTargets.push_back(lu.output.generic_string()); + if (!pkgTargets.empty()) { + mcpp::build::BuildOptions aOpts; + aOpts.ninjaTargets = pkgTargets; + auto a = backend->build(ctx->plan, aOpts); + if (!a) { + std::fflush(stdout); + mcpp::ui::error(a.error().message); + // Surface the compiler/linker stderr (parity with run_build_plan) — + // otherwise `mcpp test` failures show only "build failed" with no + // diagnostic, which is undebuggable (notably on CI). + if (!a.error().diagnosticOutput.empty()) { + std::fputs(a.error().diagnosticOutput.c_str(), stderr); + if (a.error().diagnosticOutput.back() != '\n') + std::fputc('\n', stderr); + } + return 1; } - return 1; - } - // M3.2: populate BMI cache for deps that did NOT hit cache. - for (auto& task : ctx->depsToPopulate) { - auto pr = mcpp::bmi_cache::populate_from(task.key, ctx->outputDir, task.artifacts); - if (!pr) { - mcpp::ui::warning(std::format( - "bmi cache populate failed for {}@{}: {}", - task.key.packageName, task.key.version, pr.error())); + // M3.2: populate BMI cache for deps that did NOT hit cache — deps + // are package-level artifacts, so this belongs right after Phase A. + for (auto& task : ctx->depsToPopulate) { + auto pr = mcpp::bmi_cache::populate_from(task.key, ctx->outputDir, task.artifacts); + if (!pr) { + mcpp::ui::warning(std::format( + "bmi cache populate failed for {}@{}: {}", + task.key.packageName, task.key.version, pr.error())); + } } - } - mcpp::ui::finished("test", buildResult->elapsed); + mcpp::ui::finished("test", a->elapsed); + } - // 6. Run each test binary in sequence; collect pass/fail. + // 6. Phase B: build + run each test in sequence; collect results. auto t0 = std::chrono::steady_clock::now(); - int passed = 0; - int failed = 0; - std::vector failures; auto runtimeEnvKey = mcpp::platform::env::runtime_library_path_key(); auto runtimeEnvValue = mcpp::platform::env::prepend_path_list( @@ -808,6 +841,17 @@ export int run_tests(std::span passthrough, for (auto& lu : ctx->plan.linkUnits) { if (lu.kind != mcpp::build::LinkUnit::TestBinary) continue; + + mcpp::build::BuildOptions bOpts; + bOpts.ninjaTargets = {lu.output.generic_string()}; + auto b = backend->build(ctx->plan, bOpts); + if (!b) { + std::println("{} ... FAIL (compile)", lu.targetName); + results.push_back({lu.targetName, TestResult::St::CompileFail, 0, + b.error().diagnosticOutput, {}}); + continue; + } + auto exe = ctx->outputDir / lu.output; mcpp::ui::status("Running", std::format("bin/{}", lu.targetName)); @@ -839,17 +883,24 @@ export int run_tests(std::span passthrough, if (exitCode == 0) { std::println("{} ... ok", lu.targetName); - ++passed; + results.push_back({lu.targetName, TestResult::St::Pass, 0, {}, {}}); } else { std::println("{} ... FAIL (exit {})", lu.targetName, exitCode); - ++failed; - failures.push_back(lu.targetName); + results.push_back({lu.targetName, TestResult::St::RunFail, exitCode, {}, {}}); } } auto elapsed = std::chrono::duration_cast( std::chrono::steady_clock::now() - t0); // 7. Summary. + int passed = 0; + int failed = 0; + std::vector failures; + for (auto& r : results) { + if (r.status == TestResult::St::Pass) ++passed; + else { ++failed; failures.push_back(r.name); } + } + std::println(""); if (failed == 0) { mcpp::ui::status("test result", @@ -863,6 +914,13 @@ export int run_tests(std::span passthrough, std::println(""); std::println("failures:"); for (auto& n : failures) std::println(" {}", n); + // Compile diagnostics per failed test, on stderr, after the summary — + // grouped so a learner scrolls to exactly their test's errors. + for (auto& r : results) { + if (r.status != TestResult::St::CompileFail || r.compileOutput.empty()) continue; + std::println(stderr, "\n--- {} (compile) ---", r.name); + std::fputs(r.compileOutput.c_str(), stderr); + } return 1; } diff --git a/tests/e2e/119_test_isolation.sh b/tests/e2e/119_test_isolation.sh new file mode 100755 index 0000000..6602c9e --- /dev/null +++ b/tests/e2e/119_test_isolation.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# requires: +# Per-test compile isolation: a non-compiling test fails alone; others build & run. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "isolation" +version = "0.1.0" +standard = "c++23" +EOF +cat > tests/ok.cpp <<'EOF' +int main() { return 0; } +EOF +cat > tests/runfail.cpp <<'EOF' +int main() { return 1; } +EOF +cat > tests/nocompile.cpp <<'EOF' +int main() { D2X_YOUR_ANSWER x = 1; return x; } +EOF + +set +e +out=$("$MCPP" test 2>&1) +code=$? +set -e +[[ $code -eq 1 ]] || { echo "expected exit 1, got $code: $out"; exit 1; } +[[ "$out" == *"ok ... ok"* ]] || { echo "passing test did not run: $out"; exit 1; } +[[ "$out" == *"runfail ... FAIL (exit 1)"* ]] || { echo "runtime failure not reported: $out"; exit 1; } +[[ "$out" == *"nocompile ... FAIL (compile)"* ]] || { echo "compile failure not isolated: $out"; exit 1; } +[[ "$out" == *"1 passed"* && "$out" == *"2 failed"* ]] || { echo "bad summary: $out"; exit 1; } + +# Package-level failure stays a hard error: break a src module, all-red is wrong, +# 'build failed' is right. +mkdir -p src +cat > src/isolation.cppm <<'EOF' +export module isolation; +this is not C++; +EOF +set +e +out2=$("$MCPP" test 2>&1) +code2=$? +set -e +[[ $code2 -ne 0 ]] || { echo "package error must fail: $out2"; exit 1; } +[[ "$out2" != *"nocompile ... FAIL (compile)"* ]] || { echo "package error misattributed to tests: $out2"; exit 1; } +echo OK From 16da2162f139804bafcbe32b4fddacf840b26a53 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 21:16:42 +0800 Subject: [PATCH 05/22] feat(test): positional pattern filters tests by path-based name --- src/build/execute.cppm | 30 +++++++++++++++++++++++++++--- src/cli.cppm | 2 ++ src/cli/cmd_build.cppm | 7 +++++-- tests/e2e/120_test_filter.sh | 32 ++++++++++++++++++++++++++++++++ 4 files changed, 66 insertions(+), 5 deletions(-) create mode 100755 tests/e2e/120_test_filter.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 2ef7530..8081ff9 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -672,10 +672,15 @@ export int build_run_target(const std::optional& targetName, return mcpp::platform::process::run_exec(argv, childEnv) == 0 ? 0 : 1; } +export struct TestOptions { + std::string filter; // substring match on the path-based test name; empty = all +}; + // `mcpp test` driver: discover tests/**/*.cpp, synthesize targets, build // with dev-deps, run each test binary, summarize. export int run_tests(std::span passthrough, - BuildOverrides overrides = {}) { + BuildOverrides overrides = {}, + TestOptions testOpts = {}) { auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (!root) { mcpp::ui::error("no mcpp.toml found in current directory or any parent"); @@ -732,6 +737,25 @@ export int run_tests(std::span passthrough, std::move(overrides)); if (!ctx) { mcpp::ui::error(ctx.error()); return 2; } + // Filter guard. The filter selects at the build/run stage ONLY — the plan + // above always contains every test, so build.ninja and + // compile_commands.json stay complete (clangd depends on the latter; a + // filtered run must not clobber it down to one entry). + auto filter_match = [&](const mcpp::build::LinkUnit& lu) { + return lu.kind == mcpp::build::LinkUnit::TestBinary + && (testOpts.filter.empty() + || lu.targetName.find(testOpts.filter) != std::string::npos); + }; + if (!testOpts.filter.empty()) { + bool any = false; + for (auto& lu : ctx->plan.linkUnits) + if (filter_match(lu)) { any = true; break; } + if (!any) { + mcpp::ui::error(std::format("no tests match '{}'", testOpts.filter)); + return 2; + } + } + // 4. "Compiling test_X (test)" lines for the test binaries. std::set cachedNames; for (auto& label : ctx->cachedDepLabels) { @@ -760,7 +784,7 @@ export int run_tests(std::span passthrough, } // List test binaries. for (auto& lu : ctx->plan.linkUnits) { - if (lu.kind == mcpp::build::LinkUnit::TestBinary) { + if (filter_match(lu)) { mcpp::ui::status("Compiling", std::format("{} (test)", lu.targetName)); } @@ -840,7 +864,7 @@ export int run_tests(std::span passthrough, runtimeEnvKey, ctx->plan.runtimeLibraryDirs); for (auto& lu : ctx->plan.linkUnits) { - if (lu.kind != mcpp::build::LinkUnit::TestBinary) continue; + if (!filter_match(lu)) continue; mcpp::build::BuildOptions bOpts; bOpts.ninjaTargets = {lu.output.generic_string()}; diff --git a/src/cli.cppm b/src/cli.cppm index f8ef694..26e9f0d 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -251,6 +251,8 @@ int run(int argc, char** argv) { }))) .subcommand(cl::App("test") .description("Build + run all tests/**/*.cpp (after `--`, args go to each test binary)") + .arg(cl::Arg("pattern") + .help("Run only tests whose name contains PATTERN (optional)")) .option(cl::Option("profile").takes_value().value_name("NAME") .help("Build profile for the test build: release (default) | dev | dist | <[profile.*] name>")) .option(cl::Option("features").takes_value().value_name("LIST") diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 646e4ea..cc34155 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -125,6 +125,9 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, ov.strict = parsed.is_flag_set("strict"); if (auto p = parsed.value("package")) ov.package_filter = *p; + mcpp::build::TestOptions to; + if (parsed.positional_count() > 0) to.filter = parsed.positional(0); + // Workspace fan-out: test every member through run_tests (which scopes its // discovery to the member). Continue-on-failure + per-member summary so one // red member never hides the rest. @@ -136,7 +139,7 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, mcpp::build::BuildOverrides mo = ov; mo.package_filter = mp; mcpp::ui::status("Workspace", std::format("testing member '{}'", mp)); - int r = mcpp::build::run_tests(passthrough, mo); + int r = mcpp::build::run_tests(passthrough, mo, to); if (r != 0) { rc = r; failed.push_back(mp); } } if (failed.empty()) @@ -149,7 +152,7 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, return s; }())); return rc; } - return mcpp::build::run_tests(passthrough, ov); + return mcpp::build::run_tests(passthrough, ov, to); } export int cmd_clean(const mcpplibs::cmdline::ParsedArgs& parsed) { diff --git a/tests/e2e/120_test_filter.sh b/tests/e2e/120_test_filter.sh new file mode 100755 index 0000000..53d999a --- /dev/null +++ b/tests/e2e/120_test_filter.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +# requires: +# `mcpp test `: substring filter on path-based test names +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests/00-a pkg/tests/01-b +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "filter" +version = "0.1.0" +standard = "c++23" +EOF +echo 'int main() { return 0; }' > tests/00-a/0.cpp +echo 'int main() { return 0; }' > tests/01-b/0.cpp + +out=$("$MCPP" test 00-a 2>&1) || { echo "filtered run failed: $out"; exit 1; } +[[ "$out" == *"00-a/0 ... ok"* ]] || { echo "filtered test did not run: $out"; exit 1; } +[[ "$out" != *"01-b/0"* ]] || { echo "filter leaked other tests: $out"; exit 1; } +[[ "$out" == *"1 passed"* ]] || { echo "bad summary: $out"; exit 1; } + +set +e +out2=$("$MCPP" test does-not-exist 2>&1) +code2=$? +set -e +[[ $code2 -eq 2 ]] || { echo "no-match should exit 2, got $code2: $out2"; exit 1; } +[[ "$out2" == *"no tests match"* ]] || { echo "missing no-match error: $out2"; exit 1; } +echo OK From f2c78308c4006121241dab52e4b32fc38e41dc92 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 21:19:27 +0800 Subject: [PATCH 06/22] =?UTF-8?q?feat(test):=20--message-format=20json=20?= =?UTF-8?q?=E2=80=94=20NDJSON=20records=20for=20tooling=20(d2x=20provider?= =?UTF-8?q?=20et=20al.)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/build/execute.cppm | 88 +++++++++++++++++++++++++++++++++++--- src/cli.cppm | 2 + src/cli/cmd_build.cppm | 7 +++ tests/e2e/121_test_json.sh | 50 ++++++++++++++++++++++ 4 files changed, 140 insertions(+), 7 deletions(-) create mode 100755 tests/e2e/121_test_json.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 8081ff9..3e6707d 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -672,15 +672,45 @@ export int build_run_target(const std::optional& targetName, return mcpp::platform::process::run_exec(argv, childEnv) == 0 ? 0 : 1; } +export enum class TestMessageFormat { Human, Json }; + export struct TestOptions { - std::string filter; // substring match on the path-based test name; empty = all + std::string filter; // substring match on the path-based test name; empty = all + TestMessageFormat format = TestMessageFormat::Human; }; +// Minimal JSON string escaping for the --message-format json records. Same +// shape as json_escape in cmd_xpkg.cppm — kept local (15 lines) rather than +// shared across the cli/build module boundary. +static std::string test_json_escape(std::string_view s) { + std::string out; + out.reserve(s.size() + 8); + for (char c : s) { + switch (c) { + case '"': out += "\\\""; break; + case '\\': out += "\\\\"; break; + case '\n': out += "\\n"; break; + case '\r': out += "\\r"; break; + case '\t': out += "\\t"; break; + default: + if (static_cast(c) < 0x20) + out += std::format("\\u{:04x}", static_cast(c)); + else out += c; + } + } + return out; +} + // `mcpp test` driver: discover tests/**/*.cpp, synthesize targets, build // with dev-deps, run each test binary, summarize. export int run_tests(std::span passthrough, BuildOverrides overrides = {}, TestOptions testOpts = {}) { + const bool json = (testOpts.format == TestMessageFormat::Json); + // JSON mode: stdout carries NDJSON only. All ui::status/info lines print + // to stdout, so silence them wholesale; errors already go to stderr. + if (json) mcpp::ui::set_quiet(true); + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (!root) { mcpp::ui::error("no mcpp.toml found in current directory or any parent"); @@ -751,6 +781,9 @@ export int run_tests(std::span passthrough, for (auto& lu : ctx->plan.linkUnits) if (filter_match(lu)) { any = true; break; } if (!any) { + if (json) + std::println("{{\"error\":\"no-tests-matched\",\"filter\":\"{}\"}}", + test_json_escape(testOpts.filter)); mcpp::ui::error(std::format("no tests match '{}'", testOpts.filter)); return 2; } @@ -805,6 +838,23 @@ export int run_tests(std::span passthrough, }; std::vector results; + // Streaming NDJSON: one record per test, emitted as it finishes — a + // consumer (e.g. the d2x provider) sees progress live, and a crash + // mid-run still leaves the completed records on stdout. + auto emit_json = [&](const TestResult& r) { + if (!json) return; + const char* st = r.status == TestResult::St::Pass ? "pass" + : r.status == TestResult::St::CompileFail ? "compile_fail" + : "run_fail"; + std::string signal = (r.exitCode > 128 && r.exitCode < 128 + 65) + ? std::to_string(r.exitCode - 128) : "null"; + std::println("{{\"test\":\"{}\",\"status\":\"{}\",\"exit_code\":{},\"signal\":{}," + "\"compile_output\":\"{}\",\"run_output\":\"{}\"}}", + test_json_escape(r.name), st, r.exitCode, signal, + test_json_escape(r.compileOutput), test_json_escape(r.runOutput)); + std::fflush(stdout); + }; + auto backend = mcpp::build::make_ninja_backend(); // Phase A goal set: every shared prerequisite — all package/dep compile @@ -830,6 +880,9 @@ export int run_tests(std::span passthrough, auto a = backend->build(ctx->plan, aOpts); if (!a) { std::fflush(stdout); + if (json) + std::println("{{\"error\":\"package\",\"compile_output\":\"{}\"}}", + test_json_escape(a.error().diagnosticOutput)); mcpp::ui::error(a.error().message); // Surface the compiler/linker stderr (parity with run_build_plan) — // otherwise `mcpp test` failures show only "build failed" with no @@ -870,9 +923,10 @@ export int run_tests(std::span passthrough, bOpts.ninjaTargets = {lu.output.generic_string()}; auto b = backend->build(ctx->plan, bOpts); if (!b) { - std::println("{} ... FAIL (compile)", lu.targetName); + if (!json) std::println("{} ... FAIL (compile)", lu.targetName); results.push_back({lu.targetName, TestResult::St::CompileFail, 0, b.error().diagnosticOutput, {}}); + emit_json(results.back()); continue; } @@ -903,15 +957,28 @@ export int run_tests(std::span passthrough, } } - int exitCode = mcpp::platform::process::run_exec(argv, childEnv); + // JSON mode captures the test's combined stdout+stderr into the + // record; human mode streams it to the terminal as before. + int exitCode; + std::string runOutput; + if (json) { + auto rr = mcpp::platform::process::capture_exec(argv, childEnv); + exitCode = rr.exit_code; + runOutput = std::move(rr.output); + } else { + exitCode = mcpp::platform::process::run_exec(argv, childEnv); + } if (exitCode == 0) { - std::println("{} ... ok", lu.targetName); - results.push_back({lu.targetName, TestResult::St::Pass, 0, {}, {}}); + if (!json) std::println("{} ... ok", lu.targetName); + results.push_back({lu.targetName, TestResult::St::Pass, 0, {}, + std::move(runOutput)}); } else { - std::println("{} ... FAIL (exit {})", lu.targetName, exitCode); - results.push_back({lu.targetName, TestResult::St::RunFail, exitCode, {}, {}}); + if (!json) std::println("{} ... FAIL (exit {})", lu.targetName, exitCode); + results.push_back({lu.targetName, TestResult::St::RunFail, exitCode, {}, + std::move(runOutput)}); } + emit_json(results.back()); } auto elapsed = std::chrono::duration_cast( std::chrono::steady_clock::now() - t0); @@ -925,6 +992,13 @@ export int run_tests(std::span passthrough, else { ++failed; failures.push_back(r.name); } } + if (json) { + std::println("{{\"summary\":{{\"passed\":{},\"failed\":{},\"elapsed_ms\":{}}}}}", + passed, failed, elapsed.count()); + std::fflush(stdout); + return failed == 0 ? 0 : 1; + } + std::println(""); if (failed == 0) { mcpp::ui::status("test result", diff --git a/src/cli.cppm b/src/cli.cppm index 26e9f0d..654e14e 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -253,6 +253,8 @@ int run(int argc, char** argv) { .description("Build + run all tests/**/*.cpp (after `--`, args go to each test binary)") .arg(cl::Arg("pattern") .help("Run only tests whose name contains PATTERN (optional)")) + .option(cl::Option("message-format").takes_value().value_name("FMT") + .help("Output format: human (default) | json (NDJSON, one record per test)")) .option(cl::Option("profile").takes_value().value_name("NAME") .help("Build profile for the test build: release (default) | dev | dist | <[profile.*] name>")) .option(cl::Option("features").takes_value().value_name("LIST") diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index cc34155..b264398 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -127,6 +127,13 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, mcpp::build::TestOptions to; if (parsed.positional_count() > 0) to.filter = parsed.positional(0); + if (auto mf = parsed.value("message-format")) { + if (*mf == "json") to.format = mcpp::build::TestMessageFormat::Json; + else if (*mf != "human") { + mcpp::ui::error(std::format("unknown --message-format '{}' (human|json)", *mf)); + return 2; + } + } // Workspace fan-out: test every member through run_tests (which scopes its // discovery to the member). Continue-on-failure + per-member summary so one diff --git a/tests/e2e/121_test_json.sh b/tests/e2e/121_test_json.sh new file mode 100755 index 0000000..186ba6f --- /dev/null +++ b/tests/e2e/121_test_json.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# requires: +# --message-format json: NDJSON per test, package error record, pure stdout +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "jsonout" +version = "0.1.0" +standard = "c++23" +EOF +cat > tests/ok.cpp <<'EOF' +#include +int main() { std::puts("hello from ok"); return 0; } +EOF +echo 'int main() { return 1; }' > tests/runfail.cpp +echo 'int main() { D2X_YOUR_ANSWER x = 1; return x; }' > tests/nocompile.cpp + +set +e +out=$("$MCPP" test --message-format json 2>/dev/null) +code=$? +set -e +[[ $code -eq 1 ]] || { echo "expected exit 1, got $code"; exit 1; } + +# stdout must be pure NDJSON: every line parses as a JSON object. +while IFS= read -r line; do + [[ -z "$line" ]] && continue + [[ "$line" == "{"* ]] || { echo "non-JSON line on stdout: $line"; exit 1; } +done <<< "$out" + +echo "$out" | grep -q '"test":"ok","status":"pass"' || { echo "missing pass record"; exit 1; } +echo "$out" | grep -q '"test":"runfail","status":"run_fail"' || { echo "missing run_fail record"; exit 1; } +echo "$out" | grep -q '"exit_code":1' || { echo "missing exit_code"; exit 1; } +echo "$out" | grep -q '"test":"nocompile","status":"compile_fail"' || { echo "missing compile_fail record"; exit 1; } +echo "$out" | grep -q 'D2X_YOUR_ANSWER' || { echo "compile_output missing diagnostics"; exit 1; } +echo "$out" | grep -q 'hello from ok' || { echo "run_output not captured"; exit 1; } +echo "$out" | grep -q '"summary":{"passed":1,"failed":2' || { echo "missing summary"; exit 1; } + +# Invalid format value → usage error +set +e +"$MCPP" test --message-format yaml >/dev/null 2>&1 +[[ $? -eq 2 ]] || { echo "invalid format should exit 2"; exit 1; } +set -e +echo OK From c1bf311c463bd1dc13b010e76cf957960de25fb7 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 21:40:18 +0800 Subject: [PATCH 07/22] =?UTF-8?q?fix(process):=20strip=20private-glibc=20e?= =?UTF-8?q?ntries=20from=20inherited=20loader=20paths=20=E2=80=94=20nested?= =?UTF-8?q?=20mcpp=20no=20longer=20segfaults=20its=20tools?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/platform/process.cppm | 35 ++++++++++++++++++++++++- tests/e2e/122_nested_ld_library_path.sh | 33 +++++++++++++++++++++++ 2 files changed, 67 insertions(+), 1 deletion(-) create mode 100755 tests/e2e/122_nested_ld_library_path.sh diff --git a/src/platform/process.cppm b/src/platform/process.cppm index 01a33e9..8c0cb21 100644 --- a/src/platform/process.cppm +++ b/src/platform/process.cppm @@ -153,6 +153,32 @@ char** host_environ() { #endif } +// An outer `mcpp run`/`mcpp test` points LD_LIBRARY_PATH at mcpp's private +// glibc payload so ITS child (a sandbox-linked user binary) can load. When +// that child spawns mcpp again (e.g. a course provider driving `mcpp test`), +// the same value would flow on into the inner mcpp's own children — and the +// sandbox ninja/gcc (host-glibc binaries) then resolve a MISMATCHED libc and +// segfault inside the dynamic linker before main (trace signature: a bare +// `__vdso_time` line). Strip exactly the private-glibc payload entries from +// inherited loader paths: user-supplied entries survive, and an `extra` +// override (the correct per-child value) always wins over the inherited var. +std::string strip_private_glibc(std::string_view paths) { + std::string cleaned; + std::size_t start = 0; + while (start <= paths.size()) { + auto end = paths.find(':', start); + if (end == std::string_view::npos) end = paths.size(); + auto item = paths.substr(start, end - start); + if (!item.empty() && item.find("/xim-x-glibc/") == std::string_view::npos) { + if (!cleaned.empty()) cleaned += ':'; + cleaned += item; + } + if (end == paths.size()) break; + start = end + 1; + } + return cleaned; +} + // Build a child environment block = the current environ with `extra` overrides // applied. Returned vector owns the strings; the caller derives a NUL-terminated // char* array from it. Built in the PARENT so the child env never requires a @@ -167,7 +193,14 @@ std::vector merged_environ( std::string_view entry(*e); auto eq = entry.find('='); std::string key(eq == std::string_view::npos ? entry : entry.substr(0, eq)); - if (!overridden.contains(key)) out.emplace_back(entry); + if (overridden.contains(key)) continue; + if (eq != std::string_view::npos + && (key == "LD_LIBRARY_PATH" || key == "DYLD_LIBRARY_PATH")) { + auto cleaned = strip_private_glibc(entry.substr(eq + 1)); + if (!cleaned.empty()) out.push_back(key + "=" + cleaned); + continue; // nothing legitimate left → drop the var entirely + } + out.emplace_back(entry); } return out; } diff --git a/tests/e2e/122_nested_ld_library_path.sh b/tests/e2e/122_nested_ld_library_path.sh new file mode 100755 index 0000000..60c7f4a --- /dev/null +++ b/tests/e2e/122_nested_ld_library_path.sh @@ -0,0 +1,33 @@ +#!/usr/bin/env bash +# requires: +# Inherited LD_LIBRARY_PATH pointing at mcpp's private glibc payload must not +# crash mcpp's own children (ninja/gcc). Reproduces the nested-mcpp chain: +# outer `mcpp run` poisons the env for its child; the child spawns mcpp again; +# the inner mcpp's tools then load a mismatched libc and segfault in the +# dynamic linker (trace signature: bare `__vdso_time` line). +set -e + +GLIBC_LIB=$(ls -d "$HOME"/.mcpp/registry/data/xpkgs/xim-x-glibc/*/lib64 2>/dev/null | head -1) +if [[ -z "$GLIBC_LIB" ]]; then + echo "SKIP: no private glibc payload installed" + exit 0 +fi + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "nest" +version = "0.1.0" +standard = "c++23" +EOF +echo 'int main() { return 0; }' > tests/t.cpp + +out=$(LD_LIBRARY_PATH="$GLIBC_LIB" "$MCPP" test 2>&1) \ + || { echo "poisoned run failed: $out"; exit 1; } +[[ "$out" == *"t ... ok"* ]] || { echo "test did not pass under poison: $out"; exit 1; } +echo OK From aaccec9165f41c64f115ecff52922e296655208a Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 21:41:08 +0800 Subject: [PATCH 08/22] docs: changelog + doc sync for mcpp test isolation/filter/json + loader-path fix --- CHANGELOG.md | 21 +++++++++++++++++++++ README.md | 2 +- README.zh-CN.md | 2 +- docs/00-getting-started.md | 1 + docs/05-mcpp-toml.md | 2 +- 5 files changed, 25 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aff3619..403616d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,27 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [未发布] + +> `mcpp test` 练习级增强批次:逐测试编译隔离、子目录路径命名、过滤器、JSON 输出——四项均为通用能力(cargo/ctest 同形),首个下游消费者是 d2mcpp「练习即测试」重设计(见 d2mcpp 仓 `.agents/docs/2026-07-23-exercises-as-tests-design.md` §4,验收标准即出自该文档)。实施计划见 `.agents/docs/2026-07-23-test-isolation-json-plan.md`。 + +### 新增 + +- **`mcpp test` 逐测试编译隔离**:两阶段构建——Phase A 先建包级共享前置(全部非测试 main 的编译单元 + 非测试 link 产物),失败是**包级错误**(报 build failed,绝不渲染成 N 个红测试);Phase B 每个测试作为独立 ninja goal 构建,编译失败只标记**该测试** `FAIL (compile)`(诊断按测试分组输出到 stderr),其余照常编译运行。此前一个编译不过的测试会让整轮 `error: build failed` 零执行。 +- **测试按 `tests/` 相对路径命名**:`tests/00-a/0.cpp` → `00-a/0`,子目录下同名 stem 不再冲突(此前直接 `duplicate test name` 硬错);平铺布局名字不变。 +- **`mcpp test `**:按路径名子串过滤要构建/运行的测试。过滤只作用于构建/运行阶段——计划始终含全部测试,`compile_commands.json` 保持完整(clangd 依赖)。无匹配时报错退出码 2。 +- **`mcpp test --message-format json`**:NDJSON 输出,逐测试一条记录流式发射(`test`/`status`=`pass|compile_fail|run_fail`/`exit_code`/`signal`/`compile_output`/`run_output`),包级失败单独 `{"error":"package",...}` 记录,末行 `{"summary":...}`;stdout 纯协议流,人读输出全部静默。供 CI/IDE/d2x Provider 消费。 + +### 修复 + +- **嵌套 mcpp 的 `LD_LIBRARY_PATH` 段错误**:外层 `mcpp run` 为其子进程指向私有 glibc payload 的 loader 路径,子进程再 spawn mcpp 时(如课程 Provider 驱动 `mcpp test`),毒化值继续流入内层 mcpp 的工具子进程——sandbox ninja/gcc 加载错配 libc 后在动态链接器里段错误(残片签名 `__vdso_time`)。现在 `merged_environ` 从继承的 `LD_LIBRARY_PATH`/`DYLD_LIBRARY_PATH` 里**只剥离** `xim-x-glibc` payload 条目:用户自己的条目保留,per-child 显式 override 一如既往优先。下游(d2x `platform.cppm`、d2mcpp `runner.cppm`)的 `unsetenv` workaround 可随本版删除。 + +### 备注 + +- e2e 新增 118(子目录命名)/119(隔离与包级归因)/120(过滤器)/121(JSON)/122(嵌套 loader 路径免疫)。 +- `BuildOptions::ninjaTargets`(构建计划子集)为支撑性通用接口,空 = 原行为。 +- musl 静态构建(`--target x86_64-linux-musl`)已验证:五个新 e2e 全绿;静态 mcpp 本体对 loader 毒化免疫,与上述修复共同覆盖嵌套链路的两端。 + ## [0.0.103] — 2026-07-22 > #267/#269:索引侧健壮化一批——mcpplibs 索引 URL 组织迁移(不再依赖 GitHub 重定向)+ 采纳 xlings 0.4.68 per-repo artifact 来源(索引同步不再硬依赖可用的 git)。设计见 `.agents/docs/2026-07-22-issue267-269-index-artifact-and-org-migration-design.md`。 diff --git a/README.md b/README.md index 27c570d..16d64c9 100644 --- a/README.md +++ b/README.md @@ -221,7 +221,7 @@ import mcpplibs.cmdline; - `mcpp new` — create a modular project; `--template [@ver][:]` uses a **library-provided template** (e.g. `--template imgui`); `--list-templates ` lists them - `mcpp run [-- args]` — build and run -- `mcpp test [-- args]` — auto-discover and run tests +- `mcpp test [pattern] [-- args]` — auto-discover and run tests (optionally filtered by name; `--message-format json` for tooling) - `mcpp search` — search package indices - `mcpp add / remove / update` — dependency management - `mcpp explain E0001` — detailed error-code explanations diff --git a/README.zh-CN.md b/README.zh-CN.md index 95d36a4..19d78d0 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -221,7 +221,7 @@ import mcpplibs.cmdline; - `mcpp new` — 创建模块化项目;`--template [@ver][:]` 使用**库自带模板**(如 `--template imgui`),`--list-templates ` 列举 - `mcpp run [-- args]` — 构建并运行 -- `mcpp test [-- args]` — 自动发现并运行测试 +- `mcpp test [pattern] [-- args]` — 自动发现并运行测试(可按名字过滤;`--message-format json` 供工具消费) - `mcpp search` — 搜索包索引 - `mcpp add / remove / update` — 依赖管理 - `mcpp explain E0001` — 错误码详细解释 diff --git a/docs/00-getting-started.md b/docs/00-getting-started.md index 98b145b..e23f995 100644 --- a/docs/00-getting-started.md +++ b/docs/00-getting-started.md @@ -83,6 +83,7 @@ showing progress and speed along the way. Once downloaded, all mcpp projects sha mcpp build # incremental build mcpp clean # clean target/ mcpp test # compile and run tests/**/*.cpp (gtest style) +mcpp test # only tests whose name contains ``` ## Adding Dependencies diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 7a0caf2..4338c74 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -328,7 +328,7 @@ qux = ">=1.0, <2.0" # Range combination gtest = "1.15.2" ``` -`mcpp build` ignores these; `mcpp test` resolves and uses them. `mcpp test` automatically discovers `tests/**/*.cpp` and compiles them into test binaries. +`mcpp build` ignores these; `mcpp test` resolves and uses them. `mcpp test` automatically discovers `tests/**/*.cpp` and compiles them into test binaries. Tests are named by their `tests/`-relative path (`tests/00-a/0.cpp` → `00-a/0`), each test compiles in isolation (a broken test fails alone; package/dep breakage is reported as a build error instead), and `mcpp test ` / `--message-format json` filter and machine-format the run. ### 2.7 `[toolchain]` — Toolchain Configuration From aa30a3e6decaf4b0adeeb6ae8f5a7090ba6855ed Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 22:36:31 +0800 Subject: [PATCH 09/22] feat(test): [build].flags globs cover test TUs; dead-glob warning now checks the disk, not just scanned sources --- src/build/execute.cppm | 23 ++++++++++++++++ src/modgraph/scanner.cppm | 9 +++++++ tests/e2e/123_test_glob_flags.sh | 46 ++++++++++++++++++++++++++++++++ 3 files changed, 78 insertions(+) create mode 100755 tests/e2e/123_test_glob_flags.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 3e6707d..b40fdd3 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -737,6 +737,23 @@ export int run_tests(std::span passthrough, return 0; } + // [build].flags globs also cover tests: a glob names files — whether they + // are scanned sources or test TUs is orthogonal. Matched entries ride the + // per-target flag channel (issue #131) on the synthesized test target. + // (Feature-folded entries are prepare-time state; tests take the base + // [build].flags — sufficient for per-test compile options.) + struct TestGlobFlags { + mcpp::manifest::GlobFlags gf; + std::set files; + }; + std::vector testGlobFlags; + if (auto mm = mcpp::manifest::load(testRoot / "mcpp.toml")) { + for (auto const& gf : mm->buildConfig.globFlags) { + auto hits = mcpp::modgraph::expand_glob(testRoot, gf.glob); + testGlobFlags.push_back({gf, {hits.begin(), hits.end()}}); + } + } + // 2. Synthesize a Target for each test file. // Name = path relative to tests/, extension dropped, '/' separators — // so tests/00-a/0.cpp and tests/01-b/0.cpp coexist as '00-a/0' and @@ -757,6 +774,12 @@ export int run_tests(std::span passthrough, t.kind = mcpp::manifest::Target::TestBinary; // Relative to the member/package root prepare_build will operate on. t.main = std::filesystem::relative(f, testRoot).string(); + for (auto const& tgf : testGlobFlags) { + if (!tgf.files.contains(f)) continue; + for (auto const& d : tgf.gf.defines) t.defines.push_back(d); + for (auto const& fl : tgf.gf.cflags) t.cflags.push_back(fl); + for (auto const& fl : tgf.gf.cxxflags) t.cxxflags.push_back(fl); + } testTargets.push_back(std::move(t)); } diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 11473d7..1dfafef 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -932,6 +932,15 @@ void scan_one_into(ScanResult& result, } for (std::size_t i = 0; i < globFlagHits.size(); ++i) { if (globFlagHits[i] == 0) { + // Zero scanned-source hits is not yet a dead glob: the entry may + // target files that are real but not scanned sources — notably + // tests/ TUs, which `mcpp test` flag-matches itself. Only a glob + // that matches NOTHING on disk is a typo worth warning about. + bool onDisk = false; + for (auto const& branch : globFlagBranches[i]) { + if (!expand_glob(root, branch).empty()) { onDisk = true; break; } + } + if (onDisk) continue; // #253: a feature-folded entry only exists when its feature is // active, so a zero hit here is a REAL dead glob either way — // name the owning feature so the author knows which table to fix. diff --git a/tests/e2e/123_test_glob_flags.sh b/tests/e2e/123_test_glob_flags.sh new file mode 100755 index 0000000..5a25f1b --- /dev/null +++ b/tests/e2e/123_test_glob_flags.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# requires: +# [build].flags globs also cover tests/ TUs (via the per-target flag channel), +# and a glob that matches real files on disk (just not scanned sources) does +# NOT warn "matched no source file". +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "globtest" +version = "0.1.0" +standard = "c++23" + +[build] +flags = [ + { glob = "tests/tagged*.cpp", cxxflags = ["-DTAG=7"] }, + { glob = "tests/never/**", cxxflags = ["-DNOPE"] }, +] +EOF +cat > tests/tagged.cpp <<'EOF' +#ifndef TAG +#error "per-glob cxxflags did not reach the test TU" +#endif +int main() { return TAG == 7 ? 0 : 1; } +EOF +cat > tests/plain.cpp <<'EOF' +#ifdef TAG +#error "per-glob cxxflags leaked into an unmatched test TU" +#endif +int main() { return 0; } +EOF + +out=$("$MCPP" test 2>&1) || { echo "mcpp test failed: $out"; exit 1; } +[[ "$out" == *"tagged ... ok"* ]] || { echo "tagged test failed: $out"; exit 1; } +[[ "$out" == *"plain ... ok"* ]] || { echo "plain test failed: $out"; exit 1; } +[[ "$out" != *"'tests/tagged*.cpp' matched no source file"* ]] \ + || { echo "spurious dead-glob warning for a tests/ glob: $out"; exit 1; } +[[ "$out" == *"'tests/never/**' matched no source file"* ]] \ + || { echo "genuinely dead glob must still warn: $out"; exit 1; } +echo OK From 6b17b895af54215a76617bfdc0f714f8ce93eb85 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 22:56:22 +0800 Subject: [PATCH 10/22] docs: changelog for test glob-flags coverage + downstream d2mcpp adoption note --- CHANGELOG.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 403616d..c7b93bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ - **测试按 `tests/` 相对路径命名**:`tests/00-a/0.cpp` → `00-a/0`,子目录下同名 stem 不再冲突(此前直接 `duplicate test name` 硬错);平铺布局名字不变。 - **`mcpp test `**:按路径名子串过滤要构建/运行的测试。过滤只作用于构建/运行阶段——计划始终含全部测试,`compile_commands.json` 保持完整(clangd 依赖)。无匹配时报错退出码 2。 - **`mcpp test --message-format json`**:NDJSON 输出,逐测试一条记录流式发射(`test`/`status`=`pass|compile_fail|run_fail`/`exit_code`/`signal`/`compile_output`/`run_output`),包级失败单独 `{"error":"package",...}` 记录,末行 `{"summary":...}`;stdout 纯协议流,人读输出全部静默。供 CI/IDE/d2x Provider 消费。 +- **`[build].flags` glob 覆盖测试 TU**:glob 指名文件,是 source 还是 test 是正交的——匹配的条目经既有 per-target flag 通道(#131)挂到合成测试 target 上,per-test 编译选项从此有 toml 承载。配套:死 glob 警告改为「磁盘上无文件匹配」才触发(此前只数被扫描的 source,指向 tests/ 的合法 glob 会被误警)。 ### 修复 @@ -20,7 +21,8 @@ ### 备注 -- e2e 新增 118(子目录命名)/119(隔离与包级归因)/120(过滤器)/121(JSON)/122(嵌套 loader 路径免疫)。 +- e2e 新增 118(子目录命名)/119(隔离与包级归因)/120(过滤器)/121(JSON)/122(嵌套 loader 路径免疫)/123(测试 TU 吃 glob flags + 死 glob 判定)。 +- 首个下游消费者已完成端到端接入:d2mcpp「练习即测试」迁移(104 练习 zh/en 52/52 全绿)+ d2x checker 闯关链路,全程使用本分支 musl 静态二进制。 - `BuildOptions::ninjaTargets`(构建计划子集)为支撑性通用接口,空 = 原行为。 - musl 静态构建(`--target x86_64-linux-musl`)已验证:五个新 e2e 全绿;静态 mcpp 本体对 loader 毒化免疫,与上述修复共同覆盖嵌套链路的两端。 From f4ba3bff399fd90fb7a88b588a4d772c61ab4f9c Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Thu, 23 Jul 2026 23:09:10 +0800 Subject: [PATCH 11/22] fix(test): interleave per-test Compiling/result/diagnostics; drop misleading mid-run Finished banner --- src/build/execute.cppm | 37 ++++++++++++++++++++++--------------- 1 file changed, 22 insertions(+), 15 deletions(-) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index b40fdd3..70b2b6b 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -839,12 +839,9 @@ export int run_tests(std::span passthrough, std::format("{} {} (dev)", name, ver)); } // List test binaries. - for (auto& lu : ctx->plan.linkUnits) { - if (filter_match(lu)) { - mcpp::ui::status("Compiling", - std::format("{} (test)", lu.targetName)); - } - } + // (Per-test "Compiling" lines print in Phase B, interleaved with each + // test's own result — announcing them all up front separated the three + // pieces of one test's story across the whole output.) // 5. Two-phase build. Phase A: package-level artifacts (everything that // is not a test binary — libs, deps). A failure here is the PACKAGE's @@ -929,7 +926,9 @@ export int run_tests(std::span passthrough, } } - mcpp::ui::finished("test", a->elapsed); + // No "Finished test" line here: Phase A only built the shared + // prerequisites. Printing a success banner right before per-test + // failures read as a contradiction; the final summary carries timing. } // 6. Phase B: build + run each test in sequence; collect results. @@ -942,11 +941,24 @@ export int run_tests(std::span passthrough, for (auto& lu : ctx->plan.linkUnits) { if (!filter_match(lu)) continue; + mcpp::ui::status("Compiling", std::format("{} (test)", lu.targetName)); + mcpp::build::BuildOptions bOpts; bOpts.ninjaTargets = {lu.output.generic_string()}; auto b = backend->build(ctx->plan, bOpts); if (!b) { - if (!json) std::println("{} ... FAIL (compile)", lu.targetName); + if (!json) { + // The test's own diagnostics, right under its FAIL line — a + // reader fixes one test with one contiguous block of output. + std::println("{} ... FAIL (compile)", lu.targetName); + std::fflush(stdout); + if (!b.error().diagnosticOutput.empty()) { + std::fputs(b.error().diagnosticOutput.c_str(), stderr); + if (b.error().diagnosticOutput.back() != '\n') + std::fputc('\n', stderr); + std::fflush(stderr); + } + } results.push_back({lu.targetName, TestResult::St::CompileFail, 0, b.error().diagnosticOutput, {}}); emit_json(results.back()); @@ -1035,13 +1047,8 @@ export int run_tests(std::span passthrough, std::println(""); std::println("failures:"); for (auto& n : failures) std::println(" {}", n); - // Compile diagnostics per failed test, on stderr, after the summary — - // grouped so a learner scrolls to exactly their test's errors. - for (auto& r : results) { - if (r.status != TestResult::St::CompileFail || r.compileOutput.empty()) continue; - std::println(stderr, "\n--- {} (compile) ---", r.name); - std::fputs(r.compileOutput.c_str(), stderr); - } + // (Each compile failure's diagnostics already printed inline under its + // FAIL line in Phase B — the summary stays a compact name list.) return 1; } From 0c466d07a614b078d5b3d92ab90ac3ab148deb7e Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:14:22 +0800 Subject: [PATCH 12/22] =?UTF-8?q?docs:=20mcpp=20test=20architecture=20revi?= =?UTF-8?q?ew=20=E2=80=94=20layering=20vs=20gtest,=20batch=20evaluation,?= =?UTF-8?q?=20roadmap=20P1-P6?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2026-07-24-mcpp-test-design-review.md | 212 ++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 .agents/docs/2026-07-24-mcpp-test-design-review.md diff --git a/.agents/docs/2026-07-24-mcpp-test-design-review.md b/.agents/docs/2026-07-24-mcpp-test-design-review.md new file mode 100644 index 0000000..6e980af --- /dev/null +++ b/.agents/docs/2026-07-24-mcpp-test-design-review.md @@ -0,0 +1,212 @@ +# mcpp test 架构评估与设计方案 + +- 日期:2026-07-24 +- 分支:`feat/test-isolation-json`(评估对象 = 本分支 8 个提交 + mcpp test 既有形态) +- 关联:d2mcpp「练习即测试」重设计(d2mcpp 仓 `.agents/docs/2026-07-23-exercises-as-tests-design.md`) +- 本文定位:**设计评审记录 + 后续演进方案**。先从 mcpp 自身架构评估本批次改动与 + `mcpp test`/gtest 的定位关系,再评估对 d2mcpp 的适配性,最后给出路线图。 + +--- + +## 1. 结论摘要 + +1. `mcpp test` 的正确定位是**二进制运行器(runner of binaries)**:发现/构建/运行 + `tests/**/*.cpp`(每文件一个独立二进制)并聚合退出码。它对二进制内部使用什么 + 断言框架**零感知**——gtest 是依赖层(dev-dependencies)的方案,与运行器正交。 + 本批次改动全部落在运行器层,未破坏该分层。 +2. 六项改动中五项与 mcpp 架构自洽且为通用能力;**逐测试隔离的当前实现存在一个 + 实质代价:Phase B 跨测试构建串行化**(全绿路径最明显),有明确的改进方案(P1, + keep-going 预构建 + 存在性归因),语义不变、并行度恢复。 +3. gtest 与 mcpp test 不是竞争关系而是两层:文件级过滤(`mcpp test `)与 + 用例级过滤(`-- --gtest_filter=...` 透传)今天已可组合。不建议把 gtest 感知 + 内建进运行器;若要用例级一等公民,正确路径是**可选的测试协议**(P5,v2 提案)。 +4. 对 d2mcpp:练习=文件=二进制的模型是**必然选择**(用例级框架无法表达"尚未编译 + 通过"这一教学核心状态);判定三来源分层映射良好;遗留一处发现逻辑重复 + (Provider 目录扫描 vs 运行器 glob),可由 P2(`mcpp test --list`)收敛。 + +--- + +## 2. 分层模型:运行器与框架的正交性 + +``` +┌────────────────────────────────────────────────────────┐ +│ runner 层(mcpp test) │ +│ 发现 tests/**/*.cpp → 每文件合成 TestBinary target │ +│ 构建(dev-deps 生效) → 运行 → 退出码聚合 → 人读/JSON │ +├────────────────────────────────────────────────────────┤ +│ in-binary 层(框架自选,mcpp 零感知) │ +│ 裸 main / gtest(+gtest_main) / catch2 / d2x 练习库 … │ +└────────────────────────────────────────────────────────┘ +``` + +既有事实支撑这一分层已是 mcpp 的实际设计: + +- gtest 以 `[dev-dependencies] gtest = "1.15.2"` 进入,`plan.cppm` 的 + dependency-provided-entry 启发(gtest_main 提供 `main`)解决链接问题——框架 + 支持发生在**依赖解析层**,不在运行器; +- `mcpp test -- args` 把参数透传给**每个**测试二进制,`-- --gtest_filter=X` + 即用例级过滤,无需运行器懂 gtest; +- 判定协议只有一条:**进程退出码**。gtest 聚合其内部用例到退出码,裸 main 直接 + 返回,d2x 练习库经 atexit 收口——三者对运行器等价。 + +### 与同类工具对照 + +| 维度 | mcpp test | cargo test | ctest | gtest(单跑) | +|---|---|---|---|---| +| 测试单元 | 文件=二进制 | 集成测试同(tests/*.rs=crate);单测经 libtest 协议到用例级 | 注册的命令 | 用例(in-binary) | +| 编译失败语义 | **该测试 fail,其余照跑**(本批次) | 整体构建失败(不隔离) | 构建在外部,坏目标不阻塞其他命令 | n/a | +| 过滤 | 文件名子串(本批次) | 用例名子串 | -R 正则 | --gtest_filter | +| 机器可读 | NDJSON(本批次) | --message-format json | --output-json | --gtest_output=json | +| 判定 | 退出码 | libtest 协议/退出码 | 退出码/正则 | 内部统计 | + +结论:mcpp test 的文件级模型与 ctest 同族、比 cargo 的集成测试**多出编译隔离** +(cargo 的 tests/*.rs 一个编译失败同样拖死整轮)。用例级是 cargo 单测/gtest 的 +领地,mcpp 通过透传组合而非内建。 + +--- + +## 3. 本批次改动逐项评估 + +### 3.1 tests 相对路径命名(4b09984)✓ + +- 动机:子目录布局下 stem 撞名硬错误;名字应与文件系统布局同构。 +- 一致性:合成 target 专属规则,平铺布局名字不变,无兼容破坏。 +- **已知豁免需要记录**:合成测试名可含 `/`(`00-a/0`),而 `[targets.*]` 用户 + target 名文法不允许——两套命名空间事实上分离(合成 target 不进 manifest、不参与 + publish/xpkg)。属可接受的不一致,但应在 docs 里写明,防止未来有人"统一"它。 + +### 3.2 `BuildOptions::ninjaTargets`(bce3e7d)✓ + +- 后端接口的最小扩展(显式 goal 集合,空=全量),`producedArtifacts` 同步收窄。 +- 通用性无争议;风险点是调用方误传非 output 路径 → ninja "unknown target" 报错, + 错误信息可读,可接受。 + +### 3.3 逐测试编译隔离(ed557df + Phase A 目标集修正)——语义 ✓,实现有代价 + +语义评估: +- 包级失败(lib/deps/共享模块)≠ 测试失败的区分是**正确且必要**的:52 个红测试 + 误导用户"你全错了",而真相是基础设施坏了。Phase A 以"全部非测试入口的编译单元 + + 非测试 link 产物"为 goal 集合,覆盖了 test 模式下 lib target 被跳过、共享破坏 + 藏在模块对象里的边角(e2e 119 第二场景验证)。 +- 归因精确:每测试独立 ninja goal,诊断天然按测试分组,扫描期错误(dyndep)也 + 落在对应测试上。 + +实现代价(本文最重要的自我批评): +- **Phase B 对每个测试串行调用 `backend->build()`**。旧实现一次 ninja 调用并行 + 编译全部测试 TU;现在 N 个测试 = N 次 ninja 进程 + N 次 build.ninja/ + compile_commands 重写 + N 次(缓存命中的)hermetic 检查,且**测试之间的编译 + 不再并行**。全绿路径(CI、正常项目)代价最大;d2mcpp 的 49 题答案全量验证 + 实测 ~29s(含运行),可用但不优。 +- 运行阶段本就串行(与旧行为一致),不算回归,但同样是并行机会。 + +**改进方案(P1,语义不变)**:三段式 +1. Phase A 不变; +2. Phase B-1:**单次** ninja 调用携带全部(过滤后)测试 goal + `-k 0` + (keep going)——恢复全并行; +3. Phase B-2:逐测试按**产物存在性**判定编译成败;对失败者**逐个**重跑其 goal + (必然快速失败)以取得干净的每测试诊断,随后照常运行成功者。 + 失败是少数路径,逐个重试的代价与收益成正比。 + +### 3.4 过滤器(16da216)✓ + +- 过滤发生在**构建/运行阶段而非计划阶段**是关键决策:计划恒含全部测试, + `compile_commands.json` 保持完整(clangd 依赖)。正确。 +- 子串语义与 cargo 对齐;`…/1` 匹配 `…/10` 的邻居效应已在下游(d2mcpp Provider + 逐行精确匹配)消化。可选增强:`--exact`。 + +### 3.5 `--message-format json`(f2c7830)✓,schema 需小幅演进 + +- 流式逐测试记录 + 包级错误记录 + summary,stdout 纯协议流:与 `-q`/`ui::set_quiet` + 的既有静音机制正交组合,实现干净。 +- 缺口:per-test `duration_ms`(进度类前端需要);无 `--list`(见 P2); + `signal` 从退出码反推(128+n 启发)而非 wait status 直读,Windows 语义空缺。 + +### 3.6 输出交错与横幅(fix 提交)✓ + +`Compiling → 结果 → 该测试诊断`的连续块 + 删除 Phase A 后误导性的 +`Finished test` 横幅——cargo 形态,修复了"成功横幅紧邻全红"的自相矛盾。 + +### 3.7 平台/清单层修复 ✓ + +- `merged_environ` 剥离 `xim-x-glibc` loader 条目:只针对私有 payload、用户条目 + 保留、显式 override 优先——外科式,配合 musl 静态发行从两端封死嵌套段错误。 +- `[build].flags` glob 覆盖测试 TU + 死 glob 警告改查磁盘:glob 指名文件, + 是否被扫描为 source 与之正交;警告语义从"没命中扫描源"修正为"磁盘上无此文件", + 是普适改进而非特例。 + +--- + +## 4. mcpp test 与 gtest:定位与演进 + +### 4.1 现状判断 + +- 运行器不感知框架是**优点**:gtest/catch2/doctest/裸 main/d2x 库在退出码协议下 + 等价,mcpp 无需 per-framework 适配矩阵。 +- `docs/00-getting-started.md` 的 "(gtest style)" 措辞**具有误导性**——它描述的是 + "常配合 gtest 使用",读起来却像"运行器实现了 gtest 语义"。应改为 + "one binary per file; bring your own framework (gtest via dev-dependencies)"。 + +### 4.2 不建议:运行器内建 gtest 感知 + +内建意味着:解析 gtest 输出/JSON、维护版本兼容、并对 catch2/doctest 重复同样工作。 +分层被打穿,收益仅是省去一次 `--` 透传。否决。 + +### 4.3 建议(P5,v2 提案):可选的用例级测试协议 + +参照 cargo/libtest 的"运行器↔测试二进制"契约,定义 mcpp 自己的**可选**协议: + +``` +运行器设 MCPP_TEST_PROTOCOL=1 后: + --mcpp-list → NDJSON: {"case":"Math.Add"} … + --mcpp-run → 运行单用例,退出码判定 + (或单进程模式: --mcpp-json → 逐用例 NDJSON 结果流) +不支持协议的二进制(裸 main):探测失败即回退到整二进制模式——零破坏。 +``` + +- gtest 侧一个 ~50 行 adapter(translate 到 `--gtest_list_tests`/`--gtest_filter`/ + `--gtest_output`)即可接入;d2x 练习库天然可实现。 +- 运行器获得:用例级过滤/并行/JSON,而仍不认识任何具体框架。 +- 明确 **YAGNI 边界**:在出现第二个真实需求方之前不实现,本文仅锁定方向,防止 + 未来用"内建 gtest"这类打穿分层的方案填这个空。 + +--- + +## 5. 对 d2mcpp 的适配性评估 + +1. **文件=二进制模型是教学的必然选择**:练习的核心状态是"尚未编译通过",用例级 + 框架(gtest 等)以"可编译"为前提,无法表达这一状态;编译错误本身是教学主通道 + (`D2X_YOUR_ANSWER` 指着要填的位置)。文件级隔离恰好把"一个未完成练习"约束为 + "一个红测试"。 +2. **判定三来源分层映射良好**:mcpp JSON(编译/退出码事实)× d2x 库侧信道 + (断言语义/路障)× Provider(合并成协议 verdict)——运行器不需要为 d2mcpp + 增加任何专有语义,验证了 §2 的分层。 +3. **发现逻辑存在重复**:Provider 自行扫描 `src/*/tests/` 推导 id/order,运行器 + 另有一份 `tests/**` glob。两者约定同构但真相源有二。P2 的 + `mcpp test --list --message-format json`(输出测试名+主文件路径)可让 Provider + 的枚举改为消费运行器输出,回到单一真相源。 +4. **性能画像**:checker 单题路径(filter + 缓存)最优(~0.2s 级);e2e 全量 + 路径承受 §3.3 的串行代价(49 题 ~29s),P1 落地后预期显著下降。 +5. **已消化的风险**:子串过滤邻居效应(Provider 精确匹配);`signal` 启发式 + (练习判定不依赖 signal 字段)。 + +--- + +## 6. 路线图(按优先级) + +| # | 事项 | 层 | 验收标准 | +|---|---|---|---| +| P1 | Phase B 改 keep-going 预构建 + 存在性归因 + 失败者逐个取诊断 | runner | e2e 118–123 不变全绿;多测试全绿工程 `mcpp test` 构建墙钟时间恢复到与旧单次构建同量级 | +| P2 | `mcpp test --list [--message-format json]` | runner | 输出测试名+主文件;d2mcpp Provider 枚举可切换为消费该输出 | +| P3 | JSON 记录增加 `duration_ms`;`signal` 改从 wait status 直读(POSIX) | runner | schema 向后兼容(只增字段) | +| P4 | 每测试超时 `--timeout `(挂死测试 → run_fail + 标注) | runner | 睡死测试不拖垮整轮 | +| P5 | 用例级测试协议(§4.3) | 协议 | 出现第二个需求方后再立项 | +| P6 | docs 措辞修正:"(gtest style)" → framework-agnostic 表述;记录 §3.1 命名豁免 | docs | — | + +## 7. 风险与未决 + +- **Windows 全链路未实测**:`--quiet`(sandbox ninja 补丁旗标)、路径分隔、 + `_pclose` 退出码语义——发版前需要一轮 Windows CI。 +- P1 的存在性归因前提是"goal 失败 ⇒ 产物不存在":ninja 对失败目标不落盘产物, + 但**残留的旧产物**会造成误判——实现时须以"本轮 mtime 是否推进/restat 状态"或 + 预删除产物为判据,不能只看 exists()。 +- 协议字段一旦被 d2mcpp 之外的消费者依赖,schema 演进需按"只增不改"纪律执行。 From dc76d48b876f5dc33266001a1554910566394b2b Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:20:53 +0800 Subject: [PATCH 13/22] docs: batch-2 implementation plan (P1-P4,P6 + release 0.0.104) --- .agents/docs/2026-07-24-test-batch2-plan.md | 29 +++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 .agents/docs/2026-07-24-test-batch2-plan.md diff --git a/.agents/docs/2026-07-24-test-batch2-plan.md b/.agents/docs/2026-07-24-test-batch2-plan.md new file mode 100644 index 0000000..47c6556 --- /dev/null +++ b/.agents/docs/2026-07-24-test-batch2-plan.md @@ -0,0 +1,29 @@ +# mcpp test 演进批次二 — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** 落实 `2026-07-24-mcpp-test-design-review.md` 路线图 P1/P2/P3/P4/P6(P5 按该文档决策显式排除:出现第二个需求方前不立项),版本 0.0.103 → 0.0.104,单 PR 合入 main。 + +**Architecture:** 全部改动收敛在 `src/build/execute.cppm`(run_tests)、`src/build/backend.cppm`/`ninja_backend.cppm`(keepGoing)、`src/platform/process.cppm`(信号退出码规范化 + 带超时运行)、`src/cli*`(--list/--timeout 参数)与 docs。每项 TDD:先写 e2e(124–127)确认 RED 再实现。 + +**Tech Stack:** 既有栈;e2e bash 脚本。 + +## Global Constraints + +- 语义回归零容忍:e2e 118–123 全程保持绿。 +- JSON schema 只增不改(新增 `duration_ms`、`timed_out` 字段;`signal` 语义修正属 bug fix)。 +- 每任务一个提交;最后版本提交(mcpp.toml 0.0.104 + CHANGELOG 定版 2026-07-24)。 +- PR:单个,源分支 `feat/test-isolation-json` → main;CI 全绿后 `gh pr merge --squash --admin`(用户已明确授权 bypass squash)。 +- 生态验证:合入后以 main 构建 musl 二进制,复跑 mcpp e2e 全量 + d2mcpp e2e all + d2x 构建/单测/checker 冒烟。 + +## Tasks + +- [ ] **T1 (P1) Phase B 并行化**:`BuildOptions.keepGoing`(→ ninja `-k 0`);run_tests 在逐测试循环前插入一次携带全部过滤后 goal 的 bulk build(结果忽略,只为并行填充缓存);既有逐测试 build 保留(成功者缓存命中≈无操作,失败者快速重试取干净诊断)。验收:118–123 绿;d2mcpp cpp11 全量答案态构建墙钟显著下降(手测记录)。 +- [ ] **T2 (P3a) 信号退出码规范化**:`normalize_exit_code` 增加 `WIFSIGNALED → 128+WTERMSIG`(shell 惯例;当前返回原始 status,JSON signal 推导永不触发的 bug)。e2e 124:段错误测试 → `"exit_code":139,"signal":11`。 +- [ ] **T3 (P3b) per-test `duration_ms`**:JSON 记录增加字段(编译+运行墙钟);summary 已有 elapsed 不动。并入 e2e 124 断言字段存在。 +- [ ] **T4 (P2) `mcpp test --list`**:列出(过滤后)测试名;`--message-format json` 时逐行 `{"test":…,"main":…}` + `{"summary":{"total":N}}`;不触发构建。e2e 125。 +- [ ] **T5 (P4) `--timeout `**:平台层新增带截止的运行(POSIX spawn+WNOHANG 轮询,超时 SIGKILL;Windows 暂不支持并文档注明);超时 → `FAIL (timeout)` / JSON `"timed_out":true` + run_fail。e2e 126。 +- [ ] **T6 (P6) docs**:"(gtest style)" → framework-agnostic 措辞;记录合成测试名含 `/` 的命名豁免;README/README.zh 同步;CHANGELOG 批次二条目。 +- [ ] **T7 版本与回归**:mcpp.toml → 0.0.104;CHANGELOG 未发布 → [0.0.104] — 2026-07-24;全量 e2e(run_all)glibc + 新脚本 musl 双跑。 +- [ ] **T8 PR + CI + 合入**:push 分支,gh pr create(标题含 0.0.104),等 CI 全绿,`gh pr merge --squash --admin`。 +- [ ] **T9 生态验证**:main 重建 musl;mcpp e2e 全量;d2mcpp `e2e.sh all` + d2x mcpp build/test + checker 冒烟,全部以 main 产物执行;结果记录回本文件。 From fcde0db6b1f6cf0068ba277915bb2dda6ba0c2f7 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:22:05 +0800 Subject: [PATCH 14/22] =?UTF-8?q?perf(test):=20parallel=20keep-going=20pre?= =?UTF-8?q?-build=20for=20Phase=20B=20=E2=80=94=20full-suite=20wall=20time?= =?UTF-8?q?=2029s=20->=203.7s=20on=2049=20tests?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/build/backend.cppm | 3 +++ src/build/execute.cppm | 18 +++++++++++++++++- src/build/ninja_backend.cppm | 5 +++++ 3 files changed, 25 insertions(+), 1 deletion(-) diff --git a/src/build/backend.cppm b/src/build/backend.cppm index 99b6227..63a4931 100644 --- a/src/build/backend.cppm +++ b/src/build/backend.cppm @@ -16,6 +16,9 @@ struct BuildOptions { // Explicit ninja goal targets (LinkUnit::output paths, relative to the // plan's outputDir). Empty = build the full plan (default behavior). std::vector ninjaTargets; + // Keep building unaffected goals after a failure (ninja -k 0). Used by + // `mcpp test` to pre-build all test goals in one parallel pass. + bool keepGoing = false; }; struct BuildResult { diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 70b2b6b..a1fa7a0 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -931,7 +931,23 @@ export int run_tests(std::span passthrough, // failures read as a contradiction; the final summary carries timing. } - // 6. Phase B: build + run each test in sequence; collect results. + // 6. Phase B. First a single keep-going bulk build over every selected + // test goal — ninja parallelizes across tests and a failing test does + // not stop the rest (-k 0). The result is deliberately ignored: the + // per-test loop below re-drives each goal, where successes are cache + // hits (near no-ops) and failures re-fail fast, yielding cleanly + // attributed per-test diagnostics without sacrificing parallelism. + { + mcpp::build::BuildOptions bulk; + bulk.keepGoing = true; + for (auto& lu : ctx->plan.linkUnits) + if (filter_match(lu)) + bulk.ninjaTargets.push_back(lu.output.generic_string()); + if (!bulk.ninjaTargets.empty()) + (void)backend->build(ctx->plan, bulk); + } + + // Then build + run each test in sequence; collect results. auto t0 = std::chrono::steady_clock::now(); auto runtimeEnvKey = mcpp::platform::env::runtime_library_path_key(); diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 3778666..7b5b276 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -1195,6 +1195,11 @@ std::expected NinjaBackend::build(const BuildPlan& plan if (opts.parallelJobs) nargv.push_back(std::format("-j{}", opts.parallelJobs)); + if (opts.keepGoing) { + nargv.push_back("-k"); + nargv.push_back("0"); + } + // Explicit goal targets: ninja builds only these outputs (and their // prerequisites). Used by `mcpp test` to isolate per-test compiles. for (auto& t : opts.ninjaTargets) From 97aa3dd2f21a33903f478af0de4a3a35f1f364bb Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:23:52 +0800 Subject: [PATCH 15/22] feat(test): shell-convention signal exit codes (128+sig) + per-test duration_ms in JSON --- src/build/execute.cppm | 16 ++++++--- src/platform/process.cppm | 6 ++++ tests/e2e/124_test_signal_and_duration.sh | 41 +++++++++++++++++++++++ 3 files changed, 59 insertions(+), 4 deletions(-) create mode 100755 tests/e2e/124_test_signal_and_duration.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index a1fa7a0..5167a50 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -855,6 +855,7 @@ export int run_tests(std::span passthrough, int exitCode = 0; std::string compileOutput; std::string runOutput; + long long durationMs = 0; // build+run wall time for THIS test }; std::vector results; @@ -869,8 +870,9 @@ export int run_tests(std::span passthrough, std::string signal = (r.exitCode > 128 && r.exitCode < 128 + 65) ? std::to_string(r.exitCode - 128) : "null"; std::println("{{\"test\":\"{}\",\"status\":\"{}\",\"exit_code\":{},\"signal\":{}," + "\"duration_ms\":{}," "\"compile_output\":\"{}\",\"run_output\":\"{}\"}}", - test_json_escape(r.name), st, r.exitCode, signal, + test_json_escape(r.name), st, r.exitCode, signal, r.durationMs, test_json_escape(r.compileOutput), test_json_escape(r.runOutput)); std::fflush(stdout); }; @@ -957,6 +959,12 @@ export int run_tests(std::span passthrough, for (auto& lu : ctx->plan.linkUnits) { if (!filter_match(lu)) continue; + auto tTest = std::chrono::steady_clock::now(); + auto test_ms = [&tTest] { + return std::chrono::duration_cast( + std::chrono::steady_clock::now() - tTest).count(); + }; + mcpp::ui::status("Compiling", std::format("{} (test)", lu.targetName)); mcpp::build::BuildOptions bOpts; @@ -976,7 +984,7 @@ export int run_tests(std::span passthrough, } } results.push_back({lu.targetName, TestResult::St::CompileFail, 0, - b.error().diagnosticOutput, {}}); + b.error().diagnosticOutput, {}, test_ms()}); emit_json(results.back()); continue; } @@ -1023,11 +1031,11 @@ export int run_tests(std::span passthrough, if (exitCode == 0) { if (!json) std::println("{} ... ok", lu.targetName); results.push_back({lu.targetName, TestResult::St::Pass, 0, {}, - std::move(runOutput)}); + std::move(runOutput), test_ms()}); } else { if (!json) std::println("{} ... FAIL (exit {})", lu.targetName, exitCode); results.push_back({lu.targetName, TestResult::St::RunFail, exitCode, {}, - std::move(runOutput)}); + std::move(runOutput), test_ms()}); } emit_json(results.back()); } diff --git a/src/platform/process.cppm b/src/platform/process.cppm index 8c0cb21..d5cefbf 100644 --- a/src/platform/process.cppm +++ b/src/platform/process.cppm @@ -137,6 +137,12 @@ int normalize_exit_code(int rc) { #else if (WIFEXITED(rc)) return WEXITSTATUS(rc); + // Shell convention for signaled children: 128 + signal number. The raw + // wait-status word only *happens* to look right when the core-dump bit + // is set (SIGSEGV+core → 0x8B = 139); without it a SIGTERM death would + // surface as "exit 15" and be indistinguishable from a normal exit code. + if (WIFSIGNALED(rc)) + return 128 + WTERMSIG(rc); return rc; #endif } diff --git a/tests/e2e/124_test_signal_and_duration.sh b/tests/e2e/124_test_signal_and_duration.sh new file mode 100755 index 0000000..8cd9d2a --- /dev/null +++ b/tests/e2e/124_test_signal_and_duration.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +# requires: +# Signaled tests report shell-convention exit codes (128+sig) with the signal +# number in JSON, and every JSON test record carries duration_ms. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "sigdur" +version = "0.1.0" +standard = "c++23" +EOF +cat > tests/crash.cpp <<'EOF' +int main() { int* p = nullptr; return *p; } +EOF +cat > tests/ok.cpp <<'EOF' +int main() { return 0; } +EOF + +set +e +out=$("$MCPP" test --message-format json 2>/dev/null) +code=$? +set -e +[[ $code -eq 1 ]] || { echo "expected exit 1, got $code"; exit 1; } + +echo "$out" | grep -q '"test":"crash","status":"run_fail","exit_code":139,"signal":11' \ + || { echo "signaled test not normalized to 139/11: $out"; exit 1; } +echo "$out" | grep -q '"test":"ok","status":"pass"' || { echo "ok test missing"; exit 1; } +[[ $(echo "$out" | grep -c '"duration_ms":') -eq 2 ]] \ + || { echo "duration_ms missing from test records: $out"; exit 1; } + +# 人读输出同样用 shell 惯例退出码 +out2=$("$MCPP" test crash 2>&1) || true +[[ "$out2" == *"crash ... FAIL (exit 139)"* ]] || { echo "human output not 139: $out2"; exit 1; } +echo OK From 631b400464c71b17e2163c39e3b49431856c075a Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:25:33 +0800 Subject: [PATCH 16/22] =?UTF-8?q?feat(test):=20--list=20=E2=80=94=20enumer?= =?UTF-8?q?ate=20(filtered)=20tests=20without=20building;=20JSON=20records?= =?UTF-8?q?=20for=20tooling?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/build/execute.cppm | 25 +++++++++++++++++++++++++ src/cli.cppm | 2 ++ src/cli/cmd_build.cppm | 1 + tests/e2e/125_test_list.sh | 30 ++++++++++++++++++++++++++++++ 4 files changed, 58 insertions(+) create mode 100755 tests/e2e/125_test_list.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 5167a50..d49db72 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -677,6 +677,7 @@ export enum class TestMessageFormat { Human, Json }; export struct TestOptions { std::string filter; // substring match on the path-based test name; empty = all TestMessageFormat format = TestMessageFormat::Human; + bool list = false; // enumerate only, no build/run }; // Minimal JSON string escaping for the --message-format json records. Same @@ -783,6 +784,30 @@ export int run_tests(std::span passthrough, testTargets.push_back(std::move(t)); } + // --list: enumerate (filtered) tests and stop — no toolchain resolution, + // no build. Names/paths come straight from discovery, so this also works + // on tests that do not currently compile. + if (testOpts.list) { + std::size_t total = 0; + for (auto& t : testTargets) { + if (!testOpts.filter.empty() + && t.name.find(testOpts.filter) == std::string::npos) continue; + ++total; + auto abs = std::filesystem::absolute(testRoot / t.main) + .lexically_normal().generic_string(); + if (json) + std::println("{{\"test\":\"{}\",\"main\":\"{}\"}}", + test_json_escape(t.name), test_json_escape(abs)); + else + std::println("{}", t.name); + } + if (json) { + std::println("{{\"summary\":{{\"total\":{}}}}}", total); + std::fflush(stdout); + } + return 0; + } + // 3. prepare_build with dev-deps enabled + synthetic targets. auto ctx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, diff --git a/src/cli.cppm b/src/cli.cppm index 654e14e..762d787 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -255,6 +255,8 @@ int run(int argc, char** argv) { .help("Run only tests whose name contains PATTERN (optional)")) .option(cl::Option("message-format").takes_value().value_name("FMT") .help("Output format: human (default) | json (NDJSON, one record per test)")) + .option(cl::Option("list") + .help("List (filtered) tests without building or running them")) .option(cl::Option("profile").takes_value().value_name("NAME") .help("Build profile for the test build: release (default) | dev | dist | <[profile.*] name>")) .option(cl::Option("features").takes_value().value_name("LIST") diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index b264398..240fdff 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -127,6 +127,7 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, mcpp::build::TestOptions to; if (parsed.positional_count() > 0) to.filter = parsed.positional(0); + to.list = parsed.is_flag_set("list"); if (auto mf = parsed.value("message-format")) { if (*mf == "json") to.format = mcpp::build::TestMessageFormat::Json; else if (*mf != "human") { diff --git a/tests/e2e/125_test_list.sh b/tests/e2e/125_test_list.sh new file mode 100755 index 0000000..086ce69 --- /dev/null +++ b/tests/e2e/125_test_list.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# requires: +# `mcpp test --list`: enumerate (filtered) tests without building anything. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests/00-a pkg/tests/01-b +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "listing" +version = "0.1.0" +standard = "c++23" +EOF +echo 'int main() { return 0; }' > tests/00-a/0.cpp +echo 'this does not even parse' > tests/01-b/0.cpp # --list 不构建,坏文件也要列出 + +out=$("$MCPP" test --list 2>&1) || { echo "--list failed: $out"; exit 1; } +[[ "$out" == *"00-a/0"* && "$out" == *"01-b/0"* ]] || { echo "missing names: $out"; exit 1; } +[[ ! -d target ]] || { echo "--list must not build (target/ created)"; exit 1; } + +out=$("$MCPP" test --list 00-a --message-format json 2>/dev/null) +echo "$out" | grep -q '"test":"00-a/0","main":".*tests/00-a/0.cpp"' \ + || { echo "json list record missing/bad: $out"; exit 1; } +echo "$out" | grep -qv '"test":"01-b/0"' || { echo "filter leaked into list"; exit 1; } +echo "$out" | grep -q '"summary":{"total":1}' || { echo "missing list summary: $out"; exit 1; } +echo OK From 6455716f9f68201f6f55af1aa8fe04168545eb8f Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:28:53 +0800 Subject: [PATCH 17/22] feat(test): --timeout per-test run deadline (POSIX; Windows best-effort untimed) --- src/build/execute.cppm | 21 +++++- src/cli.cppm | 2 + src/cli/cmd_build.cppm | 9 +++ src/platform/process.cppm | 133 ++++++++++++++++++++++++++++++++++ tests/e2e/126_test_timeout.sh | 43 +++++++++++ 5 files changed, 204 insertions(+), 4 deletions(-) create mode 100755 tests/e2e/126_test_timeout.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index d49db72..b9cd280 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -678,6 +678,7 @@ export struct TestOptions { std::string filter; // substring match on the path-based test name; empty = all TestMessageFormat format = TestMessageFormat::Human; bool list = false; // enumerate only, no build/run + int timeoutSecs = 0; // per-test run deadline; 0 = unlimited }; // Minimal JSON string escaping for the --message-format json records. Same @@ -881,6 +882,7 @@ export int run_tests(std::span passthrough, std::string compileOutput; std::string runOutput; long long durationMs = 0; // build+run wall time for THIS test + bool timedOut = false; // killed by --timeout }; std::vector results; @@ -895,9 +897,10 @@ export int run_tests(std::span passthrough, std::string signal = (r.exitCode > 128 && r.exitCode < 128 + 65) ? std::to_string(r.exitCode - 128) : "null"; std::println("{{\"test\":\"{}\",\"status\":\"{}\",\"exit_code\":{},\"signal\":{}," - "\"duration_ms\":{}," + "\"duration_ms\":{},\"timed_out\":{}," "\"compile_output\":\"{}\",\"run_output\":\"{}\"}}", test_json_escape(r.name), st, r.exitCode, signal, r.durationMs, + r.timedOut ? "true" : "false", test_json_escape(r.compileOutput), test_json_escape(r.runOutput)); std::fflush(stdout); }; @@ -1043,17 +1046,27 @@ export int run_tests(std::span passthrough, // JSON mode captures the test's combined stdout+stderr into the // record; human mode streams it to the terminal as before. + auto deadline = std::chrono::milliseconds( + static_cast(testOpts.timeoutSecs) * 1000); + bool timedOut = false; int exitCode; std::string runOutput; if (json) { - auto rr = mcpp::platform::process::capture_exec(argv, childEnv); + auto rr = mcpp::platform::process::capture_exec_deadline( + argv, childEnv, deadline, &timedOut); exitCode = rr.exit_code; runOutput = std::move(rr.output); } else { - exitCode = mcpp::platform::process::run_exec(argv, childEnv); + exitCode = mcpp::platform::process::run_exec_deadline( + argv, childEnv, deadline, &timedOut); } - if (exitCode == 0) { + if (timedOut) { + if (!json) std::println("{} ... FAIL (timeout after {}s)", + lu.targetName, testOpts.timeoutSecs); + results.push_back({lu.targetName, TestResult::St::RunFail, exitCode, {}, + std::move(runOutput), test_ms(), true}); + } else if (exitCode == 0) { if (!json) std::println("{} ... ok", lu.targetName); results.push_back({lu.targetName, TestResult::St::Pass, 0, {}, std::move(runOutput), test_ms()}); diff --git a/src/cli.cppm b/src/cli.cppm index 762d787..b8c1c1c 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -257,6 +257,8 @@ int run(int argc, char** argv) { .help("Output format: human (default) | json (NDJSON, one record per test)")) .option(cl::Option("list") .help("List (filtered) tests without building or running them")) + .option(cl::Option("timeout").takes_value().value_name("SECS") + .help("Kill a test still running after SECS seconds (reported as a timeout failure)")) .option(cl::Option("profile").takes_value().value_name("NAME") .help("Build profile for the test build: release (default) | dev | dist | <[profile.*] name>")) .option(cl::Option("features").takes_value().value_name("LIST") diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 240fdff..361550e 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -128,6 +128,15 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, mcpp::build::TestOptions to; if (parsed.positional_count() > 0) to.filter = parsed.positional(0); to.list = parsed.is_flag_set("list"); + if (auto ts = parsed.value("timeout")) { + int secs = 0; + auto [p, ec] = std::from_chars(ts->data(), ts->data() + ts->size(), secs); + if (ec != std::errc{} || p != ts->data() + ts->size() || secs < 0) { + mcpp::ui::error(std::format("invalid --timeout '{}' (whole seconds >= 0)", *ts)); + return 2; + } + to.timeoutSecs = secs; + } if (auto mf = parsed.value("message-format")) { if (*mf == "json") to.format = mcpp::build::TestMessageFormat::Json; else if (*mf != "human") { diff --git a/src/platform/process.cppm b/src/platform/process.cppm index d5cefbf..06a78fe 100644 --- a/src/platform/process.cppm +++ b/src/platform/process.cppm @@ -34,6 +34,11 @@ module; #include // pipe, dup2, close, read #include // waitpid #include // posix_spawnp, posix_spawn_file_actions_* (incl. addchdir_np) +#include // kill, SIGKILL (deadline runners) +#include // errno, EINTR (deadline wait loop) +#include // poll (deadline capture) +#include // fcntl O_NONBLOCK (deadline capture) +#include // nanosleep (deadline wait loop) #if defined(__APPLE__) #include // _NSGetEnviron — direct `environ` is only linkable // from executables on Apple, not from dylibs @@ -88,6 +93,21 @@ RunResult capture_exec( const std::vector>& extraEnv = {}, std::string_view cwd = {}); +// Deadline variants (POSIX): kill the child with SIGKILL once `deadline` +// elapses and set *timed_out. A zero deadline means no limit. On Windows the +// deadline is currently ignored (no supported kill-by-handle path in the +// residual shell launcher) — callers must treat the timeout as best-effort. +int run_exec_deadline(const std::vector& argv, + const std::vector>& extraEnv, + std::chrono::milliseconds deadline, + bool* timed_out); + +RunResult capture_exec_deadline( + const std::vector& argv, + const std::vector>& extraEnv, + std::chrono::milliseconds deadline, + bool* timed_out); + // Run `command` silently (discard stdout/stderr). // On POSIX, stdin is automatically redirected from /dev/null. int run_silent(std::string_view command); @@ -451,4 +471,117 @@ RunResult capture_exec( #endif } +int run_exec_deadline(const std::vector& argv, + const std::vector>& extraEnv, + std::chrono::milliseconds deadline, + bool* timed_out) +{ + if (timed_out) *timed_out = false; + if (deadline.count() <= 0) return run_exec(argv, extraEnv); + if (argv.empty()) return 127; +#if defined(__linux__) || defined(__APPLE__) + auto envStore = merged_environ(extraEnv); + std::vector envp; + for (auto& s : envStore) envp.push_back(s.data()); + envp.push_back(nullptr); + std::vector cargv; + for (auto& a : argv) cargv.push_back(const_cast(a.c_str())); + cargv.push_back(nullptr); + + pid_t pid = 0; + if (::posix_spawnp(&pid, cargv[0], nullptr, nullptr, cargv.data(), envp.data()) != 0) + return 127; + + auto until = std::chrono::steady_clock::now() + deadline; + int status = 0; + for (;;) { + pid_t r = ::waitpid(pid, &status, WNOHANG); + if (r == pid) return normalize_exit_code(status); + if (r < 0 && errno != EINTR) return 127; + if (std::chrono::steady_clock::now() >= until) { + ::kill(pid, SIGKILL); + while (::waitpid(pid, &status, 0) < 0) { /* EINTR retry */ } + if (timed_out) *timed_out = true; + return normalize_exit_code(status); + } + struct timespec ts{0, 20'000'000}; // 20ms + ::nanosleep(&ts, nullptr); + } +#else + // Windows: the residual shell launcher has no kill-by-handle path yet — + // run untimed (documented best-effort semantics). + return run_exec(argv, extraEnv); +#endif +} + +RunResult capture_exec_deadline( + const std::vector& argv, + const std::vector>& extraEnv, + std::chrono::milliseconds deadline, + bool* timed_out) +{ + if (timed_out) *timed_out = false; + if (deadline.count() <= 0) return capture_exec(argv, extraEnv); + RunResult result; + if (argv.empty()) { result.exit_code = 127; return result; } +#if defined(__linux__) || defined(__APPLE__) + int fds[2]; + if (::pipe(fds) != 0) { result.exit_code = 127; return result; } + + auto envStore = merged_environ(extraEnv); + std::vector envp; + for (auto& s : envStore) envp.push_back(s.data()); + envp.push_back(nullptr); + std::vector cargv; + for (auto& a : argv) cargv.push_back(const_cast(a.c_str())); + cargv.push_back(nullptr); + + posix_spawn_file_actions_t fa; + ::posix_spawn_file_actions_init(&fa); + ::posix_spawn_file_actions_adddup2(&fa, fds[1], 1); + ::posix_spawn_file_actions_adddup2(&fa, fds[1], 2); + ::posix_spawn_file_actions_addclose(&fa, fds[0]); + ::posix_spawn_file_actions_addclose(&fa, fds[1]); + + pid_t pid = 0; + int sp = ::posix_spawnp(&pid, cargv[0], &fa, nullptr, cargv.data(), envp.data()); + ::posix_spawn_file_actions_destroy(&fa); + ::close(fds[1]); + if (sp != 0) { ::close(fds[0]); result.exit_code = 127; return result; } + + ::fcntl(fds[0], F_SETFL, ::fcntl(fds[0], F_GETFL) | O_NONBLOCK); + + auto until = std::chrono::steady_clock::now() + deadline; + bool killed = false; + std::array buf{}; + for (;;) { + struct pollfd pfd{fds[0], POLLIN, 0}; + ::poll(&pfd, 1, 50); + for (;;) { + ssize_t n = ::read(fds[0], buf.data(), buf.size()); + if (n > 0) { result.output.append(buf.data(), static_cast(n)); continue; } + break; + } + int status = 0; + pid_t r = ::waitpid(pid, &status, WNOHANG); + if (r == pid) { + // Drain whatever is left in the pipe after exit. + ssize_t n; + while ((n = ::read(fds[0], buf.data(), buf.size())) > 0) + result.output.append(buf.data(), static_cast(n)); + ::close(fds[0]); + result.exit_code = normalize_exit_code(status); + if (timed_out) *timed_out = killed; + return result; + } + if (!killed && std::chrono::steady_clock::now() >= until) { + ::kill(pid, SIGKILL); + killed = true; + } + } +#else + return capture_exec(argv, extraEnv); +#endif +} + } // namespace mcpp::platform::process diff --git a/tests/e2e/126_test_timeout.sh b/tests/e2e/126_test_timeout.sh new file mode 100755 index 0000000..c3ee28b --- /dev/null +++ b/tests/e2e/126_test_timeout.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# requires: +# `mcpp test --timeout `: a hung test is killed and reported as a +# timeout failure; the rest of the suite still runs. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p pkg/tests +cd pkg +cat > mcpp.toml <<'EOF' +[package] +name = "hang" +version = "0.1.0" +standard = "c++23" +EOF +cat > tests/sleepy.cpp <<'EOF' +import std; +int main() { std::this_thread::sleep_for(std::chrono::seconds(30)); return 0; } +EOF +cat > tests/quick.cpp <<'EOF' +int main() { return 0; } +EOF + +start=$(date +%s) +set +e +out=$("$MCPP" test --timeout 2 2>&1) +code=$? +set -e +elapsed=$(( $(date +%s) - start )) +[[ $code -eq 1 ]] || { echo "expected exit 1, got $code: $out"; exit 1; } +[[ $elapsed -lt 25 ]] || { echo "timeout did not take effect (took ${elapsed}s)"; exit 1; } +[[ "$out" == *"sleepy ... FAIL (timeout"* ]] || { echo "missing timeout marker: $out"; exit 1; } +[[ "$out" == *"quick ... ok"* ]] || { echo "quick test did not run: $out"; exit 1; } + +set +e +outj=$("$MCPP" test --timeout 2 --message-format json 2>/dev/null) +set -e +echo "$outj" | grep -q '"test":"sleepy","status":"run_fail"' || { echo "json: no run_fail"; exit 1; } +echo "$outj" | grep -q '"timed_out":true' || { echo "json: timed_out missing: $outj"; exit 1; } +echo OK From 6f6c90f86a705e0a5c4c68bedb394245c1dfbab7 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:29:53 +0800 Subject: [PATCH 18/22] docs(test): framework-agnostic wording (drop 'gtest style'), new flags, naming exemption note (zh+en) --- README.md | 2 +- README.zh-CN.md | 2 +- docs/00-getting-started.md | 5 ++++- docs/05-mcpp-toml.md | 2 +- docs/zh/00-getting-started.md | 6 +++++- docs/zh/05-mcpp-toml.md | 2 +- src/cli.cppm | 2 +- 7 files changed, 14 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 16d64c9..316c17a 100644 --- a/README.md +++ b/README.md @@ -221,7 +221,7 @@ import mcpplibs.cmdline; - `mcpp new` — create a modular project; `--template [@ver][:]` uses a **library-provided template** (e.g. `--template imgui`); `--list-templates ` lists them - `mcpp run [-- args]` — build and run -- `mcpp test [pattern] [-- args]` — auto-discover and run tests (optionally filtered by name; `--message-format json` for tooling) +- `mcpp test [pattern] [-- args]` — auto-discover and run tests (filter by name; `--list`, `--timeout `, `--message-format json`) - `mcpp search` — search package indices - `mcpp add / remove / update` — dependency management - `mcpp explain E0001` — detailed error-code explanations diff --git a/README.zh-CN.md b/README.zh-CN.md index 19d78d0..8271378 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -221,7 +221,7 @@ import mcpplibs.cmdline; - `mcpp new` — 创建模块化项目;`--template [@ver][:]` 使用**库自带模板**(如 `--template imgui`),`--list-templates ` 列举 - `mcpp run [-- args]` — 构建并运行 -- `mcpp test [pattern] [-- args]` — 自动发现并运行测试(可按名字过滤;`--message-format json` 供工具消费) +- `mcpp test [pattern] [-- args]` — 自动发现并运行测试(按名字过滤;`--list`、`--timeout `、`--message-format json`) - `mcpp search` — 搜索包索引 - `mcpp add / remove / update` — 依赖管理 - `mcpp explain E0001` — 错误码详细解释 diff --git a/docs/00-getting-started.md b/docs/00-getting-started.md index e23f995..5f02f4c 100644 --- a/docs/00-getting-started.md +++ b/docs/00-getting-started.md @@ -82,8 +82,11 @@ showing progress and speed along the way. Once downloaded, all mcpp projects sha ```bash mcpp build # incremental build mcpp clean # clean target/ -mcpp test # compile and run tests/**/*.cpp (gtest style) +mcpp test # compile and run tests/**/*.cpp — one binary per file, + # framework-agnostic (bare main, or gtest via [dev-dependencies]) mcpp test # only tests whose name contains +mcpp test --list # enumerate tests without building +mcpp test --timeout 30 # kill a test still running after 30s ``` ## Adding Dependencies diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 4338c74..fa1ab92 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -328,7 +328,7 @@ qux = ">=1.0, <2.0" # Range combination gtest = "1.15.2" ``` -`mcpp build` ignores these; `mcpp test` resolves and uses them. `mcpp test` automatically discovers `tests/**/*.cpp` and compiles them into test binaries. Tests are named by their `tests/`-relative path (`tests/00-a/0.cpp` → `00-a/0`), each test compiles in isolation (a broken test fails alone; package/dep breakage is reported as a build error instead), and `mcpp test ` / `--message-format json` filter and machine-format the run. +`mcpp build` ignores these; `mcpp test` resolves and uses them. `mcpp test` automatically discovers `tests/**/*.cpp` and compiles them into test binaries. The runner is framework-agnostic: each file is an independent binary judged by exit code — a bare `main`, gtest (via `[dev-dependencies]` + `gtest_main`), or any other framework all work identically, and `-- args` are forwarded to every test binary (e.g. `-- --gtest_filter=...`). Note: synthesized test target names may contain `/` (`tests/00-a/0.cpp` → `00-a/0`), unlike `[targets.*]` names — the two namespaces are intentionally separate (test targets never enter the manifest or publishing). Tests are named by their `tests/`-relative path (`tests/00-a/0.cpp` → `00-a/0`), each test compiles in isolation (a broken test fails alone; package/dep breakage is reported as a build error instead), and `mcpp test ` / `--message-format json` filter and machine-format the run. ### 2.7 `[toolchain]` — Toolchain Configuration diff --git a/docs/zh/00-getting-started.md b/docs/zh/00-getting-started.md index 0cb77ab..992466d 100644 --- a/docs/zh/00-getting-started.md +++ b/docs/zh/00-getting-started.md @@ -83,7 +83,11 @@ mcpp run ```bash mcpp build # 增量构建 mcpp clean # 清理 target/ -mcpp test # 编译并运行 tests/**/*.cpp(gtest 风格) +mcpp test # 编译并运行 tests/**/*.cpp —— 每文件一个独立二进制, + # 框架无关(裸 main,或经 [dev-dependencies] 使用 gtest) +mcpp test # 只运行名字包含 的测试 +mcpp test --list # 只枚举测试,不构建 +mcpp test --timeout 30 # 运行超过 30s 的测试将被终止并计为失败 ``` ## 添加依赖 diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index f7e0f06..0f39723 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -300,7 +300,7 @@ qux = ">=1.0, <2.0" # 范围组合 gtest = "1.15.2" ``` -`mcpp build` 忽略这些;`mcpp test` 解析并使用。`mcpp test` 会自动发现 `tests/**/*.cpp` 并编译为测试二进制。 +`mcpp build` 忽略这些;`mcpp test` 解析并使用。`mcpp test` 会自动发现 `tests/**/*.cpp` 并编译为测试二进制。运行器与断言框架无关:每个文件是一个以退出码判定的独立二进制——裸 `main`、gtest(经 `[dev-dependencies]` + `gtest_main`)或其他框架均等价,`-- args` 会透传给每个测试二进制(例如 `-- --gtest_filter=...`)。注意:合成的测试 target 名可含 `/`(`tests/00-a/0.cpp` → `00-a/0`),与 `[targets.*]` 的命名文法不同——两套命名空间刻意分离(测试 target 不进清单、不参与发布)。 ### 2.7 `[toolchain]` — 工具链配置 diff --git a/src/cli.cppm b/src/cli.cppm index b8c1c1c..d00be73 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -53,7 +53,7 @@ void print_usage() { std::println(" mcpp new Create a new package skeleton"); std::println(" mcpp build [options] Build the current package"); std::println(" mcpp run [target] [-- args...] Build + run a binary target"); - std::println(" mcpp test [-- args...] Build + run all tests/**/*.cpp"); + std::println(" mcpp test [pattern] [-- args...] Build + run tests/**/*.cpp (--list, --timeout, --message-format json)"); std::println(" mcpp clean [--bmi-cache] Remove target/ (and optionally BMI cache)"); std::println(" mcpp add [@] Add a dependency to mcpp.toml"); std::println(" mcpp remove Remove a dependency from mcpp.toml"); From d57b37b2b114411597feb4946a2c40c9d2705bb2 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:30:22 +0800 Subject: [PATCH 19/22] =?UTF-8?q?release:=200.0.104=20=E2=80=94=20mcpp=20t?= =?UTF-8?q?est=20capability=20batch=20(isolation/parallel,=20filter,=20lis?= =?UTF-8?q?t,=20timeout,=20json,=20signal=20fix)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG.md | 17 +++++++++++++---- mcpp.toml | 2 +- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c7b93bb..d274300 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,9 +3,9 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 -## [未发布] +## [0.0.104] — 2026-07-24 -> `mcpp test` 练习级增强批次:逐测试编译隔离、子目录路径命名、过滤器、JSON 输出——四项均为通用能力(cargo/ctest 同形),首个下游消费者是 d2mcpp「练习即测试」重设计(见 d2mcpp 仓 `.agents/docs/2026-07-23-exercises-as-tests-design.md` §4,验收标准即出自该文档)。实施计划见 `.agents/docs/2026-07-23-test-isolation-json-plan.md`。 +> `mcpp test` 能力批次(两轮):逐测试编译隔离、子目录路径命名、过滤器、JSON 输出——四项均为通用能力(cargo/ctest 同形),首个下游消费者是 d2mcpp「练习即测试」重设计(见 d2mcpp 仓 `.agents/docs/2026-07-23-exercises-as-tests-design.md` §4,验收标准即出自该文档)。实施计划见 `.agents/docs/2026-07-23-test-isolation-json-plan.md`。 ### 新增 @@ -15,14 +15,23 @@ - **`mcpp test --message-format json`**:NDJSON 输出,逐测试一条记录流式发射(`test`/`status`=`pass|compile_fail|run_fail`/`exit_code`/`signal`/`compile_output`/`run_output`),包级失败单独 `{"error":"package",...}` 记录,末行 `{"summary":...}`;stdout 纯协议流,人读输出全部静默。供 CI/IDE/d2x Provider 消费。 - **`[build].flags` glob 覆盖测试 TU**:glob 指名文件,是 source 还是 test 是正交的——匹配的条目经既有 per-target flag 通道(#131)挂到合成测试 target 上,per-test 编译选项从此有 toml 承载。配套:死 glob 警告改为「磁盘上无文件匹配」才触发(此前只数被扫描的 source,指向 tests/ 的合法 glob 会被误警)。 +### 新增(批次二,设计评审见 `.agents/docs/2026-07-24-mcpp-test-design-review.md`) + +- **Phase B 并行化**:逐测试隔离改为「keep-going 全量预构建(-k 0,跨测试并行)+ 逐测试缓存命中验证/失败重试取诊断」——语义不变,49 测试全量墙钟 29s → 3.7s。 +- **`mcpp test --list`**:只枚举(可过滤的)测试,不解析工具链、不构建;`--message-format json` 输出 `{"test":…,"main":…}` 逐行记录 + total 汇总,坏文件同样可列出。供 d2x Provider 等工具做单一真相源发现。 +- **`mcpp test --timeout `**:每测试运行截止,超时 SIGKILL 并计为 `FAIL (timeout)` / JSON `"timed_out":true`(POSIX;Windows 暂为尽力而为,不生效)。 +- **JSON 记录增加 `duration_ms`**(该测试构建+运行墙钟);schema 只增不改。 +- **信号退出码规范化**:`WIFSIGNALED → 128+WTERMSIG`(shell 惯例)。此前返回原始 wait status——SIGSEGV 仅因 core-dump 位碰巧显示 139,SIGTERM 死亡会伪装成 `exit 15`;JSON 的 `signal` 字段推导从未可靠触发。属 bug 修复。 + ### 修复 - **嵌套 mcpp 的 `LD_LIBRARY_PATH` 段错误**:外层 `mcpp run` 为其子进程指向私有 glibc payload 的 loader 路径,子进程再 spawn mcpp 时(如课程 Provider 驱动 `mcpp test`),毒化值继续流入内层 mcpp 的工具子进程——sandbox ninja/gcc 加载错配 libc 后在动态链接器里段错误(残片签名 `__vdso_time`)。现在 `merged_environ` 从继承的 `LD_LIBRARY_PATH`/`DYLD_LIBRARY_PATH` 里**只剥离** `xim-x-glibc` payload 条目:用户自己的条目保留,per-child 显式 override 一如既往优先。下游(d2x `platform.cppm`、d2mcpp `runner.cppm`)的 `unsetenv` workaround 可随本版删除。 ### 备注 -- e2e 新增 118(子目录命名)/119(隔离与包级归因)/120(过滤器)/121(JSON)/122(嵌套 loader 路径免疫)/123(测试 TU 吃 glob flags + 死 glob 判定)。 -- 首个下游消费者已完成端到端接入:d2mcpp「练习即测试」迁移(104 练习 zh/en 52/52 全绿)+ d2x checker 闯关链路,全程使用本分支 musl 静态二进制。 +- e2e 新增 118(子目录命名)/119(隔离与包级归因)/120(过滤器)/121(JSON)/122(嵌套 loader 路径免疫)/123(测试 TU 吃 glob flags + 死 glob 判定)/124(信号退出码+duration)/125(--list)/126(--timeout)。 +- 首个下游消费者已完成端到端接入:d2mcpp「练习即测试」迁移(104 练习 zh/en 52/52 全绿)+ d2x checker 引导链路,全程使用本分支 musl 静态二进制。 +- docs 措辞修正:"(gtest style)" → 框架无关表述;记录合成测试名含 `/` 的命名豁免(zh+en)。 - `BuildOptions::ninjaTargets`(构建计划子集)为支撑性通用接口,空 = 原行为。 - musl 静态构建(`--target x86_64-linux-musl`)已验证:五个新 e2e 全绿;静态 mcpp 本体对 loader 毒化免疫,与上述修复共同覆盖嵌套链路的两端。 diff --git a/mcpp.toml b/mcpp.toml index ab063aa..f1b71f1 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "0.0.103" +version = "0.0.104" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] From dfe8868873096d159ed4b96f39ed7255de7f1683 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:44:53 +0800 Subject: [PATCH 20/22] release: sync MCPP_VERSION constant to 0.0.104 --- src/toolchain/fingerprint.cppm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index 826fac1..95598f6 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -18,7 +18,7 @@ import mcpp.toolchain.detect; export namespace mcpp::toolchain { -inline constexpr std::string_view MCPP_VERSION = "0.0.103"; +inline constexpr std::string_view MCPP_VERSION = "0.0.104"; struct FingerprintInputs { Toolchain toolchain; From fcf37435c2c9e5b867ebebf9e3e25e2352de36e9 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 01:48:36 +0800 Subject: [PATCH 21/22] test(e2e): renumber new scripts 118-126 -> 152-160 (numeric collision with pre-existing scripts) --- .agents/docs/2026-07-24-mcpp-test-design-review.md | 2 +- CHANGELOG.md | 2 +- .../e2e/{118_test_subdir_names.sh => 152_test_subdir_names.sh} | 0 tests/e2e/{119_test_isolation.sh => 153_test_isolation.sh} | 0 tests/e2e/{120_test_filter.sh => 154_test_filter.sh} | 0 tests/e2e/{121_test_json.sh => 155_test_json.sh} | 0 ..._nested_ld_library_path.sh => 156_nested_ld_library_path.sh} | 0 tests/e2e/{123_test_glob_flags.sh => 157_test_glob_flags.sh} | 0 ...t_signal_and_duration.sh => 158_test_signal_and_duration.sh} | 0 tests/e2e/{125_test_list.sh => 159_test_list.sh} | 0 tests/e2e/{126_test_timeout.sh => 160_test_timeout.sh} | 0 11 files changed, 2 insertions(+), 2 deletions(-) rename tests/e2e/{118_test_subdir_names.sh => 152_test_subdir_names.sh} (100%) rename tests/e2e/{119_test_isolation.sh => 153_test_isolation.sh} (100%) rename tests/e2e/{120_test_filter.sh => 154_test_filter.sh} (100%) rename tests/e2e/{121_test_json.sh => 155_test_json.sh} (100%) rename tests/e2e/{122_nested_ld_library_path.sh => 156_nested_ld_library_path.sh} (100%) rename tests/e2e/{123_test_glob_flags.sh => 157_test_glob_flags.sh} (100%) rename tests/e2e/{124_test_signal_and_duration.sh => 158_test_signal_and_duration.sh} (100%) rename tests/e2e/{125_test_list.sh => 159_test_list.sh} (100%) rename tests/e2e/{126_test_timeout.sh => 160_test_timeout.sh} (100%) diff --git a/.agents/docs/2026-07-24-mcpp-test-design-review.md b/.agents/docs/2026-07-24-mcpp-test-design-review.md index 6e980af..f459577 100644 --- a/.agents/docs/2026-07-24-mcpp-test-design-review.md +++ b/.agents/docs/2026-07-24-mcpp-test-design-review.md @@ -195,7 +195,7 @@ | # | 事项 | 层 | 验收标准 | |---|---|---|---| -| P1 | Phase B 改 keep-going 预构建 + 存在性归因 + 失败者逐个取诊断 | runner | e2e 118–123 不变全绿;多测试全绿工程 `mcpp test` 构建墙钟时间恢复到与旧单次构建同量级 | +| P1 | Phase B 改 keep-going 预构建 + 存在性归因 + 失败者逐个取诊断 | runner | e2e 152–157 不变全绿;多测试全绿工程 `mcpp test` 构建墙钟时间恢复到与旧单次构建同量级 | | P2 | `mcpp test --list [--message-format json]` | runner | 输出测试名+主文件;d2mcpp Provider 枚举可切换为消费该输出 | | P3 | JSON 记录增加 `duration_ms`;`signal` 改从 wait status 直读(POSIX) | runner | schema 向后兼容(只增字段) | | P4 | 每测试超时 `--timeout `(挂死测试 → run_fail + 标注) | runner | 睡死测试不拖垮整轮 | diff --git a/CHANGELOG.md b/CHANGELOG.md index d274300..532dd00 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,7 +29,7 @@ ### 备注 -- e2e 新增 118(子目录命名)/119(隔离与包级归因)/120(过滤器)/121(JSON)/122(嵌套 loader 路径免疫)/123(测试 TU 吃 glob flags + 死 glob 判定)/124(信号退出码+duration)/125(--list)/126(--timeout)。 +- e2e 新增 152(子目录命名)/153(隔离与包级归因)/154(过滤器)/155(JSON)/156(嵌套 loader 路径免疫)/157(测试 TU 吃 glob flags + 死 glob 判定)/158(信号退出码+duration)/159(--list)/160(--timeout)。 - 首个下游消费者已完成端到端接入:d2mcpp「练习即测试」迁移(104 练习 zh/en 52/52 全绿)+ d2x checker 引导链路,全程使用本分支 musl 静态二进制。 - docs 措辞修正:"(gtest style)" → 框架无关表述;记录合成测试名含 `/` 的命名豁免(zh+en)。 - `BuildOptions::ninjaTargets`(构建计划子集)为支撑性通用接口,空 = 原行为。 diff --git a/tests/e2e/118_test_subdir_names.sh b/tests/e2e/152_test_subdir_names.sh similarity index 100% rename from tests/e2e/118_test_subdir_names.sh rename to tests/e2e/152_test_subdir_names.sh diff --git a/tests/e2e/119_test_isolation.sh b/tests/e2e/153_test_isolation.sh similarity index 100% rename from tests/e2e/119_test_isolation.sh rename to tests/e2e/153_test_isolation.sh diff --git a/tests/e2e/120_test_filter.sh b/tests/e2e/154_test_filter.sh similarity index 100% rename from tests/e2e/120_test_filter.sh rename to tests/e2e/154_test_filter.sh diff --git a/tests/e2e/121_test_json.sh b/tests/e2e/155_test_json.sh similarity index 100% rename from tests/e2e/121_test_json.sh rename to tests/e2e/155_test_json.sh diff --git a/tests/e2e/122_nested_ld_library_path.sh b/tests/e2e/156_nested_ld_library_path.sh similarity index 100% rename from tests/e2e/122_nested_ld_library_path.sh rename to tests/e2e/156_nested_ld_library_path.sh diff --git a/tests/e2e/123_test_glob_flags.sh b/tests/e2e/157_test_glob_flags.sh similarity index 100% rename from tests/e2e/123_test_glob_flags.sh rename to tests/e2e/157_test_glob_flags.sh diff --git a/tests/e2e/124_test_signal_and_duration.sh b/tests/e2e/158_test_signal_and_duration.sh similarity index 100% rename from tests/e2e/124_test_signal_and_duration.sh rename to tests/e2e/158_test_signal_and_duration.sh diff --git a/tests/e2e/125_test_list.sh b/tests/e2e/159_test_list.sh similarity index 100% rename from tests/e2e/125_test_list.sh rename to tests/e2e/159_test_list.sh diff --git a/tests/e2e/126_test_timeout.sh b/tests/e2e/160_test_timeout.sh similarity index 100% rename from tests/e2e/126_test_timeout.sh rename to tests/e2e/160_test_timeout.sh From 8a3cf3c4c6a17f48fd403ddb464658a8acba600e Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Fri, 24 Jul 2026 02:01:16 +0800 Subject: [PATCH 22/22] =?UTF-8?q?test(e2e):=20gate=20signal/timeout=20scri?= =?UTF-8?q?pts=20on=20unix-shell=20=E2=80=94=20POSIX-only=20semantics,=20d?= =?UTF-8?q?ocumented=20Windows=20limitation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- tests/e2e/158_test_signal_and_duration.sh | 2 +- tests/e2e/160_test_timeout.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/e2e/158_test_signal_and_duration.sh b/tests/e2e/158_test_signal_and_duration.sh index 8cd9d2a..03e24a0 100755 --- a/tests/e2e/158_test_signal_and_duration.sh +++ b/tests/e2e/158_test_signal_and_duration.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# requires: +# requires: unix-shell # Signaled tests report shell-convention exit codes (128+sig) with the signal # number in JSON, and every JSON test record carries duration_ms. set -e diff --git a/tests/e2e/160_test_timeout.sh b/tests/e2e/160_test_timeout.sh index c3ee28b..ea8cf7b 100755 --- a/tests/e2e/160_test_timeout.sh +++ b/tests/e2e/160_test_timeout.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# requires: +# requires: unix-shell # `mcpp test --timeout `: a hung test is killed and reported as a # timeout failure; the rest of the suite still runs. set -e