From 9da1815652bd03e842a73dd69d0c757712af9b58 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 27 Oct 2026 23:30:56 +0900 Subject: [PATCH 01/54] 2026.10.27.1: a version the local index lacks is refreshed for when its install is refused; a package whose descriptor was read is the only one asked for Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 20 ++++ docs/05-dependencies.md | 9 +- docs/90-build-from-source.md | 4 +- docs/zh/05-dependencies.md | 3 +- docs/zh/90-build-from-source.md | 4 +- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- src/build/prepare/graph_load.cpp | 27 +++++- ...on_the_index_lacks_refreshes_at_install.sh | 96 +++++++++++++++++++ 9 files changed, 155 insertions(+), 12 deletions(-) create mode 100755 tests/e2e/949_a_version_the_index_lacks_refreshes_at_install.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index cc7425fd7..c9a8e7058 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,26 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [2026.10.27.1] - 2026-10-27 + +### Fixed + +- **A version the local index lacks is refreshed for when its install is + refused.** The walk refreshes the index when a descriptor is missing, and not + when a descriptor is present without the version asked for (version tables + are per system, so that absence is not proof). The provider's "not found" for + the version is, and it now reaches the refresh policy (offline, + `[index] auto_refresh`, the debounce, one sync per run); after a refresh the + install is asked once more. A dependency on a version published after the + index was last synced (`openkal-llvm-runtime = "0.16.3"`) was refused until + `mcpp index update`. e2e 949. +- **A package whose descriptor was read is the only one asked for.** A refused + install of a namespace-omitted dependency was tried again as + `compat:` and `compat.`, shown as downloading, and reported with + three addresses, although the descriptor had named the package in + `mcpplibs`. The compat spellings are tried only when no descriptor could be + read. + ## [2026.10.26.2] - 2026-10-26 ### Added diff --git a/docs/05-dependencies.md b/docs/05-dependencies.md index eaf4fa245..95c0af0dc 100644 --- a/docs/05-dependencies.md +++ b/docs/05-dependencies.md @@ -362,9 +362,12 @@ an error, never a directory-order choice. See the normative template rows in `mcpp build` / `run` / `test` refresh the package index **only when a dependency cannot be resolved from the local copy** — never merely because time has passed. Concretely, a refresh happens when there is no local index at all, when a -dependency's descriptor is missing from it, or when a SemVer constraint matches -none of the versions it knows. A build whose dependencies all resolve locally -makes no network request, however old the local index is. +dependency's descriptor is missing from it, when a SemVer constraint matches +none of the versions it knows, or when the install of an exact version is +refused because the index does not carry it (a version published after the +last sync); after that refresh the install is asked once more. A build whose +dependencies all resolve locally makes no network request, however old the +local index is. The consequence worth knowing: a constraint like `^1.2` resolves against the **the versions the local index knows**. A `1.3.0` published upstream after the last refresh diff --git a/docs/90-build-from-source.md b/docs/90-build-from-source.md index 9af55bba1..029c1391e 100644 --- a/docs/90-build-from-source.md +++ b/docs/90-build-from-source.md @@ -82,7 +82,7 @@ src/ tests/ ├── unit/ 154 C++ tests, discovered by `mcpp test` -└── e2e/ 635 shell scripts against a real binary +└── e2e/ 636 shell scripts against a real binary ``` ## Test Organization @@ -93,7 +93,7 @@ Two layers, and they answer different questions. discovers. They exercise a module's contract with no binary and no filesystem state. -**End-to-end tests** (`tests/e2e/NN_.sh`, 635 files) run a real `mcpp` +**End-to-end tests** (`tests/e2e/NN_.sh`, 636 files) run a real `mcpp` binary against a real project. `run_all.sh` is the CI entry point. `mcpp test` does **not** run them. diff --git a/docs/zh/05-dependencies.md b/docs/zh/05-dependencies.md index 9c4225b54..18d5a4e84 100644 --- a/docs/zh/05-dependencies.md +++ b/docs/zh/05-dependencies.md @@ -325,7 +325,8 @@ package = { `mcpp build` / `run` / `test` **只在依赖无法用本地副本解析时**刷新包索引， 绝不会仅仅因为时间流逝就刷新。具体地说：本地根本没有索引、依赖的描述符不在 -其中，或一条 SemVer 约束在本地已知版本里无解，这三种情况会触发刷新。只要 +其中、一条 SemVer 约束在本地已知版本里无解，或某个确切版本因索引尚未收录 +（上次同步之后发布）而安装被拒，这四种情况会触发刷新；刷新之后安装再请求一次。只要 所有依赖都能在本地解析出来，不论本地索引多旧，构建都不会发出任何网络请求。 由此带来一个值得知道的推论：`^1.2` 这类约束，是对**本地索引已知的那些版本** diff --git a/docs/zh/90-build-from-source.md b/docs/zh/90-build-from-source.md index cbce28e3e..63fef366b 100644 --- a/docs/zh/90-build-from-source.md +++ b/docs/zh/90-build-from-source.md @@ -84,7 +84,7 @@ src/ tests/ ├── unit/ 154 C++ tests, discovered by `mcpp test` -└── e2e/ 635 shell scripts against a real binary +└── e2e/ 636 shell scripts against a real binary ``` ## 测试组织 @@ -94,7 +94,7 @@ tests/ **单元测试**（`tests/unit/`，154 个文件）是 `mcpp test` 发现的 C++ 程序。它们检验一个模块的契约，不需要二进制、也不需要文件系统状态。 -**端到端测试**（`tests/e2e/NN_.sh`，635 个文件）拿真实的 `mcpp` +**端到端测试**（`tests/e2e/NN_.sh`，636 个文件）拿真实的 `mcpp` 二进制跑真实工程。`run_all.sh` 是 CI 的入口。`mcpp test` **不**运行它们。 ```bash diff --git a/mcpp.toml b/mcpp.toml index a863fe0dd..011d9290e 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.10.26.2" +version = "2026.10.27.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 77cb6edf1..eb50e20fb 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.10.26.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.10.27.1"; } // namespace mcpp diff --git a/src/build/prepare/graph_load.cpp b/src/build/prepare/graph_load.cpp index ee28f0e9c..5ea2b49b1 100644 --- a/src/build/prepare/graph_load.cpp +++ b/src/build/prepare/graph_load.cpp @@ -1058,6 +1058,26 @@ step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { mcpp::invocation::ScopedEnv hookVar4(hookVars.at(4).first, hookVars.at(4).second); mcpp::invocation::ScopedEnv hookVar5(hookVars.at(5).first, hookVars.at(5).second); auto r = install_one(target); + // A MISS AT INSTALL REFRESHES THE INDEX, THEN ASKS ONCE MORE. The + // walk refreshes when a descriptor is missing, and deliberately not + // when a descriptor is present without the version asked for: + // version tables are per system, so that absence is not proof. The + // provider's "not found" for that version is, and it is the only + // place the stale case shows (measured: `openkal-llvm-runtime = + // "0.16.3"` against an index synced before 0.16.3, refused until + // `mcpp index update`). The policy decides (offline, `auto_refresh`, + // the debounce, one sync per run). A project's own index is read as + // it is, so it is not refreshed here. + if (r && r->exitCode != 0 && !ctx.useProjectEnv + && mcpp::pm::install_failure_policy(r->error ? r->error->code : std::string{}).refreshIndex) { + const auto env = mcpp::config::make_xlings_env(**cfg); + auto decision = mcpp::pm::decide_for_miss( + mcpp::pm::policy_for(**cfg), env, target, mcpp::pm::RefreshReason::InstallMiss); + if (auto applied = mcpp::pm::apply(decision, env); !applied) + mcpp::ui::warning(applied.error()); + else if (decision.shouldRefresh) + r = install_one(target); + } // The first address's failure is the one about the package; a // compat spelling tried after it fails as "not found" whatever // the cause was. @@ -1067,11 +1087,14 @@ step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { if (r->error) firstError += (firstError.empty() ? "" : "; ") + r->error->message; } - if (r && r->exitCode != 0 && + if (r && r->exitCode != 0 && !ctx.luaContent && (ns.empty() || ns == mcpp::pm::kDefaultNamespace)) { // Compat retry for a bare/default-namespace request whose // descriptor could not be read (no `wireAddr` to trust): the - // package may still be a `compat` one. Try BOTH spellings — a + // package may still be a `compat` one. A descriptor that was + // read names the package, and its failure is that package's: + // another namespace is not tried and not shown as downloading. + // Try BOTH spellings — a // SPEC-001 index keys it `compat:`, a pre-SPEC-001 index // keys it by the literal `compat.`. Sending only the // latter is what left the retry pointing at a name the migrated diff --git a/tests/e2e/949_a_version_the_index_lacks_refreshes_at_install.sh b/tests/e2e/949_a_version_the_index_lacks_refreshes_at_install.sh new file mode 100755 index 000000000..4da38843e --- /dev/null +++ b/tests/e2e/949_a_version_the_index_lacks_refreshes_at_install.sh @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# 949 -- a version the local index lacks is refreshed for when its install is +# refused, and the package the descriptor named is the only one asked for. +# +# The walk refreshes when a descriptor is missing, and deliberately not when a +# descriptor is present without the version asked for: version tables are per +# system, so that absence is not proof. The provider's "not found" for the +# version is. Measured: `openkal-llvm-runtime = "0.16.3"` against an index +# synced before 0.16.3 was refused until `mcpp index update`, after mcpp had +# also tried `compat:openkal-llvm-runtime` and `compat.openkal-llvm-runtime` +# and shown them as downloading. +# +# Hermetic, as 945 is: the builtin index is fabricated, and every case is +# debounced or has `auto_refresh = false`, so the decision is observed +# without a sync. +# +# Criteria: +# A. The refused install reaches the refresh policy, which declines inside +# the debounce window and says so for that address. +# B. With `[index] auto_refresh = false` the same refusal is declined for +# that reason. +# C. A descriptor that was read names the package: no other namespace is +# tried or shown as downloading, and the refusal names one address. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +INDEX="$MCPP_HOME/indices/mcpplibs" +mkdir -p "$INDEX/pkgs/l" +echo "abc1234" > "$INDEX/.xlings-index-version" +# The descriptor is here; the version asked for is not in it. +cat > "$INDEX/pkgs/l/laterpkg.lua" <<'EOF' +package = { + spec = "1", + namespace = "mcpplibs", + name = "laterpkg", + description = "A package whose next version the local index predates", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { + ["1.0.0"] = { + url = "https://example.invalid/laterpkg-1.0.0.tar.gz", + sha256 = "0000000000000000000000000000000000000000000000000000000000000000", + }, + }, + }, +} +EOF + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +cat > mcpp.toml <<'TOML' +[package] +name = "app" +version = "0.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies] +laterpkg = "1.0.1" +TOML +echo 'int main() { return 0; }' > src/main.cpp + +# A +: > "$INDEX/.mcpp-index-updated" +MCPP_VERBOSE=1 "$MCPP" build > a.log 2>&1 && fail "A: the build installed a version no index carries" a.log +grep -qE "skip refresh for mcpplibs:laterpkg@1\.0\.1: index was just refreshed" a.log \ + || fail "A: the refused install did not reach the refresh policy" a.log +grep -qE "Refreshing +package index" a.log && fail "A: a sync inside the debounce window" a.log +echo "ok: a refused install asks for a refresh (declined inside the debounce window)" + +# B +touch -d '30 days ago' "$INDEX/.mcpp-index-updated" 2>/dev/null \ + || touch -A -300000 "$INDEX/.mcpp-index-updated" 2>/dev/null || true +printf '\n[index]\nauto_refresh = false\n' >> "$MCPP_HOME/config.toml" +MCPP_VERBOSE=1 "$MCPP" build > b.log 2>&1 && fail "B: the build installed a version no index carries" b.log +grep -qE "skip refresh for mcpplibs:laterpkg@1\.0\.1: \[index\] auto_refresh = false" b.log \ + || fail "B: auto_refresh = false did not decide the refused install" b.log +echo "ok: with auto_refresh = false the refused install is declined" + +# C +grep -qE "Downloading +compat" a.log b.log && fail "C: another namespace was shown as downloading" a.log +grep -q "wire address tried: mcpplibs:laterpkg@1.0.1$" a.log \ + || fail "C: the refusal does not name the one address asked for" a.log +echo "ok: the package the descriptor named is the only one asked for" + +echo "PASS: 949 a version the index lacks refreshes at install" From a336ef01dcee3e85d2175fb1dc6a5a6f8f651753 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 27 Oct 2026 15:48:01 +0900 Subject: [PATCH 02/54] fix: a kernel-abi package over the system's C library is placed into the shared image that reaches it Every package providing an `mcpp:` layer was a boundary of static placement (#646 F1), so openkal-linux/-macos/-windows went into the programs only. On a hosted target that package is an ordinary static library over the platform's C library (mcpplibs.os depends on it), and a shared image reaching it was left with kal_* undefined: CPython importing an extension module built on mcpplibs.os failed with `undefined symbol: kal_process_close`, and Mach-O and PE refuse such a link. A kernel-abi provider is now a boundary only when a C library of the graph (`mcpp:c-abi`) sits on it. e2e 940: a program over a dependency's shared image; the image defines the kernel-abi package's symbols (fails before the fix), and a program reaching the package itself still links and runs. --- CHANGELOG.md | 9 ++ src/build/plan.cppm | 20 ++- ...system_c_library_is_in_the_shared_image.sh | 139 ++++++++++++++++++ 3 files changed, 167 insertions(+), 1 deletion(-) create mode 100755 tests/e2e/950_a_kernel_abi_package_over_the_system_c_library_is_in_the_shared_image.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index c9a8e7058..e3c735f49 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,15 @@ install is asked once more. A dependency on a version published after the index was last synced (`openkal-llvm-runtime = "0.16.3"`) was refused until `mcpp index update`. e2e 949. +- **A kernel-abi package over the system's C library is placed into the shared + image that reaches it.** Every package providing an `mcpp:` layer was a + boundary of static placement and went into programs only, which is right for + a C library or a C++ runtime. On a hosted target openkal-linux (-macos, + -windows) sits on the system's C library and is an ordinary static library + under `mcpplibs.os`; a shared image reaching it was left with `kal_*` + undefined, and CPython could not load an extension built on `mcpplibs.os` + (`undefined symbol: kal_process_close`). A `kernel-abi` provider is now a + boundary only when a C library of the graph sits on it. e2e 950. - **A package whose descriptor was read is the only one asked for.** A refused install of a namespace-omitted dependency was tried again as `compat:` and `compat.`, shown as downloading, and reported with diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 161f0c187..0a4ad5fc1 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -2536,10 +2536,28 @@ make_plan(const mcpp::manifest::Manifest& manifest, // runtime: `provides = ["mcpp:..."]`) is beneath every image, and // where its objects go is the runtime contract's decision, which // #641 made (`cxx_runtime = { shared = "self-contained" }`). + // + // A KERNEL-ABI PACKAGE UNDER NO C LIBRARY OF THE GRAPH IS NOT A LAYER. + // When the C library is the platform's (glibc, libSystem, the Windows + // CRT), openkal-linux (-macos, -windows) is a static library over it + // that a package (`mcpplibs.os`) depends on like any other. Treated as + // a layer, it went into the programs only, and a shared image that + // reached it was left with `kal_*` undefined: ELF linked it, and a + // host that had not linked the package -- CPython importing an + // extension module -- refused to load it (`undefined symbol: + // kal_process_close`); Mach-O and PE refuse the link. It is beneath + // every image when a C library of the graph (`mcpp:c-abi`, openkal- + // musl) sits on it, and placed like any static package otherwise. + const bool graphHasCLibrary = std::ranges::any_of(packages, [](const auto& pkg) { + return std::ranges::any_of(pkg.manifest.provides, [](const std::string& p) { + return p.starts_with("mcpp:c-abi"); + }); + }); std::set boundaries; for (std::size_t i = 1; i < packages.size(); ++i) { auto const& pm = packages[i].manifest; - const bool layer = std::ranges::any_of(pm.provides, [](const std::string& p) { + const bool layer = std::ranges::any_of(pm.provides, [&](const std::string& p) { + if (p.starts_with("mcpp:kernel-abi")) return graphHasCLibrary; return p.starts_with("mcpp:") || p == "hosted-standard-library"; }); if (layer || mcpp::pack::is_distribution_package(pm)) boundaries.insert(i); diff --git a/tests/e2e/950_a_kernel_abi_package_over_the_system_c_library_is_in_the_shared_image.sh b/tests/e2e/950_a_kernel_abi_package_over_the_system_c_library_is_in_the_shared_image.sh new file mode 100755 index 000000000..c343ceb7c --- /dev/null +++ b/tests/e2e/950_a_kernel_abi_package_over_the_system_c_library_is_in_the_shared_image.sh @@ -0,0 +1,139 @@ +#!/usr/bin/env bash +# requires: elf +# 950 -- a `kernel-abi` package over the platform's C library is linked into a +# shared image that reaches it, like any static package. +# +# THE DEFECT. Every package that provides an `mcpp:` layer was a boundary of +# static placement (#646 F1): never placed into a shared image, its objects +# went into the programs only. That is right for a C library or a C++ runtime, +# whose contract decides where they go. It is not right for openkal-linux +# (-macos, -windows) on a hosted target: there the C library is the system's, +# and the kernel-abi package is a static library over it that `mcpplibs.os` +# depends on like any other. A shared image reaching it was left with `kal_*` +# undefined. ELF linked it, and a host that had not linked the package itself +# could not load it -- measured with CPython importing an extension module +# built on `mcpplibs.os`: +# +# ImportError: .../_openmui.abi3.so: undefined symbol: kal_process_close +# +# and Mach-O and PE refuse the link outright. +# +# Two legs, both a program over a dependency's shared image: +# A the program reaches the kernel-abi package only through the image: the +# image defines its symbols and does not import them, as it does for the +# ordinary static package beside it. +# B the program reaches the package itself too: it links and runs as +# before. +set -e + +MCPP="${MCPP:-mcpp}" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT +cd "$work" +mkdir -p kernel/src sys/src ext/src app/src + +cat > kernel/src/kernel.c <<'EOF' +int kal_fake_answer(void) { return 42; } +EOF +cat > kernel/mcpp.toml <<'EOF' +[package] +name = "fakekernel" +version = "0.1.0" +provides = ["mcpp:kernel-abi=openkal"] + +[targets.fakekernel] +kind = "lib" +sources = ["src/*.c"] +EOF + +cat > sys/src/sys.c <<'EOF' +extern int kal_fake_answer(void); +int sys_answer(void) { return kal_fake_answer(); } +EOF +cat > sys/mcpp.toml <<'EOF' +[package] +name = "fakesys" +version = "0.1.0" + +[targets.fakesys] +kind = "lib" +sources = ["src/*.c"] + +[dependencies] +fakekernel = { path = "../kernel" } +EOF + +cat > ext/src/ext.c <<'EOF' +extern int sys_answer(void); +int ext_answer(void) { return sys_answer(); } +EOF +cat > ext/mcpp.toml <<'EOF' +[package] +name = "ext" +version = "0.1.0" + +[build] +sources = ["src/*.c"] + +[targets.ext] +kind = "shared" + +[dependencies] +fakesys = { path = "../sys" } +EOF + +write_app() { # $1 = an extra dependency line, $2 = what main checks + cat > app/src/main.c < app/mcpp.toml <&1) || { echo "$out" >&2; echo "FAIL: $1: the program must build" >&2; exit 1; } + so="$(find app/target -name 'libext.so' -type f | head -1)" + bin="$(find app/target -path '*/bin/app' -type f | head -1)" + if [ -z "$so" ] || [ -z "$bin" ]; then + echo "$out" >&2; echo "FAIL: $1: libext.so or bin/app was not produced" >&2; exit 1 + fi + "$bin" || { echo "FAIL: $1: the program ran and answered $?" >&2; exit 1; } +} + +# ── A. the image is the package's one home ──────────────────────────────── +write_app "" "ext_answer() == 42" +build_app A +nm --defined-only "$so" | grep -qw sys_answer || { + echo "FAIL: A: the shared image does not define sys_answer (its static dependency)" >&2; exit 1; } +if nm -D --undefined-only "$so" | grep -qw kal_fake_answer; then + nm -D --undefined-only "$so" >&2 + echo "FAIL: A: the shared image imports kal_fake_answer from the program; a host" \ + "that did not link fakekernel cannot load it" >&2 + exit 1 +fi +nm --defined-only "$so" | grep -qw kal_fake_answer || { + echo "FAIL: A: the shared image does not define kal_fake_answer" >&2; exit 1; } +echo "OK: A (the shared image defines the kernel-abi package's symbols)" + +# ── B. the program reaches the package itself ───────────────────────────── +write_app 'fakekernel = { path = "../kernel" }' "ext_answer() == 42 && kal_fake_answer() == 42" +build_app B +echo "OK: B (a program reaching the package links and runs)" From 5a33a2e4248f91928bc364e512abd39e222f9e5a Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 00:40:37 +0900 Subject: [PATCH 03/54] docs: 637 end-to-end tests Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/90-build-from-source.md | 4 ++-- docs/zh/90-build-from-source.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/90-build-from-source.md b/docs/90-build-from-source.md index 029c1391e..bf926cb2a 100644 --- a/docs/90-build-from-source.md +++ b/docs/90-build-from-source.md @@ -82,7 +82,7 @@ src/ tests/ ├── unit/ 154 C++ tests, discovered by `mcpp test` -└── e2e/ 636 shell scripts against a real binary +└── e2e/ 637 shell scripts against a real binary ``` ## Test Organization @@ -93,7 +93,7 @@ Two layers, and they answer different questions. discovers. They exercise a module's contract with no binary and no filesystem state. -**End-to-end tests** (`tests/e2e/NN_.sh`, 636 files) run a real `mcpp` +**End-to-end tests** (`tests/e2e/NN_.sh`, 637 files) run a real `mcpp` binary against a real project. `run_all.sh` is the CI entry point. `mcpp test` does **not** run them. diff --git a/docs/zh/90-build-from-source.md b/docs/zh/90-build-from-source.md index 63fef366b..5c453e510 100644 --- a/docs/zh/90-build-from-source.md +++ b/docs/zh/90-build-from-source.md @@ -84,7 +84,7 @@ src/ tests/ ├── unit/ 154 C++ tests, discovered by `mcpp test` -└── e2e/ 636 shell scripts against a real binary +└── e2e/ 637 shell scripts against a real binary ``` ## 测试组织 @@ -94,7 +94,7 @@ tests/ **单元测试**（`tests/unit/`，154 个文件）是 `mcpp test` 发现的 C++ 程序。它们检验一个模块的契约，不需要二进制、也不需要文件系统状态。 -**端到端测试**（`tests/e2e/NN_.sh`，636 个文件）拿真实的 `mcpp` +**端到端测试**（`tests/e2e/NN_.sh`，637 个文件）拿真实的 `mcpp` 二进制跑真实工程。`run_all.sh` 是 CI 的入口。`mcpp test` **不**运行它们。 ```bash From a1a7ba3a1d6dc37105074d2db146ce57a9c9039e Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 03:29:10 +0900 Subject: [PATCH 04/54] design: platforms -- a target class's configuration as a versioned package (SPEC-013), openkal's as the first, and the IDE line and mcppls in the same batch Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .agents/docs/2026-10-28-platforms-design.md | 557 ++++++++++++++++++++ src/okimage/okimage.cppm | 124 ----- 2 files changed, 557 insertions(+), 124 deletions(-) create mode 100644 .agents/docs/2026-10-28-platforms-design.md delete mode 100644 src/okimage/okimage.cppm diff --git a/.agents/docs/2026-10-28-platforms-design.md b/.agents/docs/2026-10-28-platforms-design.md new file mode 100644 index 000000000..1285bbb49 --- /dev/null +++ b/.agents/docs/2026-10-28-platforms-design.md @@ -0,0 +1,557 @@ +--- +subject: platform +status: accepted +--- + +# 平台：目标体系的通用扩展；openkal 作为第一个平台包；IDE 一线与 mcppls 的同批升级 + +日期：2026-10-28。前文：[part 1](2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md)（控制契约、IDE 生态规划 §13–§14）、 +[part 2](2026-10-26-ecosystem-part2-os-library-over-openkal-design.md)（ok 映像与 `-openkal`）。 +基线：mcpp `7d704820`（2026.10.26.2）+ release PR [#847](https://github.com/mcpp-community/mcpp/pull/847)（含 #846）；openkal 0.17.1 轮 +（openkal-linux 0.18.2、openkal-macos 0.15.1、openkal-windows 0.13.3、openkal-musl 0.22.3、openkal-llvm-runtime 0.16.3、openkal-image 0.1.1）； +mcpplibs.os 0.2.0；mcppls 0.0.18（[#72](https://github.com/Sunrisepeak/mcpp-language-server/pull/72)，未合入）；mcpp-ide 本地仓库 `90ae383`； +mcpp-vscode 0.8.0；mcpp-plugins [#51](https://github.com/mcpp-community/mcpp-plugins/pull/51)（0.20.0，未合入）。 +相关规范：SPEC-002、004、006、007、011、012。 + +**修订记录** + +- rev 1（2026-10-27）：openkal 内置目标栈（`[openkal]` 表、`src/openkal/`）。 +- rev 2（2026-10-28）：通用平台机制与 SPEC-013；openkal 是第一个平台包；板级支持包与目标表一并迁移。 +- rev 3：第二轮评审——键表精简为七组（§4）；宿主图与目标图分别选择平台，构建程序可以作为 ok 映像由 `okrun` 运行（§6）； + mcppls（含语义高亮的时延，§11.1）、mcpp-ide、mcpp-clion、mcpp-vscode、mcpp-plugins 纳入同批，作为生态级方案（§11）； + #846 并入 #847；每仓库一个 PR、commit 作检查点的交付计划（§13）；多视角自审（§15）。 +- **rev 4（本稿）**：第三轮评审——§16 的 1–7 定案；mcpp-clion 以插件依赖的形式依赖 mcppls 的 CLion 插件，语言功能只由 mcppls 提供 + （§11.2）；VS Code 与 JetBrains Marketplace 的发布由维护者上传，交付物为产物路径（§13.3）。§15 按定案重审，§16 改为本轮待确认项。 +- **rev 5**：§16 定案（VS Code 以 `extensionPack`、CLion 以 `` 引入 mcppls；上传交付物为本机验证后的产物目录）；开始实施。 + 实施中发现两处缺口，列入同批：os 0.2.0 在宿主三元组上以 openkal-musl 为 C 库时不能编译（L9，os 0.2.1）；M3 载体的链接取到宿主 GCC 的 + `crtbeginT.o`（L10，§4.7）。 + +## 0. 摘要 + +**平台**是一类目标的一份可复用、带版本、可发布的配置：工具链、目标侧各层的默认提供者、本机工具、编译与链接约定、 +产物命名与封装、运行器。平台包在自己的清单中以 `[platform]` 段声明；使用者在 `[target.]`（目标图）或 +`[host]`（宿主图）中以同名键覆盖；构建程序在平台阶段以 SPEC-007 指令陈述的内容与之同义。引擎只提供机制：选择、 +四层合并、把默认提供者放进依赖图、执行封装、报告来源。 + +openkal 由数据包 `openkal-platform` 描述；目标表为 `-openkal` 钉住它的一个版本，即"内置支持"。`mcpp build|run|test|pack +--target x86_64-openkal` 不写任何声明即可工作；引擎中不再有 openkal 专属代码。宿主图同样可以选择平台：选择 openkal 平台时， +构建程序编译为本机架构的 ok 映像，由 `okrun` 运行，在三个系统上行为与产物一致。 + +同批升级 IDE 一线：mcppls 修正大文件语义高亮远慢于补全的问题并迁移到平台；新仓库 mcpp-ide 交付后端、VS Code 适配层 +（沿用扩展 ID `mcpp-vscode`）与 JetBrains 适配层（`mcpp-clion`）；mcpp-vscode 转为指向 mcpp-ide。 + +## 1. 问题（实测） + +| 编号 | 观察 | 出处 | +|---|---|---| +| P1 | 净室中 `mcpp new hello && mcpp run --target x86_64-openkal` 需手写 `openkal-tools` 与 `openkal-llvm-runtime` 四行。 | 2026-10-27 净室，2026.10.26.2 | +| P2 | 缺少 `okld` 的提示写死 `openkal-tools = "0.1.0"`。 | `src/okimage/okimage.cppm` | +| P3 | 两个 openkal 版本同图时，多版本改名只改 C++ 模块名，不改 C 头文件与链接符号，故障出现在无关位置。 | part 1 §25.8 | +| P4 | 精确版本在本地索引中缺失时安装失败不刷新，并尝试 `compat:` 拼写。 | 用户报告；#847 | +| P5 | os、mcppls 在清单中重复声明运行时与工具，CI 从源码自建 `okrun`。 | os、mcppls | +| P6 | 承载 openkal 行为的引擎代码有六处：`flags`、`ninja_backend`、`prepare/plan`、`execute`、`pack`（按包名找 `openkal-tools`）、`triple`。 | `grep` | +| P7 | 报告把目标默认的运行时标为 `graph`。 | targetside 报告 | +| P8 | 体系知识分布在五种互不相通的机制：目标表行、`sysroot` 覆盖、payload 默认运行器、板级支持包的构建程序指令、`[target.]` 键。 | docs/04、22、32、34、41 | +| P9 | 构建程序由宿主工具链按宿主约定编译：同一 `build.mcpp` 在 Windows 上由 MSVC、在 Linux 上由 GCC 编译，行为与诊断随系统不同，缓存不能跨系统复用。 | SPEC-007 §10 | +| P10 | mcppls 中，大文件的补全很快生效，语义高亮往往晚很多（§11.1 分析）。 | 用户报告 | +| P11 | IDE 一线（part 1 §14）尚未交付：mcpp-ide 只有本地后端，无远端仓库；VS Code 与 JetBrains 适配层为空目录；mcpp-vscode 仍是独立实现。 | mcpp-ide `90ae383` | + +## 2. 原则 + +1. **目标的属性归平台，平台归生态。** 引擎不持有体系知识。 +2. **引擎只提供机制与规范。** +3. **一套词汇，三种写法同义**：平台包声明约定，使用者声明覆盖，构建程序陈述需要计算的部分。先例：R9.9。 +4. **声明优先，代码兜底。** +5. **内置即钉住**：内置支持是目标表中的一个版本号。 +6. **宿主与目标对称**：两张图各自选择平台，机制相同；宿主图的平台默认是宿主约定。 +7. **一致、稳定、可观察**：同一份报告、同一个锁、同一组优先级；每个值可追溯来源；拒绝时给出键、来源与出路。 + +## 3. 词汇 + +| 词 | 含义 | +|---|---| +| 目标（target） | 产物运行的机器与格式，以三元组命名 | +| 平台（platform） | 一类目标的配置包（`openkal-platform`、一块开发板的支持包）；区别于构建配置 `--profile` | +| 目标图 / 宿主图 | 为目标构建的包；为构建机器构建的程序（构建程序、工具） | +| 层（layer） | `kernel-abi`、`c-abi`、`c++-abi`、`compiler-runtime`（SPEC-002） | +| 工具（tool） | 在构建机器上运行的程序（`okld`）；区别于随产物发布的 `artifacts`（R6.4） | +| 封装（seal） | 链接之后、发布之前对产物本身的处理与检查；区别于 `artifact` 角色（生成另一个文件） | +| 运行器（runner） | `mcpp run` / `mcpp test` 执行产物的方式；`runners.` 为命名的设备动作 | +| 载体（carrier） | 把依赖加载器的产物与加载器合成一个本机程序的宿主程序包（M3） | + +## 4. 平台包规范（SPEC-013 要点，精简后的七组键） + +```toml +[platform] +targets = ["", …] # 服务的目标 +toolchain = "" # 约定的工具链 +sysroot = … # 预构建的目标基（可选；同 [target].sysroot） +runner = ["", "{}"] # 或 { cmd = […], host = "same-arch" | "any", exclusive = bool } +runners = { = [ … ] } # 命名的设备动作（flash、monitor…） +carrier = { package = "", section = "<节名>" } # M3 载体（可选） +plugin = "<模块>" # 平台阶段的插件代码（可选） + +[platform.layers] = <依赖规格> | false +[platform.tools] = <依赖规格> | { program = "<名字>" } +[platform.build] cflags / cxxflags / asmflags / ldflags = [ … ] +[platform.build.'cfg(<谓词>)'] 同上，条件化（沿用 cfg 语法，例如 arch = "aarch64"） +[platform.link] driver = "compiler" | "lld"; flags = [ … ]（driver = "lld" 时直接给 ld.lld 的参数） + start = "<符号>"; script = "<包内路径>" + inputs = [ { file = "<包内路径>" } | { from = [argv…], as = "<名字>" } ] + exe = "{name}."; shared = "lib{name}." + seal = [ [argv…], … ] +[platform.versions] <包> = "<版本>" # 一组相互一致的版本（§8.1） +[platform.hosted] toolchain、layers # 宿主三元组选用本平台时适用的子集（§5.4） +``` + +相对 rev 2 的精简：`platform-spec`、`requires-mcpp` 删去，改用 `[package] mcpp`（R9.8，平台规范版本随引擎版本）；`round` / +`round-members` 合为 `versions`（平台包自身的版本就是版本组的名字）；`artifact.*` 与 `seal` 并入 `link`（"产物怎样生成"是一件事）； +`build.` 改用已有的 cfg 子表；`run.host`、`run.exclusive` 并入 `runner` 的表形式；`pack.carrier` 改为顶层 `carrier`。 +功能不变：rev 2 的每个键都有对应。 + +约定沿用已有写法：argv 中 `{}` 为产物；程序以名字书写，按"图中为本机构建的程序 → 声明的载荷 → `PATH`"查找。 + +### 4.1 三种写法同义 + +平台包写约定（`[platform]`），使用者写覆盖（`[target.]`、`[host]`，键名相同），构建程序在平台阶段以指令陈述可变部分 +（新增指令 `link_driver`、`link_input`、`artifact_name`、`seal`、`run_host`，与已有的 `cxxflag`、`link_flag`、`link_start`、`link_script`、 +`runner`、`run_exclusive` 同列）。三者解析为同一结构 `PlatformStatement`，每个字段带来源。 + +### 4.2 声明与代码的分界 + +- **只能声明**：`targets`、`toolchain`、`sysroot`、`layers`、`tools`、`runner`、`runners`、`carrier`、`versions`、`hosted`——解析工具链与依赖图 + 时需要，那时没有构建程序可运行；`self env`、IDE、报告不经构建即可读取。 +- **首选声明**：`build`、`link`。 +- **插件代码**（`plugin`）：依使用者的图、特性或机型而变的部分；在 `platform` 阶段运行（与 R9.9 的 `toolchain` 阶段并列），只能陈述 + `build`、`link`、`runner`、`runners` 的指令，其他指令被拒绝并点名；遵守 SPEC-007 的义务；结果按身份缓存。 + +### 4.3 层的默认提供者与让位 + +`layers.` 作为根的依赖边在依赖遍历前加入，来源 `platform`。图中已有包以 `provides` 提供该层时不加入；一个包提供多层只加入一次； +`false` 表示不提供。判断读 `provides`，不比较包名。 + +### 4.4 工具 + +依赖规格：为本机构建（等价于带 `tools = [...]` 的构建依赖，同包的工具只构建一次）；`{ program }`：由载荷或 `PATH` 提供，引擎报告所用的 +程序（承担 R6.2 的报告义务）。图中已有为本机构建的同名程序时让位。 + +### 4.5 合并 + +标志按"平台 → 本机配置 → 项目"拼接，约定在前，使用者只能追加（需要不同约定时换平台）；`link.seal`、`link.inputs` 整体替换；其余按字段替换。 + +### 4.6 封装 + +`link.seal` 的步骤在链接器写出的临时产物上依次执行（`{}` 为它）；全部成功后原子发布（SPEC-011），任一步失败则不发布、保留上一次发布的 +产物，错误给出该步 argv 与输出；步骤属于链接边的命令行。 + +### 4.7 载体（M3） + +`mcpp pack --mode self-contained` 为本机构建 `carrier.package`，以 `.incbin` 把产物放入其 `section` 节（ELF、Mach-O、COFF 三种汇编由引擎生成）。 +取代 `pack.cppm` 中按包名找 `openkal-tools` 的代码；载体版本写入锁。 + +## 5. 目标图：选择与覆盖 + +### 5.1 选择 + +第一个给出答案的胜出：① `--platform <规格>` / `MCPP_TARGET__PLATFORM`（`none` 不用平台）；② `[target.] platform`（工作空间根被成员继承）； +③ `config.toml` 同键；④ 目标表的 `platform` 列；⑤ 宿主目标的内置约定。`platform` 不能写在 `cfg(platform = …)` 选择器下。 + +### 5.2 覆盖 + +```toml +[target.'cfg(platform = "openkal")'] # 两个架构共用一份 +layers.c-abi = { path = "../openkal-musl" } +tools.okld = { path = "../openkal-image/tools" } +runner = ["my-okrun", "{}"] +``` + +四层优先级与 `sysroot` 相同：`MCPP_TARGET__` > 项目 > 本机 > 平台包。 + +### 5.3 平台作为依赖 + +被选中的平台包是该配置下根的依赖：其库目标可被 `import`（板级支持包选为平台即可用其模块）。旧写法（板级支持包作为普通依赖、 +由构建程序发出 `runner`、`link_script`）继续有效，按 §4.1 合并。 + +### 5.4 宿主三元组选用平台 + +宿主三元组（`x86_64-linux`、`x86_64-windows-gnu`）只适用平台 `hosted` 子表中的 `toolchain` 与 `layers`；无 `hosted` 的平台被拒绝并说明它只服务其 +`targets`。docs/24 的 "openkal 交叉" 因此写作 `[target.'cfg(any(linux, macos, windows))'] platform = "openkal-platform@0.17.1"`；手写 +`openkal-llvm-runtime` 的旧写法由让位规则保持有效。 + +## 6. 宿主图：选择平台，构建程序作为 ok 映像 + +### 6.1 模型 + +一次构建有两张图：目标图为 `--target` 构建；宿主图为构建机器构建构建程序（`build.mcpp`、平台插件）与本机工具。两张图各自有一个平台， +机制相同（选择、合并、注入、报告）： + +```toml +[host] # mcpp.toml 或 config.toml +platform = "openkal-platform@0.17.1" # 宿主图的程序编译为 <本机架构>-openkal，由 okrun 运行 +``` + +选择顺序：`--host-platform` / `MCPP_HOST_PLATFORM` > 项目 `[host]` > 本机 `config.toml [host]` > 默认（宿主约定，即今天的行为）。 + +### 6.2 自举边界 + +宿主图选用平台 P 时，P 自己的工具与运行器（`okld`、`okrun`）总是按宿主约定原生构建：运行器不能由它自己运行。规则：**一个平台的 +`tools` 与 `runner` 所需程序，在任何图中都按宿主约定构建**。没有循环：P 的工具属于原生宿主图，构建程序属于 P 的宿主图。 + +### 6.3 意义（为何做） + +- **一致**：构建程序在 Linux、macOS、Windows 上由同一工具链（llvm）以同一约定编译，读同一个 openkal，行为、诊断与产物字节一致（P9）。 + Windows 上编译构建程序不再需要 MSVC。 +- **缓存跨系统**：同一架构的构建程序映像与系统无关，身份不含宿主系统；全局缓存与 CI 缓存可在三个系统间共享。 +- **对称**：宿主图与目标图是同一机制的两个实例，没有为构建程序另设的通道。 + +### 6.4 约束与默认值 + +- 构建程序的协议（标准输出上的指令、`MCPP_*` 环境、文件读写、查询用的进程启动）全部经 openkal 提供，映像内可用；依赖宿主原生库的 + 构建程序不能编译为映像，此时报错点名该依赖与 `[host] platform` 键。 +- 本批交付机制、验收与测量：e2e 覆盖构建程序、`build.mcpp` 的库、平台插件三类宿主程序作为映像运行；mcpp-index 的全量构建以 + `MCPP_HOST_PLATFORM=openkal-platform@0.17.1` 运行一次并报告结果（§13 mcpp-index）。默认值保持宿主约定；当测量显示索引中的构建程序全部 + 通过时，默认值改为 openkal 平台，那是一行数据（目标表的宿主平台列），不需要新的设计。 + +## 7. 内置平台 + +- **宿主目标**（Linux、macOS、Windows、Android、iOS、Emscripten）：约定留在引擎中（链接 ELF、Mach-O、PE 所必需），以平台的形状报告 + （`Platform host conventions (built-in)`），其 `pin`、`sysroot` 列即内置平台的 `toolchain`、`sysroot`，使用者以同名键覆盖。 +- **钉住的平台包**：目标表增加 `platform` 列，`-openkal` 写 `"mcpplibs.openkal-platform@0.17.1"`；增加宿主平台列，默认空（宿主约定）。 +- **获取**：平台包按普通包解析、下载、缓存、锁定，在工具链解析前读取清单；无 `plugin` 的平台包不编译。`mcpp toolchain install --target ` + （及 `--host`）预取一个配置的全部内容，之后可 `--offline` 构建。 + +## 8. 一致性与稳定性 + +### 8.1 版本组 + +图中属于平台 `versions` 的包被解析为不同版本时拒绝，消息列出双方请求链与两条出路：换平台版本，或固定造成冲突的包。 + +### 8.2 多版本改名的边界（一般规则） + +改名只改 C++ 模块名，只对接口全由具名模块构成的包成立。导出头文件目录的包、提供目标侧层的包及其依赖不改名，两版本并存时拒绝并列出请求者。 + +### 8.3 稳定性的其他保证 + +- 平台包的身份（名字、版本、来源）进入构建配置的指纹：换平台即换构建目录，不复用不相容的产物（与 2026.10.26.2 的暂存修正一起， + 杜绝升级后残留旧产物）。 +- 平台的约定只能追加，不能被使用者删除；需要不同约定时换平台，变化因此总是显式的、可在报告中看到的。 +- 锁记录平台与其注入的包（`origin = "platform"`）；引擎升级不悄然替换已锁定的版本，`mcpp update` 才前移。 +- 平台包获取、工具构建、封装、平台插件的失败都在发生处报告，带键与来源；网络问题遵循 #847 的刷新策略。 + +## 9. 命令、输出与诊断（可观察性） + +- 报告（所有目标同构）： + +``` + Target x86_64-openkal → x86_64-unknown-none-elf + Platform openkal-platform 0.17.1 (built-in) + kernel-abi openkal (the loader, okrun) + c-abi musl (openkal-musl@0.22.3, platform) + c++-abi libc++ (openkal-llvm-runtime@0.16.3, platform) + tools okld, okrun (openkal-tools@0.1.1, platform) + runner okrun (platform) +``` + + 宿主图选用了平台时多一行 `Host platform openkal-platform 0.17.1 (config.toml)`。来源取值：`built-in`、`platform`、`config.toml`、`mcpp.toml`、 + `MCPP_*`、`--platform`、`graph`。 +- `--message-format json`：`plan` 事件增加 `platform` 与 `hostPlatform` 对象（名字、版本、来源、`fields` 中每个字段的值与来源）。 +- `mcpp self env --target [--host] [--format json]`；`mcpp toolchain list` 增加平台列；`mcpp toolchain install --target `；`mcpp self doctor` + 检查平台就绪与版本组；`-v` 下每个字段的决定以 `decision` 记录（R6.5 的同一机制）。 +- 诊断：每条拒绝给出键、来源与出路，不写死版本号；新增错误码进入 `mcpp self explain`；格式遵守 SPEC-012。首次使用平台时的获取在 + 状态行中可见（`Fetching platform openkal-platform 0.17.1`）。 + +## 10. openkal-platform + +### 10.1 它做什么 + +一个**只含数据、不编译**的包，内容是 openkal 生态对 `-openkal` 的全部约定（映像的编译与链接标志、入口、注释节、`okld note/check`、 +`.oke`/`.okso`、`okrun`、M3 载体）与默认版本（运行时、工具、实现）。今天前者写在引擎里，后者要求每个项目手写。它跟随 openkal 轮次发布 +（0.17.1），只改约定时发 `0.17.1.1`；带验收测试（两个目标各构建一个最小程序，在本机架构运行）。mcpp 只钉住它的版本；项目也可以不等 +mcpp，自己写 `platform = "openkal-platform@0.18.0"`。 + +### 10.2 内容 + +```toml +[package] +namespace = "mcpplibs" +name = "openkal-platform" +version = "0.17.1" +mcpp = ">=<本批 mcpp 版本>" + +[platform] +targets = ["x86_64-openkal", "aarch64-openkal"] +toolchain = "llvm@23.1.3" +runner = { cmd = ["okrun", "{}"], host = "same-arch" } +carrier = { package = "openkal-run@0.1.1", section = ".oke" } + +[platform.layers] +compiler-runtime = "openkal-llvm-runtime@0.16.3" +c++-abi = "openkal-llvm-runtime@0.16.3" +c-abi = { name = "openkal-musl", version = "0.22.3", default-features = false, features = ["posix"] } + +[platform.tools] +okld = "openkal-tools@0.1.1" +okrun = "openkal-tools@0.1.1" + +[platform.build] +cflags = ["-fPIC", "-femulated-tls", "-fvisibility=hidden", "-fasynchronous-unwind-tables", "-Xclang", "-fstack-clash-protection"] +cxxflags = ["-fPIC", "-femulated-tls", "-fvisibility=hidden", "-fasynchronous-unwind-tables", "-Xclang", "-fstack-clash-protection"] +[platform.build.'cfg(arch = "aarch64")'] +cflags = ["-ffixed-x18", "-mno-outline-atomics"] +cxxflags = ["-ffixed-x18", "-mno-outline-atomics"] + +[platform.link] +driver = "lld" +start = "ok_main" +flags = ["-shared", "-z", "max-page-size=65536", "-z", "now", "-z", "noexecstack", "--hash-style=both", "--gc-sections", "--eh-frame-hdr"] +inputs = [{ from = ["okld", "note-source"], as = "ok-notes.S" }] +exe = "{name}.oke" +shared = "lib{name}.okso" +seal = [["okld", "note", "{}"], ["okld", "check", "{}"]] + +[platform.hosted] +toolchain = "llvm@23.1.3" +layers = { compiler-runtime = "openkal-llvm-runtime@0.16.3", c++-abi = "openkal-llvm-runtime@0.16.3", + c-abi = { name = "openkal-musl", version = "0.22.3", default-features = false, features = ["posix"] } } + +[platform.versions] +openkal = "0.17.1" +openkal-linux = "0.18.2" +openkal-macos = "0.15.1" +openkal-windows = "0.13.3" +openkal-musl = "0.22.3" +openkal-llvm-runtime = "0.16.3" +openkal-ld = "0.1.1" +openkal-run = "0.1.1" +openkal-tools = "0.1.1" +``` + +`link.flags` 是 `build.ldflags` 之外、只给 `driver = "lld"` 的链接器参数（`ld.lld` 直接调用时不经编译器驱动的 `-Wl,` 转写）。每个值取自今天的引擎 +或已发布版本，无新约定。`-ffreestanding`（图中无 C 库）、`__OPENKAL__`（由 `kernel-abi` 取值定义）、无 C++ 运行时时拒绝 `import std`，本来就是按层 +判断的通用规则，留在引擎。 + +### 10.3 仓库与发布顺序 + +新仓库 `mcpplibs/openkal-platform`（GitCode 镜像 `mcpp-res/openkal-platform`）。它引用 openkal-llvm-runtime，后者经 openkal-musl 依赖 openkal-image +的 openkal-ld，同仓会形成发布环，因此独立，是一轮中最后发布的包：实现 → openkal-image → openkal-musl → openkal-llvm-runtime → openkal-platform → 索引 → +mcpp 前移钉住。 + +## 11. 生态（同批，每仓库一个 PR） + +### 11.1 mcppls：语义高亮的时延（P10）、平台迁移与 0.0.18 内容 + +**分析（代码事实）。** + +| 环节 | 补全（快） | 语义高亮（慢） | +|---|---|---| +| 是否等待打字停顿 | 否 | 是：`textDocument/semanticTokens/*` 在 `builds_main_file_` 列表中（`src/engine/clangd.cpp:2261`），打字期间等待停顿 `typing_quiet_`（上次构建时长 ×1.2，夹在 300–1000 ms） | +| clangd 所需 | 前导（preamble）加当前文本，不建主文件 AST | 主文件的完整 AST；大文件的 AST 构建是秒级 | +| mcppls 的快速路径 | 有：`answered_from_word`（同一词内复用上一答案并重定位）、`answered_late`（迟到答案在同词内立即给出，C-2） | 无：没有上一结果的缓存，打开文件或 AST 重建期间什么也不返回 | +| 增量 | 不适用 | 无：声明 `full: true`，不声明 `delta`；客户端发来的 `full/delta` 被改写成 `full`（`workspace.cpp:1102`），每次传输整份 token | +| mcppls 自身的处理 | 轻 | `remap_core_tokens` 与 `merge` 各做一次 decode → vector → nlohmann JSON 数组的 encode；`merge` 对每个原生 token 扫描全部 core token（O(N×M)） | + +结论：高亮慢于补全的主因是**高亮必须等主文件 AST**，而补全不需要；mcppls 又没有为高亮做补全那样的上一结果复用，也没有增量。约束：不动 clangd。 + +**方案（mcppls 内）。** + +- **H1 上一结果即时返回。** 每个打开文档保存最近一次 clangd 的 token（已映射到本服务器的图例）与它所对应的文本版本。请求到达而 AST 未就绪时， + 以上一结果立即答复，按两版本之间的编辑重定位（行列平移；被编辑的行内 token 丢弃），然后在 AST 就绪后发 `workspace/semanticTokens/refresh` + （引擎已有合并后的 refresh 机制，`engine.cppm:145`）。客户端不支持 refresh 时退回等待。 +- **H2 持久缓存。** 以（文档内容摘要、编译命令摘要、mcppls 与 clangd 的版本、模块 BMI 的身份）为键，把 token 写入工作区缓存目录（遵守 C-8 + 的预算）；打开文件时键命中即立即答复，再走 H1 的刷新。键中任何一项不同即视为未命中，保证不显示错误高亮。 +- **H3 增量。** 声明 `full: { delta: true }`，为每次答复分配 `resultId`，对 `full/delta` 以上一结果计算编辑（LSP 的 SemanticTokensEdit），大文件 + 只传变化部分。 +- **H4 处理成本。** `merge` 改为两路有序合并 O(N+M)；remap、merge、encode 在一次遍历中完成，直接写出整数数组，不经中间 JSON。 +- **H5 首屏。** 客户端发 `range` 请求时（VS Code 对可见区域先发），H1/H2 的结果按范围裁剪即时答复；mcppls 自身的模块语法 token（无需 AST） + 总是立即可用。 +- **测量与判据。** devtools bench 增加任务：一个 1 万行、导入多个模块的文件，记录 didOpen 到首个补全、首个高亮（H1/H2 前后）、编辑后高亮更新 + 的时延，以及每次答复的字节数；CI 的 bench-validate 校验基线与参考对。判据：打开已缓存文件时首个高亮不晚于首个补全；编辑后高亮答复的 + 字节数与编辑规模成比例（H3）。 + +**操作系统层只来自 mcpplibs.os。** 与 mcpp 2026.10.26.2 删除 `modules/platform` 相同，mcppls 删除自己的 `modules/os` 与 `modules/platform`，直接使用 +mcpplibs.os；只在确属 mcppls 自身需要处保留基于 mcpplibs.os 的最小封装。通用且适合进入 mcpplibs.os 的接口提交到 os 本批的 PR（mcpplibs/os#7，0.2.1）， +带测试，供其他程序复用。 + +**同一 PR 的其他内容。** #72（0.0.18：服务器建于 mcpplibs.os、每架构一个 ok 映像、读取 mcpp 的结果记录）的内容；删除只为目标服务的 openkal +声明（运行时、工具、ld），映像以零声明的 `--target -openkal` 构建；CI 改用新 mcpp、`mcpp toolchain install --target`；CLion 插件 e2e 断言 +mcpp 工程不被 Compilation Database 接管（issue #71 的第三项）。版本 0.0.18，发布于 mcpp 之后。 + +### 11.2 mcpp-ide（新仓库 `mcpp-community/mcpp-ide`） + +按 part 1 §14 的结构与边界交付全部内容，并加入平台： + +| 部分 | 内容 | +|---|---| +| 后端（已有，补全） | 会话、监听（结果记录、manifest）、编排（同工程串行、取消、新取代旧）、视图模型、设置校验与迁移、自检、i18n、能力协商；新增：目标与平台视图（来自 `mcpp toolchain list --format json` 与 `plan` 事件的 `platform`），`self env --target` 定位运行器 | +| `protocol/` | JSON Schema 与取自本批 mcpp 真实输出的 golden fixture（含 `platform` 对象） | +| `spec/` | 命令表、设置、菜单、i18n；由 `build.mcpp` 生成 VS Code `package.json` 声明部分与 `plugin.xml`（不提交生成物） | +| `webui/` | 依赖图、库、缓存、设置、打包、目标与平台面板；VS Code webview 与 JetBrains JCEF 加载同一份 | +| `vscode/` | 由 mcpp-vscode 0.8.0 迁移：Task（PTY）、Testing API、调试（cppdbg / CodeLLDB / lldb-dap；ok 映像经 `okrun` 的 GDB JIT 接口调试）、诊断来自事件流、manifest 诊断来自 mcpp；以扩展 ID `mcpp-community.mcpp-vscode` 发布 1.0.0 | +| `jetbrains/` | Kotlin，CLion 2025.2+（`sinceBuild = 252`，与 mcppls 的 CLion 插件相同）：mcpp 运行配置类型（PTY 控制台、`main()` 旁 ▶）、测试树、调试、console filter、面板；以 `mcpp-clion` 发布。**语言功能只由 mcppls 提供**：`plugin.xml` 以 `io.github.sunrisepeak.mcppls` 依赖 mcppls 的 CLion 插件（Marketplace 安装 mcpp-clion 时一并安装它），mcpp-clion 不注册任何语言服务、不启用 CLion 的 Compilation Database 接管 mcpp 工程；两个插件的分工：mcppls = 编辑（补全、高亮、诊断、导航），mcpp-clion = 工程（构建、运行、测试、调试、面板） | +| `agents/` | 面向 agent 的说明与 skill | +| 分发 | 后端以 `--target -openkal` 构建为 ok 映像，经 `mcpp pack --mode self-contained` 成为各系统程序（与 mcppls 0.0.18 同法），随 vsix 与 CLion 插件分发；同时以 xlings 包 `mcpp-ide` 发布。GitHub Release 附 `mcpp-vscode-.vsix` 与 `mcpp-clion-.zip`；两个 Marketplace 由维护者上传（§13.3） | +| CI | 后端三系统测试（先 `mcpp build` 再 `mcpp test`，或使用 L5 的 `MCPP_BIN_`）；VS Code 扩展测试；JetBrains plugin verifier；README 与 README.zh-CN 按 mcppls 的结构 | + +VS Code 一端分工相同，依赖的强度不同：`vscode/` 的 `package.json` 以 `extensionPack: ["sunrisepeak.mcpp-language-server"]` 引入 mcppls 的 VS Code +扩展（`engines.vscode` 与之相同，`^1.91.0`）——一并安装，但用户可以卸载而保留 mcpp-vscode。理由：VS Code 用户可能已有自己的 C++ 语言服务，mcpp-vscode +0.7.0 起就提供"一键关闭其他 C++ 语言服务"而不是强制替换；CLion 的原生引擎不理解 mcpp 工程，那里没有可以保留的替代，因此用强依赖。两端都只经 mcppls +提供 mcpp 工程的语言功能，mcpp-ide 本身不注册语言服务。 + +### 11.3 mcpp-vscode + +mcpp-ide 首个发布之后，一个 PR：README 指向 mcpp-ide，说明扩展 ID 不变、用户无感升级；仓库随后归档（part 1 的 O9）。 + +### 11.4 mcpp-plugins + +#51（0.20.0：SYCL 一个 C++ 运行时）为本批的唯一 PR，追加：L2 通用库 `mcpp.plugins.platform`（平台插件构造平台陈述的函数，同 `mcpp.plugins.toolchain`）， +docs/rules.md 增加平台插件一节；CI 使用本批 mcpp。 + +### 11.5 其他仓库 + +| 仓库 | PR 内容 | +|---|---| +| mcpplibs/openkal-platform（新） | §10；验收测试；CI 以 mcpp 的 release 分支构建（发布前）/ 发布版（发布后）；发布 0.17.1 与 GitCode 镜像 | +| mcpplibs/mcpp-index | 收录 openkal-platform；测试成员 `openkal-platform`（零声明的 `--target x86_64-openkal` 构建与运行）；全量构建以宿主平台 openkal 运行一次的测量（§6.4，报告进 PR） | +| mcpplibs/os | 0.2.1：L9；删除 `[target.'cfg(os = "openkal")']` 下的运行时与工具声明；CI 的映像任务改用 `mcpp toolchain install --target` 与 `self env --target` | +| mcpplibs/openkal-image | README：工具由平台提供；`[target.'cfg(platform = "openkal")'] tools.okld = { path }` 用于开发工具本身 | +| openxlings/xim-pkgindex | `mcpp-ide` 包（后端，预编译）；mcpp 的版本前移由机器人 PR 完成 | + +## 12. mcpp 侧：迁移、遗留与代码组织 + +### 12.1 迁移 + +| 对象 | 变化 | +|---|---| +| 引擎的 openkal 代码 | 删除 `src/okimage/`；六处分支改为平台机制的通用调用 | +| 目标表 | 增加目标平台列与宿主平台列；`toolchain list` 显示 | +| 板级支持包 | docs/34 改写为"编写平台包"；`examples/13-platform-targets`、e2e 332、336、338 改为平台写法；333 保留指令写法作兼容判据 | +| 清单 | `[target.]` 增加 `platform`、`layers.*`、`tools.*`、`link.*`、`carrier`；`[host]`；`cfg(platform = …)`；SPEC-004 已知键表同步 | +| 引擎接口 | `mcpp.core` 协议 17：`platform` 阶段与 §4.1 的新指令 | +| 文档 | 新增 docs/35「平台」与 SPEC-013；改写 docs/24、34；增补 docs/04、22、30、41、50；中英文同步 | + +### 12.2 遗留问题 + +| 编号 | 问题 | 处理 | +|---|---|---| +| L1 | 精确版本缺失时安装失败不刷新（P4） | 已在 #847：`E_NOT_FOUND` 交给刷新策略后再请求一次；e2e 949 | +| L2 | 已读描述符的包仍尝试 `compat:` 拼写 | 已在 #847 | +| L3 | 多版本改名对带 C 接口的包静默生效（P3） | §8.2 | +| L4 | 安装失败的诊断转述 xlings 的 "the index is current"，而 mcpplibs 索引是陈旧工件 | mcpp 自己陈述：哪个索引、哪个修订、多久前同步、刷新是否进行及原因；xlings 原文降为 `-v` | +| L5 | `mcpp test` 不构建包的可执行目标 | 先构建，测试环境给出 `MCPP_BIN_`（对标 Cargo `CARGO_BIN_EXE_`） | +| L6 | 工作空间中同一依赖的缓存行打印两次 | 每包每次构建报告一次 | +| L7 | os、mcppls 的 CI 自建 `okrun` | §11 | +| L8 | kernel-abi 包在宿主目标上不进入共享映像（#846） | 已并入 #847（a336ef01，e2e 950） | +| L9 | os 0.2.0 在宿主三元组上以 openkal-musl 与 openkal-llvm-runtime 为运行时（mcppls 的形态，即 §5.4 的 `hosted`）时不能编译：macOS 源文件包含 SDK 头（`sys/sysctl.h`、`mach/mach.h`），Windows 源文件把 `path::c_str()`（此时为 `char`）传给宽字符接口 | os 0.2.1：按 Darwin ABI 自行声明所用的 libSystem 接口，Windows 显式构造宽字符串；CI 增加三系统的该形态测试 | +| L10 | `mcpp pack --mode self-contained` 为 M3 构建载体时，静态链接取到宿主 GCC 的 `crtbeginT.o`，被封闭性检查拒绝 | `:carrier` 以平台工具链为本机构建载体，C 运行时对象来自工具链载荷；e2e 覆盖 | + +### 12.3 代码组织 + +`src/platform/`，模块 `mcpp.platform`：`:model`（`PlatformStatement`，纯数据）、`:select`（目标图与宿主图的选择）、`:merge`、`:inject`（层、工具、 +载体、让位）、`:link`（driver、inputs、命名、封装）、`:carrier`、`:versions`（版本组与改名边界）、`:report`。清单解析在 `modules/manifest`；新指令在 +`hostprogram.cppm` 与 `build_program`。`:model`、`:select`、`:merge` 为纯函数，以表格测试锁定。 + +## 13. 交付：每仓库一个 PR，commit 作检查点 + +### 13.1 顺序 + +``` +openkal-platform(新) ─┐ + ├─> mcpp-index ─> mcpp #847 CI 绿 ─> 合入/发布/索引前移 ─┬─> os、openkal-image、mcppls #72、mcpp-plugins #51 + │ └─> mcpp-ide 首发 ─> xim-pkgindex(mcpp-ide) ─> mcpp-vscode +``` + +- openkal-platform 先于 mcpp-index：索引收录需要它的发布。 +- mcpp-index 的成员需要本批 mcpp（零声明构建）。解决：该成员的 `[package] mcpp` 写本批版本；索引 CI 增加与 os 相同的 `MCPP_SOURCE_REF` + 能力（成员的下限高于 CI 所用 mcpp 时，以该 ref 构建 mcpp 运行它；无 ref 时报告为跳过并点名下限）；收录条目在 mcpp 的最终 CI 之前合入，使 mcpp 的 e2e 读到索引中的平台包。 +- mcpp 的最终 commit 把目标表的钉住从开发期的 `{ git, tag }` 换成索引版本（与 os 0.2.0 的切换相同）。 +- 等待 CI 期间并行开发 mcppls、mcpp-ide、mcpp-plugins（不等 mcpp 发布即可开发，以 release 分支构建验证）。 + +### 13.2 各仓库的检查点 + +**mcpp-community/mcpp — #847（标题改为本批发布，版本号在发布当日确定）** + +1. L1/L2（已有）；L8 #846（已有） +2. SPEC-013、docs/35 初稿 +3. 清单：`[platform]`、`[target]` 新键、`[host]`、`cfg(platform)`；单元测试 +4. `src/platform` 的 `:model :select :merge`；表格测试 +5. `:inject`、锁的 `origin = "platform"` +6. `:link`（driver、inputs、命名、封装）接入 flags / ninja / plan；`:carrier` 从 pack 移出 +7. 运行器表形式与 `host`；execute +8. `:report`、JSON、`self env --target/--host`、`toolchain install --target`、`toolchain list` 平台列、doctor、explain 码 +9. 删除 `src/okimage`；目标表平台列（开发期钉 git tag） +10. 宿主图平台与自举边界；构建程序作为映像的 e2e +11. `:versions` 与改名边界 +12. L4、L5、L6 +13. 板级支持包迁移（examples/13、docs/34、e2e 332/336/338） +14. 新 e2e；942、947 改为零声明 +15. 文档（04、22、24、30、34、35、41、50，中英）、CHANGELOG、specs README +16. 钉住切换为索引版本 + +**mcpplibs/openkal-platform（新）**：初始 commit（README、LICENSE）→ PR：平台清单 → 验收测试 → CI → 发布 0.17.1、GitCode 镜像。 + +**mcpplibs/mcpp-index**：收录描述符 → 测试成员 → 宿主平台测量（报告）→ CHANGELOG。 + +**Sunrisepeak/mcpp-language-server — #72**：bench 任务（先测基线）→ H4 → H1 → H2 → H3 → H5 → bench 判据 → 平台迁移 → CI 与 CLion 断言 → CHANGELOG 与版本。 + +**mcpp-community/mcpp-ide（新）**：初始 commit（现有后端）→ PR：protocol fixture 更新 → 后端补全与平台视图 → spec 生成器 → webui → vscode 适配层（迁移）→ +jetbrains 适配层 → agents → 分发与 CI → README 中英 → 发布 0.1.0。 + +**mcpp-community/mcpp-plugins — #51**：`mcpp.plugins.platform` → 文档 → CI 前移到本批 mcpp。 + +**mcpplibs/os、openkal-image、openxlings/xim-pkgindex、mcpp-vscode**：各一个 PR，内容见 §11。 + +### 13.3 维护者的上传 + +mcpp-ide 发布时，交付本机验证后的产物目录（含两份文件与 `SHA256SUMS`）： + +- `mcpp-vscode-1.0.0.vsix`：上传到 VS Code Marketplace（publisher `mcpp-community`，扩展 ID `mcpp-community.mcpp-vscode`），可同时上传 Open VSX； +- `mcpp-clion-0.1.0.zip`：上传到 JetBrains Marketplace（插件 ID `io.github.mcpp-community.mcpp-clion`，依赖 `io.github.sunrisepeak.mcppls`）。 + +两份文件在交付前通过：vsix 的扩展测试与 `vsce ls` 清单检查；zip 的 plugin verifier（最低与当前 CLion）。mcpp-vscode 的归档在两份上传完成之后。 + +## 14. 测试 + +单元：`test_platform_merge`（四层、拼接、整体替换、继承、`cfg(platform)` 下写 `platform` 被拒）；`test_platform_select`（目标图五级、宿主图四级）； +`test_platform_inject`（让位、多层一次、`false`、工具让位、自举边界）；`test_platform_manifest`（每个键与错误位置、`hosted` 的键限制）； +`test_platform_versions`；`test_mangle_boundary`；锁的 `origin`。 + +e2e：零声明 openkal 的 build / run / test / pack（含 aarch64 构建）；942、947 零声明后照常通过；覆盖（`layers.c-abi = false`、`tools.okld = { path }`、 +`config.toml`、`MCPP_TARGET_*_PLATFORM`、`--platform`）；让位（声明 openkal-musl 时来源为 `graph`）；版本组冲突；一个与 openkal 无关的本地平台包 +（标志、封装、命名、运行器）；板级支持包两种写法；宿主三元组选用平台与被拒；宿主图平台（构建程序、库、平台插件三类作为映像运行；自举边界下 +`okrun` 原生构建）；输出与 JSON；`toolchain install --target` 后 `--offline`；L4、L5、L6；950。 + +mcppls：bench 判据（§11.1）；高亮缓存命中、未命中与键变化的单元测试；delta 的往返测试；VS Code e2e 断言首个高亮时延。 +mcpp-ide：后端单元测试与 golden fixture；两端适配层的冒烟测试。 + +## 15. 多视角自审 + +1. **兼容性。** 旧写法全部保留：手写 `openkal-tools` / `openkal-llvm-runtime`（让位）、板级支持包的指令写法（同义合并）、`[target]` 的已有键、旧锁（无 + `origin` 视为项目声明）；宿主图默认不变。破坏性只有一处：`src/okimage` 的提示文字与 `.oke` 生成由平台提供，行为不变（942、947 判据）。mcppls 的 + 高亮缓存键包含版本与编译命令，升级后不显示陈旧结果。mcpp-vscode 的用户经同一扩展 ID 无感升级。 +2. **跨平台。** 平台机制与三系统无关；载体汇编覆盖 ELF、Mach-O、COFF；`runner.host = "same-arch"` 处理跨架构不可运行并建议 `run_under`；宿主图作为 + 映像使三系统的构建程序一致（§6.3）。所有新 e2e 在三系统 CI 运行，需要 llvm 的沿用 `requires: llvm`。 +3. **用户开发体验（应用开发者）。** 已知目标零配置；覆盖只在一处、键与平台相同；报告与 JSON 给出每个值的来源；首次获取可见；离线可预取。 +4. **mcpp 使用体验（平台作者、插件作者、IDE）。** 一个包、一段 `[platform]`、一份规范；本地 `path` 开发；与规则插件共用 `mcpp.core`；`mcpp test` 在平台包中 + 对每个目标构建并在可运行的目标上运行；IDE 通过 `toolchain list` 与 `plan` 事件取得目标与平台，不解析清单。 +5. **简洁优雅。** 键从约 27 个精简为七组（§4）；不引入新模板语法；宿主图复用目标图的全部机制；openkal 在引擎中归零。风险：`[target.]` 的键增多—— + 新键全部是平台字段的同名覆盖，docs/35 以一张表列出。 +6. **稳定性。** 约定只能追加；平台身份进入指纹；版本组与改名边界在解析时拒绝而非在链接或运行时失败；锁防止静默前移；自举边界杜绝循环；宿主图 + 默认不变，映像化以测量为前提（§6.4）。mcppls 高亮缓存的每一项失配都视为未命中。 +7. **可观察性。** 报告行、JSON、`self env`、`doctor`、`-v` 的 `decision` 记录、`explain` 码；mcppls 的 bench 给出时延与字节数。 +8. **架构。** 引擎 = 机制 + 规范 + 宿主平台的官方约定；体系知识在平台包；IDE 逻辑在 mcpp-ide，mcpp 不含 IDE 概念；mcppls 不动 clangd。宿主图与目标图对称， + 为日后默认以映像运行构建程序留出的只是一行数据。 +9. **规模与风险。** 本批改动面大（mcpp、两个新仓库、六个生态仓库）。缓解：纯函数先以表格测试锁定；openkal 的现有 e2e 是行为不变的判据；与 openkal 无关的 + 平台 e2e 证明机制通用；每个仓库内以 commit 作检查点，CI 在每个检查点可运行；并行开发由 §13.1 的依赖图约束。 +10. **对外动作。** 新建公开仓库（openkal-platform、mcpp-ide 与其 GitCode 镜像）已确认；两个 Marketplace 由维护者上传，交付物为路径与校验和（§13.3）； + mcpp-vscode 在上传完成后归档。 +11. **IDE 的分工（rev 4）。** 语言功能只有一个来源：mcppls。mcpp-clion 以强依赖（``）、VS Code 适配层以扩展包（`extensionPack`，可卸载）引入它，不注册语言服务，不让 CLion 的 + Compilation Database 接管 mcpp 工程；工程功能只有一个来源：mcpp-ide 后端，经 mcpp 的控制契约。两端的能力表相同（part 1 §14.3）。风险：mcppls 的 CLion + 插件未装或版本过旧——依赖声明使 Marketplace 一并安装；后端握手时读 mcppls 的版本（`mcppls --version`），低于要求时在面板中给出升级指引，而不是静默降级。 +12. **复审 rev 3 的定案。** 宿主图的平台（§6）与自举边界无循环；mcppls 的 H1–H5 只改 mcppls，判据先测基线再改；#72、#51、#847 各为本仓库唯一 PR； + 索引 CI 的 `MCPP_SOURCE_REF` 与 os 的做法相同。未发现需要改动的定案。 + +## 16. 定案（rev 5） + +1. mcpp-clion 以 `` 强依赖 mcppls 的 CLion 插件；VS Code 适配层以 `extensionPack` 引入 mcppls 的扩展（一并安装、可卸载）；后端握手检查 mcppls 的版本。 +2. 两个 Marketplace 由维护者上传；交付物是本机验证后的产物目录；插件 ID `io.github.mcpp-community.mcpp-clion`。 +3. 按 §13 实施；L9、L10 同批解决。 diff --git a/src/okimage/okimage.cppm b/src/okimage/okimage.cppm deleted file mode 100644 index 4f9490900..000000000 --- a/src/okimage/okimage.cppm +++ /dev/null @@ -1,124 +0,0 @@ -// mcpp.okimage — what an ok image IS, for the target `-openkal` -// (openkal SPEC clause 12; design 2026-10-26 part 2, §8.5). -// -// One artifact per architecture, which a loader (`okrun`) runs on every system -// of that architecture. The facts below are the convention's, stated once: -// the compile flags that keep an image runnable everywhere, the link that -// produces the shape the loader accepts, and the note the loader reads. The -// format itself -- its reader, its checker, its signature -- is libokld's, and -// the engine reaches it through `okld`, a program of the dependency graph, -// never through a second implementation here. -// -// WHAT IS NOT HERE: the C library and the C++ run-time inside an image. They -// are packages (`provides = ["mcpp:c-abi=…"]`), and an image with neither is -// legal: its entry is its own `ok_main`. - -export module mcpp.okimage; - -import std; - -export namespace mcpp::okimage { - -// The entry of an executable image (SPEC 12.2): `int ok_main(void)`, in the -// image's convention, whose value is the image's status. A C library inside -// the image defines it and calls `main` with `kal_env_arg`'s arguments. -inline constexpr std::string_view kEntry = "ok_main"; - -// The programs of the graph the engine runs for this target. -inline constexpr std::string_view kTool = "okld"; // note, check -inline constexpr std::string_view kRunner = "okrun"; // `mcpp run`, `mcpp test` - -// FLAGS EVERY TRANSLATION UNIT OF AN IMAGE CARRIES (SPEC 12.2). -// -// -fPIC the image is ET_DYN and has no text relocation; -// -femulated-tls a variable is found through `__emutls_get_address`, -// which the image supplies: one system does not let a -// program set the thread pointer, another does not let it -// change it; -// -ffixed-x18 (AArch64) two systems reserve the platform register; -// -mno-outline-atomics -// (AArch64) the outlined helpers read the processor's -// features through an operating system's interface; -// -fvisibility=hidden -// an image exports what its source marks -// `visibility("default")` and nothing else, as the -// reference build does: everything else is the image's -// own, so the link can drop what nothing reaches, and the -// requirements note names only the interfaces the program -// uses, not every one its C library could. -// -fasynchronous-unwind-tables -// every frame, C's included, can be unwound: a C++ -// exception crosses the C of the unwinder itself -// (`_Unwind_RaiseException`) and of the C library. The -// hosted triples default to it; `-unknown-none-elf` -// does not -- measured, `_Unwind_Backtrace` in an image -// over openkal-llvm-runtime found no frame at all, and -// every `throw` terminated. -// -Xclang -fstack-clash-protection -// a frame larger than a page is probed page by page: one -// system commits a thread's stack as its guard page is -// touched, and a frame that steps over the guard faults -- -// measured, an image's 64 KiB buffer ended it on Windows -// and nowhere else. Passed to the front end because the -// driver enables the option for hosted triples only. -inline std::vector compile_flags(std::string_view arch) { - std::vector out{ "-fPIC", "-femulated-tls", "-fvisibility=hidden", - "-fasynchronous-unwind-tables", - "-Xclang", "-fstack-clash-protection" }; - if (arch == "aarch64") { - out.emplace_back("-ffixed-x18"); - out.emplace_back("-mno-outline-atomics"); - } - return out; -} - -// THE LINK, BY `ld.lld` ITSELF (SPEC 12.2). Position-independent, every -// loadable segment aligned to 64 KiB (the largest protection granularity among -// the systems), bound now, no executable stack, both hash tables (a loader may -// use either), what nothing reaches discarded, and the unwind index an -// unwinder finds through `dl_iterate_phdr`. -// -// Measured, a C program over openkal-musl before `--gc-sections` and hidden -// visibility: the whole C library was in the image and exported, and `okld -// check` refused it for importing `__mulxc3` -- from complex arithmetic the -// program never called. -inline std::string link_flags() { - return " -shared -z max-page-size=65536 -z now -z noexecstack --hash-style=both" - " --gc-sections --eh-frame-hdr"; -} - -// The notes every image carries (SPEC 12.4): the requirements, which `okld -// note` writes from the image's strong imports after the link, and the -// signature, all zero until `okld sign`. The same text `okld note-source` -// prints. -inline std::string_view note_source() { - return - "# The two notes of an ok image (openkal SPEC 12.4), written by mcpp.\n" - "\t.section .note.openkal,\"a\",@note\n" - "\t.p2align 2\n" - "\t.long 8, 16, 1\n" - "\t.asciz \"OPENKAL\"\n" - "\t.long 1, 0\n" - "\t.quad 0\n" - "\t.p2align 2\n" - "\t.long 8, 96, 2\n" - "\t.asciz \"OPENKAL\"\n" - "\t.fill 96, 1, 0\n"; -} - -// The diagnostic for a build of this target with no `okld` to finish it. It -// names the dependency that supplies both programs, and where to write it. -inline std::string no_tool_message(std::string_view triple, std::string_view tool) { - return std::format( - "'{}' builds ok images, and `{}` -- the format's tool, which writes an " - "image's requirements note and checks the convention -- is not provided.\n" - " Declare the package that builds it for this machine:\n" - "\n" - " [build-dependencies]\n" - " openkal-tools = {{ version = \"0.1.0\", tools = [\"okld\", \"okrun\"] }}\n" - "\n" - " `okrun` is then the runner of `mcpp run` and `mcpp test` for this target.", - triple, tool); -} - -} // namespace mcpp::okimage From af48dac79f4bedd005850ced323cd77444ca8c8c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 03:29:10 +0900 Subject: [PATCH 05/54] platform: a package's [platform] states a class of targets; the engine selects, merges, completes the graph with, links, seals, runs and carries it The engine keeps the mechanism (SPEC-013) and no target class's knowledge: - mcpp.manifest.platform / platform_toml: the seven groups of [platform], the same keys under [target.] and [host], read by one reader with the tier and the line of every value; cfg(platform = ...). - mcpp.platform: selection (flag, environment, project, this machine, the target table), per-field merge with appended flags, the hosted view, artifact-name patterns, seal steps with {needed}, report rows and JSON. - prepare: the platform package is the root's first dependency; the layers the graph lacks and the tools it does not build are added as the platform's; its version group is enforced; its toolchain stands where the row's pin stands; its sysroot is the base's fourth tier. - the link reads the platform: its flags, ld.lld with its flags, its entry, inputs made by a program (platform-step capture), names, and seal steps run before an artifact is published (platform-step seal), replacing okimage-finish. - the runner is one merged statement: platform, build programs, this machine, the project (any selector), MCPP_TARGET__RUNNER; runner.host = same-arch refuses another architecture's artifact. - pack --mode self-contained builds the platform's carrier with the static row's toolchain stated (the carrier's graph no longer cancels it). - src/okimage is removed; -openkal pins mcpplibs.openkal-platform@0.17.1. - GCC 16.1 crashed importing mcpp.cli when mcpp.platform imported prepare_inputs (#848): the selector evaluator is passed in instead. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- modules/manifest/src/cfg_selector.cppm | 10 +- modules/manifest/src/key_registry.cppm | 7 + modules/manifest/src/manifest.cppm | 3 + modules/manifest/src/platform.cppm | 313 ++++++++++ modules/manifest/src/platform_toml.cppm | 627 +++++++++++++++++++++ modules/manifest/src/targetside_model.cppm | 23 +- modules/manifest/src/toml.cppm | 119 +++- modules/manifest/src/types.cppm | 12 + modules/toolchain-model/src/triple.cppm | 28 +- src/build/execute.cppm | 116 ++-- src/build/flags.cppm | 87 ++- src/build/ninja_backend.cppm | 191 ++++--- src/build/plan.cppm | 44 +- src/build/prepare.cppm | 6 + src/build/prepare/base.cpp | 10 +- src/build/prepare/driver.cpp | 9 +- src/build/prepare/graph.cpp | 47 +- src/build/prepare/plan.cpp | 41 +- src/build/prepare/platform.cpp | 561 ++++++++++++++++++ src/build/prepare/scan.cpp | 43 +- src/build/prepare/state.cppm | 36 ++ src/build/prepare/target_side.cpp | 20 +- src/build/prepare/toolchain.cpp | 15 +- src/build/prepare/toolchain_decision.cpp | 3 + src/build/prepare_inputs.cppm | 13 +- src/build/refusal.cppm | 33 +- src/cli.cppm | 50 +- src/cli/cmd_build.cppm | 129 +++-- src/config.cppm | 52 +- src/pack/pack.cppm | 223 ++++---- src/pack/pipeline.cppm | 1 + src/platform/platform.cppm | 544 ++++++++++++++++++ src/toolchain/registry.cppm | 2 +- tests/unit/test_manifest.cpp | 4 +- tests/unit/test_platform.cpp | 414 ++++++++++++++ 35 files changed, 3402 insertions(+), 434 deletions(-) create mode 100644 modules/manifest/src/platform.cppm create mode 100644 modules/manifest/src/platform_toml.cppm create mode 100644 src/build/prepare/platform.cpp create mode 100644 src/platform/platform.cppm create mode 100644 tests/unit/test_platform.cpp diff --git a/modules/manifest/src/cfg_selector.cppm b/modules/manifest/src/cfg_selector.cppm index 955e0278e..cd210192b 100644 --- a/modules/manifest/src/cfg_selector.cppm +++ b/modules/manifest/src/cfg_selector.cppm @@ -40,8 +40,11 @@ export namespace mcpp::manifest::cfg { // target-side layer and are answerable only after the graph is resolved (see // the note on `kCfgLayerKeys` in mcpp.build.prepare_inputs); `accelerator` is // an input, known before resolution. +// `platform` is answered by the platform selection (SPEC-013 §3), which is +// made from the triple and the configuration before resolution, so it is read +// in the same pass as the triple's own keys. inline constexpr std::string_view kCfgTripleKeys[] = { - "arch", "env", "family", "os", + "arch", "env", "family", "os", "platform", }; inline constexpr std::string_view kCfgEarlyLayerKeys[] = { "accelerator", @@ -72,13 +75,16 @@ namespace mcpp::manifest::cfg { namespace { -constexpr unsigned kArch = 1, kOs = 2, kEnv = 4, kFamily = 8; +constexpr unsigned kArch = 1, kOs = 2, kEnv = 4, kFamily = 8, kPlatform = 16; unsigned component_of_key(std::string_view key) { if (key == "arch") return kArch; if (key == "os") return kOs | kFamily; if (key == "env") return kEnv; if (key == "family") return kFamily; + // A platform names a class of targets: one component, less specific than + // an operating system (two) and than a triple. + if (key == "platform") return kPlatform; return 0; // a layer key, `accelerator`, or an unknown key } diff --git a/modules/manifest/src/key_registry.cppm b/modules/manifest/src/key_registry.cppm index df89c41e1..d5f33a4ef 100644 --- a/modules/manifest/src/key_registry.cppm +++ b/modules/manifest/src/key_registry.cppm @@ -84,6 +84,8 @@ inline constexpr KeyClass kKeyRegistry[] = { {"target.", "cxx_runtime", KeyScope::Configuration, KeyMerge::Scalar, true, true }, {"target.", "linkage", KeyScope::Configuration, KeyMerge::Scalar, true, true }, {"target.", "min_api_level", KeyScope::Configuration, KeyMerge::Scalar, true, true }, + {"target.", "platform", KeyScope::Configuration, KeyMerge::Scalar, true, true }, + {"target.", "carrier", KeyScope::Configuration, KeyMerge::Scalar, true, true }, {"target.", "run_under", KeyScope::Configuration, KeyMerge::Scalar, true, true }, {"target.", "runner", KeyScope::Configuration, KeyMerge::Scalar, true, true }, {"target.", "sysroot", KeyScope::Configuration, KeyMerge::Scalar, true, true }, @@ -94,6 +96,9 @@ inline constexpr KeyClass kKeyRegistry[] = { {"target..xlings", "", KeyScope::Configuration, KeyMerge::Table, true, true }, {"target..targets", "", KeyScope::Package, KeyMerge::Table, false, true }, {"target..dependencies", "", KeyScope::Package, KeyMerge::Table, false, true }, + {"target..layers", "", KeyScope::Configuration, KeyMerge::Table, true, true }, + {"target..tools", "", KeyScope::Configuration, KeyMerge::Table, true, true }, + {"target..link", "", KeyScope::Configuration, KeyMerge::Table, true, true }, // whole tables {"toolchain", "", KeyScope::Configuration, KeyMerge::Table, true, false}, {"indices", "", KeyScope::Configuration, KeyMerge::Table, true, false}, @@ -109,6 +114,8 @@ inline constexpr KeyClass kKeyRegistry[] = { {"test", "", KeyScope::Package, KeyMerge::Table, false, false}, {"hooks", "", KeyScope::Package, KeyMerge::Table, false, false}, {"runtime", "", KeyScope::Package, KeyMerge::Table, false, false}, + {"platform", "", KeyScope::Package, KeyMerge::Table, false, false}, + {"host", "", KeyScope::Configuration, KeyMerge::Table, false, false}, }; // The keys of `table`, in registry order. diff --git a/modules/manifest/src/manifest.cppm b/modules/manifest/src/manifest.cppm index 877c23727..874bb0153 100644 --- a/modules/manifest/src/manifest.cppm +++ b/modules/manifest/src/manifest.cppm @@ -11,6 +11,7 @@ // mcpp.manifest.toml mcpp.toml parsing (projects / packages on disk) // mcpp.manifest.xpkg xpkg .lua `mcpp = {}` segment (index descriptors) // mcpp.manifest.flag_words the words an element of a compile-flag list stands for +// mcpp.manifest.platform what a platform states (SPEC-013), and its TOML reader // mcpp.pm.compat.workspace_position COMPAT(workspace-position): a workspace // root's own tables given to members by position @@ -21,6 +22,8 @@ export import mcpp.manifest.toml; export import mcpp.manifest.xpkg; export import mcpp.manifest.flag_words; export import mcpp.manifest.cfg_selector; +export import mcpp.manifest.platform; +export import mcpp.manifest.platform_toml; export import mcpp.manifest.key_registry; export import mcpp.pm.compat.workspace_position; export import mcpp.pm.compat.registry; diff --git a/modules/manifest/src/platform.cppm b/modules/manifest/src/platform.cppm new file mode 100644 index 000000000..b1f4e5d21 --- /dev/null +++ b/modules/manifest/src/platform.cppm @@ -0,0 +1,313 @@ +// mcpp.manifest.platform -- the data a platform states (SPEC-013; design +// 2026-10-28-platforms-design.md §4). +// +// A PLATFORM is one reusable, versioned configuration of a class of targets: +// the toolchain, the default provider of each target-side layer, the tools the +// build runs on this machine, the compile and link conventions, the artifact's +// naming and sealing, and the runner. A platform package states it in its own +// manifest under `[platform]`; a project overrides it field by field under +// `[target.]` (and the host graph's under `[host]`) with the same key +// names; a build program states the parts that must be computed with +// directives of the same meaning. All three read into `Statement`, every field +// carrying the place it was stated, so the engine merges one structure and +// reports one source per value. +// +// This module is pure data. The parser is mcpp.manifest.toml's (it owns the +// TOML document and the dependency grammar); selection, merging, injection and +// reporting are the engine's (`src/platform`). + +export module mcpp.manifest.platform; + +import std; +import mcpp.pm.dep_spec; + +export namespace mcpp::manifest::platform { + +// The tier a value was stated at. Ordered by precedence where tiers compete: +// a later tier replaces an earlier one field by field (§4.5). +enum class Tier : std::uint8_t { + BuiltIn, // the engine's own conventions for a host target + Platform, // the platform package's `[platform]` section + Program, // the platform's plugin or a build program (directives) + Config, // config.toml: `[target.]`, `[host]` + Manifest, // the project's mcpp.toml: `[target.]`, `[host]` + Env, // MCPP_TARGET__, MCPP_HOST_PLATFORM + Flag, // --platform, --host-platform + Graph, // a package of the dependency graph provides it (§4.3) +}; + +constexpr std::string_view tier_name(Tier t) { + switch (t) { + case Tier::BuiltIn: return "built-in"; + case Tier::Platform: return "platform"; + case Tier::Program: return "build-program"; + case Tier::Config: return "config.toml"; + case Tier::Manifest: return "mcpp.toml"; + case Tier::Env: return "env"; + case Tier::Flag: return "flag"; + case Tier::Graph: return "graph"; + } + return "platform"; +} + +// Where one value was stated: the tier, the file and line when there is one, +// and the key as written (`[target.'cfg(platform = "openkal")'].runner`, +// `MCPP_TARGET_X86_64_OPENKAL_PLATFORM`). +struct Origin { + Tier tier = Tier::Platform; + std::string file; + int line = 0; + std::string key; +}; + +// A package a platform names: the platform itself, a layer's provider, the +// package of a tool, or the carrier. +// +// String form `"[.]@"`; table form the dependency +// grammar (`version`, `path`, `git` with `rev`/`tag`/`branch`, `features`, +// `default-features`) plus `name`, which the string form carries in front of +// the `@`. A path or git source may leave `name` out: the package's own +// manifest then names it. +struct PackageRef { + std::string name; // "[ns.]name"; empty: the source names it + mcpp::pm::DependencySpec spec; // the source and the requests on the edge + // `tools. = { program = "" }`: a program found in a + // declared payload or on PATH, not built. + std::string program; + // `layers. = false`: the platform provides no package for the layer. + bool off = false; + + bool empty() const { + return name.empty() && program.empty() && !off && spec.path.empty() + && spec.git.empty() && spec.version.empty(); + } +}; + +// A scalar or an argv with its origin. `set` distinguishes "absent" from +// "present and empty"; the struct carries no std::optional because an +// optional of a string as a member of an exported struct instantiates its +// special members in every importer, which MSVC's standard library under +// clang refuses (see TargetEntry::sysroot in mcpp.manifest.types). +struct Text { + std::string value; + Origin origin; + bool set = false; +}; + +struct Words { + std::vector value; + Origin origin; + bool set = false; +}; + +// `runner = ["okrun", "{}"]` or `runner = { cmd = [...], host = "same-arch", +// exclusive = true }`. `host`: "" (unstated), "same-arch" (the artifact runs +// on a machine of its architecture), "any" (the runner reaches it from any +// machine, an emulator or a probe). +struct Runner { + std::vector cmd; + std::string host; + bool exclusive = false; + bool exclusiveSet = false; + Origin origin; + bool set = false; +}; + +// The four flag lists of `[platform.build]` and of each of its +// `[platform.build.'cfg(...)']` tables. +struct Flags { + std::vector cflags, cxxflags, asmflags, ldflags; + bool empty() const { + return cflags.empty() && cxxflags.empty() && asmflags.empty() && ldflags.empty(); + } +}; + +struct CfgFlags { + std::string predicate; // the selector, `cfg(arch = "aarch64")` + Flags flags; + Origin origin; +}; + +// One input of the link besides the objects: a file of the platform package, +// or the standard output of a program saved as `as` (assembled when it ends +// in `.S` or `.s`, linked as it is otherwise). +struct LinkInput { + std::string file; // absolute, inside the package that stated it + std::vector from; // argv; `from[0]` is a program name + std::string as; // the file name the output is saved under +}; + +// `[platform.link]`: how the artifact is produced (§4). +struct Link { + Text driver; // "compiler" | "lld" + Words flags; // given to ld.lld itself when driver = "lld" + Text start; // the entry symbol + Text script; // a linker script, absolute, inside the stating package + std::vector inputs; + Origin inputsOrigin; + bool inputsSet = false; + Text exe; // "{name}." + Text shared; // "lib{name}." + // Post-link steps, each an argv run on the linked file (`{}`) before it + // is published (§4.6). + std::vector> seal; + Origin sealOrigin; + bool sealSet = false; +}; + +// `carrier = { package = "", section = "", main = "" }` (M3, +// §4.7): the package a self-contained program of this machine is built over, +// the section the carried artifact is placed in, and the source of the +// program's entry, a file of the stating package that runs the artifact it +// finds between `mcpp_carried_begin` and `mcpp_carried_end`. +struct Carrier { + PackageRef package; + std::string section; + std::string main; // absolute + Origin origin; + bool set = false; +}; + +struct LayerEntry { + PackageRef ref; + Origin origin; +}; + +struct ToolEntry { + PackageRef ref; + Origin origin; +}; + +// `[platform.hosted]`: the part that applies when a HOST triple selects this +// platform (§5.4). +struct Hosted { + Text toolchain; + std::map layers; + bool set = false; +}; + +// Everything one source states about a platform, in the seven groups of §4. +struct Statement { + // [platform] + Words targets; + Text toolchain; + Text sysroot; + std::vector sysrootProvides; + Runner runner; + std::map runners; + Carrier carrier; + Text plugin; + // [platform.layers], [platform.tools] + std::map layers; + std::map tools; + // [platform.build] and its cfg tables + Flags build; + Origin buildOrigin; + std::vector buildCfg; + // [platform.link] + Link link; + // [platform.versions] + std::map versions; + Origin versionsOrigin; + // [platform.hosted] + Hosted hosted; + + bool empty() const { + return !targets.set && !toolchain.set && !sysroot.set && !runner.set + && runners.empty() && !carrier.set && !plugin.set && layers.empty() + && tools.empty() && build.empty() && buildCfg.empty() + && !link.driver.set && !link.flags.set && !link.start.set && !link.script.set + && !link.inputsSet && !link.exe.set && !link.shared.set && !link.sealSet + && versions.empty() && !hosted.set; + } +}; + +// The platform part of one `[target.]` table, or of `[host]` +// (selector empty): which platform, and the fields it overrides. +struct Override { + std::string selector; + PackageRef platform; // `platform = ""` or a table + Origin platformOrigin; + bool platformSet = false; + bool platformNone = false; // `platform = "none"` + Statement fields; +}; + +// The four target-side layers a platform may provide a package for (SPEC-002). +inline constexpr std::string_view kLayers[] = { + "kernel-abi", "c-abi", "c++-abi", "compiler-runtime", +}; + +constexpr bool is_layer(std::string_view k) { + for (auto l : kLayers) if (l == k) return true; + return false; +} + +// The keys a `[platform]` section may carry, and those of its sub-tables. One +// list per table, read by the parser's unknown-key sweep and by the tests. +inline constexpr std::string_view kPlatformKeys[] = { + "targets", "toolchain", "sysroot", "runner", "runners", "carrier", "plugin", + "layers", "tools", "build", "link", "versions", "hosted", +}; +inline constexpr std::string_view kBuildKeys[] = { "cflags", "cxxflags", "asmflags", "ldflags" }; +inline constexpr std::string_view kLinkKeys[] = { + "driver", "flags", "start", "script", "inputs", "exe", "shared", "seal", +}; +inline constexpr std::string_view kHostedKeys[] = { "toolchain", "layers" }; + +// The `[target.]` keys a project or config.toml uses to select and +// override a platform. `runner`, `runners`, `toolchain` and `sysroot` are the +// table's existing keys and keep their parsers. +inline constexpr std::string_view kOverrideKeys[] = { + "platform", "layers", "tools", "carrier", "link", +}; + +// `"[.]@"` -> the ref; the error names what is +// missing. `none` is not a ref; callers test for it first. +inline std::expected parse_ref_text(std::string_view text) { + PackageRef r; + const auto at = text.rfind('@'); + if (at == std::string_view::npos || at == 0 || at + 1 == text.size()) + return std::unexpected(std::format( + "'{}' is not a package reference; write \"@\", for example " + "\"openkal-platform@0.17.1\", or a table such as {{ path = \"\" }}", text)); + r.name = std::string(text.substr(0, at)); + r.spec.version = std::string(text.substr(at + 1)); + const auto dot = r.name.find('.'); + if (dot == std::string::npos) { + r.spec.namespace_ = std::string(mcpp::pm::kDefaultNamespace); + r.spec.shortName = r.name; + r.spec.namespaceOmitted = true; + } else { + r.spec.namespace_ = r.name.substr(0, dot); + r.spec.shortName = r.name.substr(dot + 1); + } + return r; +} + +// The name a reference is shown under: `openkal-platform@0.17.1`, +// `openkal-musl (path ../openkal-musl)`, `okrun (program)`. +inline std::string display(const PackageRef& r) { + if (r.off) return "false"; + if (!r.program.empty()) return std::format("{} (program)", r.program); + std::string n = r.spec.shortName.empty() ? r.name : r.spec.shortName; + if (!r.spec.path.empty()) + return n.empty() ? std::format("path {}", r.spec.path) + : std::format("{} (path {})", n, r.spec.path); + if (!r.spec.git.empty()) + return std::format("{}{}{} ({} {})", n, n.empty() ? "" : " ", r.spec.git, + r.spec.gitRefKind, r.spec.gitRev); + return std::format("{}@{}", n, r.spec.version); +} + +// The name `cfg(platform = "")` matches: the package name with a +// trailing `-platform` removed, so `openkal-platform` is the platform +// `openkal`. A selector may also name the package in full. +inline std::string platform_name(std::string_view packageShortName) { + constexpr std::string_view kSuffix = "-platform"; + if (packageShortName.size() > kSuffix.size() && packageShortName.ends_with(kSuffix)) + return std::string(packageShortName.substr(0, packageShortName.size() - kSuffix.size())); + return std::string(packageShortName); +} + +} // namespace mcpp::manifest::platform diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm new file mode 100644 index 000000000..6dd4b6acf --- /dev/null +++ b/modules/manifest/src/platform_toml.cppm @@ -0,0 +1,627 @@ +// mcpp.manifest.platform_toml -- the TOML reading of a platform statement +// (SPEC-013 §2; design 2026-10-28 §4, §5.2, §6.1). +// +// One reader for the three places the same keys are written: a platform +// package's `[platform]` section, a `[target.]` table of a project or +// of config.toml, and `[host]`. The tier and the file travel with every value, +// so a merged statement reports where each of its fields came from. +// +// Type errors are refusals naming the key. A key this engine does not know is +// recorded as a warning (`--strict` refuses it), as `[target.]` +// does: a package written for a newer engine still loads, and states its floor +// with `[package] mcpp` (R9.8). + +export module mcpp.manifest.platform_toml; + +import std; +import mcpp.libs.toml; +import mcpp.manifest.platform; +import mcpp.pm.dep_spec; + +export namespace mcpp::manifest::platform { + +struct Reader { + std::filesystem::path file; // the file the values are written in + Tier tier = Tier::Platform; + std::vector* warnings = nullptr; + + Origin at(const mcpp::libs::toml::Value& v, std::string key) const; + std::filesystem::path absolute(const std::string& written) const; + + std::expected, std::string> + strings(const mcpp::libs::toml::Value& v, std::string_view where) const; + + // A package: `"@"`, a dependency table with `name`, or — + // where allowed — `false` (a layer the platform does not provide) and + // `{ program = "" }` (a tool found rather than built). + std::expected + ref(const mcpp::libs::toml::Value& v, std::string_view where, bool allowOff, bool allowProgram) const; + + std::expected runner(const mcpp::libs::toml::Value& v, std::string_view where) const; + + std::expected + flags(const mcpp::libs::toml::Table& t, std::string_view where, Flags& out, + std::vector* cfg) const; + + std::expected + link(const mcpp::libs::toml::Table& t, std::string_view where, Link& out) const; + + std::expected + layers(const mcpp::libs::toml::Value& v, std::string_view where, + std::map& out) const; + + std::expected + tools(const mcpp::libs::toml::Value& v, std::string_view where, + std::map& out) const; + + std::expected carrier(const mcpp::libs::toml::Value& v, std::string_view where) const; + + // `[platform]` of a platform package. + std::expected statement(const mcpp::libs::toml::Table& t) const; + + // The platform part of one `[target.]` table (the keys of + // `kOverrideKeys`, plus `runner` and `runners`, which the table already + // had) or of `[host]` (selector empty: `platform` only). + std::expected + override_(const mcpp::libs::toml::Table& t, std::string_view selector) const; + + void warn(std::string text) const { if (warnings) warnings->push_back(std::move(text)); } +}; + +} // namespace mcpp::manifest::platform + +// ── implementation ────────────────────────────────────────────────────────── + +namespace mcpp::manifest::platform { + +namespace { + +namespace toml = mcpp::libs::toml; + +std::string list_of(std::span keys) { + std::string out; + for (auto k : keys) { if (!out.empty()) out += ", "; out += k; } + return out; +} + +bool contains(std::span keys, std::string_view k) { + return std::ranges::find(keys, k) != keys.end(); +} + +void split_name(PackageRef& r) { + if (r.name.empty()) return; + const auto dot = r.name.find('.'); + if (dot == std::string::npos) { + r.spec.namespace_ = std::string(mcpp::pm::kDefaultNamespace); + r.spec.shortName = r.name; + r.spec.namespaceOmitted = true; + } else { + r.spec.namespace_ = r.name.substr(0, dot); + r.spec.shortName = r.name.substr(dot + 1); + r.spec.namespaceOmitted = false; + } +} + +} // namespace + +Origin Reader::at(const toml::Value& v, std::string key) const { + return Origin{ tier, file.generic_string(), static_cast(v.position.line), std::move(key) }; +} + +std::filesystem::path Reader::absolute(const std::string& written) const { + std::filesystem::path p(written); + if (p.is_relative()) { + std::error_code ec; + auto dir = file.parent_path(); + if (dir.is_relative()) dir = std::filesystem::absolute(dir, ec); + p = dir / p; + } + return p.lexically_normal(); +} + +std::expected, std::string> +Reader::strings(const toml::Value& v, std::string_view where) const { + if (!v.is_array()) + return std::unexpected(std::format("{} must be an array of strings", where)); + std::vector out; + for (auto const& e : v.as_array()) { + if (!e.is_string()) + return std::unexpected(std::format("{} must contain only strings", where)); + out.push_back(e.as_string()); + } + return out; +} + +std::expected +Reader::ref(const toml::Value& v, std::string_view where, bool allowOff, bool allowProgram) const { + PackageRef r; + if (v.is_bool()) { + if (allowOff && !v.as_bool()) { r.off = true; return r; } + return std::unexpected(std::format( + "{} must be a package reference{}", where, allowOff ? " or false" : "")); + } + if (v.is_string()) { + auto parsed = parse_ref_text(v.as_string()); + if (!parsed) return std::unexpected(std::format("{}: {}", where, parsed.error())); + return std::move(*parsed); + } + if (!v.is_table()) + return std::unexpected(std::format( + "{} must be \"@\" or a table such as {{ path = \"\" }}", where)); + auto const& t = v.as_table(); + if (auto it = t.find("program"); it != t.end()) { + if (!allowProgram) + return std::unexpected(std::format( + "{}: `program` names a program found on this machine and is accepted " + "for a tool only", where)); + if (!it->second.is_string() || it->second.as_string().empty()) + return std::unexpected(std::format("{}.program must be a program name", where)); + if (t.size() != 1) + return std::unexpected(std::format( + "{}: {{ program = \"...\" }} takes no other key", where)); + r.program = it->second.as_string(); + return r; + } + for (auto const& [k, val] : t) { + auto key = std::format("{}.{}", where, k); + auto str = [&]() -> std::expected { + if (!val.is_string() || val.as_string().empty()) + return std::unexpected(std::format("{} must be a non-empty string", key)); + return val.as_string(); + }; + if (k == "name") { + auto s = str(); if (!s) return std::unexpected(s.error()); + r.name = *s; + } else if (k == "version") { + auto s = str(); if (!s) return std::unexpected(s.error()); + r.spec.version = *s; + } else if (k == "path") { + auto s = str(); if (!s) return std::unexpected(s.error()); + r.spec.path = absolute(*s).generic_string(); + } else if (k == "git") { + auto s = str(); if (!s) return std::unexpected(s.error()); + r.spec.git = *s; + } else if (k == "rev" || k == "tag" || k == "branch") { + auto s = str(); if (!s) return std::unexpected(s.error()); + if (!r.spec.gitRefKind.empty()) + return std::unexpected(std::format( + "{}: one of rev / tag / branch, not both {} and {}", where, r.spec.gitRefKind, k)); + r.spec.gitRev = *s; + r.spec.gitRefKind = k; + } else if (k == "features") { + auto s = strings(val, key); if (!s) return std::unexpected(s.error()); + r.spec.features = std::move(*s); + } else if (k == "default-features") { + if (!val.is_bool()) + return std::unexpected(std::format("{} must be true or false", key)); + r.spec.defaultFeatures = val.as_bool(); + r.spec.defaultFeaturesDeclared = true; + } else { + return std::unexpected(std::format( + "{}: '{}' is not a key of a package reference; the keys are name, version, " + "path, git, rev, tag, branch, features, default-features{}", + where, k, allowProgram ? ", or program alone" : "")); + } + } + const int sources = int(!r.spec.version.empty()) + int(!r.spec.path.empty()) + + int(!r.spec.git.empty()); + if (sources != 1) + return std::unexpected(std::format( + "{} must name exactly one source: version, path or git", where)); + if (!r.spec.git.empty() && r.spec.gitRev.empty()) + return std::unexpected(std::format("{}: a git source needs one of rev / tag / branch", where)); + if (r.spec.git.empty() && !r.spec.gitRev.empty()) + return std::unexpected(std::format("{}: {} applies to a git source", where, r.spec.gitRefKind)); + if (!r.spec.version.empty() && r.name.empty()) + return std::unexpected(std::format( + "{}: a version names the package too: {{ name = \"\", version = \"\" }}, " + "or \"@\"", where)); + split_name(r); + return r; +} + +std::expected Reader::runner(const toml::Value& v, std::string_view where) const { + Runner r; + r.origin = at(v, std::string(where)); + r.set = true; + if (v.is_array()) { + auto c = strings(v, where); + if (!c) return std::unexpected(c.error()); + r.cmd = std::move(*c); + } else if (v.is_table()) { + for (auto const& [k, val] : v.as_table()) { + auto key = std::format("{}.{}", where, k); + if (k == "cmd") { + auto c = strings(val, key); + if (!c) return std::unexpected(c.error()); + r.cmd = std::move(*c); + } else if (k == "host") { + if (!val.is_string() || (val.as_string() != "same-arch" && val.as_string() != "any")) + return std::unexpected(std::format( + "{} must be \"same-arch\" (the artifact runs on a machine of its " + "architecture) or \"any\" (the runner reaches it from any machine)", key)); + r.host = val.as_string(); + } else if (k == "exclusive") { + if (!val.is_bool()) return std::unexpected(std::format("{} must be true or false", key)); + r.exclusive = val.as_bool(); + r.exclusiveSet = true; + } else { + return std::unexpected(std::format( + "{}: '{}' is not a key of a runner; the keys are cmd, host, exclusive", where, k)); + } + } + } else { + return std::unexpected(std::format( + "{} must be an array of strings or {{ cmd = [...], host = \"same-arch\" }}", where)); + } + if (r.cmd.empty()) + return std::unexpected(std::format( + "{} has an empty command; an empty template would run nothing and report success", where)); + return r; +} + +std::expected +Reader::flags(const toml::Table& t, std::string_view where, Flags& out, + std::vector* cfg) const { + for (auto const& [k, v] : t) { + auto key = std::format("{}.{}", where, k); + if (cfg && v.is_table() && k.starts_with("cfg(") && k.ends_with(")")) { + CfgFlags c; + c.predicate = k; + c.origin = at(v, std::format("{}.'{}'", where, k)); + if (auto r = flags(v.as_table(), std::format("{}.'{}'", where, k), c.flags, nullptr); !r) + return r; + cfg->push_back(std::move(c)); + continue; + } + std::vector* dst = + k == "cflags" ? &out.cflags + : k == "cxxflags" ? &out.cxxflags + : k == "asmflags" ? &out.asmflags + : k == "ldflags" ? &out.ldflags + : nullptr; + if (!dst) + return std::unexpected(std::format( + "{} has no key '{}'; the keys are {}{}", where, k, list_of(kBuildKeys), + cfg ? ", and 'cfg(...)' tables of the same keys" : "")); + auto s = strings(v, key); + if (!s) return std::unexpected(s.error()); + dst->insert(dst->end(), s->begin(), s->end()); + } + return {}; +} + +std::expected +Reader::link(const toml::Table& t, std::string_view where, Link& out) const { + for (auto const& [k, v] : t) { + auto key = std::format("{}.{}", where, k); + auto text = [&](Text& dst, bool needsName) -> std::expected { + if (!v.is_string() || v.as_string().empty()) + return std::unexpected(std::format("{} must be a non-empty string", key)); + if (needsName && v.as_string().find("{name}") == std::string::npos) + return std::unexpected(std::format( + "{} = \"{}\" names no artifact; write the pattern with {{name}}, " + "for example \"{{name}}.oke\"", key, v.as_string())); + dst = Text{ v.as_string(), at(v, key), true }; + return {}; + }; + if (k == "driver") { + if (!v.is_string() || (v.as_string() != "compiler" && v.as_string() != "lld")) + return std::unexpected(std::format( + "{} must be \"compiler\" (the compiler driver links) or \"lld\" " + "(ld.lld itself links, with `flags`)", key)); + out.driver = Text{ v.as_string(), at(v, key), true }; + } else if (k == "flags") { + auto s = strings(v, key); + if (!s) return std::unexpected(s.error()); + out.flags = Words{ std::move(*s), at(v, key), true }; + } else if (k == "start") { + if (auto r = text(out.start, false); !r) return r; + } else if (k == "script") { + if (auto r = text(out.script, false); !r) return r; + out.script.value = absolute(out.script.value).generic_string(); + } else if (k == "exe") { + if (auto r = text(out.exe, true); !r) return r; + if (!out.exe.value.starts_with("{name}")) + return std::unexpected(std::format( + "{} = \"{}\": an executable's name begins with the target's name; " + "write \"{{name}}\", for example \"{{name}}.oke\"", key, out.exe.value)); + } else if (k == "shared") { + if (auto r = text(out.shared, true); !r) return r; + } else if (k == "inputs") { + if (!v.is_array()) + return std::unexpected(std::format( + "{} must be an array of {{ file = \"...\" }} or {{ from = [...], as = \"...\" }}", key)); + out.inputs.clear(); + for (auto const& e : v.as_array()) { + if (!e.is_table()) + return std::unexpected(std::format( + "{} must contain only tables: {{ file = \"...\" }} or " + "{{ from = [...], as = \"...\" }}", key)); + LinkInput in; + for (auto const& [ik, iv] : e.as_table()) { + auto ikey = std::format("{}.{}", key, ik); + if (ik == "file") { + if (!iv.is_string() || iv.as_string().empty()) + return std::unexpected(std::format("{} must be a path", ikey)); + in.file = absolute(iv.as_string()).generic_string(); + } else if (ik == "from") { + auto s = strings(iv, ikey); + if (!s) return std::unexpected(s.error()); + if (s->empty()) + return std::unexpected(std::format("{} is empty; it names a program", ikey)); + in.from = std::move(*s); + } else if (ik == "as") { + if (!iv.is_string() || iv.as_string().empty() + || iv.as_string().find_first_of("/\\") != std::string::npos) + return std::unexpected(std::format("{} must be a file name", ikey)); + in.as = iv.as_string(); + } else { + return std::unexpected(std::format( + "{}: '{}' is not a key of a link input; the keys are file, or from and as", + key, ik)); + } + } + if (in.file.empty() == in.from.empty()) + return std::unexpected(std::format( + "{}: an input is {{ file = \"...\" }} or {{ from = [...], as = \"...\" }}", key)); + if (!in.from.empty() && in.as.empty()) + return std::unexpected(std::format( + "{}: an input made by a program names the file it is saved as (`as`)", key)); + out.inputs.push_back(std::move(in)); + } + out.inputsOrigin = at(v, key); + out.inputsSet = true; + } else if (k == "seal") { + if (!v.is_array()) + return std::unexpected(std::format( + "{} must be an array of commands, for example [[\"okld\", \"check\", \"{{}}\"]]", key)); + out.seal.clear(); + for (auto const& e : v.as_array()) { + auto s = strings(e, key + "[]"); + if (!s) return std::unexpected(std::format( + "{} must be an array of commands, each an array of strings", key)); + if (s->empty()) + return std::unexpected(std::format("{} has an empty command", key)); + out.seal.push_back(std::move(*s)); + } + out.sealOrigin = at(v, key); + out.sealSet = true; + } else { + return std::unexpected(std::format( + "{} has no key '{}'; the keys are {}", where, k, list_of(kLinkKeys))); + } + } + return {}; +} + +std::expected +Reader::layers(const toml::Value& v, std::string_view where, + std::map& out) const { + if (!v.is_table()) + return std::unexpected(std::format( + "{} must be a table of = | false, for example " + "c-abi = \"openkal-musl@0.22.3\"", where)); + for (auto const& [k, val] : v.as_table()) { + auto key = std::format("{}.{}", where, k); + if (!is_layer(k)) + return std::unexpected(std::format( + "{}: '{}' is not a target-side layer; the layers are {}", where, k, list_of(kLayers))); + auto r = ref(val, key, /*allowOff=*/true, /*allowProgram=*/false); + if (!r) return std::unexpected(r.error()); + out[k] = LayerEntry{ std::move(*r), at(val, key) }; + } + return {}; +} + +std::expected +Reader::tools(const toml::Value& v, std::string_view where, + std::map& out) const { + if (!v.is_table()) + return std::unexpected(std::format( + "{} must be a table of = | {{ program = \"\" }}", where)); + for (auto const& [k, val] : v.as_table()) { + auto key = std::format("{}.{}", where, k); + auto r = ref(val, key, /*allowOff=*/false, /*allowProgram=*/true); + if (!r) return std::unexpected(r.error()); + out[k] = ToolEntry{ std::move(*r), at(val, key) }; + } + return {}; +} + +std::expected Reader::carrier(const toml::Value& v, std::string_view where) const { + if (!v.is_table()) + return std::unexpected(std::format( + "{} must be {{ package = \"@\", section = \"\" }}", where)); + Carrier c; + c.origin = at(v, std::string(where)); + c.set = true; + for (auto const& [k, val] : v.as_table()) { + auto key = std::format("{}.{}", where, k); + if (k == "package") { + auto r = ref(val, key, false, false); + if (!r) return std::unexpected(r.error()); + c.package = std::move(*r); + } else if (k == "section") { + if (!val.is_string() || val.as_string().empty()) + return std::unexpected(std::format("{} must be a section name", key)); + c.section = val.as_string(); + } else if (k == "main") { + if (!val.is_string() || val.as_string().empty()) + return std::unexpected(std::format("{} must be a source file of the package", key)); + c.main = absolute(val.as_string()).generic_string(); + } else { + return std::unexpected(std::format( + "{}: '{}' is not a key of a carrier; the keys are package, section, main", where, k)); + } + } + if (c.package.empty() || c.section.empty() || c.main.empty()) + return std::unexpected(std::format( + "{} names `package`, `section` and `main` (the program's entry source)", where)); + return c; +} + +std::expected Reader::statement(const toml::Table& t) const { + Statement s; + for (auto const& [k, v] : t) { + auto key = std::format("[platform].{}", k); + if (k == "targets") { + auto w = strings(v, key); + if (!w) return std::unexpected(w.error()); + if (w->empty()) return std::unexpected(std::format("{} names no target", key)); + s.targets = Words{ std::move(*w), at(v, key), true }; + } else if (k == "toolchain") { + if (!v.is_string() || v.as_string().empty()) + return std::unexpected(std::format( + "{} must be a toolchain, for example \"llvm@23.1.3\"", key)); + s.toolchain = Text{ v.as_string(), at(v, key), true }; + } else if (k == "sysroot") { + if (v.is_string()) { + s.sysroot = Text{ v.as_string(), at(v, key), true }; + } else if (v.is_table()) { + std::string path; + for (auto const& [sk, sv] : v.as_table()) { + if (sk == "path" && sv.is_string()) path = sv.as_string(); + else if (sk == "provides") { + auto p = strings(sv, key + ".provides"); + if (!p) return std::unexpected(p.error()); + s.sysrootProvides = std::move(*p); + } else return std::unexpected(std::format( + "{}: '{}' is not a key of a sysroot table; the keys are path, provides", key, sk)); + } + if (path.empty()) + return std::unexpected(std::format("{} is a table without `path`", key)); + s.sysroot = Text{ "path:" + absolute(path).generic_string(), at(v, key), true }; + } else { + return std::unexpected(std::format("{} must be a string or {{ path = \"\" }}", key)); + } + } else if (k == "runner") { + auto r = runner(v, key); + if (!r) return std::unexpected(r.error()); + s.runner = std::move(*r); + } else if (k == "runners") { + if (!v.is_table()) + return std::unexpected(std::format("{} must be a table of = [argv]", key)); + for (auto const& [rk, rv] : v.as_table()) { + auto rkey = std::format("{}.{}", key, rk); + auto w = strings(rv, rkey); + if (!w) return std::unexpected(w.error()); + if (w->empty()) return std::unexpected(std::format("{} is empty", rkey)); + s.runners[rk] = Words{ std::move(*w), at(rv, rkey), true }; + } + } else if (k == "carrier") { + auto c = carrier(v, key); + if (!c) return std::unexpected(c.error()); + s.carrier = std::move(*c); + } else if (k == "plugin") { + if (!v.is_string() || v.as_string().empty()) + return std::unexpected(std::format( + "{} must name the platform's build program, for example \"build.mcpp\"", key)); + s.plugin = Text{ absolute(v.as_string()).generic_string(), at(v, key), true }; + } else if (k == "layers") { + if (auto r = layers(v, "[platform.layers]", s.layers); !r) return std::unexpected(r.error()); + } else if (k == "tools") { + if (auto r = tools(v, "[platform.tools]", s.tools); !r) return std::unexpected(r.error()); + } else if (k == "build") { + if (!v.is_table()) return std::unexpected(std::format("{} must be a table", key)); + s.buildOrigin = at(v, "[platform.build]"); + if (auto r = flags(v.as_table(), "[platform.build]", s.build, &s.buildCfg); !r) + return std::unexpected(r.error()); + } else if (k == "link") { + if (!v.is_table()) return std::unexpected(std::format("{} must be a table", key)); + if (auto r = link(v.as_table(), "[platform.link]", s.link); !r) + return std::unexpected(r.error()); + } else if (k == "versions") { + if (!v.is_table()) + return std::unexpected(std::format("{} must be a table of = \"\"", key)); + for (auto const& [vk, vv] : v.as_table()) { + if (!vv.is_string() || vv.as_string().empty()) + return std::unexpected(std::format("[platform.versions].{} must be a version", vk)); + s.versions[vk] = vv.as_string(); + } + s.versionsOrigin = at(v, "[platform.versions]"); + } else if (k == "hosted") { + if (!v.is_table()) return std::unexpected(std::format("{} must be a table", key)); + s.hosted.set = true; + for (auto const& [hk, hv] : v.as_table()) { + auto hkey = std::format("[platform.hosted].{}", hk); + if (hk == "toolchain") { + if (!hv.is_string() || hv.as_string().empty()) + return std::unexpected(std::format("{} must be a toolchain", hkey)); + s.hosted.toolchain = Text{ hv.as_string(), at(hv, hkey), true }; + } else if (hk == "layers") { + if (auto r = layers(hv, "[platform.hosted.layers]", s.hosted.layers); !r) + return std::unexpected(r.error()); + } else { + return std::unexpected(std::format( + "[platform.hosted] has no key '{}'; a host triple takes the platform's {} only", + hk, list_of(kHostedKeys))); + } + } + } else { + warn(std::format("[platform] has unsupported key '{}' (ignored). The keys are: {}.", + k, list_of(kPlatformKeys))); + } + } + return s; +} + +std::expected +Reader::override_(const toml::Table& t, std::string_view selector) const { + Override o; + o.selector = std::string(selector); + const std::string where = selector.empty() + ? std::string("[host]") : std::format("[target.{}]", selector); + for (auto const& [k, v] : t) { + auto key = std::format("{}.{}", where, k); + if (k == "platform") { + if (v.is_string() && v.as_string() == "none") { + o.platformNone = true; + } else { + auto r = ref(v, key, false, false); + if (!r) return std::unexpected(r.error()); + o.platform = std::move(*r); + } + o.platformOrigin = at(v, key); + o.platformSet = true; + } else if (selector.empty()) { + // `[host]` selects the host graph's platform and nothing else. + return std::unexpected(std::format( + "[host] has no key '{}'; it holds `platform`, the host graph's platform", k)); + } else if (k == "layers") { + if (auto r = layers(v, key, o.fields.layers); !r) return std::unexpected(r.error()); + } else if (k == "tools") { + if (auto r = tools(v, key, o.fields.tools); !r) return std::unexpected(r.error()); + } else if (k == "carrier") { + auto c = carrier(v, key); + if (!c) return std::unexpected(c.error()); + o.fields.carrier = std::move(*c); + } else if (k == "link") { + if (!v.is_table()) return std::unexpected(std::format("{} must be a table", key)); + if (auto r = link(v.as_table(), key, o.fields.link); !r) return std::unexpected(r.error()); + } else if (k == "runner") { + auto r = runner(v, key); + if (!r) return std::unexpected(r.error()); + o.fields.runner = std::move(*r); + } else if (k == "runners") { + if (!v.is_table()) continue; // the table's own reader refuses the shape + for (auto const& [rk, rv] : v.as_table()) { + auto rkey = std::format("{}.{}", key, rk); + auto w = strings(rv, rkey); + if (!w) continue; // ditto + o.fields.runners[rk] = Words{ std::move(*w), at(rv, rkey), true }; + } + } + // Every other key belongs to the table's other readers. + } + // `platform` decides what `cfg(platform = ...)` means, so it cannot be + // stated under a selector that asks about it (§5.1). + if (o.platformSet && selector.find("platform") != std::string_view::npos + && selector.starts_with("cfg(")) + return std::unexpected(std::format( + "{}.platform: a platform cannot be selected under a selector that asks which " + "platform is selected; write it under a triple or a cfg() of os/arch/env", where)); + return o; +} + +} // namespace mcpp::manifest::platform diff --git a/modules/manifest/src/targetside_model.cppm b/modules/manifest/src/targetside_model.cppm index ad46b4e60..1f583a3a9 100644 --- a/modules/manifest/src/targetside_model.cppm +++ b/modules/manifest/src/targetside_model.cppm @@ -566,10 +566,11 @@ struct Inputs { std::string targetOs; // mcpp's own OS field, for the payload interface name std::string targetEnv; // mcpp's own ENV field ("musl", "gnu", …) bool freestandingTarget = false; - // An ok image (`-openkal`): its kernel interface is openkal's, held - // by the loader that runs it rather than by a package or a payload, and + // A target whose loader supplies the system (an ok image, `-openkal`): + // its kernel interface is the one the triple names (`targetOs`), held by + // the loader that runs it rather than by a package or a payload, and // every layer above it is the graph's or absent. - bool openkalTarget = false; + bool loaderTarget = false; // The compiler, which is always a payload and never a package (see // `layer_is_suppliable_by_package`). Present here so that the layering rule @@ -721,10 +722,10 @@ inline TargetSide resolve(const Inputs& in) { ts.compilerRuntime = { Origin::Graph, in.compilerRuntime->display_interface(), in.compilerRuntime->id(), false }; - else if (in.payloadCompilerRuntimeAbsent || in.openkalTarget) - // An ok image's runtime routines are the image's own (a C++ - // run-time package, or the program): the payload's are built for an - // operating system the image does not have. + else if (in.payloadCompilerRuntimeAbsent || in.loaderTarget) + // The runtime routines of a program its loader runs are its own (a + // C++ run-time package, or the program): the payload's are built for + // an operating system the program does not have. ts.compilerRuntime = { Origin::None, {}, {}, false }; else if (!in.compilerFamily.empty()) ts.compilerRuntime = { Origin::Payload, in.compilerFamily, {}, false }; @@ -733,10 +734,10 @@ inline TargetSide resolve(const Inputs& in) { if (in.kernelAbi) ts.kernelAbi = { Origin::Graph, in.kernelAbi->display_interface(), in.kernelAbi->id(), false }; - else if (in.openkalTarget) + else if (in.loaderTarget) // Fixed by the target: an ok image calls `kal_*` and nothing else, - // and the loader binds them to its implementation (SPEC 12.3). - ts.kernelAbi = { Origin::Payload, "openkal", "the loader (okrun)", false }; + // and the loader binds them to its implementation (openkal SPEC 12.3). + ts.kernelAbi = { Origin::Payload, in.targetOs, "the loader", false }; else if (in.freestandingTarget) // Not a gap. A bare machine has no kernel, and saying so is the // information: the same target reads `openkal` when an implementation @@ -757,7 +758,7 @@ inline TargetSide resolve(const Inputs& in) { ts.cAbi = { Origin::Xpkg, in.sysrootLibc.empty() ? xpkg_interface(in.sysrootXpkg) : in.sysrootLibc, in.sysrootXpkg, false }; - else if (in.freestandingTarget || in.openkalTarget) + else if (in.freestandingTarget || in.loaderTarget) ts.cAbi = { Origin::None, {}, {}, false }; else ts.cAbi = { Origin::Payload, payload_libc_name(in.targetOs, in.targetEnv), diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 5f4ae7f97..3fa7fe831 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -5,6 +5,8 @@ export module mcpp.manifest.toml; import mcpp.manifest.types; import mcpp.manifest.key_registry; import mcpp.manifest.cfg_selector; +import mcpp.manifest.platform; +import mcpp.manifest.platform_toml; import mcpp.targetside; import std; import mcpp.invocation; @@ -1043,13 +1045,18 @@ std::expected parse_document(const t::Document& documen // every entry. Mixed arrays (strings and tables) never reach here. "package.requires", "target.*.requires", + // `inputs = [{ file = "..." }, { from = [...], as = "..." }]` (SPEC-013 + // §2.5): the platform reader type-checks every entry. + "platform.link.inputs", + "target.*.link.inputs", }; if (auto badPath = find_disallowed_array_of_tables(doc->root(), "", kAllowedArraysOfTables)) { return std::unexpected(error(origin, std::format( "[[{}]] (array-of-tables) is not allowed for section '{}'; " "array-of-tables syntax is only supported for [[build.flags]], " "[[features..flags]], [[runtime.requirements]], " - "[[runtime.artifacts]], [[runtime.deploy]], and [xlings] deps entries", + "[[runtime.artifacts]], [[runtime.deploy]], [[platform.link.inputs]], " + "and [xlings] deps entries", *badPath, *badPath))); } @@ -3735,6 +3742,29 @@ std::expected parse_document(const t::Document& documen } } + // [platform] — the section that makes a package a PLATFORM (SPEC-013; + // design 2026-10-28 §4): the configuration of a class of targets, read + // by the engine before the toolchain is decided. And [host] — the + // platform of the host graph (§6.1). + if (auto* pt = doc->get_table("platform")) { + mcpp::manifest::platform::Reader rd{origin, mcpp::manifest::platform::Tier::Platform, + &m.schemaWarnings}; + auto st = rd.statement(*pt); + if (!st) return std::unexpected(error(origin, st.error())); + if (!st->targets.set) + return std::unexpected(error(origin, + "[platform] names the targets it serves: targets = [\"\", ...]")); + m.platform = std::move(*st); + m.platformDeclared = true; + } + if (auto* ht = doc->get_table("host")) { + mcpp::manifest::platform::Reader rd{origin, mcpp::manifest::platform::Tier::Manifest, + &m.schemaWarnings}; + auto ov = rd.override_(*ht, ""); + if (!ov) return std::unexpected(error(origin, ov.error())); + m.hostPlatform = std::move(*ov); + } + // [lib] — library root convention (cargo-style). if (auto v = doc->get_string("lib.path")) { m.lib.path = *v; @@ -3927,13 +3957,22 @@ std::expected parse_document(const t::Document& documen // scalar (the unknown-key sweep below skips it by type) nor part // of the conditional sub-table channel. if (auto it = body.find("runner"); it != body.end()) { - if (!it->second.is_array()) { + // The table form, `{ cmd = [...], host = "same-arch", exclusive = + // true }`, is the platform reader's (SPEC-013 §2.3); its command + // is the template here as well. + if (!it->second.is_array() && it->second.is_table()) { + mcpp::manifest::platform::Reader rd{origin, mcpp::manifest::platform::Tier::Manifest, + &m.schemaWarnings}; + auto r = rd.runner(it->second, std::format("[target.{}].runner", triple)); + if (!r) return std::unexpected(error(origin, r.error())); + e.runner = r->cmd; + } else if (!it->second.is_array()) { return std::unexpected(error(origin, std::format( - "[target.{}].runner must be an array of strings, " - "e.g. runner = [\"qemu-system-riscv64\", \"-kernel\"]", + "[target.{}].runner must be an array of strings or {{ cmd = [...], " + "host = \"same-arch\" }}, e.g. runner = [\"qemu-system-riscv64\", \"-kernel\"]", triple))); } - for (auto& el : it->second.as_array()) { + if (it->second.is_array()) for (auto& el : it->second.as_array()) { if (!el.is_string()) { return std::unexpected(error(origin, std::format( "[target.{}].runner must contain only strings", triple))); @@ -3986,6 +4025,52 @@ std::expected parse_document(const t::Document& documen } } + // THE PLATFORM KEYS (SPEC-013 §2.3; design 2026-10-28 §5): which + // platform this selector's targets use (`platform`), and the fields + // of it this table overrides with the platform's own key names + // (`layers`, `tools`, `carrier`, `link`, plus `runner` and + // `runners` above). One reader, mcpp.manifest.platform_toml, reads + // them here, in a platform package's `[platform]` and in `[host]`. + bool platformKeys = false; + if (auto it = body.find("platform"); it != body.end()) { + if (!it->second.is_string() && !it->second.is_table()) + return std::unexpected(error(origin, std::format( + "[target.{}].platform must be \"@\", \"none\", or a table " + "such as {{ path = \"\" }}", triple))); + platformKeys = true; + } + if (auto it = body.find("layers"); it != body.end()) { + if (!it->second.is_table()) + return std::unexpected(error(origin, std::format( + "[target.{}].layers must be a table, e.g. [target.{}.layers] c-abi = \"@\"", triple, triple))); + platformKeys = true; + } + if (auto it = body.find("tools"); it != body.end()) { + if (!it->second.is_table()) + return std::unexpected(error(origin, std::format( + "[target.{}].tools must be a table, e.g. [target.{}.tools] okld = {{ path = \"\" }}", triple, triple))); + platformKeys = true; + } + if (auto it = body.find("carrier"); it != body.end()) { + if (!it->second.is_table()) + return std::unexpected(error(origin, std::format( + "[target.{}].carrier must be a table, e.g. carrier = {{ package = \"@\", section = \"\" }}", triple))); + platformKeys = true; + } + if (auto it = body.find("link"); it != body.end()) { + if (!it->second.is_table()) + return std::unexpected(error(origin, std::format( + "[target.{}].link must be a table, e.g. [target.{}.link] start = \"\"", triple, triple))); + platformKeys = true; + } + if (platformKeys || body.contains("runner") || body.contains("runners")) { + mcpp::manifest::platform::Reader rd{origin, mcpp::manifest::platform::Tier::Manifest, + &m.schemaWarnings}; + auto ov = rd.override_(body, canon_triple(triple)); + if (!ov) return std::unexpected(error(origin, ov.error())); + m.platformOverrides.push_back(std::move(*ov)); + } + // Unsupported scalar and array keys are REPORTED, not dropped. // `[targets.]` has done this since #249; this table did not, // so a key that looks plausible — `cxx_runtime_tests` was the real @@ -4024,25 +4109,28 @@ std::expected parse_document(const t::Document& documen // arrays, so a sixth key added to the parser without a matching // entry here fails that test instead of shipping silently again. static constexpr std::string_view kKnownTargetScalars[] = { - "cxx_runtime", "linkage", "min_api_level", "sysroot", "toolchain", + "cxx_runtime", "linkage", "min_api_level", "platform", "sysroot", "toolchain", }; static constexpr std::string_view kKnownTargetArrays[] = { "requires", "run_under", "runner" }; static constexpr std::string_view kKnownTargetTables[] = { - "abi", "build", "build-dependencies", "dependencies", + "abi", "build", "build-dependencies", "carrier", "dependencies", "dev-dependencies", "feature-deps", "feature-requires-abi", - "feature-xlings", "requires_abi", "runners", "runtime", - "targets", "xlings", + "feature-xlings", "layers", "link", "requires_abi", "runners", "runtime", + "targets", "tools", "xlings", }; for (auto& [key, value] : body) { if (value.is_table()) { if (std::ranges::find(kKnownTargetTables, key) != std::end(kKnownTargetTables)) continue; - // A scalar key whose value may also be written as a table - // (`sysroot = { path = ... }`) is a value, not a section; - // it was parsed, or refused, above. + // A scalar or an array key whose value may also be written + // as a table (`sysroot = { path = ... }`, `runner = { cmd = + // [...] }`) is a value, not a section; it was parsed, or + // refused, above. if (std::ranges::find(kKnownTargetScalars, key) - != std::end(kKnownTargetScalars)) + != std::end(kKnownTargetScalars) + || std::ranges::find(kKnownTargetArrays, key) + != std::end(kKnownTargetArrays)) continue; std::string tables; for (auto name : kKnownTargetTables) { @@ -4061,8 +4149,9 @@ std::expected parse_document(const t::Document& documen if (std::ranges::find(known, key) != known.end()) continue; m.schemaWarnings.push_back(std::format( "[target.{}] has unsupported key '{}' (ignored). Supported keys: " - "cxx_runtime, linkage, min_api_level, run_under, runner, sysroot, toolchain, " - "plus the [target..runners] table for named runners. " + "cxx_runtime, linkage, min_api_level, platform, run_under, runner, sysroot, " + "toolchain, plus the [target..runners] table for named runners and " + "the platform's layers, tools, carrier and link tables. " "Per-role contracts go in [build].cxx_runtime's table form.", triple, key)); } diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 5e720e176..8f83362d3 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -11,6 +11,7 @@ import mcpp.pm.compat; // Legacy dependency-key compatibility helpers import mcpp.pm.index_spec; // IndexSpec for [indices] section import mcpp.manifest.flag_words; import mcpp.targetside; +import mcpp.manifest.platform; // SPEC-013: [platform], the platform keys of [target.], [host] export namespace mcpp::manifest { @@ -2441,6 +2442,17 @@ struct Manifest { // [target.] tables — empty if user didn't declare any. std::map targetOverrides; + // PLATFORMS (SPEC-013; design 2026-10-28). `platform` is a platform + // package's own `[platform]` section, `platformDeclared` whether it has + // one. `platformOverrides` holds the platform part of every + // `[target.]` table, in file order (selection and per-field + // overrides; the engine orders matching selectors by specificity), and + // `hostPlatform` the `[host]` table: the platform of the host graph. + mcpp::manifest::platform::Statement platform; + bool platformDeclared = false; + std::vector platformOverrides; + mcpp::manifest::platform::Override hostPlatform; + // [pack] — `mcpp pack` config (see docs/10-pack-and-release.md). PackConfig packConfig; diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 8807e4a28..e4771b3a6 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -323,6 +323,13 @@ struct Triple { // `cfg(windows)` are both false, and the system it runs on is the // loader's, known at run time (`os::host()`). bool is_openkal() const { return os == "openkal"; } + // THE LOADER SUPPLIES THE SYSTEM. The compiler is told no operating + // system, and what lies beneath the program is bound when it is loaded. + // The layer rules of such a target are the engine's (a C++ header set + // only from the graph, a freestanding C environment when the graph has no + // C library); everything else about it -- flags, link, naming, runner -- + // is its platform's (SPEC-013). + bool loader_supplies_os() const { return is_openkal(); } // WHETHER THIS ROW'S TOOLCHAIN PIN IS A CAPABILITY RATHER THAN A // CONVENTION — the distinction that decides whether an author may override @@ -476,6 +483,13 @@ struct TargetInfo { // subset alike to one libc, one ISA and one version. std::string_view sysroot; bool defaultStatic; // target's default linkage is static + // THE PLATFORM a build of this target uses when nothing selects one + // (SPEC-013 §7): a pinned version of a platform package, which is what + // "built-in support" for a class of targets is. Empty = the engine's own + // conventions for the target (every hosted row). A project selects + // another with `[target.] platform`, a machine in config.toml, + // one invocation with `--platform`. + std::string_view platform = {}; }; // (note deliberately excludes "static" — the display layer derives that tag @@ -886,11 +900,15 @@ inline constexpr TargetInfo kKnownTargets[] = { { "wasm32-emscripten", "verified", "wasm","emsdk@6.0.9","", true }, // OK IMAGES (openkal SPEC clause 12; design 2026-10-26 part 2, §8.5). One // artifact per architecture, run by `okrun` on Linux, macOS and Windows of - // that architecture. clang emits it from one binary for either row; the C - // library and the C++ run-time are the graph's (`provides`), and an image - // with neither is legal. The two architectures the loader relocates. - { "x86_64-openkal", "preview", "ok", "llvm@23.1.3","", true }, - { "aarch64-openkal", "preview", "ok", "llvm@23.1.3","", true }, + // that architecture. clang emits it from one binary for either row. What + // makes it an ok image -- its flags, its link, its note and check, its + // name, its runner, its C library and C++ run-time -- is the platform's, + // `openkal-platform` (SPEC-013), pinned here. The two architectures the + // loader relocates. + { "x86_64-openkal", "preview", "ok", "llvm@23.1.3","", true, + "mcpplibs.openkal-platform@0.17.1" }, + { "aarch64-openkal", "preview", "ok", "llvm@23.1.3","", true, + "mcpplibs.openkal-platform@0.17.1" }, }; inline std::span known_targets() { return kKnownTargets; } diff --git a/src/build/execute.cppm b/src/build/execute.cppm index dc4f143ed..e0b42ebdc 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -21,7 +21,7 @@ import mcpp.diag; import mcpp.build.plan; import mcpp.toolchain.triple; import mcpp.freestanding.runner; -import mcpp.okimage; +import mcpp.platform; import mcpp.build.runner_lookup; // #544: where the runner's program is import mcpp.home; // config.toml, for the machine's default toolchain import mcpp.libs.toml; @@ -50,6 +50,7 @@ import mcpp.provider.select; import mcpp.runtime.binding; import mcpp.log; import mcpplibs.os.platform; +import mcpp.toolchain.platform; import mcpp.invocation; import mcpp.effects; import mcpp.runtime.launch_env; @@ -1098,8 +1099,24 @@ ExecutionHost locate_run_under(const mcpp::build::BuildContext& ctx, std::string // here, and is believed. std::optional unreachable_execution(const mcpp::build::BuildContext& ctx, std::string_view tripleKey, bool noRunner) { - if (noRunner || !ctx.runUnder.empty() || mcpplibs::os::is_macos) return std::nullopt; + if (noRunner || !ctx.runUnder.empty()) return std::nullopt; auto t = mcpp::toolchain::triple::parse(tripleKey); + // A RUNNER THAT RUNS THE ARTIFACT ON A MACHINE OF ITS ARCHITECTURE + // (`runner = { ..., host = "same-arch" }`, SPEC-013): an ok image's loader + // is a program of this machine, and an image of another architecture is + // not one it can map. + if (auto const& r = ctx.plan.platform.fields.runner; t && r.set && r.host == "same-arch" + && t->arch != mcpp::toolchain::platform::host_arch) + return std::format( + "cannot run {} artifacts on this {} host: the runner `{}` ({}) runs an artifact on a " + "machine of its architecture.\n" + " Name an execution host of that architecture: [target.{}] run_under = " + "[\"remote-run\", \"\", \"--\"]\n" + " (in mcpp.toml for the team, or ~/.mcpp/config.toml for this machine).", + tripleKey, mcpp::toolchain::platform::host_arch, + r.cmd.empty() ? std::string() : r.cmd.front(), + mcpp::platform::origin_text(r.origin), tripleKey); + if (mcpplibs::os::is_macos) return std::nullopt; if (!t || !t->is_apple()) return std::nullopt; return std::format( "cannot run {} artifacts on this host: {} runs only on macOS.\n" @@ -1175,50 +1192,34 @@ RunnerChoice choose_device_action(const BuildContext& ctx, const auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); if (ft) c.freestanding = ft->is_freestanding(); c.tripleKey = ft ? ft->str() : ctx.tc.targetTriple; - // The graph's answer: the default runner, or a named one a package supplied. - if (isDefault) { - c.tmpl = ctx.manifest.buildConfig.runner; - } else if (auto it = ctx.manifest.buildConfig.namedRunners.find(std::string(which)); - it != ctx.manifest.buildConfig.namedRunners.end()) { - c.tmpl = it->second.argv; - c.longLived = it->second.longLived; - } - // The row is found as every other `[target.]` reader finds it - // (`find_target_entry`), whichever spelling the section and the toolchain - // use. The toolchain's own `targetTriple` is what the driver reported, - // which on macOS is `arm64-apple-darwin24.6.0`; an exact lookup of it - // matched on Linux and never on macOS (measured on CI, 2026-09-02), and an - // exact lookup of the canonical spelling missed a section written - // `[target.x86-windows-msvc]`. The raw form is kept as a fallback for a - // triple the parser does not know. - auto with_slot = [&](const mcpp::manifest::TargetEntry* e) { - const mcpp::manifest::TargetEntry* none = nullptr; - if (!e) return none; - if (isDefault) return e->runner.empty() ? none : e; - auto nr = e->namedRunners.find(std::string(which)); - return (nr != e->namedRunners.end() && !nr->second.empty()) ? e : none; + // THE STATEMENT PREPARE MERGED (SPEC-013 §4.5): the platform's runner, a + // build program's (`mcpp::runner`), this machine's, the project's + // `[target.]` (a cfg selector as well as a triple, whichever + // spelling), the environment's -- the later replacing the earlier. The + // project overriding what a dependency supplied is reported. + auto const& pf = ctx.plan.platform.fields; + auto overrides_program = [&](const mcpp::manifest::platform::Origin& o, bool programHad) { + using T = mcpp::manifest::platform::Tier; + return programHad && (o.tier == T::Manifest || o.tier == T::Config || o.tier == T::Env); }; - const mcpp::manifest::TargetEntry* entry = nullptr; - if (ft) { - entry = with_slot(mcpp::build::find_target_entry(ctx.manifest, *ft)); - } else if (auto it = ctx.manifest.targetOverrides.find(ctx.tc.targetTriple); - it != ctx.manifest.targetOverrides.end()) { - entry = with_slot(&it->second); - } - if (entry) { - if (isDefault) { - c.fromManifest = !ctx.manifest.buildConfig.runner.empty(); - c.tmpl = entry->runner; - } else { - c.fromManifest = ctx.manifest.buildConfig.namedRunners.contains( - std::string(which)); - c.tmpl = entry->namedRunners.at(std::string(which)); + if (isDefault) { + if (pf.runner.set) { + c.tmpl = pf.runner.cmd; + c.fromManifest = overrides_program(pf.runner.origin, + !ctx.manifest.buildConfig.runner.empty()); } + } else if (auto it = pf.runners.find(std::string(which)); it != pf.runners.end()) { + c.tmpl = it->second.value; + auto nr = ctx.manifest.buildConfig.namedRunners.find(std::string(which)); + c.fromManifest = overrides_program(it->second.origin, + nr != ctx.manifest.buildConfig.namedRunners.end()); + if (nr != ctx.manifest.buildConfig.namedRunners.end() && !c.fromManifest) + c.longLived = nr->second.longLived; } - // AND THE TOOLCHAIN'S OWN ANSWER, WHEN THE PROJECT AND ITS GRAPH GAVE NONE. + // AND THE TOOLCHAIN'S OWN ANSWER, WHEN NOTHING ABOVE GAVE ONE. // - // The third source and the last. A project's `[target.] runner` - // and a package's `mcpp::runner(...)` both outrank it, because they are + // The last source. A project's `[target.] runner`, a platform's + // and a package's `mcpp::runner(...)` all outrank it, because they are // statements about THIS program; the payload's is a statement about // everything its compiler produces. Measured 2026-09-12 in a sandbox with // no `node` on PATH: an Emscripten artefact built correctly and then @@ -1231,12 +1232,6 @@ RunnerChoice choose_device_action(const BuildContext& ctx, c.tmpl = mcpp::toolchain::payload_default_runner(ctx.tc.binaryPath); c.fromPayload = !c.tmpl.empty(); } - // AND THE TARGET'S OWN, FOR AN OK IMAGE: the loader is what runs one, on - // every system (SPEC 12). `okrun` is found as any runner's program is -- - // first among the programs the graph built for this machine - // (`openkal-tools`), then in declared payloads, then on PATH. - if (isDefault && c.tmpl.empty() && ft && ft->is_openkal()) - c.tmpl = { std::string(mcpp::okimage::kRunner) }; // `--no-runner` is the operator on THIS host stating a host fact the // manifest cannot carry: the triple is native here. On a freestanding // target that leaves nothing to execute, and the caller's existing @@ -2027,22 +2022,28 @@ export int list_runners(const std::string& package_filter, if (!ctx) { mcpp::ui::error(ctx.error()); return 2; } const auto& bc = ctx->manifest.buildConfig; + // The runners prepare merged: the platform's, a build program's, this + // machine's and the project's, each with where it was stated. + const auto& pf = ctx->plan.platform.fields; const auto ft = mcpp::toolchain::triple::parse(ctx->tc.targetTriple); const std::string key = ft ? ft->str() : ctx->tc.targetTriple; std::println("Target {}", key); - if (bc.runner.empty() && bc.namedRunners.empty()) { + if (!pf.runner.set && pf.runners.empty()) { std::println(" (none — this project reaches its artifact by executing it)"); return 0; } - if (!bc.runner.empty()) - std::println(" {:<12} {}", "(default)", bc.runner.front()); - for (auto const& [name, nr] : bc.namedRunners) { - std::println(" {:<12} {}{}", name, - nr.argv.empty() ? std::string("(no argv)") : nr.argv.front(), - nr.longLived ? " [long-lived]" : ""); + if (pf.runner.set) + std::println(" {:<12} {} ({})", "(default)", pf.runner.cmd.front(), + mcpp::platform::origin_text(pf.runner.origin)); + for (auto const& [name, w] : pf.runners) { + auto nr = bc.namedRunners.find(name); + const bool longLived = nr != bc.namedRunners.end() && nr->second.longLived; + std::println(" {:<12} {} ({}){}", name, + w.value.empty() ? std::string("(no argv)") : w.value.front(), + mcpp::platform::origin_text(w.origin), longLived ? " [long-lived]" : ""); } - if (bc.runExclusive) + if (bc.runExclusive || pf.runner.exclusive) std::println(" note: this target's runs cannot overlap; `mcpp test` " "serialises them"); return 0; @@ -3819,7 +3820,8 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, // Clamped rather than made an error: an exclusive target with one test // is an ordinary run, and refusing it would turn a correct // configuration into a failure. - const bool exclusiveDevice = ctx->manifest.buildConfig.runExclusive + const bool exclusiveDevice = (ctx->manifest.buildConfig.runExclusive + || ctx->plan.platform.fields.runner.exclusive) && !runnerChoice.tmpl.empty(); const int runJobsHere = exclusiveDevice ? 1 : runJobs; if (exclusiveDevice && runJobs > 1 && list.size() > 1) { diff --git a/src/build/flags.cppm b/src/build/flags.cppm index bf2acdd6c..e305fe53b 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -20,7 +20,7 @@ import mcpp.build.refusal; import mcpp.diag; import mcpp.freestanding.target; import mcpp.freestanding.linkline; -import mcpp.okimage; +import mcpp.platform; import mcpp.manifest.types; import mcpp.manifest.flag_words; import mcpp.modgraph.scanner; @@ -2467,40 +2467,71 @@ CompileFlags compute_flags(const BuildPlan& plan) { } } - // ── An ok image (`-openkal`): the convention's flags, and the link - // by `ld.lld` itself ── + // ── A target whose loader supplies the system: the layer rules ── // - // Applied by replacement, after every hosted decision, for the reason the - // freestanding block gives: an image has no loader path, no crt files and - // no system library, and what it holds above `kal_*` is the graph's, - // reached through the graph's own `-L`/`-l`. The compile keeps the hosted - // header model of a graph-supplied C library (`-nostdlibinc`, above); the - // C++ headers are withheld unless the graph supplies them too, because the - // payload's were configured for its own operating system. + // The compile keeps the hosted header model of a graph-supplied C library + // (`-nostdlibinc`, above); the C++ headers are withheld unless the graph + // supplies them too, because the payload's were configured for its own + // operating system. if (const auto ot = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); - ot && ot->is_openkal()) { - std::string image; - for (auto const& w : mcpp::okimage::compile_flags(ot->arch)) image += " " + w; - if (!plan.targetSide.cxx.fromGraph()) image += " -nostdinc++"; - // AN IMAGE WITH NO C LIBRARY IS A FREESTANDING C ENVIRONMENT, in the + ot && ot->loader_supplies_os()) { + std::string w; + if (!plan.targetSide.cxx.fromGraph()) w += " -nostdinc++"; + // A PROGRAM WITH NO C LIBRARY IS A FREESTANDING C ENVIRONMENT, in the // standard's sense: nothing defines `strlen`, so the compiler must not // turn a loop into a call to it. Measured: `--release` of an image // that defines only `memcpy` and `memset` was refused by `okld check` // for importing `strlen`, which `-O2` had synthesised. - if (plan.targetSide.cAbi.absent()) image += " -ffreestanding"; - f.cxx += image; - f.cc += image; - f.as += image; + if (plan.targetSide.cAbi.absent()) w += " -ffreestanding"; + f.cxx += w; + f.cc += w; + f.as += w; + } - f.ldDriver = mcpp::freestanding::resolve_lld(plan.toolchain.binaryPath); - f.ld = mcpp::okimage::link_flags() + user_ldflags; - f.ldC = f.ld; - f.sysroot.clear(); - f.bFlag.clear(); - f.ldRuntimeFallback.clear(); - f.linkage = "static"; - for (auto& r : f.ldStdlibByRole) r.clear(); - for (auto& r : f.ldStdlibCByRole) r.clear(); + // ── THE PLATFORM'S CONVENTIONS (SPEC-013 §2.5, §4.5) ── + // + // Its flags come before every unit's own, so a package adds to a + // convention and cannot remove one. A platform that links with `ld.lld` + // itself (`[platform.link] driver = "lld"`, an ok image) gets the link it + // states, by replacement, after every hosted decision, for the reason the + // freestanding block gives: the artifact has no loader path, no crt files + // and no system library of the toolchain's, and what it holds is the + // graph's, reached through the graph's own `-L`/`-l`. + if (auto const& p = plan.platform; p.active) { + auto words = [](const std::vector& v) { + std::string out; + for (auto const& w : v) + out += " " + (w.find(' ') == std::string::npos ? w : "\"" + w + "\""); + return out; + }; + f.cc += words(p.flags.cflags); + f.cxx += words(p.flags.cxxflags); + f.as += words(p.flags.asmflags); + if (p.fields.link.driver.value == "lld") { + f.ldDriver = mcpp::freestanding::resolve_lld(plan.toolchain.binaryPath); + f.ld = words(p.fields.link.flags.value) + words(p.flags.ldflags) + user_ldflags; + // `lld` by that name is the generic driver, which chooses its + // flavour from its name or from `-flavor`; a payload that ships + // only it is asked for the ELF linker explicitly. + if (f.ldDriver.stem() == "lld") f.ld = " -flavor gnu" + f.ld; + f.ldC = f.ld; + f.sysroot.clear(); + f.bFlag.clear(); + f.ldRuntimeFallback.clear(); + f.linkage = "static"; + for (auto& r : f.ldStdlibByRole) r.clear(); + for (auto& r : f.ldStdlibCByRole) r.clear(); + } else { + f.ld += words(p.flags.ldflags); + f.ldC += words(p.flags.ldflags); + } + // A linker script the platform states: `-T` is the linker's own flag, + // and both drivers pass it through. + if (p.fields.link.script.set) { + const auto t = " -T " + escape_path(p.fields.link.script.value); + f.ld += t; + f.ldC += t; + } } // A LINKER STATED BY ROLE (mcpp#755, `tools = { ld = ... }`), AFTER EVERY diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 71d597016..847cd0748 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -44,7 +44,7 @@ import mcpp.toolchain.dialect; import mcpp.toolchain.provider; import mcpp.toolchain.registry; import mcpp.toolchain.triple; // shared_soname_flag decides by TARGET, not host -import mcpp.okimage; +import mcpp.toolchain.fingerprint; // hash_string: a platform seal's digest import mcpp.toolchain.msvc_coroutines; import mcpp.provider; import mcpp.provider.select; @@ -590,6 +590,12 @@ std::string pe_link_flag(const BuildPlan& plan, bool sep, return sep ? flag : "-Wl," + flag; } +// The platform links with `ld.lld` itself (`[platform.link] driver = "lld"`): +// a flag is the linker's own spelling, not the driver's `-Wl,` form. +bool links_with_lld(const BuildPlan& plan) { + return plan.platform.active && plan.platform.fields.link.driver.value == "lld"; +} + std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { if (lu.kind != LinkUnit::SharedLibrary) return ""; const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); @@ -605,8 +611,8 @@ std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { const std::string name = lu.soname.empty() ? lu.output.filename().string() : lu.soname; if (macho) return "-Wl,-install_name,@rpath/" + name; - // An ok image is linked by `ld.lld` itself, which takes the bare flag. - if (t && t->is_openkal()) return lu.soname.empty() ? "" : "-soname " + lu.soname; + // A platform that links with `ld.lld` itself takes the bare flag. + if (links_with_lld(plan)) return lu.soname.empty() ? "" : "-soname " + lu.soname; return lu.soname.empty() ? "" : "-Wl,-soname," + lu.soname; } @@ -620,7 +626,7 @@ std::string shared_soname_default(const LinkUnit& lu, const BuildPlan& plan) { const bool pe = t ? t->is_pe() : bool(mcpplibs::os::is_windows); const bool macho = t ? t->is_mach_o() : bool(mcpplibs::os::is_macos); if (pe || macho) return ""; - if (t && t->is_openkal()) return "-soname " + lu.output.filename().string(); + if (links_with_lld(plan)) return "-soname " + lu.output.filename().string(); return "-Wl,-soname," + lu.output.filename().string(); } @@ -1528,34 +1534,92 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, placedFiles.push_back({{pl.source}, pl.dest}); } - // ── An ok image: its notes, and the step that finishes it ────────────── + // ── A platform's link inputs and seal (SPEC-013 §4, §4.6) ────────────── // - // The notes are one assembly source the engine writes (SPEC 12.4) and one - // object every image links. `okimage_finish` runs `okld note` and `okld - // check` on the linked image and only then publishes it under its name, - // so an image that breaks the convention never reaches `bin/`. - const bool okImageTarget = [&] { - const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); - return t && t->is_openkal(); - }(); - const auto okNoteSource = std::filesystem::path("obj") / "okimage" / "note.S"; - const auto okNoteObject = std::filesystem::path("obj") / "okimage" / "note.o"; - if (okImageTarget) { - std::error_code mkec; - std::filesystem::create_directories((plan.outputDir / okNoteSource).parent_path(), mkec); - write_file(plan.outputDir / okNoteSource, mcpp::okimage::note_source()); + // An input the platform names is linked into every executable and shared + // library: a file of the platform package, or the standard output of a + // program saved under `obj/platform/` by its own edge; an assembly + // source is assembled for the target first, by the compiler and nothing + // of the project's flags (the notes of an ok image are the case). The + // seal steps run on the linked file, written beneath `obj/platform/`, and + // only a file every step accepted is published under the artifact's name. + auto const& pf = plan.platform; + const bool platformLld = pf.active && pf.fields.link.driver.value == "lld"; + const bool platformSeal = pf.active && !pf.fields.link.seal.empty(); + // A program a platform names, as the platform resolved it, or its name for + // PATH at build time. + auto platform_program = [&](const std::string& name) { + auto it = pf.tools.find(name); + return it == pf.tools.end() ? name : it->second; + }; + std::vector platformInputs; + if (pf.active && !pf.fields.link.inputs.empty()) { const auto tt = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); - append("rule okimage_note\n"); - append(std::format(" command = $cc --target={} --no-default-config -c $in -o $out\n", - tt->llvm_triple())); - append(" description = NOTE $out\n\n"); - append(std::format("build {} : okimage_note {}\n\n", - escape_ninja_path(okNoteObject), escape_ninja_path(okNoteSource))); - append(std::format("okld = {}\n", ninja_tool_word(plan.okldPath))); - append("okimage_with =\n"); - append("rule okimage_finish\n"); - append(" command = $mcpp okimage-finish --okld $okld -- $in $out $okimage_with\n"); - append(" description = IMAGE $out\n\n"); + const bool clangDriver = plan.toolchain.compiler == mcpp::toolchain::CompilerId::Clang; + append("rule platform_capture\n"); + append(" command = $mcpp platform-step capture --out $out -- $argv\n"); + append(" description = GEN $out\n\n"); + append("rule platform_asm\n"); + append(std::format(" command = $cc {}-c $in -o $out\n", + clangDriver && tt ? std::format("--target={} --no-default-config ", + tt->llvm_triple()) + : std::string{})); + append(" description = AS $out\n\n"); + for (auto const& in : pf.fields.link.inputs) { + std::filesystem::path src; + if (!in.from.empty()) { + src = std::filesystem::path("obj") / "platform" / in.as; + std::string argv, implicitIn; + for (std::size_t i = 0; i < in.from.size(); ++i) { + const auto word = i == 0 ? platform_program(in.from[0]) : in.from[i]; + argv += " " + ninja_command_word(word); + } + if (auto it = pf.tools.find(in.from[0]); it != pf.tools.end()) + implicitIn = " | " + escape_ninja_path(std::filesystem::path(it->second)); + append(std::format("build {} : platform_capture{}\n", escape_ninja_path(src), implicitIn)); + append(" argv =" + argv + "\n\n"); + } else { + src = std::filesystem::path(in.file); + } + const auto ext = src.extension().string(); + if (ext == ".S" || ext == ".s") { + auto obj = std::filesystem::path("obj") / "platform" / src.filename(); + obj.replace_extension(".o"); + append(std::format("build {} : platform_asm {}\n\n", + escape_ninja_path(obj), escape_ninja_path(src))); + platformInputs.push_back(std::move(obj)); + } else { + platformInputs.push_back(std::move(src)); + } + } + } + // The shared artifacts of this plan, which a seal step's `{needed}` names. + std::set sharedOutputs; + for (auto const& lu : plan.linkUnits) + if (lu.kind == LinkUnit::SharedLibrary) sharedOutputs.insert(lu.output); + if (platformSeal) { + // The steps, one per line, words separated by a tab, beside the + // configuration; their digest is part of the command, so a changed + // step reseals. + std::string steps; + for (auto const& step : pf.fields.link.seal) { + std::string line; + for (std::size_t i = 0; i < step.size(); ++i) { + if (i) line += '\t'; + line += i == 0 ? platform_program(step[0]) : step[i]; + } + steps += line + "\n"; + } + const auto stepsFile = std::filesystem::path("obj") / "platform" / "seal.steps"; + std::error_code mkec; + std::filesystem::create_directories((plan.outputDir / stepsFile).parent_path(), mkec); + write_file(plan.outputDir / stepsFile, steps); + append("seal_with =\n"); + append("rule platform_seal\n"); + append(std::format(" command = $mcpp platform-step seal --steps {} --digest {} -- $in $out $seal_with\n", + ninja_command_word(stepsFile.generic_string()), + mcpp::toolchain::hash_string(steps))); + append(" description = SEAL $out\n\n"); } // ── The raw image a flasher takes ────────────────────────────────────── @@ -2327,18 +2391,18 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, "$ld /nologo /DLL /OUT:$out $implib_flag $def_flag " "$in $ldflags $unit_ldflags", "SHARED"); - } else if (okImageTarget) { - // AN OK IMAGE, LINKED BY `ld.lld` ITSELF (SPEC 12.2). An executable - // image is ET_DYN with the entry `ok_main`; an image another loads - // has its name and no entry. Each edge writes - // `obj/okimage/.unchecked`; the `okimage_finish` edge below - // writes the requirements note, - // checks the convention and publishes ``. - const std::string entry = std::format("-e {}", mcpp::okimage::kEntry); + } else if (platformLld) { + // A PLATFORM THAT LINKS WITH `ld.lld` ITSELF (an ok image, SPEC 12.2): + // its flags are the whole line, an executable is entered at the + // symbol the platform states, and an artifact another loads has + // its name and no entry. When the platform seals, each edge writes + // beneath `obj/platform/` and the seal edge below publishes. + const std::string entry = pf.fields.link.start.set + ? std::format(" -e {}", pf.fields.link.start.value) : std::string{}; link_rule("cxx_link", - "$ld_driver $ldflags $unit_ldflags " + entry + " -o $out $in", "LINK"); + "$ld_driver $ldflags $unit_ldflags" + entry + " -o $out $in", "LINK"); link_rule("c_link", - "$ld_driver $c_ldflags $unit_ldflags " + entry + " -o $out $in", "LINK"); + "$ld_driver $c_ldflags $unit_ldflags" + entry + " -o $out $in", "LINK"); link_rule("cxx_shared", "$ld_driver $ldflags $soname_default $soname_flag $unit_ldflags -o $out $in", "SHARED"); @@ -3348,14 +3412,16 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, for (auto& o : lu.objects) { ins += " " + escape_ninja_path(o); } - // An ok image's notes, in every image this plan links; the image is - // written beside its name and finished by its own edge (above). - const bool okImageUnit = okImageTarget && lu.kind != LinkUnit::StaticLibrary; - if (okImageUnit) ins += " " + escape_ninja_path(okNoteObject); - // Beneath obj/, not beside the name: what bin/ holds is what was - // published, and an unchecked image is not. - auto okUnchecked = std::filesystem::path("obj/okimage") / lu.output.relative_path(); - okUnchecked += ".unchecked"; + // The platform's link inputs, in every executable and shared library + // this plan links; with seal steps, the artifact is written beneath + // obj/ and published by its own edge (above). Not beside the name: + // what bin/ holds is what was published, and an unsealed file is not. + const bool platformUnit = pf.active && lu.kind != LinkUnit::StaticLibrary; + if (platformUnit) + for (auto const& in : platformInputs) ins += " " + escape_ninja_path(in); + const bool sealUnit = platformSeal && lu.kind != LinkUnit::StaticLibrary; + auto unsealed = std::filesystem::path("obj/platform") / lu.output.relative_path(); + unsealed += ".unsealed"; // mcpp#426. `separateLinker` (link.exe) takes no driver at all, so the // question does not arise there and the msvc rules are untouched. @@ -3509,7 +3575,7 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, // order-only list of the edge. if (lu.kind != LinkUnit::StaticLibrary) orderOnly += linkOrderOnly; std::string out_line = std::format("build {}{} : {}{}{}{}\n", - escape_ninja_path(okImageUnit ? okUnchecked : lu.output), implicitOut, rule, ins, + escape_ninja_path(sealUnit ? unsealed : lu.output), implicitOut, rule, ins, implicit.empty() ? std::string{} : " |" + implicit, orderOnly.empty() ? std::string{} : " ||" + orderOnly); if (auto flag = shared_soname_flag(lu, plan); !flag.empty()) @@ -3633,21 +3699,21 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, } append(std::move(out_line)); - // The finished image: its requirements written and the convention - // checked against the images it needs, then published (SPEC 12.4). - if (okImageUnit) { + // The sealed artifact: every step of the platform's seal run on it, + // the shared artifacts it needs named to each, then published (§4.6). + if (sealUnit) { std::string with, withInputs; auto add_needed = [&](const std::filesystem::path& p) { - if (p.extension() != ".okso") return; + if (!sharedOutputs.contains(p)) return; with += " " + ninja_command_word(p.generic_string()); withInputs += " " + escape_ninja_path(p); }; for (auto const& o : lu.objects) add_needed(o); for (auto const& o : lu.implicitInputs) add_needed(o); - append(std::format("build {} : okimage_finish {}{}\n", - escape_ninja_path(lu.output), escape_ninja_path(okUnchecked), + append(std::format("build {} : platform_seal {}{}\n", + escape_ninja_path(lu.output), escape_ninja_path(unsealed), withInputs.empty() ? std::string{} : " |" + withInputs)); - if (!with.empty()) append(" okimage_with =" + with + "\n"); + if (!with.empty()) append(" seal_with =" + with + "\n"); append("\n"); } @@ -4541,13 +4607,12 @@ std::expected NinjaBackend::build(const BuildPlan& plan // which names an internal rule to somebody who wrote `kind = "shared"` in a // manifest. Defining the rule would be worse: `ld.lld -shared` succeeds and // produces an object nothing on that machine can load. - // An ok image is the exception: an image another loads (`.okso`) is - // placed by the loader (DT_NEEDED) or by `dlopen`, and has its rule. - const bool okImageShared = [&] { - const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); - return t && t->is_openkal(); - }(); - if (!flags.ldDriver.empty() && !okImageShared) { + // A platform that links with ld.lld itself and names its shared + // artifacts is the exception (an ok image's `.okso`, placed by the loader + // through DT_NEEDED or by `dlopen`): it has the rule. + const bool platformShared = plan.platform.active + && plan.platform.fields.link.driver.value == "lld" && plan.platform.fields.link.shared.set; + if (!flags.ldDriver.empty() && !platformShared) { for (auto const& lu : plan.linkUnits) { if (lu.kind != LinkUnit::SharedLibrary) continue; return std::unexpected(BuildError{std::format( diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 0a4ad5fc1..fbb2ef7ab 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -28,6 +28,7 @@ import mcpp.runtime.binding; import mcpp.provider; import mcpp.provider.select; import mcpp.home; +import mcpp.platform; export namespace mcpp::build { @@ -539,10 +540,11 @@ struct BuildPlan { std::map> moduleScopes; std::filesystem::path stdCompatObjectPath; // absolute path to prebuilt std.compat.o std::filesystem::path scanDepsPath; // clang-scan-deps binary (Clang only) - // `okld` for an ok-image target (`-openkal`): the program of the graph - // (or of this machine) that writes each image's requirements note and - // checks the convention after the link (mcpp.okimage). Empty elsewhere. - std::filesystem::path okldPath; + // THE TARGET'S PLATFORM (SPEC-013): the merged statement, its flags for + // this target and the programs its link and seal run. `active` is false + // for a target that uses none; the runner fields are filled either way, + // because a build program and the project state them for any target. + mcpp::platform::Resolved platform; // NASM assembly (.asm sources). Both resolved in prepare AFTER the plan // exists — only when the plan actually contains .asm units (lazy, hard // failure when unavailable; never a silent skip). @@ -769,7 +771,10 @@ make_plan(const mcpp::manifest::Manifest& manifest, // cache address of a dependency source that lives outside its own // package root (a build.mcpp OUT_DIR product). Empty is legal and // simply makes those units uncacheable. - const std::vector& storeRoots = {}); + const std::vector& storeRoots = {}, + // The target's platform (SPEC-013): its artifact names, when it + // states them, replace the target row's. + const mcpp::platform::Resolved* platform = nullptr); // Expand one manifest `include_dirs` entry against the project root — the // #249 consistency join + the expand_dir_glob the dep path uses. Exported @@ -1639,7 +1644,8 @@ make_plan(const mcpp::manifest::Manifest& manifest, const std::filesystem::path& outputDir, const std::filesystem::path& stdBmiPath, const std::filesystem::path& stdObjectPath, - const std::vector& storeRoots) + const std::vector& storeRoots, + const mcpp::platform::Resolved* platform) { BuildPlan plan; plan.manifest = manifest; @@ -1664,7 +1670,26 @@ make_plan(const mcpp::manifest::Manifest& manifest, auto t = mcpp::toolchain::triple::parse(tc.targetTriple); return t ? *t : mcpp::toolchain::triple::Triple{}; }(); - const auto naming = naming_for(tc); + auto naming = naming_for(tc); + // A PLATFORM NAMES ITS ARTIFACTS (`[platform.link] exe`, `shared`): an + // ok image is `.oke`, an image another loads `lib.okso`. + // The patterns were checked where they were read. + // `ArtifactNaming` holds views; these are views into the platform's own + // strings, which outlive the plan's construction. + if (platform && platform->active) { + constexpr std::string_view kName = "{name}"; + auto const& l = platform->fields.link; + if (l.exe.set && mcpp::platform::split_name_pattern(l.exe.value)) { + const std::string_view v = l.exe.value; + if (v.starts_with(kName)) naming.exeSuffix = v.substr(kName.size()); + } + if (l.shared.set && mcpp::platform::split_name_pattern(l.shared.value)) { + const std::string_view v = l.shared.value; + const auto at = v.find(kName); + naming.libPrefix = v.substr(0, at); + naming.sharedLibExt = v.substr(at + kName.size()); + } + } // The loader-tag contract exists only where DT_RPATH/DT_RUNPATH do. // Mach-O and PE have neither, so they get no flag rather than a branch in @@ -3085,8 +3110,9 @@ make_plan(const mcpp::manifest::Manifest& manifest, t.name, all)); } if (!targetTriple.is_freestanding()) { - const std::string owner = targetTriple.is_openkal() - ? std::string("okrun, at `ok_main`, which the image's C library defines") + const std::string owner = targetTriple.loader_supplies_os() + ? std::string("its loader, at the entry its platform states " + "(`[platform.link] start`), which the image's C library defines") : targetTriple.is_android() ? std::string("the Android runtime, through the exports the platform names") : std::string("the C runtime (`_start`, `mainCRTStartup`), which initialises " diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 186b999e1..0f7cb7ab5 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -971,6 +971,12 @@ export struct BuildOverrides { // `prepare_build` itself, never by a caller. std::vector toolchain_statement; std::string bootstrap_spec; + // THE PLATFORM PACKAGE'S DIRECTORY, when this prepare is the second pass + // of a build whose platform states a target base (`[platform] sysroot`): + // the base is read before the graph exists, so the first pass fetched the + // package and the second reads its `[platform]` from here at once + // (SPEC-013 §7). Set by `prepare_build` itself, never by a caller. + std::filesystem::path platform_root; }; // ── git dependency helpers ────────────────────────────────────────────────── diff --git a/src/build/prepare/base.cpp b/src/build/prepare/base.cpp index 3750bd914..4b6923886 100644 --- a/src/build/prepare/base.cpp +++ b/src/build/prepare/base.cpp @@ -181,7 +181,15 @@ base_statement(PrepareState& state, const mcpp::toolchain::triple::Triple& t) { return settle(std::move(s)); } } - // 4. The engine's: the row, read where it always was (`effective_sysroot`). + // 4. The engine's: the platform's `sysroot` (SPEC-013 §2), then the row, + // read where it always was (`effective_sysroot`). + if (state.platformEarly && state.platformData.sysroot.set) { + auto const& o = state.platformData.sysroot.origin; + auto s = stamped(tb::parse_text(state.platformData.sysroot.value), "platform", o.file, + o.line, "[platform].sysroot"); + if (s) s->provides = state.platformData.sysrootProvides; + return settle(std::move(s)); + } return settle(tb::Statement{}); } diff --git a/src/build/prepare/driver.cpp b/src/build/prepare/driver.cpp index 0e0ec7bee..6369d834c 100644 --- a/src/build/prepare/driver.cpp +++ b/src/build/prepare/driver.cpp @@ -90,7 +90,14 @@ prepare_build(bool print_fingerprint, return prepare_build_pass(print_fingerprint, includeDevDeps, std::move(extraTargets), std::move(overrides)); auto first = prepare_build_pass(print_fingerprint, includeDevDeps, extraTargets, overrides); - if (first) { (void)take_toolchain_restart(); return first; } + if (first) { (void)take_toolchain_restart(); (void)take_platform_restart(); return first; } + // A platform that states a target base, fetched by this pass: the next + // reads its `[platform]` before the base is decided (SPEC-013 §7). + if (auto root = take_platform_restart(); root && overrides.platform_root.empty()) { + overrides.platform_root = std::move(*root); + return prepare_build(print_fingerprint, includeDevDeps, std::move(extraTargets), + std::move(overrides)); + } auto restart = take_toolchain_restart(); if (!restart) return first; overrides.toolchain_statement = std::move(restart->first); diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index 771d6774f..389b78ba6 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -2364,6 +2364,10 @@ std::expected phase4b_graph_worklist(PrepareState& state) { childSpec.features.push_back(depFeat); }; + // The platform package first: its `[platform]` decides what the graph + // is completed with once the root's own requests are walked (SPEC-013). + step4b_seed_platform(state); + // Seed the worklist from the main manifest. Dev-deps only when the // caller wants them; they're never propagated transitively. const std::string mainPkgLabel = state.m->package.name; @@ -2398,25 +2402,40 @@ std::expected phase4b_graph_worklist(PrepareState& state) { } - while (!state.worklist.empty()) { - WorklistItemCtx ctx; - ctx.item = std::move(state.worklist.front()); - state.worklist.pop_front(); + auto drain = [&]() -> std::expected { + while (!state.worklist.empty()) { + WorklistItemCtx ctx; + ctx.item = std::move(state.worklist.front()); + state.worklist.pop_front(); - if (auto r = step4b_resolve_identity(state, ctx); !r) - return std::unexpected(r.error()); + if (auto r = step4b_resolve_identity(state, ctx); !r) + return std::unexpected(r.error()); + + if (auto it = state.resolved.find(ctx.key); it != state.resolved.end()) { + if (auto r = step4b_handle_already_resolved(state, ctx, it); !r) + return std::unexpected(r.error()); + continue; + } - if (auto it = state.resolved.find(ctx.key); it != state.resolved.end()) { - if (auto r = step4b_handle_already_resolved(state, ctx, it); !r) + if (auto r = step4b_acquire_dependency_source(state, ctx); !r) + return std::unexpected(r.error()); + if (auto r = step4b_finalize_dependency(state, ctx); !r) return std::unexpected(r.error()); - continue; } - - if (auto r = step4b_acquire_dependency_source(state, ctx); !r) - return std::unexpected(r.error()); - if (auto r = step4b_finalize_dependency(state, ctx); !r) - return std::unexpected(r.error()); + return {}; + }; + if (auto r = drain(); !r) return r; + // THE PLATFORM COMPLETES THE GRAPH (SPEC-013 §4.3, §4.4): the layers no + // package of the graph provides and the tools it does not build are added + // as the platform's, and walked like any request. A few rounds at most: the + // packages one round adds may provide a layer the next then leaves. + for (int round = 0; round < 3; ++round) { + auto pushed = step4b_apply_platform(state); + if (!pushed) return std::unexpected(pushed.error()); + if (!*pushed) break; + if (auto r = drain(); !r) return r; } + if (auto r = step4b_check_platform_versions(state); !r) return r; if (auto r = step4b_cycle_check(state); !r) return std::unexpected(r.error()); diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 7dce6c016..a732b58f7 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -3,7 +3,6 @@ // resolution.json are written by records.cpp. module mcpp.build.prepare; -import mcpp.okimage; import :state; import mcpp.build.prepare_inputs; @@ -448,9 +447,12 @@ static std::expected step13_make_plan(PrepareState& state, Bu for (std::size_t i = 1; i < state.packages.size(); ++i) state.packages[i].linkUsage.ldflags = normalized_dependency_ldflags( state.packages[i].root, state.packages[i].manifest.buildConfig.ldflags); + // The platform's statement, final: every build program has run. + step13_platform_statement(state); auto planResult = mcpp::build::make_plan(*state.m, *state.tc, state.fp, state.scan.graph, state.report.topoOrder, state.packages, *state.root, ctx.outputDir, - state.stdBmiPath, state.stdObjectPath, state.storeRoots); + state.stdBmiPath, state.stdObjectPath, state.storeRoots, + &state.platform); if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); // `trim_paths` per package (mcpp#844, design 2026-10-26 part 2 §10.4): a @@ -1482,36 +1484,11 @@ static std::expected step13_build_graph_actions(PrepareState& } static std::expected step13_assembly_units(PrepareState& state, BuildContext& ctx) { - // AN OK IMAGE IS FINISHED BY `okld` (mcpp.okimage): the graph's, a - // declared payload's, or this machine's, in that order. Its absence is a - // refusal before anything compiles, naming the dependency that supplies - // it -- not a link that succeeds and an image no loader accepts. - // After `make plan`, which builds the plan these fields belong to. - if (auto tt = mcpp::toolchain::triple::parse(state.tc->targetTriple); tt && tt->is_openkal()) { - const std::string tool(mcpp::okimage::kTool); - if (auto it = state.rootTools.find(tool); it != state.rootTools.end()) { - ctx.plan.okldPath = it->second; - } else { - for (auto const& dir : ctx.xlingsDepBinDirs) { - for (auto const& name : { tool, tool + ".exe" }) { - std::error_code ec; - if (std::filesystem::is_regular_file(dir / name, ec)) { ctx.plan.okldPath = dir / name; break; } - } - if (!ctx.plan.okldPath.empty()) break; - } - if (ctx.plan.okldPath.empty()) - if (auto found = mcpp::invocation::which(tool)) ctx.plan.okldPath = *found; - } - // Only a build that links an image needs it: a library's archive is - // objects, and the image that takes them is checked where it is linked. - const bool linksAnImage = std::ranges::any_of(ctx.plan.linkUnits, [](auto const& lu) { - return lu.kind != mcpp::build::LinkUnit::StaticLibrary; - }); - if (ctx.plan.okldPath.empty() && linksAnImage && !state.overrides.plan_only) { - mcpp::build::refusal::record(mcpp::build::refusal::Code::ImageToolAbsent); - return std::unexpected(mcpp::okimage::no_tool_message(tt->str(), tool)); - } - } + // THE PLATFORM, FINISHED (SPEC-013): the statement every reader uses, + // with the build programs' directives in it, its flags evaluated for the + // target, and the programs its link and seal run located. After `make + // plan`, which builds the plan this belongs to. + if (auto r = step13_finish_platform(state, ctx); !r) return std::unexpected(r.error()); // Clang: discover clang-scan-deps for P1689 dyndep scanning. if (mcpp::toolchain::is_clang(*state.tc)) { if (auto sd = mcpp::toolchain::clang::find_scan_deps(*state.tc)) { diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp new file mode 100644 index 000000000..47f86f384 --- /dev/null +++ b/src/build/prepare/platform.cpp @@ -0,0 +1,561 @@ +// platform.cpp -- the platform of the target graph through the prepare phases +// (SPEC-013; design 2026-10-28 §5, §7-§9). Declared in `:state`. +// +// P1 which platform (`step1_select_platform`): before the conditional +// tables merge, so `cfg(platform = ...)` is answered with the rest of +// the target's keys. +// P4b the platform package is the root's first dependency +// (`step4b_seed_platform`); once the graph has been walked, its +// `[platform]` is read and merged with this machine's and the +// project's overrides, and the layers and tools the graph lacks are +// added to it (`step4b_apply_platform`, walked again by the caller); +// a version group the graph contradicts is refused +// (`step4b_check_platform_versions`). +// P5 its toolchain (`step5_platform_toolchain`). +// P13 the statement every reader uses (`step13_finish_platform`): +// directives merged in, flags evaluated for the target, tools located. + +module mcpp.build.prepare; +import :state; + +import std; +import mcpp.build.prepare_inputs; +import mcpp.build.refusal; +import mcpp.config; +import mcpp.invocation; +import mcpp.manifest; +import mcpp.platform; +import mcpp.pm.compat; +import mcpp.targetside; +import mcpp.toolchain.triple; +import mcpp.ui; +import mcpp.freestanding.linkline; + +namespace mcpp::build { + +namespace fs = std::filesystem; +namespace mp = mcpp::manifest::platform; + +namespace { + +mcpp::toolchain::triple::Triple target_of(const PrepareState& state) { + if (!state.overrides.target_triple.empty()) + if (auto t = mcpp::toolchain::triple::parse(state.overrides.target_triple)) return *t; + return mcpp::toolchain::triple::host_triple(); +} + +// The package short name a reference names, before the package is fetched: +// the reference's own name, the `[package]` of a directory, or the last +// segment of a repository's URL. +std::string known_short_name(const mp::PackageRef& r) { + if (!r.spec.shortName.empty()) return r.spec.shortName; + if (!r.spec.path.empty()) { + if (auto m = mcpp::manifest::load(fs::path(r.spec.path) / "mcpp.toml")) + return mcpp::pm::compat::resolve_package_name(m->package.name, m->package.namespace_) + .shortName; + return {}; + } + if (!r.spec.git.empty()) { + std::string_view u = r.spec.git; + while (!u.empty() && u.back() == '/') u.remove_suffix(1); + if (auto s = u.find_last_of("/:"); s != std::string_view::npos) u = u.substr(s + 1); + if (u.ends_with(".git")) u.remove_suffix(4); + return std::string(u); + } + return {}; +} + +// The project's override tables: the workspace layer's, then the plan root's. +std::vector project_tables(const mcpp::manifest::Manifest& m) { + std::vector out; + if (m.layer) out = m.layer->platformOverrides; + out.insert(out.end(), m.platformOverrides.begin(), m.platformOverrides.end()); + return out; +} + +// Which layers each package of the graph provides, by layer. +std::map provided_layers(const PrepareState& state) { + std::map out; + for (std::size_t i = 1; i < state.packages.size(); ++i) { + auto const& pm = state.packages[i].manifest; + for (auto const& entry : pm.provides) { + auto cap = mcpp::targetside::parse_capability(entry); + if (!cap || !*cap) continue; + auto const& c = **cap; + std::string layer(mcpp::targetside::cap_layer_name(c.layer)); + if (!mp::is_layer(layer)) continue; + out.try_emplace(layer, pm.package.version.empty() + ? pm.package.name + : std::format("{}@{}", pm.package.name, pm.package.version)); + } + } + return out; +} + +std::string qualified(const mcpp::manifest::Manifest& m) { + auto n = mcpp::pm::compat::resolve_package_name(m.package.name, m.package.namespace_); + return mcpp::pm::compat::qualified_name(n.namespace_, n.shortName); +} + +// The key a worklist item for a reference is written under: its name, or +// the short name its source gives. +std::string item_key(const mp::PackageRef& r) { + if (!r.name.empty()) return r.name; + return known_short_name(r); +} + +// The selector evaluator the platform module asks: this graph's context. +mcpp::platform::Matches matcher(cfgpred::Ctx c) { + return [c = std::move(c)](std::string_view sel) { return cfgpred::matches(std::string(sel), c); }; +} + +} // namespace + +cfgpred::Ctx platform_cfg_context(const PrepareState& state) { + return cfgpred::context_for(target_of(state).str(), state.platformPackageShortName); +} + +std::expected step1_select_platform(PrepareState& state) { + // A host sub-build (build programs, tools) is the host graph: its platform + // is the host conventions until a host platform is selected (§6), and a + // platform's own tools are always built so (the bootstrap boundary, §6.2). + if (state.overrides.host_target || state.overrides.tool_depth > 0 + || state.overrides.build_program_graph) + return {}; + const auto t = target_of(state); + const auto flag = mcpp::invocation::get("MCPP_PLATFORM").value_or(""); + const auto envKey = target_env_name(t, "PLATFORM"); + const auto env = mcpp::invocation::get(envKey).value_or(""); + auto project = project_tables(*state.m); + std::vector config; + if (auto cfg = state.get_cfg(false); cfg) config = (*cfg)->platformOverrides; + std::string column; + if (auto* row = mcpp::toolchain::triple::find_known_target(t)) column = std::string(row->platform); + mcpp::platform::ChoiceInputs in{ flag, env, envKey, project, config, column }; + auto sel = mcpp::platform::select_target(in, matcher(cfgpred::context_for(t.str()))); + if (!sel) return std::unexpected(sel.error()); + state.platform = mcpp::platform::Resolved{}; + state.platform.selection = std::move(*sel); + if (!state.platform.selection.none) { + auto const& ref = state.platform.selection.ref; + // A directory is read now, and its manifest's own error is the answer + // when it cannot be. + if (!ref.spec.path.empty() && ref.name.empty()) { + auto m = mcpp::manifest::load(fs::path(ref.spec.path) / "mcpp.toml"); + if (!m) + return std::unexpected(std::format("{}: the platform {}: {}", + mcpp::platform::origin_text(state.platform.selection.origin), + mp::display(ref), m.error().format())); + } + state.platformPackageShortName = known_short_name(ref); + if (state.platformPackageShortName.empty()) + return std::unexpected(std::format( + "{}: the platform {} names no package; write {{ name = \"\", ... }}", + mcpp::platform::origin_text(state.platform.selection.origin), + mp::display(state.platform.selection.ref))); + // READ NOW WHEN IT IS ON DISK: a directory, or the package a first + // pass fetched. The target base is decided before the graph exists, + // and a platform may state one (§7). + fs::path early = state.overrides.platform_root; + if (early.empty() && !state.platform.selection.ref.spec.path.empty()) + early = state.platform.selection.ref.spec.path; + if (!early.empty()) + if (auto m = mcpp::manifest::load(early / "mcpp.toml"); m && m->platformDeclared) { + state.platformData = m->platform; + state.platformEarly = true; + } + } + return {}; +} + +void step4b_seed_platform(PrepareState& state) { + if (state.platform.selection.none) return; + auto const& ref = state.platform.selection.ref; + WorkItem item; + item.name = item_key(ref); + item.spec = ref.spec; + if (item.spec.shortName.empty()) item.spec.shortName = state.platformPackageShortName; + if (item.spec.namespace_.empty()) { + item.spec.namespace_ = std::string(mcpp::pm::kDefaultNamespace); + item.spec.namespaceOmitted = true; + } + if (item.spec.candidates.empty()) + item.spec.candidates = {{ .namespace_ = item.spec.namespace_, .shortName = item.spec.shortName }}; + item.spec.declaredIn = "the platform"; + item.requestedBy = state.m->package.name + " (platform)"; + item.originalConstraint = item.spec.version; + item.consumerDepIndex = kMainConsumer; + state.worklist.push_front(std::move(item)); +} + +namespace { +thread_local std::optional g_platformRestart; +} // namespace + +std::optional take_platform_restart() { + auto r = std::move(g_platformRestart); + g_platformRestart.reset(); + return r; +} + +std::expected step4b_apply_platform(PrepareState& state) { + auto& P = state.platform; + if (P.selection.none) return false; + const auto t = target_of(state); + + // ── The package, once: find it, read it, merge what overrides it ────── + if (!P.active) { + const mcpp::modgraph::PackageRoot* found = nullptr; + const auto& ref = P.selection.ref; + for (std::size_t i = 1; i < state.packages.size(); ++i) { + auto const& pr = state.packages[i]; + auto n = mcpp::pm::compat::resolve_package_name(pr.manifest.package.name, + pr.manifest.package.namespace_); + const bool byPath = !ref.spec.path.empty() + && fs::weakly_canonical(pr.root) == fs::weakly_canonical(fs::path(ref.spec.path)); + if (byPath || n.shortName == state.platformPackageShortName) { found = ≺ break; } + } + if (!found) + return std::unexpected(std::format( + "the platform {} ({}) was not resolved into the graph", + mp::display(ref), mcpp::platform::origin_text(P.selection.origin))); + if (!found->manifest.platformDeclared) { + refusal::record(refusal::Code::PlatformNotAPlatform); + return std::unexpected(std::format( + "{} is selected as the platform for {} ({}), and its mcpp.toml has no " + "[platform] section.\n" + " A platform package states the configuration of its targets there " + "(SPEC-013 §2); select a platform package, or none (--platform none).", + qualified(found->manifest), t.str(), + mcpp::platform::origin_text(P.selection.origin))); + } + P.package = qualified(found->manifest); + P.version = found->manifest.package.version; + P.root = found->root; + auto hosted = mcpp::platform::serves(found->manifest.platform, P.package, t); + if (!hosted) { + refusal::record(refusal::Code::PlatformTargetUnserved); + return std::unexpected(std::format("{} ({})", hosted.error(), + mcpp::platform::origin_text(P.selection.origin))); + } + P.selection.hosted = *hosted; + state.platformData = *hosted ? mcpp::platform::hosted_view(found->manifest.platform) + : found->manifest.platform; + P.fields = state.platformData; + const auto ctx = platform_cfg_context(state); + std::vector config; + if (auto cfg = state.get_cfg(false); cfg) config = (*cfg)->platformOverrides; + for (auto const* o : mcpp::platform::matching(config, matcher(ctx))) + mcpp::platform::merge(P.fields, o->fields); + auto project = project_tables(*state.m); + for (auto const* o : mcpp::platform::matching(project, matcher(ctx))) + mcpp::platform::merge(P.fields, o->fields); + P.active = true; + state.platformInjected.insert(P.package); + // A base the platform states, which this pass decided without: the + // next pass reads the package before deciding it (§7). + if (P.fields.sysroot.set && !state.platformEarly && P.fields.sysroot.origin.tier == mp::Tier::Platform) { + g_platformRestart = P.root; + return std::unexpected(std::string("mcpp:platform-restart")); + } + mcpp::ui::info("Platform", std::format("{} ({})", P.display(), + mcpp::platform::origin_text(P.selection.origin))); + } + + // ── The layers the graph lacks (§4.3) ───────────────────────────────── + bool pushed = false; + const auto provided = provided_layers(state); + std::map> byKey; // item key -> layers + std::map entryOf; + for (auto layer : mp::kLayers) { + auto it = P.fields.layers.find(std::string(layer)); + if (it == P.fields.layers.end()) continue; + auto const& e = it->second; + if (e.ref.off) { + P.layers[std::string(layer)] = { "none", e.origin.tier }; + continue; + } + if (auto pv = provided.find(std::string(layer)); pv != provided.end()) { + // Whose: the platform's when the provider is a package the + // platform added, the graph's otherwise. + const auto providerName = pv->second.substr(0, pv->second.find('@')); + bool ours = false; + for (auto const& q : state.platformInjected) + if (q == providerName || q.ends_with("." + providerName) + || providerName.ends_with("." + q.substr(q.find('.') + 1))) + ours = true; + P.layers[std::string(layer)] = { pv->second, ours ? e.origin.tier : mp::Tier::Graph }; + continue; + } + const auto key = item_key(e.ref); + if (key.empty()) + return std::unexpected(std::format( + "{}: the package for {} names no package; write {{ name = \"\", ... }}", + mcpp::platform::origin_text(e.origin), layer)); + if (state.platformInjected.contains("layer:" + key)) continue; // already walked + byKey[key].push_back(std::string(layer)); + entryOf[key] = &e; + } + for (auto const& [key, layers] : byKey) { + auto const& e = *entryOf.at(key); + WorkItem item; + item.name = key; + item.spec = e.ref.spec; + if (item.spec.shortName.empty()) item.spec.shortName = known_short_name(e.ref); + if (item.spec.namespace_.empty()) { + item.spec.namespace_ = std::string(mcpp::pm::kDefaultNamespace); + item.spec.namespaceOmitted = true; + } + if (item.spec.candidates.empty()) + item.spec.candidates = {{ .namespace_ = item.spec.namespace_, + .shortName = item.spec.shortName }}; + std::string names; + for (auto const& l : layers) { if (!names.empty()) names += ", "; names += l; } + item.spec.declaredIn = std::format("the platform's {}", names); + item.requestedBy = std::format("{} (platform: {})", state.m->package.name, names); + item.originalConstraint = item.spec.version; + item.consumerDepIndex = kMainConsumer; + state.worklist.push_back(std::move(item)); + state.platformInjected.insert("layer:" + key); + state.platformInjected.insert(mcpp::pm::compat::qualified_name( + state.worklist.back().spec.namespace_, state.worklist.back().spec.shortName)); + pushed = true; + } + + // ── The tools the graph does not build (§4.4) ───────────────────────── + std::set requested; // tools the root already asks the graph for + for (auto const& [k, spec] : state.m->buildDependencies) + for (auto const& tool : spec.tools) requested.insert(tool); + std::map> toolsByKey; + std::map toolEntryOf; + for (auto const& [name, e] : P.fields.tools) { + if (!e.ref.program.empty()) continue; // found, not built + if (requested.contains(name)) continue; // the graph builds it already + const auto key = item_key(e.ref); + if (key.empty()) + return std::unexpected(std::format( + "{}: the package for the tool {} names no package; write {{ name = \"\", ... }}", + mcpp::platform::origin_text(e.origin), name)); + if (state.platformInjected.contains("tool:" + name)) continue; + toolsByKey[key].push_back(name); + toolEntryOf[key] = &e; + } + for (auto const& [key, tools] : toolsByKey) { + auto const& e = *toolEntryOf.at(key); + WorkItem item; + item.name = key; + item.spec = e.ref.spec; + if (item.spec.shortName.empty()) item.spec.shortName = known_short_name(e.ref); + if (item.spec.namespace_.empty()) { + item.spec.namespace_ = std::string(mcpp::pm::kDefaultNamespace); + item.spec.namespaceOmitted = true; + } + if (item.spec.candidates.empty()) + item.spec.candidates = {{ .namespace_ = item.spec.namespace_, + .shortName = item.spec.shortName }}; + item.spec.tools = tools; + std::string names; + for (auto const& n : tools) { if (!names.empty()) names += ", "; names += n; } + item.spec.declaredIn = std::format("the platform's tools ({})", names); + item.requestedBy = std::format("{} (platform tools: {})", state.m->package.name, names); + item.originalConstraint = item.spec.version; + item.consumerDepIndex = kMainConsumer; + item.buildOnly = true; + item.fromBuildTable = true; + // The root asks for them as it would for a declared build dependency, + // so the tool provisioning pass builds them for this machine. + state.m->buildDependencies[key] = item.spec; + for (auto const& n : tools) state.platformInjected.insert("tool:" + n); + state.platformInjected.insert(mcpp::pm::compat::qualified_name(item.spec.namespace_, + item.spec.shortName)); + state.worklist.push_back(std::move(item)); + pushed = true; + } + return pushed; +} + +std::expected step4b_check_platform_versions(PrepareState& state) { + auto const& P = state.platform; + if (!P.active || P.fields.versions.empty()) return {}; + for (std::size_t i = 1; i < state.packages.size(); ++i) { + auto const& pm = state.packages[i].manifest; + auto n = mcpp::pm::compat::resolve_package_name(pm.package.name, pm.package.namespace_); + auto it = P.fields.versions.find(n.shortName); + if (it == P.fields.versions.end()) continue; + if (pm.package.version.empty() || pm.package.version == it->second) continue; + // A package taken from a directory or a repository is a development + // tree, which the version group does not bind. + if (auto r = state.resolved.find({n.namespace_, n.shortName}); + r != state.resolved.end() && r->second.source != "version") + continue; + std::string by; + if (auto r = state.resolved.find({n.namespace_, n.shortName}); r != state.resolved.end()) + by = r->second.requestedBy; + refusal::record(refusal::Code::PlatformVersionGroup); + return std::unexpected(std::format( + "{} {} is in this build, and the platform {} is a version group that names {} {}.\n" + " requested by: {}\n" + " The family's packages pin one another exactly, so two versions of one of\n" + " them cannot be linked into one program. Either select the platform release\n" + " of that version ([target.] platform = \"{}@\"), or write\n" + " {} = \"{}\" where it is requested.", + n.shortName, pm.package.version, P.display(), n.shortName, it->second, + by.empty() ? std::string("the graph") : by, + P.package.substr(P.package.find('.') + 1), n.shortName, it->second)); + } + return {}; +} + +void step5_platform_toolchain(PrepareState& state) { + auto const& P = state.platform; + if (!P.active || !P.fields.toolchain.set) return; + // The platform's toolchain stands where the target row's pin stands: it + // replaces a default, never a toolchain the project or this machine named + // for the target (§4.5), and the graph supplying the system does not + // cancel it -- the platform is what supplies it. + if (tc_origin_is_user_explicit(state.tcOrigin)) return; + state.targetPinCandidate = P.fields.toolchain.value; + state.targetPinIsCapability = true; + state.platformToolchainPin = true; +} + +} // namespace mcpp::build + +namespace mcpp::build { + +namespace { + +// A program a platform names, as a runner's program is found: the programs +// the graph built for this machine, then the declared payloads, then PATH. +std::optional locate_program(const PrepareState& state, const BuildContext& ctx, + const std::string& name) { + if (auto it = state.rootTools.find(name); it != state.rootTools.end()) return it->second; + for (auto const& dir : ctx.xlingsDepBinDirs) + for (auto const& n : { name, name + ".exe" }) { + std::error_code ec; + if (fs::is_regular_file(dir / n, ec)) return dir / n; + } + return mcpp::invocation::which(name); +} + +std::vector split_words(std::string_view v) { + std::vector out; + std::string cur; + for (char c : v) { + if (c == ' ' || c == '\t') { if (!cur.empty()) out.push_back(std::exchange(cur, {})); } + else cur += c; + } + if (!cur.empty()) out.push_back(cur); + return out; +} + +} // namespace + +void step13_platform_statement(PrepareState& state) { + auto& P = state.platform; + const auto t = target_of(state); + const auto cfgCtx = platform_cfg_context(state); + + // ── The statement every reader uses: platform → build programs → this + // machine → the project → the environment (§4.5) ────────────────────── + mp::Statement s = P.active ? state.platformData : mp::Statement{}; + mp::Statement program = state.programPlatform; + // A build program's runner and named runners (`mcpp::runner`), the BSP + // spelling of the same fields. + if (!state.m->buildConfig.runner.empty()) { + program.runner.cmd = state.m->buildConfig.runner; + program.runner.set = true; + program.runner.origin = mp::Origin{ mp::Tier::Program, {}, 0, + state.runnerProvider.empty() ? state.m->package.name : state.runnerProvider }; + if (state.m->buildConfig.runExclusive) { + program.runner.exclusive = true; + program.runner.exclusiveSet = true; + } + } + for (auto const& [name, nr] : state.m->buildConfig.namedRunners) { + auto it = state.namedRunnerProvider.find(name); + program.runners[name] = mp::Words{ nr.argv, mp::Origin{ mp::Tier::Program, {}, 0, + it == state.namedRunnerProvider.end() ? state.m->package.name : it->second }, true }; + } + mcpp::platform::merge(s, program); + std::vector config; + if (auto cfg = state.get_cfg(false); cfg) config = (*cfg)->platformOverrides; + for (auto const* o : mcpp::platform::matching(config, matcher(cfgCtx))) mcpp::platform::merge(s, o->fields); + auto project = project_tables(*state.m); + for (auto const* o : mcpp::platform::matching(project, matcher(cfgCtx))) mcpp::platform::merge(s, o->fields); + const auto envKey = target_env_name(t, "RUNNER"); + if (auto e = mcpp::invocation::get(envKey); e && !e->empty()) { + if (*e == "none") { + s.runner = mp::Runner{}; + } else { + s.runner.cmd = split_words(*e); + s.runner.origin = mp::Origin{ mp::Tier::Env, {}, 0, envKey }; + s.runner.set = !s.runner.cmd.empty(); + } + } + P.fields = std::move(s); + P.flags = mcpp::platform::evaluate_flags(P.fields, matcher(cfgCtx)); +} + +std::expected step13_finish_platform(PrepareState& state, BuildContext& ctx) { + auto P = state.platform; + const auto t = target_of(state); + + // ── The programs the link and the seal run (§4.4, §4.6) ─────────────── + if (P.active) { + std::set needed; + for (auto const& in : P.fields.link.inputs) + if (!in.from.empty()) needed.insert(in.from.front()); + for (auto const& step : P.fields.link.seal) + if (!step.empty()) needed.insert(step.front()); + for (auto const& [name, e] : P.fields.tools) + if (auto found = locate_program(state, ctx, e.ref.program.empty() ? name : e.ref.program)) + P.tools[name] = found->generic_string(); + const bool linksSomething = std::ranges::any_of(ctx.plan.linkUnits, [](auto const& lu) { + return lu.kind != mcpp::build::LinkUnit::StaticLibrary; + }); + for (auto const& name : needed) { + if (P.tools.contains(name)) continue; + if (auto found = locate_program(state, ctx, name)) { + P.tools[name] = found->generic_string(); + continue; + } + if (!linksSomething || state.overrides.plan_only) continue; + refusal::record(refusal::Code::PlatformToolAbsent); + auto it = P.fields.tools.find(name); + const std::string from = it == P.fields.tools.end() + ? std::string("the platform names no package that builds it") + : std::format("{} builds it ({})", mp::display(it->second.ref), + mcpp::platform::origin_text(it->second.origin)); + return std::unexpected(std::format( + "the platform {} links {} with `{}`, and it is neither built by the graph nor found " + "on this machine; {}.\n" + " Name the package that builds it for this machine:\n" + "\n" + " [target.'cfg(platform = \"{}\")'.tools]\n" + " {} = \"@\" # or {{ path = \"\" }}, {{ program = \"\" }}", + P.display(), t.str(), name, from, mp::platform_name(state.platformPackageShortName), + name)); + } + } + // A platform that links with ld.lld needs one beside the compiler; its + // absence is said here, not as a link line that starts with a flag. + if (P.active && P.fields.link.driver.value == "lld" && !state.overrides.plan_only + && std::ranges::any_of(ctx.plan.linkUnits, [](auto const& lu) { + return lu.kind != mcpp::build::LinkUnit::StaticLibrary; }) + && mcpp::freestanding::resolve_lld(state.tc->binaryPath).empty()) { + refusal::record(refusal::Code::LldRequiredAbsent); + return std::unexpected(std::format( + "the platform {} links {} with ld.lld itself ({}), and the toolchain has no ld.lld " + "beside {}.\n" + " Use a toolchain whose payload carries it -- the platform's own is {} -- with " + "`[target.] toolchain`, or remove the line that names this one.", + P.display(), t.str(), mcpp::platform::origin_text(P.fields.link.driver.origin), + state.tc->binaryPath.generic_string(), + P.fields.toolchain.set ? P.fields.toolchain.value : std::string("an llvm toolchain"))); + } + ctx.plan.platform = std::move(P); + return {}; +} + +} // namespace mcpp::build diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index 53e4081a9..0dc8df73c 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -398,11 +398,11 @@ step11_package_std_module_source(PrepareState& state) { // Both are read only from a package that ALSO provides the capability // below. A package that named a std module without supplying the // library would be describing something it does not have. - // AN OK IMAGE'S STANDARD LIBRARY IS THE GRAPH'S OR NONE. The payload's - // libc++ was configured for the payload's own operating system; inside an - // image it would be a second answer to a question the image's C++ - // run-time package answers. - if (auto ft = mcpp::toolchain::triple::parse(state.tc->targetTriple); ft && ft->is_openkal()) + // THE STANDARD LIBRARY OF A TARGET WHOSE LOADER SUPPLIES THE SYSTEM (an + // ok image) IS THE GRAPH'S OR NONE. The payload's libc++ was configured + // for the payload's own operating system; inside such a program it would + // be a second answer to a question its C++ run-time package answers. + if (auto ft = mcpp::toolchain::triple::parse(state.tc->targetTriple); ft && ft->loader_supplies_os()) state.tc->clear_std_modules(); for (auto& pkg : state.packages) { if (pkg.manifest.stdModule.empty()) continue; @@ -719,19 +719,34 @@ step11_std_module_availability_gate(PrepareState& state, bool needsStdModule) { state.tc->targetTriple)); } if (auto ft = mcpp::toolchain::triple::parse(state.tc->targetTriple); - ft && ft->is_openkal()) + ft && ft->loader_supplies_os()) { + // Which package the platform names for the layer, and where that + // was said, or that no platform names one (SPEC-013 §4.3). + std::string why = "No platform names one for it"; + if (state.platform.active) { + auto it = state.platform.fields.layers.find("c++-abi"); + why = it == state.platform.fields.layers.end() + ? std::format("The platform {} names none", state.platform.display()) + : it->second.ref.off + ? std::format("The platform's c++-abi is switched off ({})", + mcpp::platform::origin_text(it->second.origin)) + : std::format("The platform {} names {} ({})", state.platform.display(), + mcpp::manifest::platform::display(it->second.ref), + mcpp::platform::origin_text(it->second.origin)); + } return std::unexpected(std::format( - "`import std;` on '{}' needs a C++ run-time inside the image, and " + "`import std;` on '{}' needs a C++ run-time inside the program, and " "the graph supplies none.\n" - " An ok image holds its own C library and C++ run-time, " - "above `kal_*`. Depend on\n" - " the packages that provide them for this target:\n" + " {}. Name the package that provides `mcpp:c++-abi` for this " + "target:\n" "\n" - " [target.'cfg(os = \"openkal\")'.dependencies]\n" - " openkal-musl = \"0.22.0\" # mcpp:c-abi=musl\n" - " openkal-llvm-runtime = \"0.16.0\" # mcpp:c++-abi=libc++, the std module", - state.tc->targetTriple)); + " [target.'cfg(platform = \"{}\")'.layers]\n" + " c++-abi = \"@\"", + state.tc->targetTriple, why, + state.platformPackageShortName.empty() + ? std::string("") + : mcpp::manifest::platform::platform_name(state.platformPackageShortName))); } return std::unexpected(std::format( "source imports std but toolchain '{}' provides no std module source", diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index b36b70f3a..6747b3f62 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -22,6 +22,7 @@ import mcpp.toolchain.platform; import mcpp.run; import mcpp.libs.json; import mcpp.manifest; +import mcpp.platform; import mcpp.source_kind; import mcpp.modgraph.glob; import mcpp.modgraph.graph; @@ -784,6 +785,30 @@ struct PrepareState { // (`tools = [...]`), by program name. A runner's program and an ok image's // `okld` are looked up here first. std::map rootTools; + + // ── PLATFORMS (SPEC-013; design 2026-10-28) ──────────────────────────── + // The target graph's platform: selected in P1, loaded, merged and applied + // to the graph in P4b, its toolchain read in P5, finished in P13 + // (prepare/platform.cpp). + mcpp::platform::Resolved platform; + // The platform package's own `[platform]` (or its hosted view), before + // any override. + mcpp::manifest::platform::Statement platformData; + // The package name `cfg(platform = ...)` is answered from, known in P1. + std::string platformPackageShortName; + // What the platform added to the graph: qualified package names, and + // `layer:` / `tool:` markers. A package listed here is locked + // with `origin = "platform"` and reported as the platform's. + std::set platformInjected; + // The platform's toolchain is the pin of this build (P5), for the line + // that announces the compiler. + bool platformToolchainPin = false; + // `platformData` was read in P1 (a directory, or a second pass), so the + // target base reads the platform's `sysroot` (base.cpp, tier four). + bool platformEarly = false; + // What build programs stated with the platform's directives (Tier::Program): + // a dependency's, the root's, and the platform's plugin. + mcpp::manifest::platform::Statement programPlatform; std::function foldDirectiveTailIntoPrivateBuild; @@ -1055,6 +1080,17 @@ std::expected step2_use_local_toolchain(PrepareState& state, const mcpp::toolchain::ToolchainSpec& spec); std::expected step2_apply_local_toolchain(PrepareState& state); std::expected step6_toolchain_phase(PrepareState& state); +// The platform through the phases (prepare/platform.cpp). +mcpp::build::cfgpred::Ctx platform_cfg_context(const PrepareState& state); +std::expected step1_select_platform(PrepareState& state); +void step4b_seed_platform(PrepareState& state); +std::expected step4b_apply_platform(PrepareState& state); +std::expected step4b_check_platform_versions(PrepareState& state); +void step5_platform_toolchain(PrepareState& state); +void step13_platform_statement(PrepareState& state); +std::expected step13_finish_platform(PrepareState& state, BuildContext& ctx); +// The platform root a first pass asks the second to read in P1, once. +std::optional take_platform_restart(); // The statement and bootstrap spec of a toolchain phase that asked prepare to // start again, taken once. std::optional, std::string>> take_toolchain_restart(); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index 380c4cc78..598848fd2 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -260,7 +260,7 @@ step9_gather_target_side_candidates(PrepareState& state) { in.targetOs = tt->os; in.targetEnv = tt->env; in.freestandingTarget = tt->is_freestanding(); - in.openkalTarget = tt->is_openkal(); + in.loaderTarget = tt->loader_supplies_os(); // NOT `tt->envExplicit`. By this line the triple has been // canonicalised, and the canonical form of `x86_64-linux` is // `x86_64-linux-gnu` — re-parsing it reports a segment the @@ -463,9 +463,9 @@ step9_resolve_and_realise_cabi(PrepareState& state, TargetSideGather& gather) { auto realised = mcpp::toolchain::cenv::realise( *state.resolvedTargetSide.cAbiDecl, tt ? tt->os : std::string{}, tt ? tt->arch : std::string{}, - // An ok image is told no operating system, as a bare machine - // is: the C library's identity macros are the only ones. - tt && (tt->is_freestanding() || tt->is_openkal())); + // A program its loader runs is told no operating system, as a + // bare machine is: the C library's identity macros are the only ones. + tt && (tt->is_freestanding() || tt->loader_supplies_os())); if (!realised) { refusal::record(refusal::Code::CEnvUnrealisable); return std::unexpected(realised.error()); @@ -626,12 +626,12 @@ step9_resolve_and_realise_cabi(PrepareState& state, TargetSideGather& gather) { // availability check links and runs without the archive; refusing it // would trade a diagnosed hazard for a regression. The degradation // names the platform, the file and the package that supplies it. - // An ok image's runtime routines are the image's own by design - // (mcpp.okimage), so its absence there is not a gap to report. + // The runtime routines of a program its loader runs are its own by + // design, so their absence there is not a gap to report. if (auto okt = state.tc ? mcpp::toolchain::triple::parse(state.tc->targetTriple) : std::optional{}; state.resolvedTargetSide.compilerRuntime.absent() && state.tc - && !(okt && okt->is_openkal())) { + && !(okt && okt->loader_supplies_os())) { auto tt = mcpp::toolchain::triple::parse(state.tc->targetTriple); mcpp::diag::degraded("target/compiler-runtime", std::format( "the toolchain payload carries no compiler runtime for {} " @@ -1202,9 +1202,9 @@ step9_same_os_check_and_report(PrepareState& state) { "same-OS check: '{}'(os={}) vs '{}'(os={})", reportedTargetName, want ? want->os : "", state.resolvedTargetSide.llvmTriple, got ? got->os : "")); - // An ok image is compiled for no operating system by design - // (`-unknown-none-elf`): the loader is its environment. - const bool okImage = want && got && want->is_openkal() && got->is_freestanding(); + // A program its loader runs is compiled for no operating system + // by design (`-unknown-none-elf`): the loader is its environment. + const bool okImage = want && got && want->loader_supplies_os() && got->is_freestanding(); if (want && got && !want->os.empty() && !got->os.empty() && want->os != got->os && !okImage) { refusal::record(refusal::Code::OsMismatch); diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index 13a46e644..f82df5dd2 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -1064,10 +1064,11 @@ step1_target_row_pin_and_capability_check(PrepareState& state, TargetOverrideCtx ? "A freestanding target has no per-host cross payload: " "clang and lld are\n" " cross-compilers by construction and gcc is not." - : ctx.parsed->is_openkal() - ? "An ok image is emitted by clang and lld for every host: " - "the compiler is told\n" - " no operating system, and gcc has no such target." + : ctx.parsed->loader_supplies_os() + ? "A target whose loader supplies the system is emitted by " + "clang and lld for\n" + " every host: the compiler is told no operating system, " + "and gcc has no such target." : ctx.parsed->is_wasm() ? "Nothing but Emscripten emits WebAssembly: `em++` is a " "clang whose target,\n" @@ -1202,7 +1203,8 @@ static std::expected step1_device_axis_and_layer_merge(Prepar // over these (prepare_inputs::Ctx::layer_matches); before this the field // was declared, documented, and never written, so the key matched nothing. state.cfgCtx = [&]() { - auto c = cfgpred::context_for(state.overrides.target_triple); + auto c = cfgpred::context_for(state.overrides.target_triple, + state.platformPackageShortName); for (auto const& set : mcpp::pack::parse_accel(state.resolvedAccel())) c.accelerators.push_back(set.backend); return c; @@ -1336,6 +1338,9 @@ std::expected phase1_toolchain_spec_and_axes(PrepareState& st return std::unexpected(r.error()); if (auto r = step1_define_early_toolchain_closures(state); !r) return std::unexpected(r.error()); if (auto r = step1_target_and_static_overrides(state); !r) return std::unexpected(r.error()); + // The platform, once the target is known and before the conditional + // tables merge: `cfg(platform = ...)` is one of their keys (SPEC-013 §3). + if (auto r = step1_select_platform(state); !r) return std::unexpected(r.error()); if (auto r = step1_device_axis_and_layer_merge(state); !r) return std::unexpected(r.error()); return {}; diff --git a/src/build/prepare/toolchain_decision.cpp b/src/build/prepare/toolchain_decision.cpp index 7981be6dd..88a359d7a 100644 --- a/src/build/prepare/toolchain_decision.cpp +++ b/src/build/prepare/toolchain_decision.cpp @@ -223,6 +223,9 @@ std::expected phase5_toolchain_after_graph(PrepareState& stat // chosen for a target it cannot produce. Same shape as the four // defects 2026.8.25.1 fixed: a predicate spanning two layers deciding // something that does not depend on either of them. + // The platform's toolchain, when one is selected, stands where the + // row's pin stands (SPEC-013 §4.5). + step5_platform_toolchain(state); if (!state.targetPinCandidate.empty() && (!graphSuppliesSystem || state.targetPinIsCapability)) { if (state.tcOrigin == TcOrigin::GlobalDefault && state.tcSpec.has_value() diff --git a/src/build/prepare_inputs.cppm b/src/build/prepare_inputs.cppm index 059856118..81fa0396b 100644 --- a/src/build/prepare_inputs.cppm +++ b/src/build/prepare_inputs.cppm @@ -65,6 +65,10 @@ namespace cfgpred { // `requires = ["mcpp:compiler=llvm"]` permanently unsatisfiable. struct Ctx { std::string os, arch, family, env, triple; + // The selected platform (SPEC-013 §3): the platform's name (`openkal`) + // and its package's (`openkal-platform`); `cfg(platform = ...)` matches + // either. Empty when the target uses none. + std::string platform, platformPackage; bool layersKnown = false; std::string compiler, compilerRuntime, kernelAbi, cAbi, cxxAbi; // The first MULTI-VALUED layer. One build can enable several accelerator @@ -132,9 +136,14 @@ struct Ctx { // triple parser — so the cfg vocabulary IS the canonical triple vocabulary // (os: linux|macos|windows, arch: GNU spellings, env: gnu|musl|msvc), and // alias spellings ("x86_64-w64-mingw32") evaluate identically to canonical. -inline Ctx context_for(std::string_view targetTriple) { +inline Ctx context_for(std::string_view targetTriple, + std::string_view platformPackage = {}) { namespace triple = mcpp::toolchain::triple; Ctx c; + if (!platformPackage.empty()) { + c.platformPackage = std::string(platformPackage); + c.platform = mcpp::manifest::platform::platform_name(platformPackage); + } auto t = targetTriple.empty() ? std::optional(triple::host_triple()) : triple::parse(targetTriple); @@ -271,6 +280,8 @@ struct Parser { if (k == "arch") return c.arch == v; if (k == "family") return c.family == v; if (k == "env") return c.env == v; + if (k == "platform") + return !c.platform.empty() && (c.platform == v || c.platformPackage == v); // `accelerator` is answerable whenever the context carries the build's // accel, which is from the first pass onward -- see kCfgEarlyLayerKeys. if (is_cfg_early_layer_key(k)) return c.layer_matches(k, v); diff --git a/src/build/refusal.cppm b/src/build/refusal.cppm index d384f06ae..6c9bcb07d 100644 --- a/src/build/refusal.cppm +++ b/src/build/refusal.cppm @@ -185,13 +185,24 @@ enum class Code { // discover them from (2026.10.5.2). Distinct from every other code: both // statements are the author's, and one of them has to change. LtoExportDiscovery, - // A build of `-openkal` that links an image finds no `okld` -- the - // format's tool, which writes the image's requirements note and checks the - // convention before the image is published (design 2026-10-26 part 2, - // §8.5). Distinct from LldRequiredAbsent, which is about the toolchain's - // payload: `okld` is a program of the dependency graph, and declaring - // openkal-tools under `[build-dependencies]` supplies it. - ImageToolAbsent, + // PLATFORMS (SPEC-013). A program a platform's link or seal names -- + // `okld` for an ok image -- is neither built by the graph nor found on + // this machine. Distinct from LldRequiredAbsent, which is about the + // toolchain's payload: the tool is the platform's, and its `[platform.tools]` + // (or `[target..tools]`) says which package builds it. Named + // `image-tool-absent` in 2026.10.26.2, before ok images were a platform. + PlatformToolAbsent, + // The package selected as the platform has no `[platform]` section. + PlatformNotAPlatform, + // The platform does not serve the target: the triple is not among its + // `targets`, and it has no `[platform.hosted]` for a host triple. + PlatformTargetUnserved, + // A package of the platform's version group is in the graph at another + // version (§8.1). + PlatformVersionGroup, + // A platform's build program stated a directive outside its phase's + // vocabulary (build, link, runner, runners). + PlatformPhaseDirective, Other, // a refusal that has not been given a code yet }; @@ -246,7 +257,13 @@ constexpr std::string_view name(Code c) { case Code::LocalToolchain: return "local-toolchain"; case Code::TargetBase: return "target-base"; case Code::LtoExportDiscovery: return "lto-export-discovery"; - case Code::ImageToolAbsent: return "image-tool-absent"; + case Code::PlatformToolAbsent: return "platform-tool-absent"; + case Code::PlatformNotAPlatform: return "platform-not-a-platform"; + case Code::PlatformTargetUnserved: + return "platform-target-unserved"; + case Code::PlatformVersionGroup: return "platform-version-group"; + case Code::PlatformPhaseDirective: + return "platform-phase-directive"; case Code::Other: return "other"; } return "other"; diff --git a/src/cli.cppm b/src/cli.cppm index e9a7fb9fb..10974da75 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -382,6 +382,19 @@ int run(int argc, char** argv) { if (i + 1 < argc) mcpp::invocation::set("MCPP_SYSROOT", argv[++i]); } else if (a.starts_with("--sysroot=")) mcpp::invocation::set("MCPP_SYSROOT", std::string(a.substr(10))); + // --platform and --host-platform are `MCPP_PLATFORM` and + // `MCPP_HOST_PLATFORM` for this invocation (SPEC-013 §3, §6.1), for + // the reason --sysroot is: the selection is made deep in prepare, and + // the fast paths read the variables to know the configuration moved. + else if (a == "--platform") { + if (i + 1 < argc) mcpp::invocation::set("MCPP_PLATFORM", argv[++i]); + } + else if (a.starts_with("--platform=")) mcpp::invocation::set("MCPP_PLATFORM", std::string(a.substr(11))); + else if (a == "--host-platform") { + if (i + 1 < argc) mcpp::invocation::set("MCPP_HOST_PLATFORM", argv[++i]); + } + else if (a.starts_with("--host-platform=")) + mcpp::invocation::set("MCPP_HOST_PLATFORM", std::string(a.substr(16))); else if (a.starts_with("-j") && a.size() > 2) mcpp::invocation::set("MCPP_JOBS", std::string(a.substr(2))); // --play-game rides the same channel: its value is optional @@ -583,6 +596,10 @@ int run(int argc, char** argv) { .help("Build with this toolchain for one build, e.g. llvm@22.1.8")) .option(cl::Option("sysroot").takes_value().value_name("SYSROOT") .help("The target's base for this invocation (= MCPP_SYSROOT): a package, glibc@system, path:, none")) + .option(cl::Option("platform").takes_value().value_name("SPEC") + .help("The target's platform for this invocation (= MCPP_PLATFORM): @, path:, none")) + .option(cl::Option("host-platform").takes_value().value_name("SPEC") + .help("The host graph's platform for this invocation (= MCPP_HOST_PLATFORM)")) .option(cl::Option("no-cache") .help("Deprecated alias for --cache=off (also clears the build dir)")) .option(cl::Option("target").takes_value().help( @@ -655,6 +672,10 @@ int run(int argc, char** argv) { .help("Build with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("sysroot").takes_value().value_name("SYSROOT") .help("The target's base for this invocation (= MCPP_SYSROOT): a package, glibc@system, path:, none")) + .option(cl::Option("platform").takes_value().value_name("SPEC") + .help("The target's platform for this invocation (= MCPP_PLATFORM): @, path:, none")) + .option(cl::Option("host-platform").takes_value().value_name("SPEC") + .help("The host graph's platform for this invocation (= MCPP_HOST_PLATFORM)")) .option(cl::Option("cache").takes_value().value_name("MODE") .help("Global dependency cache: global (default) | local | off")) .option(cl::Option("no-cache") @@ -755,6 +776,10 @@ int run(int argc, char** argv) { .help("Build the tests with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("sysroot").takes_value().value_name("SYSROOT") .help("The target's base for this invocation (= MCPP_SYSROOT): a package, glibc@system, path:, none")) + .option(cl::Option("platform").takes_value().value_name("SPEC") + .help("The target's platform for this invocation (= MCPP_PLATFORM): @, path:, none")) + .option(cl::Option("host-platform").takes_value().value_name("SPEC") + .help("The host graph's platform for this invocation (= MCPP_HOST_PLATFORM)")) .option(cl::Option("cache").takes_value().value_name("MODE") .help("Global dependency cache: global (default) | local | off")) .option(cl::Option("no-cache") @@ -790,6 +815,10 @@ int run(int argc, char** argv) { .help("Ask about this toolchain, e.g. llvm@22.1.8")) .option(cl::Option("sysroot").takes_value().value_name("SYSROOT") .help("The target's base for this invocation (= MCPP_SYSROOT): a package, glibc@system, path:, none")) + .option(cl::Option("platform").takes_value().value_name("SPEC") + .help("The target's platform for this invocation (= MCPP_PLATFORM): @, path:, none")) + .option(cl::Option("host-platform").takes_value().value_name("SPEC") + .help("The host graph's platform for this invocation (= MCPP_HOST_PLATFORM)")) .option(cl::Option("features").takes_value().value_name("LIST") .help("Resolve with these features, as `build --features` does (root features and /)")) .option(cl::Option("format").takes_value().value_name("json") @@ -874,6 +903,10 @@ int run(int argc, char** argv) { .help("Build with this toolchain for one invocation, e.g. llvm@22.1.8")) .option(cl::Option("sysroot").takes_value().value_name("SYSROOT") .help("The target's base for this invocation (= MCPP_SYSROOT): a package, glibc@system, path:, none")) + .option(cl::Option("platform").takes_value().value_name("SPEC") + .help("The target's platform for this invocation (= MCPP_PLATFORM): @, path:, none")) + .option(cl::Option("host-platform").takes_value().value_name("SPEC") + .help("The host graph's platform for this invocation (= MCPP_HOST_PLATFORM)")) .option(cl::Option("features").takes_value().value_name("LIST") .help("Activate root-package features, and a dependency's as /, for every build pass of the pack (comma-separated)")) // `--format` names the PACKAGE format here, so machine output is @@ -932,6 +965,10 @@ int run(int argc, char** argv) { .help("Describe the build with this toolchain, e.g. llvm@22.1.8")) .option(cl::Option("sysroot").takes_value().value_name("SYSROOT") .help("The target's base for this invocation (= MCPP_SYSROOT): a package, glibc@system, path:, none")) + .option(cl::Option("platform").takes_value().value_name("SPEC") + .help("The target's platform for this invocation (= MCPP_PLATFORM): @, path:, none")) + .option(cl::Option("host-platform").takes_value().value_name("SPEC") + .help("The host graph's platform for this invocation (= MCPP_HOST_PLATFORM)")) .option(cl::Option("accel").takes_value().value_name("SPEC") .help("Device backends and architectures (same axis as `mcpp build --accel`)")) .option(cl::Option("no-accel") @@ -1212,11 +1249,14 @@ int run(int argc, char** argv) { .action(wrap_rc([&passthrough](const cl::ParsedArgs& p) { return cmd_link_output(p, std::span(passthrough)); }))) - .subcommand(cl::App("okimage-finish") - .description("(internal: invoked by ninja for -openkal) Write an ok image's requirements note and check it with okld, then publish it") - .option(cl::Option("okld").takes_value().value_name("PATH").help("the okld program")) + .subcommand(cl::App("platform-step") + .description("(internal: invoked by ninja for a platform's link) `capture --out F -- argv…` saves a program's output; `seal --steps F -- [needed…]` runs the platform's seal steps, then publishes") + .arg(cl::Arg("mode").help("capture | seal")) + .option(cl::Option("out").takes_value().value_name("PATH").help("capture: the file the output is saved as")) + .option(cl::Option("steps").takes_value().value_name("PATH").help("seal: the steps, one per line, words separated by a tab")) + .option(cl::Option("digest").takes_value().value_name("HEX").help("seal: the steps' digest (part of the command, so a changed step reseals)")) .action(wrap_rc([&passthrough](const cl::ParsedArgs& p) { - return cmd_okimage_finish(p, std::span(passthrough)); + return cmd_platform_step(p, std::span(passthrough)); }))) .subcommand(cl::App("place-dlls") .description("(internal: invoked by ninja) Place beside a Windows program the DLLs it imports from its runtime search directories") @@ -1687,7 +1727,7 @@ int run(int argc, char** argv) { "update", "search", "publish", "pack", "emit", "xpkg", "toolchain", "cache", "index", "self", "explain", "version", "dyndep", "why", "resolve", "stage", "bmi-equal", "coff-def", - "place-dlls", "link-output", "okimage-finish", + "place-dlls", "link-output", "platform-step", "bmi-compile", "bmi-supervise", "bmi-await", }); bool ok = false; diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index d0d729de5..25b5b46b0 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -9,6 +9,7 @@ module; export module mcpp.cli.cmd_build; import std; +import mcpp.platform; import mcpplibs.cmdline; import mcpp.build.prepare; import mcpp.build.execute; @@ -1476,59 +1477,113 @@ export int cmd_link_output(const mcpplibs::cmdline::ParsedArgs& parsed, return mcpp::run::run_exec(std::vector(command.begin(), command.end())); } -// `mcpp okimage-finish --okld -- […]` -// (internal: invoked by build.ninja for the target `-openkal`; design -// 2026-10-26 part 2, §8.5). +// `mcpp platform-step capture --out -- ` +// `mcpp platform-step seal --steps --digest -- […]` +// (internal: invoked by build.ninja for a platform's link; SPEC-013 §4, §4.6). // -// AN IMAGE IS PUBLISHED ONLY ONCE IT MEETS THE CONVENTION. The linked file is -// copied beside the image's name, `okld note` writes its requirements note from -// its strong imports, `okld check` checks every clause of SPEC 12.2 against it -// and the images it needs, and only then does the copy take the name. A check -// that fails leaves no image behind, and the edge runs again next time. -export int cmd_okimage_finish(const mcpplibs::cmdline::ParsedArgs& parsed, - std::span rest) { +// CAPTURE saves a program's standard output as a link input (an ok image's +// notes, from `okld note-source`), written beside its name and renamed, so an +// interrupted run leaves no half file. +// +// SEAL PUBLISHES AN ARTIFACT ONLY ONCE EVERY STEP ACCEPTED IT. The linked file +// is copied beside the artifact's name; each step runs on the copy, `{}` being +// the copy and `{needed}` the shared artifacts it needs (an option before it +// repeated for each); only then does the copy take the name. A step that fails +// leaves the previous artifact in place and no copy behind, names the step +// and prints what it said; the edge runs again next time. +export int cmd_platform_step(const mcpplibs::cmdline::ParsedArgs& parsed, + std::span rest) { namespace fs = std::filesystem; - const std::string okld = parsed.option_or_empty("okld").value(); - if (okld.empty() || rest.size() < 2) { - std::println(stderr, "error: okimage-finish needs --okld -- [needed…]"); - return 2; - } - const fs::path linked = rest[0], image = rest[1]; - auto tmp = image; tmp += ".tmp"; - std::error_code ec; - fs::copy_file(linked, tmp, fs::copy_options::overwrite_existing, ec); - if (ec) { - std::println(stderr, "error: cannot copy {} to {}: {}", linked.string(), tmp.string(), ec.message()); - return 1; - } - auto run = [&](std::vector argv) { + const std::string mode = parsed.positional_count() > 0 ? parsed.positional(0) : std::string(); + auto run = [&](const std::vector& argv, std::string* out) { int spawnError = 0; auto r = mcpp::run::capture_exec(argv, {}, {}, &spawnError); if (spawnError != 0) { - std::println(stderr, "error: cannot start {}: {}", okld, + std::println(stderr, "error: cannot start {}: {}", argv.front(), std::generic_category().message(spawnError)); return false; } if (r.exit_code != 0) { + std::string shown; + for (auto const& a : argv) shown += (shown.empty() ? "" : " ") + a; + std::println(stderr, "error: `{}` exited {}", shown, r.exit_code); std::print(stderr, "{}", r.output); return false; } + if (out) *out = std::move(r.output); return true; }; - std::vector check{ okld, "check", tmp.string() }; - for (std::size_t i = 2; i < rest.size(); ++i) { check.push_back("--with"); check.push_back(rest[i]); } - if (!run({ okld, "note", tmp.string() }) || !run(check)) { - fs::remove(tmp, ec); - std::println(stderr, "error: {} does not meet the ok-image convention (openkal SPEC 12)", - image.filename().string()); - return 1; + std::error_code ec; + if (mode == "capture") { + const std::string out = parsed.option_or_empty("out").value(); + if (out.empty() || rest.empty()) { + std::println(stderr, "error: platform-step capture needs --out -- "); + return 2; + } + std::string text; + if (!run(std::vector(rest.begin(), rest.end()), &text)) return 1; + fs::create_directories(fs::path(out).parent_path(), ec); + auto tmp = fs::path(out); + tmp += ".tmp"; + { + std::ofstream f(tmp, std::ios::binary | std::ios::trunc); + f << text; + if (!f) { std::println(stderr, "error: cannot write {}", tmp.string()); return 1; } + } + fs::rename(tmp, out, ec); + if (ec) { std::println(stderr, "error: cannot publish {}: {}", out, ec.message()); return 1; } + return 0; } - fs::rename(tmp, image, ec); - if (ec) { - std::println(stderr, "error: cannot publish {}: {}", image.string(), ec.message()); - return 1; + if (mode == "seal") { + const std::string stepsFile = parsed.option_or_empty("steps").value(); + if (stepsFile.empty() || rest.size() < 2) { + std::println(stderr, "error: platform-step seal needs --steps -- [needed…]"); + return 2; + } + std::vector> steps; + { + std::ifstream in(stepsFile, std::ios::binary); + std::string line; + while (std::getline(in, line)) { + if (!line.empty() && line.back() == '\r') line.pop_back(); + if (line.empty()) continue; + std::vector step; + std::size_t from = 0; + for (;;) { + auto tab = line.find('\t', from); + step.push_back(line.substr(from, tab == std::string::npos ? std::string::npos : tab - from)); + if (tab == std::string::npos) break; + from = tab + 1; + } + steps.push_back(std::move(step)); + } + } + const fs::path linked = rest[0], artifact = rest[1]; + const std::vector needed(rest.begin() + 2, rest.end()); + auto tmp = artifact; + tmp += ".tmp"; + fs::copy_file(linked, tmp, fs::copy_options::overwrite_existing, ec); + if (ec) { + std::println(stderr, "error: cannot copy {} to {}: {}", linked.string(), tmp.string(), ec.message()); + return 1; + } + for (auto const& step : steps) { + if (!run(mcpp::platform::expand_step(step, tmp.string(), needed), nullptr)) { + fs::remove(tmp, ec); + std::println(stderr, "error: {} was not sealed: a step of its platform's " + "[platform.link] seal refused it", artifact.filename().string()); + return 1; + } + } + fs::rename(tmp, artifact, ec); + if (ec) { + std::println(stderr, "error: cannot publish {}: {}", artifact.string(), ec.message()); + return 1; + } + return 0; } - return 0; + std::println(stderr, "error: platform-step takes `capture` or `seal`"); + return 2; } export int cmd_stage(const mcpplibs::cmdline::ParsedArgs& parsed) { diff --git a/src/config.cppm b/src/config.cppm index f834615b3..3bbfcd6c2 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -24,6 +24,8 @@ import std; import mcpp.home; import mcpp.modgraph.glob; // try_narrow: the one UTF-8 spelling of a path import mcpp.libs.toml; +import mcpp.manifest.platform; +import mcpp.manifest.platform_toml; import mcpp.libs.json; import mcpp.pm.index_spec; import mcpp.provider; @@ -177,6 +179,12 @@ struct GlobalConfig { std::filesystem::path runUnderFile; }; std::map targetDefaults; + // This machine's platform choices and overrides (SPEC-013 §3, §5): the + // platform part of each `[target.]` table, the system file's + // before the user's (a later one applies later), and `[host]`, the host + // graph's platform. A project's mcpp.toml outranks them. + std::vector platformOverrides; + mcpp::manifest::platform::Override hostPlatform; // Resolved xlings home (registryDir unless overridden) std::filesystem::path xlingsHome() const { @@ -371,7 +379,9 @@ parse_payload_overrides(const mcpp::libs::toml::Document& doc); std::expected parse_target_defaults(const mcpp::libs::toml::Document& doc, const std::filesystem::path& file, - std::map& into); + std::map& into, + std::vector* platforms = nullptr, + mcpp::manifest::platform::Override* host = nullptr); // Normalize legacy index naming in a loaded config (exported for tests): // org migration mcpp-community/mcpp-index -> mcpplibs/mcpp-index, index name @@ -718,16 +728,40 @@ void canonicalize_legacy_index_names(GlobalConfig& cfg) { std::expected parse_target_defaults(const mcpp::libs::toml::Document& doc, const std::filesystem::path& file, - std::map& into) { + std::map& into, + std::vector* platforms, + mcpp::manifest::platform::Override* host) { + namespace mp = mcpp::manifest::platform; + const mp::Reader reader{file, mp::Tier::Config, nullptr}; + // `[host]`: the platform of this machine's host graph (SPEC-013 §6.1). A + // nearer file's choice stands. + if (auto* ht = doc.get_table("host"); ht && host && !host->platformSet) { + auto ov = reader.override_(*ht, ""); + if (!ov) return std::unexpected(std::format("{} {}", file.string(), ov.error())); + *host = std::move(*ov); + } auto* tt = doc.get_table("target"); if (!tt) return {}; + std::vector ours; for (auto const& [triple, body] : *tt) { if (!body.is_table()) return std::unexpected(std::format( "{} [target.{}] must be a table with `sysroot` and/or `run_under`", file.string(), triple)); auto& d = into[triple]; + // The platform keys (SPEC-013 §5.2), read by the reader a manifest uses. + bool platformKeys = false; + for (auto k : mp::kOverrideKeys) platformKeys |= body.as_table().contains(k); + platformKeys |= body.as_table().contains("runner") || body.as_table().contains("runners"); + if (platformKeys && platforms) { + auto ov = reader.override_(body.as_table(), triple); + if (!ov) return std::unexpected(std::format("{} {}", file.string(), ov.error())); + ours.push_back(std::move(*ov)); + } for (auto const& [k, v] : body.as_table()) { + if (std::ranges::find(mp::kOverrideKeys, k) != std::end(mp::kOverrideKeys) + || k == "runner" || k == "runners") + continue; if (k == "sysroot") { if (d.sysrootDeclared) continue; // a nearer file said it std::string text; @@ -781,11 +815,15 @@ parse_target_defaults(const mcpp::libs::toml::Document& doc, } else { return std::unexpected(std::format( "{} [target.{}]: '{}' is not a key of a machine's target defaults; " - "the keys are `sysroot` and `run_under` (everything else about a " - "target belongs in the project's mcpp.toml)", file.string(), triple, k)); + "the keys are `sysroot`, `run_under` and the platform's: platform, layers, " + "tools, carrier, link, runner, runners (everything else about a target " + "belongs in the project's mcpp.toml)", file.string(), triple, k)); } } } + // The file read first is the nearer one; a later override applies later, + // so the nearer file's tables go after those already held. + if (platforms) platforms->insert(platforms->begin(), ours.begin(), ours.end()); return {}; } @@ -984,11 +1022,13 @@ std::expected load_or_init( else return std::unexpected(ConfigError{overrides.error()}); // [target.]: this machine's defaults, the user's first. - if (auto r = parse_target_defaults(*doc, cfg.configFile, cfg.targetDefaults); !r) + if (auto r = parse_target_defaults(*doc, cfg.configFile, cfg.targetDefaults, + &cfg.platformOverrides, &cfg.hostPlatform); !r) return std::unexpected(ConfigError{r.error()}); if (systemDoc) if (auto r = parse_target_defaults(*systemDoc, mcpp::home::system_config_file(), - cfg.targetDefaults); !r) + cfg.targetDefaults, &cfg.platformOverrides, + &cfg.hostPlatform); !r) return std::unexpected(ConfigError{r.error()}); // A read-only report must not create its log directory either. diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 873cc30c3..2cbde9a9d 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -61,6 +61,7 @@ import mcpp.toolchain.triple; import mcpp.toolchain.base; import mcpp.ui; import mcpp.runtime.elf; +import mcpp.platform; export namespace mcpp::pack { @@ -251,10 +252,13 @@ struct Plan { // the file `.js` in the first place (see `artifact_naming`), so it is the // one fact this format IS recorded under. bool targetIsWasm = false; - // Is the artifact an ok image (`-openkal`, openkal SPEC 12)? From the - // triple, as for wasm: the file is an ELF shared object that no loader of - // this machine runs, and its closure is the images it names. - bool targetIsOkImage = false; + // Is the artifact one a loader runs (an ok image, `-openkal`)? From + // the triple, as for wasm: the file is an ELF shared object that no loader + // of this machine runs, and its closure is the images it names. + bool targetIsLoaded = false; + // The target's platform (SPEC-013): the carrier a self-contained pack of + // a loaded artifact is built over (§4.7). + mcpp::platform::Resolved platform; // #622 A3/A10: is `builtBinary` an `Application` target whose link form // on THIS row is a shared object (`*-linux-android`, // `toolchain::triple::application_form`)? Read from the CALLER, which @@ -661,7 +665,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, if (auto t = mcpp::toolchain::triple::parse(p.triple)) { p.targetIsWasm = t->object_format() == mcpp::toolchain::triple::ObjectFormat::Wasm; - p.targetIsOkImage = t->is_openkal(); + p.targetIsLoaded = t->loader_supplies_os(); } // THE CONTRACT REACHES PACKAGING. Until now it stopped at the compile and @@ -1973,77 +1977,71 @@ run_wasm(const Plan& plan) return make_tarball(plan.stagingRoot, plan.archivePath); } -// AN OK IMAGE (`-openkal`; design 2026-10-26 part 2, §8.2 and §8.5). +// AN ARTIFACT A LOADER RUNS (an ok image, `-openkal`; design +// 2026-10-26 part 2, §8.2 and §8.5; SPEC-013 §4.7). // // The modes keep their meaning -- what the machine that runs the package must -// provide -- and on this row they are: +// provide -- and on such a row they are: // -// system the image alone; the machine provides `okrun` and every -// image it names (M2); -// vendored the image and the images it names (`DT_NEEDED`), beside it; -// the machine provides `okrun` (M2, the default); -// self-contained a program of THIS machine that carries the image as a -// section and runs it through libokrun (M3), with the images -// it names beside it; the machine provides nothing. The image -// is not recompiled: the program is a generated entry, the -// image's bytes and `openkal-run`, built by mcpp for this -// machine. Another system's M3 is packed on that system. -// static refused: an image is loaded, never linked into one file -// with its loader by `ld`. +// system the artifact alone; the machine provides the loader and +// every artifact it names (M2); +// vendored the artifact and the artifacts it names (`DT_NEEDED`), +// beside it; the machine provides the loader (M2, the default); +// self-contained a program of THIS machine that carries the artifact as a +// section and runs it through the platform's CARRIER (M3), +// with the artifacts it names beside it; the machine provides +// nothing. The artifact is not recompiled: the program is the +// carrier package, the carrier's entry source and the bytes, +// built by mcpp for this machine with this machine's own +// conventions -- the bootstrap boundary (§6.2), so no platform +// is selected for it. Another system's M3 is packed on that +// system. +// static refused: the artifact is loaded, never linked into one +// file with its loader by `ld`. +namespace { + +// A reference as a dependency entry of the generated manifest. +std::string dependency_entry(const mcpp::manifest::platform::PackageRef& r) { + auto const& sp = r.spec; + const std::string ns = sp.namespace_.empty() ? std::string("mcpplibs") : sp.namespace_; + std::string value; + if (!sp.path.empty()) + value = std::format("{{ path = \"{}\" }}", sp.path); + else if (!sp.git.empty()) + value = std::format("{{ git = \"{}\", {} = \"{}\" }}", sp.git, + sp.gitRefKind.empty() ? "rev" : sp.gitRefKind, sp.gitRev); + else + value = std::format("\"{}\"", sp.version); + return std::format("[dependencies.{}]\n{} = {}\n", ns, sp.shortName, value); +} + +} // namespace + std::expected -okimage_build_selfcontained(const Plan& plan, const std::filesystem::path& image) +carrier_build_selfcontained(const Plan& plan, const std::filesystem::path& image) { namespace fs = std::filesystem; - // `openkal-run` from the source the project's `openkal-tools` came from: - // the same repository at the same reference selects its member by key - // (05 -- Dependencies), and a published version has a sibling of the same - // version. - // - // THE DECLARATION THE BUILD RESOLVED: after workspace inheritance, and - // with the conditional tables whose predicate the image's target meets - // (`[target.'cfg(os = "openkal")'.build-dependencies]`, which is where a - // program built for systems as well as images declares it). The raw file's - // `[build-dependencies]` alone missed both. - auto eff = mcpp::project::load_effective_manifest(plan.projectRoot); - if (!eff) return std::unexpected(Error{std::format( - "cannot read {}: {}", (plan.projectRoot / "mcpp.toml").string(), eff.error())}); - const auto& man = eff->manifest; - std::optional tools; - auto find_in = [&](const std::map& deps) { - for (auto const& [key, spec] : deps) - if (key == "openkal-tools" || spec.shortName == "openkal-tools") tools = spec; - }; - find_in(man.buildDependencies); - const auto ctx = mcpp::build::cfgpred::context_for(plan.triple); - for (auto const& cc : man.conditionalConfigs) - if (!mcpp::build::cfgpred::uses_layer(cc.predicate) - && mcpp::build::cfgpred::matches(cc.predicate, ctx)) - find_in(cc.buildDependencies); - if (!tools) return std::unexpected(Error{ - "--mode self-contained of an ok image builds a program over `openkal-run` from the " - "source of the project's `openkal-tools`, and its [build-dependencies] declare none"}); - std::string runSpec; - if (!tools->git.empty()) { - runSpec = std::format("{{ git = \"{}\", {} = \"{}\" }}", tools->git, - tools->gitRefKind.empty() ? "rev" : tools->gitRefKind, tools->gitRev); - } else if (!tools->path.empty()) { - auto root = fs::path(tools->path); - if (root.is_relative()) root = plan.projectRoot / root; - runSpec = std::format("{{ path = \"{}\" }}", (root.parent_path() / "run").generic_string()); - } else { - runSpec = std::format("\"{}\"", tools->version); - } + auto const& c = plan.platform.fields.carrier; + if (!plan.platform.active || !c.set) + return std::unexpected(Error{std::format( + "--mode self-contained of {} builds a program of this machine over its platform's " + "carrier, and {} states none ([platform] carrier = {{ package, section, main }}).", + plan.binaryName, plan.platform.display())}); + std::error_code ec; + if (c.main.empty() || !fs::is_regular_file(c.main, ec)) + return std::unexpected(Error{std::format( + "the carrier of {} names its entry source {}, which is not a file ({})", + plan.platform.display(), c.main.empty() ? std::string("(none)") : c.main, + mcpp::platform::origin_text(c.origin))}); const auto stem = fs::path(plan.binaryName).stem().string(); - const auto dir = plan.projectRoot / "target" / "okimage-pack" / stem; - std::error_code ec; + const auto dir = plan.projectRoot / "target" / "carrier-pack" / stem; fs::create_directories(dir / "src", ec); if (ec) return std::unexpected(Error{std::format("cannot create {}: {}", dir.string(), ec.message())}); - // RESOLVED AS THE BUILD THAT MADE THE IMAGE RESOLVED ITS TOOLS. The + // RESOLVED AS THE BUILD THAT MADE THE ARTIFACT RESOLVED ITS PLATFORM. The // generated project is this pack's, not a project of its own: a lock it - // kept from an earlier pack held `openkal-run` at that pack's commit while - // the image's tools had moved on, and the program then carried a runner - // older than the one the image was checked by. + // kept from an earlier pack held the carrier at that pack's commit while + // the platform had moved on. fs::remove(dir / "mcpp.lock", ec); auto write = [&](const fs::path& p, std::string_view text) -> bool { @@ -2051,58 +2049,69 @@ okimage_build_selfcontained(const Plan& plan, const std::filesystem::path& image out << text; return bool(out); }; + // ON LINUX THE PROGRAM IS STATIC (musl), the one form that runs on every + // distribution, linked by the toolchain the target table pins for this + // machine's static row -- stated, because the carrier's own graph supplies + // a kernel interface, which would otherwise cancel the row's convention and + // leave a compiler without a musl C runtime to link with (L10, measured: + // clang took the host GCC's `crtbeginT.o`). + std::string staticRow; + if constexpr (mcpplibs::os::is_linux) { + const auto row = std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch); + if (auto t = mcpp::toolchain::triple::parse(row)) + if (auto* k = mcpp::toolchain::triple::find_known_target(*t); k && !k->pin.empty()) + staticRow = std::format("\n[target.{}]\ntoolchain = \"{}\"\n", row, k->pin); + } const std::string manifest = std::format( "# Generated by `mcpp pack --mode self-contained` for {}: a program of this\n" - "# machine that carries the ok image as a section and runs it (M3).\n" + "# machine that carries it as a section and runs it through {}'s carrier (M3).\n" "[package]\nname = \"{}\"\nversion = \"{}\"\n\n" "[targets.{}]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n" - "[build]\nsources = [\"src/main.cpp\", \"src/image.S\"]\n\n" - "[dependencies]\nopenkal-run = {}\n", - plan.binaryName, stem, plan.packageVersion.empty() ? "0.0.0" : plan.packageVersion, - stem, runSpec); + "[build]\nsources = [\"src/main.cpp\", \"src/carried.S\"]\n\n" + "{}{}", + plan.binaryName, plan.platform.display(), stem, + plan.packageVersion.empty() ? "0.0.0" : plan.packageVersion, stem, + dependency_entry(c.package), staticRow); // The section, in the syntax of this machine's object format; the symbol - // spelling carries Mach-O's leading underscore. + // spelling carries Mach-O's leading underscore. The carrier's entry reads + // the bytes through `mcpp_carried_begin` / `mcpp_carried_end` and the + // artifact's name through `mcpp_carried_name`. const bool macho = mcpplibs::os::is_macos; const bool coff = mcpplibs::os::is_windows; const std::string sym = macho ? "_" : ""; - const std::string section = macho ? "\t.section __OKE,__image\n" - : coff ? "\t.section .oke,\"dr\"\n" - : "\t.section .oke,\"a\",@progbits\n"; + std::string bare = c.section; + while (!bare.empty() && bare.front() == '.') bare.erase(bare.begin()); + std::string upper = bare; + for (auto& ch : upper) ch = static_cast(std::toupper(static_cast(ch))); + const std::string section = macho ? std::format("\t.section __{},__image\n", upper.substr(0, 14)) + : coff ? std::format("\t.section {},\"dr\"\n", c.section) + : std::format("\t.section {},\"a\",@progbits\n", c.section); const std::string assembly = std::format( - "# The image's bytes, unchanged (generated by mcpp pack).\n{}" - "\t.p2align 4\n\t.globl {}ok_image_begin\n{}ok_image_begin:\n" - "\t.incbin \"{}\"\n\t.globl {}ok_image_end\n{}ok_image_end:\n", - section, sym, sym, fs::absolute(image).generic_string(), sym, sym); - const std::string main = std::format( - "// Generated by mcpp pack: runs the image this program carries (M3).\n" - "#include \n" - "#include \n" - "extern \"C\" const unsigned char ok_image_begin[], ok_image_end[];\n" - "int main(int argc, char** argv) {{\n" - " char why[512];\n" - " okrun_options options{{}};\n" - " options.carried = 1; // the program is the image's name, and what starts it again\n" - " okrun_run_bytes(ok_image_begin, static_cast(ok_image_end - ok_image_begin),\n" - " \"{}\", argc, const_cast(argv), &options, why, sizeof why);\n" - " std::fprintf(stderr, \"{}: %s\\n\", why);\n" - " return 127;\n" - "}}\n", - plan.binaryName, stem); - if (!write(dir / "mcpp.toml", manifest) || !write(dir / "src" / "image.S", assembly) - || !write(dir / "src" / "main.cpp", main)) + "# The artifact's bytes, unchanged, and its name (generated by mcpp pack).\n{}" + "\t.p2align 4\n\t.globl {}mcpp_carried_begin\n{}mcpp_carried_begin:\n" + "\t.incbin \"{}\"\n\t.globl {}mcpp_carried_end\n{}mcpp_carried_end:\n" + "\t.globl {}mcpp_carried_name\n{}mcpp_carried_name:\n\t.asciz \"{}\"\n", + section, sym, sym, fs::absolute(image).generic_string(), sym, sym, sym, sym, + plan.binaryName); + std::string mainText; + { + std::ifstream in(c.main, std::ios::binary); + mainText.assign(std::istreambuf_iterator(in), {}); + } + if (!write(dir / "mcpp.toml", manifest) || !write(dir / "src" / "carried.S", assembly) + || !write(dir / "src" / "main.cpp", mainText)) return std::unexpected(Error{std::format("cannot write the program's sources under {}", dir.string())}); // PACKED, NOT MERELY BUILT: the program obeys every rule this command // keeps for a program -- no build-machine path, its closure beside it. On - // Linux it is static (musl), the one form that runs on every distribution; - // on macOS and Windows the system's own libraries are the only ones it - // names. - mcpp::ui::info("Packing", std::format("{} into a program of this machine (okrun, M3)", plan.binaryName)); + // macOS and Windows the system's own libraries are the only ones it names. + mcpp::ui::info("Packing", std::format("{} into a program of this machine ({}'s carrier, M3)", + plan.binaryName, plan.platform.display())); // `-o` with a bare name: the package directory is `target/dist/`. const auto out = dir / "target" / "dist" / "packed"; fs::remove_all(out, ec); std::vector argv{ mcpp::invocation::self_exe_path().string(), "pack", - "--format", "dir", "-o", "packed" }; + "--format", "dir", "-o", "packed", "--platform", "none" }; if constexpr (mcpplibs::os::is_linux) { argv.insert(argv.end(), { "--target", std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch), "--mode", "static" }); @@ -2121,15 +2130,15 @@ okimage_build_selfcontained(const Plan& plan, const std::filesystem::path& image } std::expected -run_okimage(const Plan& plan) +run_loaded(const Plan& plan) { namespace fs = std::filesystem; if (plan.opts.mode == Mode::Static) return std::unexpected(Error{std::format( - "--mode static does not apply to an ok image: an image is loaded, not linked " - "with its loader into one file.\n" + "--mode static does not apply to {}: it is loaded, not linked with its loader " + "into one file.\n" " `--mode self-contained` packs a program of this machine that carries {} " - "and needs nothing installed.", plan.binaryName)}); + "and needs nothing installed.", plan.binaryName, plan.binaryName)}); std::error_code ec; fs::remove_all(plan.stagingRoot, ec); fs::create_directories(plan.stagingRoot / "bin", ec); @@ -2155,7 +2164,7 @@ run_okimage(const Plan& plan) fs::path staged; if (plan.opts.mode == Mode::BundleAll) { - auto packed = okimage_build_selfcontained(plan, plan.builtBinary); + auto packed = carrier_build_selfcontained(plan, plan.builtBinary); if (!packed) return std::unexpected(packed.error()); fs::copy(*packed, plan.stagingRoot, fs::copy_options::recursive | fs::copy_options::overwrite_existing, ec); @@ -2369,8 +2378,8 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // An ok image: the images it names travel beside it, and under // `self-contained` the image is carried inside a program of this machine. - if (plan.targetIsOkImage) { - if (auto r = detail::run_okimage(plan); !r) return std::unexpected(r.error()); + if (plan.targetIsLoaded) { + if (auto r = detail::run_loaded(plan); !r) return std::unexpected(r.error()); return ClosureResult{}; } diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 88bb98060..1e5136dfd 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -568,6 +568,7 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { // which appears in the project's own manifest. ctx.plan.runtimeRequirements, programIsSharedObject); if (!plan) return Refusal{1, about(run, m, plan.error().message)}; + plan->platform = ctx.plan.platform; // `--output ` with several members: each member's archive, or its tree // for `--format dir`, is written below the directory under the name it would // have by default. The staging tree of an archive, and of a dispatched diff --git a/src/platform/platform.cppm b/src/platform/platform.cppm new file mode 100644 index 000000000..cc6e2e868 --- /dev/null +++ b/src/platform/platform.cppm @@ -0,0 +1,544 @@ +// mcpp.platform -- the engine's half of platforms (SPEC-013; design +// 2026-10-28-platforms-design.md §3-§9). +// +// The engine holds no knowledge of any one platform. It provides the +// mechanism: which platform a graph uses (selection), what the platform's +// statement is once every tier has spoken (merge), the flags it contributes to +// a target (evaluation of its cfg tables), how an artifact is named and sealed +// (patterns and argv templates), and how all of it is reported. The data comes +// from a platform package's `[platform]` section (mcpp.manifest.platform) and +// from the tables that override it; the prepare phases apply the result to the +// graph, the toolchain, the flags, the link, the runner and the pack. +// +// Everything here is a pure function of its arguments, and the unit tests +// state the rules as tables (tests/unit/test_platform_*.cpp). Whether a +// `[target.]` applies is asked of the caller (`Matches`), which +// holds the one evaluator of selectors (mcpp.build.prepare_inputs), so this +// module depends on the data model alone. (Importing prepare_inputs here also +// made GCC 16.1 crash importing mcpp.cli, mcpp#848.) + +export module mcpp.platform; + +import std; +import mcpp.manifest; +import mcpp.toolchain.triple; +import mcpp.invocation; + +export namespace mcpp::platform { + +// Whether a selector (`cfg(...)`, a triple, an alias) applies to the target +// of this graph. +using Matches = std::function; + +// ── Selection (§5.1, §6.1) ────────────────────────────────────────────────── + +// Which platform one graph uses. `none` is the engine's own conventions for +// the target (a host target's, or nothing at all). +struct Selection { + mcpp::manifest::platform::PackageRef ref; + mcpp::manifest::platform::Origin origin; + bool none = true; + // A host triple selected the platform: only its `[platform.hosted]` part + // applies (§5.4). + bool hosted = false; +}; + +// The value a flag or an environment variable gives: `none`, a package +// reference `@`, or a directory (`path:`, or anything +// spelled as a path), relative to where mcpp was started. +std::expected parse_choice(std::string_view value, mcpp::manifest::platform::Origin origin); + +// The tables of `tables` whose selector the context meets, least specific +// first (SPEC-004 §3.1.1), so that applying them in order lets the most +// specific one decide. +std::vector matching(std::span tables, + const Matches& matches); + +// Everything a selection reads, in precedence order. +struct ChoiceInputs { + std::string_view flag; // MCPP_PLATFORM (`--platform`) + std::string_view env; // MCPP_TARGET__PLATFORM + std::string envKey; + std::span project; // the plan root's tables, the workspace layer's first + std::span config; // config.toml's + std::string_view column; // the target table's platform column +}; + +// The target graph's platform: the flag, the environment, the project, this +// machine's configuration, the target table, in that order; the first to +// answer decides. +std::expected select_target(const ChoiceInputs& in, const Matches& matches); + +// The host graph's platform: `--host-platform` / MCPP_HOST_PLATFORM, the +// project's `[host]`, config.toml's `[host]`; otherwise the host conventions. +std::expected select_host(std::string_view flagOrEnv, bool fromFlag, + const mcpp::manifest::platform::Override& project, + const mcpp::manifest::platform::Override& config); + +// ── Merge (§4.5) ──────────────────────────────────────────────────────────── + +// `over` replaces `base` field by field, with two exceptions: flag lists +// concatenate (the platform's first, so a convention cannot be removed, only +// added to), and `link.inputs` and `link.seal` are replaced whole (they are +// one sequence each). Maps (`layers`, `tools`, `runners`, `versions`) are +// merged key by key. +void merge(mcpp::manifest::platform::Statement& base, const mcpp::manifest::platform::Statement& over); + +// The part of a platform that applies when a host triple selects it: its +// `[platform.hosted]` toolchain and layers, and nothing else (§5.4). +mcpp::manifest::platform::Statement hosted_view(const mcpp::manifest::platform::Statement& platform); + +// Whether the platform serves the triple, or refuses with the reason. A host +// triple is served by a platform with a `[platform.hosted]` table. +std::expected +serves(const mcpp::manifest::platform::Statement& platform, std::string_view package, + const mcpp::toolchain::triple::Triple& t); + +// The four flag lists the platform gives every unit of a target: the +// unconditional ones, then each cfg table the context meets, in file order. +struct TargetFlags { + std::vector cflags, cxxflags, asmflags, ldflags; +}; +TargetFlags evaluate_flags(const mcpp::manifest::platform::Statement& s, const Matches& matches); + +// ── Artifacts (§4, §4.6) ──────────────────────────────────────────────────── + +// `"lib{name}.okso"` -> {"lib", ".okso"}. A pattern names `{name}` exactly once. +std::expected, std::string> +split_name_pattern(std::string_view pattern); + +// One seal step for one artifact: `{}` is the artifact; an argument `{needed}` +// stands for the shared artifacts it needs, one argument each, and when the +// argument before it is an option that takes a value (`--with`: no `=`, no +// `{}`), that option is repeated before each of them (both are left out when +// it needs none). +std::vector expand_step(const std::vector& step, + std::string_view artifact, + std::span needed); + +// ── What the build uses (§9) ──────────────────────────────────────────────── + +// The platform of one graph after every tier has spoken: what the flags, the +// link, the runner, the pack and the report read. +struct Resolved { + bool active = false; // a platform package is in use + Selection selection; + std::string package; // "mcpplibs.openkal-platform" + std::string version; // "0.17.1" + std::filesystem::path root; // its directory + mcpp::manifest::platform::Statement fields; // merged + TargetFlags flags; // evaluated for the target + // Per layer the platform names: who provides it in this build, and + // whether that is the platform's package or one the graph already had. + struct Layer { std::string provider; mcpp::manifest::platform::Tier tier = mcpp::manifest::platform::Tier::Platform; }; + std::map layers; + // Per tool: the program the build runs. + std::map tools; + + // The platform's part of a configuration's identity: name, version and + // source, so that another platform is another build directory (§8.3). + std::string identity() const; + // `openkal-platform 0.17.1` + std::string display() const; +}; + +// The `source` column of the report and of JSON: `built-in`, `platform`, +// `config.toml`, `mcpp.toml`, `env`, `flag`, `graph`, `build-program`. +std::string origin_text(const mcpp::manifest::platform::Origin& o); + +// The lines `Platform …` and its fields, as the target report prints them. +std::vector> report_rows(const Resolved& r); + +// The JSON object of the `plan` event's `platform` / `hostPlatform`. +std::string to_json(const Resolved& r); + +} // namespace mcpp::platform + +// ── implementation ────────────────────────────────────────────────────────── + +namespace mcpp::platform { + +namespace { + +namespace mp = mcpp::manifest::platform; + +bool names_a_path(std::string_view v) { + if (v.empty()) return false; + if (v.starts_with("path:")) return true; + if (v.starts_with("/") || v.starts_with("./") || v.starts_with("../") || v.starts_with("~") + || v == "." || v == "..") + return true; + if (v.size() > 2 && v[1] == ':' && (v[2] == '\\' || v[2] == '/')) return true; // C:\… + return v.find('@') == std::string_view::npos + && (v.find('/') != std::string_view::npos || v.find('\\') != std::string_view::npos); +} + +void append(std::vector& to, const std::vector& from) { + to.insert(to.end(), from.begin(), from.end()); +} + +std::string json_escape(std::string_view s) { + std::string o; + o.reserve(s.size() + 2); + for (char c : s) { + switch (c) { + case '"': o += "\\\""; break; + case '\\': o += "\\\\"; break; + case '\n': o += "\\n"; break; + case '\t': o += "\\t"; break; + case '\r': o += "\\r"; break; + default: + if (static_cast(c) < 0x20) o += std::format("\\u{:04x}", int(c)); + else o += c; + } + } + return o; +} + +std::string q(std::string_view s) { return "\"" + json_escape(s) + "\""; } + +// An option that takes the next argument as its value (`--with`, `-L`): no +// `=` and no `{}` of its own. +bool is_bare_option(std::string_view a) { + return a.starts_with("-") && a.find('=') == std::string_view::npos + && a.find("{}") == std::string_view::npos; +} + +std::string argv_text(const std::vector& v) { + std::string out; + for (auto const& a : v) { + if (!out.empty()) out += ' '; + out += a.find(' ') == std::string::npos ? a : "\"" + a + "\""; + } + return out; +} + +} // namespace + +std::expected parse_choice(std::string_view value, mp::Origin origin) { + Selection s; + s.origin = std::move(origin); + if (value == "none") return s; + if (names_a_path(value)) { + std::string dir(value.starts_with("path:") ? value.substr(5) : value); + if (dir.starts_with("~")) + if (auto h = mcpp::invocation::get("HOME"); h && !h->empty()) dir = *h + dir.substr(1); + std::error_code ec; + auto p = std::filesystem::absolute(std::filesystem::path(dir), ec).lexically_normal(); + s.ref.spec.path = p.generic_string(); + s.none = false; + return s; + } + auto r = mp::parse_ref_text(value); + if (!r) return std::unexpected(std::format("{}: {}", s.origin.key, r.error())); + s.ref = std::move(*r); + s.none = false; + return s; +} + +std::vector matching(std::span tables, + const Matches& matches) { + std::vector out; + for (auto const& t : tables) + if (matches(t.selector)) out.push_back(&t); + std::ranges::stable_sort(out, [](const mp::Override* a, const mp::Override* b) { + return mcpp::manifest::cfg::specificity(a->selector) + < mcpp::manifest::cfg::specificity(b->selector); + }); + return out; +} + +std::expected select_target(const ChoiceInputs& in, const Matches& matches) { + if (!in.flag.empty()) + return parse_choice(in.flag, mp::Origin{mp::Tier::Flag, {}, 0, "--platform"}); + if (!in.env.empty()) + return parse_choice(in.env, mp::Origin{mp::Tier::Env, {}, 0, in.envKey}); + for (auto span : {in.project, in.config}) { + const mp::Override* chosen = nullptr; + for (auto const* o : matching(span, matches)) + if (o->platformSet) chosen = o; + if (!chosen) continue; + Selection s; + s.origin = chosen->platformOrigin; + if (s.origin.key.empty()) + s.origin.key = std::format("[target.{}].platform", chosen->selector); + if (chosen->platformNone) return s; + s.ref = chosen->platform; + s.none = false; + return s; + } + if (!in.column.empty()) { + auto s = parse_choice(in.column, mp::Origin{mp::Tier::BuiltIn, {}, 0, "the target table"}); + if (!s) return s; + return s; + } + Selection none; + none.origin = mp::Origin{mp::Tier::BuiltIn, {}, 0, "the target table"}; + return none; +} + +std::expected select_host(std::string_view flagOrEnv, bool fromFlag, + const mp::Override& project, + const mp::Override& config) { + if (!flagOrEnv.empty()) + return parse_choice(flagOrEnv, mp::Origin{fromFlag ? mp::Tier::Flag : mp::Tier::Env, {}, 0, + fromFlag ? "--host-platform" : "MCPP_HOST_PLATFORM"}); + for (auto const* o : {&project, &config}) { + if (!o->platformSet) continue; + Selection s; + s.origin = o->platformOrigin; + if (s.origin.key.empty()) s.origin.key = "[host].platform"; + if (o->platformNone) return s; + s.ref = o->platform; + s.none = false; + return s; + } + Selection none; + none.origin = mp::Origin{mp::Tier::BuiltIn, {}, 0, "host conventions"}; + return none; +} + +void merge(mp::Statement& b, const mp::Statement& o) { + auto text = [](mp::Text& to, const mp::Text& from) { if (from.set) to = from; }; + auto words = [](mp::Words& to, const mp::Words& from) { if (from.set) to = from; }; + words(b.targets, o.targets); + text(b.toolchain, o.toolchain); + if (o.sysroot.set) { b.sysroot = o.sysroot; b.sysrootProvides = o.sysrootProvides; } + if (o.runner.set) { + // Field by field: an argv-only statement keeps the platform's `host` + // and `exclusive`; a table states what it states. + b.runner.cmd = o.runner.cmd; + if (!o.runner.host.empty()) b.runner.host = o.runner.host; + if (o.runner.exclusiveSet) { + b.runner.exclusive = o.runner.exclusive; + b.runner.exclusiveSet = true; + } + b.runner.origin = o.runner.origin; + b.runner.set = true; + } + for (auto const& [k, v] : o.runners) b.runners[k] = v; + if (o.carrier.set) b.carrier = o.carrier; + text(b.plugin, o.plugin); + for (auto const& [k, v] : o.layers) b.layers[k] = v; + for (auto const& [k, v] : o.tools) b.tools[k] = v; + append(b.build.cflags, o.build.cflags); + append(b.build.cxxflags, o.build.cxxflags); + append(b.build.asmflags, o.build.asmflags); + append(b.build.ldflags, o.build.ldflags); + for (auto const& c : o.buildCfg) b.buildCfg.push_back(c); + text(b.link.driver, o.link.driver); + if (o.link.flags.set) { + append(b.link.flags.value, o.link.flags.value); + if (!b.link.flags.set) b.link.flags.origin = o.link.flags.origin; + b.link.flags.set = true; + } + text(b.link.start, o.link.start); + text(b.link.script, o.link.script); + if (o.link.inputsSet) { + b.link.inputs = o.link.inputs; + b.link.inputsOrigin = o.link.inputsOrigin; + b.link.inputsSet = true; + } + text(b.link.exe, o.link.exe); + text(b.link.shared, o.link.shared); + if (o.link.sealSet) { + b.link.seal = o.link.seal; + b.link.sealOrigin = o.link.sealOrigin; + b.link.sealSet = true; + } + for (auto const& [k, v] : o.versions) b.versions[k] = v; + if (!o.versions.empty()) b.versionsOrigin = o.versionsOrigin; + if (o.hosted.set) { + text(b.hosted.toolchain, o.hosted.toolchain); + for (auto const& [k, v] : o.hosted.layers) b.hosted.layers[k] = v; + b.hosted.set = true; + } +} + +mp::Statement hosted_view(const mp::Statement& p) { + mp::Statement s; + s.targets = p.targets; + s.toolchain = p.hosted.toolchain; + s.layers = p.hosted.layers; + s.versions = p.versions; + s.versionsOrigin = p.versionsOrigin; + return s; +} + +std::expected +serves(const mp::Statement& p, std::string_view package, const mcpp::toolchain::triple::Triple& t) { + for (auto const& w : p.targets.value) { + auto pt = mcpp::toolchain::triple::parse(w); + if (pt ? pt->str() == t.str() : w == t.str()) return false; + } + std::string list; + for (auto const& w : p.targets.value) { if (!list.empty()) list += ", "; list += w; } + if (p.hosted.set) return true; + return std::unexpected(std::format( + "platform {} serves {} and not {}: it has no [platform.hosted] table, the part a host " + "triple may take (SPEC-013 §5.4).\n" + " Build for one of its targets (--target {}), or select another platform for {}", + package, list, t.str(), p.targets.value.empty() ? std::string("") : p.targets.value.front(), + t.str())); +} + +TargetFlags evaluate_flags(const mp::Statement& s, const Matches& matches) { + TargetFlags f; + append(f.cflags, s.build.cflags); + append(f.cxxflags, s.build.cxxflags); + append(f.asmflags, s.build.asmflags); + append(f.ldflags, s.build.ldflags); + for (auto const& t : s.buildCfg) { + if (!matches(t.predicate)) continue; + append(f.cflags, t.flags.cflags); + append(f.cxxflags, t.flags.cxxflags); + append(f.asmflags, t.flags.asmflags); + append(f.ldflags, t.flags.ldflags); + } + return f; +} + +std::expected, std::string> +split_name_pattern(std::string_view pattern) { + constexpr std::string_view kName = "{name}"; + const auto at = pattern.find(kName); + if (at == std::string_view::npos || pattern.find(kName, at + 1) != std::string_view::npos) + return std::unexpected(std::format( + "'{}' names the artifact as {{name}} exactly once, for example \"{{name}}.oke\"", pattern)); + auto prefix = std::string(pattern.substr(0, at)); + auto suffix = std::string(pattern.substr(at + kName.size())); + if (prefix.find_first_of("/\\") != std::string::npos || suffix.find_first_of("/\\") != std::string::npos) + return std::unexpected(std::format("'{}' is a file name pattern; it names no directory", pattern)); + return std::pair{ std::move(prefix), std::move(suffix) }; +} + +std::vector expand_step(const std::vector& step, + std::string_view artifact, + std::span needed) { + std::vector out; + for (std::size_t i = 0; i < step.size(); ++i) { + auto const& a = step[i]; + if (a == "{needed}") { + // The option in front of it, if any, was held back below. + const bool optioned = i > 0 && is_bare_option(step[i - 1]); + for (auto const& n : needed) { + if (optioned) out.push_back(step[i - 1]); + out.push_back(n); + } + continue; + } + if (i + 1 < step.size() && step[i + 1] == "{needed}" && is_bare_option(a)) continue; + if (a == "{}") { out.emplace_back(artifact); continue; } + // `{}` inside an argument (`--image={}`) stands for the artifact too. + std::string v = a; + for (std::size_t p = v.find("{}"); p != std::string::npos; p = v.find("{}", p + artifact.size())) + v.replace(p, 2, artifact); + out.push_back(std::move(v)); + } + return out; +} + +std::string Resolved::identity() const { + if (!active) return {}; + std::string id = std::format("{}@{}", package, version); + if (!selection.ref.spec.path.empty()) id += "|path:" + selection.ref.spec.path; + if (!selection.ref.spec.git.empty()) + id += std::format("|git:{}#{}", selection.ref.spec.git, selection.ref.spec.gitRev); + return id; +} + +std::string Resolved::display() const { + if (!active) return "host conventions"; + auto dot = package.find('.'); + return std::format("{} {}", dot == std::string::npos ? package : package.substr(dot + 1), version); +} + +std::string origin_text(const mp::Origin& o) { + switch (o.tier) { + case mp::Tier::Env: + case mp::Tier::Flag: return o.key; + case mp::Tier::Config: + case mp::Tier::Manifest: { + std::string file = std::filesystem::path(o.file).filename().generic_string(); + if (file.empty()) file = std::string(mp::tier_name(o.tier)); + return o.line > 0 ? std::format("{}:{}", file, o.line) : file; + } + default: return std::string(mp::tier_name(o.tier)); + } +} + +std::vector> report_rows(const Resolved& r) { + std::vector> rows; + if (!r.active) return rows; + auto const& f = r.fields; + for (auto const& [layer, l] : r.layers) + rows.emplace_back(layer, std::format("{} ({})", l.provider, mp::tier_name(l.tier))); + if (!f.tools.empty()) { + // Tools grouped by the package that builds them. + std::map, std::string>> by; + for (auto const& [name, t] : f.tools) { + auto key = mp::display(t.ref); + by[key].first.push_back(name); + by[key].second = origin_text(t.origin); + } + for (auto const& [pkg, v] : by) { + std::string names; + for (auto const& n : v.first) { if (!names.empty()) names += ", "; names += n; } + rows.emplace_back("tools", std::format("{} ({}, {})", names, pkg, v.second)); + } + } + if (f.runner.set) + rows.emplace_back("runner", std::format("{} ({})", f.runner.cmd.empty() ? std::string() : f.runner.cmd.front(), + origin_text(f.runner.origin))); + for (auto const& [name, w] : f.runners) + rows.emplace_back("runner " + name, std::format("{} ({})", argv_text(w.value), origin_text(w.origin))); + if (f.link.driver.set && f.link.driver.value != "compiler") + rows.emplace_back("link", std::format("{}{} ({})", f.link.driver.value, + f.link.start.set ? ", start " + f.link.start.value : "", + origin_text(f.link.driver.origin))); + return rows; +} + +std::string to_json(const Resolved& r) { + if (!r.active) return "null"; + auto const& f = r.fields; + std::string o = "{"; + o += "\"name\":" + q(r.package); + o += ",\"version\":" + q(r.version); + o += ",\"source\":" + q(origin_text(r.selection.origin)); + o += ",\"hosted\":" + std::string(r.selection.hosted ? "true" : "false"); + o += ",\"fields\":{"; + bool first = true; + auto field = [&](std::string_view key, std::string value, const mp::Origin& origin) { + if (!first) o += ","; + first = false; + o += q(key) + ":{\"value\":" + value + ",\"source\":" + q(origin_text(origin)) + "}"; + }; + auto arr = [](const std::vector& v) { + std::string s = "["; + for (std::size_t i = 0; i < v.size(); ++i) { if (i) s += ","; s += q(v[i]); } + return s + "]"; + }; + if (f.toolchain.set) field("toolchain", q(f.toolchain.value), f.toolchain.origin); + if (f.runner.set) field("runner", arr(f.runner.cmd), f.runner.origin); + for (auto const& [k, w] : f.runners) field("runners." + k, arr(w.value), w.origin); + for (auto const& [layer, l] : r.layers) { + mp::Origin origin{l.tier, {}, 0, {}}; + if (auto it = f.layers.find(layer); it != f.layers.end() && l.tier != mp::Tier::Graph) + origin = it->second.origin; + field("layers." + layer, q(l.provider), origin); + } + for (auto const& [name, t] : f.tools) { + auto it = r.tools.find(name); + field("tools." + name, it == r.tools.end() ? q(mp::display(t.ref)) : q(it->second), t.origin); + } + if (f.link.driver.set) field("link.driver", q(f.link.driver.value), f.link.driver.origin); + if (f.link.start.set) field("link.start", q(f.link.start.value), f.link.start.origin); + if (f.link.exe.set) field("link.exe", q(f.link.exe.value), f.link.exe.origin); + if (f.link.shared.set) field("link.shared", q(f.link.shared.value), f.link.shared.origin); + if (f.carrier.set) field("carrier", q(mp::display(f.carrier.package)), f.carrier.origin); + o += "}}"; + return o; +} + +} // namespace mcpp::platform diff --git a/src/toolchain/registry.cppm b/src/toolchain/registry.cppm index 49b907536..03b36e1fb 100644 --- a/src/toolchain/registry.cppm +++ b/src/toolchain/registry.cppm @@ -1442,7 +1442,7 @@ bool host_can_serve(const triple::Triple& target) { if (target.is_freestanding()) return true; // An ok image the same way: clang and lld emit it on every host, and what // the image holds above `kal_*` is the graph's. - if (target.is_openkal()) return true; + if (target.loader_supplies_os()) return true; return false; } diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index acafd8eeb..cd10ee218 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -5052,8 +5052,8 @@ runnerX = ["qemu-aarch64-static"] // tests/unit/test_target_scalar_keys.cpp for the test that keeps this // string (and the array beside it) from drifting from the parser again. EXPECT_NE(m->schemaWarnings[0].find( - "Supported keys: cxx_runtime, linkage, min_api_level, run_under, runner, " - "sysroot, toolchain"), + "Supported keys: cxx_runtime, linkage, min_api_level, platform, run_under, " + "runner, sysroot, toolchain"), std::string::npos) << m->schemaWarnings[0]; } diff --git a/tests/unit/test_platform.cpp b/tests/unit/test_platform.cpp new file mode 100644 index 000000000..75730dc9d --- /dev/null +++ b/tests/unit/test_platform.cpp @@ -0,0 +1,414 @@ +#include + +import std; +import mcpp.manifest; +import mcpp.platform; +import mcpp.build.prepare_inputs; +import mcpp.toolchain.triple; + +// SPEC-013 (design 2026-10-28): what a platform states, how it is read in its +// three places, which platform a graph selects, how the tiers merge, and the +// two templates of the link (artifact names, seal steps). Tables, so each rule +// is one row. + +namespace mm = mcpp::manifest; +namespace mp = mcpp::manifest::platform; +namespace pf = mcpp::platform; +namespace cfgpred = mcpp::build::cfgpred; + +namespace { + +mm::Manifest parse(std::string_view text) { + auto m = mm::parse_string(text); + if (!m) ADD_FAILURE() << m.error().format(); + return m ? *m : mm::Manifest{}; +} + +std::string parse_error(std::string_view text) { + auto m = mm::parse_string(text); + return m ? std::string{} : m.error().message; +} + +pf::Matches matcher(std::string_view triple, std::string_view platformPackage = {}) { + auto c = cfgpred::context_for(triple, platformPackage); + return [c](std::string_view sel) { return cfgpred::matches(std::string(sel), c); }; +} + +constexpr std::string_view kPlatform = R"( +[package] +name = "board-platform" +version = "1.2.0" + +[platform] +targets = ["x86_64-openkal", "aarch64-openkal"] +toolchain = "llvm@23.1.3" +runner = { cmd = ["okrun", "{}"], host = "same-arch" } +carrier = { package = "openkal-run@0.1.1", section = ".oke", main = "carrier/main.cpp" } + +[platform.layers] +"c++-abi" = "openkal-llvm-runtime@0.16.3" +c-abi = { name = "openkal-musl", version = "0.22.3", default-features = false, features = ["posix"] } +kernel-abi = false + +[platform.tools] +okld = "openkal-tools@0.1.1" +probe = { program = "probe-rs" } + +[platform.build] +cxxflags = ["-fPIC"] +[platform.build.'cfg(arch = "aarch64")'] +cxxflags = ["-ffixed-x18"] + +[platform.link] +driver = "lld" +start = "ok_main" +flags = ["-shared"] +inputs = [{ from = ["okld", "note-source"], as = "ok-notes.S" }, { file = "crt/x.o" }] +exe = "{name}.oke" +shared = "lib{name}.okso" +seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] + +[platform.hosted] +toolchain = "llvm@23.1.3" +[platform.hosted.layers] +c-abi = "openkal-musl@0.22.3" + +[platform.versions] +openkal-musl = "0.22.3" +)"; + +} // namespace + +// ── The reader ────────────────────────────────────────────────────────────── + +TEST(PlatformManifest, EveryGroupIsRead) { + auto m = parse(kPlatform); + ASSERT_TRUE(m.platformDeclared); + auto const& p = m.platform; + EXPECT_EQ(p.targets.value, (std::vector{"x86_64-openkal", "aarch64-openkal"})); + EXPECT_EQ(p.toolchain.value, "llvm@23.1.3"); + EXPECT_EQ(p.runner.cmd, (std::vector{"okrun", "{}"})); + EXPECT_EQ(p.runner.host, "same-arch"); + EXPECT_EQ(p.carrier.package.spec.shortName, "openkal-run"); + EXPECT_EQ(p.carrier.section, ".oke"); + EXPECT_TRUE(p.carrier.main.ends_with("carrier/main.cpp")); + ASSERT_TRUE(p.layers.contains("c++-abi")); + EXPECT_EQ(p.layers.at("c++-abi").ref.spec.version, "0.16.3"); + EXPECT_EQ(p.layers.at("c-abi").ref.spec.shortName, "openkal-musl"); + EXPECT_FALSE(p.layers.at("c-abi").ref.spec.defaultFeatures); + EXPECT_EQ(p.layers.at("c-abi").ref.spec.features, (std::vector{"posix"})); + EXPECT_TRUE(p.layers.at("kernel-abi").ref.off); + EXPECT_EQ(p.tools.at("okld").ref.spec.shortName, "openkal-tools"); + EXPECT_EQ(p.tools.at("probe").ref.program, "probe-rs"); + EXPECT_EQ(p.build.cxxflags, (std::vector{"-fPIC"})); + ASSERT_EQ(p.buildCfg.size(), 1u); + EXPECT_EQ(p.link.driver.value, "lld"); + EXPECT_EQ(p.link.start.value, "ok_main"); + ASSERT_EQ(p.link.inputs.size(), 2u); + EXPECT_EQ(p.link.inputs[0].as, "ok-notes.S"); + EXPECT_TRUE(p.link.inputs[1].file.ends_with("crt/x.o")); + EXPECT_EQ(p.link.seal.size(), 2u); + EXPECT_EQ(p.hosted.toolchain.value, "llvm@23.1.3"); + EXPECT_TRUE(p.hosted.layers.contains("c-abi")); + EXPECT_EQ(p.versions.at("openkal-musl"), "0.22.3"); + EXPECT_EQ(p.runner.origin.tier, mp::Tier::Platform); + EXPECT_GT(p.runner.origin.line, 0); +} + +TEST(PlatformManifest, RefusalsNameTheKey) { + const std::string head = "[package]\nname = \"p\"\nversion = \"1.0.0\"\n"; + struct Row { std::string body; std::string_view says; }; + const Row rows[] = { + { "[platform]\ntoolchain = \"llvm@23.1.3\"\n", "names the targets it serves" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.layers]\nc-lib = \"x@1\"\n", + "not a target-side layer" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.link]\ndriver = \"gold\"\n", + "\"compiler\"" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.link]\nexe = \"x.oke\"\n", + "{name}" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.link]\nexe = \"lib{name}.oke\"\n", + "begins with the target's name" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\nrunner = []\n", "empty command" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\nrunner = { cmd = [\"x\"], host = \"far\" }\n", + "same-arch" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.layers]\nc-abi = { version = \"1\" }\n", + "names the package too" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.layers]\nc-abi = { program = \"x\" }\n", + "for a tool only" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.hosted]\nrunner = [\"x\"]\n", + "a host triple takes" }, + { "[target.'cfg(platform = \"openkal\")']\nplatform = \"x@1.0.0\"\n", + "cannot be selected under a selector" }, + { "[target.x86_64-openkal]\nlayers = \"x\"\n", "must be a table" }, + { "[host]\nrunner = [\"x\"]\n", "[host] has no key" }, + }; + for (auto const& r : rows) { + auto e = parse_error(head + r.body); + EXPECT_NE(e.find(r.says), std::string::npos) << r.body << "\n=> " << e; + } +} + +TEST(PlatformManifest, OverridesAndHostAreRead) { + auto m = parse(R"( +[package] +name = "app" +version = "0.1.0" + +[host] +platform = "openkal-platform@0.17.1" + +[target.'cfg(platform = "openkal")'] +runner = { cmd = ["my-okrun", "{}"], exclusive = true } +[target.'cfg(platform = "openkal")'.layers] +c-abi = { path = "../openkal-musl" } +[target.'cfg(platform = "openkal")'.tools] +okld = { path = "../openkal-image/tools" } +[target.'cfg(platform = "openkal")'.link] +flags = ["--build-id"] + +[target.x86_64-openkal] +platform = "none" +)"); + for (auto const& w : m.schemaWarnings) ADD_FAILURE() << "unexpected warning: " << w; + EXPECT_TRUE(m.hostPlatform.platformSet); + EXPECT_EQ(m.hostPlatform.platform.spec.shortName, "openkal-platform"); + ASSERT_EQ(m.platformOverrides.size(), 2u); + auto const& o = m.platformOverrides[0]; + EXPECT_EQ(o.fields.runner.cmd.front(), "my-okrun"); + EXPECT_TRUE(o.fields.runner.exclusive); + EXPECT_TRUE(o.fields.layers.at("c-abi").ref.spec.path.ends_with("openkal-musl")); + EXPECT_TRUE(o.fields.tools.at("okld").ref.spec.path.ends_with("openkal-image/tools")); + EXPECT_EQ(o.fields.link.flags.value, (std::vector{"--build-id"})); + EXPECT_EQ(o.fields.runner.origin.tier, mp::Tier::Manifest); + EXPECT_TRUE(m.platformOverrides[1].platformNone); + // The runner's command is the row's template as well. + EXPECT_EQ(m.targetOverrides.at("cfg(platform = \"openkal\")").runner.front(), "my-okrun"); +} + +TEST(PlatformManifest, ReferencesInEveryForm) { + struct Row { std::string_view text; std::string_view ns, name, version; }; + const Row rows[] = { + { "openkal-platform@0.17.1", "mcpplibs", "openkal-platform", "0.17.1" }, + { "mcpplibs.openkal-platform@0.17.1", "mcpplibs", "openkal-platform", "0.17.1" }, + { "acme.board@1.0.0", "acme", "board", "1.0.0" }, + }; + for (auto const& r : rows) { + auto ref = mp::parse_ref_text(r.text); + ASSERT_TRUE(ref) << r.text; + EXPECT_EQ(ref->spec.namespace_, r.ns); + EXPECT_EQ(ref->spec.shortName, r.name); + EXPECT_EQ(ref->spec.version, r.version); + } + EXPECT_FALSE(mp::parse_ref_text("openkal-platform")); + EXPECT_FALSE(mp::parse_ref_text("@1.0")); + EXPECT_EQ(mp::platform_name("openkal-platform"), "openkal"); + EXPECT_EQ(mp::platform_name("stm32-bsp"), "stm32-bsp"); +} + +// ── Selection: the first tier that answers decides ───────────────────────── + +TEST(PlatformSelect, TheTiersInOrder) { + auto m = parse(R"( +[package] +name = "app" +version = "0.1.0" +[target.'cfg(os = "openkal")'] +platform = "project-platform@1.0.0" +)"); + auto cfg = parse(R"( +[package] +name = "cfg" +version = "0.1.0" +[target.x86_64-openkal] +platform = "machine-platform@1.0.0" +)"); + auto t = matcher("x86_64-openkal"); + auto pick = [&](std::string_view flag, std::string_view env, bool project, bool config, + std::string_view column) { + pf::ChoiceInputs in{ flag, env, "MCPP_TARGET_X86_64_OPENKAL_PLATFORM", + project ? std::span(m.platformOverrides) + : std::span{}, + config ? std::span(cfg.platformOverrides) + : std::span{}, + column }; + auto s = pf::select_target(in, t); + EXPECT_TRUE(s) << (s ? "" : s.error()); + return s ? (s->none ? std::string("none") : s->ref.spec.shortName) + "|" + + std::string(mp::tier_name(s->origin.tier)) + : std::string("error"); + }; + EXPECT_EQ(pick("flag-platform@1.0.0", "env-platform@1.0.0", true, true, "row-platform@1.0.0"), + "flag-platform|flag"); + EXPECT_EQ(pick("", "env-platform@1.0.0", true, true, "row-platform@1.0.0"), "env-platform|env"); + EXPECT_EQ(pick("", "", true, true, "row-platform@1.0.0"), "project-platform|mcpp.toml"); + EXPECT_EQ(pick("", "", false, true, "row-platform@1.0.0"), "machine-platform|mcpp.toml"); + EXPECT_EQ(pick("", "", false, false, "row-platform@1.0.0"), "row-platform|built-in"); + EXPECT_EQ(pick("", "", false, false, ""), "none|built-in"); + EXPECT_EQ(pick("none", "", true, true, "row-platform@1.0.0"), "none|flag"); +} + +TEST(PlatformSelect, TheMostSpecificSelectorDecides) { + auto m = parse(R"( +[package] +name = "app" +version = "0.1.0" +[target.x86_64-openkal] +platform = "triple@1.0.0" +[target.'cfg(arch = "x86_64")'] +platform = "arch@1.0.0" +)"); + pf::ChoiceInputs in{ "", "", "", m.platformOverrides, {}, "" }; + auto s = pf::select_target(in, matcher("x86_64-openkal")); + ASSERT_TRUE(s); + EXPECT_EQ(s->ref.spec.shortName, "triple"); + auto a = pf::select_target(in, matcher("aarch64-openkal")); + ASSERT_TRUE(a); + EXPECT_TRUE(a->none); +} + +TEST(PlatformSelect, TheHostGraph) { + mp::Override project, config; + config.platformSet = true; + config.platform = *mp::parse_ref_text("machine@1.0.0"); + auto s = pf::select_host("", false, project, config); + ASSERT_TRUE(s); + EXPECT_EQ(s->ref.spec.shortName, "machine"); + project.platformSet = true; + project.platformNone = true; + s = pf::select_host("", false, project, config); + ASSERT_TRUE(s); + EXPECT_TRUE(s->none); + s = pf::select_host("flag@2.0.0", true, project, config); + ASSERT_TRUE(s); + EXPECT_EQ(s->ref.spec.shortName, "flag"); + EXPECT_EQ(s->origin.tier, mp::Tier::Flag); +} + +TEST(PlatformSelect, ADirectoryIsAPath) { + auto s = pf::parse_choice("path:/opt/board", {}); + ASSERT_TRUE(s); + EXPECT_EQ(s->ref.spec.path, "/opt/board"); + s = pf::parse_choice("../board", {}); + ASSERT_TRUE(s); + EXPECT_FALSE(s->ref.spec.path.empty()); + EXPECT_FALSE(pf::parse_choice("not-a-reference", {})); +} + +// ── Merge: per field, flags appended, sequences replaced ─────────────────── + +TEST(PlatformMerge, TheRulesOfSection45) { + auto base = parse(kPlatform).platform; + mp::Statement over; + over.runner.cmd = {"my-okrun", "{}"}; + over.runner.set = true; // argv form: keeps host + over.build.cxxflags = {"-DUSER"}; + over.link.flags = { {"--build-id"}, {}, true }; + over.link.seal = { {"sign", "{}"} }; + over.link.sealSet = true; + over.layers["c-abi"] = { mp::PackageRef{ .off = true }, {} }; + pf::merge(base, over); + EXPECT_EQ(base.runner.cmd.front(), "my-okrun"); + EXPECT_EQ(base.runner.host, "same-arch"); + EXPECT_EQ(base.build.cxxflags, (std::vector{"-fPIC", "-DUSER"})); + EXPECT_EQ(base.link.flags.value, (std::vector{"-shared", "--build-id"})); + ASSERT_EQ(base.link.seal.size(), 1u); + EXPECT_EQ(base.link.seal.front().front(), "sign"); + EXPECT_EQ(base.link.inputs.size(), 2u); // not stated: kept + EXPECT_TRUE(base.layers.at("c-abi").ref.off); + EXPECT_FALSE(base.layers.at("c++-abi").ref.off); + EXPECT_EQ(base.link.start.value, "ok_main"); +} + +TEST(PlatformMerge, AHostTripleTakesTheHostedPartOnly) { + auto p = parse(kPlatform).platform; + auto h = pf::hosted_view(p); + EXPECT_EQ(h.toolchain.value, "llvm@23.1.3"); + EXPECT_TRUE(h.layers.contains("c-abi")); + EXPECT_FALSE(h.layers.contains("c++-abi")); + EXPECT_FALSE(h.runner.set); + EXPECT_FALSE(h.link.driver.set); + EXPECT_TRUE(h.build.empty()); + auto served = pf::serves(p, "board-platform", *mcpp::toolchain::triple::parse("x86_64-linux-gnu")); + ASSERT_TRUE(served); + EXPECT_TRUE(*served); + auto target = pf::serves(p, "board-platform", *mcpp::toolchain::triple::parse("x86_64-openkal")); + ASSERT_TRUE(target); + EXPECT_FALSE(*target); + p.hosted = {}; + auto refused = pf::serves(p, "board-platform", *mcpp::toolchain::triple::parse("x86_64-linux-gnu")); + ASSERT_FALSE(refused); + EXPECT_NE(refused.error().find("[platform.hosted]"), std::string::npos); +} + +TEST(PlatformMerge, CfgTablesFollowTheTarget) { + auto p = parse(kPlatform).platform; + auto x = pf::evaluate_flags(p, matcher("x86_64-openkal")); + auto a = pf::evaluate_flags(p, matcher("aarch64-openkal")); + EXPECT_EQ(x.cxxflags, (std::vector{"-fPIC"})); + EXPECT_EQ(a.cxxflags, (std::vector{"-fPIC", "-ffixed-x18"})); +} + +TEST(PlatformMerge, CfgPlatformMatchesTheNameOrThePackage) { + EXPECT_TRUE(cfgpred::matches("cfg(platform = \"openkal\")", + cfgpred::context_for("x86_64-openkal", "openkal-platform"))); + EXPECT_TRUE(cfgpred::matches("cfg(platform = \"openkal-platform\")", + cfgpred::context_for("x86_64-openkal", "openkal-platform"))); + EXPECT_FALSE(cfgpred::matches("cfg(platform = \"openkal\")", + cfgpred::context_for("x86_64-openkal"))); + EXPECT_GT(mcpp::manifest::cfg::specificity("cfg(os = \"openkal\")"), + mcpp::manifest::cfg::specificity("cfg(platform = \"openkal\")")); +} + +// ── The link's templates ──────────────────────────────────────────────────── + +TEST(PlatformLink, NamePatterns) { + auto e = pf::split_name_pattern("{name}.oke"); + ASSERT_TRUE(e); + EXPECT_EQ(e->first, ""); + EXPECT_EQ(e->second, ".oke"); + auto s = pf::split_name_pattern("lib{name}.okso"); + ASSERT_TRUE(s); + EXPECT_EQ(s->first, "lib"); + EXPECT_EQ(s->second, ".okso"); + EXPECT_FALSE(pf::split_name_pattern("name.oke")); + EXPECT_FALSE(pf::split_name_pattern("{name}{name}")); + EXPECT_FALSE(pf::split_name_pattern("bin/{name}")); +} + +TEST(PlatformLink, SealSteps) { + const std::vector check{"okld", "check", "{}", "--with", "{needed}"}; + const std::vector two{"a.okso", "b.okso"}; + EXPECT_EQ(pf::expand_step(check, "x.oke.tmp", two), + (std::vector{"okld", "check", "x.oke.tmp", "--with", "a.okso", + "--with", "b.okso"})); + EXPECT_EQ(pf::expand_step(check, "x.oke.tmp", {}), + (std::vector{"okld", "check", "x.oke.tmp"})); + EXPECT_EQ(pf::expand_step({"sign", "--image={}", "{needed}"}, "x", two), + (std::vector{"sign", "--image=x", "a.okso", "b.okso"})); +} + +// ── The report ────────────────────────────────────────────────────────────── + +TEST(PlatformReport, SourcesAndJson) { + pf::Resolved r; + r.active = true; + r.package = "mcpplibs.openkal-platform"; + r.version = "0.17.1"; + r.fields = parse(kPlatform).platform; + r.selection.origin = { mp::Tier::BuiltIn, {}, 0, "the target table" }; + r.layers["c-abi"] = { "openkal-musl@0.22.3", mp::Tier::Platform }; + r.layers["c++-abi"] = { "acme-runtime@1.0.0", mp::Tier::Graph }; + EXPECT_EQ(r.display(), "openkal-platform 0.17.1"); + EXPECT_EQ(r.identity(), "mcpplibs.openkal-platform@0.17.1"); + auto rows = pf::report_rows(r); + ASSERT_FALSE(rows.empty()); + bool graph = false; + for (auto const& [k, v] : rows) + if (k == "c++-abi") graph = v.find("(graph)") != std::string::npos; + EXPECT_TRUE(graph); + auto json = pf::to_json(r); + EXPECT_NE(json.find("\"name\":\"mcpplibs.openkal-platform\""), std::string::npos); + EXPECT_NE(json.find("\"source\":\"built-in\""), std::string::npos); + EXPECT_NE(json.find("\"layers.c++-abi\""), std::string::npos); + EXPECT_EQ(pf::to_json(pf::Resolved{}), "null"); +} From d5fb21135c6e5edf8bd27af3485a223d857c19d9 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 03:42:48 +0900 Subject: [PATCH 06/54] platform: the report names the platform and what it added; mcpp.lock says origin = "platform"; protocol 17 states the platform's fields from a program, and a platform's plugin runs in the platform phase - The Target lines mark a layer whose package the platform added as `platform`; a `Platform ()` block follows with the tools, runner and link the platform contributes; the toolchain line says when the compiler is the platform's. - A package the platform added is locked with `origin = "platform"`; a lock without the field reads as before. - mcpp.core protocol 17: link_driver, link_input / link_input_from, artifact_name, seal, run_host -- the platform statement's fields at the build-program tier, validated before they apply and merged in the order the programs ran, the root's last. - `[platform] plugin` runs the platform package's program with `mcpp::phase()` = "platform": it may state compile and link flags, the link's fields, and runners, nothing else (platform-phase-directive), and its compile flags are the platform's for every unit. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- modules/buildmcpp/src/directives.cppm | 116 +++++++++++++++++++- modules/buildmcpp/src/program_protocol.cppm | 6 +- modules/manifest/src/targetside_model.cppm | 10 +- modules/manifest/src/types.cppm | 5 + src/build/build_program.cppm | 31 ++++++ src/build/hostprogram.cppm | 28 +++++ src/build/prepare/features.cpp | 32 ++++++ src/build/prepare/platform.cpp | 15 ++- src/build/prepare/records.cpp | 4 + src/build/prepare/target_side.cpp | 16 +++ src/build/prepare/toolchain.cpp | 9 ++ src/platform/platform.cppm | 14 +-- src/pm/lock_io.cppm | 11 +- tests/unit/test_build_directives.cpp | 47 +++++++- 14 files changed, 325 insertions(+), 19 deletions(-) diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index fea054399..f34989f61 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -201,6 +201,11 @@ enum class Slot : std::size_t { // starter. Not an ldflag: the engine renders it per linker and refuses // two statements that disagree, which a flag would make two `-e` words. LinkStart, + LinkDriver, + LinkInputs, + ArtifactNames, + Seal, + RunHost, Count }; inline constexpr std::size_t kSlotCount = static_cast(Slot::Count); @@ -299,7 +304,7 @@ struct Def { int sinceProtocol; }; -inline constexpr std::array kTable{{ +inline constexpr std::array kTable{{ // wire tag slot scope transform must missingPrefix missingSuffix since {"cxxflag", "cxxflag", Slot::CxxFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, {"cflag", "cflag", Slot::CFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, @@ -492,6 +497,16 @@ inline constexpr std::array kTable{{ // as `link-script` is: it is a fact of every image linked over the board. // kCacheEpoch is not bumped, for the reason `runtime-search-dir` states. {"link-start", "link-start", Slot::LinkStart, Scope::LinkGlobal, Transform::Verbatim, false, "", "", 16}, + // v17 (SPEC-013 §4.1): the platform's link and runner fields, stated by a + // program -- a board's or a platform's plugin. LinkGlobal and RunGlobal as + // `link-script` and `runner` are; they become the build's platform + // statement at the build-program tier. kCacheEpoch is not bumped, for the + // reason `runtime-search-dir` states. + {"link-driver", "link-driver", Slot::LinkDriver, Scope::LinkGlobal, Transform::Verbatim, false, "", "", 17}, + {"link-input", "link-input", Slot::LinkInputs, Scope::LinkGlobal, Transform::Verbatim, false, "", "", 17}, + {"artifact-name", "artifact-name", Slot::ArtifactNames, Scope::LinkGlobal, Transform::Verbatim, false, "", "", 17}, + {"seal", "seal", Slot::Seal, Scope::LinkGlobal, Transform::Verbatim, false, "", "", 17}, + {"run-host", "run-host", Slot::RunHost, Scope::RunGlobal, Transform::Verbatim, false, "", "", 17}, }}; // ── Collected output of one run ──────────────────────────────────────────── @@ -640,6 +655,12 @@ bool glob_leaves_package(std::string_view pattern); // place that knows which manifest channel each slot feeds. void apply(mcpp::manifest::Manifest& m, const Directives& d); +// The platform's fields a program stated (protocol 17: `link-driver`, +// `link-input`, `artifact-name`, `seal`, `run-host`), as a platform statement +// at the build-program tier (SPEC-013 §4.1). `apply` calls it; the platform +// phase reads it for the platform's own program. +void apply_platform(mcpp::manifest::platform::Statement& s, const Directives& d); + // Decode one `mcpp:action=` JSON payload. nullopt = malformed. std::optional decode_action(std::string_view payload); @@ -1188,6 +1209,10 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { // program that states two different symbols, so the last line is the one. for (auto const& symbol : d.at(Slot::LinkStart)) m.runtimeConfig.linkIntent.start = symbol; + + // The platform's fields (protocol 17), at the build-program tier; + // `platform_directive_error` has refused a malformed line. + apply_platform(m.buildConfig.programPlatform, d); } // The five roles `mcpp::action::role` may name, in the order @@ -1339,6 +1364,55 @@ static std::string named_target_error(const mcpp::manifest::Manifest& m, return {}; } +void apply_platform(mcpp::manifest::platform::Statement& s, const Directives& d) { + namespace mp = mcpp::manifest::platform; + auto origin = [](std::string_view wire) { + return mp::Origin{ mp::Tier::Program, {}, 0, std::format("mcpp:{}", wire) }; + }; + for (auto const& v : d.at(Slot::LinkDriver)) + s.link.driver = mp::Text{ v, origin("link-driver"), true }; + if (!d.at(Slot::LinkInputs).empty()) { + std::vector inputs; + for (auto const& v : d.at(Slot::LinkInputs)) { + if (v.starts_with("file:")) { + inputs.push_back({ .file = v.substr(5) }); + continue; + } + const auto colon = v.find(':', 5); + const auto as = v.substr(5, colon - 5); + const auto token = v.substr(colon + 1); + auto it = std::ranges::find_if(inputs, [&](auto const& in) { return in.as == as; }); + if (it == inputs.end()) inputs.push_back({ .from = { token }, .as = as }); + else it->from.push_back(token); + } + s.link.inputs = std::move(inputs); + s.link.inputsOrigin = origin("link-input"); + s.link.inputsSet = true; + } + for (auto const& v : d.at(Slot::ArtifactNames)) { + const auto colon = v.find(':'); + auto& field = v.substr(0, colon) == "exe" ? s.link.exe : s.link.shared; + field = mp::Text{ v.substr(colon + 1), origin("artifact-name"), true }; + } + if (!d.at(Slot::Seal).empty()) { + std::map> steps; + for (auto const& v : d.at(Slot::Seal)) { + const auto colon = v.find(':'); + int index = 0; + std::from_chars(v.data(), v.data() + colon, index); + steps[index].push_back(v.substr(colon + 1)); + } + s.link.seal.clear(); + for (auto& [i, step] : steps) s.link.seal.push_back(std::move(step)); + s.link.sealOrigin = origin("seal"); + s.link.sealSet = true; + } + for (auto const& v : d.at(Slot::RunHost)) { + s.runner.host = v; + s.runner.origin = origin("run-host"); + } +} + std::string target_directive_error(const mcpp::manifest::Manifest& m, const Directives& d) { if (auto e = named_target_error(m, d.at(Slot::WindowsSubsystem), "windows-subsystem", "windows_subsystem", true); @@ -1348,6 +1422,46 @@ std::string target_directive_error(const mcpp::manifest::Manifest& m, const Dire "windows-entry", "windows_entry", false); !e.empty()) return e; + // The platform's fields (protocol 17). + for (auto const& v : d.at(Slot::LinkDriver)) + if (v != "lld" && v != "compiler") + return std::format("build.mcpp emitted `mcpp:link-driver={}`; the drivers are " + "\"lld\" and \"compiler\"", v); + for (auto const& v : d.at(Slot::LinkInputs)) { + const bool file = v.starts_with("file:") && v.size() > 5; + const auto colon = v.find(':', 5); + const bool from = v.starts_with("from:") && colon != std::string::npos && colon > 5 + && colon + 1 < v.size(); + if (!file && !from) + return std::format("build.mcpp emitted `mcpp:link-input={}`; an input is " + "`file:` or `from::`", v); + } + for (auto const& v : d.at(Slot::ArtifactNames)) { + const auto colon = v.find(':'); + const auto kind = v.substr(0, colon); + const auto pattern = colon == std::string::npos ? std::string() : v.substr(colon + 1); + if ((kind != "exe" && kind != "shared") || pattern.empty()) + return std::format("build.mcpp emitted `mcpp:artifact-name={}`; write " + "`exe:` or `shared:`", v); + const auto at = pattern.find("{name}"); + if (at == std::string::npos || pattern.find("{name}", at + 1) != std::string::npos + || (kind == "exe" && at != 0)) + return std::format("build.mcpp emitted `mcpp:artifact-name={}`; the pattern names " + "the artifact as {{name}} once{}", v, + kind == "exe" ? ", at its start" : ""); + } + for (auto const& v : d.at(Slot::Seal)) { + const auto colon = v.find(':'); + const auto index = v.substr(0, colon); + if (colon == std::string::npos || index.empty() + || !std::ranges::all_of(index, [](char c) { return c >= '0' && c <= '9'; }) + || colon + 1 >= v.size()) + return std::format("build.mcpp emitted `mcpp:seal={}`; write `:`", v); + } + for (auto const& v : d.at(Slot::RunHost)) + if (v != "same-arch" && v != "any") + return std::format("build.mcpp emitted `mcpp:run-host={}`; it is \"same-arch\" " + "or \"any\"", v); // `link-start` (protocol 16): a symbol, and one per program. auto const& starts = d.at(Slot::LinkStart); for (auto const& symbol : starts) { diff --git a/modules/buildmcpp/src/program_protocol.cppm b/modules/buildmcpp/src/program_protocol.cppm index b4b8c59e2..bf92a4ce6 100644 --- a/modules/buildmcpp/src/program_protocol.cppm +++ b/modules/buildmcpp/src/program_protocol.cppm @@ -122,7 +122,11 @@ export namespace mcpp::build::program_protocol { // v16 (design 2026-10-26 §24.3): `mcpp:link-start=` -- the symbol every image // linked over the package is entered at, the build-program form of a target's // `start`. No directive of v15 changes spelling. -inline constexpr int kProtocolVersion = 16; +// v17 (SPEC-013 §4.1): the platform's fields a program states -- `link-driver`, +// `link-input`, `artifact-name`, `seal`, `run-host` -- and `mcpp::phase()` +// answering "platform" for a platform's plugin. No directive of v16 changes +// spelling. +inline constexpr int kProtocolVersion = 17; // ── Cache-format epoch ───────────────────────────────────────────────────── // diff --git a/modules/manifest/src/targetside_model.cppm b/modules/manifest/src/targetside_model.cppm index 1f583a3a9..304b59366 100644 --- a/modules/manifest/src/targetside_model.cppm +++ b/modules/manifest/src/targetside_model.cppm @@ -245,6 +245,9 @@ struct Layer { std::string interfaceName; // openkal / linux / win32 / darwin / musl / glibc / libc++ std::string impl; // package@version, or an xpkg reference bool subset = false; // C++ layer: a freestanding subset, not the whole library + // The package was added to the graph by the target's platform (SPEC-013 + // §4.3): still a graph package, reported as the platform's. + bool byPlatform = false; bool absent() const { return origin == Origin::None; } bool prebuilt() const { return origin == Origin::Payload || origin == Origin::Xpkg; } @@ -1082,13 +1085,14 @@ inline std::string format_layers(const TargetSide& ts, bool verbose) { std::string suffix = l.subset ? ", subset" : ""; // One column layout whether or not an implementation is named, so the // rows read as a table rather than as a list of sentences. + const std::string_view from = l.byPlatform ? std::string_view("platform") + : origin_name(l.origin); if (l.impl.empty()) out += std::format("{}{:<17} {:<14} ({}{})\n", kIndent, label, - l.interfaceName, origin_name(l.origin), suffix); + l.interfaceName, from, suffix); else out += std::format("{}{:<17} {:<14} ({}, {}{})\n", kIndent, label, - l.interfaceName, l.impl, origin_name(l.origin), - suffix); + l.interfaceName, l.impl, from, suffix); } return out; } diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 8f83362d3..f4bab329f 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -876,6 +876,11 @@ struct BuildConfig : BuildInputs { // configuration error, not something to merge: appending would produce an // argv that is neither one's, and it would fail at exec time with no // indication of which package contributed which token. + // The platform's fields a build program stated (protocol 17; SPEC-013 + // §4.1): its link driver, link inputs, artifact names, seal steps and + // runner host, at the build-program tier. Filled by `apply`; prepare + // merges every program's into the build's platform statement. + mcpp::manifest::platform::Statement programPlatform; std::vector runner; // NAMED WAYS OF REACHING THE ARTEFACT, AND THE ENGINE KNOWS NONE OF // THEIR NAMES. diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index a95553768..89d795656 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -1305,6 +1305,37 @@ std::expected run_build_program_impl( " `mcpp::plugins::toolchain::configure` returns true.", def.wire)); } + } else if (env.phase == "platform") { + // THE PLATFORM PHASE STATES THE PLATFORM'S BUILD, LINK AND RUNNER + // FIELDS AND NOTHING ELSE (SPEC-013 §4.2): it describes a class of + // targets, not a package of sources, and what it states reaches every + // unit and every link of the target. Refused rather than dropped, + // naming the first directive that does not belong. + for (auto const& def : dirs::kTable) { + switch (def.slot) { + case Slot::CxxFlags: case Slot::CFlags: case Slot::LdFlags: + case Slot::Runner: case Slot::NamedRunner: case Slot::RunnerLongLived: + case Slot::RunExclusive: case Slot::LinkStart: case Slot::LinkDriver: + case Slot::LinkInputs: case Slot::ArtifactNames: case Slot::Seal: + case Slot::RunHost: case Slot::RerunFiles: case Slot::RerunEnv: + case Slot::RerunGlobs: case Slot::Warnings: case Slot::Diagnostics: + continue; + default: break; + } + if (d.at(def.slot).empty()) continue; + mcpp::build::refusal::record(mcpp::build::refusal::Code::PlatformPhaseDirective); + return std::unexpected(std::format( + "build.mcpp stated `mcpp:{}=` in its platform phase.\n" + " A platform's plugin states the platform's build, link and runner\n" + " fields only (compile and link flags, the link driver, inputs,\n" + " entry, artifact names, seal steps, runners); `mcpp::phase()` is\n" + " \"platform\" there (SPEC-013 §4.2).", + def.wire)); + } + if (!d.at(Slot::ToolchainStatement).empty()) + return std::unexpected(std::string( + "build.mcpp stated `mcpp:toolchain=` in its platform phase; a platform's " + "toolchain is its `[platform] toolchain`")); } else if (!d.at(Slot::ToolchainStatement).empty()) { return std::unexpected(std::string( "build.mcpp stated `mcpp:toolchain=` outside the toolchain phase.\n" diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 18509cc30..1bd400cf6 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -191,6 +191,34 @@ inline void link_flag(const char* flag) { std::printf("mcpp:link-flag= // `link_script` does, and is checked there against the symbol the program or // its entry states; an image is entered at one symbol. inline void link_start(const char* symbol) { std::printf("mcpp:link-start=%s\n", symbol); } +// THE PLATFORM'S LINK, STATED BY A PROGRAM (protocol 17, SPEC-013 §4.1): the +// same fields as `[platform.link]`, for the part a board or a platform plugin +// computes. Each reaches the consumer's link, as `link_script` does. +// +// `link_driver("lld")`: `ld.lld` itself links, with the flags `link_flag` +// states; "compiler" (the default) leaves the link to the compiler driver. +inline void link_driver(const char* driver) { std::printf("mcpp:link-driver=%s\n", driver); } +// A file linked into every executable and shared library (assembled first when +// it is a `.S`/`.s`). +inline void link_input(const char* path) { std::printf("mcpp:link-input=file:%s\n", path); } +// The standard output of a program, saved as `as` and linked in the same way; +// one token of its argv per call, in order (`mcpp::link_input_from("notes.S", +// "okld")`, then `"note-source"`). +inline void link_input_from(const char* as, const char* token) { + std::printf("mcpp:link-input=from:%s:%s\n", as, token); +} +// How an artifact is named: `kind` is "exe" or "shared", `pattern` names it +// with `{name}` ("{name}.oke", "lib{name}.okso"). +inline void artifact_name(const char* kind, const char* pattern) { + std::printf("mcpp:artifact-name=%s:%s\n", kind, pattern); +} +// One token of seal step `step` (0, 1, ...): steps run in order on the linked +// file (`{}`) before it is published, `{needed}` naming the shared artifacts +// it needs. `mcpp::seal(0, "okld"); mcpp::seal(0, "check"); mcpp::seal(0, "{}");` +inline void seal(int step, const char* token) { std::printf("mcpp:seal=%d:%s\n", step, token); } +// Where the runner runs the artifact: "same-arch" (a machine of its +// architecture) or "any" (an emulator or a probe reaches it from any machine). +inline void run_host(const char* where) { std::printf("mcpp:run-host=%s\n", where); } // The PE subsystem ("console" | "windows") and the entry function ("main" | // "wmain" | "WinMain" | "wWinMain") of an executable target of THIS package, // named by `target` (#618). The same fields as `[targets.] diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 8bd7e9c60..1244971ac 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -7,6 +7,7 @@ import :state; import mcpp.build.prepare_inputs; import std; +import mcpp.platform; import mcpp.diag; import mcpp.log; import mcpp.build.refusal; @@ -1788,6 +1789,13 @@ static std::expected step6_dependency_build_programs(PrepareS break; } } + // THE PLATFORM'S PLUGIN (SPEC-013 §4.2): the platform package's + // program runs in the `platform` phase, and states only the + // platform's build, link and runner fields. + const bool platformPhase = state.platform.active && state.platformData.plugin.set + && std::filesystem::weakly_canonical(pkg.root) + == std::filesystem::weakly_canonical(state.platform.root); + if (platformPhase) bpEnv.phase = "platform"; bpEnv.languageModules = pkg.manifest.language.modules; bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; if (auto dit = state.deviceSourcesByPackage.find(pkg.root.string()); dit != state.deviceSourcesByPackage.end()) @@ -1820,6 +1828,8 @@ static std::expected step6_dependency_build_programs(PrepareS bpEnv.programError = program.error(); } auto& bcDep = pkg.manifest.buildConfig; + const auto cxxN = bcDep.cxxflags.size(); + const auto cN = bcDep.cflags.size(); const auto mark = state.markDirectiveTail(pkg.manifest); const auto ldN = bcDep.ldflags.size(); const auto actN = bcDep.actions.size(); @@ -1935,6 +1945,28 @@ static std::expected step6_dependency_build_programs(PrepareS state.m->buildConfig.runExclusive = true; state.m->buildConfig.ldflags.insert(state.m->buildConfig.ldflags.end(), bcDep.ldflags.begin() + ldN, bcDep.ldflags.end()); + // THE PLATFORM'S FIELDS A PROGRAM STATED (protocol 17), at the + // build-program tier of the build's platform statement, named + // after the package (SPEC-013 §4.1). A plugin's compile flags are + // the platform's, for every unit of the target, not its own. + { + auto stated = std::move(bcDep.programPlatform); + bcDep.programPlatform = {}; + if (platformPhase) { + stated.build.cxxflags.assign(bcDep.cxxflags.begin() + static_cast(cxxN), + bcDep.cxxflags.end()); + stated.build.cflags.assign(bcDep.cflags.begin() + static_cast(cN), + bcDep.cflags.end()); + } + const auto who = pkg.manifest.package.name; + for (auto* o : { &stated.link.driver.origin, &stated.link.inputsOrigin, + &stated.link.exe.origin, &stated.link.shared.origin, + &stated.link.sealOrigin, &stated.runner.origin }) + if (o->tier == mcpp::manifest::platform::Tier::Program) + o->key = std::format("{} ({})", who, o->key); + if (!stated.empty() || !stated.runner.host.empty()) + mcpp::platform::merge(state.programPlatform, stated); + } } // apply() may have added interface defines to packages' publicUsage diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 47f86f384..462a41605 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -258,8 +258,6 @@ std::expected step4b_apply_platform(PrepareState& state) { g_platformRestart = P.root; return std::unexpected(std::string("mcpp:platform-restart")); } - mcpp::ui::info("Platform", std::format("{} ({})", P.display(), - mcpp::platform::origin_text(P.selection.origin))); } // ── The layers the graph lacks (§4.3) ───────────────────────────────── @@ -460,6 +458,9 @@ void step13_platform_statement(PrepareState& state) { // machine → the project → the environment (§4.5) ────────────────────── mp::Statement s = P.active ? state.platformData : mp::Statement{}; mp::Statement program = state.programPlatform; + // The root's own program states them as well (applied into its manifest); + // it is the last program to run, and it speaks for the project. + mcpp::platform::merge(program, state.m->buildConfig.programPlatform); // A build program's runner and named runners (`mcpp::runner`), the BSP // spelling of the same fields. if (!state.m->buildConfig.runner.empty()) { @@ -554,6 +555,16 @@ std::expected step13_finish_platform(PrepareState& state, Bui state.tc->binaryPath.generic_string(), P.fields.toolchain.set ? P.fields.toolchain.value : std::string("an llvm toolchain"))); } + // THE PLATFORM, REPORTED (SPEC-013 §9): which one and why, then what it + // contributes beyond the layers the Target lines already showed. + if (P.active) { + mcpp::ui::info("Platform", std::format("{} ({}{})", P.display(), + mcpp::platform::origin_text(P.selection.origin), + P.selection.hosted ? ", hosted" : "")); + for (auto const& [label, value] : mcpp::platform::report_rows(P)) + if (!mp::is_layer(label)) + mcpp::ui::info("", std::format("{:<17} {}", label, value)); + } ctx.plan.platform = std::move(P); return {}; } diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 7830d89e5..ca1098ee5 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -237,6 +237,10 @@ std::expected step13_lockfile(PrepareState& state, BuildConte // prefix claimed otherwise. `index_package_digest` is FNV-1a on // every host. lp.hash = mcpp::pm::index_package_digest(sourceIndex, lp.name, lp.version); + // The platform's packages say so (SPEC-013 §8.3). + if (state.platformInjected.contains(mcpp::pm::compat::qualified_name( + key.ns.empty() ? std::string(mcpp::pm::kDefaultNamespace) : key.ns, key.shortName))) + lp.origin = "platform"; // One entry per identity: a dependency declared by the root and by // a member under different spellings is one package. if (!versionLocked.insert(lock_identity(lp)).second) continue; diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index 598848fd2..0c54c60d7 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -1222,6 +1222,22 @@ step9_same_os_check_and_report(PrepareState& state) { reportedTargetName, reportedTargetName)); } } + // A layer whose package the platform added is the platform's in the + // report (SPEC-013 §9), whatever else the graph holds. + if (state.platform.active) { + auto injected = [&](std::string_view impl) { + const auto name = impl.substr(0, impl.find('@')); + for (auto const& q : state.platformInjected) { + const std::string_view qs = q; + const auto shortName = qs.substr(qs.find('.') == std::string_view::npos ? 0 : qs.find('.') + 1); + if (name == qs || name == shortName) return true; + } + return false; + }; + for (auto* l : { &state.resolvedTargetSide.compilerRuntime, &state.resolvedTargetSide.kernelAbi, + &state.resolvedTargetSide.cAbi, &state.resolvedTargetSide.cxx }) + if (l->fromGraph() && injected(l->impl)) l->byPlatform = true; + } mcpp::ui::info("Target", tsd::format_report( state.resolvedTargetSide, reportedTargetName, mcpp::log::is_verbose())); return {}; diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index f82df5dd2..a76729f4c 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -1572,12 +1572,21 @@ step2_resolve_explicit_spec(PrepareState& state, ToolchainResolveCtx& ctx) { ? std::string{} : std::format(", not your {} — this project only", state.graphCompilerReplaced)); + else if (!state.pinReplacedDefault.empty() && state.platformToolchainPin) + chosenBy = std::format( + "\n the platform's ({}), replacing your " + "{} — override with `[target.{}] toolchain`", + state.platform.display(), state.pinReplacedDefault, + state.overrides.target_triple.empty() ? std::string("") + : state.overrides.target_triple); else if (!state.pinReplacedDefault.empty()) chosenBy = std::format( "\n target default for {}, replacing your " "{} — override with `[target.{}] toolchain`", state.overrides.target_triple, state.pinReplacedDefault, state.overrides.target_triple); + else if (state.platformToolchainPin) + chosenBy = std::format(" (the platform's, {})", state.platform.display()); else if (state.tcOrigin == TcOrigin::TargetPin || state.tcOrigin == TcOrigin::FirstRun) chosenBy = std::format(" ({})", tc_origin_name(state.tcOrigin)); diff --git a/src/platform/platform.cppm b/src/platform/platform.cppm index cc6e2e868..1bcfa198e 100644 --- a/src/platform/platform.cppm +++ b/src/platform/platform.cppm @@ -304,18 +304,18 @@ void merge(mp::Statement& b, const mp::Statement& o) { words(b.targets, o.targets); text(b.toolchain, o.toolchain); if (o.sysroot.set) { b.sysroot = o.sysroot; b.sysrootProvides = o.sysrootProvides; } + // Field by field: an argv-only statement keeps the platform's `host` and + // `exclusive`; a table, or a program's `run-host`, states what it states. if (o.runner.set) { - // Field by field: an argv-only statement keeps the platform's `host` - // and `exclusive`; a table states what it states. b.runner.cmd = o.runner.cmd; - if (!o.runner.host.empty()) b.runner.host = o.runner.host; - if (o.runner.exclusiveSet) { - b.runner.exclusive = o.runner.exclusive; - b.runner.exclusiveSet = true; - } b.runner.origin = o.runner.origin; b.runner.set = true; } + if (!o.runner.host.empty()) b.runner.host = o.runner.host; + if (o.runner.exclusiveSet) { + b.runner.exclusive = o.runner.exclusive; + b.runner.exclusiveSet = true; + } for (auto const& [k, v] : o.runners) b.runners[k] = v; if (o.carrier.set) b.carrier = o.carrier; text(b.plugin, o.plugin); diff --git a/src/pm/lock_io.cppm b/src/pm/lock_io.cppm index be0b53239..a1a8dab8c 100644 --- a/src/pm/lock_io.cppm +++ b/src/pm/lock_io.cppm @@ -32,6 +32,12 @@ struct LockedPackage { std::string version; std::string source; // e.g. "index+mcpplibs@abc123def..." std::string hash; // "sha256:..." or "fnv1a:..." + // Who put the package in the graph when the project did not (SPEC-013 + // §8.3): "platform" for a package the target's platform added (the + // platform package itself, a layer's provider, a tool's package). Empty + // for a package the project declared or one reached through it; a lock + // written before the field reads as empty. + std::string origin; }; // Parsed form of a git source string as written to mcpp.lock. @@ -137,6 +143,7 @@ std::expected load(const std::filesystem::path& path) { if (auto it = tt.find("version"); it != tt.end() && it->second.is_string()) lp.version = it->second.as_string(); if (auto it = tt.find("source"); it != tt.end() && it->second.is_string()) lp.source = it->second.as_string(); if (auto it = tt.find("hash"); it != tt.end() && it->second.is_string()) lp.hash = it->second.as_string(); + if (auto it = tt.find("origin"); it != tt.end() && it->second.is_string()) lp.origin = it->second.as_string(); // v1 → v2 migration: default namespace to "mcpplibs". if (fileVersion < 2 && lp.namespace_.empty()) { @@ -187,7 +194,9 @@ std::string serialize(const Lockfile& lock) { } out += std::format("version = {}\n", t::escape_string(p.version)); out += std::format("source = {}\n", t::escape_string(p.source)); - out += std::format("hash = {}\n\n", t::escape_string(p.hash)); + out += std::format("hash = {}\n", t::escape_string(p.hash)); + if (!p.origin.empty()) out += std::format("origin = {}\n", t::escape_string(p.origin)); + out += "\n"; } return out; } diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index 1cfc561ce..affa04ff6 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -1062,16 +1062,55 @@ TEST(BuildDirectives, DeployRowIsProtocolElevenWithLinkGlobalScopeAndATag) { EXPECT_EQ(def->scope, dirs::Scope::LinkGlobal); EXPECT_EQ(def->sinceProtocol, 11); EXPECT_FALSE(def->tag.empty()); - EXPECT_EQ(dirs::kProtocolVersion, 16); + EXPECT_EQ(dirs::kProtocolVersion, 17); } -TEST(BuildDirectives, ProtocolSixteenIsAcceptedAndSeventeenIsNot) { - auto ok = parse("mcpp:protocol=16\n"); +TEST(BuildDirectives, ProtocolSeventeenIsAcceptedAndEighteenIsNot) { + auto ok = parse("mcpp:protocol=17\n"); EXPECT_FALSE(dirs::protocol_error(ok).has_value()); - auto no = parse("mcpp:protocol=17\n"); + auto no = parse("mcpp:protocol=18\n"); EXPECT_TRUE(dirs::protocol_error(no).has_value()); } +// SPEC-013 §4.1: the platform's fields a program states (protocol 17) become a +// platform statement at the build-program tier; malformed lines are refused. +TEST(BuildDirectives, PlatformFieldsAreProtocolSeventeenAndReachTheStatement) { + for (auto wire : {"link-driver", "link-input", "artifact-name", "seal", "run-host"}) { + auto def = dirs::find_by_wire(wire); + ASSERT_NE(def, nullptr) << wire; + EXPECT_EQ(def->sinceProtocol, 17) << wire; + EXPECT_FALSE(def->tag.empty()) << wire; + } + mcpp::manifest::Manifest m; + auto d = parse("mcpp:link-driver=lld\n" + "mcpp:link-input=from:notes.S:okld\n" + "mcpp:link-input=from:notes.S:note-source\n" + "mcpp:link-input=file:/opt/crt0.o\n" + "mcpp:artifact-name=exe:{name}.oke\n" + "mcpp:artifact-name=shared:lib{name}.okso\n" + "mcpp:seal=0:okld\nmcpp:seal=0:note\nmcpp:seal=0:{}\n" + "mcpp:seal=1:okld\nmcpp:seal=1:check\nmcpp:seal=1:{}\n" + "mcpp:run-host=same-arch\n"); + EXPECT_EQ(dirs::target_directive_error(m, d), ""); + dirs::apply(m, d); + auto const& s = m.buildConfig.programPlatform; + EXPECT_EQ(s.link.driver.value, "lld"); + ASSERT_EQ(s.link.inputs.size(), 2u); + EXPECT_EQ(s.link.inputs[0].as, "notes.S"); + EXPECT_EQ(s.link.inputs[0].from, (std::vector{"okld", "note-source"})); + EXPECT_EQ(s.link.inputs[1].file, "/opt/crt0.o"); + EXPECT_EQ(s.link.exe.value, "{name}.oke"); + EXPECT_EQ(s.link.shared.value, "lib{name}.okso"); + ASSERT_EQ(s.link.seal.size(), 2u); + EXPECT_EQ(s.link.seal[1], (std::vector{"okld", "check", "{}"})); + EXPECT_EQ(s.runner.host, "same-arch"); + EXPECT_EQ(s.link.driver.origin.tier, mcpp::manifest::platform::Tier::Program); + for (auto bad : {"mcpp:link-driver=gold\n", "mcpp:link-input=from:x\n", + "mcpp:artifact-name=exe:lib{name}\n", "mcpp:artifact-name=dll:{name}\n", + "mcpp:seal=x:okld\n", "mcpp:run-host=far\n"}) + EXPECT_NE(dirs::target_directive_error(m, parse(bad)), "") << bad; +} + // Design 2026-10-26 §24.3: a board's image entry symbol reaches the manifest's // link intent; a non-symbol and two different symbols are refused. TEST(BuildDirectives, LinkStartIsProtocolSixteenAndReachesTheLinkIntent) { From 9762ab887d4b1bf5c2fe3df269391f57718c46ba Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 03:50:05 +0900 Subject: [PATCH 07/54] platform: self env --target/--host reports one configuration; toolchain install --target installs it; toolchain list has a platform column; the plan event and the result record carry platform and hostPlatform - `mcpp self env --target [--host] [--format json]` prepares the configuration (the project's when run in one, an empty program in the home's cache otherwise) and reports the compiler and who chose it, the platform and its source, each field's source, the tools built for this machine, and the runner with the program it resolves to. - `mcpp toolchain install --target ` with no compiler named installs the toolchain, then the platform package, the packages it adds and its tools, so a build of works with --offline afterwards. - `mcpp toolchain list` shows each row's platform (JSON `platform`, null for the engine's conventions). - The `plan` event and result.json's `configuration` carry `platform` and `hostPlatform` objects (name, version, source, hosted, fields). - The commands that prepare a configuration live with the build commands: importing mcpp.build.prepare from cmd_self and cmd_toolchain made GCC 16.1 crash importing mcpp.cli again (#848). Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/build/execute.cppm | 13 +++- src/build/prepare.cppm | 15 +++++ src/build/prepare/configuration.cpp | 52 +++++++++++++++ src/build/prepare/plan.cpp | 4 +- src/build/result.cppm | 8 +++ src/cli.cppm | 28 +++++++- src/cli/cmd_build.cppm | 99 +++++++++++++++++++++++++++++ src/toolchain/lifecycle.cppm | 26 ++++++-- 8 files changed, 234 insertions(+), 11 deletions(-) create mode 100644 src/build/prepare/configuration.cpp diff --git a/src/build/execute.cppm b/src/build/execute.cppm index e0b42ebdc..188bf5b9e 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -22,6 +22,7 @@ import mcpp.build.plan; import mcpp.toolchain.triple; import mcpp.freestanding.runner; import mcpp.platform; +import mcpp.libs.json; // the plan event's platform objects import mcpp.build.runner_lookup; // #544: where the runner's program is import mcpp.home; // config.toml, for the machine's default toolchain import mcpp.libs.toml; @@ -1337,10 +1338,14 @@ mcpp::build::result::Record start_record(const BuildContext& ctx) { r.pid = mcpp::build::dirlock::process_id(); r.triple = ctx.tc.targetTriple; r.profile = ctx.profile; + r.platform = mcpp::platform::to_json(ctx.plan.platform); + r.hostPlatform = mcpp::platform::to_json(ctx.hostPlatform); mcpp::build::result::write(r); mcpp::build::result::emit("plan", {{"configuration", { {"triple", r.triple}, {"profile", r.profile}, {"fingerprint", r.fingerprint}, - {"outputDir", r.outputDir.generic_string()}}}}); + {"outputDir", r.outputDir.generic_string()}, + {"platform", nlohmann::json::parse(r.platform, nullptr, false)}, + {"hostPlatform", nlohmann::json::parse(r.hostPlatform, nullptr, false)}}}}); return r; } @@ -1381,12 +1386,16 @@ FastRecord fast_path_begin(const std::filesystem::path& projectRoot, if (previous) { f.record.triple = previous->triple; f.record.profile = previous->profile; + f.record.platform = previous->platform; + f.record.hostPlatform = previous->hostPlatform; f.artifacts = std::move(previous->artifacts); } res::write(f.record); res::emit("plan", {{"configuration", { {"triple", f.record.triple}, {"profile", f.record.profile}, - {"fingerprint", f.record.fingerprint}, {"outputDir", f.record.outputDir.generic_string()}}}}); + {"fingerprint", f.record.fingerprint}, {"outputDir", f.record.outputDir.generic_string()}, + {"platform", nlohmann::json::parse(f.record.platform, nullptr, false)}, + {"hostPlatform", nlohmann::json::parse(f.record.hostPlatform, nullptr, false)}}}}); return f; } void fast_path_end(FastRecord& f, bool ok) { diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 0f7cb7ab5..b50496dc2 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -87,6 +87,7 @@ import mcpp.toolchain.post_install; import mcpp.toolchain.abi; import mcpp.toolchain.triple; import mcpp.build.plan; +import mcpp.platform; import mcpp.build.flags; // compute_flags — the per-role contracts (#418) import mcpp.build.graph_shape; // #407: the graph says which mode wrote it import mcpp.build.build_program; @@ -524,6 +525,10 @@ export struct BuildContext { // where it was stated, for the line that announces it. std::vector runUnder; std::string runUnderOrigin; + // THE HOST GRAPH'S PLATFORM (SPEC-013 §6): what the build programs and the + // tools of this build are built for. Inactive = the host conventions. + // The target's is `plan.platform`. + mcpp::platform::Resolved hostPlatform; // The payload directory of every `[xlings]` address resolved above, for // the fast path's presence check (#716). See BuildCacheEntry::xlingsPayloads. std::vector xlingsPayloads; @@ -1040,6 +1045,16 @@ prepare_build(bool print_fingerprint, bool includeDevDeps = false, std::vector extraTargets = {}, BuildOverrides overrides = {}); +// ONE CONFIGURATION, PREPARED WITHOUT A BUILD (SPEC-013 §9): what `mcpp self +// env --target` reports and what `mcpp toolchain install --target` fetches -- +// the toolchain, the platform package, the packages it adds and the tools it +// builds for this machine -- so a later build of that target works offline. +// Inside a project the project's own configuration is prepared (its +// `[target.]` tables apply); elsewhere an empty program in the +// home's cache stands for one. Nothing is compiled but the tools. +export std::expected +prepare_configuration(std::string_view targetTriple); + // The PlanNotes a failed call recorded before the phase that failed it. // // On success, `prepare_build` copies `PrepareState::planNotes` into diff --git a/src/build/prepare/configuration.cpp b/src/build/prepare/configuration.cpp new file mode 100644 index 000000000..54c92c385 --- /dev/null +++ b/src/build/prepare/configuration.cpp @@ -0,0 +1,52 @@ +// configuration.cpp -- one configuration prepared without a build (SPEC-013 +// §9): `mcpp self env --target` and `mcpp toolchain install --target`. +// Declared in prepare.cppm. + +module mcpp.build.prepare; +import :state; + +import std; +import mcpp.home; +import mcpp.project; +import mcpp.toolchain.triple; + +namespace mcpp::build { + +namespace fs = std::filesystem; + +std::expected prepare_configuration(std::string_view targetTriple) { + BuildOverrides ov; + ov.target_triple = std::string(targetTriple); + std::error_code ec; + if (auto root = mcpp::project::find_manifest_root(fs::current_path(ec))) { + ov.project_root = *root; + return prepare_build(/*print_fingerprint=*/false, /*includeDevDeps=*/false, {}, ov); + } + // No project here: an empty program in the home's cache stands for one, + // one directory per target, so its lock and its build directory persist + // and a second run finds everything on disk. + std::string name = "host"; + if (!targetTriple.empty()) { + auto t = mcpp::toolchain::triple::parse(targetTriple); + name = t ? t->str() : std::string(targetTriple); + } + const auto dir = mcpp::home::root() / "cache" / "configurations" / name; + fs::create_directories(dir / "src", ec); + if (ec) return std::unexpected(std::format("cannot create {}: {}", dir.string(), ec.message())); + auto write_if_absent = [&](const fs::path& p, std::string_view text) { + if (fs::exists(p, ec)) return true; + std::ofstream out(p, std::ios::binary); + out << text; + return bool(out); + }; + if (!write_if_absent(dir / "mcpp.toml", + "# The configuration `mcpp self env --target` and `mcpp toolchain install\n" + "# --target` prepare when they are run outside a project (written by mcpp).\n" + "[package]\nname = \"configuration\"\nversion = \"0.0.0\"\n") + || !write_if_absent(dir / "src" / "main.cpp", "int main() { return 0; }\n")) + return std::unexpected(std::format("cannot write the configuration under {}", dir.string())); + ov.project_root = dir; + return prepare_build(/*print_fingerprint=*/false, /*includeDevDeps=*/false, {}, ov); +} + +} // namespace mcpp::build diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index a732b58f7..2b71e9912 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -2117,7 +2117,9 @@ std::expected phase13_finish(PrepareState& state) { ctx.profile = state.effectiveProfile; ctx.activeFeatureRequest = state.workspacePlan() ? state.requestedFeatures : state.overrides.features; - ctx.compilerChoice = { std::string(tc_origin_name(state.tcOrigin)), + ctx.compilerChoice = { state.platformToolchainPin && state.tcOrigin == TcOrigin::TargetPin + ? std::string("platform") + : std::string(tc_origin_name(state.tcOrigin)), state.graphCompilerRequiredBy, state.graphCompilerReplaced.empty() ? state.pinReplacedDefault : state.graphCompilerReplaced }; diff --git a/src/build/result.cppm b/src/build/result.cppm index bb8f5f838..3c272b5c7 100644 --- a/src/build/result.cppm +++ b/src/build/result.cppm @@ -79,6 +79,10 @@ struct Record { std::string triple; std::string profile; std::string fingerprint; + // The configuration's platforms (SPEC-013 §9): the target graph's and the + // host graph's, as `mcpp::platform::to_json` writes them; "null" for none. + std::string platform = "null"; + std::string hostPlatform = "null"; std::filesystem::path outputDir; std::filesystem::path projectRoot; std::filesystem::path compileDatabase; @@ -182,6 +186,8 @@ nlohmann::json to_json(const Record& r) { {"profile", r.profile}, {"fingerprint", r.fingerprint}, {"outputDir", r.outputDir.generic_string()}, + {"platform", nlohmann::json::parse(r.platform, nullptr, false)}, + {"hostPlatform", nlohmann::json::parse(r.hostPlatform, nullptr, false)}, }}, {"compileDatabase", r.compileDatabase.empty() ? nlohmann::json(nullptr) : nlohmann::json(r.compileDatabase.generic_string())}, @@ -208,6 +214,8 @@ std::optional read(const std::filesystem::path& outputDir) { r.profile = c.value("profile", ""); r.fingerprint = c.value("fingerprint", ""); r.outputDir = c.value("outputDir", ""); + if (c.contains("platform")) r.platform = c["platform"].dump(); + if (c.contains("hostPlatform")) r.hostPlatform = c["hostPlatform"].dump(); } if (d.contains("compileDatabase") && d["compileDatabase"].is_string()) r.compileDatabase = d["compileDatabase"].get(); diff --git a/src/cli.cppm b/src/cli.cppm index 10974da75..225039933 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -208,6 +208,24 @@ int enter_system_home(int argc, char** argv) { return 0; } +// `mcpp self env` with `--target` or `--host` reports one configuration +// (SPEC-013 §9), prepared by the build commands' module. +int cmd_env_any(const mcpplibs::cmdline::ParsedArgs& p) { + if (p.value("target") || p.is_flag_set("host")) return cmd_env_configuration(p); + return cmd_env(p); +} + +// `mcpp toolchain install --target ` with no compiler named installs the +// target's whole configuration once its toolchain is in (SPEC-013 §7). +int cmd_toolchain_any(const mcpplibs::cmdline::ParsedArgs& p) { + const int rc = cmd_toolchain(p); + if (rc != 0 || p.subcommand_name() != "install" || !p.subcommand_matches) return rc; + auto const& sub = *p.subcommand_matches; + auto target = sub.value("target"); + if (!target || target->empty() || !sub.positional(0).empty()) return rc; + return cmd_install_configuration(*target); +} + int run(int argc, char** argv) { namespace cl = mcpplibs::cmdline; ReportUnnarrowablePaths reportUnnarrowable_; @@ -1058,7 +1076,7 @@ int run(int argc, char** argv) { .arg(cl::Arg("version").help("(optional, alternative to @-form)")) .option(cl::Option("target").takes_value().help( "Remove the payload for instead of the host one"))) - .action(wrap_rc(cmd_toolchain))) + .action(wrap_rc(cmd_toolchain_any))) .subcommand(cl::App("cache") .description("Inspect and manage the global build cache") .subcommand(cl::App("dir") @@ -1161,7 +1179,11 @@ int run(int argc, char** argv) { .subcommand(cl::App("doctor") .description("Diagnose mcpp environment health")) .subcommand(cl::App("env") - .description("Print mcpp paths and configuration") + .description("Print mcpp paths and configuration; with --target, one configuration: toolchain, platform, tools, runner") + .option(cl::Option("target").takes_value().value_name("TRIPLE") + .help("Report this target's configuration (prepared, not built; the project's when run in one)")) + .option(cl::Option("host") + .help("Report the host graph's configuration: what build programs and tools are built for")) .option(cl::Option("format").takes_value().value_name("json") .help("Machine-readable output (enveloped; see docs/50-machine-output.md)"))) .subcommand(cl::App("config") @@ -1183,7 +1205,7 @@ int run(int argc, char** argv) { {"store", cmd_self_store}, {"init", cmd_self_init}, {"doctor", cmd_doctor}, - {"env", cmd_env}, + {"env", cmd_env_any}, {"config", cmd_self_config}, {"version", cmd_self_version}, {"explain", cmd_explain_action}, diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 25b5b46b0..07fc0a333 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -10,6 +10,7 @@ export module mcpp.cli.cmd_build; import std; import mcpp.platform; +import mcpp.build.runner_lookup; import mcpplibs.cmdline; import mcpp.build.prepare; import mcpp.build.execute; @@ -1586,6 +1587,104 @@ export int cmd_platform_step(const mcpplibs::cmdline::ParsedArgs& parsed, return 2; } +// `mcpp self env --target ` / `--host` (SPEC-013 §9): one configuration, +// prepared and not built -- the toolchain, the platform and its source, the +// tools it builds for this machine, the runner and where its program is. +export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { + bool json = false; + if (auto f = parsed.value("format")) { + if (!mcpp::wire::parse_format(*f)) { + std::println(stderr, "error: {}", mcpp::wire::unsupported_format(*f)); + return 2; + } + json = true; + } + const std::string target = parsed.value("target").value_or(""); + const bool host = parsed.is_flag_set("host"); + const bool wasQuiet = mcpp::ui::is_quiet(); + if (json) mcpp::ui::set_quiet(true); + auto ctx = mcpp::build::prepare_configuration(host ? std::string_view{} : std::string_view(target)); + mcpp::ui::set_quiet(wasQuiet); + if (!ctx) { + if (json) { + mcpp::wire::Envelope env{.kind = "mcpp.env", .effects = {}}; + env.data = nullptr; + env.diagnostics.push_back({"MCPP_CONFIGURATION_FAILED", mcpp::wire::Severity::Error, + ctx.error()}); + std::println("{}", mcpp::wire::to_json(env).dump()); + } else { + mcpp::ui::error(ctx.error()); + } + return 1; + } + auto const& P = host ? ctx->hostPlatform : ctx->plan.platform; + auto const& runner = ctx->plan.platform.fields.runner; + std::string runnerProgram; + if (!host && runner.set && !runner.cmd.empty()) { + auto found = mcpp::build::runner_lookup::locate( + runner.cmd.front(), ctx->xlingsDepBinDirs, + mcpp::invocation::get("PATH").value_or("")); + if (found.program) runnerProgram = found.program->generic_string(); + } + if (json) { + nlohmann::json tools = nlohmann::json::object(); + for (auto const& [name, path] : P.tools) tools[name] = path; + nlohmann::json data{ + {"triple", ctx->tc.targetTriple}, + {"graph", host ? "host" : "target"}, + {"toolchain", {{"compiler", ctx->tc.binaryPath.generic_string()}, + {"origin", ctx->compilerChoice.origin}}}, + {"platform", nlohmann::json::parse(mcpp::platform::to_json(P), nullptr, false)}, + {"tools", std::move(tools)}, + {"runner", host || !runner.set ? nlohmann::json(nullptr) : nlohmann::json{ + {"argv", runner.cmd}, {"host", runner.host}, {"exclusive", runner.exclusive}, + {"program", runnerProgram.empty() ? nlohmann::json(nullptr) : nlohmann::json(runnerProgram)}, + {"source", mcpp::platform::origin_text(runner.origin)}}}, + {"runUnder", ctx->runUnder}, + }; + mcpp::wire::emit({.kind = "mcpp.env", .effects = {}, .data = std::move(data)}); + return 0; + } + std::println("{:<10} {}", "target", ctx->tc.targetTriple); + std::println("{:<10} {} ({})", "compiler", ctx->tc.binaryPath.generic_string(), + ctx->compilerChoice.origin); + std::println("{:<10} {}{}", "platform", P.display(), + P.active ? std::format(" ({})", mcpp::platform::origin_text(P.selection.origin)) + : std::string()); + for (auto const& [label, value] : mcpp::platform::report_rows(P)) + std::println("{:<10} {:<17} {}", "", label, value); + for (auto const& [name, path] : P.tools) + std::println("{:<10} {:<17} {}", "tool", name, path); + if (!host && runner.set) { + std::string argv; + for (auto const& a : runner.cmd) argv += (argv.empty() ? "" : " ") + a; + std::println("{:<10} {} ({}){}", "runner", argv, mcpp::platform::origin_text(runner.origin), + runnerProgram.empty() ? std::string(" -- not found on this machine") + : " -> " + runnerProgram); + } + if (!ctx->runUnder.empty()) { + std::string argv; + for (auto const& a : ctx->runUnder) argv += (argv.empty() ? "" : " ") + a; + std::println("{:<10} {} ({})", "run_under", argv, ctx->runUnderOrigin); + } + return 0; +} + + +// `mcpp toolchain install --target ` with no compiler named installs THE +// CONFIGURATION (SPEC-013 §7): the toolchain (already installed by the +// caller), then the platform package, the packages the platform adds and the +// tools it builds for this machine, so a build of the target works with +// `--offline` afterwards. +export int cmd_install_configuration(std::string_view target) { + auto ctx = mcpp::build::prepare_configuration(target); + if (!ctx) { mcpp::ui::error(ctx.error()); return 1; } + if (ctx->plan.platform.active) + mcpp::ui::info("Installed", std::format("{} for {}: {} tool(s) built for this machine", + ctx->plan.platform.display(), target, ctx->plan.platform.tools.size())); + return 0; +} + export int cmd_stage(const mcpplibs::cmdline::ParsedArgs& parsed) { // `--list FILE` (#734 E4): many placements, one process. Each destination // keeps the single-file semantics below -- content comparison, an diff --git a/src/toolchain/lifecycle.cppm b/src/toolchain/lifecycle.cppm index a1809bf61..62e8c7b1c 100644 --- a/src/toolchain/lifecycle.cppm +++ b/src/toolchain/lifecycle.cppm @@ -648,6 +648,7 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, // So the convention travels in its own field, empty when there is // none. Same defect family as the release this was written for. std::string pin; // the target table's convention pin, or empty + std::string platform; // the target table's platform column (SPEC-013 §7), or empty std::string status; // installed | available | planned bool isDefault = false; int rank = 0; // display order: installed < available < planned @@ -691,7 +692,11 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, r.note = note_for(t); r.toolchain = tcLabel; for (auto const& info : mcpp::toolchain::triple::known_targets()) - if (info.canonical == r.target) { r.pin = std::string(info.pin); break; } + if (info.canonical == r.target) { + r.pin = std::string(info.pin); + r.platform = std::string(info.platform); + break; + } r.status = "installed"; r.rank = 0; r.isDefault = defSpec @@ -770,6 +775,7 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, if (auto at = pin.find('@'); at != std::string::npos) pin[at] = ' '; r.toolchain = pin.empty() ? "—" : pin; r.pin = std::string(info.pin); + r.platform = std::string(info.platform); // Three answers, not two. "available" means a payload here produces it; // "via dependency graph" means the compiler is here and the system has // to come from packages — a different thing to do next, so a different @@ -793,6 +799,9 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, {"note", r.note}, {"toolchain", r.toolchain}, {"pin", r.pin}, + // The platform a build of the target uses when nothing selects + // one (SPEC-013 §7); null: the engine's own conventions. + {"platform", r.platform.empty() ? nlohmann::json(nullptr) : nlohmann::json(r.platform)}, {"status", r.status}, {"default", r.isDefault}, }); @@ -811,12 +820,19 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, if (!targetRows.empty()) { std::println(""); std::println("Targets:"); - std::println(" {:<3}{:<24}{:<22}{:<18}{}", - "", "TARGET", "NOTE", "TOOLCHAIN", "STATUS"); + std::println(" {:<3}{:<24}{:<22}{:<18}{:<26}{}", + "", "TARGET", "NOTE", "TOOLCHAIN", "PLATFORM", "STATUS"); for (auto& r : targetRows) { - std::println(" {:<3}{:<24}{:<22}{:<18}{}", + // The platform package without its namespace, `name version`. + std::string platform = "—"; + if (!r.platform.empty()) { + platform = r.platform.substr(r.platform.find('.') == std::string::npos + ? 0 : r.platform.find('.') + 1); + if (auto at = platform.find('@'); at != std::string::npos) platform[at] = ' '; + } + std::println(" {:<3}{:<24}{:<22}{:<18}{:<26}{}", r.isDefault ? "*" : "", - r.target, r.note, r.toolchain, r.status); + r.target, r.note, r.toolchain, platform, r.status); } } From 1bb80f09153b1636b914c1b301906374454feea1 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 03:59:40 +0900 Subject: [PATCH 08/54] platform: the host graph's platform -- build programs built as its images and run by its runner; a workspace's members stand on the platform's layers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - [host] platform (mcpp.toml, config.toml), --host-platform and MCPP_HOST_PLATFORM select the host graph's platform in the outermost prepare. The build programs' graph is then built for the platform's target of this machine's architecture, by the platform's toolchain, and each program runs through the platform's runner (`okrun {}`); the runner is recorded beside the program so a current program runs the same way. The group's identity is the platform's, so changing it rebuilds elsewhere. - The bootstrap boundary: every nested prepare (tools, the programs' own build programs) uses the host conventions, so a platform's tools and runner are never built by the platform. - The layer providers the platform adds are declared by the root, and by each member of a workspace plan, as if written by hand: a member links its own closure (measured: build programs linked without musl and libc++). - The plan reports `Host platform …` when the build programs were built under one; the fast path declines when a platform flag, variable or the config's [host] changed. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/build/build_program.cppm | 16 ++- src/build/execute.cppm | 10 +- src/build/prepare.cppm | 4 + src/build/prepare/driver.cpp | 7 ++ src/build/prepare/features.cpp | 1 + src/build/prepare/graph.cpp | 1 + src/build/prepare/host_graph.cpp | 55 +++++++++- src/build/prepare/local_toolchain.cpp | 1 + src/build/prepare/platform.cpp | 140 ++++++++++++++++++++++++-- src/build/prepare/state.cppm | 21 ++++ src/build/prepare/target_side.cpp | 2 + 11 files changed, 246 insertions(+), 12 deletions(-) diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 89d795656..8a3e55e70 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -376,6 +376,10 @@ struct BuildProgramEnv { std::string programIdentity; std::chrono::milliseconds programCompile{0}; std::string programError; + // THE ARGV THAT RUNS IT, `{}` the program: the host graph's platform's + // runner when the program is one of its images (SPEC-013 §6, `okrun {}`); + // empty: the program is run directly. + std::vector programRunner; }; // The env-var name `hostprogram::xpkg_dir` reads back. One spelling of the @@ -1201,8 +1205,18 @@ std::expected run_build_program_impl( // the one who knows how long it takes. const auto deadline = dirs::run_timeout_for(m.buildConfig.buildProgramTimeoutSecs); + std::vector argv{bin.string()}; + if (!env.programRunner.empty()) { + argv.clear(); + bool placed = false; + for (auto const& a : env.programRunner) { + if (a == "{}") { argv.push_back(bin.string()); placed = true; } + else argv.push_back(a); + } + if (!placed) argv.push_back(bin.string()); + } auto rres = mcpp::run::capture_exec_deadline( - {bin.string()}, childEnv, deadline, &timedOut, root.string()); + argv, childEnv, deadline, &timedOut, root.string()); if (timedOut) { // Name the manifest to edit. Without this the user edits their OWN // mcpp.toml when a dependency's build program is what timed out, and diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 188bf5b9e..5f76f306d 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -160,7 +160,8 @@ std::string toolchain_request_identity() { const auto configFile = mcpp::home::root() / "config.toml"; auto target_table = [&](const std::filesystem::path& file) { auto doc = mcpp::libs::toml::parse_file(file); - if (!doc || !doc->get_table("target")) return; + // `[host]` too: the host graph's platform (SPEC-013 §6.1). + if (!doc || (!doc->get_table("target") && !doc->get_table("host"))) return; std::ifstream is(file, std::ios::binary); std::string text((std::istreambuf_iterator(is)), std::istreambuf_iterator()); std::uint64_t h = 1469598103934665603ull; @@ -176,9 +177,14 @@ std::string toolchain_request_identity() { target_table(mcpp::home::system_config_file()); std::string overrides; for (auto const& e : mcpp::invocation::with_prefix("MCPP_TARGET_")) - if (e.find("_SYSROOT=") != std::string::npos || e.find("_RUN_UNDER=") != std::string::npos) + if (e.find("_SYSROOT=") != std::string::npos || e.find("_RUN_UNDER=") != std::string::npos + || e.find("_PLATFORM=") != std::string::npos || e.find("_RUNNER=") != std::string::npos) overrides += e + ";"; if (auto s = mcpp::invocation::get("MCPP_SYSROOT")) overrides += std::format("MCPP_SYSROOT={};", *s); + // The platforms a flag or the environment selects (SPEC-013 §5.1, §6.1). + if (auto s = mcpp::invocation::get("MCPP_PLATFORM")) overrides += std::format("MCPP_PLATFORM={};", *s); + if (auto s = mcpp::invocation::get("MCPP_HOST_PLATFORM")) + overrides += std::format("MCPP_HOST_PLATFORM={};", *s); if (overrides.empty() && targets.empty()) return std::format("cli={};default={}", cli, machineDefault); return std::format("cli={};default={};base={}{}", cli, machineDefault, overrides, targets); diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index b50496dc2..54129fbcc 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -982,6 +982,10 @@ export struct BuildOverrides { // package and the second reads its `[platform]` from here at once // (SPEC-013 §7). Set by `prepare_build` itself, never by a caller. std::filesystem::path platform_root; + // THE PLATFORM THIS PREPARE'S TARGET USES, stated by the caller: the + // host graph's platform, given to the build programs' sub-build (SPEC-013 + // §6). Read as the flag tier; empty on every other prepare. + std::string platform_choice; }; // ── git dependency helpers ────────────────────────────────────────────────── diff --git a/src/build/prepare/driver.cpp b/src/build/prepare/driver.cpp index 6369d834c..72b27dc53 100644 --- a/src/build/prepare/driver.cpp +++ b/src/build/prepare/driver.cpp @@ -129,6 +129,13 @@ void read_build_programs_marker(BuildOverrides& ov) { if (eq == std::string::npos) continue; const auto key = line.substr(0, eq), value = line.substr(eq + 1); if (key == "toolchain" && ov.toolchain.empty()) ov.toolchain = value; + // A host graph with a platform (SPEC-013 §6): the programs are built + // for its target, by its choice. + else if (key == "platform" && ov.platform_choice.empty()) ov.platform_choice = value; + else if (key == "target" && ov.target_triple.empty()) { + ov.target_triple = value; + ov.host_target = false; + } else if (key == "profile" && ov.profile.empty()) ov.profile = value; else if (key == "index-store" && ov.project_index_root.empty()) ov.project_index_root = value; else if (key == "pin") diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 1244971ac..fc9bbadd2 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -1822,6 +1822,7 @@ static std::expected step6_dependency_build_programs(PrepareS if (!program && !state.overrides.plan_only) return std::unexpected(program.error()); if (program) { bpEnv.programBinary = program->path; + bpEnv.programRunner = program->runner; bpEnv.programIdentity = program->identity; bpEnv.programCompile = program->compile; } else { diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index 389b78ba6..b350787b8 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -2436,6 +2436,7 @@ std::expected phase4b_graph_worklist(PrepareState& state) { if (auto r = drain(); !r) return r; } if (auto r = step4b_check_platform_versions(state); !r) return r; + step4b_platform_member_edges(state); if (auto r = step4b_cycle_check(state); !r) return std::unexpected(r.error()); diff --git a/src/build/prepare/host_graph.cpp b/src/build/prepare/host_graph.cpp index 86056eeeb..48f56e6ce 100644 --- a/src/build/prepare/host_graph.cpp +++ b/src/build/prepare/host_graph.cpp @@ -210,12 +210,30 @@ std::string features_of(const PrepareState& state, std::size_t d) { struct HostToolchain { std::string spec; std::string identity; + // The host graph's platform (SPEC-013 §6): the spelling the sub-build is + // told it by and the target of this machine's architecture it serves. + // Empty: the build programs are this machine's programs. + std::string platform; + std::string platformTarget; }; std::expected host_toolchain(PrepareState& state) { + HostToolchain t; + // UNDER A HOST PLATFORM the build programs are images of this machine's + // architecture: the platform's toolchain compiles them and its runner + // runs them, the same on every system (§6.3). Their identity is the + // platform's, with no trace of this machine's compiler. + if (!state.hostPlatformSel.none) { + if (auto r = load_host_platform(state); !r) return std::unexpected(r.error()); + t.platform = host_platform_choice(state); + t.platformTarget = state.hostPlatformTarget; + t.spec = state.hostPlatform.fields.toolchain.value; + t.identity = std::format("platform {}|{}|{}", state.hostPlatform.identity(), + t.platformTarget, t.spec); + return t; + } auto host = state.host_tc_for_build_program(); if (!host) return std::unexpected(host.error()); - HostToolchain t; t.spec = state.host_spec_for_build_program(); t.identity = std::format("{}|{}|{}|{}", t.spec, path_text(host->first), host->second.version, host->second.targetTriple); @@ -606,6 +624,11 @@ void write_group(PrepareState& state, const fs::path& dir, const std::vector sub.target_triple = ""; sub.host_target = true; sub.toolchain = tc.spec; + // Under a host platform: the platform's target of this architecture, and + // its toolchain by the platform's own pin. + if (!tc.platform.empty()) { + sub.target_triple = tc.platformTarget; + sub.host_target = false; + sub.toolchain = ""; + sub.platform_choice = tc.platform; + } sub.profile = "build-program"; sub.cache_mode = state.overrides.cache_mode; sub.strict = state.overrides.strict; @@ -749,6 +780,20 @@ build_group(PrepareState& state, const fs::path& dir, const std::vector describe(*ctx); if (goals.empty()) return std::map{}; + // THE RUNNER, under a host platform: the platform's, with its program + // as that build located it (§6.1). A platform with none cannot run what + // it built, which is said here rather than at the first program's run. + if (!tc.platform.empty()) { + auto const& pr = ctx->plan.platform.fields.runner; + if (!pr.set || pr.cmd.empty()) + return std::unexpected(std::format( + "the host graph's platform {} states no runner, so the build programs it " + "builds cannot be run", ctx->plan.platform.display())); + std::vector argv = pr.cmd; + if (auto it = ctx->plan.platform.tools.find(argv.front()); it != ctx->plan.platform.tools.end()) + argv.front() = it->second; + state.hostPlatformRunner = std::move(argv); + } std::map binaries; std::vector ninjaGoals; for (auto const& lu : ctx->plan.linkUnits) { @@ -980,8 +1025,11 @@ ensure_build_programs(PrepareState& state, const std::vector& packa && fs::exists(it->value("binary", std::string{}), ec); if (current) { place_program_copy(state, p.index, fs::path(it->value("binary", std::string{}))); + std::vector runner; + if (it->contains("runner") && (*it)["runner"].is_array()) + for (auto const& a : (*it)["runner"]) runner.push_back(a.get()); state.buildProgramBinaries[p.index] = {fs::path(it->value("binary", std::string{})), - p.identity}; + p.identity, {}, std::move(runner)}; mcpp::log::verbose("build-program", std::format( "{}: identity {} current", qualified(manifest_of(state, p.index)), p.identity)); continue; @@ -1022,11 +1070,12 @@ ensure_build_programs(PrepareState& state, const std::vector& packa std::chrono::steady_clock::now() - began); for (auto const* p : stale) { const auto bin = built->at(p->member); - state.buildProgramBinaries[p->index] = {bin, p->identity, took}; + state.buildProgramBinaries[p->index] = {bin, p->identity, took, state.hostPlatformRunner}; place_program_copy(state, p->index, bin); nlohmann::json row; row["identity"] = p->identity; row["binary"] = path_text(bin); + if (!state.hostPlatformRunner.empty()) row["runner"] = state.hostPlatformRunner; row["package"] = qualified(manifest_of(state, p->index)); row["source"] = p->source.file.empty() ? std::string("synthesised") : path_text(p->source.file); diff --git a/src/build/prepare/local_toolchain.cpp b/src/build/prepare/local_toolchain.cpp index 4623c720d..fc9c9fd41 100644 --- a/src/build/prepare/local_toolchain.cpp +++ b/src/build/prepare/local_toolchain.cpp @@ -388,6 +388,7 @@ std::expected step6_toolchain_phase(PrepareState& state) { if (!program) return std::unexpected(std::format("the root build program's toolchain phase: {}", program.error())); bpEnv.programBinary = program->path; + bpEnv.programRunner = program->runner; bpEnv.programIdentity = program->identity; bpEnv.programCompile = program->compile; bpEnv.phase = "toolchain"; diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 462a41605..0085b9583 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -28,6 +28,7 @@ import mcpp.platform; import mcpp.pm.compat; import mcpp.targetside; import mcpp.toolchain.triple; +import mcpp.toolchain.platform; import mcpp.ui; import mcpp.freestanding.linkline; @@ -117,13 +118,27 @@ cfgpred::Ctx platform_cfg_context(const PrepareState& state) { std::expected step1_select_platform(PrepareState& state) { // A host sub-build (build programs, tools) is the host graph: its platform - // is the host conventions until a host platform is selected (§6), and a - // platform's own tools are always built so (the bootstrap boundary, §6.2). - if (state.overrides.host_target || state.overrides.tool_depth > 0 - || state.overrides.build_program_graph) - return {}; + // is the one its caller states (the build programs' graph under a host + // platform), or the host conventions -- and a platform's own tools are + // always built so (the bootstrap boundary, §6.2). + const bool nested = state.overrides.host_target || state.overrides.tool_depth > 0 + || state.overrides.build_program_graph; + if (nested && state.overrides.platform_choice.empty()) return {}; + // THE HOST GRAPH'S PLATFORM, chosen by the outermost prepare (§6.1). + if (!nested) { + mp::Override config; + if (auto cfg = state.get_cfg(false); cfg) config = (*cfg)->hostPlatform; + const auto hostFlag = mcpp::invocation::get("MCPP_HOST_PLATFORM").value_or(""); + mp::Override project = state.m->hostPlatform; + if (!project.platformSet && state.m->layer) project = state.m->layer->hostPlatform; + auto hs = mcpp::platform::select_host(hostFlag, false, project, config); + if (!hs) return std::unexpected(hs.error()); + state.hostPlatformSel = std::move(*hs); + } const auto t = target_of(state); - const auto flag = mcpp::invocation::get("MCPP_PLATFORM").value_or(""); + const auto flag = !state.overrides.platform_choice.empty() + ? state.overrides.platform_choice + : mcpp::invocation::get("MCPP_PLATFORM").value_or(""); const auto envKey = target_env_name(t, "PLATFORM"); const auto env = mcpp::invocation::get(envKey).value_or(""); auto project = project_tables(*state.m); @@ -313,6 +328,10 @@ std::expected step4b_apply_platform(PrepareState& state) { item.requestedBy = std::format("{} (platform: {})", state.m->package.name, names); item.originalConstraint = item.spec.version; item.consumerDepIndex = kMainConsumer; + // The root declares it, as it would by hand: every reader of the + // root's edges (the link's closure among them) sees it. + state.m->dependencies[key] = item.spec; + state.platformLayerSpecs[key] = item.spec; state.worklist.push_back(std::move(item)); state.platformInjected.insert("layer:" + key); state.platformInjected.insert(mcpp::pm::compat::qualified_name( @@ -404,6 +423,35 @@ std::expected step4b_check_platform_versions(PrepareState& st return {}; } +// A WORKSPACE'S MEMBERS STAND ON THE PLATFORM'S LAYERS AS A ROOT DOES (§4.3). +// The packages the platform added are the virtual root's dependencies, and a +// member links its own closure; so each member of the plan is given an edge to +// each layer provider the platform added, as if it had declared it. (Measured: +// a build programs' graph under a host platform linked its programs without +// the C library and the C++ run-time, and `okld check` refused `free`.) +void step4b_platform_member_edges(PrepareState& state) { + if (!state.platform.active || !state.workspacePlan() || state.platformLayerSpecs.empty()) return; + for (std::size_t m = 1; m < state.packages.size(); ++m) { + if (!state.packages[m].selectedMember) continue; + auto& deps = state.packages[m].manifest.dependencies; + for (auto const& [key, spec] : state.platformLayerSpecs) { + if (deps.contains(key)) continue; + deps[key] = spec; + for (std::size_t l = 1; l < state.packages.size(); ++l) { + auto const& pm = state.packages[l].manifest; + auto n = mcpp::pm::compat::resolve_package_name(pm.package.name, pm.package.namespace_); + if (n.shortName != spec.shortName) continue; + DependencyEdge e; + e.consumerPackageIndex = m; + e.dependencyPackageIndex = l; + e.requestedFeatures = spec.features; + e.defaultFeatures = spec.defaultFeatures; + state.dependencyEdges.push_back(std::move(e)); + } + } + } +} + void step5_platform_toolchain(PrepareState& state) { auto const& P = state.platform; if (!P.active || !P.fields.toolchain.set) return; @@ -566,7 +614,87 @@ std::expected step13_finish_platform(PrepareState& state, Bui mcpp::ui::info("", std::format("{:<17} {}", label, value)); } ctx.plan.platform = std::move(P); + // THE HOST GRAPH'S, when the build programs were built under it (§6.1). + ctx.hostPlatform = state.hostPlatform; + if (ctx.hostPlatform.active) + mcpp::ui::info("Host", std::format("platform {} ({}): build programs are {} images run by {}", + ctx.hostPlatform.display(), mcpp::platform::origin_text(ctx.hostPlatform.selection.origin), + state.hostPlatformTarget, + state.hostPlatformRunner.empty() ? std::string("its runner") + : std::filesystem::path(state.hostPlatformRunner.front()).filename().string())); return {}; } } // namespace mcpp::build + +namespace mcpp::build { + +// THE HOST GRAPH'S PLATFORM, LOADED (SPEC-013 §6): the package, and the +// target among its own that has this machine's architecture -- what the build +// programs are compiled for and run as. Read when the build programs are +// built; the package is fetched as any dependency is. +std::expected load_host_platform(PrepareState& state) { + auto& H = state.hostPlatform; + auto const& sel = state.hostPlatformSel; + if (sel.none || H.active) return {}; + const auto& ref = sel.ref; + fs::path root; + std::optional m; + if (!ref.spec.path.empty()) { + root = ref.spec.path; + auto r = mcpp::manifest::load(root / "mcpp.toml"); + if (!r) return std::unexpected(std::format("{}: the host platform {}: {}", + mcpp::platform::origin_text(sel.origin), mp::display(ref), r.error().format())); + m = std::move(*r); + } else if (!ref.spec.version.empty()) { + if (!state.loadVersionDep) + return std::unexpected("the host platform is read once the graph's loader exists"); + auto r = state.loadVersionDep(ref.name, ref.spec.namespace_, ref.spec.shortName, ref.spec.version); + if (!r) return std::unexpected(std::format("{}: the host platform {}: {}", + mcpp::platform::origin_text(sel.origin), mp::display(ref), r.error())); + root = r->first; + m = std::move(r->second); + } else { + return std::unexpected(std::format( + "{}: the host platform {} is named by a version or a directory", + mcpp::platform::origin_text(sel.origin), mp::display(ref))); + } + if (!m->platformDeclared) { + refusal::record(refusal::Code::PlatformNotAPlatform); + return std::unexpected(std::format( + "{} is selected as the host graph's platform ({}), and its mcpp.toml has no " + "[platform] section", qualified(*m), mcpp::platform::origin_text(sel.origin))); + } + const std::string arch(mcpp::toolchain::platform::host_arch); + for (auto const& w : m->platform.targets.value) { + auto t = mcpp::toolchain::triple::parse(w); + if (t && t->arch == arch) { state.hostPlatformTarget = t->str(); break; } + } + if (state.hostPlatformTarget.empty()) { + refusal::record(refusal::Code::PlatformTargetUnserved); + return std::unexpected(std::format( + "the host graph's platform {} ({}) serves no {} target, so this machine's build " + "programs cannot be built for it; its targets are {}", + qualified(*m), mcpp::platform::origin_text(sel.origin), arch, + [&] { std::string l; for (auto const& w : m->platform.targets.value) + l += (l.empty() ? "" : ", ") + w; return l; }())); + } + H.active = true; + H.selection = sel; + H.package = qualified(*m); + H.version = m->package.version; + H.root = root; + H.fields = m->platform; + return {}; +} + +// The spelling a sub-build is told the host platform by (`platform_choice`). +std::string host_platform_choice(const PrepareState& state) { + auto const& r = state.hostPlatformSel.ref; + if (!r.spec.path.empty()) return "path:" + r.spec.path; + return std::format("{}.{}@{}", r.spec.namespace_.empty() ? std::string(mcpp::pm::kDefaultNamespace) + : r.spec.namespace_, + r.spec.shortName, r.spec.version); +} + +} // namespace mcpp::build diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 6747b3f62..af91f4cb8 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -253,6 +253,9 @@ struct BuildProgramBinary { std::filesystem::path path; std::string identity; std::chrono::milliseconds compile{0}; + // The argv that runs it, `{}` the program, when the host graph has a + // platform (an ok image run by okrun, SPEC-013 §6); empty: run directly. + std::vector runner; }; // PrepareState carries prepare_build's working state across the phases it @@ -800,6 +803,9 @@ struct PrepareState { // `layer:` / `tool:` markers. A package listed here is locked // with `origin = "platform"` and reported as the platform's. std::set platformInjected; + // The layer providers the platform added, by the key the root declares + // them under: a workspace's members are given the same declarations. + std::map platformLayerSpecs; // The platform's toolchain is the pin of this build (P5), for the line // that announces the compiler. bool platformToolchainPin = false; @@ -809,6 +815,18 @@ struct PrepareState { // What build programs stated with the platform's directives (Tier::Program): // a dependency's, the root's, and the platform's plugin. mcpp::manifest::platform::Statement programPlatform; + // THE HOST GRAPH'S PLATFORM (SPEC-013 §6): selected in P1 of the + // outermost prepare only -- a nested one (tools, build programs) builds + // with the host conventions, which is the bootstrap boundary: a + // platform's own tools and runner are never built by the platform. + mcpp::platform::Selection hostPlatformSel; + // Loaded when the build programs are built: the package, its version and + // the target of this machine's architecture it serves. + mcpp::platform::Resolved hostPlatform; + std::string hostPlatformTarget; + // The runner the host graph's platform runs a build program with, its + // program located by the build programs' sub-build. + std::vector hostPlatformRunner; std::function foldDirectiveTailIntoPrivateBuild; @@ -1086,11 +1104,14 @@ std::expected step1_select_platform(PrepareState& state); void step4b_seed_platform(PrepareState& state); std::expected step4b_apply_platform(PrepareState& state); std::expected step4b_check_platform_versions(PrepareState& state); +void step4b_platform_member_edges(PrepareState& state); void step5_platform_toolchain(PrepareState& state); void step13_platform_statement(PrepareState& state); std::expected step13_finish_platform(PrepareState& state, BuildContext& ctx); // The platform root a first pass asks the second to read in P1, once. std::optional take_platform_restart(); +std::expected load_host_platform(PrepareState& state); +std::string host_platform_choice(const PrepareState& state); // The statement and bootstrap spec of a toolchain phase that asked prepare to // start again, taken once. std::optional, std::string>> take_toolchain_restart(); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index 0c54c60d7..22da6ce2d 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -1740,6 +1740,7 @@ static std::expected step9_root_build_program(PrepareState& s // The program, as the build programs' graph compiled it (mcpp#797). if (auto program = build_program_binary(state, 0); program) { bpEnv.programBinary = program->path; + bpEnv.programRunner = program->runner; bpEnv.programIdentity = program->identity; bpEnv.programCompile = program->compile; } else if (!state.overrides.plan_only) { @@ -2137,6 +2138,7 @@ static std::expected step9_member_build_programs(PrepareState // The program, as the build programs' graph compiled it (mcpp#797). if (auto program = build_program_binary(state, i); program) { bpEnv.programBinary = program->path; + bpEnv.programRunner = program->runner; bpEnv.programIdentity = program->identity; bpEnv.programCompile = program->compile; } else { From 104191e51ea577cf85606f579dd40219976c5ddf Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 04:10:32 +0900 Subject: [PATCH 09/54] platform: hosted targets; two versions of a header or layer package are refused; L4-L6; the std module of a macOS build over a graph C library takes the deployment floor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - [platform.hosted] targets names the host triples an arrangement is for; a platform selected for another (x86_64-windows-msvc under a C library of the graph) is refused naming the row that is served. - A multi-version rename separates named modules only: a package that exports a header directory or provides a target-side layer is refused at two versions, naming both requests (design §8.2). - L4: a version the index does not have is stated by mcpp -- which index, its revision and age, whether it was refreshed now and why not; the provider's own report is kept for -v. - L5: `mcpp test` builds the package's programs first (the pass `mcpp build` runs) and gives each test MCPP_BIN_. - L6: a package line is written once per command, not once per configuration. - The std module of a native macOS build whose C library is the graph's took the running system's version (std.pcm 15.0, units 14.0; measured by mcpplibs.os): it now takes the floor every unit takes. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- modules/manifest/src/platform.cppm | 6 ++- modules/manifest/src/platform_toml.cppm | 6 ++- src/build/execute.cppm | 45 +++++++++++++++++++++++ src/build/prepare/graph.cpp | 30 +++++++++++++++ src/build/prepare/graph_load.cpp | 49 ++++++++++++++++++++++--- src/build/prepare/scan.cpp | 18 +++++++++ src/build/progress.cppm | 10 ++++- src/platform/platform.cppm | 24 +++++++++++- 8 files changed, 178 insertions(+), 10 deletions(-) diff --git a/modules/manifest/src/platform.cppm b/modules/manifest/src/platform.cppm index b1f4e5d21..d4625676c 100644 --- a/modules/manifest/src/platform.cppm +++ b/modules/manifest/src/platform.cppm @@ -181,6 +181,10 @@ struct ToolEntry { // `[platform.hosted]`: the part that applies when a HOST triple selects this // platform (§5.4). struct Hosted { + // The host triples the arrangement is for (`x86_64-windows-gnu` and not + // `-msvc`: a C library of the graph needs the GNU ABI on Windows). Unset: + // every host triple. + Words targets; Text toolchain; std::map layers; bool set = false; @@ -253,7 +257,7 @@ inline constexpr std::string_view kBuildKeys[] = { "cflags", "cxxflags", "asmfla inline constexpr std::string_view kLinkKeys[] = { "driver", "flags", "start", "script", "inputs", "exe", "shared", "seal", }; -inline constexpr std::string_view kHostedKeys[] = { "toolchain", "layers" }; +inline constexpr std::string_view kHostedKeys[] = { "targets", "toolchain", "layers" }; // The `[target.]` keys a project or config.toml uses to select and // override a platform. `runner`, `runners`, `toolchain` and `sysroot` are the diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm index 6dd4b6acf..0e331ca39 100644 --- a/modules/manifest/src/platform_toml.cppm +++ b/modules/manifest/src/platform_toml.cppm @@ -545,7 +545,11 @@ std::expected Reader::statement(const toml::Table& t) co s.hosted.set = true; for (auto const& [hk, hv] : v.as_table()) { auto hkey = std::format("[platform.hosted].{}", hk); - if (hk == "toolchain") { + if (hk == "targets") { + auto w = strings(hv, hkey); + if (!w) return std::unexpected(w.error()); + s.hosted.targets = Words{ std::move(*w), at(hv, hkey), true }; + } else if (hk == "toolchain") { if (!hv.is_string() || hv.as_string().empty()) return std::unexpected(std::format("{} must be a toolchain", hkey)); s.hosted.toolchain = Text{ hv.as_string(), at(hv, hkey), true }; diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 5f76f306d..b753ee60e 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -3473,6 +3473,9 @@ struct TestBuild { bool bulkBuiltEverything = false; std::filesystem::path ninjaLog; std::uintmax_t logFrom = 0; + // MCPP_BIN_ for each program of the package, built before the + // tests (L5). + std::vector> programEnv; // How the test binaries are executed, resolved once for the build // (#544): every test of one build shares a target. @@ -4056,6 +4059,8 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, // A GL test that cannot find a driver fails the same way a GL program // does, so it must be told the same things. for (auto& kv : subosEnv) childEnv.push_back(kv); + // ...and where the package's own programs are (L5). + for (auto& kv : tb.programEnv) childEnv.push_back(kv); // ...and, through a runner, the files the test carries with it, as // `mcpp run` hands them over (see `runtime_files_for`). Written here, // in the single-threaded pass, one list per test program. @@ -4275,6 +4280,45 @@ export int run_tests(std::span passthrough, } return 0; } + // THE PACKAGE'S PROGRAMS, BUILT BEFORE ITS TESTS AND NAMED TO THEM (L5). + // A test that runs the program it tests finds it as MCPP_BIN_ (the + // target's name, upper-cased, every other character `_`), the way a Cargo + // integration test finds CARGO_BIN_EXE_. Built by the pass `mcpp + // build` runs -- its fast path when nothing changed -- so a program is + // never linked against the tests' dev-dependencies. Before this a package + // whose tests ran its own executable had to be built first by hand. + std::vector> programEnv; + { + bool hasPrograms = false; + if (auto m = mcpp::manifest::load(testRoot / "mcpp.toml")) { + hasPrograms = std::ranges::any_of(m->targets, [](auto const& t) { + return t.kind == mcpp::manifest::Target::Binary + || t.kind == mcpp::manifest::Target::Application; + }); + std::error_code ec; + if (m->targets.empty() && std::filesystem::exists(testRoot / "src" / "main.cpp", ec)) + hasPrograms = true; + } + if (hasPrograms) { + auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, {}, overrides); + if (!binCtx) { mcpp::ui::error(binCtx.error()); return 2; } + if (std::ranges::any_of(binCtx->plan.linkUnits, + [](auto const& lu) { return lu.kind == LinkUnit::Binary; })) { + if (int rc = run_build_plan(*binCtx, mcpp::log::is_verbose(), false); rc != 0) + return rc; + for (auto const& lu : binCtx->plan.linkUnits) { + if (lu.kind != LinkUnit::Binary) continue; + std::string key = "MCPP_BIN_"; + for (char c : lu.targetName) + key += (c >= 'a' && c <= 'z') ? char(c - 'a' + 'A') + : ((c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) ? c : '_'; + programEnv.emplace_back(std::move(key), + (binCtx->outputDir / lu.output).lexically_normal().string()); + } + } + } + } + // 3. prepare_build with dev-deps enabled + synthetic targets. // A test binary is executed, so the run tier applies here exactly as it // does to `mcpp run` — `[xlings.workspace]` has no separate `test` tier @@ -4286,6 +4330,7 @@ export int run_tests(std::span passthrough, std::move(overrides)); if (!prepared) { mcpp::ui::error(prepared.error()); return 2; } TestBuild tb; + tb.programEnv = std::move(programEnv); tb.ctx.emplace(std::move(*prepared)); tb.backend = mcpp::build::make_ninja_backend(); tb.ninjaLog = tb.ctx->outputDir / ".ninja_log"; diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index b350787b8..d732e5755 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -988,6 +988,36 @@ step4b_identity_version_merge(PrepareState& state, WorklistItemCtx& ctx, spec.version)); } + // THE RENAME'S BOUNDARY (design 2026-10-28 §8.2): it renames + // C++ modules and nothing else, so it separates two versions + // only when their whole interface is named modules. A package + // that exports a header directory, or provides a target-side + // layer, has C names and link symbols the rename cannot touch: + // two of them meet in one link, or in one include path, at a + // place that names neither (measured with two openkal + // releases in one graph). Refused here, naming both requests. + { + const bool headers = !secondaryManifest.buildConfig.includeDirs.empty(); + const bool layer = std::ranges::any_of(secondaryManifest.provides, + [](const std::string& c) { return c.starts_with("mcpp:"); }); + if (headers || layer) + return std::unexpected(std::format( + "dependency '{}{}{}' has irreconcilable versions:\n" + " '{}' requested by '{}'\n" + " '{}' requested by '{}'\n" + "Two versions cannot share one build: the package {}, and a " + "renamed copy keeps those names, so they would collide in the " + "link or in the include path. Align the requests on one version " + "(pin it in your mcpp.toml){}.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.version, it->second.requestedBy, + spec.version, item.requestedBy, + layer ? "provides a target-side layer" : "exports a header directory", + state.platform.active + ? std::format(", or select the release of {} that carries the " + "version you need", state.platform.display()) + : std::string())); + } // Module names are authored API and are not required to // mirror package identity. Discover every provided module // root from the secondary's source text, then rewrite the diff --git a/src/build/prepare/graph_load.cpp b/src/build/prepare/graph_load.cpp index 5ea2b49b1..2b9e5bb5d 100644 --- a/src/build/prepare/graph_load.cpp +++ b/src/build/prepare/graph_load.cpp @@ -1068,15 +1068,27 @@ step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { // `mcpp index update`). The policy decides (offline, `auto_refresh`, // the debounce, one sync per run). A project's own index is read as // it is, so it is not refreshed here. - if (r && r->exitCode != 0 && !ctx.useProjectEnv - && mcpp::pm::install_failure_policy(r->error ? r->error->code : std::string{}).refreshIndex) { + // What mcpp did about the miss, for the message if it stays one + // (L4): the refresh, and why it ran or did not. + std::string refreshNote; + const bool missedVersion = r && r->exitCode != 0 + && mcpp::pm::install_failure_policy(r->error ? r->error->code : std::string{}).refreshIndex; + if (missedVersion && !ctx.useProjectEnv) { const auto env = mcpp::config::make_xlings_env(**cfg); auto decision = mcpp::pm::decide_for_miss( mcpp::pm::policy_for(**cfg), env, target, mcpp::pm::RefreshReason::InstallMiss); - if (auto applied = mcpp::pm::apply(decision, env); !applied) + if (auto applied = mcpp::pm::apply(decision, env); !applied) { mcpp::ui::warning(applied.error()); - else if (decision.shouldRefresh) + refreshNote = std::format("refreshing it failed: {}", applied.error()); + } else if (decision.shouldRefresh) { r = install_one(target); + refreshNote = "refreshed it now, and the version is still not in it"; + } else { + refreshNote = std::format("did not refresh it: {}", + mcpp::pm::reason_text(decision.reason)); + } + } else if (missedVersion) { + refreshNote = "did not refresh it: a project's own index is read as it is"; } // The first address's failure is the one about the package; a // compat spelling tried after it fails as "not found" whatever @@ -1136,13 +1148,38 @@ step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { childErr += r->error->message; } auto target = std::format("{}@{}", depName, version); - auto diag = mcpp::pm::format_install_failure_diagnostic( - target, r->exitCode, indexRepos, childErr); std::string tried; for (auto& a : attempted) { if (!tried.empty()) tried += ", "; tried += a; } + // A VERSION THE INDEX DOES NOT HAVE IS SAID BY MCPP (L4): which + // index, at which revision and how old, and what was done about + // it. The provider's own sentence ("the index is current") is + // about its index, not mcpp's, and reads as the opposite of the + // cause; it stays available with -v. + if (missedVersion) { + std::string diag = std::format( + "the package index has no {} to install\n" + " index: {}\n" + " mcpp {}\n" + " if the version was published since, `mcpp index update` fetches it; " + "otherwise the version is not in the index", + target, ctx.useProjectEnv ? std::string("the project's own index") + : mcpp::pm::staleness_note(mcpp::config::make_xlings_env(**cfg)), + refreshNote.empty() ? std::string("did not refresh it") : refreshNote); + if (auto hint = mcpp::pm::unusable_index_hint(); !hint.empty()) diag += "\n" + hint; + if (mcpp::log::is_verbose()) + diag += "\n " + mcpp::pm::format_install_failure_diagnostic( + target, r->exitCode, indexRepos, childErr); + else if (!childErr.empty()) + diag += "\n (the provider's own report: run with -v)"; + diag += std::format("\n wire address{} tried: {}", + attempted.size() == 1 ? "" : "es", tried); + return std::unexpected(std::move(diag)); + } + auto diag = mcpp::pm::format_install_failure_diagnostic( + target, r->exitCode, indexRepos, childErr); diag += std::format("\n wire address{} tried: {}", attempted.size() == 1 ? "" : "es", tried); return std::unexpected(std::move(diag)); diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index 0dc8df73c..1de03af92 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -584,6 +584,24 @@ step11_package_std_module_source(PrepareState& state) { if (flags.find(q) == std::string::npos) flags += q; } } + // THE DEPLOYMENT FLOOR WHEN THE C LIBRARY IS THE GRAPH'S. Above, it + // comes with the prebuilt C library's tokens; a graph C library (the + // hosted arrangement: openkal-musl on macOS, SPEC-013 §5.4) has none, + // and the precompile then took the driver's default -- the running + // system's version -- while every unit was built for the floor + // (measured by mcpplibs.os: std.pcm for 15.0, the units for 14.0, and + // each import refused as a configuration mismatch). The value is the + // one every unit reads (flags.cppm), and a cross target carries it in + // its triple instead. + if (!state.tc->cAbiPrebuilt && state.tc->crossTargetFlag.empty()) { + auto tt = mcpp::toolchain::triple::parse(state.tc->targetTriple); + const auto floor = mcpp::toolchain::apple::deployment_target( + tt && tt->os == "macos", state.m->buildConfig.macosDeploymentTarget); + if (tt && tt->os == "macos" && !floor.empty()) { + const auto q = " -mmacosx-version-min=" + floor; + if (flags.find(q) == std::string::npos) flags += q; + } + } // The same words the package's own units receive from this list // (mcpp.manifest.flag_words), one quoted word each. for (auto& w : mcpp::manifest::flag_words(state.targetSideUsage.cxxflags)) diff --git a/src/build/progress.cppm b/src/build/progress.cppm index 77b637dd5..03e429f98 100644 --- a/src/build/progress.cppm +++ b/src/build/progress.cppm @@ -769,6 +769,10 @@ struct Report { std::vector started; // packages announced since the last frame long long lastFrame = 0; // ms since command start std::size_t lastDone = 0; + // The lines a package was announced with in this command (L6): a + // workspace builds several configurations, and a dependency they share + // was written once per configuration with the same words. + std::set announcedLines; }; Report& report() { @@ -888,7 +892,11 @@ void announce(Report& r, Build::Impl& b, std::size_t i, std::vector const auto& p = b.record->packages[i]; if (s.announced || p.name == "std" || p.name.empty()) return; s.announced = true; - out.push_back(package_line(r, b, i)); + auto line = package_line(r, b, i); + // One line per package and per command: the same package, with the same + // words, in a second configuration says nothing new. + if (!r.announcedLines.insert(line).second) return; + out.push_back(std::move(line)); r.started.push_back(p.cachedUnits > 0 ? screen::Source::Cache : source_of(p.source)); } diff --git a/src/platform/platform.cppm b/src/platform/platform.cppm index 1bcfa198e..2b3abee0b 100644 --- a/src/platform/platform.cppm +++ b/src/platform/platform.cppm @@ -373,7 +373,29 @@ serves(const mp::Statement& p, std::string_view package, const mcpp::toolchain:: } std::string list; for (auto const& w : p.targets.value) { if (!list.empty()) list += ", "; list += w; } - if (p.hosted.set) return true; + if (p.hosted.set) { + if (!p.hosted.targets.set) return true; + std::string hosts; + for (auto const& w : p.hosted.targets.value) { + auto pt = mcpp::toolchain::triple::parse(w); + if (pt ? pt->str() == t.str() : w == t.str()) return true; + if (!hosts.empty()) hosts += ", "; + hosts += w; + } + // The arrangement is for some host triples and not this one (a C + // library of the graph on Windows needs the GNU ABI): name the row of + // the same system and architecture it does serve. + std::string near; + for (auto const& w : p.hosted.targets.value) + if (auto pt = mcpp::toolchain::triple::parse(w); pt && pt->arch == t.arch && pt->os == t.os) + near = pt->str(); + return std::unexpected(std::format( + "platform {} hosts {} and not {} ([platform.hosted] targets).{}", + package, hosts, t.str(), + near.empty() ? std::string() + : std::format("\n Build for {} (--target {}, or [build] target = \"{}\")", + near, near, near))); + } return std::unexpected(std::format( "platform {} serves {} and not {}: it has no [platform.hosted] table, the part a host " "triple may take (SPEC-013 §5.4).\n" From 2d67dc6d1f24c09897dbeeff2267bc02cca96b9c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 05:04:09 +0900 Subject: [PATCH 10/54] platform: --pkg-format; a platform package's tests run per target; the version group is checked before each round; workspace roots carry the platform rows; self env reports a configuration whose sources would be refused - pack and run take --pkg-format (the earlier --format warns); e2e renamed - `mcpp test` on a platform package builds and runs its tests for each of its targets (the other architecture built, not run, under same-arch) - the version group is checked after each completion round, so a family version the project pinned is refused as the group's, not as a second version of a package the platform added - [target.] platform rows and [host] travel through the workspace layer and the virtual root, and enter root_position_key - self env / toolchain install take --platform; a source-level refusal is a warning of the configuration report (configuration_only) - the import-std remedy names the form that applies (layer or dependency) - progress test isolation; L6 unit test - part 2 design (packaging and distribution), main design rev 6 Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .agents/docs/2026-10-28-platforms-design.md | 43 +++- ...part2-packaging-and-distribution-design.md | 238 ++++++++++++++++++ modules/manifest/src/platform.cppm | 24 ++ src/build/execute.cppm | 10 +- src/build/prepare.cppm | 5 + src/build/prepare/configuration.cpp | 1 + src/build/prepare/graph.cpp | 5 + src/build/prepare/platform.cpp | 12 +- src/build/prepare/scan.cpp | 28 ++- src/build/prepare/target_side.cpp | 5 +- src/build/prepare/toolchain.cpp | 5 +- src/cli.cppm | 22 +- src/cli/cmd_build.cppm | 44 +++- src/cli/cmd_publish.cppm | 29 ++- src/pack/pack.cppm | 2 +- src/pack/pipeline.cppm | 20 +- src/project.cppm | 19 ++ tests/e2e/202_machine_output_contract.sh | 4 +- tests/e2e/257_shared_library_pe.sh | 2 +- tests/e2e/259_shared_library_macho.sh | 2 +- .../261_pack_follows_the_module_extension.sh | 4 +- tests/e2e/638_pack_format_dispatch.sh | 22 +- tests/e2e/650_wasm_row_names_its_launcher.sh | 6 +- ...build_program_deploys_what_it_generated.sh | 6 +- tests/e2e/652_an_application_is_a_kind.sh | 2 +- ...lication_on_android_is_a_shared_library.sh | 10 +- ...n_hands_the_distributable_to_the_runner.sh | 26 +- ...tages_declared_files_before_the_closure.sh | 4 +- ...a_universal_apk_is_two_legs_in_one_tree.sh | 4 +- ...aches_a_dispatched_format_with_its_tree.sh | 4 +- .../667_an_android_pack_stages_its_closure.sh | 12 +- ...acho_program_is_staged_with_its_closure.sh | 4 +- ...669_an_elf_host_pack_states_its_closure.sh | 6 +- ...chain_is_an_option_of_run_test_and_pack.sh | 2 +- ...receives_the_files_the_artifact_carries.sh | 4 +- ...mat_is_run_by_the_runner_named_after_it.sh | 16 +- tests/e2e/689_pack_takes_features.sh | 4 +- .../722_pack_strips_what_the_graph_built.sh | 12 +- .../e2e/723_pack_reports_what_it_produced.sh | 16 +- ...d_program_declares_a_runtime_search_dir.sh | 6 +- ..._finds_a_dll_through_runtime_search_dir.sh | 2 +- ...796_runtime_search_dir_on_pe_under_wine.sh | 2 +- ...he_llvm_row_defaults_to_the_dynamic_crt.sh | 4 +- ...ram_takes_its_runtime_from_one_resolver.sh | 2 +- tests/e2e/841_pack_summarizes_many_outputs.sh | 6 +- ...ers_plans_builds_and_runs_programs_once.sh | 8 +- ..._is_refused_before_anything_is_compiled.sh | 12 +- ...members_names_each_member_in_its_report.sh | 2 +- ...l_members_resolves_and_fails_per_member.sh | 6 +- ...ild_and_every_drive_takes_the_job_count.sh | 8 +- ...ok_image_is_built_once_and_run_by_okrun.sh | 55 ++-- .../951_an_ok_image_needs_no_declaration.sh | 138 ++++++++++ tests/unit/test_build_progress.cpp | 29 ++- 53 files changed, 766 insertions(+), 198 deletions(-) create mode 100644 .agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md create mode 100755 tests/e2e/951_an_ok_image_needs_no_declaration.sh diff --git a/.agents/docs/2026-10-28-platforms-design.md b/.agents/docs/2026-10-28-platforms-design.md index 1285bbb49..88f95ae03 100644 --- a/.agents/docs/2026-10-28-platforms-design.md +++ b/.agents/docs/2026-10-28-platforms-design.md @@ -22,6 +22,7 @@ mcpp-vscode 0.8.0；mcpp-plugins [#51](https://github.com/mcpp-community/mcpp-pl #846 并入 #847；每仓库一个 PR、commit 作检查点的交付计划（§13）；多视角自审（§15）。 - **rev 4（本稿）**：第三轮评审——§16 的 1–7 定案；mcpp-clion 以插件依赖的形式依赖 mcppls 的 CLion 插件，语言功能只由 mcppls 提供 （§11.2）；VS Code 与 JetBrains Marketplace 的发布由维护者上传，交付物为产物路径（§13.3）。§15 按定案重审，§16 改为本轮待确认项。 +- **rev 6**：补充设计 part 2（打包与分发、okb、`.okso` 本机绑定、体积、openkal 0.17.2 轮）定案，以 mcppls 与 mcpp-ide 交叉验证（§17）。 - **rev 5**：§16 定案（VS Code 以 `extensionPack`、CLion 以 `` 引入 mcppls；上传交付物为本机验证后的产物目录）；开始实施。 实施中发现两处缺口，列入同批：os 0.2.0 在宿主三元组上以 openkal-musl 为 C 库时不能编译（L9，os 0.2.1）；M3 载体的链接取到宿主 GCC 的 `crtbeginT.o`（L10，§4.7）。 @@ -88,8 +89,8 @@ toolchain = "" # 约定的工具链 sysroot = … # 预构建的目标基（可选；同 [target].sysroot） runner = ["", "{}"] # 或 { cmd = […], host = "same-arch" | "any", exclusive = bool } runners = { = [ … ] } # 命名的设备动作（flash、monitor…） -carrier = { package = "", section = "<节名>" } # M3 载体（可选） -plugin = "<模块>" # 平台阶段的插件代码（可选） +carrier = { package = "", section = "<节名>", main = "<入口源码>" } # M3 载体（可选） +plugin = "build.mcpp" # 平台包的构建程序，在平台阶段运行（可选） [platform.layers] = <依赖规格> | false [platform.tools] = <依赖规格> | { program = "<名字>" } @@ -101,7 +102,7 @@ plugin = "<模块>" # 平台阶段的插件代码（ exe = "{name}."; shared = "lib{name}." seal = [ [argv…], … ] [platform.versions] <包> = "<版本>" # 一组相互一致的版本（§8.1） -[platform.hosted] toolchain、layers # 宿主三元组选用本平台时适用的子集（§5.4） +[platform.hosted] targets、toolchain、layers # 宿主三元组选用本平台时适用的子集（§5.4）；targets 限定可用的宿主三元组 ``` 相对 rev 2 的精简：`platform-spec`、`requires-mcpp` 删去，改用 `[package] mcpp`（R9.8，平台规范版本随引擎版本）；`round` / @@ -146,8 +147,10 @@ plugin = "<模块>" # 平台阶段的插件代码（ ### 4.7 载体（M3） -`mcpp pack --mode self-contained` 为本机构建 `carrier.package`，以 `.incbin` 把产物放入其 `section` 节（ELF、Mach-O、COFF 三种汇编由引擎生成）。 -取代 `pack.cppm` 中按包名找 `openkal-tools` 的代码；载体版本写入锁。 +`mcpp pack --mode self-contained` 为本机构建 `carrier.package`，以 `.incbin` 把产物放入其 `section` 节（ELF、Mach-O、COFF 三种汇编由引擎生成）， +并给出符号 `mcpp_carried_begin`、`mcpp_carried_end`、`mcpp_carried_name`；程序入口是平台包的 `carrier.main`（引擎不含任何加载器的调用）。 +取代 `pack.cppm` 中按包名找 `openkal-tools` 的代码。载体按宿主约定构建（自举边界）：Linux 上为 `-linux-musl` 行并显式写出该行的工具链 +（载体的图提供 kernel-abi，否则会取消行的约定，L10）。 ## 5. 目标图：选择与覆盖 @@ -282,7 +285,7 @@ mcpp = ">=<本批 mcpp 版本>" targets = ["x86_64-openkal", "aarch64-openkal"] toolchain = "llvm@23.1.3" runner = { cmd = ["okrun", "{}"], host = "same-arch" } -carrier = { package = "openkal-run@0.1.1", section = ".oke" } +carrier = { package = "openkal-run@0.1.1", section = ".oke", main = "carrier/main.cpp" } [platform.layers] compiler-runtime = "openkal-llvm-runtime@0.16.3" @@ -307,12 +310,13 @@ flags = ["-shared", "-z", "max-page-size=65536", "-z", "now", "-z", "noexecstac inputs = [{ from = ["okld", "note-source"], as = "ok-notes.S" }] exe = "{name}.oke" shared = "lib{name}.okso" -seal = [["okld", "note", "{}"], ["okld", "check", "{}"]] +seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] [platform.hosted] +targets = ["x86_64-linux-gnu", "aarch64-linux-gnu", "x86_64-linux-musl", "aarch64-linux-musl", + "x86_64-macos", "aarch64-macos", "x86_64-windows-gnu", "x86_64-windows-musl"] toolchain = "llvm@23.1.3" -layers = { compiler-runtime = "openkal-llvm-runtime@0.16.3", c++-abi = "openkal-llvm-runtime@0.16.3", - c-abi = { name = "openkal-musl", version = "0.22.3", default-features = false, features = ["posix"] } } +[platform.hosted.layers] # 与 [platform.layers] 相同 [platform.versions] openkal = "0.17.1" @@ -424,7 +428,7 @@ docs/rules.md 增加平台插件一节；CI 使用本批 mcpp。 |---|---| | 引擎的 openkal 代码 | 删除 `src/okimage/`；六处分支改为平台机制的通用调用 | | 目标表 | 增加目标平台列与宿主平台列；`toolchain list` 显示 | -| 板级支持包 | docs/34 改写为"编写平台包"；`examples/13-platform-targets`、e2e 332、336、338 改为平台写法；333 保留指令写法作兼容判据 | +| 板级支持包 | docs/34 改写为"编写平台包"（板级支持包即平台包，指令写法同义保留）；e2e 952 以一个与 openkal 无关的本地平台包覆盖标志、链接脚本、命名、封装与运行器；332、336、338（目标行的启动）与 333（指令写法）不变，作兼容判据 | | 清单 | `[target.]` 增加 `platform`、`layers.*`、`tools.*`、`link.*`、`carrier`；`[host]`；`cfg(platform = …)`；SPEC-004 已知键表同步 | | 引擎接口 | `mcpp.core` 协议 17：`platform` 阶段与 §4.1 的新指令 | | 文档 | 新增 docs/35「平台」与 SPEC-013；改写 docs/24、34；增补 docs/04、22、30、41、50；中英文同步 | @@ -555,3 +559,22 @@ mcpp-ide：后端单元测试与 golden fixture；两端适配层的冒烟测试 1. mcpp-clion 以 `` 强依赖 mcppls 的 CLion 插件；VS Code 适配层以 `extensionPack` 引入 mcppls 的扩展（一并安装、可卸载）；后端握手检查 mcppls 的版本。 2. 两个 Marketplace 由维护者上传；交付物是本机验证后的产物目录；插件 ID `io.github.mcpp-community.mcpp-clion`。 3. 按 §13 实施；L9、L10 同批解决。 + +## 17. 补充设计：part 2（打包与分发） + +第四、五轮讨论把打包与分发、`.okso` 的本机绑定、体积与新增的 openkal 0.17.2 轮定为本批内容，写在 +[part 2](2026-10-28-platforms-part2-packaging-and-distribution-design.md)。要点： + +- `mcpp pack` 的选择拆为链接（`--linkage`）、闭包（`--mode`）、容器（`--pkg-format`）三个正交维度与构建腿；`--mode static` 是简写； + `--pkg-format` 取代 pack、run 的 `--format`（旧写法提示）。 +- 平台以 `universal = true` 声明其目标默认打通用包：一个包含全部架构，解压即可在任何系统、任何平台架构上运行；`--systems`、`--merge` + 产出免安装的多系统包。 +- 单文件是通用容器（原生：启动桩加载荷；openkal：载体从内存加载）；okb 是 openkal 的通用映像包，格式由 openkal-image 定义，由 + openkal-platform 的插件作为包格式提供。 +- 新规范 SPEC-014（打包与分发）；体积在构建、剥离、容器、去重、共享五处优化并实测。 +- `.okso` 的本机绑定由 openkal 体系的构建插件 `openkal-bind` 完成；openkal 0.17.2 轮（openkal-image 0.1.2、openkal-musl 0.22.4、 + openkal-llvm-runtime 0.16.4、openkal-platform 0.17.2）在本批最后；APE 单文件为后续（mcpp#849）。 +- 定案（part 2 §14）：`[platform] universal`；okb 由平台插件作为包格式提供；`tar` 默认 gzip、zstd 显式选择；openkal-platform 0.17.1 先行、 + 0.17.2 为最终钉住。 +- 交叉验证（part 2 §15）：mcppls 与 mcpp-ide 逐项采用零声明目标、通用包、okb、单文件、xlings 单资产、宿主图平台与体积优化—— + 编辑器扩展只带一个 okb 与各系统 × 架构的 okrun，一个 vsix 服务全部平台；判据写入两个 CI。 diff --git a/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md b/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md new file mode 100644 index 000000000..a249004c6 --- /dev/null +++ b/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md @@ -0,0 +1,238 @@ +--- +subject: platform +status: accepted +--- + +# 平台 part 2：打包与分发——三维模型、通用包、单文件、okb、体积，以及 openkal 0.17.2 轮 + +日期：2026-10-28。前文：[platforms design](2026-10-28-platforms-design.md)（SPEC-013，平台机制）。本文是它的补充，覆盖第四、五轮讨论 +（2026-10-28）定下的打包与分发、`.okso` 的本机绑定、体积，以及因此新增的 openkal 0.17.2 轮。相关规范：SPEC-007（构建插件）、 +SPEC-012（输出）、SPEC-013（平台）、新增 SPEC-014（打包与分发）。 + +**修订记录** + +- rev 1：定案——三维模型与 `--pkg-format`；okb v1 由 openkal-image 定义；体积优化本批实施并实测；`.okso` 本机绑定由 + openkal 体系的构建插件完成；APE 单文件记为后续（mcpp#849）。 +- rev 2（本稿）：§14 的四项定案；新增 §15，以 mcppls 与 mcpp-ide 交叉验证 openkal 体系的独有特性。 + +## 0. 摘要 + +`mcpp pack` 的选择拆成三个正交维度，各自回答一个问题：**链接**（`--linkage`，程序属性：库在程序里还是在旁边）、**闭包** +（`--mode`，打包属性：运行的机器要提供什么）、**容器**（`--pkg-format`，打包属性：交付成什么），外加**构建腿**（`--target`，可重复）。 +一个平台可以声明其目标的包默认是**通用**的：一个包含全部架构，解压即可在任何系统、任何该平台架构上运行。单文件是通用的容器 +（原生目标为启动桩加载荷，openkal 为载体从内存加载）；okb 是 openkal 的通用映像包，由 openkal-image 定义格式、由 openkal-platform 的 +插件作为包格式提供，引擎不含 openkal 知识。体积在构建、剥离、容器、去重、共享五处优化，本批实测后写入。openkal 侧新增一轮 0.17.2： +okb 与 okrun 的 bundle 和库接口、`.okso` 本机绑定插件、openkal-musl 的 renameat2 修正。 + +## 1. 问题（讨论中确认） + +| 编号 | 观察 | +|---|---| +| Q1 | `--format` 在报告类命令里是输出编码（`self env --format json`），在 `pack`、`run` 里是包格式；同名异义。 | +| Q2 | `--mode static` 把"程序全静态"（链接）与"机器什么都不用提供"（闭包）合在一个词里；openkal 有静态（一个 `.oke`）与共享（`.oke` 加 `.okso`）之分，却被说成"不适用"。 | +| Q3 | openkal 的价值——一份二进制对应"架构"而不是"系统 × 架构"——在包里没有体现：M2 是单架构的目录，M3 是单系统单架构的程序。 | +| Q4 | 没有单文件交付；用户需要自己解压。 | +| Q5 | 打包已近九千行，有包格式提供者（构建插件）的机制，却没有一份规范说明模型、阶段、暂存树与接口。 | +| Q6 | 体积：`--gc-sections` 已有而没有 `-ffunction-sections/-fdata-sections`，容器只有 gzip，资源按腿重复。 | +| Q7 | 一个 C 库要为每个系统各编一份原生库；`.okso` 只能被 ok 映像使用。 | +| Q8 | openkal-musl 的 `renameat2` 忽略 `RENAME_EXCHANGE`，第二个条目被覆盖（数据丢失；mcpplibs.os 0.2.1 的审计发现）。 | + +## 2. 模型：三个维度与构建腿 + +| 维度 | 选项 | 归属 | 原生目标 | openkal 目标 | +|---|---|---|---|---| +| 链接 `--linkage` | `static` / `shared` | 程序属性；对应 `[build] dependency_linkage`，打包时按它重建 | Linux 上 `static` 还选 `-linux-musl` 行（全静态，今天的 `--mode static` 行为） | `static`：一个 `.oke`，无 `.okso`；`shared`：`.oke` 加 `.okso` | +| 闭包 `--mode` | `system` / `vendored` / `self-contained` | 打包属性：机器提供什么 | 系统库 / 加非系统库 / 加 C、C++ 运行时 | 机器有加载器与 `.okso` / 带 `.okso` / 再带加载器（载体） | +| 容器 `--pkg-format` | `dir` `tar` `zip` `single-file` 与提供者的格式（`okb`、`appimage`、`msi`…） | 打包属性：交付成什么 | `single-file`：启动桩加压缩载荷 | `single-file`：载体从内存加载；`okb`：通用映像包 | +| 腿 `--target`（可重复） | 三元组 | 一次打包覆盖哪些目标 | 每条一份产物，或合并为一棵树（已有：APK 的 universal） | 平台声明 `universal` 时默认覆盖平台的全部 `targets` | + +- `--mode static` 保留为简写：`--linkage static --mode self-contained`。对原生目标与今天的行为相同；对 openkal 目标是"一个静态 `.oke` 加载体"。 +- 组合由引擎在打包开始前检查，不可能的组合拒绝并给出出路（例如 `--mode system --pkg-format single-file`：单文件必须自带它所运行的 + 东西）。 +- **`--pkg-format`**：`pack` 与 `run` 的包格式选项改名为 `--pkg-format`；旧的 `--format` 继续接受并提示是旧写法。全局用法规则：构建类命令 + （build、test、run、pack）的机器输出是 `--message-format`；报告类命令的 `--format` 是输出编码；包格式只叫 `--pkg-format`。 +- `[pack]` 增加 `linkage`、`format`（包格式的项目默认）、`compress`；`default_mode` 照旧。 + +## 3. 通用包（openkal 的默认） + +平台在 `[platform]` 中写 `universal = true`（openkal-platform 写），其任一目标的 `mcpp pack` 默认覆盖平台的全部 `targets`；`--no-universal` +只打所选目标。布局（`dir`、`tar`、`zip` 相同）： + +``` +hello-0.1.0-openkal/ 包名带平台名，不带系统与架构 + bin/hello POSIX 启动器 + bin/hello.cmd Windows 启动器 + lib/x86_64/hello.oke …okso 每个架构一组映像 + lib/aarch64/hello.oke …okso + libexec/okrun/-/okrun[.exe] 仅 --mode self-contained + share/… 资源只放一份 + mcpp-package.json SPEC-014 的包元数据 +``` + +- **启动器由引擎生成**，内容只来自平台的 `runner` 与腿：按 `uname -m` / `%PROCESSOR_ARCHITECTURE%` 选架构目录（映射是引擎的通用知识： + `x86_64|amd64|AMD64 → x86_64`，`aarch64|arm64|ARM64 → aarch64`），按 `runner.cmd` 起动；`self-contained` 时先找包内 + `libexec/okrun/-/`，再找 PATH。 +- **self-contained 的多系统加载器**：`--systems linux,macos,windows`（默认本机系统）为每个系统 × 架构交叉构建载体的加载器程序。缺少某个 + 系统的 SDK 载荷时拒绝并点名；CI 矩阵可各自产出后用 `mcpp pack --merge a.zip b.zip …` 按 `mcpp-package.json` 合并为一棵树（腿与系统取并集， + 同名映像的摘要必须相同）。 +- 结果：同一个解压目录在任何系统、任何平台架构上可运行（vendored 需要机器上有 okrun；self-contained 什么都不需要）。 + +## 4. 单文件（通用） + +| 组合 | 产物 | 运行 | +|---|---|---| +| 原生 `vendored`/`self-contained` + `single-file` | 引擎的启动桩 + zstd 压缩的暂存树 | 首次运行把载荷解到用户缓存（按载荷摘要寻址，原子改名），之后直接 exec；缓存根：`$XDG_CACHE_HOME`/`~/.cache`、`~/Library/Caches`、`%LOCALAPPDATA%` | +| 原生 `static` + `single-file`（Linux） | 静态启动桩 + 静态程序 | 同上，零依赖 | +| openkal `self-contained` + `single-file` | 平台的载体 + 本架构的映像组（okb 切片） | 载体经 openkal-run 的 bundle 接口从内存加载，不写盘；Linux 上载体静态，零依赖 | +| openkal `vendored` + `okb` | 通用映像包（§5） | `okrun app.okb`，或 xlings 的 shim；任何系统、任何架构 | + +- 启动桩的源码随引擎提供（同 engine package 的机制），打包时为目标构建，基于 mcpplibs.os；它只做"校验、解压、exec"。 +- 载体从内存加载需要 openkal-run 的 bundle 接口（§8）；载体的入口源码是平台的（`carrier.main`），引擎只把字节放进节、给出 + `mcpp_carried_*` 符号。 +- 一个文件在所有系统、所有架构上运行（APE，Cosmopolitan）不在本批：需要基于 Cosmopolitan 的 openkal 实现；记于 mcpp#849。 + +## 5. okb v1（openkal-image 定义） + +格式规范写在 openkal-image 的 `docs/okb.md`，是 okrun 的输入格式，不进入 openkal 核心规范（SPEC 12），因此不引起实现与 musl 的级联。 + +- **容器**：标准 ZIP（允许 zip64），任何 zip 工具可查看。 +- **映像**：以 stored 存放，数据起点按 64 KiB 对齐（与映像的 `max-page-size=65536` 一致，用额外字段补齐，同 zipalign），okrun 直接 mmap 原文件 + 的段，不复制、可跨进程共享页。 +- **v1 只含映像与清单**。程序在运行时读取的数据文件随通用包交付；映像从 okb 内读取资源需要加载器提供的文件视图，属于格式的后续版本， + v1 的清单以 `okb` 版本号为将来留出位置。 +- **清单 `okb.json`**：`okb`（版本 1）、`name`、`version`、`entry`、每个架构的目录与入口映像、每个条目的 sha256。 +- **解析顺序**：本架构目录 → 包内 `lib/` → okrun 的搜索路径（含 xlings 的包存储，§7）。 +- **信任**：每个映像各自 `okld sign`；清单可签名，签名覆盖全部条目摘要；okrun 的 `require_signature` 对整个 okb 生效。 +- **工具**：`okld bundle …` 写一个 okb（格式的拥有者提供写入器），`okld bundle check` 校验对齐、摘要与签名。 +- **由平台作为包格式提供**：openkal-platform 的插件（`plugin = "build.mcpp"`，平台阶段）以已有的包格式机制（SPEC-007 `pack-format`、 + `role = "artifact"` 的 action，第二遍读暂存树）声明 `okb`，动作调用 `okld bundle`。引擎不认识 okb；平台阶段的指令表因此加入 + `pack-format` 与 artifact action。 + +## 6. SPEC-014：打包与分发 + +新规范，本批写定，`pack` 的实现与之对齐。 + +1. **模型**：§2 的三个维度与腿；组合的合法性表；`--mode static` 的定义。 +2. **管线**：构建各腿 → 按 `mode` 收集闭包 → 生成暂存树 → 变换（strip、relocate、平台的 `seal`、签名）→ 写容器 → 产物、校验和、 + `mcpp.pack` 信封。`--merge` 是"读入若干暂存树 → 合并 → 写容器"。 +3. **暂存树与元数据**：布局（§3）；`mcpp-package.json` 的 schema（名称、版本、平台、腿、系统、每个文件的角色与摘要、启动方式）。这是 + 包格式提供者读的输入，也是 `--merge` 与 `mcpp emit xpkg` 读的输入。 +4. **接口**（面向构建插件、框架与系统）： + - 包格式提供者：构建插件或平台插件以 `pack-format` 声明、以 artifact action 生成（已有，规范化输入输出契约：暂存树、 + `mcpp-package.json`、`MCPP_PACK_*` 环境，产物路径与种类）。 + - 启动器与载体：引擎的通用启动桩；平台的 `carrier`。 + - 签名器：`[pack] sign = [argv]`（`{}` 为文件）或插件；签名在写容器之前、对暂存树内的程序与映像。 + - 分发描述：`mcpp emit xpkg` 由 `mcpp-package.json` 生成索引条目——每个系统 × 架构一项，或"一个通用资产对应所有系统"一项。 +5. **机器输出**：`mcpp.pack` 信封增加 `linkage`、`mode`、`pkgFormat`、`legs`、`systems`，`artifacts[]` 带 sha256。 +6. **全局用法规则**：§2 的 `--format`、`--pkg-format`、`--message-format` 分工。 + +## 7. 分发：经 xlings 不改 xlings + +xim-pkgindex 的包描述可以表达 openkal 程序： + +- 一个资产（通用包或 okb）对应 linux、macosx、windows 三个块，URL 相同。 +- `deps` 依赖 okrun（openkal-tools）；xvm 注册时把程序命令配成 `okrun <安装目录>/.okb`（或通用包内的启动器）。 +- 共享的 `.okso` 可作为独立的 xlings 包分发（每个架构一份，三系统通用），okrun 的搜索路径包含 xlings 的包存储，多个程序共用一份。 + +本批在 xim-pkgindex 的 PR 中加入一个 openkal 程序的示例与规则说明（同一 PR 还有 mcpp-ide 包）。 + +## 8. `.okso` 的本机绑定：openkal 体系的构建插件 + +- **openkal-run 增加库接口**：`okrun_open_library(path | bytes)`、`okrun_symbol(lib, name)`、`okrun_close(lib)`，以及 §4、§5 用到的 bundle 接口 + （一组内存中的映像按名字相互解析）。 +- **插件 `mcpplibs.openkal-bind`**（openkal-image 仓库，与 openkal-run 同版本发布；模块 `mcpp.mcpplibs.openkal_bind`，按 SPEC-007 R9.6）：一个 + 以 `.okso` 加 C 头文件发布的库，在非 openkal 目标上以构建依赖引入它；库自己的 build.mcpp 调用插件，插件用工具链的 clang 读出头文件中 + 的函数原型（AST JSON），生成绑定源码——每个函数一个同名本机导出，首次调用时经 openkal-run 加载 `.okso`，经带 `sysv_abi` 的函数指针转发 + （调用约定由编译器转换）——作为本包的源文件编译。`.okso` 与 openkal-run 经 `[runtime] deploy` 放到程序旁边。消费方照常依赖并 + `#include`，无感。 +- **边界由插件检查并拒绝**：只有 C 接口；边界类型不得含 `long`、`unsigned long`、`wchar_t`、`long double`（LP64 与 LLP64 不同）；资源归属 + 在一侧（接口成对创建、销毁）；回调参数由插件生成反向 thunk，原型必须完整；异常与 C++ 类型不跨边界。 +- mcpp 引擎不需要新机制；若实现中发现缺口（例如部署 `.okso` 的条件化），在 #847 中补齐。 + +## 9. 体积 + +本批全部实施，并在 hello（C++，`import std`）与 mcppls 服务器两个映像、一个原生 hello 上实测前后，表写入本节（实施时填写）。 + +| 处 | 做法 | +|---|---| +| 构建 | openkal-platform 增加 `-ffunction-sections -fdata-sections`（`--gc-sections` 已有，回收粒度因此到函数与数据）与 `--icf=safe`；`release` 配置可选 `-Oz`；LTO 已有 | +| 剥离 | 默认 strip；`--debug-symbols DIR` 输出分离的调试信息（已有） | +| 容器 | `--compress zstd\|gzip\|xz\|none[:level]`，`tar` 默认仍为 gzip（兼容），`.tar.zst` 可选；单文件载荷用 zstd；okb 内映像不压缩（为 mmap） | +| 去重 | 通用包与 okb 中资源只放一份，只有映像按架构；分发矩阵从"系统 × 架构"降为"架构" | +| 共享 | 共享 `.okso` 作为独立 xlings 包，一个架构装一次，多个程序共用（§7） | + +## 10. openkal 0.17.2 轮（本批最后） + +| 包 | 版本 | 内容 | +|---|---|---| +| openkal-image（openkal-ld、openkal-run、openkal-tools，同版本） | 0.1.2 | okb v1（`docs/okb.md`）、`okld bundle`/`bundle check`、okrun 读 okb、bundle 接口、库接口；新包 `openkal-bind`（同仓同版本） | +| openkal-musl | 0.22.4 | `renameat2` 的 `RENAME_EXCHANGE` 等标志按语义实现或拒绝（Q8）；钉住 openkal-ld 0.1.2 | +| openkal-llvm-runtime | 0.16.4 | 钉住 musl 0.22.4 | +| openkal-platform | 0.17.2 | 版本组前移；体积标志；`universal = true`；okb 包格式插件；载体入口改用 bundle 接口 | +| openkal 与三个实现 | 不变 | — | + +顺序：openkal-image → openkal-musl → openkal-llvm-runtime → openkal-platform → 索引 → mcpp 目标表钉住 0.17.2（#847 的最终 commit）。 +openkal-platform 0.17.1 先行发布，用于 #847 开发期的 CI 与索引；0.17.2 是本批的最终钉住。os 0.2.1 的钉住不涉及 musl，dev 依赖的 +llvm-runtime 随平台迁移移除；mcppls 的平台迁移后不再自己声明运行时。 + +## 11. 交付（在 part 1 §13 的基础上追加） + +| 仓库 | PR | 追加内容 | +|---|---|---| +| mcpp | #847 | 三维模型、`--pkg-format`（旧写法提示）、通用包与启动器、`--systems`、`--merge`、单文件启动桩、通用载体字节嵌入、签名接口、压缩、`mcpp-package.json`、平台阶段的包格式指令、SPEC-014、docs/10 重写（中英）、e2e | +| mcpplibs/openkal-image | 新 PR（0.1.2） | §5、§8 | +| mcpplibs/openkal-musl | 新 PR（0.22.4） | Q8 | +| mcpplibs/openkal-llvm-runtime | 新 PR（0.16.4） | 钉住 | +| mcpplibs/openkal-platform | 首个 PR 发 0.17.1；第二个 PR 发 0.17.2 | §3 的 `universal`、§5 的插件、§9 的标志 | +| mcpplibs/mcpp-index | 同 part 1 的 PR | 0.17.2 轮条目 | +| openxlings/xim-pkgindex | 同 part 1 的 PR | §7 的示例与规则 | + +openkal-platform 有两个 PR：0.17.1 是平台机制的首发（#847 开发期需要它进入索引），0.17.2 随新一轮；二者内容不同，各自对应一个 +发布，不合并。 + +## 12. 测试 + +e2e：三维组合与拒绝；`--pkg-format` 与旧写法；通用包两架构布局与启动器（本机架构运行；另一架构在 qemu-user 可用时运行）； +`--merge`；原生单文件（vendored 与 static）首次解压与复用；openkal 单文件从内存运行（不写盘：缓存目录无新文件）；okb 的生成、 +`okld bundle check`、`okrun app.okb`；签名步骤；`mcpp emit xpkg` 的通用资产条目；体积标志生效（`--gc-sections` 后未引用函数消失、 +`--icf` 合并）。openkal-image：okb 读写、对齐、摘要、签名的单元测试；绑定插件对一个 C 库（含拒绝的类型）的 e2e，三系统 CI。 + +## 13. 自审 + +1. **一致性**：三个维度各回答一个问题，旧选项保留为简写或旧写法，用户的已有命令不变；`--pkg-format` 消除同名异义。 +2. **引擎边界**：okb、载体入口、绑定插件都在 openkal 体系；引擎只提供通用机制（启动桩、字节嵌入、包格式提供者、启动器生成）。 +3. **跨平台**：通用包与 okb 让 openkal 的产物数等于架构数；self-contained 的多系统加载器在一台机器上可产出（SDK 缺失时拒绝），或由 + `--merge` 合并 CI 矩阵的结果。 +4. **稳定性与安全**：okb 映像对齐后 mmap，对齐不足或摘要不符时拒绝而非退回复制；单文件解压先校验载荷摘要，解到同目录的临时名再原子改名， + 按摘要寻址，不复用不完整的缓存，缓存目录只对当前用户可写。 +5. **用户体验**：默认即通用；解压即用；单文件免解压；xlings 一条命令安装。 +6. **可观察性**：`mcpp-package.json` 与 `mcpp.pack` 信封记录腿、系统、每个文件的摘要。 +7. **规模与风险**：本批再增一轮 openkal 发布与 SPEC-014。缓解：okb 不进入核心规范，级联止于 image、musl、llvm-runtime、platform； + openkal-platform 0.17.1 先行使 #847 的 CI 不等待新一轮。 +8. **未纳入**：APE 单文件（mcpp#849）。 + +## 14. 定案 + +1. `[platform] universal` 声明其目标默认打通用包（openkal-platform 写 `true`），`--no-universal` 只打所选目标。 +2. okb 由 openkal-platform 的插件作为包格式提供，引擎不认识 okb；平台阶段的指令表加入 `pack-format` 与 artifact action。 +3. `tar` 默认仍为 gzip，zstd 经 `--compress zstd` 或 `[pack] compress` 选择。 +4. openkal-platform 0.17.1 先行发布，0.17.2 为本批最终钉住。 + +## 15. 交叉验证：mcppls 与 mcpp-ide 用上 openkal 的独有特性 + +两个 IDE 一线的程序是本批特性的第一批真实用户：它们的服务器与后端本来就以 ok 映像分发，因此逐项采用本批的机制， +既是功能交付，也是对平台、打包与 okb 的交叉验证。二者的 PR 在 mcpp 发布与 openkal 0.17.2 轮之后完成这部分，判据写入各自 CI。 + +| 特性 | mcppls（#72） | mcpp-ide（0.1.0） | 验证的是 | +|---|---|---|---| +| 零声明的 `--target -openkal`（SPEC-013） | 服务器删去运行时、工具、ld 的声明 | 后端同样零声明 | 平台的层、工具、链接、封装、运行器 | +| 通用包（`universal`） | 一份 `mcppls--openkal.zip` 代替每系统 × 架构的服务器包 | 一份后端包 | 布局、启动器、`--systems` 的多系统加载器、`--merge` | +| okb | 编辑器扩展（VS Code vsix、CLion 插件、Zed）只带一个 `mcppls.okb` 与各系统 × 架构的 okrun，一个 vsix 服务全部平台 | vsix 与 mcpp-clion 的 zip 同样只带一个 `mcpp-ide.okb`，各平台一个包而不是按平台分包 | okb v1 的生成、`okld bundle check`、okrun 读 okb、mmap 与签名 | +| 单文件（openkal） | 命令行用户的 `mcppls` 单文件（每系统 × 架构，免安装） | 无（由扩展启动） | 载体从内存加载、bundle 接口 | +| xlings 单资产分发 | xim-pkgindex 的 mcppls 包改为一个资产对应三系统，依赖 okrun | 新增 mcpp-ide 包同法 | §7 的写法 | +| 宿主图的平台（SPEC-013 §6） | CI 一条以 `--host-platform openkal-platform@…` 构建（devtools 的构建程序作为映像运行） | CI 同一条（`spec/` 生成器作为映像运行） | 构建程序、其库作为映像，自举边界 | +| 体积（§9） | 服务器映像的实测基线与优化后数字写入本方案 §9 | 后端同 | 体积标志、zstd、去重 | +| 运行时版本组 | 以 openkal-platform 0.17.2 的版本组构建 | 同 | §8.1 | + +判据：两个 CI 各有一条"扩展包内只有一个 okb、每个系统 × 架构的 okrun 都能启动它"的检查；vsix 与 CLion 插件 zip 的体积在本方案 +记录前后对比。交付给维护者上传的产物目录（part 1 §13.3）中的 vsix 与 zip 即为采用 okb 后的版本。 diff --git a/modules/manifest/src/platform.cppm b/modules/manifest/src/platform.cppm index d4625676c..fae475c06 100644 --- a/modules/manifest/src/platform.cppm +++ b/modules/manifest/src/platform.cppm @@ -304,6 +304,30 @@ inline std::string display(const PackageRef& r) { return std::format("{}@{}", n, r.spec.version); } +// What a comparison of root positions keys an override on: its selector and +// the places its entries were written. One declaration is one key, so two +// workspace members whose plans an override would tell apart do not share one. +inline std::string identity(const Override& o) { + std::string s = o.selector; + auto add = [&](const Origin& g) { s += std::format("|{}:{}:{}", g.file, g.line, g.key); }; + if (o.platformSet) { + s += o.platformNone ? std::string("|none") : "|" + display(o.platform); + add(o.platformOrigin); + } + auto const& f = o.fields; + for (auto const& [k, e] : f.layers) add(e.origin); + for (auto const& [k, e] : f.tools) add(e.origin); + for (auto const& [k, w] : f.runners) add(w.origin); + if (f.carrier.set) add(f.carrier.origin); + if (f.runner.set) add(f.runner.origin); + for (auto const* t : { &f.link.driver, &f.link.start, &f.link.script, &f.link.exe, &f.link.shared }) + if (t->set) add(t->origin); + if (f.link.flags.set) add(f.link.flags.origin); + if (f.link.inputsSet) add(f.link.inputsOrigin); + if (f.link.sealSet) add(f.link.sealOrigin); + return s; +} + // The name `cfg(platform = "")` matches: the package name with a // trailing `-platform` removed, so `openkal-platform` is the platform // `openkal`. A selector may also name the package in full. diff --git a/src/build/execute.cppm b/src/build/execute.cppm index b753ee60e..b383bcf19 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -2823,7 +2823,7 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, if (!format_name.empty() && choice.tmpl.empty() && std::filesystem::is_directory(exe, dirEc)) { std::println(stderr, - "error: --format {} produced a directory, '{}', and no runner reaches it.\n" + "error: --pkg-format {} produced a directory, '{}', and no runner reaches it.\n" " A directory is not executed directly: a runner named '{}' runs\n" " it. A package supplies one with `mcpp::runner(\"{}\", …)`, or the\n" " project declares it:\n" @@ -3023,7 +3023,7 @@ export int build_run_target(const std::optional& targetName, // no reading for a file this host was never going to execute on its own. if (!format.empty() && no_runner) { std::println(stderr, - "error: --format and --no-runner cannot be combined: a " + "error: --pkg-format and --no-runner cannot be combined: a " "distributable is not something this host executes directly."); return 2; } @@ -3099,7 +3099,7 @@ export int build_run_target(const std::optional& targetName, // shape that reports zero artifacts fails LOUDLY here instead of // dereferencing past the end of an empty vector below. std::println(stderr, - "error: --format {} reported success and named no artifact to run", format); + "error: --pkg-format {} reported success and named no artifact to run", format); return 1; } // THE RUNNER RESOLUTION NEEDS A BuildContext, AND build_and_pack's OWN @@ -3132,7 +3132,7 @@ export int build_run_target(const std::optional& targetName, names += a.string(); } std::println(stderr, - "error: --format {} produced {} distributables ({}); mcpp run needs " + "error: --pkg-format {} produced {} distributables ({}); mcpp run needs " "exactly one to hand to the runner", format, outcome.artifacts.size(), names); return 1; } @@ -3228,7 +3228,7 @@ export int build_run_target(const std::optional& targetName, std::println(stderr, "error: '{}' is an application, and on {} an application is " "a shared library that a package installs. Run it through a " - "distributable: mcpp run --format , where is " + "distributable: mcpp run --pkg-format , where is " "one of: {}", t.name, ctx->tc.targetTriple, formats.empty() ? std::string("none declared") : formats); diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 54129fbcc..9c3c804ff 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -808,6 +808,11 @@ export struct BuildOverrides { // project root, as the resolution input it is, while the lock this planning // produces is written under `work_dir`. Build programs still run. bool plan_only = false; + // THE CONFIGURATION, NOT THE SOURCES (`prepare_configuration`, SPEC-013 + // §9): what the project's sources would be refused for -- `import std` + // with no C++ run-time in the graph -- is a note of the report, and the + // configuration is still reported. + bool configuration_only = false; // #355 tool provisioning re-enters prepare_build for the tool package. A // tool package's own build.mcpp may legitimately want another tool (gRPC's // wants protoc), so the depth cannot be 1 — but an unbounded chain is a diff --git a/src/build/prepare/configuration.cpp b/src/build/prepare/configuration.cpp index 54c92c385..9a58af88a 100644 --- a/src/build/prepare/configuration.cpp +++ b/src/build/prepare/configuration.cpp @@ -17,6 +17,7 @@ namespace fs = std::filesystem; std::expected prepare_configuration(std::string_view targetTriple) { BuildOverrides ov; ov.target_triple = std::string(targetTriple); + ov.configuration_only = true; std::error_code ec; if (auto root = mcpp::project::find_manifest_root(fs::current_path(ec))) { ov.project_root = *root; diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index d732e5755..26926c464 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -2459,9 +2459,14 @@ std::expected phase4b_graph_worklist(PrepareState& state) { // package of the graph provides and the tools it does not build are added // as the platform's, and walked like any request. A few rounds at most: the // packages one round adds may provide a layer the next then leaves. + // The version group is checked before each round's walk as well as after + // the last: a package of the family the project pinned outside the group + // is refused as such, before the platform's own packages would meet it as + // a second version of themselves. for (int round = 0; round < 3; ++round) { auto pushed = step4b_apply_platform(state); if (!pushed) return std::unexpected(pushed.error()); + if (auto r = step4b_check_platform_versions(state); !r) return r; if (!*pushed) break; if (auto r = drain(); !r) return r; } diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 0085b9583..1e8633a94 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -66,12 +66,10 @@ std::string known_short_name(const mp::PackageRef& r) { return {}; } -// The project's override tables: the workspace layer's, then the plan root's. +// The project's override tables: the plan root's, which hold the workspace +// layer's first (`merge_layer_root_position`). std::vector project_tables(const mcpp::manifest::Manifest& m) { - std::vector out; - if (m.layer) out = m.layer->platformOverrides; - out.insert(out.end(), m.platformOverrides.begin(), m.platformOverrides.end()); - return out; + return m.platformOverrides; } // Which layers each package of the graph provides, by layer. @@ -130,7 +128,6 @@ std::expected step1_select_platform(PrepareState& state) { if (auto cfg = state.get_cfg(false); cfg) config = (*cfg)->hostPlatform; const auto hostFlag = mcpp::invocation::get("MCPP_HOST_PLATFORM").value_or(""); mp::Override project = state.m->hostPlatform; - if (!project.platformSet && state.m->layer) project = state.m->layer->hostPlatform; auto hs = mcpp::platform::select_host(hostFlag, false, project, config); if (!hs) return std::unexpected(hs.error()); state.hostPlatformSel = std::move(*hs); @@ -395,6 +392,9 @@ std::expected step4b_check_platform_versions(PrepareState& st auto const& P = state.platform; if (!P.active || P.fields.versions.empty()) return {}; for (std::size_t i = 1; i < state.packages.size(); ++i) { + // A package built for this machine (a tool's) is not linked with the + // target's, and a group's exact pins are about one link. + if (state.packages[i].buildTimeOnly) continue; auto const& pm = state.packages[i].manifest; auto n = mcpp::pm::compat::resolve_package_name(pm.package.name, pm.package.namespace_); auto it = P.fields.versions.find(n.shortName); diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index 1de03af92..c52e2d8db 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -753,18 +753,23 @@ step11_std_module_availability_gate(PrepareState& state, bool needsStdModule) { mcpp::manifest::platform::display(it->second.ref), mcpp::platform::origin_text(it->second.origin)); } + // The remedy in the form that applies: the platform's layer when + // a platform is selected, a dependency of the target otherwise. + const std::string remedy = state.platform.active + ? std::format(" [target.'cfg(platform = \"{}\")'.layers]\n" + " \"c++-abi\" = \"@\"", + mcpp::manifest::platform::platform_name(state.platformPackageShortName)) + : std::format(" [target.{}.dependencies]\n" + " = \"\"", + state.tc->targetTriple); return std::unexpected(std::format( "`import std;` on '{}' needs a C++ run-time inside the program, and " "the graph supplies none.\n" " {}. Name the package that provides `mcpp:c++-abi` for this " "target:\n" "\n" - " [target.'cfg(platform = \"{}\")'.layers]\n" - " c++-abi = \"@\"", - state.tc->targetTriple, why, - state.platformPackageShortName.empty() - ? std::string("") - : mcpp::manifest::platform::platform_name(state.platformPackageShortName))); + "{}", + state.tc->targetTriple, why, remedy)); } return std::unexpected(std::format( "source imports std but toolchain '{}' provides no std module source", @@ -1072,8 +1077,15 @@ std::expected phase11_scan(PrepareState& state) { if (auto r = step11_apple_sdk_cxx_runtime(state, *needsStdModule); !r) return std::unexpected(r.error()); - if (auto r = step11_std_module_availability_gate(state, *needsStdModule); !r) - return std::unexpected(r.error()); + if (auto r = step11_std_module_availability_gate(state, *needsStdModule); !r) { + if (!state.overrides.configuration_only) return std::unexpected(r.error()); + // The configuration is reported; the sources' refusal travels with it. + PlanNote note; // a warning: the document is complete without the sources + note.code = "MCPP_CONFIGURATION_SOURCES_REFUSED"; + note.message = r.error(); + state.planNotes.push_back(std::move(note)); + *needsStdModule = false; + } step11_compute_fingerprint(state); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index 22da6ce2d..edad07d4a 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -1086,9 +1086,12 @@ step9_pin_and_linkage_diagnostics(PrepareState& state) { auto tt = mcpp::toolchain::triple::parse(state.resolvedTargetCanonical); return tt && tt->pin_is_capability(); }(); + // AND NOT WHEN THE PIN IS THE PLATFORM'S (SPEC-013 §4.5): the platform + // is what supplies the system here, and its toolchain is part of that + // statement, not a row's guess made before the graph was known. if (!state.pinReplacedDefault.empty() && state.resolvedTargetSide.system_from_graph() - && !pinIsCapability) { + && !pinIsCapability && !state.platformToolchainPin) { mcpp::diag::warning("toolchain", std::format( "this project's target side comes from its dependency graph, so " "{} would have served {}.\n" diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index a76729f4c..d479bb76d 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -1577,8 +1577,9 @@ step2_resolve_explicit_spec(PrepareState& state, ToolchainResolveCtx& ctx) { "\n the platform's ({}), replacing your " "{} — override with `[target.{}] toolchain`", state.platform.display(), state.pinReplacedDefault, - state.overrides.target_triple.empty() ? std::string("") - : state.overrides.target_triple); + state.overrides.target_triple.empty() + ? mcpp::toolchain::triple::host_triple().str() + : state.overrides.target_triple); else if (!state.pinReplacedDefault.empty()) chosenBy = std::format( "\n target default for {}, replacing your " diff --git a/src/cli.cppm b/src/cli.cppm index 225039933..b842051d4 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -745,10 +745,12 @@ int run(int argc, char** argv) { // given a run-specific spelling, because it is the same request: // "produce this distributable", asked from a command that also // executes it afterward. - .option(cl::Option("format").takes_value() - .help("Run the distributable a `--format ` pack would produce " - "(same values as `mcpp pack --format`); refused together with " + .option(cl::Option("pkg-format").takes_value().value_name("FORMAT") + .help("Run the distributable a `--pkg-format ` pack would produce " + "(same values as `mcpp pack --pkg-format`); refused together with " "--no-runner")) + .option(cl::Option("format").takes_value() + .help("Earlier spelling of --pkg-format")) .option(cl::Option("play-game") .help("Play a game in the status row while it builds: --play-game=snake|stack|runner, or one at random")) .option(cl::Option("print-launch") @@ -903,9 +905,11 @@ int run(int argc, char** argv) { // `mcpp::provides_pack_format` and `--format ` dispatches to // it; an unknown value names what IS available rather than a // constant, so the help text does not have to enumerate them. - .option(cl::Option("format").takes_value() - .help("tar (default; .zip for a Windows target) | dir | any " + .option(cl::Option("pkg-format").takes_value().value_name("FORMAT") + .help("The package format: tar (default; .zip for a Windows target) | dir | any " "format the resolved graph provides (e.g. appimage, msi)")) + .option(cl::Option("format").takes_value() + .help("Earlier spelling of --pkg-format")) .option(cl::Option("output").short_name('o').takes_value() .help("Override output path; with several members, the directory each archive or tree is written below")) // Packaging builds RELEASE by default — the artifact leaves this @@ -1055,7 +1059,9 @@ int run(int argc, char** argv) { .arg(cl::Arg("compiler").help("gcc | llvm | msvc | emsdk | android-ndk (or gcc@16.1.0; legacy aliases accepted)")) .arg(cl::Arg("version").help("e.g. 16.1.0, 15, 15.1")) .option(cl::Option("target").takes_value().help( - "Install the toolchain payload for (e.g. x86_64-windows-gnu)"))) + "Install the toolchain payload for (e.g. x86_64-windows-gnu)")) + .option(cl::Option("platform").takes_value().value_name("SPEC") + .help("With --target: the target's platform for this invocation (= MCPP_PLATFORM)"))) .subcommand(cl::App("default") .description("Set the default toolchain (and optionally the default target)") // Same dual-form as `install`: `gcc@16.1.0` or `gcc 16.1.0`, @@ -1184,6 +1190,10 @@ int run(int argc, char** argv) { .help("Report this target's configuration (prepared, not built; the project's when run in one)")) .option(cl::Option("host") .help("Report the host graph's configuration: what build programs and tools are built for")) + .option(cl::Option("platform").takes_value().value_name("SPEC") + .help("The target's platform for this invocation (= MCPP_PLATFORM): @, path:, none")) + .option(cl::Option("host-platform").takes_value().value_name("SPEC") + .help("The host graph's platform for this invocation (= MCPP_HOST_PLATFORM)")) .option(cl::Option("format").takes_value().value_name("json") .help("Machine-readable output (enveloped; see docs/50-machine-output.md)"))) .subcommand(cl::App("config") diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 07fc0a333..3f9c95797 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -10,6 +10,8 @@ export module mcpp.cli.cmd_build; import std; import mcpp.platform; +import mcpp.toolchain.platform; +import mcpp.toolchain.triple; import mcpp.build.runner_lookup; import mcpplibs.cmdline; import mcpp.build.prepare; @@ -976,7 +978,12 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, // pack --format`; `run` only adds the pairing with `--no-runner`, checked // in build_run_target where the runner is actually chosen. std::string format; - if (auto f = parsed.value("format")) format = *f; + if (auto f = parsed.value("pkg-format")) format = *f; + else if (auto f = parsed.value("format")) { + // The earlier spelling (`mcpp pack --pkg-format`'s note says why). + mcpp::ui::warning("`mcpp run --format` is the earlier spelling of --pkg-format"); + format = *f; + } if (parsed.is_flag_set("list-runners")) return mcpp::build::list_runners(package_filter, cache_mode, no_cache, target_triple, features, profile, accel); @@ -1286,6 +1293,34 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, mcpp::ui::plain(std::format(" slowest: {}", slowest)); return rc; } + // A PLATFORM PACKAGE IS TESTED ON THE TARGETS IT SERVES (SPEC-013 §2): + // with no --target, its tests are built for each of its `targets` with the + // package itself as the platform, and run where its runner can run them + // (`host = "same-arch"`: the target of this machine's architecture); the + // others are built and reported as not run. + if (ov.target_triple.empty() && ov.package_filter.empty()) { + std::error_code ec; + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path(ec)); + auto m = root ? mcpp::manifest::load(*root / "mcpp.toml") + : std::expected( + std::unexpected(mcpp::manifest::ManifestError{})); + if (m && m->platformDeclared && !m->platform.targets.value.empty()) { + mcpp::invocation::set("MCPP_PLATFORM", "path:" + root->generic_string()); + int rc = 0; + for (auto const& t : m->platform.targets.value) { + auto tov = ov; + tov.target_triple = t; + auto tto = to; + auto tt = mcpp::toolchain::triple::parse(t); + const bool here = tt && tt->arch == mcpp::toolchain::platform::host_arch; + if (!here && m->platform.runner.host == "same-arch") tto.noRun = true; + mcpp::ui::info("Testing", std::format("platform {} on {}{}", m->package.name, t, + tto.noRun ? " (built, not run here)" : "")); + if (int r = mcpp::build::run_tests(passthrough, tov, tto); r != 0 && rc == 0) rc = r; + } + return rc; + } + } return mcpp::build::run_tests(passthrough, ov, to); } @@ -1642,9 +1677,14 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { {"source", mcpp::platform::origin_text(runner.origin)}}}, {"runUnder", ctx->runUnder}, }; - mcpp::wire::emit({.kind = "mcpp.env", .effects = {}, .data = std::move(data)}); + mcpp::wire::Envelope env{.kind = "mcpp.env", .effects = {}}; + env.data = std::move(data); + for (auto const& n : ctx->planNotes) + env.diagnostics.push_back({n.code, n.severity, n.message}); + mcpp::wire::emit(env); return 0; } + for (auto const& n : ctx->planNotes) mcpp::ui::warning(n.message); std::println("{:<10} {}", "target", ctx->tc.targetTriple); std::println("{:<10} {} ({})", "compiler", ctx->tc.binaryPath.generic_string(), ctx->compilerChoice.origin); diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 89509e2f8..9af765aa4 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -175,6 +175,25 @@ export int cmd_place_dlls(const mcpplibs::cmdline::ParsedArgs& parsed) { namespace { +// THE PACKAGE FORMAT IS `--pkg-format`. `--format` names the output encoding +// of a reporting command everywhere else in mcpp (`self env --format json`), +// and `mcpp pack`'s machine output is `--message-format`; pack's own +// `--format` is accepted as the earlier spelling and says so. +std::optional pack_format_option(const mcpplibs::cmdline::ParsedArgs& parsed) { + auto pkg = parsed.value("pkg-format"); + auto old = parsed.value("format"); + if (old) { + if (pkg && *pkg != *old) { + mcpp::ui::warning(std::format( + "--format '{}' and --pkg-format '{}' disagree; --pkg-format is used", *old, *pkg)); + } else if (!pkg) { + mcpp::ui::warning("`mcpp pack --format` is the earlier spelling of --pkg-format, " + "the package format; --format names an output encoding elsewhere"); + } + } + return pkg ? pkg : old; +} + int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, mcpp::pack::PackOutcome* report, mcpp::pack::LibraryPackReport* libraryReport, @@ -318,7 +337,7 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { nlohmann::json stage = nullptr; nlohmann::json stages = nullptr; if (libraryRoute) { - const auto fmt = parsed.value("format").value_or("tar"); + const auto fmt = pack_format_option(parsed).value_or("tar"); artifacts.push_back(artifact_json(library.artifact, fmt, library.targets)); } else if (outcome.members.size() > 1) { // Several members (member selection design 2026-09-30, K1). Fields are @@ -506,11 +525,11 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, // exists to remove. The refusal moves to `build_and_pack`, after build // programs have declared what they provide and before anything is // compiled, where it can name what IS available instead of a fixed list. - if (auto v = parsed.value("format")) { + if (auto v = pack_format_option(parsed)) { if (*v == "tar") opts.format = mcpp::pack::Format::Tar; else if (*v == "dir") opts.format = mcpp::pack::Format::Dir; else if (v->empty()) { - mcpp::ui::error("--format needs a value: tar | dir | a format the " + mcpp::ui::error("--pkg-format needs a value: tar | dir | a format the " "resolved graph provides"); return 2; } else { @@ -604,12 +623,12 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, // a library package while the user asked for an installer. if (opts.format == mcpp::pack::Format::Dispatched) { mcpp::ui::error(std::format( - "--format {} is a distributable produced from a program's staged " + "--pkg-format {} is a distributable produced from a program's staged " "bundle, and '{}' is a library target.\n" " A library package ships an interface plus prebuilt binaries " "per triple; there is no\n" " single staged tree to hand a distribution format.\n" - " use: --format tar | dir, or name a program target", + " use: --pkg-format tar | dir, or name a program target", opts.formatName, route->targetName)); return 2; } diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 2cbde9a9d..d72ee4921 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -2111,7 +2111,7 @@ carrier_build_selfcontained(const Plan& plan, const std::filesystem::path& image const auto out = dir / "target" / "dist" / "packed"; fs::remove_all(out, ec); std::vector argv{ mcpp::invocation::self_exe_path().string(), "pack", - "--format", "dir", "-o", "packed", "--platform", "none" }; + "--pkg-format", "dir", "-o", "packed", "--platform", "none" }; if constexpr (mcpplibs::os::is_linux) { argv.insert(argv.end(), { "--target", std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch), "--mode", "static" }); diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 1e5136dfd..acd13ff0e 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -716,7 +716,7 @@ std::optional check_format(PackRun& run, GroupJob& g) { avail += ", " + f; } return Refusal{2, std::format( - "unknown --format '{}'.\n" + "unknown --pkg-format '{}'.\n" " available in this build: {}\n" " A format past `tar` and `dir` comes from a package in the " "resolved graph, which declares\n" @@ -758,7 +758,7 @@ std::optional check_format(PackRun& run, GroupJob& g) { } if (!missing.empty()) return Refusal{2, std::format( - "--format '{}' is not provided for every packed member.{}\n" + "--pkg-format '{}' is not provided for every packed member.{}\n" " A member is packed into its own staged tree by a provider in its own build " "program, or in a\n" " package that only that member reaches.{}", @@ -784,7 +784,7 @@ std::optional check_format(PackRun& run, GroupJob& g) { if (mcpp::pack::is_builtin_pack_format(f)) mcpp::ui::warning(std::format( "a package in this graph declares `mcpp:pack-format={}`, which " - "is one of the archive shapes `mcpp pack` owns; `--format {}` " + "is one of the archive shapes `mcpp pack` owns; `--pkg-format {}` " "will always select the built-in and never that package", f, f)); return std::nullopt; } @@ -880,7 +880,7 @@ std::optional stage_member(PackRun& run, GroupJob& g, MemberJob& m) { return Refusal{1, about(run, m, r.error().message)}; m.stageFailure = r.error().message; mcpp::ui::warning(about(run, m, std::format( - "no staged tree for --format {}: {}\n" + "no staged tree for --pkg-format {}: {}\n" " A format that consumes ${{mcpp.stage_dir}} cannot be produced " "here; one that names a\n" " built file with ${{mcpp.target_file:}} is unaffected.", @@ -1023,7 +1023,7 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba for (auto& m : g.members) { if (m.result.rc != 0 || m.stageFailure.empty()) continue; mcpp::ui::error(about(run, m, std::format( - "no staged tree for --format {}: {}", run.opts.formatName, m.stageFailure))); + "no staged tree for --pkg-format {}: {}", run.opts.formatName, m.stageFailure))); m.result.rc = 1; } if (std::ranges::none_of(g.members, [](const MemberJob& m) { return m.result.rc == 0; })) @@ -1103,7 +1103,7 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba auto [at, fresh] = writer.try_emplace(absolute_of(o), m.name); if (!fresh && at->second != m.name) return Refusal{1, std::format( - "members '{}' and '{}' would both write {} for --format '{}'.\n" + "members '{}' and '{}' would both write {} for --pkg-format '{}'.\n" " A file has one producer: write each member's output below a " "directory of its own,\n" " for example under `mcpp::out_dir()`, which is the member's.", @@ -1125,7 +1125,7 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba auto it = submitted.find(m.name); if (it != submitted.end() && !it->second.outputs.empty()) { any = true; continue; } mcpp::ui::error(about(run, m, std::format( - "no action claimed --format '{}'.\n" + "no action claimed --pkg-format '{}'.\n" " A package declared it provides this format, and no build " "program submitted a new\n" " `role = \"artifact\"` action when it was asked for.\n" @@ -1142,7 +1142,7 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba for (auto const& m : g.members) if (m.result.rc == 0) - mcpp::ui::info("Distributing", std::format("{} v{} (--format {})", + mcpp::ui::info("Distributing", std::format("{} v{} (--pkg-format {})", m.plan->packageName, m.plan->packageVersion, run.opts.formatName)); // NO EXPLICIT GOALS. Everything but the dist edges is already up to @@ -1196,9 +1196,9 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba // A drive that failed says why above; its missing file is // the member's failure, not a success that carried nothing. mcpp::ui::error(about(run, m, driveFailed - ? std::format("--format {} did not produce {}: its step failed", + ? std::format("--pkg-format {} did not produce {}: its step failed", run.opts.formatName, abs.string()) - : std::format("--format {} reported success and produced nothing at {}", + : std::format("--pkg-format {} reported success and produced nothing at {}", run.opts.formatName, abs.string()))); m.result.rc = 1; produced = false; diff --git a/src/project.cppm b/src/project.cppm index e3eb80741..fe663461e 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -398,6 +398,15 @@ void merge_layer_root_position(mcpp::manifest::Manifest& member, if (to.runner.empty()) to.runner = from.runner; for (auto const& [n, argv] : from.namedRunners) to.namedRunners.try_emplace(n, argv); } + + // The platform part of `[workspace.target.]` and `[workspace.host]` + // (SPEC-013 §4.5): the workspace's tables merge first, so the member's own + // come after them and win field by field. + member.platformOverrides.insert(member.platformOverrides.begin(), + layer.platformOverrides.begin(), + layer.platformOverrides.end()); + if (!member.hostPlatform.platformSet && layer.hostPlatform.platformSet) + member.hostPlatform = layer.hostPlatform; } } // namespace detail @@ -1073,6 +1082,14 @@ export std::string root_position_key( for (auto const& [name, argv] : e.namedRunners) list("target." + triple + ".runners." + name, argv); } + // The platform rows (SPEC-013 §4.5). A row that asks which platform is + // selected cannot be decided before the platform is, so it always counts. + for (auto const& o : m.platformOverrides) { + if (applies && !applies(o.selector) && o.selector.find("platform") == std::string::npos) + continue; + field("platform", mcpp::manifest::platform::identity(o)); + } + if (m.hostPlatform.platformSet) field("host.platform", mcpp::manifest::platform::identity(m.hostPlatform)); field("standard", m.package.standard); auto const& b = m.buildConfig; auto dialect = b.dialectCxxflags; @@ -1147,6 +1164,8 @@ virtual_workspace_root(const mcpp::manifest::Manifest& workspace, v.cppStandard = first.cppStandard; v.toolchain = first.toolchain; v.targetOverrides = first.targetOverrides; + v.platformOverrides = first.platformOverrides; + v.hostPlatform = first.hostPlatform; v.indices = first.indices; v.profiles = first.profiles; v.capabilityPins = first.capabilityPins; diff --git a/tests/e2e/202_machine_output_contract.sh b/tests/e2e/202_machine_output_contract.sh index 9e19c640a..f39954531 100755 --- a/tests/e2e/202_machine_output_contract.sh +++ b/tests/e2e/202_machine_output_contract.sh @@ -9,7 +9,7 @@ # # mcpp cache list --format json -> stdout: "Error: unknown option: --format" # stderr: (empty) rc=1 -# mcpp pack --format bogus -> stderr: "invalid --format ..." rc=2 +# mcpp pack --pkg-format bogus -> stderr: "invalid --pkg-format ..." rc=2 # # Two shapes for the same class of mistake, and the first one writes human text # into the channel a protocol owns. `mcpp cache list --format json | jq` gets a @@ -50,7 +50,7 @@ check "unknown option (top level)" 2 "$MCPP" --no-such-option # is the branch that answers "I know the option, not that value". check "unsupported value (self env)" 2 "$MCPP" self env --format yaml check "unsupported value (cache list)" 2 "$MCPP" cache list --format yaml -check "unsupported value (pack)" 2 "$MCPP" pack --format bogus +check "unsupported value (pack)" 2 "$MCPP" pack --pkg-format bogus # And the supported ones must still be JSON on stdout. Asserted here because # every check above is about what does NOT happen; without this, deleting the diff --git a/tests/e2e/257_shared_library_pe.sh b/tests/e2e/257_shared_library_pe.sh index 71ec74e0d..133ee9412 100755 --- a/tests/e2e/257_shared_library_pe.sh +++ b/tests/e2e/257_shared_library_pe.sh @@ -92,7 +92,7 @@ grep -qE "^build bin/libmathkit\.dll \| bin/libmathkit\.dll\.a : " "$nj" || { # ── 2. the package carries both, and names the right one to link ──────── rm -rf target -"$MCPP" pack mathkit --target "$TRIPLE" --format dir > pack.log 2>&1 \ +"$MCPP" pack mathkit --target "$TRIPLE" --pkg-format dir > pack.log 2>&1 \ || { cat pack.log; echo "pack failed"; exit 1; } pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0*' | head -1)" [[ -f "$pkg/lib/$TRIPLE/libmathkit.dll" ]] || { find "$pkg" -type f; echo "FAIL: no DLL in the package"; exit 1; } diff --git a/tests/e2e/259_shared_library_macho.sh b/tests/e2e/259_shared_library_macho.sh index 358b4dedd..ca54f511d 100755 --- a/tests/e2e/259_shared_library_macho.sh +++ b/tests/e2e/259_shared_library_macho.sh @@ -74,7 +74,7 @@ fi # ── 3. pack it ────────────────────────────────────────────────────────── rm -rf target -"$MCPP" pack mathkit --format dir > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +"$MCPP" pack mathkit --pkg-format dir > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } pkgrel="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0*' | head -1)" [[ -n "$pkgrel" ]] || { cat pack.log; echo "FAIL: no package"; exit 1; } # OUT of the producer's tree, so the next step can delete that tree entirely. diff --git a/tests/e2e/261_pack_follows_the_module_extension.sh b/tests/e2e/261_pack_follows_the_module_extension.sh index efed13543..423defcf0 100755 --- a/tests/e2e/261_pack_follows_the_module_extension.sh +++ b/tests/e2e/261_pack_follows_the_module_extension.sh @@ -62,7 +62,7 @@ kind = "lib" EOF cd mathkit -"$MCPP" pack mathkit --format dir > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +"$MCPP" pack mathkit --pkg-format dir > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" # ── the closure actually started ──────────────────────────────────────── @@ -139,7 +139,7 @@ sources = ["src/*.cppm"] [targets.plain] kind = "lib" EOF -( cd plain && "$MCPP" pack plain --format dir > pack.log 2>&1 ) || { +( cd plain && "$MCPP" pack plain --pkg-format dir > pack.log 2>&1 ) || { cat plain/pack.log; echo "plain pack failed"; exit 1; } plainpkg="$(find plain/target/dist -maxdepth 1 -type d -name 'plain-0.1.0-*' | head -1)" grep -q 'module_extensions' "$plainpkg/mcpp.toml" && { diff --git a/tests/e2e/638_pack_format_dispatch.sh b/tests/e2e/638_pack_format_dispatch.sh index 9dc346bbb..683aa9a60 100755 --- a/tests/e2e/638_pack_format_dispatch.sh +++ b/tests/e2e/638_pack_format_dispatch.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # requires: gcc -# 638_pack_format_dispatch.sh — `mcpp pack --format ` dispatches to a +# 638_pack_format_dispatch.sh — `mcpp pack --pkg-format ` dispatches to a # package, and no distribution format lives in the engine. # # `tar` and `dir` answer the same question `msi` and `appimage` answer -- what @@ -125,11 +125,11 @@ G=$(find_graph) # ── 2. an unknown format is refused, BEFORE the build, naming what exists ── set +e -"$MCPP" pack --format bogus > b2.log 2>&1 +"$MCPP" pack --pkg-format bogus > b2.log 2>&1 rc=$? set -e [ "$rc" -ne 0 ] || { cat b2.log; echo "FAIL: --format bogus was accepted"; exit 1; } -grep -q "unknown --format 'bogus'" b2.log \ +grep -q "unknown --pkg-format 'bogus'" b2.log \ || { cat b2.log; echo "FAIL: the refusal does not name the value"; exit 1; } # THE LIST IS THE POINT. A refusal that named a fixed list would be the # coupling this mechanism removes; this one names the graph's own answer. @@ -146,15 +146,15 @@ grep -q "Compiling app" b2.log \ # cannot be decided until prepare has run -- and prepare narrates what it # resolves. Deciding it later is what broke the contract once. set +e -so=$("$MCPP" pack --format bogus 2>/dev/null); rc=$? -se=$("$MCPP" pack --format bogus 2>&1 >/dev/null) +so=$("$MCPP" pack --pkg-format bogus 2>/dev/null); rc=$? +se=$("$MCPP" pack --pkg-format bogus 2>&1 >/dev/null) set -e [ -z "$so" ] || { echo "FAIL: the refusal wrote to stdout: $(echo "$so" | head -1)"; exit 1; } [ -n "$se" ] || { echo "FAIL: the refusal said nothing on stderr"; exit 1; } [ "$rc" = 2 ] || { echo "FAIL: the refusal exited $rc, expected 2"; exit 1; } # ── 3. the dispatched format produces a file, and it saw the staged tree ─── -"$MCPP" pack --format zap > b3.log 2>&1 || { cat b3.log; echo "FAIL: pack --format zap failed"; exit 1; } +"$MCPP" pack --pkg-format zap > b3.log 2>&1 || { cat b3.log; echo "FAIL: pack --pkg-format zap failed"; exit 1; } Z=$(find target -name 'app.zap' | head -1) [ -n "$Z" ] || { cat b3.log; echo "FAIL: --format zap produced nothing"; exit 1; } # The engine handed the build program the rest of `[package]`. @@ -242,7 +242,7 @@ int main() { } EOF set +e -"$MCPP" pack --format zap > b6.log 2>&1 +"$MCPP" pack --pkg-format zap > b6.log 2>&1 rc=$? set -e [ "$rc" -ne 0 ] || { cat b6.log; echo "FAIL: a source action reached the staged tree"; exit 1; } @@ -261,11 +261,11 @@ import mcpp; int main() { mcpp::provides_pack_format("zap"); return 0; } EOF set +e -"$MCPP" pack --format zap > b7.log 2>&1 +"$MCPP" pack --pkg-format zap > b7.log 2>&1 rc=$? set -e [ "$rc" -ne 0 ] || { cat b7.log; echo "FAIL: a format nothing claimed reported success"; exit 1; } -grep -q "no action claimed --format 'zap'" b7.log \ +grep -q "no action claimed --pkg-format 'zap'" b7.log \ || { cat b7.log; echo "FAIL: the refusal does not name the unclaimed format"; exit 1; } # ── 8. an artifact action that predates the request is not the package ───── @@ -312,7 +312,7 @@ int main() { return 0; } EOF -"$MCPP" pack --format zap > b8.log 2>&1 || { cat b8.log; echo "FAIL: a member that reads no staged tree was refused"; exit 1; } +"$MCPP" pack --pkg-format zap > b8.log 2>&1 || { cat b8.log; echo "FAIL: a member that reads no staged tree was refused"; exit 1; } grep -q "app.zap" b8.log \ || { cat b8.log; echo "FAIL: the gated action was not reported as the package"; exit 1; } grep -q "size.stamp" b8.log \ @@ -357,7 +357,7 @@ int main() { return 0; } EOF -"$MCPP" pack --format zap > b9.log 2>&1 || { cat b9.log; echo "FAIL: a provider that reads no staged tree was refused"; exit 1; } +"$MCPP" pack --pkg-format zap > b9.log 2>&1 || { cat b9.log; echo "FAIL: a provider that reads no staged tree was refused"; exit 1; } grep -q "app.zap" b9.log \ || { cat b9.log; echo "FAIL: the provider was not reported as the package"; exit 1; } # It really did not consume the tree: no stage manifest edge was added, so the diff --git a/tests/e2e/650_wasm_row_names_its_launcher.sh b/tests/e2e/650_wasm_row_names_its_launcher.sh index 4b22add5b..e8e0eb279 100755 --- a/tests/e2e/650_wasm_row_names_its_launcher.sh +++ b/tests/e2e/650_wasm_row_names_its_launcher.sh @@ -114,8 +114,8 @@ echo " ok: bin/app.js and bin/app.wasm exist, and no bare bin/app does" grep -q '^1-2-3$' run.log || fail "mcpp run did not print 1-2-3" run.log echo " ok: mcpp run prints 1-2-3" -"$MCPP" pack --target wasm32-emscripten --format dir > pack.log 2>&1 \ - || fail "mcpp pack --format dir failed" pack.log +"$MCPP" pack --target wasm32-emscripten --pkg-format dir > pack.log 2>&1 \ + || fail "mcpp pack --pkg-format dir failed" pack.log staged=$(find target/dist -maxdepth 1 -name 'app-0.1.0-wasm32-emscripten' -type d | head -1) [ -n "$staged" ] || fail "no staged directory under target/dist" pack.log [ -f "$staged/bin/app.js" ] || fail "the staged tree has no bin/app.js" pack.log @@ -124,7 +124,7 @@ manifest="target/dist/$(basename "$staged").stage-manifest" [ -f "$manifest" ] || fail "no .stage-manifest sibling of the staged tree" pack.log grep -q 'bin/app.js' "$manifest" || fail "the stage manifest does not list bin/app.js" "$manifest" grep -q 'bin/app.wasm' "$manifest" || fail "the stage manifest does not list bin/app.wasm" "$manifest" -echo " ok: mcpp pack --format dir stages both files and the manifest lists both" +echo " ok: mcpp pack --pkg-format dir stages both files and the manifest lists both" # `--no-entry` reaches the link line as an ordinary ldflag on this row -- no # engine mechanism checks for a `main` symbol (docs/21, docs/22). diff --git a/tests/e2e/651_a_build_program_deploys_what_it_generated.sh b/tests/e2e/651_a_build_program_deploys_what_it_generated.sh index 9fecef01d..368d72e1d 100755 --- a/tests/e2e/651_a_build_program_deploys_what_it_generated.sh +++ b/tests/e2e/651_a_build_program_deploys_what_it_generated.sh @@ -156,10 +156,10 @@ grep -qE "^ *build\.mcpp .* cached" b3.log \ grep -qx "changed resource" "$DEPLOYED" \ || fail "the restored file does not carry the action's latest output" "$DEPLOYED" -# ── 5. `mcpp pack --format dir` stages the deployed file ──────────────────── -"$MCPP" pack --format dir > b4.log 2>&1 || fail "pack --format dir failed" b4.log +# ── 5. `mcpp pack --pkg-format dir` stages the deployed file ──────────────────── +"$MCPP" pack --pkg-format dir > b4.log 2>&1 || fail "pack --pkg-format dir failed" b4.log STAGED=$(find target/dist -name res.bin | head -1) -[ -n "$STAGED" ] || fail "mcpp pack --format dir did not stage the deployed file" b4.log +[ -n "$STAGED" ] || fail "mcpp pack --pkg-format dir did not stage the deployed file" b4.log case "$STAGED" in */bin/app.resources/res.bin) : ;; *) fail "the staged file is not at bin/app.resources/res.bin" b4.log ;; diff --git a/tests/e2e/652_an_application_is_a_kind.sh b/tests/e2e/652_an_application_is_a_kind.sh index 0c7052fe1..ad1decb8f 100755 --- a/tests/e2e/652_an_application_is_a_kind.sh +++ b/tests/e2e/652_an_application_is_a_kind.sh @@ -97,7 +97,7 @@ grep -q "TOOL-RUN" <<<"$out" || fail "mcpp run tool did not print its marker" <( echo "mcpp run tool (bin) unaffected OK" # ── 5. `mcpp pack` treats `app` as the program route, same as `bin` ──────── -"$MCPP" pack --format dir > pack.log 2>&1 || fail "mcpp pack failed" pack.log +"$MCPP" pack --pkg-format dir > pack.log 2>&1 || fail "mcpp pack failed" pack.log staged=$(ls -d target/dist/myapp-0.1.0-*/ 2>/dev/null | head -1) [ -n "$staged" ] || fail "no staged tree under target/dist" pack.log [ -x "${staged}bin/myapp" ] || fail "the staged tree has no bin/myapp" pack.log diff --git a/tests/e2e/652b_an_application_on_android_is_a_shared_library.sh b/tests/e2e/652b_an_application_on_android_is_a_shared_library.sh index 46c6b7f51..048b2eb45 100755 --- a/tests/e2e/652b_an_application_on_android_is_a_shared_library.sh +++ b/tests/e2e/652b_an_application_on_android_is_a_shared_library.sh @@ -72,10 +72,10 @@ echo "bin keeps meaning binary on Android OK" # ── 3. `mcpp run` of the app, with no --format, is refused naming it ─────── out=$("$MCPP" run myapp --target "$TARGET" 2>&1) \ && fail "mcpp run of an app whose form is a library must be refused" <(echo "$out") -expected="error: 'myapp' is an application, and on $TARGET an application is a shared library that a package installs. Run it through a distributable: mcpp run --format , where is one of: none declared" +expected="error: 'myapp' is an application, and on $TARGET an application is a shared library that a package installs. Run it through a distributable: mcpp run --pkg-format , where is one of: none declared" grep -qF "$expected" <<<"$out" \ || fail "the refusal sentence does not match the design record's exact wording" <(echo "$out") -echo "mcpp run of the app without --format is refused, naming --format OK" +echo "mcpp run of the app without --pkg-format is refused, naming --pkg-format OK" # Negative-direction control for (3): `tool` is a real Binary on this same # row, so it is not caught by the app-only refusal above. @@ -144,8 +144,8 @@ int main() { } EOF -"$MCPP" pack --format blob --target "$TARGET" > pack.log 2>&1 \ - || fail "mcpp pack --format blob on Android failed" pack.log +"$MCPP" pack --pkg-format blob --target "$TARGET" > pack.log 2>&1 \ + || fail "mcpp pack --pkg-format blob on Android failed" pack.log staged=$(ls -d target/dist/myapp-0.1.0-*/ 2>/dev/null | head -1) [ -n "$staged" ] || fail "no staged tree under target/dist for the Android pack" pack.log [ -f "${staged}lib/libmyapp.so" ] \ @@ -154,7 +154,7 @@ staged=$(ls -d target/dist/myapp-0.1.0-*/ 2>/dev/null | head -1) || fail "the deploy'd file was not staged under bin/myres/ on the Android row" pack.log [ -n "$(find target -name 'myapp.blob' 2>/dev/null)" ] \ || fail "the reported artifact myapp.blob does not exist" pack.log -echo "mcpp pack --format blob on Android stages lib/libmyapp.so OK" +echo "mcpp pack --pkg-format blob on Android stages lib/libmyapp.so OK" # ── 5. MCPP_TARGET_MIN_PLATFORM_VERSION: numeric on Android, empty on host ─ # `mcpp::out_dir()` for a build PROGRAM is one directory per configuration and diff --git a/tests/e2e/656_run_hands_the_distributable_to_the_runner.sh b/tests/e2e/656_run_hands_the_distributable_to_the_runner.sh index 7975d1e53..ba0e4af13 100755 --- a/tests/e2e/656_run_hands_the_distributable_to_the_runner.sh +++ b/tests/e2e/656_run_hands_the_distributable_to_the_runner.sh @@ -4,7 +4,7 @@ # ` (#622 A10, design record # 2026-09-12-622-a-ui-framework-on-android-ios-and-web.md §2.10). # -# `mcpp run --format ` IS `mcpp pack --format ` -- the two +# `mcpp run --pkg-format ` IS `mcpp pack --pkg-format ` -- the two # prepares, the build, the staging, the provider's action -- followed by the # ORDINARY run, with the artifact the pack pipeline reported as the operand. # Runner resolution is unchanged: this project's `[target.] runner`. @@ -124,7 +124,7 @@ HOST=$(ls target | head -1) # THE PLAIN GRAPH'S OWN PATH, captured now rather than re-derived at the # end. `--format` (a `pack_format` override) resolves to a DIFFERENT # fingerprint directory than a plain build -- see mcpp.build.plan's -# fingerprint inputs -- so once `run --format blob` has run, target/ holds +# fingerprint inputs -- so once `run --pkg-format blob` has run, target/ holds # both a plain and a "blob" build.ninja side by side; a plain build never # touches the "blob" one, and `ninja` files already at the byte content # being written are not re-touched, so their mtimes do not order the two. @@ -143,8 +143,8 @@ chmod +x "$TMP/runner.sh" printf '\n[target.%s]\nrunner = ["%s"]\n' "$HOST" "$TMP/runner.sh" >> mcpp.toml -# ── 1. `mcpp run --format blob` hands the runner the distributable ──────── -out=$("$MCPP" run --format blob 2>&1) || fail "mcpp run --format blob failed" <(echo "$out") +# ── 1. `mcpp run --pkg-format blob` hands the runner the distributable ──────── +out=$("$MCPP" run --pkg-format blob 2>&1) || fail "mcpp run --pkg-format blob failed" <(echo "$out") grep -q "RUNNER: .*/app\.final$" <<<"$out" \ || fail "the runner's operand was not the terminal artifact app.final" <(echo "$out") grep -q "RUNNER: .*/app\.blob$" <<<"$out" \ @@ -155,7 +155,7 @@ grep -q "Packed .*app\.blob" <<<"$out" \ && fail "the intermediate app.blob was reported as Packed" <(echo "$out") grep -q "1-2-3" <<<"$out" \ || fail "the program's marker did not print through the runner" <(echo "$out") -echo "mcpp run --format blob hands the runner the distributable OK" +echo "mcpp run --pkg-format blob hands the runner the distributable OK" # ── 2. a plain `mcpp run` is unaffected: the operand is the link output ─── out=$("$MCPP" run 2>&1) || fail "mcpp run failed" <(echo "$out") @@ -166,31 +166,31 @@ echo "mcpp run alone is unaffected OK" # ── 3. an unknown format is refused, naming what the graph provides ─────── set +e -out=$("$MCPP" run --format bogus 2>&1); rc=$? +out=$("$MCPP" run --pkg-format bogus 2>&1); rc=$? set -e -[ "$rc" -ne 0 ] || fail "mcpp run --format bogus was accepted" <(echo "$out") -grep -q "unknown --format 'bogus'" <<<"$out" \ +[ "$rc" -ne 0 ] || fail "mcpp run --pkg-format bogus was accepted" <(echo "$out") +grep -q "unknown --pkg-format 'bogus'" <<<"$out" \ || fail "the refusal does not name the value" <(echo "$out") grep -q "available in this build:.*blob" <<<"$out" \ || fail "the refusal does not name what is available" <(echo "$out") -echo "mcpp run --format bogus is refused, naming blob OK" +echo "mcpp run --pkg-format bogus is refused, naming blob OK" # ── 4. --format with --no-runner is refused before anything runs ────────── set +e -out=$("$MCPP" run --format blob --no-runner 2>&1); rc=$? +out=$("$MCPP" run --pkg-format blob --no-runner 2>&1); rc=$? set -e [ "$rc" -ne 0 ] || fail "--format with --no-runner was accepted" <(echo "$out") -grep -q -- "--format and --no-runner cannot be combined" <<<"$out" \ +grep -q -- "--pkg-format and --no-runner cannot be combined" <<<"$out" \ || fail "the refusal does not name the pairing" <(echo "$out") grep -q "1-2-3" <<<"$out" && fail "the program ran despite the refusal" <(echo "$out") echo "--format with --no-runner is refused OK" # ── 5. a plain build afterward has a plain graph, with no blob edge ─────── find target -name 'app.blob' -delete 2>/dev/null || true -"$MCPP" build > b5.log 2>&1 || fail "the build after run --format failed" b5.log +"$MCPP" build > b5.log 2>&1 || fail "the build after run --pkg-format failed" b5.log [ -f "$G0" ] || fail "the plain build's own build.ninja disappeared" b5.log [ "$(graph_dist "$G0")" = "none" ] \ - || { sed -n '2p' "$G0"; fail "the graph after run --format is not dist=none" b5.log; } + || { sed -n '2p' "$G0"; fail "the graph after run --pkg-format is not dist=none" b5.log; } grep -q "blob" "$G0" \ && fail "a plain build's graph still carries the blob edge" b5.log [ -z "$(find target -name 'app.blob' 2>/dev/null)" ] \ diff --git a/tests/e2e/662_pack_stages_declared_files_before_the_closure.sh b/tests/e2e/662_pack_stages_declared_files_before_the_closure.sh index 2ad8c7de1..baad1bcfb 100755 --- a/tests/e2e/662_pack_stages_declared_files_before_the_closure.sh +++ b/tests/e2e/662_pack_stages_declared_files_before_the_closure.sh @@ -67,7 +67,7 @@ linux = "gcc@16.1.0" deploy_files = ["share/notes.txt"] EOF -"$MCPP" pack --format dir > dir.log 2>&1 || fail "mcpp pack --format dir failed" dir.log +"$MCPP" pack --pkg-format dir > dir.log 2>&1 || fail "mcpp pack --pkg-format dir failed" dir.log tree=$(find target/dist -maxdepth 1 -type d -name 'app-0.1.0-*' | head -1) [ -n "$tree" ] || fail "no staged directory under target/dist" dir.log @@ -147,7 +147,7 @@ int main() { } EOF -"$MCPP" pack --format zap > zap.log 2>&1 || fail "mcpp pack --format zap failed" zap.log +"$MCPP" pack --pkg-format zap > zap.log 2>&1 || fail "mcpp pack --pkg-format zap failed" zap.log Z=$(find target -name 'app.zap' | head -1) [ -n "$Z" ] || fail "--format zap produced nothing" zap.log diff --git a/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh b/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh index af5f92cf6..4625f9dd0 100755 --- a/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh +++ b/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh @@ -46,7 +46,7 @@ printf "resource-1\n" > res.txt # empty. Also asserted by 652b for the tarball route; repeated here, against # `--format dir`, because THIS is the format the several-triple case below # also uses, and the two must be compared under the same format. -"$MCPP" pack myapp --target aarch64-linux-android --format dir > pack1.log 2>&1 \ +"$MCPP" pack myapp --target aarch64-linux-android --pkg-format dir > pack1.log 2>&1 \ || fail "single-target Android pack failed" pack1.log staged1=$(ls -d target/dist/myapp-0.1.0-*/ 2>/dev/null | head -1) [ -n "$staged1" ] || fail "no staged tree for the single-target pack" pack1.log @@ -61,7 +61,7 @@ echo "one --target stages flat lib/libmyapp.so, unchanged OK" # ── 2. two Android triples: one tree, two lib// legs ───────────────── rm -rf target/dist "$MCPP" pack myapp --target aarch64-linux-android --target x86_64-linux-android \ - --format dir > pack2.log 2>&1 \ + --pkg-format dir > pack2.log 2>&1 \ || fail "two-target Android pack failed" pack2.log staged2=$(ls -d target/dist/myapp-0.1.0-*/ 2>/dev/null | head -1) [ -n "$staged2" ] || fail "no staged tree for the two-target pack" pack2.log diff --git a/tests/e2e/666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh b/tests/e2e/666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh index 9b81c0742..9786c86aa 100755 --- a/tests/e2e/666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh +++ b/tests/e2e/666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh @@ -63,7 +63,7 @@ int main() { EOF # The built-in archive stages the program: its closure is only the OS's. -"$MCPP" pack --format dir > dir.log 2>&1 || fail "--format dir of a Mach-O program failed" dir.log +"$MCPP" pack --pkg-format dir > dir.log 2>&1 || fail "--pkg-format dir of a Mach-O program failed" dir.log D=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'zapapp-0.1.0-*' | head -1) [ -n "$D" ] || fail "--format dir staged no tree" dir.log [ -x "$D/bin/zapapp" ] || fail "the staged tree has no bin/zapapp" dir.log @@ -71,7 +71,7 @@ D=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'zapapp-0.1.0-*' | he || fail "the dir manifest does not say closure = walked" "$D.stage-manifest" echo " --format dir stages a Mach-O program with closure = walked" -"$MCPP" pack --format zap > zap.log 2>&1 || fail "mcpp pack --format zap failed" zap.log +"$MCPP" pack --pkg-format zap > zap.log 2>&1 || fail "mcpp pack --pkg-format zap failed" zap.log if grep -q 'staged without its dependency closure' zap.log; then fail "pack still reports the tree as staged without its closure" zap.log fi diff --git a/tests/e2e/667_an_android_pack_stages_its_closure.sh b/tests/e2e/667_an_android_pack_stages_its_closure.sh index 0f66d6c81..ad51959cc 100755 --- a/tests/e2e/667_an_android_pack_stages_its_closure.sh +++ b/tests/e2e/667_an_android_pack_stages_its_closure.sh @@ -48,8 +48,8 @@ printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > app/sr cd app # ── 1. one triple: the closure is staged under lib/ ─────────────────────── -"$MCPP" pack --target x86_64-linux-android --format dir > dir.log 2>&1 \ - || fail "mcpp pack --format dir failed" dir.log +"$MCPP" pack --target x86_64-linux-android --pkg-format dir > dir.log 2>&1 \ + || fail "mcpp pack --pkg-format dir failed" dir.log tree=target/dist/app-0.1.0-x86_64-linux-android [ -d "$tree" ] || fail "no staged tree at $tree" dir.log got=$(cd "$tree/lib" && find . -type f | sort | tr '\n' ' ') @@ -90,7 +90,7 @@ libraries = ["ext"] EOF printf 'import fw;\nextern "C" int ext_value();\nint main() { return fw_anchor() == 41 && ext_value() == 1 ? 0 : 1; }\n' > src/main.cpp -"$MCPP" pack --target x86_64-linux-android --format dir > ext.log 2>&1 \ +"$MCPP" pack --target x86_64-linux-android --pkg-format dir > ext.log 2>&1 \ || fail "the pack with a prebuilt library failed" ext.log [ -f "$tree/lib/libext.so" ] || fail "the prebuilt libext.so is not staged" ext.log grep -qxF "needs${TAB}libext.so${TAB}lib/libext.so" "$manifest" \ @@ -100,7 +100,7 @@ echo " ok: a library from [runtime] link_library_dirs is staged" # The build is up to date, so the pack does not relink; the file is simply gone. rm ext/lib/libext.so for fmt in dir tar; do - if "$MCPP" pack --target x86_64-linux-android --format "$fmt" > "gone-$fmt.log" 2>&1; then + if "$MCPP" pack --target x86_64-linux-android --pkg-format "$fmt" > "gone-$fmt.log" 2>&1; then fail "--format $fmt succeeded with libext.so gone" "gone-$fmt.log" fi grep -q "libext.so: found in none of" "gone-$fmt.log" \ @@ -140,7 +140,7 @@ int main() { return 0; } EOF -"$MCPP" pack --target x86_64-linux-android --format copy > copy.log 2>&1 \ +"$MCPP" pack --target x86_64-linux-android --pkg-format copy > copy.log 2>&1 \ || fail "the dispatched format failed" copy.log seen=$(find target -name stage.txt | head -1) [ -f "$seen" ] || fail "the provider wrote nothing" copy.log @@ -166,7 +166,7 @@ demo.fw = { path = "../fw", linkage = "shared" } EOF printf 'import fw;\nint main() { return fw_anchor() == 41 ? 0 : 1; }\n' > src/main.cpp rm -rf target/dist -"$MCPP" pack --target x86_64-linux-android --target aarch64-linux-android --format dir \ +"$MCPP" pack --target x86_64-linux-android --target aarch64-linux-android --pkg-format dir \ > two.log 2>&1 || fail "the two-triple pack failed" two.log tree2=$(find target/dist -mindepth 1 -maxdepth 1 -type d | head -1) [ -n "$tree2" ] || fail "no staged tree for two triples" two.log diff --git a/tests/e2e/668_a_macho_program_is_staged_with_its_closure.sh b/tests/e2e/668_a_macho_program_is_staged_with_its_closure.sh index 3fb15c4b6..59a27708a 100755 --- a/tests/e2e/668_a_macho_program_is_staged_with_its_closure.sh +++ b/tests/e2e/668_a_macho_program_is_staged_with_its_closure.sh @@ -11,7 +11,7 @@ # removed -- a tree that ran only while the build directory existed would pass # every check but that one. Measured on macos-15 before the change (the # triage record's reading C5): exit 7 staged by hand, exit 134 without the -# dylib, and `mcpp pack --format dir` refused the program. +# dylib, and `mcpp pack --pkg-format dir` refused the program. set -e TMP=$(mktemp -d) @@ -51,7 +51,7 @@ EOF cd app # ── 1. --format dir stages the program and its dylib ────────────────────── -"$MCPP" pack --format dir > dir.log 2>&1 || fail "mcpp pack --format dir failed" dir.log +"$MCPP" pack --pkg-format dir > dir.log 2>&1 || fail "mcpp pack --pkg-format dir failed" dir.log tree=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'app-0.1.0-*' | head -1) [ -n "$tree" ] || fail "no staged tree under target/dist" dir.log [ -x "$tree/bin/app" ] || fail "the staged tree has no bin/app" dir.log diff --git a/tests/e2e/669_an_elf_host_pack_states_its_closure.sh b/tests/e2e/669_an_elf_host_pack_states_its_closure.sh index 7302d8f97..78c4e7b3c 100755 --- a/tests/e2e/669_an_elf_host_pack_states_its_closure.sh +++ b/tests/e2e/669_an_elf_host_pack_states_its_closure.sh @@ -77,7 +77,7 @@ named=$(find "$bin" -maxdepth 1 -name 'libnamed.so' -type f | head -1) echo " ok: libfw.so records its file name; a declared soname keeps libnamed.so.1" # ── 2. A3: the manifest states the closure the tree carries ─────────────── -"$MCPP" pack --format dir > dir.log 2>&1 || fail "mcpp pack --format dir failed" dir.log +"$MCPP" pack --pkg-format dir > dir.log 2>&1 || fail "mcpp pack --pkg-format dir failed" dir.log tree=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'app-0.1.0-*' | head -1) [ -n "$tree" ] || fail "no staged tree" dir.log manifest="$tree.stage-manifest" @@ -111,12 +111,12 @@ ldflags = ["-L$TMP/ext/lib", "-lext", "-Wl,-rpath,$TMP/ext/lib"] EOF printf 'int ext_value();\nint main() { return ext_value() == 1 ? 0 : 1; }\n' > extapp/src/main.cpp cd extapp -"$MCPP" pack --format dir > ext.log 2>&1 || fail "the pack with a prebuilt library failed" ext.log +"$MCPP" pack --pkg-format dir > ext.log 2>&1 || fail "the pack with a prebuilt library failed" ext.log etree=$(find target/dist -mindepth 1 -maxdepth 1 -type d -name 'extapp-0.1.0-*' | head -1) grep -qxF "needs${TAB}libext.so${TAB}lib/libext.so" "$etree.stage-manifest" \ || fail "libext.so is not stated as bundled" "$etree.stage-manifest" rm "$TMP/ext/lib/libext.so" -if "$MCPP" pack --format dir > gone.log 2>&1; then +if "$MCPP" pack --pkg-format dir > gone.log 2>&1; then fail "--format dir succeeded with libext.so gone" gone.log fi grep -q "libext.so: the loader finds no file for it" gone.log \ diff --git a/tests/e2e/671_toolchain_is_an_option_of_run_test_and_pack.sh b/tests/e2e/671_toolchain_is_an_option_of_run_test_and_pack.sh index d0d2771d8..7c69d249e 100644 --- a/tests/e2e/671_toolchain_is_an_option_of_run_test_and_pack.sh +++ b/tests/e2e/671_toolchain_is_an_option_of_run_test_and_pack.sh @@ -34,7 +34,7 @@ fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2 cd tcopt # ── 1. the option is accepted and its value reaches resolution ───────────── -for spelling in "run" "test" "pack --format dir"; do +for spelling in "run" "test" "pack --pkg-format dir"; do read -ra words <<<"$spelling" set +e "$MCPP" "${words[@]}" --toolchain nosuch@0.0.1 > c1.log 2>&1 diff --git a/tests/e2e/672_a_runner_receives_the_files_the_artifact_carries.sh b/tests/e2e/672_a_runner_receives_the_files_the_artifact_carries.sh index 24374053e..82e4601dd 100644 --- a/tests/e2e/672_a_runner_receives_the_files_the_artifact_carries.sh +++ b/tests/e2e/672_a_runner_receives_the_files_the_artifact_carries.sh @@ -208,11 +208,11 @@ int main() { return 0; } EOF -"$MCPP" run --format blob > r5.log 2>&1 || fail "mcpp run --format blob failed" r5.log +"$MCPP" run --pkg-format blob > r5.log 2>&1 || fail "mcpp run --pkg-format blob failed" r5.log rec="$TMP/rec/lone.blob.rec" [ -f "$rec" ] && ! grep -q '^UNSET$' "$rec" || fail "the distributable's runner did not receive the variable" r5.log "$rec" list=$(sed -n 's/^LIST=//p' "$rec") [ -f "$list" ] && [ ! -s "$list" ] || fail "the distributable's list is not an existing empty file" "$rec" -echo "a distributable run with --format receives an empty list OK" +echo "a distributable run with --pkg-format receives an empty list OK" echo "PASS: 672_a_runner_receives_the_files_the_artifact_carries" diff --git a/tests/e2e/674_a_format_is_run_by_the_runner_named_after_it.sh b/tests/e2e/674_a_format_is_run_by_the_runner_named_after_it.sh index a5ecc9196..ec7a27f6c 100644 --- a/tests/e2e/674_a_format_is_run_by_the_runner_named_after_it.sh +++ b/tests/e2e/674_a_format_is_run_by_the_runner_named_after_it.sh @@ -6,8 +6,8 @@ # spawn (#634 B3). # # Measured on macos-15 against 2026.9.14.1: with a named runner `app` declared, -# `mcpp run --format app --runner app` ran the bundle and returned its status, -# while `mcpp run --format app` alone handed the bundle directory to the kernel: +# `mcpp run --pkg-format app --runner app` ran the bundle and returned its status, +# while `mcpp run --pkg-format app` alone handed the bundle directory to the kernel: # # ...RApp.app could not be started: Permission denied (error 13) exit 126 # @@ -112,7 +112,7 @@ cp mcpp.toml mcpp.toml.base # ── 1. the runner named after the format ────────────────────────────────── set +e -"$MCPP" run --format blob > r1.log 2>&1; rc=$? +"$MCPP" run --pkg-format blob > r1.log 2>&1; rc=$? set -e grep -q "RUNNER-blob: .*/app\.blob$" r1.log \ || fail "the runner named 'blob' did not receive the distributable" r1.log @@ -120,11 +120,11 @@ grep -q "PROGRAM-RAN" r1.log || fail "the program did not run through the runner [ "$rc" -eq 7 ] || fail "the program's status 7 did not come back (got $rc)" r1.log grep -q "Running \`.*runner-blob.sh" r1.log \ || fail "the status line does not say Running for the format's runner" r1.log -echo "mcpp run --format blob runs through the runner named blob OK" +echo "mcpp run --pkg-format blob runs through the runner named blob OK" # ── 2. a typed --runner wins ─────────────────────────────────────────────── set +e -"$MCPP" run --format blob --runner other > r2.log 2>&1; rc=$? +"$MCPP" run --pkg-format blob --runner other > r2.log 2>&1; rc=$? set -e grep -q "RUNNER-other: .*/app\.blob$" r2.log || fail "--runner other did not win" r2.log grep -q "RUNNER-blob" r2.log && fail "the format's runner ran although --runner was typed" r2.log @@ -142,7 +142,7 @@ echo "a plain mcpp run executes the link output directly OK" # ── 4. a directory no runner reaches ─────────────────────────────────────── set +e -"$MCPP" run --format bundle > r4.log 2>&1; rc=$? +"$MCPP" run --pkg-format bundle > r4.log 2>&1; rc=$? set -e [ "$rc" -eq 126 ] || fail "the directory without a runner did not exit 126 (got $rc)" r4.log grep -q "produced a directory" r4.log || fail "the refusal does not say the distributable is a directory" r4.log @@ -155,7 +155,7 @@ echo "a directory no runner reaches is refused before the spawn OK" # ── 5. a manifest runner named after the format reaches the directory ───── printf 'bundle = ["%s"]\n' "$TMP/runner-bundle.sh" >> mcpp.toml set +e -"$MCPP" run --format bundle > r5.log 2>&1; rc=$? +"$MCPP" run --pkg-format bundle > r5.log 2>&1; rc=$? set -e grep -q "RUNNER-bundle: .*/app\.bundle$" r5.log || fail "the runner named 'bundle' did not receive the directory" r5.log [ "$rc" -eq 7 ] || fail "the bundle's program status did not come back (got $rc)" r5.log @@ -165,7 +165,7 @@ echo "a manifest runner named bundle runs the directory OK" cp mcpp.toml.base mcpp.toml printf '\n[target.%s]\nrunner = ["%s"]\n' "$HOST" "$TMP/runner-default.sh" >> mcpp.toml set +e -"$MCPP" run --format bundle > r6.log 2>&1; rc=$? +"$MCPP" run --pkg-format bundle > r6.log 2>&1; rc=$? set -e grep -q "RUNNER-default: .*/app\.bundle$" r6.log || fail "the default runner did not receive the directory" r6.log grep -q "no runner reaches it" r6.log && fail "a refusal was printed although a default runner exists" r6.log diff --git a/tests/e2e/689_pack_takes_features.sh b/tests/e2e/689_pack_takes_features.sh index 0552358bc..0bc4f0f0b 100644 --- a/tests/e2e/689_pack_takes_features.sh +++ b/tests/e2e/689_pack_takes_features.sh @@ -100,7 +100,7 @@ packed_answer() { # $1 = log; runs the program the last pack staged tool_entries() { find "$MCPP_HOME/build-cache" -path '*/tool/*' -name 'gen*' -type f 2>/dev/null | wc -l; } # ── The pack without the feature: no tool, the fallback source ───────────── -"$MCPP" pack --format dir > p0.log 2>&1 || fail "pack without the feature failed" p0.log +"$MCPP" pack --pkg-format dir > p0.log 2>&1 || fail "pack without the feature failed" p0.log [ "$(packed_answer p0.log)" = "ANSWER=0" ] || fail "pack without the feature: expected 0" p0.log [ "$(tool_entries)" -eq 0 ] || fail "a pack without the feature built the host tool" p0.log grep -q "host tool" p0.log && fail "a pack without the feature mentioned the host tool" p0.log @@ -108,7 +108,7 @@ echo "ok: a pack without --features builds no host tool" # ── The pack with the feature: the tool is built and its output shipped ──── rm -rf target -"$MCPP" pack --format dir --features installer > p1.log 2>&1 \ +"$MCPP" pack --pkg-format dir --features installer > p1.log 2>&1 \ || fail "mcpp pack --features installer failed" p1.log grep -q "unknown option" p1.log && fail "pack did not accept --features" p1.log grep -q "Building.*host tool" p1.log || fail "the pack with the feature did not build the tool" p1.log diff --git a/tests/e2e/722_pack_strips_what_the_graph_built.sh b/tests/e2e/722_pack_strips_what_the_graph_built.sh index 97b14633f..77dbe5919 100755 --- a/tests/e2e/722_pack_strips_what_the_graph_built.sh +++ b/tests/e2e/722_pack_strips_what_the_graph_built.sh @@ -102,7 +102,7 @@ printf 'int dep_answer();\nint main() { return dep_answer() == 42 ? 0 : 1; }\n' cd hostapp # ── A ─────────────────────────────────────────────────────────────────────── -"$MCPP" pack --format tar > a.log 2>&1 || fail "A: pack failed" a.log +"$MCPP" pack --pkg-format tar > a.log 2>&1 || fail "A: pack failed" a.log grep -q "Packing hostapp v0.1.0 (vendored, stripped)" a.log || fail "A: the Packing line" a.log rm -rf xa && mkdir xa && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xa lib=$(ls xa/*/lib/libdep.so); prog=$(ls xa/*/bin/hostapp) @@ -116,7 +116,7 @@ case "$fp" in "symtab=0 debug=0 "*) ;; *) fail "A: bin/hostapp is not stripped: echo "ok: A, the program and the graph-built library are stripped and run" # ── B ─────────────────────────────────────────────────────────────────────── -"$MCPP" pack --format tar --no-strip > b.log 2>&1 || fail "B: pack failed" b.log +"$MCPP" pack --pkg-format tar --no-strip > b.log 2>&1 || fail "B: pack failed" b.log grep -q "Packing hostapp v0.1.0 (vendored)" b.log || fail "B: the Packing line" b.log rm -rf xb && mkdir xb && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xb fb=$(elf_facts "$(ls xb/*/lib/libdep.so)") @@ -125,7 +125,7 @@ case "$fb" in "symtab=1 "*) ;; *) fail "B: --no-strip did not reach lib/libdep.s echo "ok: B, --no-strip reaches the graph-built library" # ── C ─────────────────────────────────────────────────────────────────────── -"$MCPP" pack --format tar --debug-symbols "$TMP/dbg" > c.log 2>&1 || fail "C: pack failed" c.log +"$MCPP" pack --pkg-format tar --debug-symbols "$TMP/dbg" > c.log 2>&1 || fail "C: pack failed" c.log [ -s "$TMP/dbg/libdep.so.debug" ] || fail "C: no libdep.so.debug" c.log rm -rf xc && mkdir xc && tar -xzf target/dist/hostapp-0.1.0-x86_64-linux-gnu.tar.gz -C xc fc=$(elf_facts "$(ls xc/*/lib/libdep.so)") @@ -158,10 +158,10 @@ int main() { } EOF -"$MCPP" pack --format probe --debug-symbols "$TMP/dbg2" > d.log 2>&1 || fail "D: pack failed" d.log +"$MCPP" pack --pkg-format probe --debug-symbols "$TMP/dbg2" > d.log 2>&1 || fail "D: pack failed" d.log probe=$(find target -name probe.txt | head -1) [ "$(cat "$probe")" = "strip=1 debug=$TMP/dbg2" ] || fail "D: the program read '$(cat "$probe" 2>/dev/null)'" d.log -"$MCPP" pack --format probe --no-strip > d2.log 2>&1 || fail "D: pack --no-strip failed" d2.log +"$MCPP" pack --pkg-format probe --no-strip > d2.log 2>&1 || fail "D: pack --no-strip failed" d2.log [ "$(cat "$probe")" = "strip=0 debug=" ] || fail "D: under --no-strip the program read '$(cat "$probe")'" d2.log echo "ok: D, a dispatched format's program reads the strip decision" @@ -194,7 +194,7 @@ int dep_answer(); int main() { return app_entry() == 43 ? 0 : 1; } EOF cd app - "$MCPP" pack --target x86_64-linux-android --format tar > e.log 2>&1 || fail "E: pack failed" e.log + "$MCPP" pack --target x86_64-linux-android --pkg-format tar > e.log 2>&1 || fail "E: pack failed" e.log grep -q "Packing app v0.1.0 (vendored, stripped)" e.log || fail "E: the Packing line" e.log rm -rf xe && mkdir xe && tar -xzf target/dist/app-0.1.0-x86_64-linux-android.tar.gz -C xe for f in libapp.so libdep.so libc++_shared.so; do diff --git a/tests/e2e/723_pack_reports_what_it_produced.sh b/tests/e2e/723_pack_reports_what_it_produced.sh index dd3bba3ac..61e12a1ba 100755 --- a/tests/e2e/723_pack_reports_what_it_produced.sh +++ b/tests/e2e/723_pack_reports_what_it_produced.sh @@ -11,11 +11,11 @@ # `mcpp build` given the same line built `dev`. # # Legs: -# A. `pack --format tar --message-format json`: stdout is one `mcpp.pack` +# A. `pack --pkg-format tar --message-format json`: stdout is one `mcpp.pack` # envelope and nothing else; the artifact path is absolute and exists, its # type is `file`, its format `tar`, its target the host triple; the staged # tree and its manifest exist, and the closure is `walked`. -# B. `pack --format dir --message-format json`: the artifact is a directory. +# B. `pack --pkg-format dir --message-format json`: the artifact is a directory. # C. `pack --release` and `pack --dev` are accepted; `--dev` builds `dev`. # D. `run --profile dev --release` builds `dev`, as `build` does. # E. `pack --message-format yaml` is refused with exit 2. @@ -42,7 +42,7 @@ printf '#include \nint main() { std::puts("app ran"); return 0; }\n' > a cd app # ── A ─────────────────────────────────────────────────────────────────────── -"$MCPP" pack --format tar --message-format json > a.json 2> a.err || fail "A: pack failed" a.err a.json +"$MCPP" pack --pkg-format tar --message-format json > a.json 2> a.err || fail "A: pack failed" a.err a.json python3 - a.json <<'EOF' || fail "A: the envelope" a.json a.err import json, os, sys text = open(sys.argv[1]).read() @@ -64,7 +64,7 @@ grep -q "Packed" a.err || fail "A: the human lines did not go to stderr" a.err echo "ok: A, one envelope on stdout names the archive, its legs and the staged tree" # ── B ─────────────────────────────────────────────────────────────────────── -"$MCPP" pack --format dir --message-format json > b.json 2> b.err || fail "B: pack failed" b.err +"$MCPP" pack --pkg-format dir --message-format json > b.json 2> b.err || fail "B: pack failed" b.err python3 - b.json <<'EOF' || fail "B: the envelope" b.json import json, os, sys a = json.load(open(sys.argv[1]))["data"]["artifacts"][0] @@ -78,13 +78,13 @@ echo "ok: B, a directory artifact is reported as one" # from an empty `target/`, so the one graph there is the one this pack wrote. built_dev() { grep -q -- " -O0" "$(ls target/*-linux-gnu/*/build.ninja | head -1)"; } rm -rf target -"$MCPP" pack --release --format tar > c1.log 2>&1 || fail "C: pack --release was refused" c1.log +"$MCPP" pack --release --pkg-format tar > c1.log 2>&1 || fail "C: pack --release was refused" c1.log built_dev && fail "C: pack --release built dev" c1.log rm -rf target -"$MCPP" pack --dev --format tar > c2.log 2>&1 || fail "C: pack --dev was refused" c2.log +"$MCPP" pack --dev --pkg-format tar > c2.log 2>&1 || fail "C: pack --dev was refused" c2.log built_dev || fail "C: pack --dev did not build dev" c2.log rm -rf target -"$MCPP" pack --profile release --dev --format tar > c3.log 2>&1 || fail "C: pack --profile --dev failed" c3.log +"$MCPP" pack --profile release --dev --pkg-format tar > c3.log 2>&1 || fail "C: pack --profile --dev failed" c3.log built_dev && fail "C: --dev won over --profile release on pack" c3.log echo "ok: C, pack takes --release and --dev, and --profile wins" @@ -97,7 +97,7 @@ echo "ok: D, run and build resolve --profile over the shorthands alike" # ── E ─────────────────────────────────────────────────────────────────────── set +e -"$MCPP" pack --format tar --message-format yaml > e.out 2> e.err +"$MCPP" pack --pkg-format tar --message-format yaml > e.out 2> e.err rc=$? set -e [ "$rc" -eq 2 ] || fail "E: --message-format yaml exited $rc" e.err diff --git a/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh b/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh index 246304739..957baf696 100755 --- a/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh +++ b/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh @@ -22,7 +22,7 @@ # on a build.mcpp CACHE HIT (not a re-run) must keep the RUNPATH and keep # `mcpp run` working -- the same replay criterion `mcpp:deploy=` is held # to in 651_a_build_program_deploys_what_it_generated.sh. -# 4. `mcpp pack --format dir` STAGES IT into the bundle's `lib/` -- the +# 4. `mcpp pack --pkg-format dir` STAGES IT into the bundle's `lib/` -- the # same closure search `runtime_search_dirs` already feeds. Checked # LAST: `mcpp pack` builds under a different profile and would otherwise # invalidate the `dev`-profile build.mcpp cache the replay check (3) @@ -129,8 +129,8 @@ grep -F -- "-Wl,-rpath,$RTDIR" "$G2" >/dev/null \ "$MCPP" run > run2.log 2>&1 \ || fail "run failed after the directive was replayed from a build.mcpp cache hit" run2.log b2.log -# ── 4. mcpp pack --format dir stages it into the bundle's lib/ ───────────── -"$MCPP" pack --format dir > pack1.log 2>&1 || fail "pack --format dir failed" pack1.log +# ── 4. mcpp pack --pkg-format dir stages it into the bundle's lib/ ───────────── +"$MCPP" pack --pkg-format dir > pack1.log 2>&1 || fail "pack --pkg-format dir failed" pack1.log STAGED=$(find target/dist -name 'libruntime_plugin.so' | head -1) [ -n "$STAGED" ] || fail "the runtime-search-dir closure was not staged by mcpp pack" pack1.log case "$STAGED" in diff --git a/tests/e2e/794_a_windows_program_finds_a_dll_through_runtime_search_dir.sh b/tests/e2e/794_a_windows_program_finds_a_dll_through_runtime_search_dir.sh index c11588952..8262b5d2a 100644 --- a/tests/e2e/794_a_windows_program_finds_a_dll_through_runtime_search_dir.sh +++ b/tests/e2e/794_a_windows_program_finds_a_dll_through_runtime_search_dir.sh @@ -99,7 +99,7 @@ rc=0; out="$("$MCPP" run 2>&1)" || rc=$? [[ $rc -eq 0 ]] || { echo "FAIL: mcpp run failed (rc=$rc): $out"; exit 1; } # ── 2. `mcpp pack` places the DLL beside the packed executable ────────────── -"$MCPP" pack --format dir > pack.log 2>&1 || { cat pack.log; echo "FAIL: pack failed"; exit 1; } +"$MCPP" pack --pkg-format dir > pack.log 2>&1 || { cat pack.log; echo "FAIL: pack failed"; exit 1; } PACKED_EXE="$(find target/dist -name 'app.exe' | head -1)" [[ -n "$PACKED_EXE" ]] || { find target/dist -type f; echo "FAIL: no packed app.exe"; exit 1; } [[ -f "$(dirname "$PACKED_EXE")/libmathkit.dll" ]] || { diff --git a/tests/e2e/796_runtime_search_dir_on_pe_under_wine.sh b/tests/e2e/796_runtime_search_dir_on_pe_under_wine.sh index 2399d9782..e0308d887 100644 --- a/tests/e2e/796_runtime_search_dir_on_pe_under_wine.sh +++ b/tests/e2e/796_runtime_search_dir_on_pe_under_wine.sh @@ -121,7 +121,7 @@ rc=0; out="$("$MCPP" run --target "$TRIPLE" 2>&1)" || rc=$? [[ $rc -eq 0 ]] || { echo "FAIL: mcpp run failed (rc=$rc): $out"; exit 1; } # ── 2. `mcpp pack` places the DLL beside the packed executable ───────────── -"$MCPP" pack --target "$TRIPLE" --format dir > pack.log 2>&1 || { cat pack.log; echo "FAIL: pack failed"; exit 1; } +"$MCPP" pack --target "$TRIPLE" --pkg-format dir > pack.log 2>&1 || { cat pack.log; echo "FAIL: pack failed"; exit 1; } PACKED_EXE="$(find target/dist -name 'app.exe' | head -1)" [[ -n "$PACKED_EXE" ]] || { find target/dist; echo "FAIL: no packed app.exe"; exit 1; } [[ -f "$(dirname "$PACKED_EXE")/libmathkit.dll" ]] || { diff --git a/tests/e2e/814_the_llvm_row_defaults_to_the_dynamic_crt.sh b/tests/e2e/814_the_llvm_row_defaults_to_the_dynamic_crt.sh index 15ec7883c..1bc9ebb71 100755 --- a/tests/e2e/814_the_llvm_row_defaults_to_the_dynamic_crt.sh +++ b/tests/e2e/814_the_llvm_row_defaults_to_the_dynamic_crt.sh @@ -144,7 +144,7 @@ echo "ok: switching A, B, A keeps both std BMI identities valid" # and not this fixture's row: it never wrote `cxx_runtime` down). # # `resolution.json` is a property of the BUILD, not of one `pack` invocation, -# so the observable here is the package's own contents: `--format dir` avoids +# so the observable here is the package's own contents: `--pkg-format dir` avoids # needing an unzip step to look inside. MARKER="$TMP/marker"; touch "$MARKER" pack_out=$("$MCPP" pack 2>&1) || fail "the default pack failed" <(echo "$pack_out") @@ -155,7 +155,7 @@ DIST=$(find target/dist -maxdepth 1 -mindepth 1 -newer "$MARKER" | head -1) || fail "the default pack did not carry vcruntime140.dll" <(echo "$pack_out"; echo "$DIST") touch "$MARKER" -system_out=$("$MCPP" pack --mode system --format dir 2>&1) \ +system_out=$("$MCPP" pack --mode system --pkg-format dir 2>&1) \ || fail "--mode system refused a defaulted (never-declared) toolchain-coupled contract" \ <(echo "$system_out") SYSTEM_DIST=$(find target/dist -maxdepth 1 -mindepth 1 -newer "$MARKER" | head -1) diff --git a/tests/e2e/820_a_windows_program_takes_its_runtime_from_one_resolver.sh b/tests/e2e/820_a_windows_program_takes_its_runtime_from_one_resolver.sh index bde5b4852..581fb1c61 100755 --- a/tests/e2e/820_a_windows_program_takes_its_runtime_from_one_resolver.sh +++ b/tests/e2e/820_a_windows_program_takes_its_runtime_from_one_resolver.sh @@ -175,7 +175,7 @@ echo "ok: W4 a runtime file declared under host-coupled is refused" # ── W5 ────────────────────────────────────────────────────────────────────── touch marker -pack_out=$(cd w1 && "$MCPP" pack --format dir 2>&1) || fail "W5: mcpp pack failed" <(echo "$pack_out") +pack_out=$(cd w1 && "$MCPP" pack --pkg-format dir 2>&1) || fail "W5: mcpp pack failed" <(echo "$pack_out") DIST=$(find w1/target/dist -maxdepth 1 -mindepth 1 -newer marker | head -1) [[ -d "$DIST" ]] || fail "W5: no package directory" <(echo "$pack_out"; find w1/target/dist) packed=0 diff --git a/tests/e2e/841_pack_summarizes_many_outputs.sh b/tests/e2e/841_pack_summarizes_many_outputs.sh index e803c376f..0f1ebb380 100644 --- a/tests/e2e/841_pack_summarizes_many_outputs.sh +++ b/tests/e2e/841_pack_summarizes_many_outputs.sh @@ -57,7 +57,7 @@ int main() { } EOF -"$MCPP" pack --format tree > p1.log 2>&1 || fail "pack failed" p1.log +"$MCPP" pack --pkg-format tree > p1.log 2>&1 || fail "pack failed" p1.log # P1 n=$(grep -c "Packed" p1.log) @@ -67,12 +67,12 @@ grep -qE "Packed .*tree[/\\\\]data \(6 files\)" p1.log || fail "P1: data/ is not grep -qE "Packed .*tree[/\\\\]manifest$" p1.log || fail "P1: the single file is not named" p1.log # P2 -"$MCPP" pack --format tree --verbose > p2.log 2>&1 || fail "pack --verbose failed" p2.log +"$MCPP" pack --pkg-format tree --verbose > p2.log 2>&1 || fail "pack --verbose failed" p2.log n=$(grep -c "Packed" p2.log) [ "$n" = 13 ] || fail "P2: $n Packed lines under --verbose, expected 13" p2.log # P3 -"$MCPP" pack --format tree --message-format json > p3.json 2> p3.err || fail "pack json failed" p3.err +"$MCPP" pack --pkg-format tree --message-format json > p3.json 2> p3.err || fail "pack json failed" p3.err python3 - p3.json <<'EOF' || fail "P3" p3.json import json, sys a = json.load(open(sys.argv[1]))["data"]["artifacts"] diff --git a/tests/e2e/867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh b/tests/e2e/867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh index 009f8950f..d5cc1b6a2 100644 --- a/tests/e2e/867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh +++ b/tests/e2e/867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh @@ -17,7 +17,7 @@ # and the staged tree it was told. # # Criteria (#749, A to D): -# A. `mcpp pack --workspace --format zap` runs `core`'s program once in all, and +# A. `mcpp pack --workspace --pkg-format zap` runs `core`'s program once in all, and # each provider's program once for each of the two passes a dispatched # format prepares: once when no format is asked, and once when it is. # B. Each member's packed tree has the same files with the same content as @@ -144,7 +144,7 @@ stage_of() { ls -d "$1"/target/dist/"$1"-0.1.0-* | grep -v -e '\.stage-manifest$ objects() { awk -F'\t' '$4 ~ /\.o$/' "$1/.ninja_log" | wc -l | tr -d ' '; } # ── A ──────────────────────────────────────────────────────────────────────── -"$MCPP" pack --workspace --format zap > a.log 2>&1 || fail "A: the pack failed" a.log +"$MCPP" pack --workspace --pkg-format zap > a.log 2>&1 || fail "A: the pack failed" a.log # `core` is not a program, so it is not packed, and its program ran once in all: # in the pass that asks for no format, and never again for a member whose answer # does not depend on the request. @@ -181,7 +181,7 @@ for m in cli gui; do cp -R "$(stage_of $m)" multi/$m/tree cp "$(stage_of $m).stage-manifest" multi/$m/stage-manifest cp $(outdir $m)/$m.zap multi/$m/dist - "$MCPP" pack -p $m --format zap > b-$m.log 2>&1 || fail "B: mcpp pack -p $m failed" b-$m.log + "$MCPP" pack -p $m --pkg-format zap > b-$m.log 2>&1 || fail "B: mcpp pack -p $m failed" b-$m.log cp -R "$(stage_of $m)" single/$m/tree cp "$(stage_of $m).stage-manifest" single/$m/stage-manifest cp $(outdir $m)/$m.zap single/$m/dist @@ -196,7 +196,7 @@ dirs=$(find target -name build.ninja -not -path '*/.build-mcpp/*' -exec dirname [ "$(echo "$dirs" | wc -l | tr -d ' ')" = 1 ] || fail "C: expected one build directory, got: $dirs" c-build.log before=$(objects "$dirs") [ "$before" -gt 0 ] || fail "C: the build compiled nothing" "$dirs/.ninja_log" -"$MCPP" pack --workspace --format zap > c-pack.log 2>&1 || fail "C: the pack failed" c-pack.log +"$MCPP" pack --workspace --pkg-format zap > c-pack.log 2>&1 || fail "C: the pack failed" c-pack.log [ "$(find target -name build.ninja -not -path '*/.build-mcpp/*' | wc -l | tr -d ' ')" = 1 ] || fail "C: the pack used another build directory" c-pack.log after=$(objects "$dirs") [ "$after" = "$before" ] || fail "C: the pack compiled $((after - before)) object(s) after the build" "$dirs/.ninja_log" c-pack.log diff --git a/tests/e2e/868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh b/tests/e2e/868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh index 069d48eac..a9873e63c 100644 --- a/tests/e2e/868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh +++ b/tests/e2e/868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh @@ -113,7 +113,7 @@ printf 'export module zlib;\nexport int f() { return 1; }\n' > lib/src/lib.cppm # ── E ──────────────────────────────────────────────────────────────────────── set +e -"$MCPP" pack --mode system --workspace --format zap > e.log 2>&1 +"$MCPP" pack --mode system --workspace --pkg-format zap > e.log 2>&1 rc=$? set -e [ "$rc" -ne 0 ] || fail "E: a member without the format was packed" e.log @@ -238,13 +238,13 @@ for m in cli2 gui2; do program_member $m "prov = { path = \"$PROV_HOST\" }" done set +e -"$MCPP" pack --mode system --workspace --format zap > j.log 2>&1 +"$MCPP" pack --mode system --workspace --pkg-format zap > j.log 2>&1 rc=$? set -e [ "$rc" -ne 0 ] && grep -q "member 'cli2'" j.log && grep -q "member 'gui2'" j.log && grep -q "prov" j.log \ || fail "J: members served only by a shared provider were not refused by name" j.log compiled_nothing . || fail "J: the refusal arrived after a compile" j.log -"$MCPP" pack --mode system -p cli2 --format zap > j2.log 2>&1 || fail "J: one member over the same provider was not packed" j2.log +"$MCPP" pack --mode system -p cli2 --pkg-format zap > j2.log 2>&1 || fail "J: one member over the same provider was not packed" j2.log provided=$(find . -name prov.zap | head -1) [ -n "$provided" ] && grep -qx "./bin/cli2" "$provided" \ || fail "J: the provider was not given the member's tree" j2.log @@ -261,7 +261,7 @@ write_dist . program_member cli3 "prov = { path = \"$PROV_HOST\" }" program_member gui3 "" provider gui3 submit -"$MCPP" pack --mode system --workspace --format zap > j3.log 2>&1 || fail "J: a provider one member reaches did not serve it beside another member's own" j3.log +"$MCPP" pack --mode system --workspace --pkg-format zap > j3.log 2>&1 || fail "J: a provider one member reaches did not serve it beside another member's own" j3.log provided=$(find . -name prov.zap | head -1) [ -n "$provided" ] && grep -qx "./bin/cli3" "$provided" && ! grep -q "gui3" "$provided" \ || fail "J: the provider was not given cli3's tree alone" j3.log @@ -283,11 +283,11 @@ provider good submit program_member mute "" provider mute silent set +e -"$MCPP" pack --mode system --workspace --format zap > k.log 2>&1 +"$MCPP" pack --mode system --workspace --pkg-format zap > k.log 2>&1 rc=$? set -e [ "$rc" -ne 0 ] || fail "K: a member that claimed nothing was reported as packed" k.log -grep -q "member 'mute': no action claimed --format 'zap'" k.log \ +grep -q "member 'mute': no action claimed --pkg-format 'zap'" k.log \ || fail "K: the failure does not name the member" k.log [ -n "$(find target/.build-mcpp/out -path '*/good/good.zap' | head -1)" ] || fail "K: the member that claimed the format was not packed" k.log grep -q "Packed .*good.zap" k.log || fail "K: the member that was packed was not reported" k.log diff --git a/tests/e2e/869_a_pack_over_several_members_names_each_member_in_its_report.sh b/tests/e2e/869_a_pack_over_several_members_names_each_member_in_its_report.sh index 0e333989c..8b7df095a 100644 --- a/tests/e2e/869_a_pack_over_several_members_names_each_member_in_its_report.sh +++ b/tests/e2e/869_a_pack_over_several_members_names_each_member_in_its_report.sh @@ -113,7 +113,7 @@ for m in cli gui; do || fail "M: $m's archive is not below the directory --output names" m1.log done [ "$(find out -maxdepth 1 -name '*.tar.gz' | wc -l | tr -d ' ')" = 2 ] || fail "M: the directory holds more than the two archives" m1.log -"$MCPP" pack --mode system --workspace --format dir -o out-trees > m2.log 2>&1 || fail "M: --format dir with --output did not pack" m2.log +"$MCPP" pack --mode system --workspace --pkg-format dir -o out-trees > m2.log 2>&1 || fail "M: --pkg-format dir with --output did not pack" m2.log for m in cli gui; do t=$(find out-trees -maxdepth 1 -type d -name "$m-0.1.0-*" | head -1) [ -n "$t" ] && [ -f "$t/bin/$m" ] || fail "M: $m's tree is not below the directory --output names" m2.log diff --git a/tests/e2e/870_a_pack_over_several_members_resolves_and_fails_per_member.sh b/tests/e2e/870_a_pack_over_several_members_resolves_and_fails_per_member.sh index 8ca4e9258..f75e98e0b 100644 --- a/tests/e2e/870_a_pack_over_several_members_resolves_and_fails_per_member.sh +++ b/tests/e2e/870_a_pack_over_several_members_resolves_and_fails_per_member.sh @@ -99,7 +99,7 @@ member c ctool c++20 zap() { ls -d target/.build-mcpp/out/*/"$1"/"$1".zap 2>/dev/null | head -1; } # ── A ──────────────────────────────────────────────────────────────────────── -"$MCPP" pack --workspace --format zap > a.log 2>&1 || fail "A: the pack failed" a.log +"$MCPP" pack --workspace --pkg-format zap > a.log 2>&1 || fail "A: the pack failed" a.log for m in a b c; do [ -x "$(zap $m)" ] || fail "A: no distributable for $m" a.log [ "$("$(zap $m)")" = "$m" ] || fail "A: $m's distributable runs '$("$(zap $m)")', not $m's program" a.log @@ -107,7 +107,7 @@ done echo "ok: A, each member's distributable is its own program, although a and b both name the target tool" # ── B ──────────────────────────────────────────────────────────────────────── -"$MCPP" pack --workspace --format zap --message-format json > b.json 2> b.err \ +"$MCPP" pack --workspace --pkg-format zap --message-format json > b.json 2> b.err \ || fail "B: the JSON pack failed" b.err b.json order=$(python3 -c 'import json,sys d = json.load(open(sys.argv[1])) @@ -118,7 +118,7 @@ echo "ok: B, the stages are listed in [workspace] members order across configura # ── C ──────────────────────────────────────────────────────────────────────── touch fail-b rc=0 -"$MCPP" pack --workspace --format zap > c.log 2>&1 || rc=$? +"$MCPP" pack --workspace --pkg-format zap > c.log 2>&1 || rc=$? [ "$rc" -ne 0 ] || fail "C: the pack exited 0 although b's step failed" c.log for m in a c; do [ -x "$(zap $m)" ] && [ "$("$(zap $m)")" = "$m" ] || fail "C: $m was not packed beside b's failure" c.log diff --git a/tests/e2e/871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh b/tests/e2e/871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh index 02218b517..db043f2cc 100644 --- a/tests/e2e/871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh +++ b/tests/e2e/871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh @@ -12,7 +12,7 @@ # and the open report. # # Criteria: -# A. A cold `mcpp pack --format tar` writes a `Compiling` line for the +# A. A cold `mcpp pack --pkg-format tar` writes a `Compiling` line for the # package and `Finished` before `Packing`. # B. Under `--message-format json`, standard output is one JSON document. # C. With `[build] jobs = 1`, no two compile steps overlap in `.ninja_log` @@ -84,7 +84,7 @@ no_overlap() { } # ── A ────────────────────────────────────────────────────────────────────── -"$MCPP" pack --format tar > a.log 2>&1 || fail "A: pack failed" a.log +"$MCPP" pack --pkg-format tar > a.log 2>&1 || fail "A: pack failed" a.log grep -q 'Compiling app v0.1.0' a.log || fail "A: the pack's build wrote no Compiling line" a.log fin=$(grep -n 'Finished' a.log | head -1 | cut -d: -f1) pk=$(grep -n 'Packing' a.log | head -1 | cut -d: -f1) @@ -94,7 +94,7 @@ echo "ok: A, the pack states its build and Finished precedes Packing" # ── B ────────────────────────────────────────────────────────────────────── rm -rf target -"$MCPP" pack --format tar --message-format json > b.json 2> b.err || fail "B: pack failed" b.err +"$MCPP" pack --pkg-format tar --message-format json > b.json 2> b.err || fail "B: pack failed" b.err python3 -c 'import json, sys; json.load(open(sys.argv[1]))' b.json \ || fail "B: standard output is not one JSON document" b.json b.err echo "ok: B, the JSON output is one document" @@ -107,7 +107,7 @@ rm -rf target; touch "$TMP/stamp"; sleep 1 "$MCPP" test > c2.log 2>&1 || fail "C: test failed" c2.log no_overlap "C test" rm -rf target; touch "$TMP/stamp"; sleep 1 -"$MCPP" pack --format tar > c3.log 2>&1 || fail "C: pack failed" c3.log +"$MCPP" pack --pkg-format tar > c3.log 2>&1 || fail "C: pack failed" c3.log no_overlap "C pack" echo "ok: C, build, test and pack each run one compile at a time under jobs = 1" diff --git a/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh b/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh index c6d64f5ee..3cd77617d 100755 --- a/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh +++ b/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh @@ -3,12 +3,13 @@ # 942 -- an ok image is built for `-openkal` and run by okrun (design # 2026-10-26 part 2, §8.5; openkal SPEC clause 12). # -# The target compiles for no operating system (`-unknown-none-elf`) with -# the convention's flags, links with `ld.lld` into an ET_DYN image entered at -# `ok_main`, and publishes the image only after `okld note` has written its -# requirements and `okld check` has checked every clause. `okld` and `okrun` are -# programs of the graph: `openkal-tools`, built for this machine through -# `[build-dependencies]`. `okrun` is the target's runner. +# The target's platform, openkal-platform (SPEC-013), compiles for no operating +# system (`-unknown-none-elf`) with the convention's flags, links with +# `ld.lld` into an ET_DYN image entered at `ok_main`, and publishes the image +# only after its seal -- `okld note`, then `okld check` -- has accepted it. +# `okld` and `okrun` are the platform's tools, built for this machine; `okrun` +# is its runner. The image here has no C library and no C++ run-time: the +# project switches the platform's layers off. # # Criteria: # A. Both rows build: `x86_64-openkal` and `aarch64-openkal` each produce @@ -20,15 +21,14 @@ # C. An image that imports a C library's name (`puts`) is refused by the # check, which names the name; the image under `bin/` is still the one the # last successful build published. -# D. `import std;` with no C++ run-time in the graph is refused before -# anything compiles, naming the packages that provide one. +# D. `import std;` with the platform's C++ run-time switched off is refused +# before anything compiles, saying it was switched off and where. # E. A second build is up to date. # F. `mcpp pack` stages the image (M2), and `--mode self-contained` stages a # program of this machine that carries it and runs with nothing installed # (M3), its image given the program's arguments and environment. -# G. A workspace's root package declares the tools as any root does: the -# image builds and runs through okrun (the selected package is not the -# graph's first, and its tools were not recorded as the root's). +# G. A workspace's root package is built as any root is: the image builds +# and runs through okrun (the selected package is not the graph's first). set -e TMP=$(mktemp -d) @@ -41,7 +41,11 @@ case "$(uname -m)" in *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; esac -TOOLS='openkal-tools = { version = "0.1.0", tools = ["okld", "okrun"] }' +# No C library, no C++ run-time, no compiler run-time: the image is its own. +NO_LAYERS='[target.'"'"'cfg(platform = "openkal")'"'"'.layers] +c-abi = false +"c++-abi" = false +compiler-runtime = false' mkdir -p "$TMP/app/src" cd "$TMP/app" @@ -54,8 +58,7 @@ version = "0.1.0" kind = "bin" main = "src/main.c" -[build-dependencies] -$TOOLS +$NO_LAYERS TOML cat > src/main.c <<'C' /* An image with no C library: it calls kal_* and supplies the two routines @@ -153,9 +156,9 @@ grep -q "Compiling" again.log && fail "E: the second build compiled again" again echo "ok: a second build is up to date" # F -"$MCPP" pack --target "$HOST_ARCH-openkal" --format dir -o m2 > m2.log 2>&1 || fail "F: pack (M2)" m2.log +"$MCPP" pack --target "$HOST_ARCH-openkal" --pkg-format dir -o m2 > m2.log 2>&1 || fail "F: pack (M2)" m2.log [ -f target/dist/m2/bin/app.oke ] || fail "F: the M2 package has no bin/app.oke" m2.log -"$MCPP" pack --target "$HOST_ARCH-openkal" --mode self-contained --format dir -o m3 > m3.log 2>&1 \ +"$MCPP" pack --target "$HOST_ARCH-openkal" --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 \ || fail "F: pack --mode self-contained (M3)" m3.log m3=$(ls target/dist/m3/bin/app target/dist/m3/bin/app.exe 2>/dev/null | head -1) [ -n "$m3" ] || fail "F: the M3 package has no program" m3.log @@ -176,7 +179,7 @@ if "$MCPP" build --target "$HOST_ARCH-openkal" > refuse.log 2>&1; then fail "C: an image importing puts was published" refuse.log fi grep -q "puts" refuse.log || fail "C: the refusal does not name the import" refuse.log -grep -q "ok-image convention" refuse.log || fail "C: the refusal does not name the convention" refuse.log +grep -q "was not sealed" refuse.log || fail "C: the refusal does not name the seal" refuse.log after=$(cksum < "$(ls target/"$HOST_ARCH"-openkal/*/bin/app.oke | head -1)" ) [ "$before" = "$after" ] || fail "C: the refused image took the name in bin/" refuse.log echo "ok: an image that imports a C library's name is refused and not published" @@ -189,16 +192,19 @@ cat > mcpp.toml < src/main.cpp if "$MCPP" build --target "$HOST_ARCH-openkal" > std.log 2>&1; then fail "D: import std built without a C++ run-time" std.log fi -grep -q "needs a C++ run-time inside the image" std.log || fail "D: the refusal's reason" std.log -grep -q "openkal-llvm-runtime" std.log || fail "D: the refusal does not name the provider" std.log -echo "ok: import std without a C++ run-time is refused, naming the provider" +grep -q "needs a C++ run-time inside the program" std.log || fail "D: the refusal's reason" std.log +grep -q "switched off" std.log || fail "D: the refusal does not say the layer was switched off" std.log +grep -q "mcpp.toml" std.log || fail "D: the refusal does not say where" std.log +echo "ok: import std with the C++ run-time switched off is refused, saying where" # G mkdir -p "$TMP/ws/src" "$TMP/ws/m/src" @@ -215,14 +221,13 @@ version = "0.1.0" kind = "bin" main = "src/main.c" -[build-dependencies] -$TOOLS +$NO_LAYERS TOML printf '[package]\nname = "m"\nversion = "0.1.0"\n\n[targets.m]\nkind = "lib"\n' > "$TMP/ws/m/mcpp.toml" printf 'int m_value(void) { return 0; }\n' > "$TMP/ws/m/src/m.c" (cd "$TMP/ws" && "$MCPP" run --target "$HOST_ARCH-openkal" -- one two > run.log 2>&1) \ || fail "G: a workspace root's image did not build or run" "$TMP/ws/run.log" grep -q "image: three arguments" "$TMP/ws/run.log" || fail "G: the workspace root's image" "$TMP/ws/run.log" -echo "ok: a workspace root declares the tools as any root does" +echo "ok: a workspace root's image is built and run as any root's" echo "PASS: 942 an ok image is built once and run by okrun" diff --git a/tests/e2e/951_an_ok_image_needs_no_declaration.sh b/tests/e2e/951_an_ok_image_needs_no_declaration.sh new file mode 100755 index 000000000..2ec1e672c --- /dev/null +++ b/tests/e2e/951_an_ok_image_needs_no_declaration.sh @@ -0,0 +1,138 @@ +#!/usr/bin/env bash +# requires: llvm python3 +# 951 -- `-openkal` needs no declaration: the target's platform supplies +# the C library, the C++ run-time, the tools and the runner (SPEC-013; design +# 2026-10-28 §7, §10). +# +# The target table pins `openkal-platform` for both ok-image rows. A project +# that names nothing but its sources builds a C++ image with `import std`, runs +# it, tests it and packs it; what the platform added is reported as the +# platform's and locked as such; the overrides take their place by the rules +# of SPEC-013 §4.3-§4.5. +# +# Criteria: +# A. `mcpp new`-shaped project, `mcpp run --target -openkal`: the +# image runs through okrun, with exceptions and the standard library. +# B. The report names the platform and its source, marks the layers it added +# `platform`, and the toolchain line says the toolchain is the platform's. +# C. mcpp.lock records the packages the platform added with +# `origin = "platform"`. +# D. `mcpp test` builds and runs a test image; `mcpp pack` stages the image +# and its loader-run program (M3). +# E. `mcpp self env --target … --format json` names the platform, its fields' +# sources and the runner's program; `mcpp toolchain list --format json` +# gives the row's platform. +# F. A package of the graph that provides a layer takes the platform's place +# (reported `graph`); a version of the family the platform's group does not +# name is refused, naming the group and both remedies. +# G. `--platform none` selects no platform for the target. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64) HOST_ARCH=x86_64 ;; + aarch64|arm64) HOST_ARCH=aarch64 ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac +T="$HOST_ARCH-openkal" + +mkdir -p "$TMP/app/src" "$TMP/app/tests" +cd "$TMP/app" +printf '[package]\nname = "app"\nversion = "0.1.0"\n' > mcpp.toml +cat > src/main.cpp <<'CPP' +import std; +int main(int argc, char** argv) { + try { throw std::runtime_error("caught"); } + catch (const std::exception& e) { std::println("image: {} ({} args)", e.what(), argc); } + return 0; +} +CPP +cat > tests/smoke.cpp <<'CPP' +import std; +int main() { std::vector v{2, 1}; std::ranges::sort(v); return v.front() == 1 ? 0 : 1; } +CPP + +# A +"$MCPP" run --target "$T" -- one two > run.log 2>&1 || fail "A: the image did not build or run" run.log +grep -q "image: caught (3 args)" run.log || fail "A: the image's output" run.log +echo "ok: a project that declares nothing runs as an ok image" + +# B +"$MCPP" build --target "$T" -v > build.log 2>&1 || fail "B: rebuild" build.log +rm -rf target +"$MCPP" build --target "$T" > fresh.log 2>&1 || fail "B: fresh build" fresh.log +# The source is the target table's column, or the variable a run before the +# platform's release selects it with. +grep -qE "Platform +openkal-platform [0-9.]+ \((built-in|MCPP_TARGET_[A-Z0-9_]+_PLATFORM)\)" fresh.log || fail "B: no Platform line naming the source" fresh.log +grep -qE "c-abi +musl +\(openkal-musl@[0-9.]+, platform\)" fresh.log || fail "B: the C library is not reported as the platform's" fresh.log +grep -qE "c\+\+-abi +libc\+\+ +\(openkal-llvm-runtime@[0-9.]+, platform\)" fresh.log || fail "B: the C++ run-time is not the platform's" fresh.log +grep -qE "tools +okld, okrun" fresh.log || fail "B: the tools are not reported" fresh.log +echo "ok: the report names the platform and what it added" + +# C +grep -A5 '"openkal-musl"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: openkal-musl is not locked as the platform's" mcpp.lock +grep -A5 '"openkal-llvm-runtime"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: openkal-llvm-runtime is not locked as the platform's" mcpp.lock +echo "ok: the lock says which packages the platform added" + +# D +"$MCPP" test --target "$T" > test.log 2>&1 || fail "D: mcpp test" test.log +grep -q "smoke ... ok" test.log || fail "D: the test image did not pass" test.log +"$MCPP" pack --target "$T" --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 || fail "D: pack (M3)" m3.log +m3=$(ls target/dist/m3/bin/app target/dist/m3/bin/app.exe 2>/dev/null | head -1) +[ -n "$m3" ] || fail "D: the M3 package has no program" m3.log +"$m3" x > m3run.log 2>&1 || fail "D: the M3 program did not run" m3run.log +grep -q "image: caught (2 args)" m3run.log || fail "D: the M3 program's image" m3run.log +echo "ok: test and pack work with no declaration" + +# E +"$MCPP" self env --target "$T" --format json > env.json 2> env.err || fail "E: self env" env.err env.json +python3 - env.json <<'PY' || fail "E: self env --format json" env.json +import json, sys +d = json.load(open(sys.argv[1]))["data"] +p = d["platform"] +assert p and p["name"].endswith("openkal-platform"), p +assert p["source"] == "built-in" or p["source"].startswith("MCPP_TARGET_"), p["source"] +assert p["fields"]["runner"]["source"] == "platform" +assert p["fields"]["layers.c-abi"]["source"] == "platform" +assert d["runner"]["program"] and d["runner"]["program"].replace("\\", "/").split("/")[-1].startswith("okrun"), d["runner"] +PY +"$MCPP" toolchain list --format json > list.json 2>/dev/null || fail "E: toolchain list" list.json +python3 - list.json "$T" <<'PY' || fail "E: toolchain list's platform column" list.json +import json, sys +rows = [r for r in json.load(open(sys.argv[1]))["data"]["targets"] if r["target"] == sys.argv[2]] +assert rows and rows[0]["platform"].split("@")[0].endswith("openkal-platform"), rows +PY +echo "ok: self env and toolchain list state the platform" + +# F +mkdir -p "$TMP/own/src" +cd "$TMP/own" +cat > mcpp.toml <<'TOML' +[package] +name = "own" +version = "0.1.0" + +[dependencies] +openkal-musl = { version = "0.22.2", default-features = false, features = ["posix"] } +TOML +cp "$TMP/app/src/main.cpp" src/main.cpp +if "$MCPP" build --target "$T" > group.log 2>&1; then + fail "F: a version outside the platform's group was built" group.log +fi +grep -q "version group" group.log || fail "F: the refusal does not name the group" group.log +grep -q "0.22.3" group.log || fail "F: the refusal does not name the group's version" group.log +echo "ok: a version outside the platform's group is refused" + +# G +"$MCPP" self env --target "$T" --platform none --format json > none.json 2> none.err || fail "G: self env --platform none" none.err +python3 - none.json <<'PY' || fail "G: --platform none still selected a platform" none.json +import json, sys +d = json.load(open(sys.argv[1]))["data"] +assert d["platform"] is None, d["platform"] +PY +echo "ok: --platform none selects no platform" + +echo "PASS: 951 an ok image needs no declaration" diff --git a/tests/unit/test_build_progress.cpp b/tests/unit/test_build_progress.cpp index 3b80e38af..f0fc2d0b9 100644 --- a/tests/unit/test_build_progress.cpp +++ b/tests/unit/test_build_progress.cpp @@ -458,7 +458,9 @@ TEST(ProgressModel, AStepWhoseEntriesAreReadInTwoPiecesCountsOnce) { mcpp::ui::disable_color(); Tmp tmp; Record rec; - rec.packages = {{"lib", true, "lib", 0, 2, "v0.1.0 (lib)", "project"}}; + // The report is the process's, as it is the command's: a name no other + // test of this binary announced. + rec.packages = {{"mlib", true, "mlib", 0, 2, "v0.1.0 (mlib)", "project"}}; rec.owner = {{"obj/lib.m.o", 0}, {"pcm.cache/lib.pcm", 0}, {"bin/lib.a", 0}}; rec.step = {{"obj/lib.m.o", 0}, {"pcm.cache/lib.pcm", 0}, {"bin/lib.a", 1}}; Build b(tmp.path); @@ -473,7 +475,30 @@ TEST(ProgressModel, AStepWhoseEntriesAreReadInTwoPiecesCountsOnce) { b.pass_end(); b.finish(true); auto out = testing::internal::GetCapturedStderr(); - EXPECT_EQ(count(out, "Compiling lib"), 1u) << out; + EXPECT_EQ(count(out, "Compiling mlib"), 1u) << out; +} + +TEST(ProgressModel, APackageTwoConfigurationsShareIsAnnouncedOncePerCommand) { + // A workspace builds one plan per configuration; a dependency they share + // is the same line in each, and the command writes it once (L6). + mcpp::ui::disable_color(); + testing::internal::CaptureStderr(); + for (int config = 0; config < 2; ++config) { + Tmp tmp; + Record rec; + rec.packages = {{"shared", true, "shared", 0, 1, "v0.2.0 (shared)", "project"}}; + rec.owner = {{"obj/s.o", 0}}; + rec.step = {{"obj/s.o", 0}}; + Build b(tmp.path); + b.set_record(rec); + b.pass_begin(); + append(tmp.path / ".ninja_log", "# ninja log v6\n1\t10\t0\tobj/s.o\taa\n"); + b.status({1, 1, 10, {}, {}}); + b.pass_end(); + b.finish(true); + } + auto out = testing::internal::GetCapturedStderr(); + EXPECT_EQ(count(out, "Compiling shared v0.2.0 (shared)"), 1u) << out; } // `Finished` is written once per command. `mcpp run --format ` packs, and From df41c53f04ca6ff6eb109939b740b1f53838608a Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 05:15:18 +0900 Subject: [PATCH 11/54] platform: why toolchain says platform for a layer the platform added; the matrix's ok-image rows build (payload) or meet the version group (graph); retarget step in its own file (toolchain.cpp under the length limit) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .agents/docs/2026-10-28-platforms-design.md | 2 +- ...part2-packaging-and-distribution-design.md | 2 +- .agents/docs/README.md | 9 +- src/build/prepare/retarget.cpp | 411 ++++++++++++++++++ src/build/prepare/state.cppm | 2 + src/build/prepare/toolchain.cpp | 350 --------------- src/doctor.cppm | 4 + tests/matrix/expected.tsv | 35 +- 8 files changed, 445 insertions(+), 370 deletions(-) create mode 100644 src/build/prepare/retarget.cpp diff --git a/.agents/docs/2026-10-28-platforms-design.md b/.agents/docs/2026-10-28-platforms-design.md index 88f95ae03..5450b1f83 100644 --- a/.agents/docs/2026-10-28-platforms-design.md +++ b/.agents/docs/2026-10-28-platforms-design.md @@ -1,6 +1,6 @@ --- subject: platform -status: accepted +status: active --- # 平台：目标体系的通用扩展；openkal 作为第一个平台包；IDE 一线与 mcppls 的同批升级 diff --git a/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md b/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md index a249004c6..760c1229f 100644 --- a/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md +++ b/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md @@ -1,6 +1,6 @@ --- subject: platform -status: accepted +status: active --- # 平台 part 2：打包与分发——三维模型、通用包、单文件、okb、体积，以及 openkal 0.17.2 轮 diff --git a/.agents/docs/README.md b/.agents/docs/README.md index e6d722b61..f2ac87124 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -338 records. +340 records. ## By subject @@ -91,6 +91,11 @@ Records that declare one. Everything else is listed by date below. - [openkal 生态：完整性收尾与验收方案](2026-09-21-openkal-ecosystem-completion-and-acceptance.md) — active - [C 环境生态方案：执行计划](2026-09-20-ecosystem-execution-plan.md) — active +### platform + +- [平台 part 2：打包与分发——三维模型、通用包、单文件、okb、体积，以及 openkal 0.17.2 轮](2026-10-28-platforms-part2-packaging-and-distribution-design.md) — active +- [平台：目标体系的通用扩展；openkal 作为第一个平台包；IDE 一线与 mcppls 的同批升级](2026-10-28-platforms-design.md) — active + ### plugins - [The category the plugin taxonomy does not name, and what a platform actually decomposes into](2026-09-11-distribution-plugins-and-platform-decomposition.md) — active @@ -144,6 +149,8 @@ Records that declare one. Everything else is listed by date below. ### 2026-10 +- [平台 part 2：打包与分发——三维模型、通用包、单文件、okb、体积，以及 openkal 0.17.2 轮](2026-10-28-platforms-part2-packaging-and-distribution-design.md) — active +- [平台：目标体系的通用扩展；openkal 作为第一个平台包；IDE 一线与 mcppls 的同批升级](2026-10-28-platforms-design.md) — active - [生态设计 part 2：openkal 的最小原子与二进制映像、其上的 `mcpplibs.os`，以及 part 1 的收尾与回归记录](2026-10-26-ecosystem-part2-os-library-over-openkal-design.md) — active - [控制契约、并发与性能、单一 C++ 运行时，以及 IDE 生态（生态级，#838、#839）](2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md) — active - [目标基座与执行宿主：sysroot 的通用模型、运行器契约、自备输入，以及同批的发布产物路径（#832）](2026-10-23-target-base-and-execution-host-design.md) — landed diff --git a/src/build/prepare/retarget.cpp b/src/build/prepare/retarget.cpp new file mode 100644 index 000000000..e1ea16711 --- /dev/null +++ b/src/build/prepare/retarget.cpp @@ -0,0 +1,411 @@ +// retarget.cpp -- telling a retargetable driver what it targets (phase 2 of +// prepare_build): the target triple, the sysroot and the multilib directory a +// clang driver is given when the toolchain resolved for a cross target. Moved +// out of toolchain.cpp whole; declared in :state. + +module mcpp.build.prepare; +import :state; +import mcpp.build.prepare_inputs; +import std; +import mcpp.build.engine_package; +import mcpp.diag; +import mcpp.build.refusal; +import mcpp.build.version_floor; +import mcpplibs.os.platform; +import mcpp.toolchain.platform; +import mcpp.toolchain.apple_sdk; +import mcpp.effects; +import mcpp.build.command_line; +import mcpp.manifest; +import mcpp.source_kind; +import mcpp.toolchain.hostflags; // the compile-token producer the package std module reuses +import mcpp.toolchain.detect; +import mcpp.toolchain.dialect; +import mcpp.toolchain.fingerprint; +import mcpp.toolchain.msvc; +import mcpp.toolchain.registry; +import mcpp.toolchain.linkmodel; +import mcpp.toolchain.lifecycle; +import mcpp.toolchain.stdmod; +import mcpp.freestanding.target; // the target sysroot layout (libdir) +import mcpp.freestanding.linkline; // the ISA profile, for the std module command +import mcpp.toolchain.post_install; +import mcpp.toolchain.abi; +import mcpp.toolchain.triple; +import mcpp.toolchain.base; +import mcpp.toolchain.base_probe; +import mcpp.home; +import mcpp.build.plan; +import mcpp.build.flags; // compute_flags — the per-role contracts (#418) +import mcpp.build.graph_shape; // #407: the graph says which mode wrote it +import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against +import mcpp.pack.prebuilt; // …and the check itself +import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its manifest +import mcpp.build.build_program; +import mcpp.build.backend; // BuildOptions for the tool sub-build +import mcpp.build.ninja; // make_ninja_backend — driving that sub-build +import mcpp.config; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.runtime_selection; +import mcpp.runtime.binding; +import mcpp.toolchain.post_install; +import mcpp.fetcher; +import mcpp.fetcher.progress; +import mcpp.ui; +import mcpp.invocation; + +namespace mcpp::build { + +std::expected +step2_retarget_for_retargetable_driver(PrepareState& state) { + // ── A retargetable driver has to be TOLD what it is targeting ──────── + // + // `tc.targetTriple` comes from `-dumpmachine`, and for every cross target + // that worked before this it was right for a reason that does not + // generalise: those targets use a DISTINCT compiler binary + // (`x86_64-w64-mingw32-g++`, `aarch64-linux-musl-g++`), whose own + // -dumpmachine reports the cross triple. Clang is ONE binary that emits + // every target it was built with, so -dumpmachine always answers with the + // host — and nothing downstream ever learns otherwise. + // + // Measured before this line existed: + // + // $ mcpp build --target riscv64-none-elf + // Resolved llvm@22.1.8 → riscv64-none-elf → …/bin/clang++ + // Finished dev [unoptimized + debuginfo] in 0.47s + // $ ls target/ + // x86_64-linux-gnu/ ← an ELF for the host, reported as riscv64 + // + // That is E1: success reported, host artifact produced. The output + // directory, the fingerprint, the cache key and the flag layer all read + // `tc.targetTriple`, so correcting it here corrects all of them at once — + // which is the point of there being one field rather than five answers. + // + // THIS USED TO BE SCOPED TO FREESTANDING, WITH THIS REASON: + // + // The hosted cross targets already resolve a per-target binary, and + // overwriting their probed triple would replace a measured fact with + // an assumed one for no gain. + // + // That was true while every hosted cross was served by a payload. It + // stops being true when the TARGET SIDE comes from the dependency graph: + // the C library, the C++ runtime and the platform's own implementation are + // then packages built from source, and the compiler is an ordinary clang — + // whose `-dumpmachine` answers the host, exactly as the paragraph above + // describes for freestanding. + // + // Measured 2026-08-23, with an explicit `[target.aarch64-macos] + // toolchain = "llvm@…"`. The manifest's cfg evaluation used the REQUESTED + // target, so the C library's aarch64 headers were on the command line; the + // toolchain's own triple was still the host's, so code generation was + // x86_64. Two answers to one question, in one command: + // + // okm_float_assert.c: the C library and the compiler disagree about + // LDBL_DIG ('33 == 18') 33 = aarch64 binary128, 18 = x87 + // + // ⇒ The condition is now the property the first paragraph of this comment + // already names: a RETARGETABLE driver has to be told. gcc is not one — a + // gcc payload IS its target — so the mingw and musl-gcc crosses keep + // answering from `-dumpmachine`, which for them remains a measured fact. + if (!state.overrides.target_triple.empty()) { + if (auto want = mcpp::toolchain::triple::parse(state.overrides.target_triple); + want && (want->is_freestanding() + || state.tc->compiler == mcpp::toolchain::CompilerId::Clang)) + { + state.tc->targetTriple = want->str(); + + // AND THE GATE THAT ALREADY EXISTS FOR THIS, APPLIED WHERE THE + // ANSWER IS KNOWN. + // + // `discover_link_runtime_dirs` refuses to report these + // directories for a target that carries its own sysroot, and the + // refusal never fired: that function runs during DETECTION, + // before this line, when `targetTriple` is still the HOST's. The + // gate read a host triple and answered correctly about it. + // + // The artefact is what showed it. An Android link line carried + // + // -L /toolchains/llvm/prebuilt/linux-x86_64/lib/ + // x86_64-unknown-linux-gnu + // + // whose last component is this machine's triple, produced by + // `root / "lib" / targetTriple` -- so the string names the + // question that was asked. Those are the compiler's own host + // runtime directories; an Android artefact must resolve libc++, + // the crt objects and the loader from the NDK's sysroot, and the + // hermetic check reported exactly that failure with six host + // objects. + // + // Cleared rather than re-derived. Re-running the discovery with + // the final triple would also change what every OTHER clang cross + // target gets, and those are measured as they stand; the claim + // being made here is only the one the gate already states. + if (want->has_own_sysroot()) state.tc->linkRuntimeDirs.clear(); + + // And the flag that says it to the driver — for a HOSTED target + // only. Freestanding already emits its own `--target`, together + // with the ISA flags that must accompany it + // (freestanding/target.cppm); a second one here would be the same + // decision in two places. + if (!want->is_freestanding() + && state.tc->compiler == mcpp::toolchain::CompilerId::Clang) { + state.tc->crossTargetFlag = + "--target=" + want->llvm_triple( + min_platform_version(*state.m, *want, state.tc->binaryPath)); + + // AND THE SAME FLAG ON THE std MODULE'S OWN COMMANDS, FOR A + // PAYLOAD THAT SERVES MORE THAN ONE TARGET. + // + // The std module is built by its own command assembly + // (clang.cppm), not by the compile flags, so a decision made + // only here reaches every translation unit and not that. For + // most toolchains the omission cannot be seen: a payload + // whose compiler IS its target finds its own headers, and a + // package-provided module carries the target inside + // `stdModuleFlags`. + // + // ONE NDK SERVES BOTH ANDROID ARCHES, which is the property + // that makes this necessary and is stated in the row's own + // pin: `android-ndk@` names no arch, so `--target` is the + // only thing that says which. Without it the precompile + // resolved libc++'s `#include <__config>` against the + // building machine and stopped there. + // + // NOT `has_own_sysroot()`, though both rows that answer true + // to it are SDKs with their own sysroot. Emscripten's `em++` + // serves exactly one target and needs no flag -- the verified + // wasm loop is measured without it -- so widening the gate to + // the predicate would add a flag to a command that does not + // want one. The property here is "one payload, several + // targets", and Android is the only row that has it; a future + // row brings its own measurement. + if (want->is_android()) { + state.tc->stdModuleTargetFlags = " " + state.tc->crossTargetFlag; + // BIONIC'S ctype HEADER AND A MODULE'S EXPORT RULES. + // + // bionic declares `isalnum` and its neighbours + // `static inline`, and libc++'s module surface exports + // them with `using std::isalnum`. A using-declaration + // cannot export a name with internal linkage, so the + // precompile fails on 14 names at once. Defining the + // macro empty makes those declarations extern, which is + // what every other C library this engine compiles + // against already does. + // + // Scoped to the std module and not to every unit: the + // rule being satisfied is about exporting from a module, + // and a translation unit that includes + // directly is entitled to bionic's inline definitions. + // `xim:android-ndk`'s own install-time self-test reaches + // the identical conclusion from the other direction. + // + // AND THE PAYLOAD MAY SAY SO ITSELF. The recipe applies + // this same define in that self-test, so it is a fact + // the payload already holds; `std_module_defines` in + // `.mcpp-toolchain.json` is the channel for it, and the + // define below is what a payload that ships no + // descriptor still gets. The two are not added + // together: a descriptor that names defines is the + // payload's complete answer for this channel, and + // appending to it would mean a payload could not + // withdraw a define this engine once needed. + auto stdDefines = [&]() -> std::vector { + auto desc = + mcpp::toolchain::payload_descriptor_for_compiler( + state.tc->binaryPath); + if (desc && *desc && !(*desc)->stdModuleDefines.empty()) + return (*desc)->stdModuleDefines; + return { "__BIONIC_CTYPE_INLINE=" }; + }(); + for (auto const& def : stdDefines) + state.tc->stdModuleTargetFlags += " -D" + def; + } + + // ── iOS: THE COMPILER IS OURS, THE SDK IS THE MACHINE'S ── + // + // The three iOS rows pin `llvm@23.1.3` -- any sufficiently + // new clang emits arm64 Mach-O for an iOS deployment target + // -- and take their headers and stub libraries from the + // machine's Xcode, which is where the whole item shrinks to + // a located sysroot. `aarch64-macos` is verified on exactly + // this split and is the precedent. + // + // LOCATED HERE, ONCE. Three later sites need the answer (the + // compile flags, the link line, and the std module's own + // command), and a function that probes the machine is the + // wrong thing to call three times: `xcrun` shells out, and + // three answers can differ if the developer directory + // changes mid-build. + // + // AND ITS ABSENCE IS A REFUSAL THAT NAMES THE SDK. The + // recorded host-surface rule is that a host dependency must + // be minimal, named, and never a fallthrough; the iOS SDK + // and `simctl` are the two this platform adds, both in the + // "proprietary runtime that exists only on its own OS" + // category. A build that continued without the SDK would + // fail in the driver's header search, naming a file rather + // than the thing that is missing. + if (want->is_ios()) { + // READ, NOT RE-DERIVED. The refusal above located it + // before any payload was resolved, and that is the one + // `xcrun` call this build makes. + // + // An empty answer here cannot happen through the + // `--target` path, and a line that prints when it does + // is cheaper than a branch that pretends it cannot: the + // row could be reached one day by a route that skipped + // the gate, and an iOS build with no `-isysroot` is a + // macOS artefact with an iOS triple on it. + if (!state.appleSdkLocated) { + return std::unexpected(std::format( + "internal: target {} reached toolchain " + "resolution without its SDK being located; the " + "gate that locates it did not run for this " + "request", want->str())); + } + state.tc->base.apple.root = *state.appleSdkLocated; + if (state.appleSdkFound) { + state.tc->base.apple.name = state.appleSdkFound->name; + state.tc->base.apple.version = state.appleSdkFound->version; + } + auto sdk = state.appleSdkLocated; + // AND THE std MODULE'S OWN COMMAND, WHICH IS A SEPARATE + // CHANNEL. Same reason the Android rows set it: the + // module is precompiled by `clang.cppm`'s own assembly + // rather than by the compile flags, so a decision made + // only in the flag builder reaches every translation + // unit and not the module they all import. Without the + // SDK here the precompile resolves libc++'s + // `#include <__config>` against the macOS SDK and the + // module is built for the wrong platform. + // + // QUOTED, as every path this string carries is (see the + // package-provided producer, which uses `shq` for each + // `-isystem`). The string is spliced into a shell + // command, and an Xcode installed as `Xcode 16.app` is + // a path with a space in it. + state.tc->stdModuleTargetFlags = + " " + state.tc->crossTargetFlag + + " -isysroot " + mcpp::build::command_line::quote(sdk->string()); + } + } + } + if (auto want = mcpp::toolchain::triple::parse(state.overrides.target_triple); + want && want->is_freestanding()) + { + // `import std` is structurally hosted, and turning it off is the + // SAME fact as the line above, not a second policy: libc++'s + // std.cppm is one module over the whole library, including the + // parts that are threads, filesystem and iostreams. There is no + // subset of it to precompile. + // + // Left on, the failure is neither early nor legible — measured: + // + // error: std module precompile failed (rc=1): + // .../include/c++/v1/__config:13:10: fatal error: + // '__config_site' file not found + // + // which reads as a broken toolchain payload and says nothing about + // the target. The freestanding std subset a user actually wants is + // an ordinary package (`mcpplibs.std.freestanding`), so mcpp's job + // here is to stop pretending the hosted one exists and to say + // where the other one is. + state.tc->clear_std_modules(); + + // ── The target's C library, resolved like its compiler ───────── + // + // The row in kKnownTargets names it, exactly as it names the + // toolchain pin, and it is installed through the same channel a + // project's `[xlings] deps` use (see the materialization above). + // Resolved HERE because the config is already open; the flag + // builder only reads the result. + // + // Absent is not an error at this point: the install happens + // earlier in this function and may legitimately not have run yet + // on a first pass. What follows would then simply not add the + // paths, and the link fails naming the missing libc — which is the + // truthful message either way. + const std::string want_sysroot = + mcpp::toolchain::triple::effective_sysroot( + *want, base_statement_text(state, *want)); + // A C library named by path (design 2026-10-23 §3.1): the same + // `/{include,lib}/` layout, nothing to install. + if (want_sysroot.starts_with("path:")) { + const std::filesystem::path dir = want_sysroot.substr(5); + std::error_code ec2; + if (!std::filesystem::is_directory(dir, ec2)) { + refusal::record(refusal::Code::TargetBase); + return std::unexpected(std::format( + "[target.{}].sysroot = {{ path = \"{}\" }} does not exist", want->str(), + dir.generic_string())); + } + if (auto spec = mcpp::freestanding::resolve(*want)) { + state.tc->base.multilibRoot = dir; + state.tc->base.package = dir.filename().string(); + const auto inc = dir / "include" / std::string(spec->libdir); + const auto lib = dir / "lib" / std::string(spec->libdir); + if (std::filesystem::is_directory(inc, ec2)) state.tc->base.multilibInclude = inc; + if (std::filesystem::is_directory(lib, ec2)) state.tc->base.multilibLib = lib; + } + } else if (!want_sysroot.empty()) { + if (auto cfg3 = state.get_cfg(true); cfg3) { + auto ref = mcpp::provider::parse_ref(want_sysroot); + auto xl = mcpp::config::make_xlings_env(**cfg3); + // INSTALLED, NOT MERELY LOOKED UP — THE SAME CHANNEL + // THE ROW'S TOOLCHAIN PIN GOES THROUGH. + // + // The row names two things and only one of them used to + // be made to exist: `pin` went through + // `resolve_xpkg_path(…, PayloadIntent::Use, …)` while + // `sysroot` was a pure lookup that returned nullopt and + // let the whole block below be skipped without a word. + // + // Measured 2026-08-26 in a clean environment (an empty + // home, so mcpp's registry starts fresh): + // + // Target riscv64-none-elf + // c-abi picolibc-riscv (…, prebuilt) + // error: 'stdio.h' file not found + // + // The report named the C library and the build could not + // find its headers. mcpp's own bare-metal CI installs it + // by hand, which is why no test ever saw this — every + // bare-metal e2e runs on a machine where the gap has + // already been papered over. + // + // OFFLINE AND `MCPP_NO_AUTO_INSTALL` ARE THE FETCHER'S + // DECISION, not re-derived here. One question, one place + // that answers it — asking it twice is the shape this + // whole release exists to remove. + mcpp::fetcher::Fetcher srFetcher(**cfg3); + mcpp::fetcher::InstallProgressHandler srProgress; + std::optional dir; + if (auto p = srFetcher.resolve_xpkg_path( + want_sysroot, mcpp::fetcher::PayloadIntent::Use, &srProgress)) + dir = p->root; + else + dir = mcpp::provider::payload(*mcpp::provider::open(xl), ref); + if (dir) { + if (auto spec = mcpp::freestanding::resolve(*want)) { + const auto inc = + *dir / "include" / std::string(spec->libdir); + const auto lib = + *dir / "lib" / std::string(spec->libdir); + std::error_code ec2; + state.tc->base.multilibRoot = *dir; + state.tc->base.package = ref.name; + if (std::filesystem::is_directory(inc, ec2)) + state.tc->base.multilibInclude = inc; + if (std::filesystem::is_directory(lib, ec2)) + state.tc->base.multilibLib = lib; + } + } + } + } + } + } + return {}; +} + +} // namespace mcpp::build diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index af91f4cb8..accfb7d51 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -1108,6 +1108,8 @@ void step4b_platform_member_edges(PrepareState& state); void step5_platform_toolchain(PrepareState& state); void step13_platform_statement(PrepareState& state); std::expected step13_finish_platform(PrepareState& state, BuildContext& ctx); +// retarget.cpp: a retargetable driver told its target, sysroot and multilib. +std::expected step2_retarget_for_retargetable_driver(PrepareState& state); // The platform root a first pass asks the second to read in P1, once. std::optional take_platform_restart(); std::expected load_host_platform(PrepareState& state); diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index d479bb76d..d22cce5b6 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -1892,356 +1892,6 @@ step2_detect_toolchain(PrepareState& state) { return {}; } -static std::expected -step2_retarget_for_retargetable_driver(PrepareState& state) { - // ── A retargetable driver has to be TOLD what it is targeting ──────── - // - // `tc.targetTriple` comes from `-dumpmachine`, and for every cross target - // that worked before this it was right for a reason that does not - // generalise: those targets use a DISTINCT compiler binary - // (`x86_64-w64-mingw32-g++`, `aarch64-linux-musl-g++`), whose own - // -dumpmachine reports the cross triple. Clang is ONE binary that emits - // every target it was built with, so -dumpmachine always answers with the - // host — and nothing downstream ever learns otherwise. - // - // Measured before this line existed: - // - // $ mcpp build --target riscv64-none-elf - // Resolved llvm@22.1.8 → riscv64-none-elf → …/bin/clang++ - // Finished dev [unoptimized + debuginfo] in 0.47s - // $ ls target/ - // x86_64-linux-gnu/ ← an ELF for the host, reported as riscv64 - // - // That is E1: success reported, host artifact produced. The output - // directory, the fingerprint, the cache key and the flag layer all read - // `tc.targetTriple`, so correcting it here corrects all of them at once — - // which is the point of there being one field rather than five answers. - // - // THIS USED TO BE SCOPED TO FREESTANDING, WITH THIS REASON: - // - // The hosted cross targets already resolve a per-target binary, and - // overwriting their probed triple would replace a measured fact with - // an assumed one for no gain. - // - // That was true while every hosted cross was served by a payload. It - // stops being true when the TARGET SIDE comes from the dependency graph: - // the C library, the C++ runtime and the platform's own implementation are - // then packages built from source, and the compiler is an ordinary clang — - // whose `-dumpmachine` answers the host, exactly as the paragraph above - // describes for freestanding. - // - // Measured 2026-08-23, with an explicit `[target.aarch64-macos] - // toolchain = "llvm@…"`. The manifest's cfg evaluation used the REQUESTED - // target, so the C library's aarch64 headers were on the command line; the - // toolchain's own triple was still the host's, so code generation was - // x86_64. Two answers to one question, in one command: - // - // okm_float_assert.c: the C library and the compiler disagree about - // LDBL_DIG ('33 == 18') 33 = aarch64 binary128, 18 = x87 - // - // ⇒ The condition is now the property the first paragraph of this comment - // already names: a RETARGETABLE driver has to be told. gcc is not one — a - // gcc payload IS its target — so the mingw and musl-gcc crosses keep - // answering from `-dumpmachine`, which for them remains a measured fact. - if (!state.overrides.target_triple.empty()) { - if (auto want = mcpp::toolchain::triple::parse(state.overrides.target_triple); - want && (want->is_freestanding() - || state.tc->compiler == mcpp::toolchain::CompilerId::Clang)) - { - state.tc->targetTriple = want->str(); - - // AND THE GATE THAT ALREADY EXISTS FOR THIS, APPLIED WHERE THE - // ANSWER IS KNOWN. - // - // `discover_link_runtime_dirs` refuses to report these - // directories for a target that carries its own sysroot, and the - // refusal never fired: that function runs during DETECTION, - // before this line, when `targetTriple` is still the HOST's. The - // gate read a host triple and answered correctly about it. - // - // The artefact is what showed it. An Android link line carried - // - // -L /toolchains/llvm/prebuilt/linux-x86_64/lib/ - // x86_64-unknown-linux-gnu - // - // whose last component is this machine's triple, produced by - // `root / "lib" / targetTriple` -- so the string names the - // question that was asked. Those are the compiler's own host - // runtime directories; an Android artefact must resolve libc++, - // the crt objects and the loader from the NDK's sysroot, and the - // hermetic check reported exactly that failure with six host - // objects. - // - // Cleared rather than re-derived. Re-running the discovery with - // the final triple would also change what every OTHER clang cross - // target gets, and those are measured as they stand; the claim - // being made here is only the one the gate already states. - if (want->has_own_sysroot()) state.tc->linkRuntimeDirs.clear(); - - // And the flag that says it to the driver — for a HOSTED target - // only. Freestanding already emits its own `--target`, together - // with the ISA flags that must accompany it - // (freestanding/target.cppm); a second one here would be the same - // decision in two places. - if (!want->is_freestanding() - && state.tc->compiler == mcpp::toolchain::CompilerId::Clang) { - state.tc->crossTargetFlag = - "--target=" + want->llvm_triple( - min_platform_version(*state.m, *want, state.tc->binaryPath)); - - // AND THE SAME FLAG ON THE std MODULE'S OWN COMMANDS, FOR A - // PAYLOAD THAT SERVES MORE THAN ONE TARGET. - // - // The std module is built by its own command assembly - // (clang.cppm), not by the compile flags, so a decision made - // only here reaches every translation unit and not that. For - // most toolchains the omission cannot be seen: a payload - // whose compiler IS its target finds its own headers, and a - // package-provided module carries the target inside - // `stdModuleFlags`. - // - // ONE NDK SERVES BOTH ANDROID ARCHES, which is the property - // that makes this necessary and is stated in the row's own - // pin: `android-ndk@` names no arch, so `--target` is the - // only thing that says which. Without it the precompile - // resolved libc++'s `#include <__config>` against the - // building machine and stopped there. - // - // NOT `has_own_sysroot()`, though both rows that answer true - // to it are SDKs with their own sysroot. Emscripten's `em++` - // serves exactly one target and needs no flag -- the verified - // wasm loop is measured without it -- so widening the gate to - // the predicate would add a flag to a command that does not - // want one. The property here is "one payload, several - // targets", and Android is the only row that has it; a future - // row brings its own measurement. - if (want->is_android()) { - state.tc->stdModuleTargetFlags = " " + state.tc->crossTargetFlag; - // BIONIC'S ctype HEADER AND A MODULE'S EXPORT RULES. - // - // bionic declares `isalnum` and its neighbours - // `static inline`, and libc++'s module surface exports - // them with `using std::isalnum`. A using-declaration - // cannot export a name with internal linkage, so the - // precompile fails on 14 names at once. Defining the - // macro empty makes those declarations extern, which is - // what every other C library this engine compiles - // against already does. - // - // Scoped to the std module and not to every unit: the - // rule being satisfied is about exporting from a module, - // and a translation unit that includes - // directly is entitled to bionic's inline definitions. - // `xim:android-ndk`'s own install-time self-test reaches - // the identical conclusion from the other direction. - // - // AND THE PAYLOAD MAY SAY SO ITSELF. The recipe applies - // this same define in that self-test, so it is a fact - // the payload already holds; `std_module_defines` in - // `.mcpp-toolchain.json` is the channel for it, and the - // define below is what a payload that ships no - // descriptor still gets. The two are not added - // together: a descriptor that names defines is the - // payload's complete answer for this channel, and - // appending to it would mean a payload could not - // withdraw a define this engine once needed. - auto stdDefines = [&]() -> std::vector { - auto desc = - mcpp::toolchain::payload_descriptor_for_compiler( - state.tc->binaryPath); - if (desc && *desc && !(*desc)->stdModuleDefines.empty()) - return (*desc)->stdModuleDefines; - return { "__BIONIC_CTYPE_INLINE=" }; - }(); - for (auto const& def : stdDefines) - state.tc->stdModuleTargetFlags += " -D" + def; - } - - // ── iOS: THE COMPILER IS OURS, THE SDK IS THE MACHINE'S ── - // - // The three iOS rows pin `llvm@23.1.3` -- any sufficiently - // new clang emits arm64 Mach-O for an iOS deployment target - // -- and take their headers and stub libraries from the - // machine's Xcode, which is where the whole item shrinks to - // a located sysroot. `aarch64-macos` is verified on exactly - // this split and is the precedent. - // - // LOCATED HERE, ONCE. Three later sites need the answer (the - // compile flags, the link line, and the std module's own - // command), and a function that probes the machine is the - // wrong thing to call three times: `xcrun` shells out, and - // three answers can differ if the developer directory - // changes mid-build. - // - // AND ITS ABSENCE IS A REFUSAL THAT NAMES THE SDK. The - // recorded host-surface rule is that a host dependency must - // be minimal, named, and never a fallthrough; the iOS SDK - // and `simctl` are the two this platform adds, both in the - // "proprietary runtime that exists only on its own OS" - // category. A build that continued without the SDK would - // fail in the driver's header search, naming a file rather - // than the thing that is missing. - if (want->is_ios()) { - // READ, NOT RE-DERIVED. The refusal above located it - // before any payload was resolved, and that is the one - // `xcrun` call this build makes. - // - // An empty answer here cannot happen through the - // `--target` path, and a line that prints when it does - // is cheaper than a branch that pretends it cannot: the - // row could be reached one day by a route that skipped - // the gate, and an iOS build with no `-isysroot` is a - // macOS artefact with an iOS triple on it. - if (!state.appleSdkLocated) { - return std::unexpected(std::format( - "internal: target {} reached toolchain " - "resolution without its SDK being located; the " - "gate that locates it did not run for this " - "request", want->str())); - } - state.tc->base.apple.root = *state.appleSdkLocated; - if (state.appleSdkFound) { - state.tc->base.apple.name = state.appleSdkFound->name; - state.tc->base.apple.version = state.appleSdkFound->version; - } - auto sdk = state.appleSdkLocated; - // AND THE std MODULE'S OWN COMMAND, WHICH IS A SEPARATE - // CHANNEL. Same reason the Android rows set it: the - // module is precompiled by `clang.cppm`'s own assembly - // rather than by the compile flags, so a decision made - // only in the flag builder reaches every translation - // unit and not the module they all import. Without the - // SDK here the precompile resolves libc++'s - // `#include <__config>` against the macOS SDK and the - // module is built for the wrong platform. - // - // QUOTED, as every path this string carries is (see the - // package-provided producer, which uses `shq` for each - // `-isystem`). The string is spliced into a shell - // command, and an Xcode installed as `Xcode 16.app` is - // a path with a space in it. - state.tc->stdModuleTargetFlags = - " " + state.tc->crossTargetFlag - + " -isysroot " + mcpp::build::command_line::quote(sdk->string()); - } - } - } - if (auto want = mcpp::toolchain::triple::parse(state.overrides.target_triple); - want && want->is_freestanding()) - { - // `import std` is structurally hosted, and turning it off is the - // SAME fact as the line above, not a second policy: libc++'s - // std.cppm is one module over the whole library, including the - // parts that are threads, filesystem and iostreams. There is no - // subset of it to precompile. - // - // Left on, the failure is neither early nor legible — measured: - // - // error: std module precompile failed (rc=1): - // .../include/c++/v1/__config:13:10: fatal error: - // '__config_site' file not found - // - // which reads as a broken toolchain payload and says nothing about - // the target. The freestanding std subset a user actually wants is - // an ordinary package (`mcpplibs.std.freestanding`), so mcpp's job - // here is to stop pretending the hosted one exists and to say - // where the other one is. - state.tc->clear_std_modules(); - - // ── The target's C library, resolved like its compiler ───────── - // - // The row in kKnownTargets names it, exactly as it names the - // toolchain pin, and it is installed through the same channel a - // project's `[xlings] deps` use (see the materialization above). - // Resolved HERE because the config is already open; the flag - // builder only reads the result. - // - // Absent is not an error at this point: the install happens - // earlier in this function and may legitimately not have run yet - // on a first pass. What follows would then simply not add the - // paths, and the link fails naming the missing libc — which is the - // truthful message either way. - const std::string want_sysroot = - mcpp::toolchain::triple::effective_sysroot( - *want, base_statement_text(state, *want)); - // A C library named by path (design 2026-10-23 §3.1): the same - // `/{include,lib}/` layout, nothing to install. - if (want_sysroot.starts_with("path:")) { - const std::filesystem::path dir = want_sysroot.substr(5); - std::error_code ec2; - if (!std::filesystem::is_directory(dir, ec2)) { - refusal::record(refusal::Code::TargetBase); - return std::unexpected(std::format( - "[target.{}].sysroot = {{ path = \"{}\" }} does not exist", want->str(), - dir.generic_string())); - } - if (auto spec = mcpp::freestanding::resolve(*want)) { - state.tc->base.multilibRoot = dir; - state.tc->base.package = dir.filename().string(); - const auto inc = dir / "include" / std::string(spec->libdir); - const auto lib = dir / "lib" / std::string(spec->libdir); - if (std::filesystem::is_directory(inc, ec2)) state.tc->base.multilibInclude = inc; - if (std::filesystem::is_directory(lib, ec2)) state.tc->base.multilibLib = lib; - } - } else if (!want_sysroot.empty()) { - if (auto cfg3 = state.get_cfg(true); cfg3) { - auto ref = mcpp::provider::parse_ref(want_sysroot); - auto xl = mcpp::config::make_xlings_env(**cfg3); - // INSTALLED, NOT MERELY LOOKED UP — THE SAME CHANNEL - // THE ROW'S TOOLCHAIN PIN GOES THROUGH. - // - // The row names two things and only one of them used to - // be made to exist: `pin` went through - // `resolve_xpkg_path(…, PayloadIntent::Use, …)` while - // `sysroot` was a pure lookup that returned nullopt and - // let the whole block below be skipped without a word. - // - // Measured 2026-08-26 in a clean environment (an empty - // home, so mcpp's registry starts fresh): - // - // Target riscv64-none-elf - // c-abi picolibc-riscv (…, prebuilt) - // error: 'stdio.h' file not found - // - // The report named the C library and the build could not - // find its headers. mcpp's own bare-metal CI installs it - // by hand, which is why no test ever saw this — every - // bare-metal e2e runs on a machine where the gap has - // already been papered over. - // - // OFFLINE AND `MCPP_NO_AUTO_INSTALL` ARE THE FETCHER'S - // DECISION, not re-derived here. One question, one place - // that answers it — asking it twice is the shape this - // whole release exists to remove. - mcpp::fetcher::Fetcher srFetcher(**cfg3); - mcpp::fetcher::InstallProgressHandler srProgress; - std::optional dir; - if (auto p = srFetcher.resolve_xpkg_path( - want_sysroot, mcpp::fetcher::PayloadIntent::Use, &srProgress)) - dir = p->root; - else - dir = mcpp::provider::payload(*mcpp::provider::open(xl), ref); - if (dir) { - if (auto spec = mcpp::freestanding::resolve(*want)) { - const auto inc = - *dir / "include" / std::string(spec->libdir); - const auto lib = - *dir / "lib" / std::string(spec->libdir); - std::error_code ec2; - state.tc->base.multilibRoot = *dir; - state.tc->base.package = ref.name; - if (std::filesystem::is_directory(inc, ec2)) - state.tc->base.multilibInclude = inc; - if (std::filesystem::is_directory(lib, ec2)) - state.tc->base.multilibLib = lib; - } - } - } - } - } - } - return {}; -} static std::expected step2_bind_msvc_toolset(PrepareState& state) { diff --git a/src/doctor.cppm b/src/doctor.cppm index 81c422c9a..cefc31c27 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -1355,8 +1355,12 @@ export int why_toolchain_json(std::string_view target, std::string_view tcSpec) {"layer", std::string(label)}, {"interface", l.interfaceName}, {"impl", l.impl}, + // `platform` for a layer the target's platform added (SPEC-013 + // §8), as the report says it. {"origin", l.absent() ? std::string("none") + : l.byPlatform + ? std::string("platform") : std::string(mcpp::targetside::origin_name(l.origin))}, {"subset", l.subset}, }; diff --git a/tests/matrix/expected.tsv b/tests/matrix/expected.tsv index e8b671ebf..44a21b15c 100644 --- a/tests/matrix/expected.tsv +++ b/tests/matrix/expected.tsv @@ -16,7 +16,8 @@ # unsupported / host-tool-toolchain 交叉目标下的 build.mcpp 没有可解析的宿主工具链 # unsupported / lld-required-absent 这一行直接经 lld 链接,而本机没有 # unsupported / std-module-precompile std 模块预编译不过 -# unsupported / image-tool-absent 链接 ok 映像而图里没有 okld(扫描工程不声明 openkal-tools) +# unsupported / platform-version-group 图里有平台版本组之外的同族版本(graph 模式钉的 openkal-musl 0.3.5 与 +# openkal-llvm-runtime 0.1.3,ok 映像行的平台是 openkal-platform) # # 这张表里**没有 `other`**,而这是一条不变量而不是巧合。`other` 在 # `src/build/refusal.cppm` 里的定义是「拒绝了而这一处分支还没有名字」,所以一行 @@ -62,7 +63,7 @@ graph linux-aarch64 aarch64-linux-gnu llvm@23.1.3 aarch64-unknown-linux-gnu payl graph linux-aarch64 aarch64-linux-musl gcc@16.1.0 - - - - - unsupported layer-requirement graph linux-aarch64 aarch64-linux-musl llvm@23.1.3 aarch64-unknown-linux-musl payload musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph linux-aarch64 aarch64-openkal gcc@16.1.0 - - - - - unsupported capability-pin -graph linux-aarch64 aarch64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +graph linux-aarch64 aarch64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph linux-aarch64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned graph linux-aarch64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned graph linux-aarch64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported capability-pin @@ -74,7 +75,7 @@ graph linux-aarch64 x86_64-linux-musl llvm@23.1.3 x86_64-unknown-linux-musl payl graph linux-aarch64 x86_64-macos gcc@16.1.0 - - - - - unsupported tier-planned graph linux-aarch64 x86_64-macos llvm@23.1.3 - - - - - unsupported tier-planned graph linux-aarch64 x86_64-openkal gcc@16.1.0 - - - - - unsupported capability-pin -graph linux-aarch64 x86_64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +graph linux-aarch64 x86_64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph linux-aarch64 x86_64-windows-gnu gcc@16.1.0 - - - - - unsupported host-cannot-serve graph linux-aarch64 x86_64-windows-gnu llvm@23.1.3 x86_64-w64-windows-gnu none musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph linux-aarch64 x86_64-windows-musl gcc@16.1.0 - - - - - unsupported capability-pin @@ -86,7 +87,7 @@ graph linux-x86_64 aarch64-linux-gnu llvm@23.1.3 aarch64-unknown-linux-gnu paylo graph linux-x86_64 aarch64-linux-musl gcc@16.1.0 - - - - - unsupported layer-requirement graph linux-x86_64 aarch64-linux-musl llvm@23.1.3 aarch64-unknown-linux-musl payload musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph linux-x86_64 aarch64-openkal gcc@16.1.0 - - - - - unsupported capability-pin -graph linux-x86_64 aarch64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +graph linux-x86_64 aarch64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph linux-x86_64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned graph linux-x86_64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned graph linux-x86_64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported capability-pin @@ -100,7 +101,7 @@ graph linux-x86_64 x86_64-linux-musl llvm@23.1.3 x86_64-unknown-linux-musl paylo graph linux-x86_64 x86_64-macos gcc@16.1.0 - - - - - unsupported tier-planned graph linux-x86_64 x86_64-macos llvm@23.1.3 - - - - - unsupported tier-planned graph linux-x86_64 x86_64-openkal gcc@16.1.0 - - - - - unsupported capability-pin -graph linux-x86_64 x86_64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +graph linux-x86_64 x86_64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph linux-x86_64 x86_64-windows-gnu gcc@16.1.0 - - - - - unsupported layer-requirement graph linux-x86_64 x86_64-windows-gnu llvm@23.1.3 x86_64-w64-windows-gnu none musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph linux-x86_64 x86_64-windows-musl gcc@16.1.0 - - - - - unsupported capability-pin @@ -109,13 +110,13 @@ graph macos-arm64 aarch64-linux-android llvm@23.1.3 - - - - - unsupported capabi graph macos-arm64 aarch64-linux-gnu llvm@23.1.3 aarch64-unknown-linux-gnu host musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph macos-arm64 aarch64-linux-musl llvm@23.1.3 aarch64-unknown-linux-musl host musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph macos-arm64 aarch64-macos llvm@23.1.3 arm64-apple-macos14.0 host musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none -graph macos-arm64 aarch64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +graph macos-arm64 aarch64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph macos-arm64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned graph macos-arm64 wasm32-emscripten llvm@23.1.3 - - - - - unsupported capability-pin graph macos-arm64 x86_64-linux-android llvm@23.1.3 - - - - - unsupported capability-pin graph macos-arm64 x86_64-linux-musl llvm@23.1.3 x86_64-unknown-linux-musl host musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph macos-arm64 x86_64-macos llvm@23.1.3 - - - - - unsupported tier-planned -graph macos-arm64 x86_64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +graph macos-arm64 x86_64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph macos-arm64 x86_64-windows-gnu llvm@23.1.3 x86_64-w64-windows-gnu none musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph macos-arm64 x86_64-windows-musl llvm@23.1.3 x86_64-w64-windows-gnu host musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph windows-x86_64 aarch64-linux-android llvm@23.1.3 - - - - - unsupported capability-pin @@ -124,7 +125,7 @@ graph windows-x86_64 aarch64-linux-gnu llvm@23.1.3 aarch64-unknown-linux-gnu non graph windows-x86_64 aarch64-linux-gnu msvc@system - - - - - unsupported host-tool-toolchain graph windows-x86_64 aarch64-linux-musl llvm@23.1.3 aarch64-unknown-linux-musl none musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph windows-x86_64 aarch64-linux-musl msvc@system - - - - - unsupported host-tool-toolchain -graph windows-x86_64 aarch64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +graph windows-x86_64 aarch64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph windows-x86_64 aarch64-openkal msvc@system - - - - - unsupported capability-pin graph windows-x86_64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned graph windows-x86_64 riscv64-linux-musl msvc@system - - - - - unsupported tier-planned @@ -136,7 +137,7 @@ graph windows-x86_64 x86_64-linux-musl llvm@23.1.3 x86_64-unknown-linux-musl non graph windows-x86_64 x86_64-linux-musl msvc@system - - - - - unsupported host-tool-toolchain graph windows-x86_64 x86_64-macos llvm@23.1.3 - - - - - unsupported tier-planned graph windows-x86_64 x86_64-macos msvc@system - - - - - unsupported tier-planned -graph windows-x86_64 x86_64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +graph windows-x86_64 x86_64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph windows-x86_64 x86_64-openkal msvc@system - - - - - unsupported capability-pin graph windows-x86_64 x86_64-windows-gnu llvm@23.1.3 x86_64-w64-windows-gnu none musl(graph) libc++(graph) openkal-llvm-runtime@0.1.3 ok none graph windows-x86_64 x86_64-windows-gnu msvc@system - - - - - unsupported host-tool-toolchain @@ -153,7 +154,7 @@ payload linux-aarch64 aarch64-linux-musl llvm@23.1.3 - - - - - unsupported conve payload linux-aarch64 aarch64-none-elf gcc@16.1.0 - - - - - unsupported capability-pin payload linux-aarch64 aarch64-none-elf llvm@23.1.3 aarch64-none-elf payload - - - ok none payload linux-aarch64 aarch64-openkal gcc@16.1.0 - - - - - unsupported capability-pin -payload linux-aarch64 aarch64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +payload linux-aarch64 aarch64-openkal llvm@23.1.3 aarch64-unknown-none-elf payload musl(platform) libc++(platform) openkal-musl@0.22.3 ok none payload linux-aarch64 armv7a-none-eabi gcc@16.1.0 - - - - - unsupported capability-pin payload linux-aarch64 armv7a-none-eabi llvm@23.1.3 armv7a-none-eabi payload - - - ok none payload linux-aarch64 armv7a-none-eabihf gcc@16.1.0 - - - - - unsupported capability-pin @@ -189,7 +190,7 @@ payload linux-aarch64 x86_64-macos llvm@23.1.3 - - - - - unsupported tier-planne payload linux-aarch64 x86_64-none-elf gcc@16.1.0 - - - - - unsupported capability-pin payload linux-aarch64 x86_64-none-elf llvm@23.1.3 x86_64-none-elf payload - - - ok none payload linux-aarch64 x86_64-openkal gcc@16.1.0 - - - - - unsupported capability-pin -payload linux-aarch64 x86_64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +payload linux-aarch64 x86_64-openkal llvm@23.1.3 x86_64-unknown-none-elf payload musl(platform) libc++(platform) openkal-musl@0.22.3 ok none payload linux-aarch64 x86_64-windows-gnu gcc@16.1.0 - - - - - unsupported host-cannot-serve payload linux-aarch64 x86_64-windows-gnu llvm@23.1.3 - - - - - unsupported convention-unreplaced payload linux-aarch64 x86_64-windows-musl gcc@16.1.0 - - - - - unsupported capability-pin @@ -203,7 +204,7 @@ payload linux-x86_64 aarch64-linux-musl llvm@23.1.3 - - - - - unsupported conven payload linux-x86_64 aarch64-none-elf gcc@16.1.0 - - - - - unsupported capability-pin payload linux-x86_64 aarch64-none-elf llvm@23.1.3 aarch64-none-elf payload - - - ok none payload linux-x86_64 aarch64-openkal gcc@16.1.0 - - - - - unsupported capability-pin -payload linux-x86_64 aarch64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +payload linux-x86_64 aarch64-openkal llvm@23.1.3 aarch64-unknown-none-elf payload musl(platform) libc++(platform) openkal-musl@0.22.3 ok none payload linux-x86_64 armv7a-none-eabi gcc@16.1.0 - - - - - unsupported capability-pin payload linux-x86_64 armv7a-none-eabi llvm@23.1.3 armv7a-none-eabi payload - - - ok none payload linux-x86_64 armv7a-none-eabihf gcc@16.1.0 - - - - - unsupported capability-pin @@ -241,7 +242,7 @@ payload linux-x86_64 x86_64-macos llvm@23.1.3 - - - - - unsupported tier-planned payload linux-x86_64 x86_64-none-elf gcc@16.1.0 - - - - - unsupported capability-pin payload linux-x86_64 x86_64-none-elf llvm@23.1.3 x86_64-none-elf payload - - - ok none payload linux-x86_64 x86_64-openkal gcc@16.1.0 - - - - - unsupported capability-pin -payload linux-x86_64 x86_64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +payload linux-x86_64 x86_64-openkal llvm@23.1.3 x86_64-unknown-none-elf payload musl(platform) libc++(platform) openkal-musl@0.22.3 ok none payload linux-x86_64 x86_64-windows-gnu gcc@16.1.0 x86_64-w64-windows-gnu none ucrt(payload) libstdc++(payload) - ok none payload linux-x86_64 x86_64-windows-gnu llvm@23.1.3 - - - - - unsupported convention-unreplaced payload linux-x86_64 x86_64-windows-musl gcc@16.1.0 - - - - - unsupported capability-pin @@ -253,7 +254,7 @@ payload macos-arm64 aarch64-linux-gnu llvm@23.1.3 - - - - - unsupported host-can payload macos-arm64 aarch64-linux-musl llvm@23.1.3 - - - - - unsupported convention-unreplaced payload macos-arm64 aarch64-macos llvm@23.1.3 arm64-apple-macos14.0 host libSystem(payload) libc++(payload) - ok none payload macos-arm64 aarch64-none-elf llvm@23.1.3 aarch64-none-elf host - - - ok none -payload macos-arm64 aarch64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +payload macos-arm64 aarch64-openkal llvm@23.1.3 aarch64-unknown-none-elf payload musl(platform) libc++(platform) openkal-musl@0.22.3 ok none payload macos-arm64 armv7a-none-eabi llvm@23.1.3 armv7a-none-eabi host - - - ok none payload macos-arm64 armv7a-none-eabihf llvm@23.1.3 armv7a-none-eabihf host - - - ok none payload macos-arm64 riscv32-none-elf llvm@23.1.3 riscv32-none-elf host - - - ok none @@ -272,7 +273,7 @@ payload macos-arm64 x86_64-linux-android llvm@23.1.3 - - - - - unsupported capab payload macos-arm64 x86_64-linux-musl llvm@23.1.3 - - - - - unsupported convention-unreplaced payload macos-arm64 x86_64-macos llvm@23.1.3 - - - - - unsupported tier-planned payload macos-arm64 x86_64-none-elf llvm@23.1.3 x86_64-none-elf host - - - ok none -payload macos-arm64 x86_64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +payload macos-arm64 x86_64-openkal llvm@23.1.3 x86_64-unknown-none-elf payload musl(platform) libc++(platform) openkal-musl@0.22.3 ok none payload macos-arm64 x86_64-windows-gnu llvm@23.1.3 - - - - - unsupported convention-unreplaced payload macos-arm64 x86_64-windows-musl llvm@23.1.3 - - - - - unsupported host-cannot-serve payload windows-x86_64 aarch64-linux-android llvm@23.1.3 - - - - - unsupported capability-pin @@ -283,7 +284,7 @@ payload windows-x86_64 aarch64-linux-musl llvm@23.1.3 - - - - - unsupported conv payload windows-x86_64 aarch64-linux-musl msvc@system - - - - - unsupported convention-unreplaced payload windows-x86_64 aarch64-none-elf llvm@23.1.3 aarch64-none-elf none - - - ok none payload windows-x86_64 aarch64-none-elf msvc@system - - - - - unsupported capability-pin -payload windows-x86_64 aarch64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +payload windows-x86_64 aarch64-openkal llvm@23.1.3 aarch64-unknown-none-elf payload musl(platform) libc++(platform) openkal-musl@0.22.3 ok none payload windows-x86_64 aarch64-openkal msvc@system - - - - - unsupported capability-pin payload windows-x86_64 armv7a-none-eabi llvm@23.1.3 armv7a-none-eabi none - - - ok none payload windows-x86_64 armv7a-none-eabi msvc@system - - - - - unsupported capability-pin @@ -319,7 +320,7 @@ payload windows-x86_64 x86_64-macos llvm@23.1.3 - - - - - unsupported tier-plann payload windows-x86_64 x86_64-macos msvc@system - - - - - unsupported tier-planned payload windows-x86_64 x86_64-none-elf llvm@23.1.3 - - - - - unsupported lld-required-absent payload windows-x86_64 x86_64-none-elf msvc@system - - - - - unsupported capability-pin -payload windows-x86_64 x86_64-openkal llvm@23.1.3 - - - - - unsupported image-tool-absent +payload windows-x86_64 x86_64-openkal llvm@23.1.3 x86_64-unknown-none-elf payload musl(platform) libc++(platform) openkal-musl@0.22.3 ok none payload windows-x86_64 x86_64-openkal msvc@system - - - - - unsupported capability-pin payload windows-x86_64 x86_64-windows-gnu llvm@23.1.3 - - - - - unsupported convention-unreplaced payload windows-x86_64 x86_64-windows-gnu msvc@system - - - - - unsupported convention-unreplaced From cc4e5fb30f3f3cc07a561c0d8e17d61172a1c2dc Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 05:20:50 +0900 Subject: [PATCH 12/54] docs: SPEC-013 (platforms), SPEC-014 (packaging and distribution); docs/35 platforms; docs/50 platform fields and reason tokens - SPEC-013: [platform] groups, tiers, target/host selection, cfg(platform), merge, graph completion and version group, toolchain, link/name/seal, runner, carrier, platform phase and protocol 17, workspace, lock, report, JSON, commands, refusal codes - SPEC-014: three axes and legs, legality table, option usage rule, [pack] keys, pipeline, universal layout, launchers, --systems, mcpp-package.json, --merge, compression, single-file, providers, signing, emit xpkg, mcpp.pack fields (per-clause status) - SPEC-007 v0.11: protocol 17 row, R9.10 platform phase - SPEC-004 v1.19: platform plane, [target.] platform keys, [host] - docs/35 (en, zh); indexes; docs/50: self env --target, toolchain list platform, plan/result platform fields, platform-* reason tokens, --pkg-format Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/35-platforms.md | 266 +++++++++++++++++++ docs/50-machine-output.md | 62 ++++- docs/README.md | 10 +- docs/specs/README.md | 2 + docs/specs/build-plugins.md | 14 +- docs/specs/manifest-semantics.md | 13 +- docs/specs/packaging.md | 280 ++++++++++++++++++++ docs/specs/platforms.md | 431 +++++++++++++++++++++++++++++++ docs/zh/35-platforms.md | 234 +++++++++++++++++ docs/zh/50-machine-output.md | 56 +++- docs/zh/README.md | 10 +- 11 files changed, 1341 insertions(+), 37 deletions(-) create mode 100644 docs/35-platforms.md create mode 100644 docs/specs/packaging.md create mode 100644 docs/specs/platforms.md create mode 100644 docs/zh/35-platforms.md diff --git a/docs/35-platforms.md b/docs/35-platforms.md new file mode 100644 index 000000000..8f6ac08de --- /dev/null +++ b/docs/35-platforms.md @@ -0,0 +1,266 @@ +# 35 — Platforms + +**Reader:** someone building for a target a platform package configures (an ok +image, `-openkal`), changing part of that configuration, or writing a +platform package for a class of targets. + +**The question this chapter answers:** how a class of targets is configured as +one package, how a build selects it, and how a project overrides or replaces it. + +**Not here:** the target-side layers a platform names, which are +[22 — The Target Side](22-target-side.md); openkal's model and its hosted +cross-compilation, which are [24 — Cross-Compilation Over openkal](24-openkal-cross.md); +packaging a program, which is [10 — Packaging an Application for Release](10-pack-and-release.md); +the exact rules, which are [SPEC-013](specs/platforms.md). + +Before: [34 — Authoring a Board-Support Package](34-authoring-a-bsp.md). +After: [40 — Bare-Metal and Freestanding Targets](40-baremetal.md). + +A **platform** is one versioned package that states everything a class of +targets needs: the toolchain, the package that provides each target-side layer, +the tools the build runs on this machine, the compile and link conventions, the +name and seal of an artifact, and the runner. A project states none of it. The +target table selects `mcpplibs.openkal-platform` for `x86_64-openkal` and +`aarch64-openkal` (2026.10.27.1+), which is what built-in support for those two +targets is. + +## The shortest run + +A project that names only its sources: + +```toml +[package] +name = "app" +version = "0.1.0" +``` + +```sh +mcpp run --target x86_64-openkal +``` + +``` + Resolved llvm@23.1.3 → x86_64-openkal → …/xim-x-llvm/23.1.3/bin/clang++ + the platform's (openkal-platform 0.17.1), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` + Target x86_64-openkal → x86_64-unknown-none-elf + compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.3, platform) + c-abi musl (openkal-musl@0.22.3, platform) + c++-abi libc++ (openkal-llvm-runtime@0.16.3, platform) + Platform openkal-platform 0.17.1 (built-in) + tools okld, okrun (openkal-tools@0.1.1, platform) + runner okrun (platform) + link lld, start ok_main (platform) + Compiling app v0.1.0 (.) + Running `okrun … target/x86_64-openkal/…/bin/app.oke` +``` + +Every value is reported with its source. `mcpp test`, `mcpp pack` and +`--target aarch64-openkal` need nothing more. `mcpp.lock` records the packages +the platform added with `origin = "platform"`. + +## What a platform states + +| Group | Keys | Effect | +|---|---|---| +| the targets | `targets`, `toolchain`, `sysroot` | which triples it serves, the toolchain they use, a prebuilt target base | +| the layers | `[platform.layers]` | the package for `c-abi`, `c++-abi`, `compiler-runtime`, `kernel-abi`, or `false` | +| the tools | `[platform.tools]` | programs built for this machine, or found (`{ program = "" }`) | +| the compile | `[platform.build]`, `[platform.build.'cfg(…)']` | flags for every unit of the target | +| the link | `[platform.link]` | `driver`, `flags`, `start`, `script`, `inputs`, the names `exe` and `shared`, the `seal` steps | +| the run | `runner`, `runners` | how `mcpp run` and `mcpp test` start an artifact, and named device actions | +| the carrier | `carrier` | the program `mcpp pack --mode self-contained` builds around a loaded artifact | +| the code | `plugin` | a build program run in the platform phase, for what depends on the graph | +| the group | `[platform.versions]` | one version of each package of a family | +| the host part | `[platform.hosted]` | what a host triple takes when it selects the platform | + +## Overriding part of a platform + +A project overrides a field under `[target.]` with the platform's own +key name. `cfg(platform = "openkal")` matches the platform whatever the +architecture: + +```toml +[target.'cfg(platform = "openkal")'.layers] +c-abi = { path = "../openkal-musl" } # a working tree in place of the release + +[target.'cfg(platform = "openkal")'.tools] +okld = { path = "../openkal-image/tools" } + +[target.'cfg(platform = "openkal")'] +runner = ["my-okrun", "{}"] +``` + +| Field | Rule | +|---|---| +| compile flags, `link.flags` | appended after the platform's; a convention cannot be removed | +| `link.inputs`, `link.seal` | replaced whole | +| `layers`, `tools`, `runners` | replaced key by key; ` = false` leaves the layer out | +| `runner` | the argv is replaced; `host` and `exclusive` are kept unless stated | +| every other field | replaced | + +The same tables are accepted in `~/.mcpp/config.toml` for one machine. A layer +a package of the graph already provides is left to that package and reported +`graph`. The package of the C++ run-time also provides the compiler run-time, so +`"c++-abi" = false` takes effect only with `compiler-runtime = false` beside it. + +## Selecting a platform, or none + +The first of these that answers selects the target's platform: + +| Source | Spelling | +|---|---| +| the command line | `--platform openkal-platform@0.17.1`, `--platform path:../openkal-platform`, `--platform none` | +| the environment | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | +| the project | `[target.] platform = "openkal-platform@0.17.1"`, or `{ path = "…" }` | +| this machine | the same key in `~/.mcpp/config.toml` | +| the target table | the platform column (`mcpp toolchain list`) | + +`platform` is not accepted under a `cfg(platform = …)` selector. `none` builds +with the engine's conventions for the target and nothing else. + +## The host graph's platform + +Build programs and the tools the build runs are compiled for this machine by +its own conventions. `[host] platform` selects a platform for them instead: + +```toml +[host] +platform = "openkal-platform@0.17.1" +``` + +``` + Host platform openkal-platform 0.17.1 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun +``` + +Every build program then compiles as an ok image of this machine's +architecture, with one toolchain on Linux, macOS and Windows, and runs through +the platform's runner. The platform's own tools are still built natively. +`--host-platform` and `MCPP_HOST_PLATFORM` select it for one run, and +`[host]` in `~/.mcpp/config.toml` for one machine. + +## The hosted arrangement + +A host triple that selects a platform takes only the platform's +`[platform.hosted]` part: its toolchain and its layers. With openkal-platform +that is the family's C library and C++ run-time over the system's own openkal +implementation: + +```toml +[target.'cfg(any(linux, macos, windows))'] +platform = "openkal-platform@0.17.1" +``` + +On Windows the arrangement is the GNU ABI's row (`--target x86_64-windows-gnu`); +a triple outside `[platform.hosted] targets` is refused, naming the row of the +same system and architecture that it serves. + +## The version group + +`[platform.versions]` names one version of each package of a family whose +packages pin one another exactly. A graph that holds another version of one of +them is refused before anything compiles: + +``` +error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.1 is a version group that names openkal-musl 0.22.3. + requested by: own + The family's packages pin one another exactly, so two versions of one of + them cannot be linked into one program. Either select the platform release + of that version ([target.] platform = "openkal-platform@"), or write + openkal-musl = "0.22.3" where it is requested. +``` + +A package taken from a directory or a repository is not bound by the group. + +## Writing a platform package + +A platform package is a package with a `[platform]` section. openkal-platform, +abridged: + +```toml +[package] +namespace = "mcpplibs" +name = "openkal-platform" +version = "0.17.1" +mcpp = ">=2026.10.27.1" + +[platform] +targets = ["x86_64-openkal", "aarch64-openkal"] +toolchain = "llvm@23.1.3" +runner = { cmd = ["okrun", "{}"], host = "same-arch" } +carrier = { package = "openkal-run@0.1.1", section = ".oke", main = "carrier/main.cpp" } + +[platform.layers] +compiler-runtime = "openkal-llvm-runtime@0.16.3" +"c++-abi" = "openkal-llvm-runtime@0.16.3" +c-abi = { name = "openkal-musl", version = "0.22.3", default-features = false, features = ["posix"] } + +[platform.tools] +okld = "openkal-tools@0.1.1" +okrun = "openkal-tools@0.1.1" + +[platform.build] +cxxflags = ["-fPIC", "-femulated-tls", "-fvisibility=hidden", "-fasynchronous-unwind-tables"] + +[platform.build.'cfg(arch = "aarch64")'] +cxxflags = ["-ffixed-x18", "-mno-outline-atomics"] + +[platform.link] +driver = "lld" +start = "ok_main" +flags = ["-shared", "-z", "max-page-size=65536", "-z", "now", "--gc-sections", "--eh-frame-hdr"] +inputs = [{ from = ["okld", "note-source"], as = "ok-notes.S" }] +exe = "{name}.oke" +shared = "lib{name}.okso" +seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] +``` + +- **Programs are named, not located.** `okld` in `inputs` and `seal` is found among + the programs the graph built for this machine, then in declared payloads, then + on `PATH`. +- **`{}` is the artifact.** In a seal step `{needed}` expands to the shared images + the artifact needs; a preceding `--with` is repeated before each. +- **A seal step that fails publishes nothing.** The artifact published by the + last successful build stays in place. +- **What depends on the graph is code.** `plugin = "build.mcpp"` runs in the + platform phase (`mcpp::phase()` is `"platform"`) and may state flags, link + fields and runners (`mcpp::link_driver`, `mcpp::link_input_from`, + `mcpp::artifact_name`, `mcpp::seal`, `mcpp::run_host`); any other directive is + refused. A board-support package's build program states the same fields in + its build phase ([34](34-authoring-a-bsp.md)). + +During development a project selects the working tree with +`--platform path:` or `platform = { path = "" }`. `mcpp test` in the +platform package, with no `--target`, builds its tests for each of its +`targets` with the package as the platform, runs those of this machine's +architecture, and reports the others as built and not run. + +## Inspecting a configuration + +```sh +mcpp self env --target x86_64-openkal # toolchain, platform, tools, runner +mcpp self env --target x86_64-openkal --format json +mcpp self env --host # the host graph +mcpp toolchain list # the PLATFORM column +mcpp toolchain install --target x86_64-openkal # fetch and build what the target needs +``` + +`--message-format json` carries `platform` and `hostPlatform` in the `plan` +event and in `result.json` ([50](50-machine-output.md)). + +## Refusals + +| Reason | Cause | Remedy | +|---|---|---| +| `platform-not-a-platform` | the selected package has no `[platform]` | select a platform package, or `none` | +| `platform-target-unserved` | the platform does not serve the triple | build for one of its `targets`, or for the host row it names | +| `platform-tool-absent` | a program of the link or the seal is neither built nor found | name its package under `[target.'cfg(platform = "…")'.tools]` | +| `platform-version-group` | another version of a family package is in the graph | select the platform release of that version, or pin the group's version | +| `platform-phase-directive` | the plugin stated a directive the platform phase does not take | state it from the package's own build program | +| `lld-required-absent` | the platform links with `ld.lld` and the toolchain has none | use the platform's toolchain | + +## Current limitations + +- A platform states one value per field; a field that differs per architecture + is written under `[platform.build.'cfg(arch = "…")']` for flags only. +- The host graph's platform is selected per run or per project; its default is + the host conventions. +- `[host]` accepts `platform` only. diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 19d9c4e4e..168926358 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -102,10 +102,11 @@ mcpp --format json streaming case and is **not** accepted — asking for it is an error, not a silent fallback. -A command whose `--format` already names its **product** asks for machine -output with `--message-format json` instead, as `mcpp test` does. `mcpp pack ---format` names the package format (`tar`, `dir`, `msi`), so its report is -`mcpp pack --message-format json` (2026.9.16.1+). The shape follows the kind: +A command that builds asks for machine output with `--message-format json` +instead, as `mcpp test` does: `build`, `test`, `run` and `pack`. The package +format of `mcpp pack` and `mcpp run` is `--pkg-format` (`tar`, `dir`, `msi`; +2026.10.27.1+; `--format` there is the earlier spelling and warns), so the +report of a pack is `mcpp pack --message-format json` (2026.9.16.1+). The shape follows the kind: `mcpp test` streams a record per test because tests finish over time, and `mcpp pack` prints one envelope because a pack has one result. `mcpp build --message-format json` (2026.10.26.1+) streams events while it builds and ends @@ -253,8 +254,8 @@ and only the first belongs to the program: |---|---| | `0`–`124` | the program ran; this is its own status, passed through unchanged. The exception is `101`, below | | `101` | `mcpp run` could not build the program: its planning or its build failed, and nothing was started. This is the status Cargo gives a failed `cargo run` build | -| `125`–`127` | the spawn was attempted and refused — `127` not found, `126` found but not executable, `125` anything else; `126` also answers `mcpp run --format ` for a distributable that is a directory and meets no runner, refused before the spawn with the same meaning (2026.9.14.2+) | -| `2` | mcpp refused the request before building anything, or after building it and before starting anything: a usage error, `--format` with `--no-runner`, a program or a runner that the project does not declare | +| `125`–`127` | the spawn was attempted and refused — `127` not found, `126` found but not executable, `125` anything else; `126` also answers `mcpp run --pkg-format ` for a distributable that is a directory and meets no runner, refused before the spawn with the same meaning (2026.9.14.2+) | +| `2` | mcpp refused the request before building anything, or after building it and before starting anything: a usage error, `--pkg-format` with `--no-runner`, a program or a runner that the project does not declare | Until 2026.9.30.2 a failed build exited `1` and a failed planning `2`, so that a compile error and a program that returns `1` could not be told apart by a @@ -337,6 +338,33 @@ That is why this exists at all: without it a client has to reimplement mcpp's home resolution, including the part where the `mcpp` on `PATH` may be an xlings shim rather than the real binary. +**One configuration** *(2026.10.27.1+)*. With `--target ` (or `--host` +for the host graph) `self env` prepares that configuration, without building, +and reports it; in a project it is the project's configuration, elsewhere an +empty program's under the home's cache. `--platform` and `--host-platform` +select as they do for a build ([35](35-platforms.md)). + +```jsonc +{ + "triple": "x86_64-openkal", + "graph": "target", // or "host" + "toolchain": {"compiler": "/…/xim-x-llvm/23.1.3/bin/clang++", "origin": "platform"}, + "platform": {"name": "mcpplibs.openkal-platform", "version": "0.17.1", "source": "built-in", + "hosted": false, "fields": {"runner": {"value": ["okrun", "{}"], "source": "platform"}}}, + "tools": {"okld": "/…/okld", "okrun": "/…/okrun"}, + "runner": {"argv": ["okrun", "{}"], "host": "same-arch", "exclusive": false, + "program": "/…/okrun", "source": "platform"}, + "runUnder": [] +} +``` + +`platform` is `null` when no platform is selected; its shape is the `plan` +event's (§9). A reason the project's sources would be refused for (an `import +std` with no C++ run-time in the graph) is a `warning` diagnostic with the code +`MCPP_CONFIGURATION_SOURCES_REFUSED`, and the configuration is still reported. +A configuration that cannot be prepared carries `MCPP_CONFIGURATION_FAILED` and +no `data`. + `defaultToolchain` is the toolchain a build with nothing configured resolves on this host: the one answer the first run installs, and the value the tables of docs/01 and docs/20 are checked against on each CI host. On Windows it depends @@ -454,6 +482,8 @@ mcpp toolchain list --format json installation, which is located on the machine rather than installed by mcpp. A target row is `{target, note, toolchain, pin, status, default}`, and `status` is one of `installed` / `available` / `via dependency graph` / `planned`. +A target row also carries `platform` (2026.10.27.1+): the platform package the +target table selects for it (`mcpplibs.openkal-platform@0.17.1`), or `null`. **`toolchain` and `pin` are not the same field twice.** `toolchain` is what the row is associated with — the installed payload on an installed row, the @@ -569,7 +599,11 @@ a program classifying the outcome reads `reason`: | `std-module-precompile` | the standard library's module could not be precompiled for this configuration | | `msvc-redist-unavailable` | an explicit `cxx_runtime = "toolchain-coupled"` on an MSVC-ABI row whose toolset has no redistributable directory to stage *(2026.9.28.1+)* | | `crt-declared-under-host-coupled` | a file of the MSVC C++ runtime is declared beside the program while its contract is host-coupled, under which the system's runtime serves it and no copy is placed *(2026.9.28.2+)* | -| `image-tool-absent` | a build of `-openkal` links an image and finds no `okld`, the program of the graph that notes and checks an ok image *(2026.10.26.2+)* | +| `platform-not-a-platform` | the package selected as the target's platform has no `[platform]` section *(2026.10.27.1+)* | +| `platform-target-unserved` | the selected platform does not serve the target: the triple is not among its `targets`, and a host triple is not among its `[platform.hosted] targets` *(2026.10.27.1+)* | +| `platform-tool-absent` | a program the platform's link or seal runs (`okld` for an ok image) is neither built by the graph nor found on this machine *(2026.10.27.1+; `image-tool-absent` in 2026.10.26.2)* | +| `platform-version-group` | a package of the platform's version group is in the graph at another version *(2026.10.27.1+)* | +| `platform-phase-directive` | a platform's plugin stated a directive its phase does not take *(2026.10.27.1+)* | | `other` | a refusal whose branch has not been given a token yet | **One token is also printed by `mcpp build` itself.** @@ -672,7 +706,7 @@ fails there still fails the build. ### `mcpp.pack` — the products of a pack *(mcpp 2026.9.16.1+)* ``` -mcpp pack [target] [--format ] [--target ...] --message-format json +mcpp pack [target] [--pkg-format ] [--target ...] --message-format json ``` The envelope is printed once, after the command finishes; every human line goes @@ -681,8 +715,10 @@ to stderr, including what the build programs and tools the pack starts print. | field | | |---|---| -| `artifacts` | one record per produced artifact: `path` (absolute), `type` (`file` or `directory`), `format` (the `--format` value, `tar` when omitted) and `targets` (the canonical triple of each leg that went into it). A dispatched format reports the terminal outputs of the actions the request introduced; a several-`--target` Android pack reports one artifact whose `targets` lists every leg | +| `artifacts` | one record per produced artifact: `path` (absolute), `type` (`file` or `directory`), `format` (the `--pkg-format` value, `tar` when omitted), `targets` (the canonical triple of each leg that went into it) and `sha256` (a file's digest; `null` for a directory; 2026.10.27.1+). A dispatched format reports the terminal outputs of the actions the request introduced; a several-`--target` Android pack reports one artifact whose `targets` lists every leg | | `stage` | the tree the artifact was made from: `dir`, `manifest` (the stage manifest below) and `closure` (`walked` or `not-walked`); `null` for a library package, when no tree was staged, and for a pack of several members | +| `linkage`, `mode`, `pkgFormat` *(2026.10.27.1+)* | the three choices of the pack ([10](10-pack-and-release.md)) | +| `legs`, `systems` *(2026.10.27.1+)* | the canonical triple of each leg, and the systems whose loaders a self-contained universal package carries | | `stages` *(2026.10.1.1+)* | present only for a pack of several members (`--workspace`, or `-p` repeated): one record per member in `[workspace] members` order, with `member` (the qualified package name) and the `dir`, `manifest` and `closure` of `stage` | For a pack of several members `artifacts` lists every member's artifacts, in the @@ -842,7 +878,7 @@ Standard output is NDJSON. Each line but the last is an event: | event | carries | |---|---| -| `plan` | `configuration`: the configuration whose build directory the build has entered | +| `plan` | `configuration`: the configuration whose build directory the build has entered; `platform` and `hostPlatform` (2026.10.27.1+) are the platforms of the target and host graphs, or `null` | | `waiting` | `holder` (`pid`, `command`, `client`, `startedAt`) and `waitedMs`: another build holds this build directory, and this one waits for it (§11) | | `progress` | `finished`, `total`: ninja's count of steps; a build may run more than one ninja pass, and each restarts the count | | `diagnostic` | `diagnostic`: one diagnostic of a compiler, from the output of a failed step, in the shape of `diagnostics[]` of the record below | @@ -870,7 +906,7 @@ terminal is therefore visible to an IDE that reads the record, and the reverse. | `pid` | the process that wrote the record | | `startedAt`, `finishedAt` | milliseconds since the epoch; `finishedAt` is `null` while `running` and after `cancelled` | | `status` | `running`, `ok`, `failed`, or `cancelled` | -| `configuration` | `triple`, `profile`, `fingerprint`, `outputDir` | +| `configuration` | `triple`, `profile`, `fingerprint`, `outputDir`, `platform`, `hostPlatform` (2026.10.27.1+) | | `compileDatabase` | the configuration's `compile_commands.json`, or `null` | | `artifacts` | `name`, `kind` (`bin`, `test`, `shared`, `static`), `path`, for a runnable product `launch`, and `entry` when the program's beginning or its entry symbol was stated | | `diagnostics` | the compilers' diagnostics of a failed build: `severity`, `message`, `path`, `range` | @@ -907,7 +943,7 @@ mcpp run [target] --print-launch Builds the program as `mcpp run` does and prints the envelope instead of starting it. `data` is `program`, `argv`, `cwd`, `env` and `via`, as -`launch` above. `--print-launch` takes no `--format`: on `mcpp run`, `--format` +`launch` above. `--print-launch` takes no `--format`: on `mcpp run`, `--pkg-format` names a distributable. ## 10. Denying effects — `--deny` *(mcpp 2026.10.26.1+)* @@ -981,7 +1017,7 @@ mcpp index list --format json project's `[indices]` entries, when run in a project). The packages are `mcpp search`'s answer. -### `mcpp.pack.formats` — the formats `mcpp pack --format` accepts +### `mcpp.pack.formats` — the formats `mcpp pack --pkg-format` accepts ``` mcpp pack --list-formats diff --git a/docs/README.md b/docs/README.md index 6c74499d3..54396ef5c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,11 +28,11 @@ anything a reader needs is written here or in a specification instead. | write a program | [01](01-getting-started.md), [04](04-mcpp-toml.md) §1 | [`01-hello`](../examples/01-hello/), [`02-with-deps`](../examples/02-with-deps/) | | write a library others import | [11](11-publishing-a-library.md), [06](06-features-and-capabilities.md), [04](04-mcpp-toml.md) §2.4 | [`04-workspace`](../examples/04-workspace/), [`11-features`](../examples/11-features/) | | publish it | [10](10-pack-and-release.md), [11](11-publishing-a-library.md), [12](12-binary-distribution.md) | [`03-pack-static`](../examples/03-pack-static/), [`05-lib-distribution`](../examples/05-lib-distribution/) | -| build for another machine | [21](21-the-target-triple.md), [24](24-openkal-cross.md), [40](40-baremetal.md) | [`06-openkal-cross`](../examples/06-openkal-cross/), `mcpp new … --template riscv-virt-rt` | +| build for another machine | [21](21-the-target-triple.md), [24](24-openkal-cross.md), [35](35-platforms.md), [40](40-baremetal.md) | [`06-openkal-cross`](../examples/06-openkal-cross/), `mcpp new … --template riscv-virt-rt` | | use a GPU or an accelerator | [42](42-heterogeneous-builds.md), [41](41-devices.md) | [`09-heterogeneous`](../examples/09-heterogeneous/), starting at [`boundary/`](../examples/09-heterogeneous/boundary/) | | add a rule, a language or a generator | [31](31-authoring-a-rule-package.md), [30](30-build-mcpp.md) | [`08-build-rules`](../examples/08-build-rules/), [`12-a-new-device-language`](../examples/12-a-new-device-language/) | | add a package to the index | [11](11-publishing-a-library.md), [SPEC-001](specs/package-identity.md) | [09](09-commands-by-scenario.md) — the publishing scenarios | -| package a tool, a driver or a board for others | [32](32-authoring-a-payload.md), [33](33-authoring-an-adapter.md), [34](34-authoring-a-bsp.md) | the descriptors in `xim-pkgindex` and `mcpp-index` | +| package a tool, a driver, a board or a platform for others | [32](32-authoring-a-payload.md), [33](33-authoring-an-adapter.md), [34](34-authoring-a-bsp.md), [35](35-platforms.md) | the descriptors in `xim-pkgindex` and `mcpp-index` | | change mcpp itself | [90](90-build-from-source.md), [92](92-release.md), [51](51-supported-versions.md) | — | Lessons also arrive as project templates, which a package ships and `mcpp new @@ -91,6 +91,7 @@ Within a part the order is a reading order, not an alphabet. - [32 — Authoring a Payload](32-authoring-a-payload.md) — a tool or a prebuilt library mcpp installs - [33 — Authoring a Runtime Adapter](33-authoring-an-adapter.md) — reaching a library the host supplies - [34 — Authoring a Board-Support Package](34-authoring-a-bsp.md) — a board, and the way in +- [35 — Platforms](35-platforms.md) — the configuration of a class of targets, as a package ### 4x — Devices and accelerators @@ -131,6 +132,7 @@ token in front of a reader to the chapter that owns it. | `min_api_level` | [04](04-mcpp-toml.md) | `macos_deployment_target` | [04](04-mcpp-toml.md) | | `[target..targets.] kind` | [22](22-target-side.md) | `[index.repos.]` in `config.toml` | [32](32-authoring-a-payload.md) | | `[test] discover` | [04](04-mcpp-toml.md), [08](08-testing.md) | `MCPP_RUNTIME_FILES` | [41](41-devices.md) | +| `[platform]`, `[host]`, `cfg(platform = …)` | [35](35-platforms.md) | `[target.] platform`, `layers`, `tools`, `link`, `carrier` | [35](35-platforms.md) | **Commands** @@ -140,6 +142,7 @@ token in front of a reader to the chapter that owns it. | `new`, `new --template` | [01](01-getting-started.md) | `add`, `update`, `why` | [05](05-dependencies.md) | | `pack` | [10](10-pack-and-release.md) | `publish`, `emit xpkg`, `xpkg parse` | [11](11-publishing-a-library.md) | | `toolchain` | [20](20-toolchains.md) | `clean`, `cache`, `index`, `self …` | [09](09-commands-by-scenario.md) | +| `--platform`, `--host-platform`, `self env --target` | [35](35-platforms.md) | `pack --pkg-format`, `--linkage`, `--merge` | [10](10-pack-and-release.md) | **Concepts** @@ -151,6 +154,7 @@ token in front of a reader to the chapter that owns it. | an island, a seam, `accel`, `MCPP_EXPORT_C` | [42](42-heterogeneous-builds.md) | a descriptor, an index | [11](11-publishing-a-library.md) | | an ABI tag, a prebuilt artifact | [12](12-binary-distribution.md) | exit codes, JSON output | [50](50-machine-output.md) | | an `xim:` payload, `[xlings.workspace]` | [32](32-authoring-a-payload.md), [23](23-the-project-environment.md) | a `compat:` adapter, `runtime.library_dirs` | [33](33-authoring-an-adapter.md) | +| a platform, a platform package, a version group | [35](35-platforms.md) | a universal package, a single-file package, `mcpp-package.json` | [10](10-pack-and-release.md) | ## Specifications @@ -171,3 +175,5 @@ downstream tooling. - [SPEC-010 — Home selection and initialization](specs/home-initialization.md) - [SPEC-011 — Concurrent runs: the build directory lock, whole outputs, optimistic writes, cancellation](specs/concurrency.md) - [SPEC-012 — Output for people: two streams, two media, the kinds of line, colour](specs/human-output.md) + - [SPEC-013 — Platforms: the configuration of a class of targets, its selection, merge, graph completion, link, seal, runner and carrier](specs/platforms.md) + - [SPEC-014 — Packaging and distribution: link, closure and container, legs, the staged tree, universal and single-file packages](specs/packaging.md) diff --git a/docs/specs/README.md b/docs/specs/README.md index fb202045d..a5f49f390 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -42,6 +42,8 @@ | [SPEC-010](home-initialization.md) | home 选择、单跳入口、初始化、网络边界与单次运行的 `--home` | 草案 v0.2 | 2026-10-26 | mcpp >= 2026.10.19.1；网络能力逐条标注；§6 mcpp >= 2026.10.26.1、xlings >= 2026.10.26.1 | | [SPEC-011](concurrency.md) | 并发运行：三段、构建目录锁、原子发布、`mcpp.lock` 的乐观写入、取消 | 评审中 v1.0 | 2026-10-26 | mcpp >= 2026.10.26.1 | | [SPEC-012](human-output.md) | 面向人的输出：两条流、两种介质、行的种类、颜色 | 评审中 v1.0 | 2026-10-26 | mcpp >= 2026.10.26.1 | +| [SPEC-013](platforms.md) | 平台:`[platform]` 的七组键、版本组与 `hosted`,三种写法与层次,目标图与宿主图的选择,`cfg(platform)`,依赖图的补全,链接、命名与封装,运行器与载体,平台阶段与协议 17 | 评审中 v1.0 | 2026-10-28 | mcpp >= 2026.10.27.1 | +| [SPEC-014](packaging.md) | 打包与分发:链接、闭包、容器三个维度与构建腿,暂存树与 `mcpp-package.json`,通用包、单文件、签名、压缩、合并与分发描述 | 评审中 v1.0 | 2026-10-28 | mcpp >= 2026.10.27.1;逐条标注 | ## 文档约定 diff --git a/docs/specs/build-plugins.md b/docs/specs/build-plugins.md index 538e15430..42ebabdb2 100644 --- a/docs/specs/build-plugins.md +++ b/docs/specs/build-plugins.md @@ -5,9 +5,9 @@ | 规范编号 | SPEC-007 | | 标题 | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | | 状态 | 草案 v0.9 | -| 版本 | 0.10 | -| 最后修改 | 2026-10-27 | -| 对应实现 | 逐条标注。未注明版本的「已实现」条款对应 mcpp >= 2026.9.26.1;注明 mcpp#702 的条款对应 mcpp >= 2026.9.26.2;注明 mcpp#707、#708、#709、#711 的条款对应 mcpp >= 2026.9.27.1;注明 mcpp#723 的条款对应 mcpp >= 2026.9.28.1;注明 mcpp 2026.9.28.2 的条款对应该版本;§9 与注明 mcpp#734 的条款对应 mcpp >= 2026.9.28.3;注明 mcpp#778 的条款对应 mcpp >= 2026.10.10.1;§10 对应 mcpp >= 2026.10.11.1 | +| 版本 | 0.11 | +| 最后修改 | 2026-10-28 | +| 对应实现 | 逐条标注。未注明版本的「已实现」条款对应 mcpp >= 2026.9.26.1;注明 mcpp#702 的条款对应 mcpp >= 2026.9.26.2;注明 mcpp#707、#708、#709、#711 的条款对应 mcpp >= 2026.9.27.1;注明 mcpp#723 的条款对应 mcpp >= 2026.9.28.1;注明 mcpp 2026.9.28.2 的条款对应该版本;§9 与注明 mcpp#734 的条款对应 mcpp >= 2026.9.28.3;注明 mcpp#778 的条款对应 mcpp >= 2026.10.10.1;§10 对应 mcpp >= 2026.10.11.1;R9.10 与协议 17 对应 mcpp >= 2026.10.27.1 | | 相关设计文档 | `.agents/docs/2026-09-26-compile-database-and-issue-699-design.md`(§5)、`.agents/docs/2026-09-28-ecosystem-design-and-optimisation-plan.md`(WS1、WS3)、`.agents/docs/2026-09-28-build-cost-foreign-toolsets-and-library-surface-design.md`(§3、§4)、`.agents/docs/2026-10-10-one-source-one-meaning-design.md`(§4)、`.agents/docs/2026-10-11-build-dependencies-one-pipeline-design.md` | | 相关 issue | mcpp#699、mcpp#701、mcpp#702、mcpp#703、mcpp#707、mcpp#708、mcpp#709、mcpp#711、mcpp#734、mcpp#755、mcpp#778、mcpp#797 | | 使用文档 | [docs/30 - build.mcpp](../30-build-mcpp.md)、[docs/31 - 编写规则包](../31-authoring-a-rule-package.md) | @@ -258,6 +258,7 @@ | 14 | 2026.9.28.3 | `mcpp.core`、构建信息(R9.3)、`mcpp::report`(R9.4) | | 15 | 2026.10.1.3 | 来源(R6.5、R6.6、R9.9):`xpkg_source`、`xpkg_program`、`xpkg_request`、`xpkg_pending`、`phase`、`decision`、`toolchain` | | 16 | 2026.10.26.2 | 镜像入口:`link_start`(`mcpp:link-start=`,docs/04 `start`) | + | 17 | 2026.10.27.1 | 平台的链接与运行器字段(SPEC-013 §12):`link_driver`、`link_input`、`link_input_from`、`artifact_name`、`seal`、`run_host`;平台阶段(`mcpp::phase()` 为 `"platform"`) | - **R9.3** 构建信息以事实陈述解析出的工具链,不针对任何外部构建系统:`tool(role)`(这一行的 工具)、`abi_tool(role)`(目标 ABI 的原生工具,MSVC ABI 上为工具集的 `cl`、`link`、`lib`、 @@ -289,6 +290,12 @@ 提供构造这些陈述的函数。编译并运行构建程序的是 bootstrap 工具链,因此一个无法使用的 自定义工具链**必须**仍能让构建程序运行并报告原因。(**已实现**,mcpp 2026.10.1.3) +- **R9.10** 平台包的 `[platform] plugin` 在平台阶段运行(`mcpp::phase()` 为 `"platform"`,`MCPP_PHASE=platform`)。 + 该阶段只能陈述编译与链接标志、`link_script`、`link_start`、协议 17 的指令、`runner` 与命名运行器、 + `run_exclusive`,以及重跑条件、警告与诊断;其他指令**必须**被拒绝(拒绝码 `platform-phase-directive`), + 消息点名第一条不属于该阶段的指令。陈述的内容在 SPEC-013 的 Program 层次合并;协议 17 的指令也可以由 + 任何构建程序在构建阶段发出。(**已实现**,mcpp 2026.10.27.1) + ## 10. 构建程序的库与构建插件(mcpp#797) - **R10.1** 构建程序的依赖是其包 `[build-dependencies]` 中请求库的条目(SPEC-004 §12 第 5 条), @@ -330,3 +337,4 @@ | 0.2 | 2026-09-26 | 随 mcpp 2026.9.26.2 落地:R1.3 的警告、R2.1 的 `runtime_search_dir`、R2.4、R3.3 的 `prepare`(目录须含文件;链接边等待所有 `prepare`)、R3.5、R3.6、R4.1、R4.3、R5.2、R5.3 标为已实现。 | | 0.9 | 2026-10-11 | 随 mcpp 2026.10.11.1(mcpp#797):新增 §10 构建程序的库与构建插件(R10.1–R10.6,P1–P5);变更记录移为 §11。 | | 0.10 | 2026-10-27 | 随 mcpp 2026.10.26.2(mcpp#845):R9.2 的协议表增加协议 16(`link_start`,镜像入口)。 | +| 0.11 | 2026-10-28 | 随 mcpp 2026.10.27.1(mcpp#847):R9.2 的协议表增加协议 17(平台的链接与运行器字段);新增 R9.10 平台阶段(SPEC-013)。 | diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 2d2e4fe43..df9fc4415 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -5,8 +5,8 @@ | **规范编号** | SPEC-004 | | **标题** | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | | **状态** | **草案(Draft)** | -| **版本** | 1.18 | -| **最后修改** | 2026-10-27 | +| **版本** | 1.19 | +| **最后修改** | 2026-10-28 | | **最低实现版本** | 条件化形状:mcpp **2026.8.29.1**(`[target..build-dependencies]` 起齐备);目标轴:mcpp **2026.9.6.4** | | **作者/维护** | mcpp-community | | **相关设计文档** | `.agents/docs/2026-09-07-mcpp-toml-unified-semantics-design.md`
`.agents/docs/2026-06-04-manifest-schema-ownership.md`
`.agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md`
`.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md`
`.agents/docs/2026-09-27-eight-reports-by-home-and-one-optimisation-plan.md`
`.agents/docs/2026-10-05-std-module-pair-msvc-lto-and-export-discovery-design.md`
`.agents/docs/2026-10-06-windows-x86-arch-vocabulary-and-rc-follow-ups-design.md`
`.agents/docs/2026-10-10-one-source-one-meaning-design.md`
`.agents/docs/2026-10-11-build-dependencies-one-pipeline-design.md`
`.agents/docs/2026-10-23-target-base-and-execution-host-design.md`
`.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md`(§24) | @@ -54,6 +54,7 @@ | 条件 | `[target..
]` | 在哪个目标上要 | | 产物元数据 | `[runtime]`、`[resources]` | 产出的东西是什么 | | 生命周期 | `[hooks]` | 构建前后跑什么 | +| 平台 | `[platform]`、`[host]` | 一类目标的配置;宿主图的平台(SPEC-013) | **状态:已实现。** @@ -66,8 +67,9 @@ **值的层次。** 同一个键的值按四档取第一个:覆盖(命令行与环境变量)、工程(`mcpp.toml`, 含工作空间层)、机器默认(`~/.mcpp/config.toml`,其次系统级 `config.toml`)、引擎默认。 -`[target.<三元组>].sysroot` 与 `[target.<三元组>].run_under` 两个键**可以**同时出现在工程与 -机器默认中;其余键只属于工程。环境变量中空值视为未设置,`none` 表示显式清除。 +`[target.<三元组>].sysroot`、`[target.<三元组>].run_under`、`[target.]` 的平台键(`platform`、 +`layers`、`tools`、`link`、`carrier`、`runner`、`runners`)与 `[host] platform` **可以**同时出现在工程与机器默认中; +其余键只属于工程。平台键的层次另见 SPEC-013 §2。环境变量中空值视为未设置,`none` 表示显式清除。 机器默认**禁止**放宽工程的约束:`requires` 与版本下限照样核对。 **库依赖与工具是两个平面,不是一个。** 库有模块与 ABI,参与解析与链接;载荷是可执行 @@ -196,6 +198,8 @@ feature-deps feature-xlings ← 限定词是门 | `[target..xlings…]` | 目标 | **已实现**(2026.9.6.4) | | `[target.].sysroot` | 目标 | **已实现** | | `[target.].run_under` | 目标(执行宿主) | **已实现**(2026.10.24.1) | +| `[target.]` 的 `platform`、`layers`、`tools`、`link`、`carrier` | 目标 | **已实现**(2026.10.27.1,SPEC-013) | +| `[host] platform` | 宿主 | **已实现**(2026.10.27.1,SPEC-013 §4) | 顶层平台键对象**不是**遗留拼法,它是宿主轴**正确**的写法:工具必须能在这台机器上 执行。`[target.].sysroot` 是目标轴 xpkg 引用的既有先例。 @@ -787,3 +791,4 @@ mcpp 2026.10.10.1,#785、#786;第 14 至 16 条与第 13 条的依赖位置为 m | 1.16 | 2026-10-21 | 新增 §13，明确依赖 EXE、DLL 的资源只进入其所属包的 image，且按计划内目标校验 manifest 冲突(#826)。 | | 1.17 | 2026-10-23 | mcpp 2026.10.24.1:§2 陈述值的四档层次,`sysroot` 与 `run_under` 可出现在工程与机器默认中;§4.2 增加 `run_under`,陈述 `sysroot` 的表写法。 | | 1.18 | 2026-10-27 | mcpp 2026.10.26.2(mcpp#845):新增 §14 程序起点与镜像入口(`kind = "entry"`、`main = { from }`、`start`、`[test] main` 与 `start`、`mcpp:link-start`);§3.1.1 的按行形式可陈述起点。 | +| 1.19 | 2026-10-28 | mcpp 2026.10.27.1(mcpp#847):§2 增加平台平面(`[platform]`、`[host]`),平台键可出现在工程与机器默认中;§4.2 增加 `[target.]` 的平台键与 `[host] platform`(SPEC-013)。 | diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md new file mode 100644 index 000000000..56171f33d --- /dev/null +++ b/docs/specs/packaging.md @@ -0,0 +1,280 @@ +# SPEC-014:打包与分发 + +| 项 | 值 | +|---|---| +| 规范编号 | SPEC-014 | +| 标题 | 打包与分发:链接、闭包、容器三个维度与构建腿,暂存树与包元数据,通用包、单文件、签名、压缩、合并与分发描述 | +| 状态 | 评审中 v1.0 | +| 版本 | 1.0 | +| 最后修改 | 2026-10-28 | +| 对应实现 | mcpp >= 2026.10.27.1;逐条标注 | +| 相关设计文档 | `.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`(§2–§7、§14)、`.agents/docs/2026-10-28-platforms-design.md`(§4.7) | +| 相关 issue | mcpp#847、mcpp#849 | +| 相关规范 | [SPEC-007](build-plugins.md)(包格式提供者、artifact action)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-013](platforms.md)(平台、载体) | +| 使用文档 | [10 —— 发布打包](../zh/10-pack-and-release.md) | + +用语按 RFC 2119:**必须 / 禁止**为强制,**应当**为建议,**可以**为可选。 + +## 0. 范围 + +本规范规定 `mcpp pack` 为一个**程序**目标产出可分发物的模型、管线、暂存树、包元数据、容器与接口,以及 +`mcpp run --pkg-format` 与 `mcpp emit xpkg` 读取的部分。库的预编译包(docs/12)不在本规范内。 + +## 1. 模型:三个维度与构建腿 + +一次打包的选择由三个正交维度与一组构建腿组成,每个维度回答一个问题: + +| 维度 | 选项 | 回答的问题 | 归属 | +|---|---|---|---| +| 链接 `--linkage` | `static`、`shared` | 库在程序里,还是在程序旁边 | 程序属性 | +| 闭包 `--mode` | `system`、`vendored`、`self-contained` | 运行的机器要提供什么 | 打包属性 | +| 容器 `--pkg-format` | `dir`、`tar`、`zip`、`single-file`,以及提供者的格式 | 交付成什么 | 打包属性 | +| 腿 `--target`(可重复) | 三元组 | 一次打包覆盖哪些目标 | 打包属性 | + +### 1.1 链接 + +`--linkage` 对应 `[build] dependency_linkage`,打包按它构建。原生 Linux 目标上 `static` 选择 `-linux-musl` +行,产物为全静态程序;openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` 加其所需的 `.okso`。 +未给出时取 `[pack] linkage`,再取构建的默认值。**未实现** + +### 1.2 闭包 + +| 模式 | 原生目标:包携带 | openkal 目标:包携带 | 运行的机器提供 | +|---|---|---|---| +| `system` | 程序 | 映像 | 全部共享库;openkal 上为加载器与映像所需的 `.okso` | +| `vendored`(缺省) | 程序与非系统共享库 | 映像与其 `.okso` | C 库、C++ 运行时与加载器;openkal 上为加载器 | +| `self-contained` | 程序、全部共享库、C 与 C++ 运行时、加载器 | 映像、`.okso` 与本机程序形式的加载器(载体,SPEC-013 §11) | 无 | + +`bundle-project` 与 `bundle-all` 是 `vendored` 与 `self-contained` 的永久别名。**已实现** + +`--mode static` 是 `--linkage static --mode self-contained` 的简写:原生目标上为全静态程序;openkal 目标上为一个静态 +`.oke` 加载体。**部分实现**:原生目标已实现;openkal 目标上 `--mode static` 当前被拒绝。 + +### 1.3 容器 + +| 值 | 产物 | +|---|---| +| `dir` | 暂存树本身 | +| `tar` | 一个归档,单一顶层目录与归档同名;缺省压缩为 gzip(§5.1) | +| `zip` | 一个 ZIP 归档,条目以 stored 存放;PE 目标的 `tar` 产出 `zip` | +| `single-file` | 一个可执行文件(§5.2) | +| 其他名字 | 依赖图中的提供者以 `pack-format` 声明的格式(§6) | + +`dir`、`tar` 与提供者格式 **已实现**;`zip` 作为显式取值与 `single-file` **未实现**。 + +### 1.4 构建腿 + +`--target` 可以重复。每条腿是一次构建;各腿暂存进同一棵树(§4)。平台以 `[platform] universal = true` 声明其目标 +默认打**通用包**:对该平台任一目标的打包缺省覆盖平台的全部 `targets`;`--no-universal` 只打所选目标。一个产物为 +可执行文件的目标,只有在通用包中或在产物为共享对象的行(Android 的 `kind = "app"`)上接受多条腿。 +Android 的多腿 **已实现**;`universal` 与 `--no-universal` **未实现**。 + +### 1.5 组合的合法性 + +组合在任何构建开始之前检查;不可能的组合**必须**被拒绝,消息写出冲突的两个选择与出路。 + +| 组合 | 结果 | +|---|---| +| `--pkg-format single-file` 与 `--mode system` | 拒绝:单文件携带它所运行的东西;出路 `--mode vendored` 或 `self-contained` | +| 需要宿主提供能力的程序(docs/04 §2.11)与 `--mode self-contained` 或 `static` | 拒绝:携带自己 C 库的包不能使用宿主提供的库;出路 `--mode vendored` | +| `cxx_runtime = "toolchain-coupled"` 与 `--mode system` | 拒绝 | +| 产物为可执行文件、非通用包,且多条 `--target` | 拒绝:一个可执行文件属于一个目标 | +| `--systems` 而非 `--mode self-contained` 的通用包 | 拒绝:只有自带加载器的包按系统区分 | +| `--merge` 的输入缺少 `mcpp-package.json`,或名字、版本、平台不同 | 拒绝,写出不一致的字段 | +| `--merge` 的两个输入含同名映像而摘要不同 | 拒绝,写出该映像与两个摘要 | +| 库目标(`lib`、`shared`)与提供者格式 | 拒绝:库包没有单一暂存树 | + +前三行与库目标一行 **已实现**;其余 **未实现**。 + +## 2. 选项与配置 + +### 2.1 选项的全局用法 + +| 选项 | 命令 | 含义 | +|---|---|---| +| `--message-format json` | 构建类命令:`build`、`test`、`run`、`pack` | 机器输出 | +| `--format <编码>` | 报告类命令:`self env`、`toolchain list`、`emit build-database` 等 | 输出编码 | +| `--pkg-format <格式>` | `pack`、`run` | 包格式 | + +`pack` 与 `run` 继续接受 `--format <格式>`,行为与 `--pkg-format` 相同,并以一条警告写出它是旧写法。**已实现** + +### 2.2 `[pack]` + +| 键 | 语义 | +|---|---| +| `default_mode` | 未给 `--mode` 时的闭包模式 | +| `linkage` | 未给 `--linkage` 时的链接 | +| `format` | 未给 `--pkg-format` 时的容器 | +| `compress` | 未给 `--compress` 时的压缩(§5.1) | +| `sign` | 签名步骤的 argv,`{}` 为文件(§7) | +| `strip`、`debug_symbols`、`include`、`exclude`、`[pack.bundle-project]` | 既有键 | + +`default_mode` 与既有键 **已实现**;`linkage`、`format`、`compress`、`sign` **未实现**。 + +## 3. 管线 + +一次打包依次经过: + +1. **构建各腿**:按维度与腿构建,`release` 为缺省构建配置。 +2. **收集闭包**:按 `--mode` 决定每条腿携带的库;闭包记录在暂存清单(docs/50)中,未解析的名字使 `dir` 与 `tar` + 拒绝。 +3. **生成暂存树**:§4 的布局;通用包生成启动器;`self-contained` 加入加载器或载体。 +4. **变换**:strip、重定位、平台的封装(SPEC-013 §9.3)、签名(§7)。 +5. **写容器**:§5;提供者格式在第二遍准备中读取暂存树(§6)。 +6. **报告**:产物、每个产物的校验和、`mcpp.pack` 信封(§9)。 + +`--merge` 是"读入若干暂存树 → 合并 → 写容器",不经第 1、2 步。第 1、2、3、4(strip、重定位、封装)、5、6 步的 +单腿形式 **已实现**;签名、通用包与 `--merge` **未实现**。 + +## 4. 暂存树与包元数据 + +### 4.1 单腿布局 + +单腿包的布局由目标的二进制格式决定(ELF 的 `bin/` 与 `lib/`,PE 的扁平目录,Mach-O 的 `bin/`,Android 的 `lib/`, +WebAssembly 的同名文件族),规定见 docs/10。**已实现** + +### 4.2 通用包布局 + +``` +--/ + bin/ POSIX 启动器 + bin/.cmd Windows 启动器 + lib//.oke, *.okso 每个架构一组映像 + libexec//-/... 仅 --mode self-contained + share/... 资源,只放一份 + mcpp-package.json +``` + +包名带平台名,不带系统与架构。**未实现** + +### 4.3 启动器 + +启动器由引擎生成,内容只来自平台的 `runner` 与腿: + +- 架构目录按 `uname -m`(POSIX)或 `%PROCESSOR_ARCHITECTURE%`(Windows)选择;映射为 `x86_64`、`amd64`、`AMD64` → + `x86_64`,`aarch64`、`arm64`、`ARM64` → `aarch64`;没有对应目录时以非零状态退出,写出本机架构与包含的架构。 +- 运行 `runner.cmd`,`{}` 替换为所选架构目录中的入口映像,其余参数原样传递;`self-contained` 时先在 + `libexec//-/` 中找加载器,再找 `PATH`。 + +**未实现** + +### 4.4 `--systems` + +`--systems [,...]`(缺省为本机系统)为每个系统 × 平台架构构建加载器程序并放入 `libexec/`。缺少某个系统的 +SDK 载荷时拒绝,写出该系统与载荷。**未实现** + +### 4.5 `mcpp-package.json` + +暂存树根的 `mcpp-package.json` 是包格式提供者、`--merge` 与 `mcpp emit xpkg` 的输入: + +```json +{ + "schema": 1, + "name": "hello", + "version": "0.1.0", + "platform": "mcpplibs.openkal-platform@0.17.2", + "linkage": "shared", + "mode": "vendored", + "legs": ["x86_64-openkal", "aarch64-openkal"], + "systems": [], + "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, + "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "..."}] +} +``` + +| 字段 | 语义 | +|---|---| +| `schema` | 本 schema 的版本,当前为 1 | +| `name`、`version` | 包的名字与版本 | +| `platform` | 平台包与版本;无平台时为 `null` | +| `linkage`、`mode` | §1 的两个维度 | +| `legs` | 腿的规范三元组 | +| `systems` | `libexec/` 中加载器所属的系统;无加载器时为空 | +| `launch` | 运行方式:`runner` 的 argv 与每个架构的入口;单腿原生包为程序的相对路径 | +| `files` | 每个文件:`path`(相对于树根,`/` 分隔)、`role`(`program`、`image`、`library`、`loader`、`launcher`、`resource`、`metadata`)、`leg`(所属的腿,共用文件为 `null`)、`sha256` | + +**未实现** + +### 4.6 合并 + +`mcpp pack --merge ...` 读入若干包(目录或归档),按 `mcpp-package.json` 合并为一棵树:腿与系统取并集,同一路径 +的文件摘要**必须**相同,资源只放一份;再按 `--pkg-format` 写容器。**未实现** + +## 5. 容器 + +### 5.1 压缩 + +`--compress [:]` 与 `[pack] compress` 选择 `tar` 的压缩;`tar` 的缺省为 gzip,产物扩展名为 +`.tar.gz`、`.tar.zst`、`.tar.xz` 或 `.tar`。`zip` 的条目以 stored 存放。单文件的载荷使用 zstd。okb 中的映像不压缩 +(SPEC-013 平台插件的格式规定)。gzip **已实现**;`--compress` 与其余取值 **未实现**。 + +### 5.2 单文件 + +| 组合 | 产物 | 运行 | +|---|---|---| +| 原生目标,`vendored` 或 `self-contained` | 引擎的启动桩加 zstd 压缩的暂存树 | 首次运行把载荷解到用户缓存,之后直接执行 | +| 原生 Linux 目标,`--linkage static` | 静态启动桩加静态程序 | 同上,不依赖任何系统库 | +| openkal 目标,`self-contained` | 平台的载体加本架构的映像组 | 载体经加载器库的 bundle 接口从内存加载,不写盘 | + +启动桩的规定: + +1. 源码随引擎提供,打包时为目标构建;它只做校验、解压与执行。 +2. 缓存根:`$XDG_CACHE_HOME`,未设置时 `~/.cache`(Linux);`~/Library/Caches`(macOS);`%LOCALAPPDATA%` + (Windows);其下按载荷的 sha256 寻址。 +3. 解压前**必须**校验载荷摘要;解到同一目录下的临时名,完成后原子改名;**禁止**复用不完整的目录;缓存目录只对 + 当前用户可写。 +4. 以原有的参数与环境执行暂存树中的程序,退出状态即程序的退出状态。 + +**未实现** + +### 5.3 一个文件在多个系统上运行 + +一个文件同时在多个系统与架构上运行(APE)不在本规范内(mcpp#849)。 + +## 6. 包格式提供者 + +一个包以构建程序的 `mcpp::provides_pack_format("")` 声明一个格式,以 `role = "artifact"` 的 action 生成产物 +(SPEC-007)。`--pkg-format ` 找到图中声明该名字的包,在第二遍准备中把暂存树交给它:暂存树目录 +(`mcpp::pack_stage_dir()`、`${mcpp.stage_dir}`)、`mcpp-package.json`、`MCPP_PACK_*` 环境;它报告的终端输出即产物。 +未知的名字被拒绝,消息列出图中可用的格式。**已实现**(`mcpp-package.json` 除外,**未实现**) + +平台插件(SPEC-013 §12)**可以**在平台阶段声明包格式与 artifact action;平台阶段的指令表因此包含 `pack-format` +与 artifact action。该平台的目标打包时,这些格式与引擎的格式并列。**未实现** + +## 7. 签名 + +`[pack] sign = ["", ..., "{}"]` 对暂存树中的每个程序与映像运行一次,`{}` 为文件,在写容器之前;一个包格式 +提供者或构建插件**可以**代替它签名。任一次失败使打包失败,写出 argv 与输出。**未实现** + +## 8. 分发描述 + +`mcpp emit xpkg` 由 `mcpp-package.json` 生成索引条目:每个系统 × 架构一项;或通用包时一个资产对应所有系统, +URL 相同,并写出运行所需的依赖(加载器的包)。**未实现** + +## 9. 机器输出 + +`mcpp pack --message-format json` 打印一个 `mcpp.pack` 信封(docs/50)。`data` 增加: + +| 字段 | 语义 | +|---|---| +| `linkage`、`mode`、`pkgFormat` | 三个维度的取值 | +| `legs` | 腿的规范三元组 | +| `systems` | 加载器所属的系统 | +| `artifacts[].sha256` | 每个文件产物的 sha256;目录产物为 `null` | + +`artifacts[].format` 保留,与 `pkgFormat` 相同。**未实现** + +## 10. 判据 + +1. 每个不合法的组合在任何编译之前被拒绝,消息写出冲突的选择与出路。 +2. `--format` 在 `pack` 与 `run` 上与 `--pkg-format` 等价,并写出警告。 +3. 通用包在每个平台架构的机器上经启动器运行;`self-contained` 的通用包在未安装加载器的机器上运行。 +4. `--merge` 的结果与一次打出全部腿与系统的包逐文件相同。 +5. 单文件在载荷摘要不符、缓存目录不完整时不执行不可信的内容;openkal 单文件运行时缓存目录中无新文件。 +6. `mcpp-package.json` 的 `files` 覆盖树中每个文件,摘要与文件一致。 + +## 变更记录 + +| 版本 | 日期 | 变更 | +|---|---|---| +| 1.0 | 2026-10-28 | 首版(mcpp#847):三个维度与构建腿、组合的合法性、选项的全局用法与 `--pkg-format`、`[pack]` 的新键、管线、单腿与通用包的布局、启动器、`--systems`、`mcpp-package.json`、`--merge`、压缩、单文件、包格式提供者与平台插件、签名、分发描述、`mcpp.pack` 的新字段。 | diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md new file mode 100644 index 000000000..f225fec9b --- /dev/null +++ b/docs/specs/platforms.md @@ -0,0 +1,431 @@ +# SPEC-013:平台 + +| 项 | 值 | +|---|---| +| 规范编号 | SPEC-013 | +| 标题 | 平台:一类目标的可复用配置,其声明、选择、合并、对依赖图的补全、链接与封装、运行器与载体 | +| 状态 | 评审中 v1.0 | +| 版本 | 1.0 | +| 最后修改 | 2026-10-28 | +| 对应实现 | mcpp >= 2026.10.27.1;逐条标注 | +| 相关设计文档 | `.agents/docs/2026-10-28-platforms-design.md`、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md` | +| 相关 issue | mcpp#846、mcpp#847、mcpp#848 | +| 相关规范 | [SPEC-002](target-side.md)(目标侧的层)、[SPEC-004](manifest-semantics.md)(条件化形状)、[SPEC-006](toolchain-management.md)(工具链选择)、[SPEC-007](build-plugins.md)(构建程序与协议)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-014](packaging.md)(打包) | +| 使用文档 | [35 —— 平台](../zh/35-platforms.md) | + +用语按 RFC 2119:**必须 / 禁止**为强制,**应当**为建议,**可以**为可选。 + +## 0. 范围与词汇 + +本规范规定**平台**:一类目标的一份可复用、带版本、可发布的配置。平台由一个包在自己的清单中以 +`[platform]` 声明,由引擎选择并合并,其内容进入工具链的决定、依赖图、编译与链接的命令、产物的 +命名与封装、运行器与打包。引擎不持有任何一个平台的知识;它提供本规范规定的机制。 + +| 词 | 含义 | +|---|---| +| 平台包 | 清单含 `[platform]` 段的包 | +| 目标图 | 为 `--target` 构建的包 | +| 宿主图 | 为构建机器构建的程序:构建程序(`build.mcpp`、平台插件)与本机工具 | +| 层 | `kernel-abi`、`c-abi`、`c++-abi`、`compiler-runtime`(SPEC-002) | +| 工具 | 在构建机器上运行的程序(例:`okld`) | +| 封装(seal) | 链接之后、发布之前对产物本身执行的步骤 | +| 运行器 | `mcpp run`、`mcpp test` 执行产物的方式;`runners.` 为命名的设备动作 | +| 载体 | 把依赖加载器的产物与加载器合成为一个本机程序的宿主程序包(M3) | +| 平台陈述 | 一份平台数据(`Statement`):本规范 §1 的全部字段,每个值带来源 | + +## 1. `[platform]` 段 + +一个包的清单含 `[platform]` 段时,该包是平台包。`[platform]` **必须**含 `targets`;否则清单被拒绝。 +**已实现** + +```toml +[platform] +targets = ["", ...] +toolchain = "" +sysroot = "" | { path = "", provides = [...] } +runner = ["", "{}"] | { cmd = [...], host = "same-arch" | "any", exclusive = } +runners = { = ["", ...] } +carrier = { package = "", section = "", main = "" } +plugin = "build.mcpp" + +[platform.layers] = | false +[platform.tools] = | { program = "" } +[platform.build] cflags / cxxflags / asmflags / ldflags = [...] +[platform.build.'cfg()'] 同上 +[platform.link] driver, flags, start, script, inputs, exe, shared, seal +[platform.versions] = "" +[platform.hosted] targets, toolchain, layers +``` + +### 1.1 键 + +| 键 | 类型 | 语义 | +|---|---|---| +| `targets` | 非空字符串数组 | 平台服务的目标三元组 | +| `toolchain` | 字符串 | 约定的工具链规格(SPEC-006),例 `llvm@23.1.3` | +| `sysroot` | 字符串或 `{ path, provides }` | 预构建的目标基,写法同 `[target.].sysroot`;`path` 相对于平台包 | +| `runner` | 字符串数组或表 | §10 | +| `runners` | ` = [argv]` | 命名的设备动作,每个 argv 非空 | +| `carrier` | 表 | §11;`package`、`section`、`main` 三键都**必须**给出 | +| `plugin` | 字符串 | 平台包中的构建程序,在平台阶段运行(§12);路径相对于平台包 | +| `layers` | 表 | §7.2;键只能是 §0 的四个层名 | +| `tools` | 表 | §7.3 | +| `build` | 表 | 四个标志列表,以及键形如 `cfg(...)` 的子表(同四个键) | +| `link` | 表 | §9 | +| `versions` | ` = ""` | 版本组(§7.5) | +| `hosted` | 表 | 宿主三元组选用本平台时适用的部分(§3.4);键只能是 `targets`、`toolchain`、`layers` | + +**包引用。** `layers`、`tools`、`carrier.package` 与选择平台的值都是包引用:字符串 +`"[.]@"`,或表。表的键为 `name`、`version`、`path`、`git` 与 +`rev` / `tag` / `branch` 之一、`features`、`default-features`;**必须**恰好给出 `version`、`path`、 +`git` 之一;`version` 与 `name` 同时出现;`path` 相对于写它的文件;`path` 或 `git` 的引用**可以** +省略 `name`,此时由该包自己的清单命名。`false` 只在 `layers` 中有效;`{ program = "" }` 只在 +`tools` 中有效,且不带其他键。**已实现** + +**argv 约定。** argv 中的 `{}` 为产物;程序以名字书写,按"图中为本机构建的程序 → 声明的载荷 → +`PATH`"查找。**已实现** + +**错误。** 类型错误是拒绝,消息写出键;`[platform]` 与 `[platform.hosted]`、`[platform.link]` 等子表中 +引擎不认识的键记为警告(`--strict` 下拒绝),使为更新的引擎写的平台包仍能被读取;平台包以 +`[package] mcpp = ">="` 声明下限(SPEC-007 R9.8)。**已实现** + +### 1.2 只能声明的键与可由代码陈述的键 + +`targets`、`toolchain`、`sysroot`、`layers`、`tools`、`carrier`、`versions`、`hosted` 在解析工具链与依赖图 +时需要,那时没有构建程序可以运行,因此**只能**在清单中声明。`build`、`link`、`runner`、`runners` **可以** +由平台插件或构建程序陈述(§12)。**已实现** + +## 2. 三种写法与层次 + +同一组字段有三种同义的写法,解析为同一结构,每个值带来源: + +| 写法 | 位置 | 层次 | +|---|---|---| +| 约定 | 平台包的 `[platform]` | Platform | +| 陈述 | 平台插件与构建程序的协议 17 指令(§12) | Program | +| 覆盖 | `config.toml` 的 `[target.]`、`[host]` | Config | +| 覆盖 | 工程 `mcpp.toml` 的 `[target.]`、`[host]` | Manifest | +| 覆盖 | 环境变量 `MCPP_TARGET__PLATFORM`、`MCPP_TARGET__RUNNER`、`MCPP_HOST_PLATFORM` | Env | +| 覆盖 | `--platform`、`--host-platform` | Flag | + +层次按 Platform < Program < Config < Manifest < Env < Flag 排列,后者按字段替换前者(§6)。另有两个 +来源:`built-in`(目标表或宿主约定)与 `graph`(依赖图中的包已经提供,§7.2)。**已实现** + +`[target.]` 中与平台对应的键为 `platform`、`layers`、`tools`、`carrier`、`link`,以及表原有的 +`runner`、`runners`、`toolchain`、`sysroot`;`[host]` 只接受 `platform`。覆盖的键名与 `[platform]` 相同。 +`` 为三元组大写、非字母数字字符改为 `_` 的形式(例:`MCPP_TARGET_X86_64_OPENKAL_PLATFORM`)。**已实现** + +## 3. 目标图的选择 + +### 3.1 顺序 + +目标图的平台由第一个给出答案的来源决定: + +1. `--platform <值>`,即本次调用的 `MCPP_PLATFORM`; +2. `MCPP_TARGET__PLATFORM`; +3. 工程的 `[target.] platform`(工作空间层先于根包,§13); +4. `config.toml` 的 `[target.] platform`; +5. 目标表的平台列(§14); +6. 无:宿主目标的内置约定,或无平台。 + +第 3、4 步在命中的表中取具体程度最高者(SPEC-004 §3.1.1)。**已实现** + +### 3.2 值 + +| 值 | 含义 | +|---|---| +| `none` | 不使用平台 | +| `"@"`、包引用表 | 一个平台包,按普通依赖解析、获取与锁定 | +| `path:`,或写成路径的值(以 `/`、`./`、`../`、`~` 开头,Windows 盘符,或含路径分隔符且不含 `@`) | 一个目录中的平台包;命令行与环境变量中的相对路径相对于 mcpp 的启动目录 | + +一个目录的清单不可读时,选择**必须**以该清单的错误失败。**已实现** + +### 3.3 `platform` 的位置限制 + +`platform` **禁止**写在询问平台的选择器下(`[target.'cfg(platform = "...")'] platform`),因为该选择器的 +答案由 `platform` 决定;清单被拒绝,消息给出可用的选择器(三元组,或 `os`、`arch`、`env` 的 `cfg`)。 +**已实现** + +### 3.4 宿主三元组选用平台 + +被选中的平台不服务所求目标时: + +- 目标在 `targets` 中:适用整个平台。 +- 目标是宿主三元组,平台有 `[platform.hosted]`,且 `hosted.targets` 未写或含该三元组:只适用 + `hosted` 的 `toolchain` 与 `layers`,以及 `versions`;报告与 JSON 标为 `hosted`。 +- `hosted.targets` 不含该三元组:拒绝(`platform-target-unserved`),消息列出 `hosted.targets`,并在其中 + 有同系统同架构的三元组时给出改用它的命令。 +- 平台无 `[platform.hosted]`:拒绝(`platform-target-unserved`),消息列出 `targets`。 + +**已实现** + +## 4. 宿主图的平台 + +### 4.1 选择 + +宿主图的平台由第一个给出答案的来源决定:`--host-platform` / `MCPP_HOST_PLATFORM`;工程的 `[host] platform`; +`config.toml` 的 `[host] platform`;否则宿主约定。选择只在最外层的准备中进行。**已实现** + +### 4.2 模型 + +宿主图选用平台 P 时,构建程序与其库编译为 `<本机架构>-

` 的产物(P 的 `targets` 中与本机架构 +相同的那个三元组),由 P 的运行器运行;构建程序的身份含 P 的身份。报告多一行 +`Host platform (<来源>): build programs are images run by `。**已实现** + +### 4.3 自举边界 + +一个平台的 `tools` 与运行器所需的程序,在任何图中都**必须**按宿主约定构建:运行器不能由它自己运行。 +嵌套的准备(工具的子构建、宿主目标的子构建、构建程序图)不选择平台,除非调用者为它指定。因此不形成 +循环:P 的工具属于原生宿主图,构建程序属于 P 的宿主图。**已实现** + +## 5. `cfg(platform = ...)` + +`cfg()` 谓词增加键 `platform`。它匹配所选平台的名字(平台包短名去掉结尾的 `-platform`,`openkal-platform` +的名字是 `openkal`)或平台包的完整短名。其具体程度位于三元组之下、`os` 之上(SPEC-004 §3.1.1)。无平台时, +`cfg(platform = ...)` 不匹配。**已实现** + +## 6. 合并 + +一个图的平台陈述按"平台 → 平台插件与构建程序 → `config.toml` → 工程 → 环境变量"的顺序合并;每一层只改 +它写出的字段: + +| 字段 | 规则 | +|---|---| +| `build` 的四个列表、`link.flags` | 拼接,平台的在前;使用者只能追加,不能删除平台的约定 | +| `build` 的 `cfg(...)` 子表 | 追加 | +| `link.inputs`、`link.seal` | 整体替换 | +| `layers`、`tools`、`runners`、`versions` | 按键替换 | +| `runner` | `cmd` 替换;`host`、`exclusive` 各自独立替换,只写 argv 的覆盖保留平台的 `host` 与 `exclusive` | +| 其余 | 按字段替换 | + +`MCPP_TARGET__RUNNER` 以空白分隔的 argv 替换 `runner.cmd`;值为 `none` 时清除运行器。**已实现** + +## 7. 依赖图的补全 + +### 7.1 平台包是根的依赖 + +被选中的平台包作为根的第一个依赖进入依赖图,按普通包解析、获取、缓存与锁定;其库目标可被 `import`。 +无 `plugin` 且无源文件的平台包不编译。被选中的包没有 `[platform]` 段时拒绝(`platform-not-a-platform`)。 +**已实现** + +### 7.2 层 + +第一轮遍历之后,对 `layers` 中的每一层: + +1. 值为 `false`:平台不为该层提供包;报告为 `none`。 +2. 图中已有包以 `provides` 提供该层:平台让位;该包由平台加入时报告为 `platform`,否则为 `graph`。 + 判断读 `provides`,不比较包名。 +3. 否则:该包作为根的依赖加入(来源 `the platform's `),一个包提供的多层只加入一次。 + +加入的包再次遍历;补全最多进行三轮。一个包提供多个层时,关闭其中一层不移除该包:例如 `c++-abi = false` +而 `compiler-runtime` 仍由同一包提供时,该包的 `provides` 使 `c++-abi` 由图提供。**已实现** + +### 7.3 工具 + +`tools` 中的每个包引用作为根的构建依赖加入,带 `tools = []`,为本机构建;同一包的多个工具只构建一次。 +根已经以构建依赖请求同名工具时让位;`{ program = "" }` 不构建,在声明的载荷与 `PATH` 中查找。 +**已实现** + +### 7.4 工作空间成员 + +工作空间的计划中,平台加入的层的包同时成为每个被选中成员的依赖边,使成员的链接闭包含有这些层。**已实现** + +### 7.5 版本组 + +`versions` 中的包以不同于组中版本的版本出现在图中时拒绝(`platform-version-group`),消息写出该包、两个版本、 +请求链与两条出路:选择携带所需版本的平台版本,或在请求处把该包写成组中的版本。只为本机构建的包(工具) +与以 `path`、`git` 取得的包不受版本组约束。检查在每一轮补全之前与最后一轮之后进行,使工程固定的一个版本以 +版本组的冲突报告,而不是以平台加入的包遇到第二个版本报告。**已实现** + +### 7.6 多版本改名的边界 + +两个版本的同一个包以改名并存只改 C++ 模块名。导出头文件目录的包与以 `provides` 提供 `mcpp:` 层的包 +**禁止**以改名并存:两个版本同时被请求时拒绝,消息列出两个请求者,有平台时另给出换平台版本的出路。 +**已实现** + +## 8. 工具链 + +平台的 `toolchain` 位于目标表钉住的工具链的位置:它替换默认值,**禁止**替换工程或本机为该目标写出的工具链 +(`[target.] toolchain`、`[toolchain]`、`--toolchain`)。图中提供 `kernel-abi` 的包不取消平台的 +工具链。工具链报告行说明它是平台的,以及它替换了哪个默认值与覆盖它的键。**已实现** + +平台写 `sysroot` 时,目标基在依赖图存在之前决定;平台包不在本地时,第一遍准备获取它,第二遍以它的 +`sysroot` 重新决定目标基。**已实现** + +## 9. 编译、链接、命名与封装 + +### 9.1 编译 + +平台陈述的 `build` 标志加在目标图每个翻译单元的命令行前部,`cfg(...)` 子表按目标求值后追加。**已实现** + +### 9.2 链接 + +| 键 | 语义 | +|---|---| +| `driver` | `"compiler"`(编译器驱动链接,缺省)或 `"lld"`(`ld.lld` 直接链接) | +| `flags` | `driver = "lld"` 时给 `ld.lld` 的参数,位于其他链接参数之前 | +| `start` | 入口符号(SPEC-004 §14);入口未定义是链接错误 | +| `script` | 平台包中的链接脚本,以 `-T` 给出 | +| `inputs` | `{ file = "<包内路径>" }`,或 `{ from = [argv], as = "<文件名>" }`:运行程序,把其标准输出保存为 `as`;以 `.S`、`.s` 结尾时先汇编,否则原样参与链接 | +| `exe` | 可执行产物的名字模式,**必须**以 `{name}` 开头 | +| `shared` | 共享产物的名字模式,`{name}` 恰好一次,不含目录 | +| `seal` | 封装步骤,每步一个 argv(§9.3) | + +`driver = "lld"` 而工具链载荷中没有 `ld.lld` 时拒绝(`lld-required-absent`),消息写出平台的工具链。程序名为 +`lld` 时引擎加 `-flavor gnu`。链接或封装命名的程序既非图中为本机构建、也不在本机可找到时拒绝 +(`platform-tool-absent`),消息写出负责构建它的包,或写出 `[target.'cfg(platform = "")'.tools]` 的写法。 +**已实现** + +### 9.3 封装 + +`seal` 的步骤在链接器写出的临时产物上依次执行,`{}` 为该临时产物;参数 `{needed}` 展开为该产物所需的共享产物, +每个一个参数,当它前面的参数是取值的选项(以 `-` 开头、不含 `=` 与 `{}`)时,该选项在每个产物前重复, +不需要任何共享产物时二者都省略。全部步骤成功后产物原子发布(SPEC-011 §3);任一步失败则不发布,保留上一次 +发布的产物,错误给出该步的 argv 与输出。步骤属于链接边的命令行,改变步骤即重新链接。**已实现** + +## 10. 运行器 + +`runner` 的数组形式为 argv;表形式: + +| 键 | 语义 | +|---|---| +| `cmd` | argv,非空;`{}` 为产物 | +| `host` | `"same-arch"`:产物只在其架构的机器上运行;`"any"`:运行器可以从任何机器到达产物(模拟器、探针) | +| `exclusive` | 为真时该目标的运行不重叠,`mcpp test` 串行运行 | + +`host = "same-arch"` 而产物的架构不是本机架构时,`mcpp run` 与 `mcpp test` 拒绝运行,消息给出 +`[target.] run_under` 的写法;`run_under` 存在或给出 `--no-runner` 时不拒绝。运行器的合并顺序与 §6 +相同;`mcpp run` 与 `mcpp test` 读合并后的值。**已实现** + +## 11. 载体(M3) + +`mcpp pack --mode self-contained` 对一个依赖加载器的产物生成一个本机程序工程:依赖 `carrier.package`;入口是 +平台包中的 `carrier.main`;一个汇编文件以 `.incbin` 把产物的字节放进节 `carrier.section`(ELF 写作该名, +Mach-O 写作 `__<大写名>,__image`,COFF 写作该名并带 `"dr"` 标志),给出符号 `mcpp_carried_begin`、 +`mcpp_carried_end`、`mcpp_carried_name`(产物的名字)。该程序按宿主约定构建(自举边界),不使用平台:Linux 上 +为 `<本机架构>-linux-musl` 行的静态程序,该行的工具链被显式写出;macOS 与 Windows 上只依赖系统库。 +引擎不含任何加载器的调用。平台未声明 `carrier` 时,`--mode self-contained` 被拒绝并写出该键。**已实现** + +## 12. 平台阶段与协议 17 + +### 12.1 平台阶段 + +平台包的 `plugin` 在**平台阶段**运行:`MCPP_PHASE` 为 `platform`,`mcpp::phase()` 返回 `"platform"`。该阶段只能 +陈述编译与链接标志、`link_script`、`link_start` 与协议 17 的指令、`runner` 与命名运行器、`run_exclusive`,以及 +重跑条件、警告与诊断;其他指令被拒绝(`platform-phase-directive`),消息点名第一条不属于该阶段的指令。 +陈述工具链在平台阶段被拒绝。**已实现** + +### 12.2 协议 17 的指令 + +| `mcpp.core` 函数 | 指令 | 字段 | +|---|---|---| +| `link_driver(d)` | `mcpp:link-driver=` | `link.driver` | +| `link_input(path)` | `mcpp:link-input=file:` | `link.inputs` 的 `file` | +| `link_input_from(as, token)` | `mcpp:link-input=from::` | `link.inputs` 的 `from` 与 `as`,同一 `as` 的 token 依次组成 argv | +| `artifact_name(kind, pattern)` | `mcpp:artifact-name=:` | `link.exe`、`link.shared` | +| `seal(step, token)` | `mcpp:seal=:` | `link.seal`,按步号分组 | +| `run_host(h)` | `mcpp:run-host=` | `runner.host` | + +协议 17 的指令也可以由任何构建程序在构建阶段发出(板级支持包的写法),在 Program 层次合并。格式错误的 +指令被拒绝,消息写出合法形式。**已实现** + +## 13. 工作空间 + +`[target.]` 的平台键与 `[host]` 是根位置的键,从计划的根读取。工作空间层的 +`[workspace.target.]` 先于成员自己的表合并;`[host]` 不是工作空间层的表。虚拟根携带第一个被选中 +成员的平台键与 `[host]`;根位置的比较键包含适用的平台表与 `[host]`,使二者不同的成员不共用一个计划。 +**已实现** + +## 14. 内置平台与目标表 + +宿主目标(Linux、macOS、Windows、Android、iOS、Emscripten)的约定留在引擎中,无平台包。目标表有平台列: +`x86_64-openkal`、`aarch64-openkal` 的值为 `mcpplibs.openkal-platform@`,即这两个目标的内置支持; +选择来源报告为 `built-in`。项目可以不等引擎更新,以 `[target.] platform` 选择另一个版本。 +**已实现** + +## 15. 锁、身份与报告 + +### 15.1 锁 + +平台加入的包在 `mcpp.lock` 中带 `origin = "platform"`;无 `origin` 的条目视为工程声明。引擎升级不替换已锁定的 +版本。**已实现** + +### 15.2 身份 + +平台包与它加入的包是依赖图的成员,进入配置指纹的依赖部分;换平台或平台版本即换构建目录。平台的身份为 +`@`,`path`、`git` 来源另带来源;宿主图平台的身份进入构建程序的身份。**已实现** + +### 15.3 报告 + +目标报告中,平台加入的层的来源写作 `platform`,图已提供的写作 `graph`。层之后为平台块: + +``` + Platform openkal-platform 0.17.1 (built-in) + tools okld, okrun (openkal-tools@0.1.1, platform) + runner okrun (platform) + link lld, start ok_main (platform) +``` + +来源取值:`built-in`、`platform`、`build-program`、`config.toml:<行>`、`mcpp.toml:<行>`、环境变量名、 +`--platform`、`graph`。宿主三元组选用平台时来源后加 `, hosted`。**已实现** + +### 15.4 机器输出 + +`--message-format json` 的 `plan` 事件与 `result.json` 的 `configuration` 含 `platform` 与 `hostPlatform`,无平台时为 +`null`;对象形式: + +```json +{"name": "mcpplibs.openkal-platform", "version": "0.17.1", "source": "built-in", "hosted": false, + "fields": {"toolchain": {"value": "llvm@23.1.3", "source": "platform"}, + "layers.c-abi": {"value": "openkal-musl@0.22.3", "source": "platform"}}} +``` + +`fields` 的键为 `toolchain`、`runner`、`runners.`、`layers.`、`tools.`(值为所用程序的路径)、 +`link.driver`、`link.start`、`link.exe`、`link.shared`、`carrier`。**已实现** + +## 16. 命令 + +| 命令 | 规定 | +|---|---| +| `--platform <值>` | `build`、`run`、`test`、`why`、`pack`、`emit build-database`、`self env`、`toolchain install`;即 `MCPP_PLATFORM` | +| `--host-platform <值>` | `build`、`run`、`test`、`why`、`pack`、`emit build-database`、`self env`;即 `MCPP_HOST_PLATFORM` | +| `mcpp self env --target [--format json]` | 准备并报告一个配置:三元组、编译器与其来源、平台、工具、运行器与其程序、`run_under`;在工程中为该工程的配置,工程外为家目录缓存中的空程序;工程源码会被拒绝的原因(例:无 C++ 运行时而 `import std`)作为警告报告,配置照常报告 | +| `mcpp self env --host` | 宿主图的配置 | +| `mcpp toolchain install --target `(不指定编译器) | 安装该目标的工具链后准备该配置:获取平台包,构建其工具 | +| `mcpp toolchain list` | 目标行有 PLATFORM 列,JSON 有 `platform` 字段 | +| `mcpp test`(在平台包中,未给 `--target`) | 对 `targets` 中的每个目标以本包为平台构建测试;`host = "same-arch"` 时只运行本机架构的目标,其余报告为已构建未运行 | + +**已实现** + +## 17. 拒绝码 + +机器可读的拒绝码出现在结果记录的 `reason` 中(docs/50)。 + +| 码 | 条件 | +|---|---| +| `platform-not-a-platform` | 被选中的包没有 `[platform]` | +| `platform-target-unserved` | 平台不服务该目标(§3.4) | +| `platform-tool-absent` | 链接或封装命名的程序既非图中构建、也未找到(§9.2) | +| `platform-version-group` | 图中的包与版本组冲突(§7.5) | +| `platform-phase-directive` | 平台阶段陈述了不属于该阶段的指令(§12.1) | +| `lld-required-absent` | `driver = "lld"` 而工具链无 `ld.lld` | + +`platform-tool-absent` 取代 2026.10.26.2 的 `image-tool-absent`。**已实现** + +## 18. 判据 + +1. `[platform]` 的每个键的类型错误在读清单时拒绝,消息写出键与文件;未知键为警告。 +2. 选择按 §3.1、§4.1 的顺序;`none` 不使用平台;目录形式读取目录中的清单。 +3. `cfg(platform = ...)` 下写 `platform` 被拒绝。 +4. 图已提供某层时平台不加入该层的包,报告为 `graph`;`false` 不加入。 +5. 版本组冲突在任何链接之前拒绝,消息给出两条出路。 +6. 封装失败不发布产物,保留上一次的产物。 +7. `same-arch` 的运行器不运行其他架构的产物,并给出 `run_under` 的写法。 +8. 平台的工具在任何图中按宿主约定构建。 +9. 锁中平台加入的包带 `origin = "platform"`;报告与 JSON 给出每个字段的来源。 + +## 变更记录 + +| 版本 | 日期 | 变更 | +|---|---|---| +| 1.0 | 2026-10-28 | 首版(mcpp 2026.10.27.1,mcpp#847):`[platform]` 的七组键与 `versions`、`hosted`;三种写法与层次;目标图与宿主图的选择;`cfg(platform)`;合并;依赖图的补全、版本组与改名边界;工具链;链接、命名与封装;运行器的表形式;载体;平台阶段与协议 17;工作空间;目标表的平台列;锁、报告、机器输出、命令与拒绝码。 | diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md new file mode 100644 index 000000000..cc1d768b1 --- /dev/null +++ b/docs/zh/35-platforms.md @@ -0,0 +1,234 @@ +# 35 —— 平台 + +**读者:** 为一个由平台包配置的目标构建的人(ok 映像,`-openkal`)、要改动该配置一部分的人, +以及为一类目标编写平台包的人。 + +**本章回答的那一个问题:** 一类目标怎样作为一个包来配置,构建怎样选择它,工程怎样覆盖或替换它。 + +**不在这里:** 平台所命名的目标侧各层,见 [22 —— 目标侧](22-target-side.md);openkal 的模型与宿主上的 +交叉构建,见 [24 —— 基于 openkal 的交叉构建](24-openkal-cross.md);打包一个程序,见 +[10 —— 发布打包](10-pack-and-release.md);精确规则,见 [SPEC-013](../specs/platforms.md)。 + +在此之前:[34 —— 编写板级支持包](34-authoring-a-bsp.md)。 +在此之后:[40 —— 裸机与 freestanding 目标](40-baremetal.md)。 + +**平台**是一个带版本的包,陈述一类目标所需的全部内容:工具链、目标侧各层的提供包、构建在本机运行的 +工具、编译与链接约定、产物的命名与封装,以及运行器。工程一项都不必写。目标表为 `x86_64-openkal` 与 +`aarch64-openkal` 选择 `mcpplibs.openkal-platform`(2026.10.27.1+),这两个目标的内置支持即是如此。 + +## 最短可跑 + +一个只写出源码的工程: + +```toml +[package] +name = "app" +version = "0.1.0" +``` + +```sh +mcpp run --target x86_64-openkal +``` + +``` + Resolved llvm@23.1.3 → x86_64-openkal → …/xim-x-llvm/23.1.3/bin/clang++ + the platform's (openkal-platform 0.17.1), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` + Target x86_64-openkal → x86_64-unknown-none-elf + compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.3, platform) + c-abi musl (openkal-musl@0.22.3, platform) + c++-abi libc++ (openkal-llvm-runtime@0.16.3, platform) + Platform openkal-platform 0.17.1 (built-in) + tools okld, okrun (openkal-tools@0.1.1, platform) + runner okrun (platform) + link lld, start ok_main (platform) + Compiling app v0.1.0 (.) + Running `okrun … target/x86_64-openkal/…/bin/app.oke` +``` + +每个值都带来源报告。`mcpp test`、`mcpp pack` 与 `--target aarch64-openkal` 无需更多声明。`mcpp.lock` 以 +`origin = "platform"` 记录平台加入的包。 + +## 平台陈述的内容 + +| 组 | 键 | 作用 | +|---|---|---| +| 目标 | `targets`、`toolchain`、`sysroot` | 服务哪些三元组、它们使用的工具链、预构建的目标基 | +| 层 | `[platform.layers]` | `c-abi`、`c++-abi`、`compiler-runtime`、`kernel-abi` 的提供包,或 `false` | +| 工具 | `[platform.tools]` | 为本机构建的程序,或查找到的程序(`{ program = "" }`) | +| 编译 | `[platform.build]`、`[platform.build.'cfg(…)']` | 目标每个翻译单元的标志 | +| 链接 | `[platform.link]` | `driver`、`flags`、`start`、`script`、`inputs`、名字 `exe` 与 `shared`、封装步骤 `seal` | +| 运行 | `runner`、`runners` | `mcpp run` 与 `mcpp test` 怎样启动产物,以及具名的设备动作 | +| 载体 | `carrier` | `mcpp pack --mode self-contained` 为依赖加载器的产物构建的程序 | +| 代码 | `plugin` | 在平台阶段运行的构建程序,陈述随依赖图而变的部分 | +| 版本组 | `[platform.versions]` | 一个家族中每个包的一个版本 | +| 宿主部分 | `[platform.hosted]` | 宿主三元组选用该平台时适用的部分 | + +## 覆盖平台的一部分 + +工程在 `[target.]` 下以平台自己的键名覆盖一个字段。`cfg(platform = "openkal")` 不论架构都匹配该平台: + +```toml +[target.'cfg(platform = "openkal")'.layers] +c-abi = { path = "../openkal-musl" } # a working tree in place of the release + +[target.'cfg(platform = "openkal")'.tools] +okld = { path = "../openkal-image/tools" } + +[target.'cfg(platform = "openkal")'] +runner = ["my-okrun", "{}"] +``` + +| 字段 | 规则 | +|---|---| +| 编译标志、`link.flags` | 追加在平台的之后;约定不能删除 | +| `link.inputs`、`link.seal` | 整体替换 | +| `layers`、`tools`、`runners` | 按键替换;` = false` 不提供该层 | +| `runner` | argv 替换;未写出时保留 `host` 与 `exclusive` | +| 其余字段 | 替换 | + +同样的表也可以写在 `~/.mcpp/config.toml` 中,对一台机器生效。图中已有包提供的层交给该包,报告为 `graph`。 +C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只有与 `compiler-runtime = false` 同时写出才生效。 + +## 选择平台,或不用平台 + +第一个给出答案的来源决定目标的平台: + +| 来源 | 写法 | +|---|---| +| 命令行 | `--platform openkal-platform@0.17.1`、`--platform path:../openkal-platform`、`--platform none` | +| 环境变量 | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | +| 工程 | `[target.] platform = "openkal-platform@0.17.1"`,或 `{ path = "…" }` | +| 本机 | `~/.mcpp/config.toml` 中的同一个键 | +| 目标表 | 平台列(`mcpp toolchain list`) | + +`platform` 不能写在 `cfg(platform = …)` 选择器下。`none` 只以引擎对该目标的约定构建。 + +## 宿主图的平台 + +构建程序与构建运行的工具按本机的约定为本机编译。`[host] platform` 为它们选择一个平台: + +```toml +[host] +platform = "openkal-platform@0.17.1" +``` + +``` + Host platform openkal-platform 0.17.1 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun +``` + +此后每个构建程序都编译为本机架构的 ok 映像,在 Linux、macOS 与 Windows 上使用同一个工具链,经平台的运行器 +运行。平台自己的工具仍以原生方式构建。`--host-platform` 与 `MCPP_HOST_PLATFORM` 对一次运行选择它, +`~/.mcpp/config.toml` 中的 `[host]` 对一台机器选择它。 + +## 宿主上的安排 + +宿主三元组选用一个平台时,只适用平台的 `[platform.hosted]` 部分:它的工具链与它的层。对 openkal-platform 而言, +即家族的 C 库与 C++ 运行时,位于系统自己的 openkal 实现之上: + +```toml +[target.'cfg(any(linux, macos, windows))'] +platform = "openkal-platform@0.17.1" +``` + +Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[platform.hosted] targets` 之外的 +三元组被拒绝,消息给出它所服务的同系统同架构的行。 + +## 版本组 + +`[platform.versions]` 为一个包与包之间精确钉住的家族列出每个包的一个版本。依赖图中出现其中某个包的 +另一个版本时,在任何编译之前拒绝: + +``` +error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.1 is a version group that names openkal-musl 0.22.3. + requested by: own + The family's packages pin one another exactly, so two versions of one of + them cannot be linked into one program. Either select the platform release + of that version ([target.] platform = "openkal-platform@"), or write + openkal-musl = "0.22.3" where it is requested. +``` + +从目录或仓库取得的包不受版本组约束。 + +## 编写平台包 + +平台包是带 `[platform]` 段的包。openkal-platform 的节选: + +```toml +[package] +namespace = "mcpplibs" +name = "openkal-platform" +version = "0.17.1" +mcpp = ">=2026.10.27.1" + +[platform] +targets = ["x86_64-openkal", "aarch64-openkal"] +toolchain = "llvm@23.1.3" +runner = { cmd = ["okrun", "{}"], host = "same-arch" } +carrier = { package = "openkal-run@0.1.1", section = ".oke", main = "carrier/main.cpp" } + +[platform.layers] +compiler-runtime = "openkal-llvm-runtime@0.16.3" +"c++-abi" = "openkal-llvm-runtime@0.16.3" +c-abi = { name = "openkal-musl", version = "0.22.3", default-features = false, features = ["posix"] } + +[platform.tools] +okld = "openkal-tools@0.1.1" +okrun = "openkal-tools@0.1.1" + +[platform.build] +cxxflags = ["-fPIC", "-femulated-tls", "-fvisibility=hidden", "-fasynchronous-unwind-tables"] + +[platform.build.'cfg(arch = "aarch64")'] +cxxflags = ["-ffixed-x18", "-mno-outline-atomics"] + +[platform.link] +driver = "lld" +start = "ok_main" +flags = ["-shared", "-z", "max-page-size=65536", "-z", "now", "--gc-sections", "--eh-frame-hdr"] +inputs = [{ from = ["okld", "note-source"], as = "ok-notes.S" }] +exe = "{name}.oke" +shared = "lib{name}.okso" +seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] +``` + +- **程序写名字,不写位置。** `inputs` 与 `seal` 中的 `okld` 依次在图为本机构建的程序、声明的载荷、`PATH` 中查找。 +- **`{}` 是产物。** 封装步骤中的 `{needed}` 展开为产物所需的共享映像;它前面的 `--with` 在每个映像前重复。 +- **封装步骤失败时不发布。** 上一次成功构建发布的产物保留原位。 +- **随依赖图而变的部分写成代码。** `plugin = "build.mcpp"` 在平台阶段运行(`mcpp::phase()` 为 `"platform"`), + 可以陈述标志、链接字段与运行器(`mcpp::link_driver`、`mcpp::link_input_from`、`mcpp::artifact_name`、 + `mcpp::seal`、`mcpp::run_host`);其他指令被拒绝。板级支持包的构建程序在构建阶段陈述同样的字段 + ([34](34-authoring-a-bsp.md))。 + +开发期间,工程以 `--platform path:

` 或 `platform = { path = "" }` 选择工作树。在平台包中执行 +`mcpp test` 而不给 `--target` 时,以该包为平台为其每个 `targets` 构建测试,运行本机架构的那些,其余报告为 +已构建、未运行。 + +## 查看一个配置 + +```sh +mcpp self env --target x86_64-openkal # toolchain, platform, tools, runner +mcpp self env --target x86_64-openkal --format json +mcpp self env --host # the host graph +mcpp toolchain list # the PLATFORM column +mcpp toolchain install --target x86_64-openkal # fetch and build what the target needs +``` + +`--message-format json` 在 `plan` 事件与 `result.json` 中给出 `platform` 与 `hostPlatform` +([50](50-machine-output.md))。 + +## 拒绝 + +| 原因 | 条件 | 出路 | +|---|---|---| +| `platform-not-a-platform` | 所选的包没有 `[platform]` | 选择一个平台包,或 `none` | +| `platform-target-unserved` | 平台不服务该三元组 | 为它的某个 `targets` 构建,或为消息给出的宿主行构建 | +| `platform-tool-absent` | 链接或封装要运行的程序既未构建、也未找到 | 在 `[target.'cfg(platform = "…")'.tools]` 下写出构建它的包 | +| `platform-version-group` | 家族中某个包的另一个版本在图中 | 选择携带该版本的平台版本,或钉住版本组中的版本 | +| `platform-phase-directive` | 插件陈述了平台阶段不接受的指令 | 改由包自己的构建程序陈述 | +| `lld-required-absent` | 平台以 `ld.lld` 链接,而工具链没有它 | 使用平台的工具链 | + +## 当前边界 + +- 一个平台的每个字段只有一个值;随架构不同的字段只有标志可以写在 `[platform.build.'cfg(arch = "…")']` 下。 +- 宿主图的平台按一次运行或一个工程选择;缺省为宿主约定。 +- `[host]` 只接受 `platform`。 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index bcd88fbf5..f532fad18 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -91,10 +91,10 @@ mcpp --format json 目前只支持 `json`。`ndjson` 为将来真正需要流式的场景保留，**现在不被接受** —— 请求它是一个错误，而不是静默地回落到别的格式。 -`--format` 已经用来表示某种**产物**的命令，改用 `--message-format json` 来 -请求机器输出，`mcpp test` 就是这样做的。`mcpp pack --format` 表示的是包格式 -（`tar`、`dir`、`msi`），所以它的报告方式是 -`mcpp pack --message-format json`（2026.9.16.1+）。具体形状随 kind 而定： +构建类命令改用 `--message-format json` 来请求机器输出，`mcpp test` 就是这样做的： +`build`、`test`、`run` 与 `pack`。`mcpp pack` 与 `mcpp run` 的包格式是 `--pkg-format` +（`tar`、`dir`、`msi`；2026.10.27.1+；在这两个命令上 `--format` 是旧写法，会给出警告），所以打包的 +报告方式是 `mcpp pack --message-format json`（2026.9.16.1+）。具体形状随 kind 而定： `mcpp test` 的测试是随时间陆续完成的，所以它按测试逐条输出流式记录；一次 `mcpp pack` 只有一个结果，所以它输出一个信封。`mcpp build --message-format json` （2026.10.26.1+）在构建过程中输出事件流，以一个信封结束（§9）。 @@ -219,8 +219,8 @@ mcpp cache list --json -> {"root": …, "entries": [ … ]} |---|---| | `0`–`124` | 程序运行过了；这是它自己的状态，原样透传。例外是 `101`，见下一行 | | `101` | `mcpp run` 无法构建程序：规划或构建失败，没有启动任何东西。这是 `cargo run` 构建失败时 Cargo 给出的状态 | -| `125`–`127` | 尝试过启动但被拒绝 —— `127` 是找不到，`126` 是找到了但不可执行，`125` 是其他原因；对 `mcpp run --format ` 而言，一个是目录、且没有任何 runner 能到达的分发物同样得到 `126`，它在启动之前就被拒绝，含义相同（2026.9.14.2+） | -| `2` | mcpp 在构建任何东西之前拒绝了请求，或在构建之后、启动任何东西之前拒绝了请求：用法错误、`--format` 与 `--no-runner` 同用、项目没有声明的程序或 runner | +| `125`–`127` | 尝试过启动但被拒绝 —— `127` 是找不到，`126` 是找到了但不可执行，`125` 是其他原因；对 `mcpp run --pkg-format ` 而言，一个是目录、且没有任何 runner 能到达的分发物同样得到 `126`，它在启动之前就被拒绝，含义相同（2026.9.14.2+） | +| `2` | mcpp 在构建任何东西之前拒绝了请求，或在构建之后、启动任何东西之前拒绝了请求：用法错误、`--pkg-format` 与 `--no-runner` 同用、项目没有声明的程序或 runner | 在 2026.9.30.2 之前，构建失败退出 `1`，规划失败退出 `2`，脚本因此无法区分 一次编译错误和一个返回 `1` 的程序。现在构建失败是 `101`；`mcpp build`、 @@ -298,6 +298,28 @@ mcpp self env --format json 解析逻辑 —— 包括「PATH 上的 `mcpp` 可能是一个 xlings shim 而不是真正的 二进制」这一部分。 +**一个配置** *（2026.10.27.1+）*。带 `--target `（宿主图用 `--host`）时，`self env` +准备该配置而不构建，并报告它；在工程中是该工程的配置，工程之外是 home 缓存中一个空程序的配置。 +`--platform` 与 `--host-platform` 的选择与构建相同（[35](35-platforms.md)）。 + +```jsonc +{ + "triple": "x86_64-openkal", + "graph": "target", // or "host" + "toolchain": {"compiler": "/…/xim-x-llvm/23.1.3/bin/clang++", "origin": "platform"}, + "platform": {"name": "mcpplibs.openkal-platform", "version": "0.17.1", "source": "built-in", + "hosted": false, "fields": {"runner": {"value": ["okrun", "{}"], "source": "platform"}}}, + "tools": {"okld": "/…/okld", "okrun": "/…/okrun"}, + "runner": {"argv": ["okrun", "{}"], "host": "same-arch", "exclusive": false, + "program": "/…/okrun", "source": "platform"}, + "runUnder": [] +} +``` + +未选择平台时 `platform` 为 `null`；其形状与 `plan` 事件的相同（§9）。工程源码会被拒绝的原因 +（图中没有 C++ 运行时而 `import std`）是一条代码为 `MCPP_CONFIGURATION_SOURCES_REFUSED` 的 +`warning` 诊断，配置照常报告。无法准备的配置带 `MCPP_CONFIGURATION_FAILED`，没有 `data`。 + `defaultToolchain` 是在这台宿主上、什么都没有配置时一次构建所解析的工具链：首次运行 安装的正是它，各 CI 宿主也以它核对 docs/01 与 docs/20 的表格。在 Windows 上，它取决于 是否存在可用的 MSVC（来自 Visual Studio 或受管的工具集）。 @@ -408,6 +430,8 @@ mcpp toolchain list --format json `source: "system"` —— 它是在机器上被定位到的，不是由 mcpp 安装的。一行目标 的形状是 `{target, note, toolchain, pin, status, default}`，`status` 取 `installed` / `available` / `via dependency graph` / `planned` 之一。 +目标行另有 `platform`（2026.10.27.1+）：目标表为该目标选择的平台包 +（`mcpplibs.openkal-platform@0.17.1`），或 `null`。 **`toolchain` 与 `pin` 不是把同一个字段写了两遍。** `toolchain` 表示这一行 关联到什么 —— 已装的行是装好的载荷，词表行是它自己的约定。`pin` 只承载目标 @@ -516,7 +540,11 @@ replaced}` —— `origin` 与构建的状态行使用的是同一句话 | `std-module-precompile` | 标准库的模块在这个配置下无法被预编译 | | `msvc-redist-unavailable` | 在 MSVC ABI 的行上显式写了 `cxx_runtime = "toolchain-coupled"`，而该行的工具集没有可放置的 redistributable 目录 *(2026.9.28.1+)* | | `crt-declared-under-host-coupled` | 程序的契约是 host-coupled（由系统的运行时服务、不放置任何副本），而程序旁声明了 MSVC C++ 运行时的文件 *(2026.9.28.2+)* | -| `image-tool-absent` | 一个 `-openkal` 的构建要链接一个映像，而找不到 `okld` —— 为 ok 映像写需求注记并检查约定的、图中的程序 *(2026.10.26.2+)* | +| `platform-not-a-platform` | 被选为该目标平台的包没有 `[platform]` 段 *(2026.10.27.1+)* | +| `platform-target-unserved` | 所选平台不服务该目标：三元组不在其 `targets` 中，宿主三元组也不在其 `[platform.hosted] targets` 中 *(2026.10.27.1+)* | +| `platform-tool-absent` | 平台的链接或封装要运行的程序（ok 映像的 `okld`）既不由图构建，也不在本机 *(2026.10.27.1+；2026.10.26.2 中为 `image-tool-absent`)* | +| `platform-version-group` | 平台版本组中的一个包以另一个版本出现在图中 *(2026.10.27.1+)* | +| `platform-phase-directive` | 平台插件陈述了其阶段不接受的指令 *(2026.10.27.1+)* | | `other` | 一个尚未被赋予记号的拒绝分支 | **其中一个记号也由 `mcpp build` 自己打印。** `interface-not-provided` 会 @@ -605,7 +633,7 @@ mcpp emit build-database [--spec s1|compile-commands] --format json ### `mcpp.pack` —— 一次打包的产物 *(mcpp 2026.9.16.1+)* ``` -mcpp pack [target] [--format ] [--target ...] --message-format json +mcpp pack [target] [--pkg-format ] [--target ...] --message-format json ``` 信封在命令结束后输出一次；所有给人看的行都改走 stderr，包括打包过程启动的 @@ -613,8 +641,10 @@ mcpp pack [target] [--format ] [--target ...] --message-format json | 字段 | | |---|---| -| `artifacts` | 每个产出的产物一条记录：`path`（绝对路径）、`type`（`file` 或 `directory`）、`format`（`--format` 的取值，省略时为 `tar`），以及 `targets`（进入该产物的每条腿的规范三元组）。被分派的格式报告本次请求引入的那些 action 的终端输出；一个多 `--target` 的 Android 打包报告一个产物，其 `targets` 列出每一条腿 | +| `artifacts` | 每个产出的产物一条记录：`path`（绝对路径）、`type`（`file` 或 `directory`）、`format`（`--pkg-format` 的取值，省略时为 `tar`）、`targets`（进入该产物的每条腿的规范三元组），以及 `sha256`（文件的摘要；目录为 `null`；2026.10.27.1+）。被分派的格式报告本次请求引入的那些 action 的终端输出；一个多 `--target` 的 Android 打包报告一个产物，其 `targets` 列出每一条腿 | | `stage` | 该产物所来自的那棵树：`dir`、`manifest`（即下文的暂存清单）与 `closure`（`walked` 或 `not-walked`）；库包、没有暂存任何树的情形，以及对多个成员的打包，此字段为 `null` | +| `linkage`、`mode`、`pkgFormat` *（2026.10.27.1+）* | 本次打包的三个选择（[10](10-pack-and-release.md)） | +| `legs`、`systems` *（2026.10.27.1+）* | 每条腿的规范三元组，以及自带加载器的通用包所携带的加载器所属的系统 | | `stages` *（2026.10.1.1+）* | 只在对多个成员的打包（`--workspace`，或重复的 `-p`）中出现：每个成员一条记录，按 `[workspace] members` 的顺序，带 `member`（限定的包名），以及与 `stage` 相同的 `dir`、`manifest`、`closure` | 对多个成员的打包，`artifacts` 按同样的成员顺序列出每个成员的产物，每条记录多出 @@ -757,7 +787,7 @@ mcpp build [target] [-p ] [--release] --message-format json | 事件 | 携带 | |---|---| -| `plan` | `configuration`：构建已进入其构建目录的那个配置 | +| `plan` | `configuration`：构建已进入其构建目录的那个配置；`platform` 与 `hostPlatform`（2026.10.27.1+）是目标图与宿主图的平台，或 `null` | | `waiting` | `holder`（`pid`、`command`、`client`、`startedAt`）与 `waitedMs`：另一次构建持有这个构建目录，本次在等待（§11） | | `progress` | `finished`、`total`：ninja 的步骤计数；一次构建可以运行不止一轮 ninja，每轮重新计数 | | `diagnostic` | `diagnostic`：失败步骤输出中的一条编译器诊断，形状同下文记录的 `diagnostics[]` | @@ -783,7 +813,7 @@ mcpp build [target] [-p ] [--release] --message-format json | `pid` | 写下记录的进程 | | `startedAt`、`finishedAt` | 自纪元起的毫秒数；`running` 期间与 `cancelled` 之后 `finishedAt` 为 `null` | | `status` | `running`、`ok`、`failed` 或 `cancelled` | -| `configuration` | `triple`、`profile`、`fingerprint`、`outputDir` | +| `configuration` | `triple`、`profile`、`fingerprint`、`outputDir`、`platform`、`hostPlatform`（2026.10.27.1+） | | `compileDatabase` | 该配置的 `compile_commands.json`，或 `null` | | `artifacts` | `name`、`kind`（`bin`、`test`、`shared`、`static`）、`path`；可运行的产物另有 `launch`；陈述了程序起点或入口符号时另有 `entry` | | `diagnostics` | 失败构建的编译器诊断：`severity`、`message`、`path`、`range` | @@ -817,7 +847,7 @@ mcpp run [target] --print-launch 按 `mcpp run` 的方式构建程序，然后打印信封而不启动它。`data` 为 `program`、 `argv`、`cwd`、`env` 与 `via`，同上文的 `launch`。`--print-launch` 不接受 -`--format`：在 `mcpp run` 上，`--format` 表示可分发物的格式。 +`--format`：在 `mcpp run` 上，`--pkg-format` 表示可分发物的格式。 ## 10. 拒绝副作用 —— `--deny` *（mcpp 2026.10.26.1+）* @@ -879,7 +909,7 @@ mcpp index list --format json `data` 为 `registries`（`name`、`url`、`default`）与 `indices`（在工程中运行时， 工程的 `[indices]` 条目）。包的目录由 `mcpp search` 回答。 -### `mcpp.pack.formats` —— `mcpp pack --format` 接受的格式 +### `mcpp.pack.formats` —— `mcpp pack --pkg-format` 接受的格式 ``` mcpp pack --list-formats diff --git a/docs/zh/README.md b/docs/zh/README.md index 5a300e6aa..6ebd79aa2 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -25,11 +25,11 @@ | 写一个程序 | [01](01-getting-started.md)、[04](04-mcpp-toml.md) §1 | [`01-hello`](../../examples/01-hello/)、[`02-with-deps`](../../examples/02-with-deps/) | | 写一个供他人 import 的库 | [11](11-publishing-a-library.md)、[06](06-features-and-capabilities.md)、[04](04-mcpp-toml.md) §2.4 | [`04-workspace`](../../examples/04-workspace/)、[`11-features`](../../examples/11-features/) | | 发布它 | [10](10-pack-and-release.md)、[11](11-publishing-a-library.md)、[12](12-binary-distribution.md) | [`03-pack-static`](../../examples/03-pack-static/)、[`05-lib-distribution`](../../examples/05-lib-distribution/) | -| 为另一台机器构建 | [21](21-the-target-triple.md)、[24](24-openkal-cross.md)、[40](40-baremetal.md) | [`06-openkal-cross`](../../examples/06-openkal-cross/)、`mcpp new … --template riscv-virt-rt` | +| 为另一台机器构建 | [21](21-the-target-triple.md)、[24](24-openkal-cross.md)、[35](35-platforms.md)、[40](40-baremetal.md) | [`06-openkal-cross`](../../examples/06-openkal-cross/)、`mcpp new … --template riscv-virt-rt` | | 使用 GPU 或加速器 | [42](42-heterogeneous-builds.md)、[41](41-devices.md) | [`09-heterogeneous`](../../examples/09-heterogeneous/)，从 [`boundary/`](../../examples/09-heterogeneous/boundary/) 开始 | | 增加一条规则、一种语言或一个生成器 | [31](31-authoring-a-rule-package.md)、[30](30-build-mcpp.md) | [`08-build-rules`](../../examples/08-build-rules/)、[`12-a-new-device-language`](../../examples/12-a-new-device-language/) | | 向索引添加一个包 | [11](11-publishing-a-library.md)、[SPEC-001](../specs/package-identity.md) | [09](09-commands-by-scenario.md)——发布相关场景 | -| 为别人打包一个工具、一个驱动或一块板子 | [32](32-authoring-a-payload.md)、[33](33-authoring-an-adapter.md)、[34](34-authoring-a-bsp.md) | `xim-pkgindex` 与 `mcpp-index` 里的描述符 | +| 为别人打包一个工具、一个驱动、一块板子或一个平台 | [32](32-authoring-a-payload.md)、[33](33-authoring-an-adapter.md)、[34](34-authoring-a-bsp.md)、[35](35-platforms.md) | `xim-pkgindex` 与 `mcpp-index` 里的描述符 | | 修改 mcpp 本身 | [90](90-build-from-source.md)、[92](92-release.md)、[51](51-supported-versions.md) | — | 课程也可以以**项目模板**的形式到达：模板由包提供，`mcpp new --template` @@ -87,6 +87,7 @@ - [32 —— 编写一个载荷](32-authoring-a-payload.md)——由 mcpp 安装的工具或预编译库 - [33 —— 编写运行时适配包](33-authoring-an-adapter.md)——够到宿主提供的库 - [34 —— 编写板级支持包](34-authoring-a-bsp.md)——一块板子，以及抵达它的方式 +- [35 —— 平台](35-platforms.md)——一类目标的配置，作为一个包 ### 4x —— 设备与加速器 @@ -127,6 +128,7 @@ | `min_api_level` | [04](04-mcpp-toml.md) | `macos_deployment_target` | [04](04-mcpp-toml.md) | | `[target..targets.] kind` | [22](22-target-side.md) | `config.toml` 中的 `[index.repos.]` | [32](32-authoring-a-payload.md) | | `[test] discover` | [04](04-mcpp-toml.md)、[08](08-testing.md) | `MCPP_RUNTIME_FILES` | [41](41-devices.md) | +| `[platform]`、`[host]`、`cfg(platform = …)` | [35](35-platforms.md) | `[target.] platform`、`layers`、`tools`、`link`、`carrier` | [35](35-platforms.md) | **命令** @@ -136,6 +138,7 @@ | `new`、`new --template` | [01](01-getting-started.md) | `add`、`update`、`why` | [05](05-dependencies.md) | | `pack` | [10](10-pack-and-release.md) | `publish`、`emit xpkg`、`xpkg parse` | [11](11-publishing-a-library.md) | | `toolchain` | [20](20-toolchains.md) | `clean`、`cache`、`index`、`self …` | [09](09-commands-by-scenario.md) | +| `--platform`、`--host-platform`、`self env --target` | [35](35-platforms.md) | `pack --pkg-format`、`--linkage`、`--merge` | [10](10-pack-and-release.md) | **概念** @@ -147,6 +150,7 @@ | 岛、接缝、`accel`、`MCPP_EXPORT_C` | [42](42-heterogeneous-builds.md) | 描述符、索引 | [11](11-publishing-a-library.md) | | ABI tag、预建产物 | [12](12-binary-distribution.md) | 退出码、JSON 输出 | [50](50-machine-output.md) | | `xim:` 载荷、`[xlings.workspace]` | [32](32-authoring-a-payload.md)、[23](23-the-project-environment.md) | `compat:` 适配包、`runtime.library_dirs` | [33](33-authoring-an-adapter.md) | +| 平台、平台包、版本组 | [35](35-platforms.md) | 通用包、单文件包、`mcpp-package.json` | [10](10-pack-and-release.md) | ## 规范文档 @@ -166,3 +170,5 @@ - [SPEC-010 —— home 选择与初始化](../specs/home-initialization.md) - [SPEC-011 —— 并发运行：构建目录锁、整体发布、乐观写入与取消](../specs/concurrency.md) - [SPEC-012 —— 面向人的输出：两条流、两种介质、行的种类与颜色](../specs/human-output.md) + - [SPEC-013 —— 平台：一类目标的配置，其选择、合并、对依赖图的补全、链接、封装、运行器与载体](../specs/platforms.md) + - [SPEC-014 —— 打包与分发：链接、闭包与容器，构建腿，暂存树，通用包与单文件包](../specs/packaging.md) From cf07df5800e30317a85c6c7f220c93dadabd2ffc Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 05:23:55 +0900 Subject: [PATCH 13/54] docs/10: packaging around three axes and legs; --pkg-format; universal and single-file packages, mcpp-package.json, compression and signing Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/10-pack-and-release.md | 253 +++++++++++++++++++++++---------- docs/zh/10-pack-and-release.md | 218 +++++++++++++++++++--------- 2 files changed, 331 insertions(+), 140 deletions(-) diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index f9d667aa3..7971f0726 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -55,21 +55,34 @@ when glibc is actually required. Read the `/proc/self/exe` note below before choosing it: launching through a bundled loader changes what the program sees about itself. -## Two axes: target (libc) × mode (bundling depth) +## Three axes and legs -Distribution is two orthogonal choices: +A pack is decided by three independent choices and a set of legs (2026.10.27.1+): -- **libc / static** — a *build-target* property: `--target …-linux-gnu` (glibc) - vs `--target …-linux-musl` (musl, static). `--target …-musl` implies `static`. -- **bundling depth** — a *pack* property: how much of the shared-lib closure - travels with the artifact. This is what `--mode` selects. +| Axis | Option | Question | Values | +|---|---|---|---| +| linkage | `--linkage` | are the libraries inside the program or beside it | `static`, `shared` | +| closure | `--mode` | what the machine that runs the package provides | `system`, `vendored` (default), `self-contained` | +| container | `--pkg-format` | what is delivered | `tar` (default), `dir`, `zip`, `single-file`, a format a package provides | +| legs | `--target`, repeatable | which targets one package covers | triples | + +- **Linkage** is a property of the program, built as `[build] dependency_linkage` + states it. On Linux `--linkage static` selects the `-linux-musl` row and + produces a fully static program; `--target …-linux-musl` implies it. On an ok + image it is one `.oke` with no `.okso`. +- **Closure** is a property of the package: how much of the shared-library + closure travels with the program. `--mode static` is shorthand for + `--linkage static --mode self-contained`. +- **Container** is the shape of the delivery, described below. +- **Legs**: each `--target` is one build, staged into the same tree. A platform + that declares `universal = true` covers all of its targets by default. | Mode | Host must provide | Size | Use case | |---|---|---|---| | `system` | every `.so` (incl. third-party) | smallest | `.deb`/`.rpm`, same-distro fleet (pkg manager declares deps) | | `vendored` (default) | libc / libstdc++ / loader | +a few MB | Mainstream distros (Ubuntu 22+, Debian 12+, RHEL 9+) | | `self-contained` | nothing | +30–50 MB | Any Linux incl. older glibc; bundles closure + `run.sh` wrapper | -| `static` | nothing (single file) | +5–10 MB | musl; matching Linux x86_64 or aarch64 host, Docker scratch, Alpine | +| `static` (`--linkage static --mode self-contained`) | nothing (single file) | +5–10 MB | musl; matching Linux x86_64 or aarch64 host, Docker scratch, Alpine | How to choose: @@ -88,6 +101,16 @@ it is "the target provides everything", which is a statement about the target and cannot be spelled with this machine's absolute paths. e2e 215 sweeps every ELF in the bundle for anything under `$MCPP_HOME` and fails on a hit. +A combination that cannot hold is refused before anything compiles, naming both +choices: + +| Combination | Remedy | +|---|---| +| `--pkg-format single-file` with `--mode system` | `--mode vendored` or `self-contained`: a single file carries what it runs | +| several `--target` for an executable outside a universal package | one pack per target, then `--merge` | +| `--systems` on anything but a self-contained universal package | `--mode self-contained`: only a package that carries its loader differs per system | +| a dispatched format for a `lib` or `shared` target | the library route, [12](12-binary-distribution.md) | + ### A program that needs the HOST to provide something "Self-contained" has a floor. Some libraries can only come from the target @@ -144,32 +167,42 @@ follow the rename: `vendored` → no suffix, `self-contained` → `-bundle-all`, ## Commands ```bash -mcpp pack # vendored by default +mcpp pack # vendored by default mcpp pack --mode system mcpp pack --mode static -mcpp pack --mode self-contained # alias: --mode bundle-all -mcpp pack --target x86_64-linux-musl # equivalent to --mode static -mcpp pack --target aarch64-linux-musl # ARM64 equivalent -mcpp pack --format dir # output as a directory, no tarball -mcpp pack --format appimage # a format a package in the graph provides -mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz -mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path -mcpp pack --profile dev # build with a different profile (default: release) -mcpp pack --dev # the same, as `build` and `run` spell it; --profile wins over it -mcpp pack --message-format json # one mcpp.pack envelope on stdout (mcpp 2026.9.16.1+) -mcpp pack --no-strip # ship the artifacts as built -mcpp pack -p app --format release # a workspace member, as if run in its directory -mcpp pack --workspace --format release # every member with a program, planned and built once -mcpp pack -p cli -p gui -o dist/ # two members, each archive below dist/ -mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ -mcpp pack --format msi --features installer # activate root-package features for the pack +mcpp pack --mode self-contained # alias: --mode bundle-all +mcpp pack --target x86_64-linux-musl # equivalent to --mode static +mcpp pack --target aarch64-linux-musl # ARM64 equivalent +mcpp pack --pkg-format dir # output as a directory, no tarball +mcpp pack --pkg-format appimage # a format a package in the graph provides +mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz +mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path +mcpp pack --profile dev # build with a different profile (default: release) +mcpp pack --dev # the same, as `build` and `run` spell it; --profile wins over it +mcpp pack --message-format json # one mcpp.pack envelope on stdout (mcpp 2026.9.16.1+) +mcpp pack --no-strip # ship the artifacts as built +mcpp pack -p app --pkg-format release # a workspace member, as if run in its directory +mcpp pack --workspace --pkg-format release # every member with a program, planned and built once +mcpp pack -p cli -p gui -o dist/ # two members, each archive below dist/ +mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ +mcpp pack --pkg-format msi --features installer # activate root-package features for the pack +mcpp pack --linkage static # the libraries inside the program +mcpp pack --pkg-format single-file # one executable file +mcpp pack --pkg-format tar --compress zstd # .tar.zst +mcpp pack --target x86_64-openkal # a universal package: every architecture of the platform +mcpp pack --target x86_64-openkal --mode self-contained --systems linux,macos,windows +mcpp pack --merge a.zip b.zip -o all.zip # one tree from packages built on several machines ``` +`--format` is accepted on `pack` and `run` as the earlier spelling of +`--pkg-format` and prints a warning (2026.10.27.1+). On every other command +`--format` is the output encoding (`--format json`). + `--features ` (mcpp 2026.9.15.2+) activates root-package features for every build pass the pack performs: each `--target` leg, and both passes of a dispatched format. It is the value `mcpp build --features` takes, so a host tool needed only for one distribution is declared under `[feature-deps.]` with `tools = [...]` and built -only by the pack that names ``. `mcpp run --format --features ` hands +only by the pack that names ``. `mcpp run --pkg-format --features ` hands the same features to the pack it performs. `-p ` (mcpp 2026.9.28.3+) packs a workspace member from the workspace @@ -185,7 +218,7 @@ A pack states the build it performs as `mcpp build` states one (mcpp 2026.10.1.2+): a line for each package that does work, the status row while it runs, and `Finished` before the first `Packing` line. A pack over several configurations builds every configuration first and writes one `Finished`. The -second pass of a dispatched `--format` is stated in the same way, and its +second pass of a dispatched `--pkg-format` is stated in the same way, and its `Distributing` and `Packed` lines follow it. The build runs `[build] jobs` compiles at once, as every command that compiles does. Before 2026.10.1.2 the build of a pack wrote none of this, and a pack that recompiled for minutes @@ -199,9 +232,9 @@ planned as one selection and built once, and each is then staged, and packed, in a tree of its own: ```bash -mcpp pack --workspace --format release # every member with a program +mcpp pack --workspace --pkg-format release # every member with a program mcpp pack --workspace --exclude updater # every such member but one -mcpp pack -p cli -p gui --format release +mcpp pack -p cli -p gui --pkg-format release ``` Packing each member with its own `mcpp pack -p ` plans the graph, runs @@ -218,7 +251,7 @@ What each member receives is what `mcpp pack -p ` gives it: packed alone, below its own `target/dist/`. With `--output ` it is written below the directory, which is created when it is missing, under the name it has by default. -- **A staged tree of its own.** With a dispatched `--format`, the pack prepares +- **A staged tree of its own.** With a dispatched `--pkg-format`, the pack prepares twice, as above, and the second pass tells each build program the staged tree of the member it acts for. `mcpp::pack_stage_dir()` and `${mcpp.stage_dir}` answer per member. A program acts for the member it belongs to; a package that @@ -250,7 +283,7 @@ packs them all. | more than one `--target` | a program is built for one target, and the several-target Android pack stages the legs of one member | | `--output` that exists as a file | each member's archive or tree is written below the directory | | a member named with `-p` that has no program target | there is nothing to pack for it | -| with a dispatched `--format`, a member no provider acts for | the format is provided by a package the member does not reach, or by a package several members reach | +| with a dispatched `--pkg-format`, a member no provider acts for | the format is provided by a package the member does not reach, or by a package several members reach | | two members that would write one archive or tree | `--output` names one directory, and two packages of one name, version and target share an archive name | A package with a build program that provides a format for several members cannot @@ -261,39 +294,38 @@ one at a time. `--message-format json` (mcpp 2026.9.16.1+) prints one `mcpp.pack` envelope on stdout after the command finishes and sends every human line to stderr. Its `data.artifacts` holds each produced file or directory with its absolute path, -its `type` (`file` or `directory`), the `--format` value and the triples of its +its `type` (`file` or `directory`), the `--pkg-format` value and the triples of its legs; `data.stage` holds the staged tree, its manifest and whether the closure was walked ([50 — Machine Output](50-machine-output.md)). A pack of several members lists every member's artifacts, each naming its `member`, leaves `data.stage` null and names one tree per member in `data.stages` -(2026.10.1.1+). `--format` names the +(2026.10.1.1+). `--pkg-format` names the package format on this command, so machine output is asked for the way `mcpp test` asks for it. -### `--format` owns one axis, and the engine owns two of its values +### `--pkg-format` owns one axis, and the engine owns four of its values -`tar` and `dir` answer the same question `msi` and `appimage` answer — what -shape does the output take — so they are values of one flag rather than the -beginning of a second one. The split between what the engine holds and what a -package holds is: +`tar`, `dir`, `zip` and `single-file` answer the same question `msi` and +`appimage` answer — what shape the output takes — so they are values of one +flag. The split between what the engine holds and what a package holds is: -> **`mcpp pack` owns the mechanism and the one universal format. Every other -> format lives in a package, and `mcpp pack` dispatches to it.** +> **`mcpp pack` owns the mechanism and the formats that need no other release: +> a tree, an archive and a single file. Every other format lives in a package, +> and `mcpp pack` dispatches to it.** -The universal format is what it already produces: an archive that extracts and -runs. It is universal in the only sense that matters here — it needs no -knowledge of anyone else's release. Everything past it does. dpkg's control +Those four extract, or start, and run, and need no knowledge of anyone else's +release. Everything past it does. dpkg's control fields, AppImage's runtime, WiX's schema, Apple's notarisation, Android's signing scheme: each one bound into the engine would couple an mcpp release to a release mcpp does not control. The same argument the project already made for languages, where Slang is supported without being named in the engine. -So the value set is open (mcpp 2026.9.11.1+). `--format ` finds the +So the value set is open (mcpp 2026.9.11.1+). `--pkg-format ` finds the package in the resolved graph that declares `` and hands it the staged tree; an unknown value names what *is* available rather than a fixed list: ``` -error: unknown --format 'bogus'. +error: unknown --pkg-format 'bogus'. available in this build: tar, dir, appimage A format past `tar` and `dir` comes from a package in the resolved graph, which declares it with `mcpp::provides_pack_format("")` in its build program. Add the package @@ -311,7 +343,7 @@ A dispatched format applies to a **program** target — `kind = "app"` included, on every row, whatever file that row links it to (see [04 §2.2](04-mcpp-toml.md)). A library package ships an interface plus prebuilt binaries per triple and has no single staged tree, so -`mcpp pack --format ` is refused rather than ignored. +`mcpp pack --pkg-format ` is refused rather than ignored. **A `kind = "app"` target whose artifact is a shared object accepts more than one `--target`** (mcpp 2026.9.13.2+): on every Android row an application @@ -334,18 +366,18 @@ when it includes a directory (relative or absolute), the literal path is used. For the full set of options, see `mcpp pack --help`. -### `mcpp run --format ` (mcpp 2026.9.12.3+) +### `mcpp run --pkg-format ` (mcpp 2026.9.12.3+) ```bash -mcpp run --target x86_64-linux-android --format apk -mcpp run --target aarch64-ios-sim --format app +mcpp run --target x86_64-linux-android --pkg-format apk +mcpp run --target aarch64-ios-sim --pkg-format app ``` -`--format` is the same flag `mcpp pack` takes, and reused here for the case a +`--pkg-format` is the same flag `mcpp pack` takes, and reused here for the case a plain `mcpp run` cannot reach: an Android application is a `.apk` and an installed iOS application a `.app`, and neither is the link output `mcpp run` -executes by default. `mcpp run --format ` packs the target for `` -— the same two passes and staged tree as `mcpp pack --format ` — and +executes by default. `mcpp run --pkg-format ` packs the target for `` +— the same two passes and staged tree as `mcpp pack --pkg-format ` — and then runs the artifact the pack reported, through the runner resolved for a program: the project's `[target.] runner`, then a dependency's `mcpp::runner(...)`, then the payload descriptor's. @@ -355,7 +387,7 @@ The artifact a pack reports is the request's **terminal** one: among the action consumes. A provider is often a chain (`dist-apk`: link, add the libraries, align, sign), and every output in it is verified to exist, but only the last is the distributable and only it is printed as `Packed`. A format -whose chain ends in two files is refused by `mcpp run --format`, naming both, +whose chain ends in two files is refused by `mcpp run --pkg-format`, naming both, because a runner takes one operand. A format may report many outputs, one per file of a distribution tree. Up to @@ -365,10 +397,10 @@ files)`), and `--verbose` prints every one (2026.9.29.5+). `--message-format json` lists every output in either case. An unknown `` is refused naming the format set the resolved graph -provides, the same set `mcpp pack --format bogus` reports. `--format` together +provides, the same set `mcpp pack --pkg-format bogus` reports. `--pkg-format` together with `--no-runner` is refused — an `.apk` or an installed `.app` cannot be executed directly. Running a `kind = "app"` target on a row where its form is -a library, with no `--format`, is refused naming the flag and that same +a library, with no `--pkg-format`, is refused naming the flag and that same format set. `mcpp test` is unaffected: a test binary is a program on every row. @@ -448,7 +480,7 @@ path. every library name the closure resolved: the staged path of a library the tree carries, `platform` for one the target provides, and `unresolved` for one found nowhere (the line format is in [50](50-machine-output.md#the-stage-manifest)). -When a name is unresolved, `--format tar` and `--format dir` fail naming it, +When a name is unresolved, `--pkg-format tar` and `--pkg-format dir` fail naming it, because a tree without that library does not start where it is installed; a dispatched format still receives the tree, with `closure = not-walked`. A library the target provides and this machine does not have is named in @@ -675,7 +707,8 @@ unaffected. ### ok images (`-openkal`): the image, or a program that carries it An ok image runs through `okrun` on every system of its architecture (see -[24](24-openkal-cross.md)). The mode keeps its meaning — what the machine that +[24](24-openkal-cross.md)), and its platform states how it is packed +([35](35-platforms.md)). The mode keeps its meaning — what the machine that runs the package must provide: | Mode | The package holds | The machine provides | @@ -683,14 +716,15 @@ runs the package must provide: | `system` | `bin/.oke` | `okrun` and every image it names | | `vendored` (default) | `bin/.oke` and the `.okso` images it names | `okrun` | | `self-contained` | `bin/` — a program of this machine that carries the image as a section (`.oke`, `__OKE,__image`) and runs it through libokrun — and the `.okso` images beside it | nothing | -| `static` | refused | — | +| `static` | `bin/`, a program of this machine that carries one static `.oke` | nothing | -The image is not recompiled for `self-contained`: mcpp generates the entry and -the section, depends on `openkal-run` from the source of the project's -`openkal-tools`, and packs that program as it packs any other — static (musl) -on Linux, with only the system's libraries on macOS and Windows. The bytes in -the section are the `.oke`'s. Another system's program is packed on that -system. +`--linkage static` builds the image with no `.okso`. The image is not +recompiled for `self-contained`: mcpp builds the platform's carrier (`[platform] +carrier`, a package and its entry source), places the image's bytes in the +carrier's section, and packs that program as it packs any other — static (musl) +on Linux, with only the system's libraries on macOS and Windows. Another +system's program is packed on that system, or by `--systems` in a universal +package. ### Android: the application object and its closure under `lib/` @@ -718,6 +752,76 @@ own closure. `--mode` does not apply on this row: the device provides exactly the platform libraries, so there is no host set to leave out and no loader to carry. +### A universal package (`[platform] universal`) + +A platform that declares `universal = true` (openkal-platform does) makes a +pack of any of its targets cover all of them; `--no-universal` packs the +selected target alone. One tree serves every system and every architecture of +the platform: + +``` +target/dist/hello-0.1.0-openkal/ +├── bin/hello ← POSIX launcher +├── bin/hello.cmd ← Windows launcher +├── lib/x86_64/hello.oke ← one set of images per architecture +├── lib/aarch64/hello.oke +├── libexec/okrun/-/ ← only with --mode self-contained +├── share/… ← resources, once +└── mcpp-package.json +``` + +The launchers are generated from the platform's `runner`: each picks the +architecture directory from `uname -m` or `%PROCESSOR_ARCHITECTURE%` and starts +the runner on the image there, looking first in `libexec/` and then on `PATH`. +`--systems linux,macos,windows` builds the loader for each system and +architecture into `libexec/` (the default is this machine's system); a system +whose SDK payload this machine lacks is refused, naming it. Packages built on +several machines are joined with `mcpp pack --merge a.zip b.zip …`: legs and +systems are united, and two files at one path must have the same digest. + +### A single-file package + +`--pkg-format single-file` delivers one executable file: + +| Target and mode | The file | On start | +|---|---|---| +| native, `vendored` or `self-contained` | the engine's stub and the staged tree, zstd-compressed | the first run unpacks the tree into the user's cache, and later runs start it from there | +| native Linux, `--linkage static` | a static stub and the static program | as above, with no system library | +| ok image, `self-contained` | the platform's carrier and the images of its architecture | the carrier loads the images from memory and writes nothing to disk | + +The cache is `$XDG_CACHE_HOME` (else `~/.cache`) on Linux, `~/Library/Caches` +on macOS and `%LOCALAPPDATA%` on Windows, below a directory named by the +payload's sha256. The payload's digest is checked before it is unpacked, it is +unpacked under a temporary name and renamed into place, and an incomplete +directory is never used. + +### `mcpp-package.json` + +Every staged tree carries `mcpp-package.json`: the package, its platform, its +three choices, its legs and systems, how it starts, and each file with its role +and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it +([SPEC-014](specs/packaging.md) §4.5): + +```json +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "mcpplibs.openkal-platform@0.17.1", + "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], + "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, + "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} +``` + +`mcpp emit xpkg` writes one index entry per system and architecture, or, for a +universal package, one asset for every system with the loader's package as a +dependency. + +### Compression and signing + +`--compress zstd|gzip|xz|none[:level]` (or `[pack] compress`) chooses the +compression of `tar`; the default is gzip. A `zip` stores its entries +uncompressed, and the images inside an okb are stored so that the loader maps +them in place. `[pack] sign = ["", …, "{}"]` runs once on each program and +image of the staged tree before the container is written; a failure stops the +pack. + ## Configuration Packaging behavior is configured via the `[pack]` section in `mcpp.toml`. The @@ -726,6 +830,10 @@ common fields are: ```toml [pack] default_mode = "static" # override the normal vendored default for bare `mcpp pack` +linkage = "shared" # the default of --linkage +format = "tar" # the default of --pkg-format +compress = "zstd" # the default of --compress +sign = ["signtool", "sign", "{}"] # run on each program and image before the container is written strip = true # default. false ships the artifacts as built debug_symbols = "dist/debug" # separate the debug info here instead of discarding it include = ["share/**", "config/*.toml"] # extra files to bundle @@ -747,15 +855,14 @@ The `static` mode additionally requires a musl toolchain configured under `[target.]`; for the full setup, see the `mcpp.toml` in [`examples/03-pack-static`](../examples/03-pack-static/). -## Planned Support - -Windows DLL bundling beyond the current `.zip` is on the roadmap. - -Distribution formats such as `.deb`, `.rpm`, AppImage and `.msi` are **not** on -this list, and that is a decision rather than an omission: they live in -packages and reach the user through `--format `, for the reason the -section above gives. Nothing further needs to join `[pack]`'s built-in modes. +## Current limitations -This document evolves alongside the `mcpp pack` implementation; for the latest -options, refer to `mcpp pack --help`. +- Distribution formats such as `.deb`, `.rpm`, AppImage and `.msi` come from + packages through `--pkg-format `; the engine provides `tar`, `dir`, `zip` + and `single-file`. +- A `zip` stores its entries uncompressed. +- One file that runs on several systems is not produced; a single-file package + serves one system and one architecture. +- An ELF program's closure is resolved by running its dynamic linker, so a Linux + program is not packed from Windows. diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 94919095e..37b15ce77 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -49,21 +49,32 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 它之前先读下面 `/proc/self/exe` 一节：经由内置 loader 启动会改变程序对「自己 在哪」的认知。 -## 两条轴：target（libc）× mode（打包深度） +## 三个维度与构建腿 -分发有两项正交的选择： +一次打包由三个相互独立的选择与一组构建腿决定（2026.10.27.1+）： -- **libc / static** 是一项*构建 target* 属性：`--target …-linux-gnu`（glibc） - 与 `--target …-linux-musl`（musl，静态）。`--target …-musl` 隐含 `static`。 -- **打包深度** 是一项*pack* 属性：产物随身携带多少共享库闭包，由 `--mode` - 选择。 +| 维度 | 选项 | 回答的问题 | 取值 | +|---|---|---|---| +| 链接 | `--linkage` | 库在程序里，还是在程序旁边 | `static`、`shared` | +| 闭包 | `--mode` | 运行该包的机器提供什么 | `system`、`vendored`（默认）、`self-contained` | +| 容器 | `--pkg-format` | 交付成什么 | `tar`（默认）、`dir`、`zip`、`single-file`、某个包提供的格式 | +| 腿 | `--target`，可重复 | 一个包覆盖哪些目标 | 三元组 | + +- **链接**是程序的属性，按 `[build] dependency_linkage` 构建。Linux 上 `--linkage static` + 选择 `-linux-musl` 行，产出全静态程序；`--target …-linux-musl` 隐含它。ok 映像上 + 它是一个 `.oke`、没有 `.okso`。 +- **闭包**是包的属性：共享库闭包中有多少随程序携带。`--mode static` 是 + `--linkage static --mode self-contained` 的简写。 +- **容器**是交付的形状，见下文。 +- **腿**：每个 `--target` 是一次构建，暂存进同一棵树。声明了 `universal = true` 的平台 + 缺省覆盖它的全部目标。 | Mode | 宿主必须提供 | 体积 | 使用场景 | |---|---|---|---| | `system` | 每一个 `.so`（含第三方） | 最小 | `.deb`/`.rpm`，同发行版机群（由包管理器声明依赖） | | `vendored`（默认） | libc / libstdc++ / loader | +几 MB | 主流发行版（Ubuntu 22+、Debian 12+、RHEL 9+） | | `self-contained` | 无 | +30–50 MB | 任意 Linux，含较旧的 glibc；携带闭包与 `run.sh` wrapper | -| `static` | 无（单文件） | +5–10 MB | musl；架构匹配的 Linux x86_64 或 aarch64，Docker scratch，Alpine | +| `static`（`--linkage static --mode self-contained`） | 无（单文件） | +5–10 MB | musl；架构匹配的 Linux x86_64 或 aarch64，Docker scratch，Alpine | 如何选择： @@ -81,6 +92,15 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 来。e2e 215 会扫描 bundle 里每一个 ELF，查找任何落在 `$MCPP_HOME` 之下的路径， 命中即失败。 +不能成立的组合在任何编译之前被拒绝，消息写出两个选择： + +| 组合 | 出路 | +|---|---| +| `--pkg-format single-file` 与 `--mode system` | `--mode vendored` 或 `self-contained`：单文件携带它所运行的东西 | +| 通用包之外、对一个可执行文件给出多个 `--target` | 每个目标打一次，再 `--merge` | +| 自带加载器的通用包之外使用 `--systems` | `--mode self-contained`：只有携带加载器的包随系统不同 | +| 对 `lib` 或 `shared` 目标使用被分派的格式 | 库的路线，[12](12-binary-distribution.md) | + ### 需要宿主提供某种能力的程序 「自包含」有一个下限。有些库只能来自目标机器：图形驱动的用户态部分与正在 @@ -128,32 +148,41 @@ wire 格式（由 `install.sh` 消费），**不**跟随改名：`vendored` 无 ## 命令 ```bash -mcpp pack # vendored by default +mcpp pack # vendored by default mcpp pack --mode system mcpp pack --mode static -mcpp pack --mode self-contained # alias: --mode bundle-all -mcpp pack --target x86_64-linux-musl # equivalent to --mode static -mcpp pack --target aarch64-linux-musl # ARM64 equivalent -mcpp pack --format dir # output as a directory, no tarball -mcpp pack --format appimage # a format a package in the graph provides -mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz -mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path -mcpp pack --profile dev # build with a different profile (default: release) -mcpp pack --dev # the same, as `build` and `run` spell it; --profile wins over it -mcpp pack --message-format json # one mcpp.pack envelope on stdout (mcpp 2026.9.16.1+) -mcpp pack --no-strip # ship the artifacts as built -mcpp pack -p app --format release # a workspace member, as if run in its directory -mcpp pack --workspace --format release # every member with a program, planned and built once -mcpp pack -p cli -p gui -o dist/ # two members, each archive below dist/ -mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ -mcpp pack --format msi --features installer # activate root-package features for the pack +mcpp pack --mode self-contained # alias: --mode bundle-all +mcpp pack --target x86_64-linux-musl # equivalent to --mode static +mcpp pack --target aarch64-linux-musl # ARM64 equivalent +mcpp pack --pkg-format dir # output as a directory, no tarball +mcpp pack --pkg-format appimage # a format a package in the graph provides +mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz +mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path +mcpp pack --profile dev # build with a different profile (default: release) +mcpp pack --dev # the same, as `build` and `run` spell it; --profile wins over it +mcpp pack --message-format json # one mcpp.pack envelope on stdout (mcpp 2026.9.16.1+) +mcpp pack --no-strip # ship the artifacts as built +mcpp pack -p app --pkg-format release # a workspace member, as if run in its directory +mcpp pack --workspace --pkg-format release # every member with a program, planned and built once +mcpp pack -p cli -p gui -o dist/ # two members, each archive below dist/ +mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ +mcpp pack --pkg-format msi --features installer # activate root-package features for the pack +mcpp pack --linkage static # the libraries inside the program +mcpp pack --pkg-format single-file # one executable file +mcpp pack --pkg-format tar --compress zstd # .tar.zst +mcpp pack --target x86_64-openkal # a universal package: every architecture of the platform +mcpp pack --target x86_64-openkal --mode self-contained --systems linux,macos,windows +mcpp pack --merge a.zip b.zip -o all.zip # one tree from packages built on several machines ``` +`pack` 与 `run` 接受 `--format` 作为 `--pkg-format` 的旧写法，并打印一条警告（2026.10.27.1+）。 +其他命令上的 `--format` 是输出编码（`--format json`）。 + `--features `（mcpp 2026.9.15.2+）为打包所执行的每一次构建启用根包 feature：每一条 `--target` 腿，以及被分派格式的两遍构建。它接受的取值与 `mcpp build --features` 相同，因此只为某一种发布才需要的宿主工具，可以声明在 带 `tools = [...]` 的 `[feature-deps.]` 下，只由点名 `` 的那次打包构建 -出来。`mcpp run --format --features ` 把同样的 feature 交给它 +出来。`mcpp run --pkg-format --features ` 把同样的 feature 交给它 执行的那次打包。 `-p `（mcpp 2026.9.28.3+）在工作区根目录打包某个成员：成员的解析方式与其他 `-p` @@ -165,7 +194,7 @@ feature：每一条 `--target` 腿，以及被分派格式的两遍构建。它 打包对它所执行的构建的陈述方式与 `mcpp build` 相同（mcpp 2026.10.1.2+）：每个有工作的包 一行，运行期间显示状态行，并在第一行 `Packing` 之前写出 `Finished`。跨多个配置的打包先构建 -全部配置，只写一次 `Finished`。由包提供的 `--format` 的第二遍以同样方式陈述，其后是 +全部配置，只写一次 `Finished`。由包提供的 `--pkg-format` 的第二遍以同样方式陈述，其后是 `Distributing` 与 `Packed` 行。构建同时运行 `[build] jobs` 个编译，与每一条会编译的命令相同。 2026.10.1.2 之前，打包中的构建不写出上述任何内容，重新编译数分钟的打包只显示 `Planning`。 @@ -176,9 +205,9 @@ feature：每一条 `--target` 腿，以及被分派格式的两遍构建。它 然后每个成员在各自的暂存树中暂存并打包： ```bash -mcpp pack --workspace --format release # 每个有程序的成员 +mcpp pack --workspace --pkg-format release # 每个有程序的成员 mcpp pack --workspace --exclude updater # 除一个之外的每个这样的成员 -mcpp pack -p cli -p gui --format release +mcpp pack -p cli -p gui --pkg-format release ``` 对每个成员分别运行 `mcpp pack -p `，每次都会重新规划图、重新运行构建程序、 @@ -190,7 +219,7 @@ mcpp pack -p cli -p gui --format release - **产物。** 成员的归档或目录，就是它单独打包时得到的那一份，位于它自己的 `target/dist/` 之下。给出 `--output ` 时，它写在该目录之下（目录不存在时会创建），使用默认的名字。 -- **自己的暂存树。** 使用被分派的 `--format` 时，打包如上所述准备两次，第二遍告诉每个 +- **自己的暂存树。** 使用被分派的 `--pkg-format` 时，打包如上所述准备两次，第二遍告诉每个 构建程序它所代表的成员的暂存树。`mcpp::pack_stage_dir()` 与 `${mcpp.stage_dir}` 按成员回答。一个程序代表它所属的成员；只有一个被打包成员依赖到的包（成员所依赖的 提供者）代表那个成员；被多个被打包成员依赖到的包不代表其中任何一个，它既不会被告知 @@ -214,7 +243,7 @@ mcpp pack -p cli -p gui --format release | 多于一个 `--target` | 一个程序只为一个 target 构建，而多 target 的 Android 打包暂存的是一个成员的各条腿 | | 已存在且是文件的 `--output` | 每个成员的归档或目录写在该目录之下 | | 用 `-p` 点名、却没有程序 target 的成员 | 它没有可打包的东西 | -| 使用被分派的 `--format` 时，没有任何提供者代表它的成员 | 该格式由成员依赖不到的包提供，或由被多个成员依赖到的包提供 | +| 使用被分派的 `--pkg-format` 时，没有任何提供者代表它的成员 | 该格式由成员依赖不到的包提供，或由被多个成员依赖到的包提供 | | 两个会写同一个归档或目录的成员 | `--output` 只点名一个目录，而名字、版本与 target 相同的两个包共用同一个归档名 | 一个为多个成员提供格式、带构建程序的包，无法同时服务它们：提供者只会得到一棵暂存树。 @@ -223,35 +252,33 @@ mcpp pack -p cli -p gui --format release `--message-format json`（mcpp 2026.9.16.1+）在命令结束后于 stdout 上输出一个 `mcpp.pack` 信封，所有给人看的行都改走 stderr。它的 `data.artifacts` 列出 产出的每一个文件或目录，带绝对路径、`type`（`file` 或 `directory`）、 -`--format` 取值以及各条腿的三元组；`data.stage` 给出暂存树、它的 manifest， +`--pkg-format` 取值以及各条腿的三元组；`data.stage` 给出暂存树、它的 manifest， 以及闭包是否已经走通（见 [50 —— 机器输出](50-machine-output.md)）。对多个成员的 打包列出每个成员的产物，每一条点名它的 `member`，`data.stage` 为 null，并在 `data.stages` 中为每个成员给出一棵树（2026.10.1.1+）。这条命令 -上的 `--format` 表示的是包格式，因此机器输出改用 `mcpp test` 请求它的那种 +上的 `--pkg-format` 表示的是包格式，因此机器输出改用 `mcpp test` 请求它的那种 方式。 -### `--format` 只是一根轴，引擎只拥有它的两个取值 +### `--pkg-format` 只是一根轴，引擎拥有它的四个取值 -`tar` 与 `dir` 回答的问题，和 `msi` 与 `appimage` 回答的问题是同一个 —— -输出取什么形状 —— 所以它们是同一个 flag 的不同取值，而不是第二个 flag 的开端。 -引擎持有什么、包持有什么，分界如下： +`tar`、`dir`、`zip` 与 `single-file` 回答的问题，和 `msi` 与 `appimage` 回答的问题是同一个 —— +输出取什么形状 —— 所以它们是同一个 flag 的不同取值。引擎持有什么、包持有什么，分界如下： -> **`mcpp pack` 拥有机制，以及那一种通用格式。其余每一种格式都住在某个包里， -> 由 `mcpp pack` 分派过去。** +> **`mcpp pack` 拥有机制，以及不依赖任何其他发布的格式：一棵树、一个归档、一个文件。 +> 其余每一种格式都住在某个包里，由 `mcpp pack` 分派过去。** -那种通用格式就是它已经在产出的东西：一个解开即可运行的归档。它「通用」只在这 -里唯一要紧的那个意义上成立 —— 它不需要了解任何别人的发布方式。此外的一切都 +这四种解开或启动即可运行，不需要了解任何别人的发布方式。此外的一切都 需要。dpkg 的 control 字段、AppImage 的 runtime、WiX 的 schema、Apple 的 公证、Android 的签名方案：其中任何一个被绑进引擎，都会把一次 mcpp 的发布耦合 到一次 mcpp 并不控制的发布上。这与本项目已经为语言做过的论证是同一个 —— Slang 被支持，而引擎里没有它的名字。 -所以取值集合是开放的（mcpp 2026.9.11.1+）。`--format ` 在解析后的图里 +所以取值集合是开放的（mcpp 2026.9.11.1+）。`--pkg-format ` 在解析后的图里 找到声明了 `` 的那个包，并把暂存树交给它；一个未知的取值点名的是 **当下确实可用**的那些，而不是一份固定清单： ``` -error: unknown --format 'bogus'. +error: unknown --pkg-format 'bogus'. available in this build: tar, dir, appimage A format past `tar` and `dir` comes from a package in the resolved graph, which declares it with `mcpp::provides_pack_format("")` in its build program. Add the package @@ -267,7 +294,7 @@ error: unknown --format 'bogus'. 被分派的格式作用于一个**程序** target —— `kind = "app"` 也不例外，无论这一行 把它链接成什么文件（见 [04 §2.2](04-mcpp-toml.md)）。库包发布的是一份接口 加上按三元组给出的预构建产物，没有单独一棵暂存树，所以 -`mcpp pack <库> --format ` 会被拒绝，而不是被忽略。 +`mcpp pack <库> --pkg-format ` 会被拒绝，而不是被忽略。 **产物是共享目标文件的 `kind = "app"` target 可以接受一个以上的 `--target`** （mcpp 2026.9.13.2+）：在每一行 Android 上，应用*就是*平台加载的那个共享库， @@ -287,17 +314,17 @@ target，第二个 `--target` 依旧被拒绝：为多个三元组打包同一 完整选项参见 `mcpp pack --help`。 -### `mcpp run --format `（mcpp 2026.9.12.3+） +### `mcpp run --pkg-format `（mcpp 2026.9.12.3+） ```bash -mcpp run --target x86_64-linux-android --format apk -mcpp run --target aarch64-ios-sim --format app +mcpp run --target x86_64-linux-android --pkg-format apk +mcpp run --target aarch64-ios-sim --pkg-format app ``` -`--format` 与 `mcpp pack` 用的是同一个 flag，这里复用它是为了覆盖一种普通 +`--pkg-format` 与 `mcpp pack` 用的是同一个 flag，这里复用它是为了覆盖一种普通 `mcpp run` 够不到的情形：一个 Android 应用程序是一个 `.apk`，一个已安装的 iOS 应用程序是一个 `.app`，两者都不是 `mcpp run` 默认执行的链接产物。 -`mcpp run --format ` 先为 `` 打包 —— 与 `mcpp pack --format +`mcpp run --pkg-format ` 先为 `` 打包 —— 与 `mcpp pack --pkg-format ` 相同的两遍构建与暂存树 —— 再经由为一个程序解析出的 runner 运行打包 报出的那个产物：先看项目的 `[target.] runner`，其次看依赖的 `mcpp::runner(...)`，再次看载荷描述文件的。 @@ -306,16 +333,16 @@ mcpp run --target aarch64-ios-sim --format app 没有被任何其他被引入的动作当作输入的那一个输出。一个提供者常常是一条链 （`dist-apk`：链接、加库、对齐、签名），链上每个输出都会被核验存在，但只有 最后一个是发布物，也只有它以 `Packed` 报出。链的末端产出两个文件的格式会被 -`mcpp run --format` 拒绝并点名两者，因为一个 runner 只接受一个操作数。 +`mcpp run --pkg-format` 拒绝并点名两者，因为一个 runner 只接受一个操作数。 一个格式可能报出很多输出，发布目录中的每个文件各一个。不超过八个时逐行打印；更多时按它们 在共同目录之下所处的条目打印，并附数量（`Packed Release/app (1309 files)`），`--verbose` 打印全部（2026.9.29.5+）。`--message-format json` 在两种情况下都列出全部输出。 未知的 `` 会被拒绝，点名已解析图提供的格式集合，与 -`mcpp pack --format bogus` 报出的是同一个集合。`--format` 与 `--no-runner` +`mcpp pack --pkg-format bogus` 报出的是同一个集合。`--pkg-format` 与 `--no-runner` 同时出现会被拒绝 —— 一个 `.apk` 或一个已安装的 `.app` 无法被直接执行。在 -`kind = "app"` 的形态是一个库的那一行上、不带 `--format` 运行它，同样会被 +`kind = "app"` 的形态是一个库的那一行上、不带 `--pkg-format` 运行它，同样会被 拒绝，点名该 flag 与同一个格式集合。`mcpp test` 不受影响：测试二进制在每一行 上都是一个程序。 @@ -384,7 +411,7 @@ tarball 的内容包在一个顶层目录里，该目录的名字与 tarball 文 暂存树旁边的 `.stage-manifest` 列出闭包解析过的每一个库名：树里携带的 库给出其暂存路径，目标机自己提供的库记为 `platform`，哪里都找不到的记为 `unresolved`（行格式见 [50](50-machine-output.md#暂存清单)）。只要有一个名字 -是 unresolved，`--format tar` 与 `--format dir` 就会失败并点名它，因为一棵 +是 unresolved，`--pkg-format tar` 与 `--pkg-format dir` 就会失败并点名它，因为一棵 缺了这个库的树在安装处无法启动；被分派的格式仍然会收到这棵树，并带上 `closure = not-walked`。目标机提供、而本机没有的库，写进 `[pack.bundle-project] also_skip`。不打包任何东西的 mode（`system`、 @@ -591,20 +618,20 @@ dylib 的消费者，其链接所带的 `@loader_path` rpath 本就会查找的 ### ok 映像（`-openkal`）：映像本身，或携带它的程序 -ok 映像经 `okrun` 在其架构的每个系统上运行（见 [24](24-openkal-cross.md)）。mode -的含义不变 —— 运行该包的机器必须提供什么： +ok 映像经 `okrun` 在其架构的每个系统上运行（见 [24](24-openkal-cross.md)），怎样打包由它的平台陈述 +（[35](35-platforms.md)）。mode 的含义不变 —— 运行该包的机器必须提供什么： | Mode | 包内内容 | 机器需提供 | |---|---|---| | `system` | `bin/.oke` | `okrun` 与它所需的每个映像 | | `vendored`（默认） | `bin/.oke` 与它所需的 `.okso` 映像 | `okrun` | | `self-contained` | `bin/`：本机的程序，以段（`.oke`、`__OKE,__image`）携带映像并经 libokrun 运行，旁边是所需的 `.okso` 映像 | 无 | -| `static` | 拒绝 | — | +| `static` | `bin/`：本机的程序，携带一个静态 `.oke` | 无 | -`self-contained` 不重新编译映像：mcpp 生成入口与段，依赖与工程的 `openkal-tools` -同源的 `openkal-run`，再按任何程序的方式打包该程序 —— Linux 上为静态（musl）， -macOS 与 Windows 上只引用系统自带的库。段中的字节即 `.oke` 的字节。其他系统的 -程序在该系统上打包。 +`--linkage static` 构建不带 `.okso` 的映像。`self-contained` 不重新编译映像：mcpp 构建平台的载体 +（`[platform] carrier`，一个包与它的入口源码），把映像的字节放进载体的段，再按任何程序的方式打包该程序 —— +Linux 上为静态（musl），macOS 与 Windows 上只引用系统自带的库。其他系统的程序在该系统上打包，或在通用包中 +由 `--systems` 构建。 ### Android：应用目标文件与它的闭包在 `lib/` 下 @@ -630,6 +657,61 @@ target/dist/myapp-0.1.0-x86_64-linux-android/ 闭包。`--mode` 不适用于这一行：设备恰好提供了平台库，因此既没有可以留给宿主 的库集合，也没有需要携带的 loader。 +### 通用包（`[platform] universal`） + +声明了 `universal = true` 的平台（openkal-platform 即是）使它任一目标的打包覆盖其全部目标；`--no-universal` +只打所选的目标。一棵树服务该平台的每个系统与每个架构： + +``` +target/dist/hello-0.1.0-openkal/ +├── bin/hello ← POSIX launcher +├── bin/hello.cmd ← Windows launcher +├── lib/x86_64/hello.oke ← one set of images per architecture +├── lib/aarch64/hello.oke +├── libexec/okrun/-/ ← only with --mode self-contained +├── share/… ← resources, once +└── mcpp-package.json +``` + +启动器由平台的 `runner` 生成：按 `uname -m` 或 `%PROCESSOR_ARCHITECTURE%` 选择架构目录，以该目录中的映像 +启动运行器，先在 `libexec/` 中、再在 `PATH` 上查找运行器。`--systems linux,macos,windows` 为每个系统与架构构建 +加载器放入 `libexec/`（缺省为本机系统）；本机缺少某个系统的 SDK 载荷时拒绝并点名。多台机器打出的包以 +`mcpp pack --merge a.zip b.zip …` 合并：腿与系统取并集，同一路径上的两个文件摘要必须相同。 + +### 单文件包 + +`--pkg-format single-file` 交付一个可执行文件： + +| 目标与模式 | 文件 | 启动时 | +|---|---|---| +| 原生，`vendored` 或 `self-contained` | 引擎的启动桩加 zstd 压缩的暂存树 | 首次运行把树解到用户缓存，之后从那里启动 | +| 原生 Linux，`--linkage static` | 静态启动桩加静态程序 | 同上，不依赖任何系统库 | +| ok 映像，`self-contained` | 平台的载体加本架构的映像 | 载体从内存加载映像，不写盘 | + +缓存在 Linux 上为 `$XDG_CACHE_HOME`（否则 `~/.cache`），macOS 上为 `~/Library/Caches`，Windows 上为 +`%LOCALAPPDATA%`，其下以载荷的 sha256 命名目录。解压前校验载荷摘要，以临时名解压后原子改名，不完整的目录 +从不使用。 + +### `mcpp-package.json` + +每棵暂存树都带 `mcpp-package.json`：包、它的平台、三个选择、腿与系统、启动方式，以及每个文件的角色与 sha256。 +提供的格式、`--merge` 与 `mcpp emit xpkg` 读取它（[SPEC-014](../specs/packaging.md) §4.5）： + +```json +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "mcpplibs.openkal-platform@0.17.1", + "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], + "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, + "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} +``` + +`mcpp emit xpkg` 为每个系统与架构写一个索引条目；通用包则写一个对应所有系统的资产，并以加载器的包为依赖。 + +### 压缩与签名 + +`--compress zstd|gzip|xz|none[:level]`（或 `[pack] compress`）选择 `tar` 的压缩；缺省为 gzip。`zip` 的条目 +不压缩存放；okb 中的映像以 stored 存放，使加载器原地映射。`[pack] sign = ["", …, "{}"]` 在写容器之前 +对暂存树中的每个程序与映像各运行一次；失败使打包停止。 + ## 配置项 打包行为通过 `mcpp.toml` 中的 `[pack]` 节配置，常用字段如下： @@ -637,6 +719,10 @@ target/dist/myapp-0.1.0-x86_64-linux-android/ ```toml [pack] default_mode = "static" # override the normal vendored default for bare `mcpp pack` +linkage = "shared" # the default of --linkage +format = "tar" # the default of --pkg-format +compress = "zstd" # the default of --compress +sign = ["signtool", "sign", "{}"] # run on each program and image before the container is written strip = true # default. false ships the artifacts as built debug_symbols = "dist/debug" # separate the debug info here instead of discarding it include = ["share/**", "config/*.toml"] # extra files to bundle @@ -658,12 +744,10 @@ force_bundle = ["libfoo.so"] # bundle even if matched by the PEP 600 list 参见 [`examples/03-pack-static`](../../examples/03-pack-static/) 中的 `mcpp.toml`。 -## 待支持 - -当前 `.zip` 之外的 Windows DLL 分发方式在规划中。 - -`.deb`、`.rpm`、AppImage、`.msi` 这些分发格式**不在**这份清单上，而这是一个 -决定而不是一处遗漏：它们住在各自的包里，经由 `--format ` 到达用户， -理由见上一节。`[pack]` 内建的这几种 mode 不需要再添任何新成员。 +## 当前边界 -本文档随 `mcpp pack` 的实现演进；最新选项以 `mcpp pack --help` 为准。 +- `.deb`、`.rpm`、AppImage、`.msi` 等分发格式经 `--pkg-format ` 由包提供；引擎提供 `tar`、`dir`、 + `zip` 与 `single-file`。 +- `zip` 的条目不压缩存放。 +- 不产出在多个系统上运行的单个文件；一个单文件包服务一个系统与一个架构。 +- ELF 程序的闭包由其动态链接器解析，因此不能在 Windows 上打包 Linux 程序。 From 8749431d68edf898004ba51f79fd2e929bb38640 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 05:28:43 +0900 Subject: [PATCH 14/54] docs: platforms in 04, 08, 09, 21, 22, 24, 30, 34, 41 (en, zh); --pkg-format for pack and run - 04: the platform keys of [target.]; the platform's runner - 08: MCPP_BIN_; a platform package tested on its targets - 09: --platform / --host-platform for one invocation - 21, 22: the target table selects openkal-platform; `platform` source - 24: zero-declaration ok images; the hosted arrangement as a platform - 30: protocol 17 directives; mcpp::phase() "platform" - 34: the board as a platform package - 41: the runner table form (host, exclusive) and its order Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/04-mcpp-toml.md | 45 +++++++++++++++++++++----- docs/08-testing.md | 12 +++++++ docs/09-commands-by-scenario.md | 8 ++++- docs/21-the-target-triple.md | 2 +- docs/22-target-side.md | 4 +++ docs/24-openkal-cross.md | 45 +++++++++++++++----------- docs/30-build-mcpp.md | 25 ++++++++------ docs/31-authoring-a-rule-package.md | 2 +- docs/34-authoring-a-bsp.md | 25 ++++++++++++++ docs/41-devices.md | 26 ++++++++++++--- docs/zh/04-mcpp-toml.md | 39 ++++++++++++++++++---- docs/zh/08-testing.md | 8 +++++ docs/zh/09-commands-by-scenario.md | 6 +++- docs/zh/21-the-target-triple.md | 2 +- docs/zh/22-target-side.md | 3 ++ docs/zh/24-openkal-cross.md | 39 +++++++++++----------- docs/zh/30-build-mcpp.md | 25 ++++++++------ docs/zh/31-authoring-a-rule-package.md | 2 +- docs/zh/34-authoring-a-bsp.md | 19 +++++++++++ docs/zh/41-devices.md | 21 +++++++++--- 20 files changed, 269 insertions(+), 89 deletions(-) diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 9351f82a0..6eef022ce 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -249,9 +249,9 @@ platform's contract, not a name mcpp assigns. `exports` (above) applies to an `windows_entry` (below) accept `app` exactly as they accept `bin`. `mcpp run` of an `app` target, on a row where its form is a shared library, -refuses without `--format`, naming the flag and the formats the resolved -graph provides. `mcpp pack --format apk` stages the library where the -closure already places a shared object. See `mcpp run --format` in +refuses without `--pkg-format`, naming the flag and the formats the resolved +graph provides. `mcpp pack --pkg-format apk` stages the library where the +closure already places a shared object. See `mcpp run --pkg-format` in [10 — Pack and Release](10-pack-and-release.md). An engine older than 2026.9.12.3 does not know the value and refuses it, @@ -1382,11 +1382,10 @@ different argv (`-bios default` for an OpenSBI boot, `-bios none -semihosting` for a picolibc image) — and an engine that guesses one is an engine the other board has to fight. A board-support package normally supplies it. -The one target whose runner is a property of the target rather than of a -board is `-openkal` (2026.10.26.2): an ok image is run by `okrun` on every -system, so `okrun` is that target's runner when the project declares none. It is -found as a declared runner's program is, with the programs the dependency graph -built for this machine (`tools = [...]`) searched first ([24](24-openkal-cross.md)). +A target configured by a platform takes the platform's runner when the project +declares none: `-openkal` is run by `okrun` on every system (2026.10.27.1+, +[35](35-platforms.md)). It is found as a declared runner's program is, with the +programs the dependency graph built for this machine searched first. ### 2.7.3 `runner` on a hosted target (2026.9.2.1+) @@ -1490,6 +1489,36 @@ when a base is stated, `MCPP_SYSROOT`. Its exit status is the program's. An Apple artifact on a host that is not macOS, with no `run_under`, is refused before anything is spawned, naming this key. +#### 2.7.3.3 The platform keys (2026.10.27.1+) + +`[target.]` also selects and overrides the target's **platform**, with +the platform's own key names ([35 — Platforms](35-platforms.md)): + +```toml +[target.'cfg(os = "openkal")'] +platform = "openkal-platform@0.17.1" # or { path = "…" }, or "none" + +[target.'cfg(platform = "openkal")'.layers] +c-abi = { path = "../openkal-musl" } + +[target.'cfg(platform = "openkal")'] +runner = { cmd = ["okrun", "{}"], host = "same-arch", exclusive = false } +``` + +| Key | Value | +|---|---| +| `platform` | the platform package of the selected targets, `none`, or a table with `path` / `git` | +| `layers.` | the package that provides `c-abi`, `c++-abi`, `compiler-runtime` or `kernel-abi`, or `false` | +| `tools.` | the package that builds a tool for this machine, or `{ program = "" }` | +| `link` | `driver`, `flags`, `start`, `script`, `inputs`, `exe`, `shared`, `seal` | +| `carrier` | `{ package, section, main }`: the program `--mode self-contained` builds | +| `runner` | an argv, or `{ cmd, host, exclusive }`: `host = "same-arch"` runs an artifact only on a machine of its architecture | + +`cfg(platform = "")` selects by platform; `platform` itself is not +accepted under such a selector. The same keys are read from +`~/.mcpp/config.toml`. A package that is a platform states them under +`[platform]`, and `[host] platform` selects the platform of the build programs. + ### 2.8 `[features]` — Features Moved to [06 — Features and Capabilities](06-features-and-capabilities.md), diff --git a/docs/08-testing.md b/docs/08-testing.md index e6f93e32c..ffbd62f2c 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -107,6 +107,18 @@ test's verdict, the output of a test that failed, the `test result` line and, fo collects the report, and `mcpp test 2>&1 | tee test.log` collects both ([09 — Commands by scenario](09-commands-by-scenario.md#output-streams)). +**A test finds the package's programs by name** (2026.10.27.1+). `mcpp test` builds +the package's programs first, as `mcpp build` does, and gives each test +`MCPP_BIN_`: the path of program ``, upper-cased, every character +other than a letter or a digit written `_` (`MCPP_BIN_MY_TOOL` for `my-tool`). +A test that runs the program it tests reads it from there. + +**A platform package is tested on its targets** (2026.10.27.1+). `mcpp test` in a +package with a `[platform]` section, with no `--target`, builds the tests for +each of the platform's `targets` with the package itself as the platform. The +targets its runner can run on this machine are run; the others are built and +reported as not run ([35](35-platforms.md)). + ### In a workspace ```bash diff --git a/docs/09-commands-by-scenario.md b/docs/09-commands-by-scenario.md index e0eae8f68..9e10d95d7 100644 --- a/docs/09-commands-by-scenario.md +++ b/docs/09-commands-by-scenario.md @@ -115,7 +115,7 @@ which selects the compiler for that invocation and writes nothing: ```bash mcpp test --toolchain llvm@23.1.3 mcpp run --toolchain gcc@16.1.0 -mcpp pack --toolchain llvm@23.1.3 --format dir +mcpp pack --toolchain llvm@23.1.3 --pkg-format dir ``` For that invocation the option takes the place of `[toolchain] default` in @@ -130,6 +130,12 @@ executable has moved or been removed, the next `mcpp build` plans the graph again and writes the record anew; nothing under `target/` needs to be deleted by hand. +`--platform ` and `--host-platform ` select the target's platform +and the build programs' platform for one invocation in the same way +(2026.10.27.1+, [35](35-platforms.md)); `--platform none` builds with the +target's conventions alone, and `mcpp self env --target ` reports the +configuration a build would use. + ## Explaining a resolution `mcpp why` reports what a build would resolve, and builds nothing: diff --git a/docs/21-the-target-triple.md b/docs/21-the-target-triple.md index adc35e630..7c9a7f85c 100644 --- a/docs/21-the-target-triple.md +++ b/docs/21-the-target-triple.md @@ -539,7 +539,7 @@ installed by mcpp · `SDK` the platform's own · `—` unreachable from this hos | `x86_64-windows-musl` | Windows by payload; anywhere by graph | no gcc emits PE+musl, and LLVM cannot spell the triple | | `aarch64-macos` | macOS | the SDK is the machine's | | `*-none-elf` | every host | clang and lld are cross-compilers by construction | -| `*-openkal` | every host | the same; the image is compiled for no operating system and runs on every system through `okrun` ([24](24-openkal-cross.md)) | +| `*-openkal` | every host | the same; the image is compiled for no operating system and runs on every system through `okrun` ([24](24-openkal-cross.md)); the target table selects `openkal-platform` for both rows ([35](35-platforms.md)) | | `wasm32-emscripten`, `*-linux-android` | every host | the SDK ships its own sysroot and upstream publishes it per host; one archive serves every guest arch | **A `—` is about payloads, not about possibility.** `host_can_serve` answers diff --git a/docs/22-target-side.md b/docs/22-target-side.md index 52525b8e9..aa521815e 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -211,6 +211,10 @@ Interface and implementation are separate columns. `openkal` is an interface and `openkal-windows` an implementation of it; collapsing the two would conceal why one source reaches several machines. +The source column reads `graph` for a package the project's graph supplied and +`platform` for one the target's platform added (2026.10.27.1+, +[35](35-platforms.md)); a platform's report follows on a `Platform` line. + ## Closure Visibility The five-layer report above answers "where does each layer come from"; it diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index 7db7130ea..cd13d4800 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -110,15 +110,16 @@ practice. ## What A Project Writes ```toml -[dependencies] -openkal-llvm-runtime = "0.1.1" - -[toolchain] -default = "llvm@23.1.3" +[target.'cfg(any(linux, macos, windows))'] +platform = "openkal-platform@0.17.1" ``` -Two lines. The first selects three layers of the target side; the second names -a compiler and says nothing about where anything else comes from. +One table (2026.10.27.1+). It selects the openkal platform for the host triples, +whose hosted part names the compiler and the packages of three layers of the +target side; the system's openkal implementation provides the fourth +([35 — Platforms](35-platforms.md)). Naming `openkal-llvm-runtime` as a +dependency and `llvm@23.1.3` as `[toolchain] default` states the same by hand, +and remains valid: the platform leaves a layer to the package that provides it. Targets are given on the command line: @@ -145,7 +146,7 @@ preprocessor directive is required in the source. A worked example is | `openkal-musl` | `c-abi` | musl redirected onto openkal, ported once | | `openkal-llvm-runtime` | `compiler-runtime`, `c++-abi` | compiler-rt builtins, libunwind, libc++abi and libc++, configured for openkal-musl | -A project names the last of these. The others follow from its dependencies. +The platform names the last two, or a project names the last; the others follow from their dependencies. ## The Reason The Compiler Must Be LLVM @@ -396,8 +397,9 @@ loader runs on every system of that architecture. `okrun` is that loader on Linux, macOS and Windows. ```toml -[build-dependencies] -openkal-tools = { version = "0.1.0", tools = ["okld", "okrun"] } +[package] +name = "app" +version = "0.1.0" ``` ```sh @@ -405,25 +407,30 @@ mcpp build --target x86_64-openkal # bin/app.oke mcpp run --target x86_64-openkal # okrun bin/app.oke ``` -| | the engine's action | the reason | +Nothing else is declared (2026.10.27.1+): the target table selects +`openkal-platform` for both rows, and the platform states the toolchain, the +layers, the tools, the conventions below and the runner ([35](35-platforms.md)). + +| | the platform's statement | the reason | |---|---|---| | compile | `--target=-unknown-none-elf -fPIC -femulated-tls -fvisibility=hidden -fasynchronous-unwind-tables -Xclang -fstack-clash-protection`; on AArch64 also `-ffixed-x18 -mno-outline-atomics`; `-ffreestanding` when the graph supplies no C library | an image is told no operating system; it does not address the thread pointer and does not use the platform register, which two systems reserve; it exports what its source marks `visibility("default")` | | link | `ld.lld -shared -z max-page-size=65536 -z now --gc-sections --eh-frame-hdr`, entry `ok_main`; an image another loads (`kind = "shared"`) is `lib.okso` with its name | ET_DYN, every segment aligned to the largest protection granularity among the systems; what nothing reaches is discarded, so the requirements note names only the interfaces the program uses | | finish | `okld note` writes the interfaces the image's strong imports require; `okld check` checks every clause against the image and the `.okso` it needs; only then is `bin/.oke` published | a loader refuses an image that requires an interface the implementation lacks, naming it, and an image that breaks the convention is never published | | run | `okrun` is the target's default runner, found among the programs the graph built for this machine, then in declared payloads, then on PATH | the loader holds the openkal implementation; the image carries none | -**The kernel interface is fixed; every layer above it is the graph's.** The -report reads `kernel-abi openkal (the loader)`. The C library and the C++ -run-time are packages that provide `mcpp:c-abi` and `mcpp:c++-abi` for this -target (`openkal-musl`, `openkal-llvm-runtime`), and an image with neither is -legal: it defines `ok_main` itself and supplies `__emutls_get_address`. The +**The kernel interface is fixed; every layer above it is a package.** The C +library and the C++ run-time are the packages the platform names +(`openkal-musl`, `openkal-llvm-runtime`), reported `platform`; a package of the +graph that provides the layer takes the platform's place. An image with neither +is legal: `[target.'cfg(platform = "openkal")'.layers]` switches them off, and +the image defines `ok_main` itself and supplies `__emutls_get_address`. The payload's libc++ is not offered, because it was configured for the payload's own system; `import std;` without a C++ run-time in the graph is refused before -anything compiles, naming the packages. +anything compiles, saying where the layer was switched off. **Missing `okld` is a refusal, not a warning.** An image linked without its -requirements note would be refused by every loader; the engine names the -dependency to add instead. +requirements note would be refused by every loader; the refusal +(`platform-tool-absent`) names the package that builds it. ## Bare Metal diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 29f13264a..bd8d87a14 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -181,11 +181,16 @@ int main() { | `mcpp::runtime_search_dir(dir)` *(protocol 12)* | `mcpp:runtime-search-dir=` — see below | | `mcpp::link_script(p)` *(2026.8.19+)* | `mcpp:link-script=` | | `mcpp::link_start(symbol)` *(protocol 16)* | `mcpp:link-start=` | +| `mcpp::link_driver(d)` *(protocol 17)* | `mcpp:link-driver=` — `"lld"` (`ld.lld` itself links) or `"compiler"`; a platform's `link.driver` ([35](35-platforms.md)) | +| `mcpp::link_input(path)` / `mcpp::link_input_from(as, token)` *(protocol 17)* | `mcpp:link-input=` — a file the link takes, or the output of a program saved as `as` (one token per call); `link.inputs` | +| `mcpp::artifact_name(kind, pattern)` *(protocol 17)* | `mcpp:artifact-name=` — `"exe"` or `"shared"` and a pattern with `{name}`; `link.exe`, `link.shared` | +| `mcpp::seal(step, token)` *(protocol 17)* | `mcpp:seal=` — one token of seal step `step`, run on the linked file (`{}`) before it is published; `link.seal` | +| `mcpp::run_host(h)` *(protocol 17)* | `mcpp:run-host=` — `"same-arch"` or `"any"`; `runner.host` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` — see below | | `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | the payload directory of a package declared in `[xlings.workspace]` — by this manifest, or by a dependency compiled into this build program *(2026.9.6.6+)*; `""` when it was not declared or is not installed (see below) | | `mcpp::xpkg_source(ns, name)` / `mcpp::xpkg_program(ns, name)` *(protocol 15)* | where that payload comes from — `payload`, `override`, `pending`, or `""` — and the program an override named | | `mcpp::xpkg_request(ns, name)` / `mcpp::xpkg_pending()` *(protocol 15)* | the directory of a payload declared `provision = "on-request"`, asking for it when it is not installed yet; `xpkg_pending()` is then true and the program should return (see below) | -| `mcpp::phase()` *(protocol 15)* | `"toolchain"` while a root program states the build toolchain, `"build"` otherwise | +| `mcpp::phase()` *(protocol 15)* | `"toolchain"` while a root program states the build toolchain, `"platform"` while a platform's `plugin` runs (protocol 17), `"build"` otherwise | | `mcpp::decision(subject, from, value, file, line, payload)` *(protocol 15)* | `mcpp:decision=` — record where a tool came from. `mcpp.plugins.tool` states it for every member | | `mcpp::toolchain(key, value)` *(protocol 15)* | `mcpp:toolchain=` — state the build toolchain in the toolchain phase. `mcpp.plugins.toolchain` builds the statement | | `mcpp::warning(text)` *(2026.8.21.2+)* | `mcpp:warning=` — see below | @@ -300,7 +305,7 @@ neither one's. whichever library of the program makes the call. A program that chooses between two runners chooses before it emits either. A runner named after a pack format reaches that format's distributable under `mcpp run ---format ` ([41 — Reaching a Device](41-devices.md)). +--pkg-format ` ([41 — Reaching a Device](41-devices.md)). ### Asking instead of declaring: `toolchain_dir` / `sysroot_dir` (2026.8.19.4+) @@ -980,7 +985,7 @@ usual work and all of them are `artifact`: each consumes **link outputs** and produces something a user installs. What the engine adds for them is a mechanism and no format at all. -`mcpp pack --format ` resolves `` through the resolved graph, the +`mcpp pack --pkg-format ` resolves `` through the resolved graph, the same way `--target` reaches a triple the engine did not have to know individually. `tar` and `dir` remain the archive shapes `mcpp pack` owns; everything past them comes from a package. @@ -1012,7 +1017,7 @@ int main() { ``` **Declare unconditionally, submit conditionally.** The declaration is what lets -the engine answer a question the requesting build cannot: `--format bogus` +the engine answer a question the requesting build cannot: `--pkg-format bogus` names what *is* available, and `--help` says "any format the resolved graph provides". Both read the set collected from a pass that asked for nothing. A member that declared only when asked still works for its author — they always @@ -1020,7 +1025,7 @@ pass their own format — and makes the set unknowable for everyone else. mcpp refuses a format nothing submitted for, rather than reporting a pack that produced no package. -**`mcpp pack --format ` prepares twice.** An `artifact` action is a ninja +**`mcpp pack --pkg-format ` prepares twice.** An `artifact` action is a ninja edge and the staged tree is produced by mcpp *after* the link, so the tree cannot be an input of the pass that built it. The first pass collects the declarations and refuses an unknown format before anything is compiled; the @@ -1036,7 +1041,7 @@ strip policy, the debug-symbol split and `include`/`exclude`. An AppImage, a (`.deb`, `.rpm`) owns the re-layout, because which directory a file belongs in is that format's knowledge and not the engine's. -`mcpp pack --format dir` writes the same tree to a path and stops, which is how +`mcpp pack --pkg-format dir` writes the same tree to a path and stops, which is how a person inspects what a member will be handed. **`mcpp pack` reports the artifact actions the request introduced.** An action @@ -1061,7 +1066,7 @@ reported as up to date. files before it resolves the artifact's dependency closure, so the tree can exist without a complete one: a closure with a name that resolves to no file the tree can carry (2026.9.14.2+), or a non-PE, non-Mach-O artifact packed from -a Windows host. `--format tar` and `--format dir` fail the command in that +a Windows host. `--pkg-format tar` and `--pkg-format dir` fail the command in that case, since the archive IS the closure; a dispatched format receives the tree regardless, with a second manifest line, `reason = `. The `needs` lines that follow the header state each needed name and what satisfies it @@ -1085,7 +1090,7 @@ none to rely on), and the only interpolations are a closed set: | `${mcpp.bin_dir}` | where produced binaries land: `bin/`, or, for an action a workspace member's build program declares, that member's product directory (2026.9.29.4+) | | `${mcpp.compile_db}` | path to `compile_commands.json` (what clang-tidy's `-p` wants) | | `${mcpp.target_file:}` | the built file of target `` | -| `${mcpp.stage_dir}` *(2026.9.11.1+)* | the tree `mcpp pack` staged, absolute. `artifact` role only, and only under `mcpp pack --format ` | +| `${mcpp.stage_dir}` *(2026.9.11.1+)* | the tree `mcpp pack` staged, absolute. `artifact` role only, and only under `mcpp pack --pkg-format ` | | `${mcpp.self}` *(2026.9.13.1+)* | the engine's own executable, absolute. An action's command is an argv with no shell, so a build program has no portable way to copy a file; the engine is present wherever a build runs, and `${mcpp.self} stage --verify content --output ` copies one file, creates the destination's parent, and writes only when the bytes differ. That argument shape is a contract from 2026.9.13.1 on; a build program that names it declares that release as its floor | Every path a `${mcpp.*}` token expands to is spelled with **forward slashes**. @@ -1346,7 +1351,7 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_PKG_AUTHORS` *(2026.9.11.1+)* | `mcpp::package_authors()` | The `[package] authors`, joined with `;`. Not `,`: an author entry is conventionally `Name ` and a name may carry a comma, so a comma-joined list cannot be split back into the entries it was made from | | `MCPP_PKG_REPO` *(2026.9.11.1+)* | `mcpp::package_repo()` | The `[package] repo` | | `MCPP_TARGET_MIN_PLATFORM_VERSION` *(2026.9.12.3+)* | `mcpp::min_platform_version()` | The project's floor for this triple, in the platform's own words: on macOS, `[build] macos_deployment_target` or the engine's own default `14.0`; on iOS, `[build] ios_deployment_target` verbatim, empty when the project states none; on `*-linux-android`, `[target.] min_api_level`, or a fallback the resolved NDK payload states; empty on every other row. The value the effective triple carries, and part of the re-run key | -| `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | The `--format` value of the `mcpp pack` pass this program is part of; empty for every ordinary build. The empty value is the one that carries the meaning — a member gates its submission on this, so `mcpp build` has the graph it always had | +| `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | The `--pkg-format` value of the `mcpp pack` pass this program is part of; empty for every ordinary build. The empty value is the one that carries the meaning — a member gates its submission on this, so `mcpp build` has the graph it always had | | `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | Where `mcpp pack` has already staged the closure, absolute; empty when this build is not packaging. Read it to decide the shape of the work; write `${mcpp.stage_dir}` into the action, so the path in the graph and the path the program read cannot disagree | | `MCPP_PACK_STRIP` *(2026.9.16.1+)* | `mcpp::pack_strip()` | `1` when the `mcpp pack` pass this program is part of strips, `0` under `--no-strip`; empty for every ordinary build. A member that stages libraries of its own follows it, so one switch governs every file in the package | | `MCPP_PACK_DEBUG_SYMBOLS_DIR` *(2026.9.16.1+)* | `mcpp::pack_debug_symbols_dir()` | Where `--debug-symbols` sends the separated `*.debug` files, absolute; empty when they are discarded or the build is not packing | @@ -1501,7 +1506,7 @@ the content is unchanged, so calling it unconditionally costs no rebuild. A `dist-*` member fits neither definition. It does not compile a translation unit and it does not run while the build program runs: it consumes link outputs and produces something a user installs, through an `artifact` action and -`mcpp pack --format `. The prefix matters because the taxonomy is +`mcpp pack --pkg-format `. The prefix matters because the taxonomy is load-bearing — a consumer reading `rules-wix` would expect a compiler and a translation unit, and there is neither. diff --git a/docs/31-authoring-a-rule-package.md b/docs/31-authoring-a-rule-package.md index 646513fd4..3db8d3b43 100644 --- a/docs/31-authoring-a-rule-package.md +++ b/docs/31-authoring-a-rule-package.md @@ -346,7 +346,7 @@ A `dist-*` member is neither a rule nor a tool. It does not compile a translation unit and it does not do its work while the build program runs: it consumes **link outputs** and produces something a user installs — an `.msi`, a `.deb`, an AppImage, a signed `.app`. The mechanism is an `artifact` action and -`mcpp pack --format `, documented in +`mcpp pack --pkg-format `, documented in [30 — Producing a distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111). Everything in this chapter applies unchanged. Six things bind harder here, and diff --git a/docs/34-authoring-a-bsp.md b/docs/34-authoring-a-bsp.md index 3bd11f6df..f6cb5a942 100644 --- a/docs/34-authoring-a-bsp.md +++ b/docs/34-authoring-a-bsp.md @@ -125,6 +125,31 @@ something else, and neither has to be known to mcpp. cannot overlap, and `mcpp test` then serialises them. A project never has to remember `-j1`. +## The board as a platform package + +The fields a board's build program states — the link script, the entry, the +runner and the named runners — are platform fields (2026.10.27.1+, +[35 — Platforms](35-platforms.md)). A BSP may state the fixed ones in its +manifest instead and keep the build program for what depends on the selected +feature: + +```toml +[platform] +targets = ["thumbv7em-none-eabihf"] +runner = ["qemu-system-arm", "-machine", "mps2-an386", "-nographic", "-kernel", "{}"] +runners = { flash = ["probe-rs", "download", "--chip", "STM32L475VG", "{}"] } + +[platform.link] +script = "cortex-m.ld" +start = "Reset_Handler" +``` + +A project then selects the board as its target's platform +(`[target.thumbv7em-none-eabihf] platform = "@"`) rather than as +a dependency. Both forms merge into one statement: a value the build program +states replaces the manifest's, and a project's `[target.]` keys replace +both. + ## Current limitations - A BSP declares its emulator and its probe driver as payloads, so a board diff --git a/docs/41-devices.md b/docs/41-devices.md index 9328f7c85..4dcbb2672 100644 --- a/docs/41-devices.md +++ b/docs/41-devices.md @@ -94,6 +94,22 @@ a dependency supplied, and the override is reported rather than applied in silence. Exactly one dependency may supply a given name; a second is an error naming both packages. +A runner may also be a table (2026.10.27.1+), which states where the artifact +runs and whether runs overlap: + +```toml +[target.'cfg(platform = "openkal")'] +runner = { cmd = ["okrun", "{}"], host = "same-arch", exclusive = false } +``` + +`host = "same-arch"` runs an artifact only on a machine of its architecture, and +an artifact of another architecture is refused naming `run_under`; `"any"` is a +runner that reaches the artifact from any machine. A target's platform supplies +its default runner ([35](35-platforms.md)); a dependency's build program, this +machine's `config.toml`, the project and `MCPP_TARGET__RUNNER` each +replace it in that order, and an argv alone keeps the platform's `host` and +`exclusive`. + ## What a runner receives A runner receives an argv — its own tokens, with the artifact's path appended @@ -115,7 +131,7 @@ The first field is the destination relative to the artifact's directory, with `/` separators; it begins with `../` for a test program discovered in a subdirectory. The second is the absolute path of the file in the output tree. A TAB separates them. The file exists for every runner invocation and is empty -when there is nothing to carry; a distributable run with `--format` receives +when there is nothing to carry; a distributable run with `--pkg-format` receives an empty file, because the distributable holds its own files. A runner that executes the artifact where it lies ignores the variable. A @@ -124,7 +140,7 @@ copies each listed file to the destination beside the moved artifact. ## Running a distributable -`mcpp run --format ` packs the distributable `` and runs it. Without +`mcpp run --pkg-format ` packs the distributable `` and runs it. Without `--runner`, it is reached by the runner named `` when the graph or the manifest supplies one, and by the default runner otherwise: @@ -133,12 +149,12 @@ mcpp::runner("app", ""); // a package names a runner after its fo ``` ```bash -mcpp run --format app # through the runner named `app` -mcpp run --format app --runner other # a typed --runner still wins +mcpp run --pkg-format app # through the runner named `app` +mcpp run --pkg-format app --runner other # a typed --runner still wins ``` A distributable that is a directory, such as an application bundle, is not -executed directly. When no runner reaches it, `mcpp run --format ` refuses +executed directly. When no runner reaches it, `mcpp run --pkg-format ` refuses before starting anything, with exit status `126`, and names the runner `` and the manifest table that would declare it. diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index d6848ee66..119d4679f 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -251,9 +251,9 @@ main = "src/main.cpp" 接受 `app` 的方式也与接受 `bin` 完全相同。 在其形式为共享库的那一行上，`mcpp run` 一个 `app` 目标，若不带 -`--format` 会拒绝执行，并点名这个旗标与所解析出的图提供的那些格式。 -`mcpp pack --format apk` 会把这个库放到闭包已经放置共享对象的位置。见 -[10 —— 打包与发布](10-pack-and-release.md)中的 `mcpp run --format`。 +`--pkg-format` 会拒绝执行，并点名这个旗标与所解析出的图提供的那些格式。 +`mcpp pack --pkg-format apk` 会把这个库放到闭包已经放置共享对象的位置。见 +[10 —— 打包与发布](10-pack-and-release.md)中的 `mcpp run --pkg-format`。 早于 2026.9.12.3 的引擎不认识这个取值，会拒绝并点名它认识的三种： @@ -1279,10 +1279,8 @@ argv（OpenSBI 启动用 `-bios default`，picolibc 镜像用 `-bios none -semihosting`）——一个替某一块板子猜一个值的引擎，就是另一块板子必须 与之对抗的引擎。一个板级支持包通常会提供它。 -唯一一个 runner 属于 target 而非板子的 target 是 `-openkal` -（2026.10.26.2）：ok 映像在每个系统上都由 `okrun` 运行，因此工程未声明 runner -时，`okrun` 就是该 target 的 runner。它的查找方式与声明的 runner 程序相同，并 -优先查找依赖图为本机构建的程序（`tools = [...]`）（[24](24-openkal-cross.md)）。 +由平台配置的目标在工程未声明 runner 时使用平台的 runner：`-openkal` 在每个系统上都由 `okrun` 运行 +（2026.10.27.1+，[35](35-platforms.md)）。它的查找方式与声明的 runner 程序相同，并优先查找依赖图为本机构建的程序。 ### 2.7.3 hosted 目标上的 `runner`（2026.9.2.1+） @@ -1376,6 +1374,33 @@ runner 与 `run_under`。 以及陈述了基座时的 `MCPP_SYSROOT`;其退出码即程序的退出码。非 macOS 宿主上的 Apple 产物在没有 `run_under` 时于执行前被拒绝,拒绝信息点名本键。 +#### 2.7.3.3 平台键（2026.10.27.1+） + +`[target.]` 还以平台自己的键名选择并覆盖目标的**平台**（[35 —— 平台](35-platforms.md)）： + +```toml +[target.'cfg(os = "openkal")'] +platform = "openkal-platform@0.17.1" # or { path = "…" }, or "none" + +[target.'cfg(platform = "openkal")'.layers] +c-abi = { path = "../openkal-musl" } + +[target.'cfg(platform = "openkal")'] +runner = { cmd = ["okrun", "{}"], host = "same-arch", exclusive = false } +``` + +| 键 | 取值 | +|---|---| +| `platform` | 所选目标的平台包、`none`，或带 `path` / `git` 的表 | +| `layers.` | 提供 `c-abi`、`c++-abi`、`compiler-runtime` 或 `kernel-abi` 的包，或 `false` | +| `tools.` | 为本机构建一个工具的包，或 `{ program = "" }` | +| `link` | `driver`、`flags`、`start`、`script`、`inputs`、`exe`、`shared`、`seal` | +| `carrier` | `{ package, section, main }`：`--mode self-contained` 构建的程序 | +| `runner` | 一个 argv，或 `{ cmd, host, exclusive }`：`host = "same-arch"` 只在产物架构的机器上运行它 | + +`cfg(platform = "")` 按平台选择；`platform` 本身不能写在这样的选择器下。同样的键也从 +`~/.mcpp/config.toml` 读取。平台包在 `[platform]` 下陈述它们，`[host] platform` 选择构建程序的平台。 + ### 2.8 `[features]` —— Feature 已移至 [06 —— Feature 与能力](06-features-and-capabilities.md)，连同 diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index e309fdd3a..1ea4a9e18 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -99,6 +99,14 @@ mcpp test -- --verbose # everything after `--` goes to each test binary `mcpp test 2>&1 | tee test.log` 两者都收集 （见 [09 —— 按场景选命令](09-commands-by-scenario.md#输出流)）。 +**测试按名字找到包的程序**（2026.10.27.1+）。`mcpp test` 先按 `mcpp build` 的方式构建包的程序，并给每个测试 +`MCPP_BIN_`：程序 `` 的路径，名字转为大写，字母与数字之外的字符写作 `_`（`my-tool` 为 +`MCPP_BIN_MY_TOOL`）。运行被测程序的测试从这里读取它。 + +**平台包在它的目标上测试**（2026.10.27.1+）。在带 `[platform]` 段的包中执行 `mcpp test` 而不给 `--target` 时， +以该包本身为平台，为平台的每个 `targets` 构建测试。其 runner 能在本机运行的目标被运行；其余已构建、报告为未运行 +（[35](35-platforms.md)）。 + ### 在工作空间中 ```bash diff --git a/docs/zh/09-commands-by-scenario.md b/docs/zh/09-commands-by-scenario.md index 88e8df6ae..b1322706e 100644 --- a/docs/zh/09-commands-by-scenario.md +++ b/docs/zh/09-commands-by-scenario.md @@ -105,7 +105,7 @@ $ mcpp search imgui ```bash mcpp test --toolchain llvm@23.1.3 mcpp run --toolchain gcc@16.1.0 -mcpp pack --toolchain llvm@23.1.3 --format dir +mcpp pack --toolchain llvm@23.1.3 --pkg-format dir ``` 对这一次调用，这个选项取代 `mcpp.toml` 中的 `[toolchain] default`，优先级 @@ -113,6 +113,10 @@ mcpp pack --toolchain llvm@23.1.3 --format dir 每个工具链构建到它自己的输出目录，一次记录下来的构建只为记录它的那个工具链 请求重放。 +`--platform ` 与 `--host-platform ` 以同样方式为一次调用选择目标的平台与构建程序的平台 +（2026.10.27.1+，[35](35-platforms.md)）；`--platform none` 只以目标的约定构建， +`mcpp self env --target ` 报告一次构建将使用的配置。 + ## 解释一次解析 `mcpp why` 报告一次构建会解析出什么，并且不构建任何东西： diff --git a/docs/zh/21-the-target-triple.md b/docs/zh/21-the-target-triple.md index 62b5334eb..891b687f7 100644 --- a/docs/zh/21-the-target-triple.md +++ b/docs/zh/21-the-target-triple.md @@ -507,7 +507,7 @@ system · `系统` 在机器上被找到，不是 mcpp 安装的 · `SDK` 平台 | `x86_64-windows-musl` | payload 只在 Windows 上；图则任意宿主 | 没有 gcc 能发出 PE + musl，且 LLVM 拼不出这个三元组 | | `aarch64-macos` | macOS | SDK 是那台机器自己的 | | `*-none-elf` | 每一台宿主 | clang 与 lld 按构造就是交叉编译器 | -| `*-openkal` | 每一台宿主 | 同上；映像为“无操作系统”编译，经 `okrun` 在每个系统上运行（[24](24-openkal-cross.md)） | +| `*-openkal` | 每一台宿主 | 同上；映像为“无操作系统”编译，经 `okrun` 在每个系统上运行（[24](24-openkal-cross.md)）；目标表为两行选择 `openkal-platform`（[35](35-platforms.md)） | | `wasm32-emscripten`、`*-linux-android` | 每一台宿主 | SDK 自带 sysroot，且上游按宿主分别发布；一份归档服务每个 guest 架构 | **一个 `—` 讲的是 payload，不是可能性。** `host_can_serve` 回答的是 diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index b4337f3b4..9b48cd2ce 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -178,6 +178,9 @@ openkal-llvm-runtime = "0.1" 接口与实现是两列分开写的。`openkal` 是一个接口，`openkal-windows` 是它的一个 实现；把二者合并会掩盖一份源码为何能够到达多台机器。 +来源一列对工程依赖图提供的包写作 `graph`，对目标的平台加入的包写作 `platform`（2026.10.27.1+， +[35](35-platforms.md)）；平台的报告随后在 `Platform` 一行给出。 + ## 闭包可见性 上面这份五层报告回答的是「每一层来自哪里」，不回答图中「还有哪些别的包」—— diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index 97498cdd8..b14747061 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -92,15 +92,13 @@ SDK，或者区分 `linux`/`windows`/`macos`。那些是 C 环境层或平台层 ## 工程要写的内容 ```toml -[dependencies] -openkal-llvm-runtime = "0.1.1" - -[toolchain] -default = "llvm@23.1.3" +[target.'cfg(any(linux, macos, windows))'] +platform = "openkal-platform@0.17.1" ``` -两行。第一行选定目标侧的三个层；第二行命名一个编译器，并且对其余一切来自 -何处只字未提。 +一张表（2026.10.27.1+）。它为宿主三元组选择 openkal 平台，平台的宿主部分命名编译器与目标侧三个层的包； +第四层由系统自己的 openkal 实现提供（[35 —— 平台](35-platforms.md)）。把 `openkal-llvm-runtime` 写成依赖、 +把 `llvm@23.1.3` 写成 `[toolchain] default` 是同一内容的手写形式，仍然有效：平台把一个层让给提供它的包。 target 在命令行上给出： @@ -126,7 +124,7 @@ mcpp build --target x86_64-windows-gnu | `openkal-musl` | `c-abi` | 被重定向到 openkal 上的 musl，只移植过一次 | | `openkal-llvm-runtime` | `compiler-runtime`、`c++-abi` | compiler-rt 内建函数、libunwind、libc++abi 与 libc++，为 openkal-musl 配置 | -一个工程命名其中最后一个，其余的由它的依赖推出。 +平台命名其中最后两个，或者工程命名最后一个；其余的由它们的依赖推出。 ## 编译器必须是 LLVM 的原因 @@ -350,8 +348,9 @@ wasm-ld: error: obj/main.o: undefined symbol: kal_process_spawn 由加载器在该架构的每个系统上运行。`okrun` 是 Linux、macOS 与 Windows 上的加载器。 ```toml -[build-dependencies] -openkal-tools = { version = "0.1.0", tools = ["okld", "okrun"] } +[package] +name = "app" +version = "0.1.0" ``` ```sh @@ -359,22 +358,24 @@ mcpp build --target x86_64-openkal # bin/app.oke mcpp run --target x86_64-openkal # okrun bin/app.oke ``` -| | mcpp 的处理 | 原因 | +无需其他声明（2026.10.27.1+）：目标表为两行选择 `openkal-platform`，由平台陈述工具链、各层、工具、下表的约定 +与 runner（[35](35-platforms.md)）。 + +| | 平台的陈述 | 原因 | |---|---|---| | 编译 | `--target=-unknown-none-elf -fPIC -femulated-tls -fvisibility=hidden -fasynchronous-unwind-tables -Xclang -fstack-clash-protection`；AArch64 另加 `-ffixed-x18 -mno-outline-atomics`；依赖图不提供 C 库时加 `-ffreestanding` | 映像按“无操作系统”编译；不访问线程指针，不使用两个系统保留的平台寄存器；只导出源码标注 `visibility("default")` 的名字 | | 链接 | `ld.lld -shared -z max-page-size=65536 -z now --gc-sections --eh-frame-hdr`，入口 `ok_main`；被其他映像加载的映像（`kind = "shared"`）为带名字的 `lib.okso` | ET_DYN，每个段按各系统中最大的保护粒度对齐；丢弃不可达的代码，需求 note 只列出程序实际使用的接口 | | 收尾 | `okld note` 由映像的强导入写入所需接口；`okld check` 对映像及其所需的 `.okso` 检查每一条约定；通过后才发布 `bin/.oke` | 加载器拒绝需要实现所缺接口的映像并指名该接口；违反约定的映像不会被发布 | | 运行 | `okrun` 是该 target 的默认 runner，依次在依赖图为本机构建的程序、声明的载荷、PATH 中查找 | 加载器持有 openkal 实现，映像不携带实现 | -**内核接口固定，其上每一层来自依赖图。** 报告显示 `kernel-abi openkal (the -loader)`。C 库与 C++ 运行时是为该 target 提供 `mcpp:c-abi`、`mcpp:c++-abi` 的包 -（`openkal-musl`、`openkal-llvm-runtime`）；两者都没有的映像也合法：它自行定义 -`ok_main` 并提供 `__emutls_get_address`。载荷的 libc++ 为载荷自身的系统配置，不 -提供给映像；依赖图中没有 C++ 运行时时，`import std;` 在编译之前被拒绝，并指名 -提供它的包。 +**内核接口固定，其上每一层都是一个包。** C 库与 C++ 运行时是平台命名的包（`openkal-musl`、 +`openkal-llvm-runtime`），报告为 `platform`；依赖图中提供该层的包取代平台的包。两者都没有的映像也合法： +`[target.'cfg(platform = "openkal")'.layers]` 关闭它们，映像自行定义 `ok_main` 并提供 +`__emutls_get_address`。载荷的 libc++ 为载荷自身的系统配置，不提供给映像；依赖图中没有 C++ 运行时时， +`import std;` 在编译之前被拒绝，并指出该层在何处被关闭。 -**缺少 `okld` 是拒绝而非警告。** 没有需求 note 的映像会被每个加载器拒绝；引擎 -转而指出应添加的依赖。 +**缺少 `okld` 是拒绝而非警告。** 没有需求 note 的映像会被每个加载器拒绝；拒绝（`platform-tool-absent`） +指名构建它的包。 ## 裸机 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 62e950cc7..b6057e40d 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -165,11 +165,16 @@ int main() { | `mcpp::runtime_search_dir(dir)` *(protocol 12)* | `mcpp:runtime-search-dir=` —— 见下 | | `mcpp::link_script(p)` *(2026.8.19+)* | `mcpp:link-script=` | | `mcpp::link_start(symbol)` *(协议 16)* | `mcpp:link-start=` | +| `mcpp::link_driver(d)` *(协议 17)* | `mcpp:link-driver=` —— `"lld"`（由 `ld.lld` 本身链接）或 `"compiler"`；即平台的 `link.driver`（[35](35-platforms.md)） | +| `mcpp::link_input(path)` / `mcpp::link_input_from(as, token)` *(协议 17)* | `mcpp:link-input=` —— 链接所取的文件，或一个程序的输出、保存为 `as`（每次调用一个 token）；即 `link.inputs` | +| `mcpp::artifact_name(kind, pattern)` *(协议 17)* | `mcpp:artifact-name=` —— `"exe"` 或 `"shared"` 与带 `{name}` 的模式；即 `link.exe`、`link.shared` | +| `mcpp::seal(step, token)` *(协议 17)* | `mcpp:seal=` —— 封装步骤 `step` 的一个 token，在链接出的文件（`{}`）发布之前运行；即 `link.seal` | +| `mcpp::run_host(h)` *(协议 17)* | `mcpp:run-host=` —— `"same-arch"` 或 `"any"`；即 `runner.host` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` —— 见下 | | `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | `[xlings.workspace]` 里声明的包的载荷目录 —— 本 manifest 声明的，或编进本构建程序的某个依赖声明的（2026.9.6.6+）；没声明或没安装时返回 `""`（见下） | | `mcpp::xpkg_source(ns, name)` / `mcpp::xpkg_program(ns, name)` *(协议 15)* | 该载荷从哪里来 —— `payload`、`override`、`pending` 或 `""` —— 以及一条覆盖点名的程序 | | `mcpp::xpkg_request(ns, name)` / `mcpp::xpkg_pending()` *(协议 15)* | 一个声明了 `provision = "on-request"` 的载荷的目录；尚未安装时请求它，此时 `xpkg_pending()` 为真，程序应当返回（见下） | -| `mcpp::phase()` *(协议 15)* | 根程序陈述构建工具链时为 `"toolchain"`，其余为 `"build"` | +| `mcpp::phase()` *(协议 15)* | 根程序陈述构建工具链时为 `"toolchain"`，平台的 `plugin` 运行时为 `"platform"`（协议 17），其余为 `"build"` | | `mcpp::decision(subject, from, value, file, line, payload)` *(协议 15)* | `mcpp:decision=` —— 记录一个工具从哪里来。`mcpp.plugins.tool` 为每个成员陈述它 | | `mcpp::toolchain(key, value)` *(协议 15)* | `mcpp:toolchain=` —— 在工具链阶段陈述构建工具链。`mcpp.plugins.toolchain` 构造该陈述 | | `mcpp::warning(text)` *(2026.8.21.2+)* | `mcpp:warning=` —— 见下 | @@ -263,7 +268,7 @@ for (auto a : {"-machine","virt","-nographic","-no-reboot","-kernel"}) 调用追加到默认 runner，每次 `mcpp::runner(name, tok)` 调用追加到名为 `name` 的 runner，按发出顺序，无论调用来自这个程序的哪个库。要在两个 runner 之间选择的程序，在发出任何一个之前完成选择。以打包格式命名的 runner 在 -`mcpp run --format ` 下抵达该格式的可分发物（[41 —— 抵达一台设备](41-devices.md)）。 +`mcpp run --pkg-format ` 下抵达该格式的可分发物（[41 —— 抵达一台设备](41-devices.md)）。 ### 问，而不是声明：`toolchain_dir` / `sysroot_dir`（2026.8.19.4+） @@ -832,7 +837,7 @@ int main() { 惯常活计，而它们全都是 `artifact`：每一个都消费**链接产物**，产出用户去安装的东西。 引擎为它们加的是一套机制，而不是任何一种格式。 -`mcpp pack --format ` 把 `` 交给解析后的图去解决，方式与 `--target` +`mcpp pack --pkg-format ` 把 `` 交给解析后的图去解决，方式与 `--target` 够到一个引擎不必逐个认识的三元组相同。`tar` 与 `dir` 仍然是 `mcpp pack` 自己拥有的 归档形状；此外的一切都来自某个包。 @@ -863,13 +868,13 @@ int main() { ``` **无条件声明，有条件提交。** 声明是让引擎能回答一个发起请求的那次构建自己回答不了的 -问题：`--format bogus` 要点名**当下确实可用**的那些格式，`--help` 要说「解析后的图 +问题：`--pkg-format bogus` 要点名**当下确实可用**的那些格式，`--help` 要说「解析后的图 提供的任何格式」。两者读的都是一次「什么格式都没要」的 pass 收集到的集合。一个只在被 问到时才声明的成员，对它的作者仍然照常工作 —— 作者永远传的是自己那个格式 —— 而对其他 所有人，这个集合变成不可知的。对一个谁都没为之提交的格式，mcpp 会拒绝，而不是报告一次 「什么包都没产出」的成功打包。 -**`mcpp pack --format ` 会 prepare 两次。** 一条 `artifact` action 是一条 +**`mcpp pack --pkg-format ` 会 prepare 两次。** 一条 `artifact` action 是一条 ninja 边，而那棵暂存树是 mcpp 在链接**之后**产出的，所以这棵树不可能成为构建出它自己 那一次 pass 的输入。第一次 pass 收集声明，并在任何东西被编译之前拒绝未知的格式；随后 才是构建与暂存；第二次 pass 设上 `pack_format` 与 `pack_stage_dir`，并构建提供方提交 @@ -882,7 +887,7 @@ pass 和那次暂存已经回答过的。 而要一棵 FHS 树的格式（`.deb`、`.rpm`）自己负责重排布局，因为一个文件该落在哪个目录是 那个格式的知识，不是引擎的。 -`mcpp pack --format dir` 把同一棵树写到一个路径上就停下，这是人去查看一个成员将会拿到 +`mcpp pack --pkg-format dir` 把同一棵树写到一个路径上就停下，这是人去查看一个成员将会拿到 什么的方式。 **`mcpp pack` 报告的是这次请求**引入**的那些 artifact action。** 一条无论有没有人 @@ -901,8 +906,8 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件，永不 **这份 manifest 的第一行是 `closure = walked` 或 `closure = not-walked`。** `mcpp pack` 会先暂存程序本身与它声明过的运行期文件，再解析该产物的依赖闭包 —— 所以这棵 树可以在闭包不完整的情况下存在：闭包里有一个名字解析不到树能携带的文件（2026.9.14.2+）, -或者在 Windows 宿主上打包一个既非 PE 也非 Mach-O 的产物。`--format tar` 与 -`--format dir` 在这种情况下让命令失败，因为归档本身就是闭包；一个被分发出去的格式无论如何 +或者在 Windows 宿主上打包一个既非 PE 也非 Mach-O 的产物。`--pkg-format tar` 与 +`--pkg-format dir` 在这种情况下让命令失败，因为归档本身就是闭包；一个被分发出去的格式无论如何 都会拿到这棵树，manifest 上多出第二行 `reason = <原因>`。头部之后的 `needs` 行给出每个 被需要的名字以及满足它的东西（[50 —— 机器输出](50-machine-output.md)），自行放置库的 提供方读这些行，而不是从 `lib/` 里推断闭包。 @@ -921,7 +926,7 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件，永不 | `${mcpp.bin_dir}` | 产出的二进制所在目录:`bin/`;工作区成员的构建程序声明的 action 中为该成员的产物目录(2026.9.29.4+) | | `${mcpp.compile_db}` | `compile_commands.json` 的路径（clang-tidy 的 `-p` 要的就是它） | | `${mcpp.target_file:}` | target `` 构建出的文件 | -| `${mcpp.stage_dir}` *(2026.9.11.1+)* | `mcpp pack` 暂存出的那棵树，绝对路径。仅 `artifact` role 可用，且仅在 `mcpp pack --format ` 下可用 | +| `${mcpp.stage_dir}` *(2026.9.11.1+)* | `mcpp pack` 暂存出的那棵树，绝对路径。仅 `artifact` role 可用，且仅在 `mcpp pack --pkg-format ` 下可用 | | `${mcpp.self}` *(2026.9.13.1+)* | 引擎自己的可执行文件，绝对路径。action 的命令是没有 shell 的 argv，构建程序因此没有可移植的拷贝手段；引擎在构建运行的每台机器上都在，`${mcpp.self} stage --verify content --output ` 拷贝一个文件、创建目标的父目录、只在字节不同时写入。这个参数形状自 2026.9.13.1 起是契约；写下它的构建程序即以该版本为下限 | `${mcpp.*}` 展开出的每个路径都用**正斜杠**拼写。token 会被拼进构建程序写下的 @@ -1148,7 +1153,7 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来，缓存 | `MCPP_PKG_AUTHORS` *(2026.9.11.1+)* | `mcpp::package_authors()` | `[package] authors`，以 `;` 连接。不用 `,`：一条 author 的惯例写法是 `Name `，名字里可能带逗号，以逗号连接的列表无法再切回原来的条目 | | `MCPP_PKG_REPO` *(2026.9.11.1+)* | `mcpp::package_repo()` | `[package] repo` | | `MCPP_TARGET_MIN_PLATFORM_VERSION` *(2026.9.12.3+)* | `mcpp::min_platform_version()` | 项目对这个三元组的下限，用平台自己的措辞：macOS 上是 `[build] macos_deployment_target` 或引擎自带的默认值 `14.0`；iOS 上是 `[build] ios_deployment_target` 原样给出，项目没写就是空；`*-linux-android` 上是 `[target.] min_api_level`，或已解析 NDK 载荷给出的回落值；其余每一行都是空。取的是有效三元组携带的那个值，并进入重跑键 | -| `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | 本程序所处的这次 `mcpp pack` 的 `--format` 取值；任何普通构建下都为空。承载含义的正是这个空值 —— 成员据此为自己的提交加闸，于是 `mcpp build` 拿到的还是它一直以来的那张图 | +| `MCPP_PACK_FORMAT` *(2026.9.11.1+)* | `mcpp::pack_format()` | 本程序所处的这次 `mcpp pack` 的 `--pkg-format` 取值；任何普通构建下都为空。承载含义的正是这个空值 —— 成员据此为自己的提交加闸，于是 `mcpp build` 拿到的还是它一直以来的那张图 | | `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | `mcpp pack` 已经把闭包暂存到的位置，绝对路径；本次构建不在打包时为空。读它来判断这次要干的活是什么形状，而把 `${mcpp.stage_dir}` 写进 action —— 这样图里的路径与程序读到的路径不可能不一致 | | `MCPP_PACK_STRIP` *(2026.9.16.1+)* | `mcpp::pack_strip()` | 本程序所处的这次 `mcpp pack` 剥离时为 `1`，`--no-strip` 下为 `0`；任何普通构建下都为空。自己暂存库的成员跟随它，于是一个开关管住包里的每个文件 | | `MCPP_PACK_DEBUG_SYMBOLS_DIR` *(2026.9.16.1+)* | `mcpp::pack_debug_symbols_dir()` | `--debug-symbols` 放置分离出的 `*.debug` 文件的位置，绝对路径；丢弃调试信息或本次构建不在打包时为空 | diff --git a/docs/zh/31-authoring-a-rule-package.md b/docs/zh/31-authoring-a-rule-package.md index f6a673e58..641a05079 100644 --- a/docs/zh/31-authoring-a-rule-package.md +++ b/docs/zh/31-authoring-a-rule-package.md @@ -313,7 +313,7 @@ mcpp::floor("cuda.driver", runtime_needs); 一个 `dist-*` 成员既不是规则也不是工具。它不编译翻译单元，也不在构建程序 运行期间把活干完：它消费**链接产物**，产出用户要去安装的东西 —— 一个 `.msi`、一个 `.deb`、一个 AppImage、一个签过名的 `.app`。机制是一条 -`artifact` action 加上 `mcpp pack --format `，见 +`artifact` action 加上 `mcpp pack --pkg-format `，见 [30 —— 产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111)。 本章其余内容原样适用。有六条在这里绑得更紧，每一条都是这个类别会犯、而 diff --git a/docs/zh/34-authoring-a-bsp.md b/docs/zh/34-authoring-a-bsp.md index 762c98ce2..f4620cad1 100644 --- a/docs/zh/34-authoring-a-bsp.md +++ b/docs/zh/34-authoring-a-bsp.md @@ -112,6 +112,25 @@ int main() { **一个探针，一个使用者。** `mcpp::run_exclusive()` 声明这个目标的运行不可 重叠，`mcpp test` 随后把它们串行化。工程方永远不需要记得加 `-j1`。 +## 作为平台包的板子 + +板子的构建程序陈述的字段——链接脚本、入口、runner 与具名 runner——都是平台的字段（2026.10.27.1+， +[35 —— 平台](35-platforms.md)）。BSP 可以改在清单中陈述其中固定的部分，构建程序只保留随所选 feature 变化的部分： + +```toml +[platform] +targets = ["thumbv7em-none-eabihf"] +runner = ["qemu-system-arm", "-machine", "mps2-an386", "-nographic", "-kernel", "{}"] +runners = { flash = ["probe-rs", "download", "--chip", "STM32L475VG", "{}"] } + +[platform.link] +script = "cortex-m.ld" +start = "Reset_Handler" +``` + +工程随后把这块板子选为目标的平台（`[target.thumbv7em-none-eabihf] platform = "@"`），而不是作为依赖。 +两种写法合并为一份陈述：构建程序陈述的值替换清单中的值，工程的 `[target.]` 键替换二者。 + ## 当前边界 - BSP 把它的模拟器与探针驱动声明为载荷，因此工具链未在某个平台发布的板子， diff --git a/docs/zh/41-devices.md b/docs/zh/41-devices.md index f940db252..b6a29a4c5 100644 --- a/docs/zh/41-devices.md +++ b/docs/zh/41-devices.md @@ -84,6 +84,17 @@ monitor = ["probe-rs", "attach", "--chip", "STM32L475VG"] 被默默应用。同一个名字只允许一个依赖提供，出现第二个是一个点名两个包的 错误。 +runner 也可以是一张表（2026.10.27.1+），陈述产物在哪里运行、运行能否重叠： + +```toml +[target.'cfg(platform = "openkal")'] +runner = { cmd = ["okrun", "{}"], host = "same-arch", exclusive = false } +``` + +`host = "same-arch"` 只在产物架构的机器上运行它，其他架构的产物被拒绝并点名 `run_under`；`"any"` 表示 +runner 能从任何机器抵达产物。目标的平台提供它的缺省 runner（[35](35-platforms.md)）；依赖的构建程序、本机的 +`config.toml`、工程与 `MCPP_TARGET__RUNNER` 依次替换它，只写 argv 时保留平台的 `host` 与 `exclusive`。 + ## runner 收到的内容 runner 收到一段 argv——它自己的 token，产物路径被追加或替换 `{}`—— @@ -103,7 +114,7 @@ libfw.so/abs/project/target///bin/libfw.so 第一个字段是相对于产物所在目录的目的路径，分隔符为 `/`；对于在子目录中 发现的测试程序，它以 `../` 开头。第二个字段是该文件在输出树中的绝对 路径。两者以 TAB 分隔。每次启动 runner 时这个文件都存在，没有需要携带 -的东西时为空；以 `--format` 运行的可分发物收到的是空文件，因为可分发物 +的东西时为空；以 `--pkg-format` 运行的可分发物收到的是空文件，因为可分发物 自带其文件。 就地执行产物的 runner 会忽略这个变量。把产物移走的 runner——移到设备、 @@ -112,7 +123,7 @@ libfw.so/abs/project/target///bin/libfw.so ## 运行一个可分发物 -`mcpp run --format ` 打包可分发物 `` 并运行它。不带 `--runner` +`mcpp run --pkg-format ` 打包可分发物 `` 并运行它。不带 `--runner` 时，若图或清单提供了名为 `` 的 runner，就由它抵达，否则由默认 runner 抵达： @@ -121,12 +132,12 @@ mcpp::runner("app", ""); // a package names a runner after its fo ``` ```bash -mcpp run --format app # through the runner named `app` -mcpp run --format app --runner other # a typed --runner still wins +mcpp run --pkg-format app # through the runner named `app` +mcpp run --pkg-format app --runner other # a typed --runner still wins ``` 目录形式的可分发物（例如应用包）不会被直接执行。没有 runner 能抵达它 -时，`mcpp run --format ` 会在启动任何东西之前拒绝，退出状态为 +时，`mcpp run --pkg-format ` 会在启动任何东西之前拒绝，退出状态为 `126`，并点名能抵达它的 runner `` 以及本该声明它的清单表。 ## 终止由声明决定，不由推断 From 51a15cc76003969065d94ea3b634aa272927954d Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 05:30:34 +0900 Subject: [PATCH 15/54] docs: CHANGELOG 2026.10.27.1 (platforms, packaging axes, --pkg-format, MCPP_BIN_, refusals); docs/90 e2e count Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 85 ++++++++++++++++++++++++++++++++- docs/90-build-from-source.md | 4 +- docs/zh/90-build-from-source.md | 4 +- 3 files changed, 88 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e3c735f49..70dcedbd4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,81 @@ ## [2026.10.27.1] - 2026-10-27 -### Fixed +### Added + +- **Platforms: the configuration of a class of targets as one versioned package** + (SPEC-013; docs/35). A package's `[platform]` section states the targets it + serves, their toolchain and target base, the package for each target-side + layer (`[platform.layers]`, or `false`), the tools built for this machine + (`[platform.tools]`), the compile flags (`[platform.build]` and its + `cfg(...)` tables), the link (`[platform.link]`: `driver`, `flags`, `start`, + `script`, `inputs`, the names `exe` and `shared`, the `seal` steps), the + runner (`{ cmd, host = "same-arch" | "any", exclusive }`) and named runners, + the carrier of a self-contained program, a plugin run in the platform phase, + a version group (`[platform.versions]`) and the part a host triple takes + (`[platform.hosted]`). The engine selects a platform for the target graph + (`--platform`, `MCPP_TARGET__PLATFORM`, `[target.] platform`, + `config.toml`, the target table) and for the host graph (`--host-platform`, + `MCPP_HOST_PLATFORM`, `[host] platform`), merges the overrides a project and + this machine write under the same key names, adds the layers and tools the + graph lacks (a layer a package of the graph provides is left to it), links, + seals and names the artifact, runs it, and reports every value with its + source. `cfg(platform = "")` selects by platform. +- **`x86_64-openkal` and `aarch64-openkal` need no declaration.** The target + table selects `mcpplibs.openkal-platform` for both rows, and the platform + states what a project wrote by hand in 2026.10.26.2: `llvm@23.1.3`, + openkal-musl, openkal-llvm-runtime, `okld` and `okrun`, the image's flags, + its link and seal, `{name}.oke` and `lib{name}.okso`, the runner and the M3 + carrier. A project that names only its sources builds, runs, tests and + packs an ok image. The engine holds no openkal code (`src/okimage` is + removed). +- **The host graph's platform.** With `[host] platform` build programs and + their libraries are compiled as ok images of this machine's architecture, + with one toolchain on Linux, macOS and Windows, and run by the platform's + runner; a platform's own tools are always built natively. +- **Protocol 17** (SPEC-007 R9.10): `link_driver`, `link_input`, + `link_input_from`, `artifact_name`, `seal` and `run_host`, stated by a + platform's plugin in the platform phase (`mcpp::phase()` is `"platform"`) + or by any build program in its build phase. +- **One configuration, without a build.** `mcpp self env --target ` + (`--host`, `--format json`) reports the toolchain, the platform with each + field's source, the tools, the runner and its program; `mcpp toolchain + install --target ` fetches the target's platform and builds its + tools; `mcpp toolchain list` has a PLATFORM column. +- **`mcpp test` builds the package's programs first** and gives each test + `MCPP_BIN_`, the path of the program ``. In a platform package + with no `--target`, it builds the tests for each of the platform's targets + and runs those of this machine's architecture. +- **Packaging along three axes** (SPEC-014; docs/10): `--linkage + static|shared` (the program), `--mode system|vendored|self-contained` (what + the machine provides; `--mode static` is `--linkage static --mode + self-contained`), `--pkg-format` (what is delivered) and the legs + (`--target`, repeatable). A platform with `universal = true` packs every + architecture into one tree with generated launchers (`--no-universal`, + `--systems`, `--merge`); `--pkg-format single-file` delivers one executable; + `--compress zstd|gzip|xz|none`; `[pack] linkage`, `format`, `compress`, + `sign`; every staged tree carries `mcpp-package.json`; the `mcpp.pack` + envelope gains `linkage`, `mode`, `pkgFormat`, `legs`, `systems` and each + artifact's `sha256`. +- `mcpp.lock` records the packages a platform added with + `origin = "platform"`; the `plan` event and `result.json` carry + `configuration.platform` and `hostPlatform`. + +### Changed + +- **`--pkg-format` is the package format of `mcpp pack` and `mcpp run`.** + `--format` there is the earlier spelling and prints a warning; on every other + command `--format` is the output encoding. +- The reason `image-tool-absent` is `platform-tool-absent`; new reasons + `platform-not-a-platform`, `platform-target-unserved`, + `platform-version-group` and `platform-phase-directive`. +- A layer the target's platform added is reported `platform`, and the + platform follows on a `Platform` line (`Host platform` for the host graph). +- A version of a dependency the index does not have is stated by mcpp: which + index, its revision and age, and whether it was refreshed now and why not; + the provider's own report is printed under `-v`. +- A package shared by several configurations of one command is announced once. + - **A version the local index lacks is refreshed for when its install is refused.** The walk refreshes the index when a descriptor is missing, and not @@ -32,6 +106,15 @@ three addresses, although the descriptor had named the package in `mcpplibs`. The compat spellings are tried only when no descriptor could be read. +- **Two versions of a package that exports a header directory or provides a + target-side layer are refused**, naming both requests. A renamed copy kept + its C names and link symbols, and the two met in one link or one include + path at a place that named neither. +- **A version group is checked before the platform completes the graph.** A + family package the project pinned outside the group is refused as the + group's conflict, with both remedies. +- **The std module of a macOS build whose C library is the graph's takes the + deployment floor** every unit takes; it took the running system's version. ## [2026.10.26.2] - 2026-10-26 diff --git a/docs/90-build-from-source.md b/docs/90-build-from-source.md index bf926cb2a..f1f503e37 100644 --- a/docs/90-build-from-source.md +++ b/docs/90-build-from-source.md @@ -82,7 +82,7 @@ src/ tests/ ├── unit/ 154 C++ tests, discovered by `mcpp test` -└── e2e/ 637 shell scripts against a real binary +└── e2e/ 638 shell scripts against a real binary ``` ## Test Organization @@ -93,7 +93,7 @@ Two layers, and they answer different questions. discovers. They exercise a module's contract with no binary and no filesystem state. -**End-to-end tests** (`tests/e2e/NN_.sh`, 637 files) run a real `mcpp` +**End-to-end tests** (`tests/e2e/NN_.sh`, 638 files) run a real `mcpp` binary against a real project. `run_all.sh` is the CI entry point. `mcpp test` does **not** run them. diff --git a/docs/zh/90-build-from-source.md b/docs/zh/90-build-from-source.md index 5c453e510..74d38c85b 100644 --- a/docs/zh/90-build-from-source.md +++ b/docs/zh/90-build-from-source.md @@ -84,7 +84,7 @@ src/ tests/ ├── unit/ 154 C++ tests, discovered by `mcpp test` -└── e2e/ 637 shell scripts against a real binary +└── e2e/ 638 shell scripts against a real binary ``` ## 测试组织 @@ -94,7 +94,7 @@ tests/ **单元测试**（`tests/unit/`，154 个文件）是 `mcpp test` 发现的 C++ 程序。它们检验一个模块的契约，不需要二进制、也不需要文件系统状态。 -**端到端测试**（`tests/e2e/NN_.sh`，637 个文件）拿真实的 `mcpp` +**端到端测试**（`tests/e2e/NN_.sh`，638 个文件）拿真实的 `mcpp` 二进制跑真实工程。`run_all.sh` 是 CI 的入口。`mcpp test` **不**运行它们。 ```bash From 8252daa6a26211544b1b11a1562159eeab45dd7f Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 05:31:24 +0900 Subject: [PATCH 16/54] =?UTF-8?q?docs:=20SPEC-013=20=C2=A75=20states=20cfg?= =?UTF-8?q?(platform)'s=20specificity=20as=20the=20selector=20code=20count?= =?UTF-8?q?s=20it;=20=C2=A79.1=20where=20the=20platform's=20ldflags=20go?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/specs/platforms.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index f225fec9b..517430e45 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -181,8 +181,9 @@ plugin = "build.mcpp" ## 5. `cfg(platform = ...)` `cfg()` 谓词增加键 `platform`。它匹配所选平台的名字(平台包短名去掉结尾的 `-platform`,`openkal-platform` -的名字是 `openkal`)或平台包的完整短名。其具体程度位于三元组之下、`os` 之上(SPEC-004 §3.1.1)。无平台时, -`cfg(platform = ...)` 不匹配。**已实现** +的名字是 `openkal`)或平台包的完整短名。它在具体程度中计一个分量,与 `arch`、`env` 相同,低于 `os`(计 +`os` 与族两个分量)与三元组;`all(...)` 中的分量相加(SPEC-004 §3.1.1)。无平台时,`cfg(platform = ...)` +不匹配。**已实现** ## 6. 合并 @@ -256,7 +257,8 @@ plugin = "build.mcpp" ### 9.1 编译 -平台陈述的 `build` 标志加在目标图每个翻译单元的命令行前部,`cfg(...)` 子表按目标求值后追加。**已实现** +平台陈述的 `build` 标志进入目标图每个翻译单元的命令行(`cflags` 给 C,`cxxflags` 给 C++,`asmflags` 给汇编, +`ldflags` 给链接,`driver = "lld"` 时位于 `link.flags` 之后、直接给 `ld.lld`),`cfg(...)` 子表按目标求值后追加在无条件的标志之后。**已实现** ### 9.2 链接 From 9206221ce6183c97a5397ed92c84f5cb9c04eb48 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 05:55:20 +0900 Subject: [PATCH 17/54] pack: the three axes (--linkage, --mode, --pkg-format tar|zip|dir|single-file) with [pack] linkage/format/compress/sign/universal; --compress gzip|zstd|xz|none in mcpp's own codecs; universal packages of a platform's targets with generated launchers and per-system loaders (--systems); single files (the engine's launcher over a zstd tree, a platform's carrier over its images); mcpp-package.json; [pack] sign; --merge; the platform phase provides pack formats - compat.zlib, compat.zstd, compat.xz: mcpp.pack.compress (one-shot codecs); a zip's members deflated, aligned stored members, links - mcpp.pack.distribution: sha256, containers, unpack, launchers, mcpp-package.json, the single file's payload and launcher source, --merge - [platform] universal; BuildOverrides::dependency_linkage - the carrier generalised (carried bytes or nothing, any system/arch row, the section by the target system); the static row pinned per arch - envelope: linkage, mode, pkgFormat, legs, systems, artifacts[].sha256 - the platform phase accepts pack-format and artifact actions - contributing skill: no three-level loop nesting without a stated reason Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .agents/skills/mcpp-contributing/SKILL.md | 2 + mcpp.toml | 8 + modules/buildmcpp/src/directives.cppm | 3 + modules/manifest/src/platform.cppm | 13 +- modules/manifest/src/platform_toml.cppm | 4 + modules/manifest/src/toml.cppm | 18 + modules/manifest/src/types.cppm | 14 + src/build/build_program.cppm | 21 +- src/build/prepare.cppm | 4 + src/build/prepare/toolchain.cpp | 4 + src/cli.cppm | 23 +- src/cli/cmd_publish.cppm | 194 ++++- src/pack/compress.cppm | 185 +++++ src/pack/compress_codecs.cpp | 154 ++++ src/pack/distribution.cppm | 963 ++++++++++++++++++++++ src/pack/library_pipeline.cppm | 3 +- src/pack/pack.cppm | 747 +++++++++++++---- src/pack/pipeline.cppm | 116 ++- src/pack/zip.cppm | 85 +- src/platform/platform.cppm | 1 + 20 files changed, 2339 insertions(+), 223 deletions(-) create mode 100644 src/pack/compress.cppm create mode 100644 src/pack/compress_codecs.cpp create mode 100644 src/pack/distribution.cppm diff --git a/.agents/skills/mcpp-contributing/SKILL.md b/.agents/skills/mcpp-contributing/SKILL.md index d6ebec57b..019ac5d0c 100644 --- a/.agents/skills/mcpp-contributing/SKILL.md +++ b/.agents/skills/mcpp-contributing/SKILL.md @@ -121,6 +121,8 @@ git checkout -b / - 遵循现有代码风格（查看相邻代码） - 模块导入用 `import std;` 和 `import mcpp.xxx;` - 只改需要改的，不顺手重构不相关代码 +- **循环嵌套不超过两层**：第三层改为具名的辅助函数、一次扁平的遍历（例如带深度的 + `recursive_directory_iterator`）或先收集再处理；确有必要保留三层时，在该处注释写明理由 - 平台相关代码放在 `src/platform/` 目录下 **构建验证** diff --git a/mcpp.toml b/mcpp.toml index 011d9290e..8afd1bfe8 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -93,6 +93,14 @@ mcpplibs.cmdline = "0.0.1" [dependencies.mcpplibs] os = "0.2.0" +# The codecs of a package's container (SPEC-014 §5): gzip and raw DEFLATE, +# zstd, xz. The reference libraries, so an archive is written the same way on +# every host and nothing is looked for on PATH. +[dependencies.compat] +zlib = "1.3.2" +zstd = "1.5.7" +xz = "5.8.3" + [target.linux.build] sources = ["!src/native/windows/**"] diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index f34989f61..da5257262 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -668,6 +668,9 @@ std::optional decode_action(std::string_view payloa // caller can refuse BEFORE applying anything — a half-applied action set is // worse than none. std::string action_error(const Directives& d); +// The role an `mcpp:action=` payload names ("source" when absent), or nothing +// for a payload that is not JSON. +std::optional action_role_of(std::string_view payload); // Non-empty when a `windows-subsystem` or `windows-entry` directive names // something `apply` cannot honour: a value that is not `:`, a diff --git a/modules/manifest/src/platform.cppm b/modules/manifest/src/platform.cppm index fae475c06..53fa649de 100644 --- a/modules/manifest/src/platform.cppm +++ b/modules/manifest/src/platform.cppm @@ -100,6 +100,12 @@ struct Words { bool set = false; }; +struct Flag { + bool value = false; + Origin origin; + bool set = false; +}; + // `runner = ["okrun", "{}"]` or `runner = { cmd = [...], host = "same-arch", // exclusive = true }`. `host`: "" (unstated), "same-arch" (the artifact runs // on a machine of its architecture), "any" (the runner reaches it from any @@ -201,6 +207,9 @@ struct Statement { std::map runners; Carrier carrier; Text plugin; + // `universal = true`: `mcpp pack` of any of the targets packs all of them + // into one package (SPEC-014 §3). + Flag universal; // [platform.layers], [platform.tools] std::map layers; std::map tools; @@ -218,7 +227,7 @@ struct Statement { bool empty() const { return !targets.set && !toolchain.set && !sysroot.set && !runner.set - && runners.empty() && !carrier.set && !plugin.set && layers.empty() + && runners.empty() && !carrier.set && !plugin.set && !universal.set && layers.empty() && tools.empty() && build.empty() && buildCfg.empty() && !link.driver.set && !link.flags.set && !link.start.set && !link.script.set && !link.inputsSet && !link.exe.set && !link.shared.set && !link.sealSet @@ -251,7 +260,7 @@ constexpr bool is_layer(std::string_view k) { // list per table, read by the parser's unknown-key sweep and by the tests. inline constexpr std::string_view kPlatformKeys[] = { "targets", "toolchain", "sysroot", "runner", "runners", "carrier", "plugin", - "layers", "tools", "build", "link", "versions", "hosted", + "universal", "layers", "tools", "build", "link", "versions", "hosted", }; inline constexpr std::string_view kBuildKeys[] = { "cflags", "cxxflags", "asmflags", "ldflags" }; inline constexpr std::string_view kLinkKeys[] = { diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm index 0e331ca39..cf7b371c4 100644 --- a/modules/manifest/src/platform_toml.cppm +++ b/modules/manifest/src/platform_toml.cppm @@ -518,6 +518,10 @@ std::expected Reader::statement(const toml::Table& t) co return std::unexpected(std::format( "{} must name the platform's build program, for example \"build.mcpp\"", key)); s.plugin = Text{ absolute(v.as_string()).generic_string(), at(v, key), true }; + } else if (k == "universal") { + if (!v.is_bool()) + return std::unexpected(std::format("{} must be true or false", key)); + s.universal = Flag{ v.as_bool(), at(v, key), true }; } else if (k == "layers") { if (auto r = layers(v, "[platform.layers]", s.layers); !r) return std::unexpected(r.error()); } else if (k == "tools") { diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 3fa7fe831..605be26c4 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -3804,6 +3804,24 @@ std::expected parse_document(const t::Document& documen m.packConfig.alsoSkip = *v; if (auto v = doc->get_string_array("pack.bundle-project.force_bundle")) m.packConfig.forceBundle = *v; + // The three axes' defaults, the signer and `universal` (SPEC-014). The + // values are checked where the command reads them, with the flags that + // take the same words. + if (auto v = doc->get_string("pack.linkage")) { + if (*v != "static" && *v != "shared") + return std::unexpected(error(origin, std::format( + "[pack].linkage = '{}' invalid; expected 'static' | 'shared'", *v))); + m.packConfig.linkage = *v; + } + if (auto v = doc->get_string("pack.format")) m.packConfig.format = *v; + if (auto v = doc->get_string("pack.compress")) m.packConfig.compress = *v; + if (auto v = doc->get_string_array("pack.sign")) { + if (v->empty()) + return std::unexpected(error(origin, std::string( + "[pack].sign is an argv, `{}` the file signed; it is empty"))); + m.packConfig.sign = *v; + } + if (auto v = doc->get_bool("pack.universal")) m.packConfig.universal = *v; // [target.] — per-target overrides. We accept both GCC // (x86_64-linux-musl) and Rust-style (x86_64-unknown-linux-musl) diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index f4bab329f..84a2a5541 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1861,6 +1861,20 @@ struct PackConfig { // assumption ("modern desktop Linux"). std::vector alsoSkip; // libs to ALSO skip on top of PEP 600 std::vector forceBundle; // libs to bundle even if PEP 600 says skip + // The three axes' project defaults (SPEC-014 §1): `linkage` ("static" | + // "shared"), `format` (the package format, as `--pkg-format` names it), + // `compress` (`[:]`, as `--compress` takes it). Empty: + // unstated. + std::string linkage; + std::string format; + std::string compress; + // The signer (SPEC-014 §4): an argv run on each program, image and library + // of the staged tree, and on a single file, before the container is + // written; `{}` is the file. + std::vector sign; + // `false`: a pack of a platform's target packs that target alone, where + // the platform states `universal = true` (SPEC-014 §3). + std::optional universal; }; // `[workspace]` — multi-package workspace support (0.0.11+). diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 8a3e55e70..47fbec886 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -1321,10 +1321,12 @@ std::expected run_build_program_impl( } } else if (env.phase == "platform") { // THE PLATFORM PHASE STATES THE PLATFORM'S BUILD, LINK AND RUNNER - // FIELDS AND NOTHING ELSE (SPEC-013 §4.2): it describes a class of - // targets, not a package of sources, and what it states reaches every - // unit and every link of the target. Refused rather than dropped, - // naming the first directive that does not belong. + // FIELDS, AND THE PACKAGE FORMATS IT PROVIDES (SPEC-013 §4.2, SPEC-014 + // §4): it describes a class of targets, not a package of sources, and + // what it states reaches every unit and every link of the target. A + // format is provided as any package provides one -- `pack-format` and + // the `artifact` actions that produce it from the staged tree. Refused + // rather than dropped, naming the first directive that does not belong. for (auto const& def : dirs::kTable) { switch (def.slot) { case Slot::CxxFlags: case Slot::CFlags: case Slot::LdFlags: @@ -1333,16 +1335,23 @@ std::expected run_build_program_impl( case Slot::LinkInputs: case Slot::ArtifactNames: case Slot::Seal: case Slot::RunHost: case Slot::RerunFiles: case Slot::RerunEnv: case Slot::RerunGlobs: case Slot::Warnings: case Slot::Diagnostics: + case Slot::PackFormats: continue; default: break; } if (d.at(def.slot).empty()) continue; + if (def.slot == Slot::Actions + && std::ranges::all_of(d.at(def.slot), [](std::string const& payload) { + return dirs::action_role_of(payload).value_or("") == "artifact"; + })) + continue; mcpp::build::refusal::record(mcpp::build::refusal::Code::PlatformPhaseDirective); return std::unexpected(std::format( "build.mcpp stated `mcpp:{}=` in its platform phase.\n" " A platform's plugin states the platform's build, link and runner\n" - " fields only (compile and link flags, the link driver, inputs,\n" - " entry, artifact names, seal steps, runners); `mcpp::phase()` is\n" + " fields (compile and link flags, the link driver, inputs, entry,\n" + " artifact names, seal steps, runners) and the package formats it\n" + " provides (`pack-format`, `artifact` actions); `mcpp::phase()` is\n" " \"platform\" there (SPEC-013 §4.2).", def.wire)); } diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 9c3c804ff..385feac65 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -879,6 +879,10 @@ export struct BuildOverrides { // same relationship --target has with [toolchain]. std::string accel; bool force_static = false; // --static (or implied by musl target) + // `mcpp pack --linkage` (SPEC-014 §1): the whole graph's dependency + // linkage for this build, over `[build] dependency_linkage` and the + // profile's. Empty: as the project states it. + std::string dependency_linkage; std::string package_filter; // -p : only build this workspace member // The workspace members this plan builds, as written in `[workspace] // members` ("." is a rooted workspace's own package), all of one diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index d22cce5b6..5dda17092 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -257,6 +257,10 @@ static std::expected step1_define_early_toolchain_closures(Pr // with the declared profile. if (pr.dependencyLinkageDeclared) state.m->buildConfig.dependencyLinkage = pr.dependencyLinkage; + // `mcpp pack --linkage`: the package's program property, stated for + // this build over the project's (SPEC-014 §1). + if (!state.overrides.dependency_linkage.empty()) + state.m->buildConfig.dependencyLinkage = state.overrides.dependency_linkage; state.m->buildConfig.optLevel = pr.optLevel; state.m->buildConfig.debug = pr.debug; state.m->buildConfig.lto = pr.lto; diff --git a/src/cli.cppm b/src/cli.cppm index b842051d4..0a03425dd 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -897,17 +897,34 @@ int run(int argc, char** argv) { // package. Omit it and mcpp picks the only packable target. .arg(cl::Arg("target").help("Target name from [targets.*] (optional)")) .option(cl::Option("mode").takes_value() - .help("system | vendored (default) | self-contained | static")) + .help("What the machine that runs it provides: system | vendored (default) | " + "self-contained | static (= --linkage static --mode self-contained)")) + .option(cl::Option("linkage").takes_value().value_name("static|shared") + .help("The program's libraries in it (static) or beside it (shared); rebuilds " + "with [build] dependency_linkage (default: as the project builds)")) .option(cl::Option("target").takes_value().multiple() .help("Triple, e.g. x86_64-linux-musl (repeatable: one leg per triple)")) + .option(cl::Option("no-universal") + .help("Pack the selected target alone where its platform packs all its targets " + "into one universal package")) + .option(cl::Option("systems").takes_value().value_name("LIST") + .help("Self-contained universal package: the systems whose loaders it carries " + "(linux,macos,windows; default: this machine's)")) + .option(cl::Option("compress").takes_value().value_name("ALG[:LEVEL]") + .help("The container's compression: gzip (tar default) | zstd | xz | none; " + "a zip deflates unless none")) + .option(cl::Option("merge").takes_value().multiple().value_name("PACKAGE") + .help("Merge packages this command wrote (universal legs, systems) into one; " + "repeatable, nothing is built")) // "plus any format the resolved graph provides", because the // values are no longer a fixed list. A package declares one with // `mcpp::provides_pack_format` and `--format ` dispatches to // it; an unknown value names what IS available rather than a // constant, so the help text does not have to enumerate them. .option(cl::Option("pkg-format").takes_value().value_name("FORMAT") - .help("The package format: tar (default; .zip for a Windows target) | dir | any " - "format the resolved graph provides (e.g. appimage, msi)")) + .help("The package format: tar | zip | dir | single-file | any format the " + "resolved graph provides (e.g. appimage, msi, okb); default: zip for a " + "Windows program, tar elsewhere")) .option(cl::Option("format").takes_value() .help("Earlier spelling of --pkg-format")) .option(cl::Option("output").short_name('o').takes_value() diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 9af765aa4..8e7cc9aa1 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -28,6 +28,10 @@ import mcpp.wire; import mcpp.home; // --list-formats plans outside the project import mcpp.pack.stage_tree; // the built-in formats import mcpp.toolchain.fingerprint; // hash_string +import mcpp.toolchain.platform; // host_arch: the static row of `--linkage static` +import mcpp.pack.compress; // --compress +import mcpp.pack.distribution; // --merge +import mcpplibs.os.platform; namespace mcpp::cli { @@ -326,11 +330,15 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { std::error_code ec; nlohmann::json t = nlohmann::json::array(); for (auto const& x : targets) t.push_back(x); + const bool dir = std::filesystem::is_directory(p, ec); return nlohmann::json{ {"path", std::filesystem::absolute(p, ec).lexically_normal().string()}, - {"type", std::filesystem::is_directory(p, ec) ? "directory" : "file"}, + {"type", dir ? "directory" : "file"}, {"format", format}, {"targets", std::move(t)}, + // SPEC-014 §9: the digest of a file, which a release publishes. + {"sha256", dir ? nlohmann::json(nullptr) + : nlohmann::json(mcpp::pack::distribution::sha256_file(p))}, }; }; nlohmann::json artifacts = nlohmann::json::array(); @@ -371,6 +379,16 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { } env.data = nlohmann::json{{"artifacts", std::move(artifacts)}, {"stage", std::move(stage)}}; if (!stages.is_null()) env.data["stages"] = std::move(stages); + // The three axes and the legs the package was made with (SPEC-014 §9). + if (!libraryRoute) { + auto const& first = outcome.members.empty() ? mcpp::pack::PackOutcome::Member{} + : outcome.members.front(); + env.data["linkage"] = outcome.linkage.empty() ? first.linkage : outcome.linkage; + env.data["mode"] = outcome.mode.empty() ? first.mode : outcome.mode; + env.data["pkgFormat"] = outcome.format; + env.data["legs"] = outcome.targets; + env.data["systems"] = outcome.systems; + } mcpp::wire::emit(env); return 0; } @@ -525,20 +543,154 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, // exists to remove. The refusal moves to `build_and_pack`, after build // programs have declared what they provide and before anything is // compiled, where it can name what IS available instead of a fixed list. - if (auto v = pack_format_option(parsed)) { - if (*v == "tar") opts.format = mcpp::pack::Format::Tar; - else if (*v == "dir") opts.format = mcpp::pack::Format::Dir; - else if (v->empty()) { - mcpp::ui::error("--pkg-format needs a value: tar | dir | a format the " - "resolved graph provides"); - return 2; - } else { + // THE PROJECT'S DEFAULTS OF THE THREE AXES (`[pack]`, SPEC-014 §1), read + // from the package this command packs; a flag states over each. + mcpp::manifest::PackConfig packCfg; + if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path())) + if (auto eff = mcpp::project::load_effective_manifest(*root)) + packCfg = eff->manifest.packConfig; + if (parsed.option("merge")) { + std::vector inputs; + for (auto const& v : parsed.option("merge")->get().values) { + std::size_t at = 0; + while (at <= v.size()) { + auto comma = v.find(',', at); + if (comma == std::string::npos) comma = v.size(); + if (comma > at) inputs.push_back(v.substr(at, comma - at)); + at = comma + 1; + } + } + mcpp::pack::distribution::MergeRequest req; + for (auto const& i : inputs) req.inputs.emplace_back(i); + if (auto v = parsed.value("output")) req.output = *v; + if (auto f = pack_format_option(parsed)) req.format = *f; + else if (!packCfg.format.empty()) req.format = packCfg.format; + std::string compressText = parsed.value("compress").value_or(packCfg.compress); + if (!compressText.empty()) { + auto c = mcpp::pack::compress::parse(compressText); + if (!c) { mcpp::ui::error(std::format("--compress: {}", c.error())); return 2; } + req.compression = *c; + } + auto merged = mcpp::pack::distribution::merge(req); + if (!merged) { mcpp::ui::error(merged.error()); return 1; } + mcpp::ui::status("Packed", merged->string()); + if (report) { + report->rc = 0; + report->artifacts.push_back(*merged); + report->format = req.format.empty() ? std::string("tar") : req.format; + } + return 0; + } + + std::string formatText; + if (auto v = pack_format_option(parsed)) formatText = *v; + else formatText = packCfg.format; + if (auto v = pack_format_option(parsed); v && v->empty()) { + mcpp::ui::error("--pkg-format needs a value: tar | zip | dir | single-file | a format " + "the resolved graph provides"); + return 2; + } + if (!formatText.empty()) { + opts.formatStated = true; + if (formatText == "tar") opts.format = mcpp::pack::Format::Tar; + else if (formatText == "dir") opts.format = mcpp::pack::Format::Dir; + else if (formatText == "zip") opts.format = mcpp::pack::Format::Zip; + else if (formatText == "single-file") opts.format = mcpp::pack::Format::SingleFile; + else { opts.format = mcpp::pack::Format::Dispatched; - opts.formatName = *v; + opts.formatName = formatText; } } if (auto v = parsed.value("output")) opts.output = outputFromUser ? outputFromUser->string() : *v; + // ─── Linkage, compression, universal, systems, signer ──────────── + { + std::string linkage = parsed.value("linkage").value_or(packCfg.linkage); + if (!linkage.empty()) { + if (linkage == "static") opts.linkage = mcpp::pack::Linkage::Static; + else if (linkage == "shared") opts.linkage = mcpp::pack::Linkage::Shared; + else { + mcpp::ui::error(std::format("invalid --linkage '{}'; expected: static | shared", + linkage)); + return 2; + } + } + // `--mode static` IS `--linkage static --mode self-contained` (SPEC-014 + // §1): the program carries its libraries, and the machine provides + // nothing. + if (opts.mode == mcpp::pack::Mode::Static) { + if (opts.linkage == mcpp::pack::Linkage::Shared) { + mcpp::ui::error("--mode static is --linkage static --mode self-contained; " + "--linkage shared contradicts it.\n" + " use: --linkage shared --mode self-contained"); + return 2; + } + opts.linkage = mcpp::pack::Linkage::Static; + } + std::string compressText = parsed.value("compress").value_or(packCfg.compress); + if (!compressText.empty()) { + auto c = mcpp::pack::compress::parse(compressText); + if (!c) { mcpp::ui::error(std::format("--compress: {}", c.error())); return 2; } + opts.compression = *c; + } + using mcpp::pack::compress::Algorithm; + const auto alg = opts.compression.algorithm; + if (opts.compression.stated) { + if (opts.format == mcpp::pack::Format::Dir || opts.format == mcpp::pack::Format::Dispatched) { + mcpp::ui::error(std::format( + "--compress {} compresses a container, and --pkg-format {} writes none.\n" + " use: --pkg-format tar | zip | single-file", + mcpp::pack::compress::name(alg), + opts.format == mcpp::pack::Format::Dir ? std::string("dir") : opts.formatName)); + return 2; + } + if (opts.format == mcpp::pack::Format::Zip && (alg == Algorithm::Zstd || alg == Algorithm::Xz)) { + mcpp::ui::error(std::format( + "a zip's members are deflated (gzip) or stored (none); --compress {} is a " + "tar's.\n use: --pkg-format tar --compress {}, or --compress gzip | none", + mcpp::pack::compress::name(alg), mcpp::pack::compress::name(alg))); + return 2; + } + if (opts.format == mcpp::pack::Format::SingleFile + && (alg == Algorithm::Gzip || alg == Algorithm::Xz)) { + mcpp::ui::error(std::format( + "a single file's payload is zstd (or none): its launcher decompresses one " + "codec.\n use: --compress zstd[:] | none")); + return 2; + } + } else if (opts.format == mcpp::pack::Format::SingleFile) { + opts.compression.algorithm = Algorithm::Zstd; + } + if (parsed.is_flag_set("no-universal")) opts.universal = false; + else opts.universal = packCfg.universal; + if (auto v = parsed.value("systems")) { + std::size_t at = 0; + while (at <= v->size()) { + auto comma = v->find(',', at); + if (comma == std::string::npos) comma = v->size(); + auto word = v->substr(at, comma - at); + if (!word.empty()) { + if (word != "linux" && word != "macos" && word != "windows") { + mcpp::ui::error(std::format( + "--systems names '{}'; the systems are linux, macos and windows", word)); + return 2; + } + if (std::ranges::find(opts.systems, word) == opts.systems.end()) + opts.systems.push_back(word); + } + at = comma + 1; + } + } + opts.sign = packCfg.sign; + // A single file carries what it runs: the machine provides nothing. + if (opts.format == mcpp::pack::Format::SingleFile && opts.mode == mcpp::pack::Mode::None) { + mcpp::ui::error("--pkg-format single-file carries what the program runs, and --mode " + "system leaves it to the machine.\n" + " use: --mode vendored | self-contained | static"); + return 2; + } + } + // `value()`, not `option_or_empty()`, for `profile` — and NOT for // anything whose name a positional shares. `ParsedArgs::value()` falls back // to a same-named positional when the option is unset, which is how @@ -562,7 +714,29 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, // form is not a shared object. std::vector triples; if (auto o = parsed.option("target")) triples = o->get().values; + // `--linkage static` OF A LINUX PROGRAM IS THE MUSL ROW (SPEC-014 §1): a + // program with no shared library at all, the C library's included, which + // is what `--mode static` has always built. On another system the + // libraries of the graph are linked in and the system's stay its own. + if (opts.linkage == mcpp::pack::Linkage::Static) { + if (triples.empty()) { + if constexpr (mcpplibs::os::is_linux) + triples.push_back(std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch)); + } else { + for (auto& t : triples) + if (auto at = t.find("-linux-gnu"); at != std::string::npos) + t.replace(at, 10, "-linux-musl"); + } + } if (!triples.empty()) opts.targetTriple = triples.back(); + // A program with no shared library is packed whole whatever the mode + // asked: there is nothing beside it to vendor, and nothing for the machine + // to provide. + if (opts.linkage == mcpp::pack::Linkage::Static + && opts.targetTriple.find("-linux-musl") != std::string::npos) { + opts.mode = mcpp::pack::Mode::Static; + modeFromUser = true; + } // ─── Several members ───────────────────────────────────────────── // diff --git a/src/pack/compress.cppm b/src/pack/compress.cppm new file mode 100644 index 000000000..a32baa5eb --- /dev/null +++ b/src/pack/compress.cppm @@ -0,0 +1,185 @@ +// mcpp.pack.compress — the compression of a package's container (SPEC-014 §5). +// +// `mcpp pack` writes an archive in one of three compressions or none, chosen by +// `--compress [:]` or `[pack] compress`: gzip (the default of +// `tar`, as every earlier release wrote it), zstd and xz. A zip's members are +// deflated or stored; a single file's payload is zstd. The codecs are the +// reference libraries (zlib, zstd, liblzma), dependencies of mcpp, so a +// package is written the same way on every host and nothing is looked for on +// PATH. +// +// Deterministic by construction: a gzip member carries no name and no time, a +// zstd frame states its content size and its checksum, an xz stream checks +// with CRC64. Two packs of one tree produce the same bytes. + +export module mcpp.pack.compress; + +import std; + +export namespace mcpp::pack::compress { + +enum class Algorithm { None, Gzip, Zstd, Xz }; + +struct Choice { + Algorithm algorithm = Algorithm::Gzip; + int level = 0; // 0: the codec's own default + bool stated = false; // written by the user (flag or manifest) +}; + +// `none`, `gzip`, `zstd`, `xz`, each optionally `:` (gzip 1-9, zstd +// 1-22, xz 0-9). +std::expected parse(std::string_view text); + +std::string_view name(Algorithm a); + +// The extension of a tar archive under `a`: `.tar.gz`, `.tar.zst`, `.tar.xz`, +// `.tar`. +std::string_view tar_extension(Algorithm a); + +// The whole of `in`, compressed to `out` under `c` (a gzip member, a zstd +// frame, an xz stream; a copy for None). +std::expected file(const std::filesystem::path& in, + const std::filesystem::path& out, Choice c); + +// `in`, compressed under `c`. +std::expected bytes(std::string_view in, Choice c); + +// `in` as raw DEFLATE (a zip member's method 8), at `level` (0: default). +std::expected deflate_raw(std::string_view in, int level); + +// A zstd frame's content, for the readers of what mcpp wrote (the tests, and +// `--merge`, which reads a package it wrote). +std::expected zstd_decompress(std::string_view frame); + +// What `bytes` compressed under `a`: a gzip member, a zstd frame, an xz +// stream, or the bytes themselves. `--merge` reads the archives it merges so. +std::expected decompress(std::string_view in, Algorithm a); + +// A raw DEFLATE stream's content, `size` bytes long (a zip member's method 8). +std::expected inflate_raw(std::string_view in, std::size_t size); + +} // namespace mcpp::pack::compress + +// The codecs themselves, in compress_codecs.cpp: the only unit that sees the +// libraries' headers. +namespace mcpp::pack::compress::codec { +std::expected gzip(std::string_view in, int level); +std::expected zstd(std::string_view in, int level); +std::expected xz(std::string_view in, int level); +std::expected deflate(std::string_view in, int level); +std::expected unzstd(std::string_view frame); +std::expected gunzip(std::string_view in); +std::expected unxz(std::string_view in); +std::expected inflate(std::string_view in, std::size_t size); +} // namespace mcpp::pack::compress::codec + +namespace mcpp::pack::compress { + +std::string_view name(Algorithm a) { + switch (a) { + case Algorithm::None: return "none"; + case Algorithm::Gzip: return "gzip"; + case Algorithm::Zstd: return "zstd"; + case Algorithm::Xz: return "xz"; + } + return "gzip"; +} + +std::string_view tar_extension(Algorithm a) { + switch (a) { + case Algorithm::None: return ".tar"; + case Algorithm::Gzip: return ".tar.gz"; + case Algorithm::Zstd: return ".tar.zst"; + case Algorithm::Xz: return ".tar.xz"; + } + return ".tar.gz"; +} + +std::expected parse(std::string_view text) { + Choice c; + c.stated = true; + std::string_view alg = text; + std::string_view lvl; + if (auto colon = text.find(':'); colon != std::string_view::npos) { + alg = text.substr(0, colon); + lvl = text.substr(colon + 1); + } + struct Row { std::string_view word; Algorithm a; int lo, hi; }; + constexpr Row rows[] = { + {"none", Algorithm::None, 0, 0}, + {"gzip", Algorithm::Gzip, 1, 9}, + {"zstd", Algorithm::Zstd, 1, 22}, + {"xz", Algorithm::Xz, 0, 9}, + }; + const Row* row = nullptr; + for (auto const& r : rows) if (r.word == alg) row = &r; + if (!row) + return std::unexpected(std::format( + "compression '{}' is not one of none | gzip | zstd | xz (each with an optional " + ":)", alg)); + c.algorithm = row->a; + if (!lvl.empty()) { + if (row->a == Algorithm::None) + return std::unexpected(std::string("compression 'none' takes no level")); + int v = 0; + auto [p, ec] = std::from_chars(lvl.data(), lvl.data() + lvl.size(), v); + if (ec != std::errc{} || p != lvl.data() + lvl.size() || v < row->lo || v > row->hi) + return std::unexpected(std::format( + "the level of {} is {} to {}; '{}' is not", row->word, row->lo, row->hi, lvl)); + c.level = v; + } + return c; +} + +std::expected bytes(std::string_view in, Choice c) { + switch (c.algorithm) { + case Algorithm::None: return std::string(in); + case Algorithm::Gzip: return codec::gzip(in, c.level); + case Algorithm::Zstd: return codec::zstd(in, c.level); + case Algorithm::Xz: return codec::xz(in, c.level); + } + return std::string(in); +} + +std::expected file(const std::filesystem::path& in, + const std::filesystem::path& out, Choice c) { + std::string data; + { + std::ifstream is(in, std::ios::binary); + if (!is) return std::unexpected(std::format("cannot read '{}'", in.string())); + data.assign(std::istreambuf_iterator(is), {}); + } + auto packed = bytes(data, c); + if (!packed) return std::unexpected(std::format("{}: {}", in.string(), packed.error())); + std::error_code ec; + if (out.has_parent_path()) std::filesystem::create_directories(out.parent_path(), ec); + std::ofstream os(out, std::ios::binary | std::ios::trunc); + if (!os) return std::unexpected(std::format("cannot write '{}'", out.string())); + os.write(packed->data(), static_cast(packed->size())); + if (!os) return std::unexpected(std::format("failed writing '{}'", out.string())); + return {}; +} + +std::expected deflate_raw(std::string_view in, int level) { + return codec::deflate(in, level); +} + +std::expected zstd_decompress(std::string_view frame) { + return codec::unzstd(frame); +} + +std::expected decompress(std::string_view in, Algorithm a) { + switch (a) { + case Algorithm::None: return std::string(in); + case Algorithm::Gzip: return codec::gunzip(in); + case Algorithm::Zstd: return codec::unzstd(in); + case Algorithm::Xz: return codec::unxz(in); + } + return std::string(in); +} + +std::expected inflate_raw(std::string_view in, std::size_t size) { + return codec::inflate(in, size); +} + +} // namespace mcpp::pack::compress diff --git a/src/pack/compress_codecs.cpp b/src/pack/compress_codecs.cpp new file mode 100644 index 000000000..93ca65883 --- /dev/null +++ b/src/pack/compress_codecs.cpp @@ -0,0 +1,154 @@ +// The codecs of mcpp.pack.compress: zlib (gzip, raw deflate), zstd and liblzma +// (xz), each through its one-shot interface. The only unit that includes the +// libraries' headers. +module; +#include +#include +#include + +module mcpp.pack.compress; + +import std; + +namespace mcpp::pack::compress::codec { + +namespace { + +// zlib's deflate with `windowBits`: 15 + 16 is a gzip member (no name, no +// time: deflate writes neither unless a header is set), -15 is raw DEFLATE. +std::expected zlib_deflate(std::string_view in, int level, + int windowBits) { + z_stream s{}; + const int lvl = level == 0 ? Z_DEFAULT_COMPRESSION : level; + if (deflateInit2(&s, lvl, Z_DEFLATED, windowBits, 8, Z_DEFAULT_STRATEGY) != Z_OK) + return std::unexpected(std::string("zlib: deflateInit2 failed")); + std::string out; + out.resize(deflateBound(&s, static_cast(in.size())) + 64); + s.next_in = reinterpret_cast(const_cast(in.data())); + s.avail_in = static_cast(in.size()); + s.next_out = reinterpret_cast(out.data()); + s.avail_out = static_cast(out.size()); + const int rc = ::deflate(&s, Z_FINISH); + const auto written = s.total_out; + deflateEnd(&s); + if (rc != Z_STREAM_END) return std::unexpected(std::format("zlib: deflate returned {}", rc)); + out.resize(written); + return out; +} + +} // namespace + +std::expected gzip(std::string_view in, int level) { + if (in.size() > std::numeric_limits::max()) + return std::unexpected(std::string("gzip: an input over 4 GiB")); + return zlib_deflate(in, level, 15 + 16); +} + +std::expected deflate(std::string_view in, int level) { + if (in.size() > std::numeric_limits::max()) + return std::unexpected(std::string("deflate: an input over 4 GiB")); + return zlib_deflate(in, level, -15); +} + +std::expected zstd(std::string_view in, int level) { + ZSTD_CCtx* cx = ZSTD_createCCtx(); + if (!cx) return std::unexpected(std::string("zstd: cannot create a context")); + ZSTD_CCtx_setParameter(cx, ZSTD_c_compressionLevel, level == 0 ? 19 : level); + ZSTD_CCtx_setParameter(cx, ZSTD_c_checksumFlag, 1); + ZSTD_CCtx_setParameter(cx, ZSTD_c_contentSizeFlag, 1); + std::string out; + out.resize(ZSTD_compressBound(in.size())); + const std::size_t n = ZSTD_compress2(cx, out.data(), out.size(), in.data(), in.size()); + ZSTD_freeCCtx(cx); + if (ZSTD_isError(n)) return std::unexpected(std::format("zstd: {}", ZSTD_getErrorName(n))); + out.resize(n); + return out; +} + +std::expected unzstd(std::string_view frame) { + const auto size = ZSTD_getFrameContentSize(frame.data(), frame.size()); + if (size == ZSTD_CONTENTSIZE_ERROR || size == ZSTD_CONTENTSIZE_UNKNOWN) + return std::unexpected(std::string("zstd: the frame does not state its content size")); + std::string out; + out.resize(static_cast(size)); + const std::size_t n = ZSTD_decompress(out.data(), out.size(), frame.data(), frame.size()); + if (ZSTD_isError(n)) return std::unexpected(std::format("zstd: {}", ZSTD_getErrorName(n))); + out.resize(n); + return out; +} + +std::expected xz(std::string_view in, int level) { + std::string out; + out.resize(lzma_stream_buffer_bound(in.size())); + std::size_t pos = 0; + const lzma_ret rc = lzma_easy_buffer_encode( + static_cast(level == 0 ? 6 : level), LZMA_CHECK_CRC64, nullptr, + reinterpret_cast(in.data()), in.size(), + reinterpret_cast(out.data()), &pos, out.size()); + if (rc != LZMA_OK) return std::unexpected(std::format("xz: lzma_easy_buffer_encode returned {}", + static_cast(rc))); + out.resize(pos); + return out; +} + +namespace { + +// zlib's inflate until the stream ends, growing the output: a gzip member +// (windowBits 15 + 32 detects the header) or raw DEFLATE (-15). +std::expected zlib_inflate(std::string_view in, int windowBits, + std::size_t hint) { + z_stream s{}; + if (inflateInit2(&s, windowBits) != Z_OK) + return std::unexpected(std::string("zlib: inflateInit2 failed")); + std::string out; + out.resize(hint > 0 ? hint : in.size() * 4 + 1024); + s.next_in = reinterpret_cast(const_cast(in.data())); + s.avail_in = static_cast(in.size()); + int rc = Z_OK; + while (rc == Z_OK) { + if (s.total_out >= out.size()) out.resize(out.size() * 2); + s.next_out = reinterpret_cast(out.data() + s.total_out); + s.avail_out = static_cast(out.size() - s.total_out); + rc = ::inflate(&s, Z_NO_FLUSH); + } + const auto written = s.total_out; + inflateEnd(&s); + if (rc != Z_STREAM_END) return std::unexpected(std::format("zlib: inflate returned {}", rc)); + out.resize(written); + return out; +} + +} // namespace + +std::expected gunzip(std::string_view in) { + return zlib_inflate(in, 15 + 32, 0); +} + +std::expected inflate(std::string_view in, std::size_t size) { + return zlib_inflate(in, -15, size + 1); +} + +std::expected unxz(std::string_view in) { + lzma_stream s = LZMA_STREAM_INIT; + if (lzma_stream_decoder(&s, UINT64_MAX, 0) != LZMA_OK) + return std::unexpected(std::string("xz: cannot start a decoder")); + std::string out; + out.resize(in.size() * 4 + 1024); + s.next_in = reinterpret_cast(in.data()); + s.avail_in = in.size(); + lzma_ret rc = LZMA_OK; + while (rc == LZMA_OK) { + if (s.total_out >= out.size()) out.resize(out.size() * 2); + s.next_out = reinterpret_cast(out.data() + s.total_out); + s.avail_out = out.size() - s.total_out; + rc = lzma_code(&s, LZMA_FINISH); + } + const auto written = s.total_out; + lzma_end(&s); + if (rc != LZMA_STREAM_END) + return std::unexpected(std::format("xz: lzma_code returned {}", static_cast(rc))); + out.resize(written); + return out; +} + +} // namespace mcpp::pack::compress::codec diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm new file mode 100644 index 000000000..66a0db2dc --- /dev/null +++ b/src/pack/distribution.cppm @@ -0,0 +1,963 @@ +// mcpp.pack.distribution — what a package is beyond its staged files +// (SPEC-014 §4, §5): the containers, the package's metadata +// (`mcpp-package.json`), the launchers of a universal package, the payload and +// the launcher program of a single file, and `--merge`. +// +// THE ENGINE KNOWS NO PLATFORM HERE. A launcher is generated from the +// platform's `runner` and the legs; a loader of a self-contained universal +// package is the platform's carrier with nothing carried; a single file's +// launcher is the engine's own program, written below as source and built for +// the target at pack time like any program. + +export module mcpp.pack.distribution; + +import std; +import mcpp.libs.json; +import mcpp.pack.compress; +import mcpp.pack.zip; +import mcpp.run; +import mcpplibs.os.platform; + +export namespace mcpp::pack::distribution { + +// ─── digests ────────────────────────────────────────────────────────────── + +// SHA-256 of `bytes`, of a file's content, in lowercase hexadecimal. +std::string sha256(std::string_view bytes); +std::string sha256_file(const std::filesystem::path& file); + +// ─── containers ─────────────────────────────────────────────────────────── + +// `root` (a staged tree) as a tar whose one top-level directory is +// `root.filename()`, compressed as `c` says. +std::expected write_tar(const std::filesystem::path& root, + const std::filesystem::path& archive, + mcpp::pack::compress::Choice c); +// The same as a zip: members deflated unless `c` is `none`, links kept, a +// file executable as its mode says or when it is named `program`. +std::expected write_zip(const std::filesystem::path& root, + const std::filesystem::path& archive, + mcpp::pack::compress::Choice c, + std::string_view program = {}); + +// A package this command wrote, as a tree under `into`: a directory as it is, +// an archive (`.zip`, `.tar`, `.tar.gz`, `.tgz`, `.tar.zst`, `.tar.xz`) +// extracted. Returns the tree's root, the archive's one top-level directory. +std::expected +unpack(const std::filesystem::path& package, const std::filesystem::path& into); + +// ─── launchers (SPEC-014 §4.3) ──────────────────────────────────────────── + +struct Launch { + std::string name; // bin/, bin/.cmd + std::vector runner; // the platform's `runner.cmd`, `{}` the image + std::map entry; // arch -> the entry image's file name + // The loader a self-contained package carries under + // `libexec//-/`; empty when it carries none. + std::string loader; +}; + +std::string posix_launcher(const Launch& l); +std::string windows_launcher(const Launch& l); + +// The architecture directory a machine name selects, as the launchers map it. +std::string_view arch_dir(std::string_view machine); + +// ─── mcpp-package.json (SPEC-014 §4.5) ──────────────────────────────────── + +inline constexpr std::string_view kFileName = "mcpp-package.json"; + +struct File { + std::string path; // relative to the tree's root, `/`-separated + std::string role; // program | image | library | loader | launcher | resource | metadata + std::string leg; // the leg's triple; empty: shared by every leg + std::string sha256; +}; + +struct Package { + int schema = 1; + std::string name, version; + std::string platform; // ".@"; empty: none + std::string linkage, mode; + std::vector legs; // canonical triples + std::vector systems; // `-` of the loaders under libexec/ + std::vector runner; // empty for a native package + std::map entry; // arch -> path of the entry (universal) + std::string program; // the program's path (one native leg) + std::vector files; +}; + +// What a caller knows of a file's role and leg; every other file's role is +// read from where it lies (`libexec/` a loader, `lib/` a library, ...). +struct Stated { + std::map role; // path -> role + std::map leg; // path -> leg +}; + +// Every file of the tree described, digested, and the document written at +// `root/mcpp-package.json`. +std::expected describe(const std::filesystem::path& root, Package pkg, + const Stated& stated); +std::string to_json(const Package& p); +std::expected read(const std::filesystem::path& root); + +// The `-` directories of `root/libexec//`: the systems whose +// loaders a self-contained universal package carries, sorted. +std::vector loader_systems(const std::filesystem::path& root); + +// ─── a single file (SPEC-014 §5.2) ──────────────────────────────────────── + +// The tree under `root` as the payload a single file's launcher extracts: the +// record stream `MCPPTREE1\n` then, per entry, a kind byte (`d` directory, +// `f` file, `x` executable file, `l` link), a 32-bit path length, the path +// (`/`-separated), a 64-bit data length and the data (a link's data is its +// target). Little-endian; entries in path order. +std::expected tree_payload(const std::filesystem::path& root); + +// The launcher program's sources: its manifest and its entry. It reads the +// carried payload (`mcpp_carried_begin` .. `mcpp_carried_end`, a zstd frame +// of `tree_payload`), extracts it once into the user's cache under the +// payload's sha256 (`mcpp_carried_digest`), and executes the program at +// `mcpp_carried_name` with its own arguments. +std::string launcher_manifest(std::string_view name, std::string_view version); +std::string_view launcher_main(); + +// ─── --merge (SPEC-014 §4.6) ────────────────────────────────────────────── + +struct MergeRequest { + std::vector inputs; + std::filesystem::path output; // empty: ./ + std::string format; // tar | zip | dir; empty: the first input's + mcpp::pack::compress::Choice compression; +}; + +std::expected merge(const MergeRequest& req); + +} // namespace mcpp::pack::distribution + +namespace mcpp::pack::distribution { + +namespace fs = std::filesystem; + +// ─── SHA-256 (FIPS 180-4) ───────────────────────────────────────────────── + +namespace { + +struct Sha256 { + std::array h{0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, + 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19}; + std::array block{}; + std::size_t used = 0; + std::uint64_t bits = 0; + + static std::uint32_t rotr(std::uint32_t x, int n) { return (x >> n) | (x << (32 - n)); } + + void compress() { + static constexpr std::uint32_t k[64] = { + 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, + 0xab1c5ed5, 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, + 0x9bdc06a7, 0xc19bf174, 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, + 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, + 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, + 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, 0xa2bfe8a1, 0xa81a664b, + 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, 0x19a4c116, + 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, + 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, + 0xc67178f2}; + std::uint32_t w[64]; + for (int i = 0; i < 16; ++i) + w[i] = (std::uint32_t(block[4 * i]) << 24) | (std::uint32_t(block[4 * i + 1]) << 16) + | (std::uint32_t(block[4 * i + 2]) << 8) | std::uint32_t(block[4 * i + 3]); + for (int i = 16; i < 64; ++i) { + const auto s0 = rotr(w[i - 15], 7) ^ rotr(w[i - 15], 18) ^ (w[i - 15] >> 3); + const auto s1 = rotr(w[i - 2], 17) ^ rotr(w[i - 2], 19) ^ (w[i - 2] >> 10); + w[i] = w[i - 16] + s0 + w[i - 7] + s1; + } + auto [a, b, c, d, e, f, g, hh] = h; + for (int i = 0; i < 64; ++i) { + const auto t1 = hh + (rotr(e, 6) ^ rotr(e, 11) ^ rotr(e, 25)) + ((e & f) ^ (~e & g)) + + k[i] + w[i]; + const auto t2 = (rotr(a, 2) ^ rotr(a, 13) ^ rotr(a, 22)) + ((a & b) ^ (a & c) ^ (b & c)); + hh = g; g = f; f = e; e = d + t1; d = c; c = b; b = a; a = t1 + t2; + } + h[0] += a; h[1] += b; h[2] += c; h[3] += d; h[4] += e; h[5] += f; h[6] += g; h[7] += hh; + } + + void update(std::string_view data) { + for (unsigned char ch : data) { + block[used++] = ch; + if (used == 64) { compress(); used = 0; } + } + bits += static_cast(data.size()) * 8; + } + + std::string hex() { + const auto total = bits; + block[used++] = 0x80; + if (used > 56) { while (used < 64) block[used++] = 0; compress(); used = 0; } + while (used < 56) block[used++] = 0; + for (int i = 7; i >= 0; --i) block[used++] = static_cast(total >> (8 * i)); + compress(); + std::string out; + for (auto v : h) out += std::format("{:08x}", v); + return out; + } +}; + +std::optional slurp(const fs::path& p) { + std::ifstream in(p, std::ios::binary); + if (!in) return std::nullopt; + return std::string(std::istreambuf_iterator(in), {}); +} + +bool spit(const fs::path& p, std::string_view bytes) { + std::error_code ec; + if (p.has_parent_path()) fs::create_directories(p.parent_path(), ec); + std::ofstream out(p, std::ios::binary | std::ios::trunc); + out.write(bytes.data(), static_cast(bytes.size())); + return bool(out); +} + +bool executable_file(const fs::path& p) { + std::error_code ec; + const auto perms = fs::status(p, ec).permissions(); + if (ec) return false; + return (perms & fs::perms::owner_exec) != fs::perms::none; +} + +} // namespace + +std::string sha256(std::string_view bytes) { + Sha256 s; + s.update(bytes); + return s.hex(); +} + +std::string sha256_file(const fs::path& file) { + std::ifstream in(file, std::ios::binary); + if (!in) return {}; + Sha256 s; + std::array buf; + while (in) { + in.read(buf.data(), buf.size()); + if (auto n = in.gcount(); n > 0) s.update(std::string_view(buf.data(), static_cast(n))); + } + return s.hex(); +} + +// ─── containers ─────────────────────────────────────────────────────────── + +std::expected write_tar(const fs::path& root, const fs::path& archive, + mcpp::pack::compress::Choice c) { + std::error_code ec; + if (archive.has_parent_path()) fs::create_directories(archive.parent_path(), ec); + // Written uncompressed by tar, compressed by mcpp's codec: `.tar.zst` and + // `.tar.xz` need no program on PATH, and a `.tar.gz` is the same bytes on + // every host. `--force-local` on Windows: GNU tar reads `C:/x.tar` as the + // rsh form `host:path`. + auto plain = archive; + plain += ".mcpp-tar"; + std::vector argv{"tar"}; + if constexpr (mcpplibs::os::is_windows) argv.push_back("--force-local"); + argv.insert(argv.end(), {"-cf", plain.string(), "-C", root.parent_path().string(), + root.filename().string()}); + auto r = mcpp::run::execute(argv); + if (r.exit_code != 0) { + fs::remove(plain, ec); + return std::unexpected(std::format("tar exited with {}: {}", r.exit_code, r.output)); + } + auto packed = mcpp::pack::compress::file(plain, archive, c); + fs::remove(plain, ec); + if (!packed) return std::unexpected(packed.error()); + return {}; +} + +std::expected write_zip(const fs::path& root, const fs::path& archive, + mcpp::pack::compress::Choice c, + std::string_view program) { + std::error_code ec; + std::vector entries; + const auto wrapper = root.filename().string(); + for (auto it = fs::recursive_directory_iterator(root, ec); + !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { + const auto& e = *it; + mcpp::pack::zip::Entry entry; + entry.name = wrapper + "/" + e.path().lexically_relative(root).generic_string(); + entry.source = e.path(); + if (e.is_symlink(ec)) { + entry.symlinkTarget = fs::read_symlink(e.path(), ec).generic_string(); + if (entry.symlinkTarget.empty()) continue; + if (e.is_directory(ec)) it.disable_recursion_pending(); + } else if (!e.is_regular_file(ec)) { + continue; + } else { + entry.executable = executable_file(e.path()) + || (!program.empty() && e.path().filename() == program); + } + entries.push_back(std::move(entry)); + } + // Deterministic order; a comparator rather than a projection onto an + // imported type's member (mcpp.pack's run_pe says why). + std::sort(entries.begin(), entries.end(), + [](const mcpp::pack::zip::Entry& a, const mcpp::pack::zip::Entry& b) { + return a.name < b.name; + }); + mcpp::pack::zip::WriteOptions o; + o.deflate = c.algorithm != mcpp::pack::compress::Algorithm::None; + o.level = c.algorithm == mcpp::pack::compress::Algorithm::Gzip ? c.level : 0; + return mcpp::pack::zip::write(archive, entries, o); +} + +namespace { + +std::uint16_t get16(std::string_view b, std::size_t at) { + return static_cast(static_cast(b[at]) + | (static_cast(b[at + 1]) << 8)); +} +std::uint32_t get32(std::string_view b, std::size_t at) { + return static_cast(get16(b, at)) | (static_cast(get16(b, at + 2)) << 16); +} + +// A relative, `/`-separated name with no `..`: what an archive may create. +bool safe_member(std::string_view name) { + if (name.empty() || name.front() == '/' || name.find('\\') != std::string_view::npos + || (name.size() > 1 && name[1] == ':')) + return false; + std::size_t at = 0; + while (at <= name.size()) { + auto slash = name.find('/', at); + if (slash == std::string_view::npos) slash = name.size(); + if (name.substr(at, slash - at) == "..") return false; + at = slash + 1; + } + return true; +} + +// The members of a zip, read through its central directory (stored and +// deflated, as mcpp writes them). +std::expected unzip(std::string_view b, const fs::path& into) { + std::size_t eocd = std::string_view::npos; + for (std::size_t i = b.size() >= 22 ? b.size() - 22 : 0; ; --i) { + if (get32(b, i) == 0x06054b50) { eocd = i; break; } + if (i == 0 || b.size() - i > 22 + 65535) break; + } + if (eocd == std::string_view::npos) return std::unexpected(std::string("not a zip: no end of central directory")); + const auto count = get16(b, eocd + 10); + std::size_t at = get32(b, eocd + 16); + for (std::uint16_t n = 0; n < count; ++n) { + if (at + 46 > b.size() || get32(b, at) != 0x02014b50) + return std::unexpected(std::string("a zip's central directory is malformed")); + const auto method = get16(b, at + 10); + const auto packed = get32(b, at + 20); + const auto size = get32(b, at + 24); + const auto nameLen = get16(b, at + 28), extraLen = get16(b, at + 30), commentLen = get16(b, at + 32); + const auto attrs = get32(b, at + 38); + const auto local = get32(b, at + 42); + const std::string name(b.substr(at + 46, nameLen)); + at += 46 + nameLen + extraLen + commentLen; + if (!safe_member(name)) return std::unexpected(std::format("a zip member names '{}'", name)); + if (name.ends_with('/')) { std::error_code ec; fs::create_directories(into / name, ec); continue; } + const auto lnameLen = get16(b, local + 26), lextraLen = get16(b, local + 28); + const auto dataAt = static_cast(local) + 30 + lnameLen + lextraLen; + if (dataAt + packed > b.size()) return std::unexpected(std::format("zip member '{}' is truncated", name)); + std::string data(b.substr(dataAt, packed)); + if (method == 8) { + auto d = mcpp::pack::compress::inflate_raw(data, size); + if (!d) return std::unexpected(std::format("zip member '{}': {}", name, d.error())); + data = std::move(*d); + } else if (method != 0) { + return std::unexpected(std::format("zip member '{}' uses method {}", name, method)); + } + const auto mode = attrs >> 16; + const auto target = into / fs::path(name); + std::error_code ec; + fs::create_directories(target.parent_path(), ec); + if ((mode & 0170000u) == 0120000u) { + fs::create_symlink(data, target, ec); + if (ec) return std::unexpected(std::format("cannot link '{}': {}", target.string(), ec.message())); + continue; + } + if (!spit(target, data)) return std::unexpected(std::format("cannot write '{}'", target.string())); + if (mode & 0100u) + fs::permissions(target, fs::perms::owner_exec | fs::perms::group_exec | fs::perms::others_exec, + fs::perm_options::add, ec); + } + return {}; +} + +} // namespace + +std::expected unpack(const fs::path& package, const fs::path& into) { + std::error_code ec; + if (fs::is_directory(package, ec)) return package; + if (!fs::is_regular_file(package, ec)) + return std::unexpected(std::format("'{}' is neither a package directory nor an archive", + package.string())); + fs::create_directories(into, ec); + const auto name = package.filename().string(); + auto bytes = slurp(package); + if (!bytes) return std::unexpected(std::format("cannot read '{}'", package.string())); + if (name.ends_with(".zip")) { + if (auto r = unzip(*bytes, into); !r) return std::unexpected(std::format("{}: {}", name, r.error())); + } else { + using mcpp::pack::compress::Algorithm; + Algorithm a = name.ends_with(".tar.gz") || name.ends_with(".tgz") ? Algorithm::Gzip + : name.ends_with(".tar.zst") ? Algorithm::Zstd + : name.ends_with(".tar.xz") ? Algorithm::Xz + : name.ends_with(".tar") ? Algorithm::None + : static_cast(-1); + if (static_cast(a) == -1) + return std::unexpected(std::format( + "'{}' is not a package this command writes (.zip, .tar, .tar.gz, .tar.zst, .tar.xz, " + "or a directory)", name)); + auto plain = mcpp::pack::compress::decompress(*bytes, a); + if (!plain) return std::unexpected(std::format("{}: {}", name, plain.error())); + const auto tarFile = into / ".mcpp-merge.tar"; + if (!spit(tarFile, *plain)) return std::unexpected(std::format("cannot write '{}'", tarFile.string())); + std::vector argv{"tar"}; + if constexpr (mcpplibs::os::is_windows) argv.push_back("--force-local"); + argv.insert(argv.end(), {"-xf", tarFile.string(), "-C", into.string()}); + auto r = mcpp::run::execute(argv); + fs::remove(tarFile, ec); + if (r.exit_code != 0) return std::unexpected(std::format("tar -xf {} exited with {}: {}", name, + r.exit_code, r.output)); + } + // The archive's one top-level directory. + std::vector tops; + for (auto const& e : fs::directory_iterator(into, ec)) + if (e.is_directory(ec)) tops.push_back(e.path()); + if (tops.size() != 1) + return std::unexpected(std::format("'{}' has {} top-level directories; a package has one", + name, tops.size())); + return tops.front(); +} + +// ─── launchers ──────────────────────────────────────────────────────────── + +std::string_view arch_dir(std::string_view machine) { + if (machine == "x86_64" || machine == "amd64" || machine == "AMD64") return "x86_64"; + if (machine == "aarch64" || machine == "arm64" || machine == "ARM64") return "aarch64"; + return {}; +} + +namespace { + +// A word of a POSIX shell command, single-quoted. +std::string sh_word(std::string_view w) { + std::string out = "'"; + for (char ch : w) { + if (ch == '\'') out += "'\\''"; + else out.push_back(ch); + } + out += "'"; + return out; +} + +} // namespace + +std::string posix_launcher(const Launch& l) { + std::string arches; + for (auto const& [arch, _] : l.entry) arches += (arches.empty() ? "" : " ") + arch; + std::string cases; + for (auto const& [arch, file] : l.entry) + cases += std::format(" {}) image=\"$top/lib/{}/{}\" ;;\n", arch, arch, file); + // The runner's argv: its program found in the package, then on PATH; `{}` + // the image; every other word as written. + std::string args; + for (std::size_t i = 1; i < l.runner.size(); ++i) + args += l.runner[i] == "{}" ? std::string(" \"$image\"") : " " + sh_word(l.runner[i]); + const std::string program = l.runner.empty() ? std::string("{}") : l.runner.front(); + std::string find; + if (!l.loader.empty()) + find = std::format( + "runner=\"$top/libexec/{0}/$os-$arch/{0}\"\n" + "[ -x \"$runner\" ] || runner={1}\n", l.loader, sh_word(program)); + else + find = std::format("runner={}\n", sh_word(program)); + return std::format( + "#!/bin/sh\n" + "# Generated by mcpp pack (SPEC-014 §4.3): runs {0} on this machine's\n" + "# architecture through the platform's runner.\n" + "top=$(CDPATH= cd -- \"$(dirname -- \"$0\")/..\" && pwd)\n" + "case \"$(uname -m)\" in\n" + " x86_64|amd64|AMD64) arch=x86_64 ;;\n" + " aarch64|arm64|ARM64) arch=aarch64 ;;\n" + " *) arch=$(uname -m) ;;\n" + "esac\n" + "case \"$(uname -s)\" in\n" + " Linux) os=linux ;;\n" + " Darwin) os=macos ;;\n" + " *) os=$(uname -s) ;;\n" + "esac\n" + "case \"$arch\" in\n" + "{1}" + " *) echo \"{0}: this machine is $arch; the package holds {2}\" >&2; exit 126 ;;\n" + "esac\n" + "{3}" + "exec \"$runner\"{4} \"$@\"\n", + l.name, cases, arches, find, args); +} + +std::string windows_launcher(const Launch& l) { + std::string arches; + for (auto const& [arch, _] : l.entry) arches += (arches.empty() ? "" : " ") + arch; + std::string picks; + for (auto const& [arch, file] : l.entry) + picks += std::format("if \"%arch%\"==\"{0}\" set \"image=%top%\\lib\\{0}\\{1}\"\r\n", arch, file); + std::string args; + for (std::size_t i = 1; i < l.runner.size(); ++i) + args += l.runner[i] == "{}" ? std::string(" \"%image%\"") : " \"" + l.runner[i] + "\""; + const std::string program = l.runner.empty() ? std::string("{}") : l.runner.front(); + std::string find = std::format("set \"runner={}\"\r\n", program); + if (!l.loader.empty()) + find = std::format( + "set \"runner=%top%\\libexec\\{0}\\windows-%arch%\\{0}.exe\"\r\n" + "if not exist \"%runner%\" set \"runner={1}\"\r\n", l.loader, program); + return std::format( + "@echo off\r\n" + "rem Generated by mcpp pack (SPEC-014 §4.3): runs {0} on this machine's\r\n" + "rem architecture through the platform's runner.\r\n" + "setlocal\r\n" + "for %%I in (\"%~dp0..\") do set \"top=%%~fI\"\r\n" + "set \"machine=%PROCESSOR_ARCHITECTURE%\"\r\n" + "if defined PROCESSOR_ARCHITEW6432 set \"machine=%PROCESSOR_ARCHITEW6432%\"\r\n" + "set \"arch=%machine%\"\r\n" + "if /i \"%machine%\"==\"AMD64\" set \"arch=x86_64\"\r\n" + "if /i \"%machine%\"==\"ARM64\" set \"arch=aarch64\"\r\n" + "set \"image=\"\r\n" + "{1}" + "if not defined image (\r\n" + " echo {0}: this machine is %arch%; the package holds {2} 1>&2\r\n" + " exit /b 126\r\n" + ")\r\n" + "{3}" + "\"%runner%\"{4} %*\r\n" + "exit /b %ERRORLEVEL%\r\n", + l.name, picks, arches, find, args); +} + +// ─── mcpp-package.json ──────────────────────────────────────────────────── + +namespace { + +std::string role_of(std::string_view path, bool executable) { + auto ends = [&](std::string_view s) { return path.ends_with(s); }; + const auto file = path.substr(path.rfind('/') == std::string_view::npos ? 0 : path.rfind('/') + 1); + if (path.starts_with("libexec/")) return "loader"; + if (file.starts_with("README") || file.starts_with("LICENSE") || file == "HOST-REQUIREMENTS" + || path.starts_with("share/licenses/")) + return "metadata"; + if (ends(".dll") || ends(".dylib") || path.find(".so") != std::string_view::npos + || ends(".okso")) + return "library"; + if (ends(".oke")) return "image"; + if (path.starts_with("bin/") && (executable || ends(".exe") || ends(".js") || ends(".wasm"))) + return "program"; + if (ends(".cmd") || ends(".sh")) return "launcher"; + // An executable at the root: the wrapper that starts the program. + if (executable && path.find('/') == std::string_view::npos) return "launcher"; + return "resource"; +} + +} // namespace + +std::string to_json(const Package& p) { + nlohmann::json files = nlohmann::json::array(); + for (auto const& f : p.files) + files.push_back({{"path", f.path}, {"role", f.role}, + {"leg", f.leg.empty() ? nlohmann::json(nullptr) : nlohmann::json(f.leg)}, + {"sha256", f.sha256}}); + nlohmann::json launch = nlohmann::json::object(); + if (!p.runner.empty()) { + launch["runner"] = p.runner; + nlohmann::json entry = nlohmann::json::object(); + for (auto const& [arch, path] : p.entry) entry[arch] = path; + launch["entry"] = std::move(entry); + } else { + launch["program"] = p.program.empty() ? nlohmann::json(nullptr) : nlohmann::json(p.program); + } + nlohmann::ordered_json j; + j["schema"] = p.schema; + j["name"] = p.name; + j["version"] = p.version; + j["platform"] = p.platform.empty() ? nlohmann::json(nullptr) : nlohmann::json(p.platform); + j["linkage"] = p.linkage; + j["mode"] = p.mode; + j["legs"] = p.legs; + j["systems"] = p.systems; + j["launch"] = std::move(launch); + j["files"] = std::move(files); + return j.dump(2) + "\n"; +} + +std::expected describe(const fs::path& root, Package pkg, const Stated& stated) { + std::error_code ec; + pkg.files.clear(); + for (auto it = fs::recursive_directory_iterator(root, ec); + !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { + const auto& e = *it; + if (e.is_symlink(ec)) { if (e.is_directory(ec)) it.disable_recursion_pending(); continue; } + if (!e.is_regular_file(ec)) continue; + const auto rel = e.path().lexically_relative(root).generic_string(); + if (rel == kFileName) continue; + File f; + f.path = rel; + if (auto r = stated.role.find(rel); r != stated.role.end()) f.role = r->second; + else f.role = role_of(rel, executable_file(e.path())); + if (auto l = stated.leg.find(rel); l != stated.leg.end()) f.leg = l->second; + else if (pkg.legs.size() == 1 && f.role != "metadata" && f.role != "resource") f.leg = pkg.legs.front(); + f.sha256 = sha256_file(e.path()); + pkg.files.push_back(std::move(f)); + } + std::sort(pkg.files.begin(), pkg.files.end(), + [](const File& a, const File& b) { return a.path < b.path; }); + if (!spit(root / kFileName, to_json(pkg))) + return std::unexpected(std::format("cannot write '{}'", (root / kFileName).string())); + return pkg; +} + +std::expected read(const fs::path& root) { + auto text = slurp(root / kFileName); + if (!text) return std::unexpected(std::format("'{}' has no {}", root.string(), kFileName)); + Package p; + try { + auto j = nlohmann::json::parse(*text); + p.schema = j.value("schema", 0); + p.name = j.value("name", std::string{}); + p.version = j.value("version", std::string{}); + if (j.contains("platform") && j["platform"].is_string()) p.platform = j["platform"].get(); + p.linkage = j.value("linkage", std::string{}); + p.mode = j.value("mode", std::string{}); + p.legs = j.value("legs", std::vector{}); + p.systems = j.value("systems", std::vector{}); + if (j.contains("launch")) { + auto const& l = j["launch"]; + if (l.contains("runner")) p.runner = l["runner"].get>(); + if (l.contains("entry") && l["entry"].is_object()) + for (auto it = l["entry"].begin(); it != l["entry"].end(); ++it) + p.entry[it.key()] = it.value().get(); + if (l.contains("program") && l["program"].is_string()) p.program = l["program"].get(); + } + for (auto const& f : j.value("files", nlohmann::json::array())) { + File x; + x.path = f.value("path", std::string{}); + x.role = f.value("role", std::string{}); + if (f.contains("leg") && f["leg"].is_string()) x.leg = f["leg"].get(); + x.sha256 = f.value("sha256", std::string{}); + p.files.push_back(std::move(x)); + } + } catch (const std::exception& e) { + return std::unexpected(std::format("{}: {}", (root / kFileName).string(), e.what())); + } + if (p.schema != 1) + return std::unexpected(std::format("{} states schema {}; this mcpp reads schema 1", + (root / kFileName).string(), p.schema)); + return p; +} + +std::vector loader_systems(const fs::path& root) { + std::vector out; + std::error_code ec; + const auto libexec = root / "libexec"; + if (!fs::is_directory(libexec, ec)) return out; + // One walk, two levels deep: `libexec//-`. + for (auto it = fs::recursive_directory_iterator(libexec, ec); + !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { + if (it.depth() < 1) continue; + it.disable_recursion_pending(); + if (it->is_directory(ec)) out.push_back(it->path().filename().string()); + } + std::ranges::sort(out); + out.erase(std::unique(out.begin(), out.end()), out.end()); + return out; +} + +// ─── a single file ──────────────────────────────────────────────────────── + +std::expected tree_payload(const fs::path& root) { + struct Item { std::string path; char kind; std::string data; }; + std::vector items; + std::error_code ec; + for (auto it = fs::recursive_directory_iterator(root, ec); + !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { + const auto& e = *it; + Item item; + item.path = e.path().lexically_relative(root).generic_string(); + if (e.is_symlink(ec)) { + item.kind = 'l'; + item.data = fs::read_symlink(e.path(), ec).generic_string(); + if (e.is_directory(ec)) it.disable_recursion_pending(); + } else if (e.is_directory(ec)) { + item.kind = 'd'; + } else if (e.is_regular_file(ec)) { + item.kind = executable_file(e.path()) ? 'x' : 'f'; + auto data = slurp(e.path()); + if (!data) return std::unexpected(std::format("cannot read '{}'", e.path().string())); + item.data = std::move(*data); + } else { + continue; + } + items.push_back(std::move(item)); + } + if (ec) return std::unexpected(std::format("cannot walk '{}': {}", root.string(), ec.message())); + std::sort(items.begin(), items.end(), [](const Item& a, const Item& b) { return a.path < b.path; }); + std::string out = "MCPPTREE1\n"; + auto put = [&](std::uint64_t v, int bytes) { + for (int i = 0; i < bytes; ++i) out.push_back(static_cast((v >> (8 * i)) & 0xff)); + }; + for (auto const& i : items) { + out.push_back(i.kind); + put(i.path.size(), 4); + out += i.path; + put(i.data.size(), 8); + out += i.data; + } + return out; +} + +std::string launcher_manifest(std::string_view name, std::string_view version) { + return std::format( + "# Generated by `mcpp pack --pkg-format single-file`: the launcher of {0}, which\n" + "# carries the package as a zstd frame and runs it from the user's cache.\n" + "[package]\nname = \"{0}\"\nversion = \"{1}\"\n\n" + "[targets.{0}]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n" + "[build]\nsources = [\"src/main.cpp\", \"src/carried.S\"]\n\n" + "[dependencies.mcpplibs]\nos = \"0.2.0\"\n\n" + "[dependencies.compat]\nzstd = \"1.5.7\"\n", + name, version.empty() ? std::string_view("0.0.0") : version); +} + +std::string_view launcher_main() { + return R"MCPP(// The launcher of a single file (SPEC-014 §5.2), written by mcpp pack. +// +// The package it runs is carried in this program as one zstd frame of a record +// stream (`mcpp_carried_begin` .. `mcpp_carried_end`). The first run extracts +// it into the user's cache, under the sha256 mcpp wrote (`mcpp_carried_digest`): +// into a name of its own beside the destination, renamed into place when every +// file is written, so a directory that is there is complete. Every later run +// executes the program at once. The frame carries its content's checksum, and +// zstd refuses a payload that does not match it. +#include + +import std; +import mcpplibs.os; + +extern "C" const unsigned char mcpp_carried_begin[]; +extern "C" const unsigned char mcpp_carried_end[]; +extern "C" const char mcpp_carried_name[]; +extern "C" const char mcpp_carried_digest[]; + +namespace { + +namespace fs = std::filesystem; +namespace os = mcpplibs::os; + +const std::string_view kName{mcpp_carried_name}; +const std::string_view kDigest{mcpp_carried_digest}; + +int fail(std::string_view what) { + std::cerr << std::format("{}: {}\n", kName, what); + return 127; +} + +bool safe(std::string_view name) { + if (name.empty() || name.front() == '/' || name.find('\\') != std::string_view::npos + || (name.size() > 1 && name[1] == ':')) + return false; + for (std::size_t at = 0; at <= name.size();) { + auto slash = name.find('/', at); + if (slash == std::string_view::npos) slash = name.size(); + if (name.substr(at, slash - at) == "..") return false; + at = slash + 1; + } + return true; +} + +std::optional extract(const fs::path& into) { + const auto* begin = reinterpret_cast(mcpp_carried_begin); + const auto size = static_cast(mcpp_carried_end - mcpp_carried_begin); + const auto content = ZSTD_getFrameContentSize(begin, size); + if (content == ZSTD_CONTENTSIZE_ERROR || content == ZSTD_CONTENTSIZE_UNKNOWN) + return "the carried payload is not a zstd frame"; + std::string tree(static_cast(content), '\0'); + const auto n = ZSTD_decompress(tree.data(), tree.size(), begin, size); + if (ZSTD_isError(n)) return std::format("the carried payload: {}", ZSTD_getErrorName(n)); + std::string_view t(tree); + if (!t.starts_with("MCPPTREE1\n")) return "the carried payload is not a tree"; + std::size_t at = 10; + auto get = [&](int bytes) { + std::uint64_t v = 0; + for (int i = 0; i < bytes; ++i) v |= std::uint64_t(static_cast(t[at + i])) << (8 * i); + at += static_cast(bytes); + return v; + }; + std::error_code ec; + while (at < t.size()) { + if (at + 13 > t.size()) return "the carried tree is truncated"; + const char kind = t[at++]; + const auto pathLen = get(4); + if (at + pathLen + 8 > t.size()) return "the carried tree is truncated"; + const std::string_view name = t.substr(at, pathLen); + at += pathLen; + const auto dataLen = get(8); + if (at + dataLen > t.size()) return "the carried tree is truncated"; + const std::string_view data = t.substr(at, dataLen); + at += dataLen; + if (!safe(name)) return std::format("the carried tree names '{}'", name); + const auto target = into / fs::path(std::string(name)); + fs::create_directories(target.parent_path(), ec); + if (kind == 'd') { fs::create_directories(target, ec); continue; } + if (kind == 'l') { + if (auto r = os::fs::create_link(fs::path(std::string(data)), target, false); !r) + return std::format("cannot link {}", target.string()); + continue; + } + std::ofstream out(target, std::ios::binary | std::ios::trunc); + out.write(data.data(), static_cast(data.size())); + if (!out) return std::format("cannot write {}", target.string()); + out.close(); + if (kind == 'x') (void)os::fs::set_executable(target, true); + } + std::ofstream(into / ".mcpp-complete") << kDigest << '\n'; + return std::nullopt; +} + +} // namespace + +int main(int argc, char** argv) { + auto cache = os::dirs::cache(); + if (!cache) return fail("this machine names no cache directory for its user"); + const fs::path base = *cache / "mcpp" / "single-file"; + const fs::path root = base / std::string(kDigest); + std::error_code ec; + if (!fs::exists(root / ".mcpp-complete", ec)) { + fs::create_directories(base, ec); + if constexpr (os::is_posix) + fs::permissions(base, fs::perms::owner_all, fs::perm_options::replace, ec); + const auto self = os::process::self(); + const fs::path tmp = base / std::format(".{}-{}", kDigest, + self ? self->id : std::uint64_t{0}); + fs::remove_all(tmp, ec); + if (auto why = extract(tmp)) { fs::remove_all(tmp, ec); return fail(*why); } + if (auto r = os::fs::rename(tmp, root, /*replace=*/false); !r) { + // Another run placed it first; that one is complete. + fs::remove_all(tmp, ec); + if (!fs::exists(root / ".mcpp-complete", ec)) + return fail(std::format("cannot place the program under {}", root.string())); + } + } + const fs::path program = root / fs::path(std::string(kName)); + os::process::spawn_options o; + o.program = program; + for (int i = 1; i < argc; ++i) o.args.emplace_back(argv[i]); + o.in = o.out = o.err = os::process::stdio::inherit; + (void)os::process::exec(std::move(o)); + return fail(std::format("cannot start {}", program.string())); +} +)MCPP"; +} + +// ─── --merge ────────────────────────────────────────────────────────────── + +std::expected merge(const MergeRequest& req) { + if (req.inputs.size() < 2) + return std::unexpected(std::string("--merge reads two packages or more")); + std::error_code ec; + const auto work = fs::temp_directory_path(ec) + / std::format("mcpp-merge-{}", std::chrono::steady_clock::now().time_since_epoch().count()); + fs::create_directories(work, ec); + struct Cleanup { fs::path p; ~Cleanup() { std::error_code e; fs::remove_all(p, e); } } cleanup{work}; + + std::vector> trees; + for (std::size_t i = 0; i < req.inputs.size(); ++i) { + auto root = unpack(req.inputs[i], work / std::format("in{}", i)); + if (!root) return std::unexpected(root.error()); + auto pkg = read(*root); + if (!pkg) return std::unexpected(pkg.error()); + trees.emplace_back(*root, std::move(*pkg)); + } + auto const& first = trees.front().second; + for (std::size_t i = 1; i < trees.size(); ++i) { + auto const& p = trees[i].second; + auto differ = [&](std::string_view field, std::string const& a, std::string const& b) + -> std::optional { + if (a == b) return std::nullopt; + return std::format("--merge reads one package's legs and systems: {} has {} '{}', and {} " + "has '{}'", req.inputs.front().string(), field, a, req.inputs[i].string(), b); + }; + for (auto r : { differ("name", first.name, p.name), differ("version", first.version, p.version), + differ("platform", first.platform, p.platform), + differ("linkage", first.linkage, p.linkage), differ("mode", first.mode, p.mode) }) + if (r) return std::unexpected(*r); + } + const auto wrapper = trees.front().first.filename(); + const auto out = work / "merged" / wrapper; + fs::create_directories(out, ec); + Package merged = first; + merged.legs.clear(); + merged.systems.clear(); + merged.entry.clear(); + Stated stated; + std::map> seen; // path -> (digest, input) + for (std::size_t i = 0; i < trees.size(); ++i) { + auto const& [root, pkg] = trees[i]; + for (auto const& l : pkg.legs) + if (std::ranges::find(merged.legs, l) == merged.legs.end()) merged.legs.push_back(l); + for (auto const& s : pkg.systems) + if (std::ranges::find(merged.systems, s) == merged.systems.end()) merged.systems.push_back(s); + for (auto const& [arch, e] : pkg.entry) merged.entry[arch] = e; + for (auto const& f : pkg.files) { + if (auto it = seen.find(f.path); it != seen.end()) { + if (it->second.first != f.sha256) + return std::unexpected(std::format( + "--merge: '{}' is in {} with sha256 {} and in {} with sha256 {}; one package " + "holds one of each file", f.path, it->second.second, it->second.first, + req.inputs[i].string(), f.sha256)); + continue; + } + seen[f.path] = {f.sha256, req.inputs[i].string()}; + stated.role[f.path] = f.role; + if (!f.leg.empty()) stated.leg[f.path] = f.leg; + fs::create_directories((out / f.path).parent_path(), ec); + fs::copy_file(root / f.path, out / f.path, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(std::format("cannot copy '{}': {}", f.path, ec.message())); + } + } + std::ranges::sort(merged.legs); + std::ranges::sort(merged.systems); + if (auto d = describe(out, merged, stated); !d) return std::unexpected(d.error()); + + // The container: as asked, else as the first input was written. + std::string format = req.format; + const auto firstName = req.inputs.front().filename().string(); + if (format.empty()) + format = fs::is_directory(req.inputs.front(), ec) ? "dir" : firstName.ends_with(".zip") ? "zip" : "tar"; + auto compression = req.compression; + if (!compression.stated && format == "tar") { + using mcpp::pack::compress::Algorithm; + compression.algorithm = firstName.ends_with(".tar.zst") ? Algorithm::Zstd + : firstName.ends_with(".tar.xz") ? Algorithm::Xz + : firstName.ends_with(".tar") ? Algorithm::None + : Algorithm::Gzip; + } + fs::path dest = req.output; + const auto ext = format == "zip" ? std::string(".zip") : format == "dir" ? std::string() + : std::string(mcpp::pack::compress::tar_extension(compression.algorithm)); + if (dest.empty()) dest = fs::current_path(ec) / (wrapper.string() + ext); + if (format == "dir") { + fs::remove_all(dest, ec); + fs::create_directories(dest.parent_path(), ec); + fs::copy(out, dest, fs::copy_options::recursive | fs::copy_options::copy_symlinks, ec); + if (ec) return std::unexpected(std::format("cannot write '{}': {}", dest.string(), ec.message())); + return dest; + } + if (format != "tar" && format != "zip") + return std::unexpected(std::format("--merge writes tar, zip or dir; '{}' is a format a package " + "provides, which reads a staged tree: merge to dir first", + format)); + auto r = format == "zip" ? write_zip(out, dest, compression) : write_tar(out, dest, compression); + if (!r) return std::unexpected(r.error()); + return dest; +} + +} // namespace mcpp::pack::distribution diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 58a3dcc62..27c3b0fc4 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -139,7 +139,8 @@ export int build_and_pack_library(const std::string& targetName, LibraryPackPlan plan; plan.builtBy = std::string(mcpp::MCPP_VERSION); - plan.writeArchive = opts.format == mcpp::pack::Format::Tar; + plan.writeArchive = opts.format == mcpp::pack::Format::Tar + || opts.format == mcpp::pack::Format::Zip; InterfaceClosure closure; bool haveClosure = false; diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index d72ee4921..ae4e8877e 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -47,6 +47,8 @@ import mcpp.pack.relocate; import mcpp.pack.stage_tree; import mcpp.pack.strip; import mcpp.pack.zip; +import mcpp.pack.compress; +import mcpp.pack.distribution; import mcpplibs.os.platform; import mcpp.toolchain.platform; import mcpp.invocation; @@ -82,7 +84,28 @@ enum class Mode { None, Static, BundleProject, BundleAll }; // anyone else's release. dpkg's control fields, AppImage's runtime, WiX's // schema and Apple's notarisation each couple an mcpp release to a release mcpp // does not control. -enum class Format { Tar, Dir, Dispatched }; +enum class Format { Tar, Dir, Zip, SingleFile, Dispatched }; + +// THE LINKAGE AXIS (SPEC-014 §1): a property of the program -- whether the +// libraries of its graph are in it or beside it -- which a pack states for the +// build it makes (`--linkage`, `[pack] linkage`). Unstated: as the project +// builds it. +enum class Linkage { Unstated, Static, Shared }; + +// The container an archive format writes, decided once in `make_plan`: a tar +// (compressed as `Options::compression` says), a zip, or none (a directory, a +// single file, a dispatched format). +enum class ArchiveKind { None, Tar, Zip }; + +// One more architecture of a universal package (SPEC-014 §3): a leg of the +// platform's targets other than the one the pack built first, built by the +// pipeline before staging. `artifact` is that leg's program image; the images +// it names are found beside it, as for the first leg. +struct LoadedLeg { + std::string triple; // canonical + std::string arch; // the directory under `lib/` + std::filesystem::path artifact; +}; // What happens when a FORMAT's dependency closure cannot be resolved on this // host, as a function of the format alone — never of the reason (a Mach-O @@ -106,6 +129,25 @@ ClosureUnavailableOutcome closure_unavailable_outcome(Format format) { struct Options { Mode mode = Mode::BundleProject; + // The three axes (SPEC-014 §1) beyond `mode` and `format`: the linkage + // the pack builds with, whether the format was stated (a format left to + // the target is a zip for a Windows program and a tar elsewhere), and the + // container's compression. + Linkage linkage = Linkage::Unstated; + bool formatStated = false; + mcpp::pack::compress::Choice compression; + // Universal packages (SPEC-014 §3). `universal`: nullopt follows the + // platform's `universal`; false packs the selected target alone. When the + // pipeline decides a universal package it sets `universalName` (the + // platform's name, which names the package instead of a triple) and + // `systems` (the systems whose loaders a self-contained package carries, + // ``, from `--systems`; empty: this machine's). + std::optional universal; + std::string universalName; + std::vector systems; + // `[pack] sign`: run on each program, image, library and loader of the + // staged tree before the container is written, `{}` the file. + std::vector sign; // The build stated its target base (design 2026-10-23 §4.11): the base it // was built against becomes the C library floor the package states. bool baseStated = false; @@ -230,6 +272,14 @@ struct Plan { std::string binaryName; // basename(builtBinary) std::filesystem::path stagingRoot; // target/dist// std::filesystem::path archivePath; // …/.tar.gz | .zip + ArchiveKind archive = ArchiveKind::None; + // The linkage the package's `mcpp-package.json` states: `static`, + // `shared`, or `declared` (each library as it declares itself). Set by the + // caller from the build. + std::string linkageName = "declared"; + // The other legs of a universal package (SPEC-014 §3), set by the caller + // after `make_plan` as `extraSharedLegs` is. + std::vector loadedLegs; std::string packageName; std::string packageVersion; std::string triple; // e.g. "x86_64-linux-musl" @@ -477,6 +527,18 @@ make_plan(const mcpp::manifest::Manifest& manifest, std::expected run(const Plan& plan, const mcpp::config::GlobalConfig& cfg); +// THE SINGLE FILE (SPEC-014 §5.2), made from the tree `run` staged: for an +// image, the platform's carrier carrying it (or, when it names shared images, +// a stored zip of all of them aligned for mapping), one file per system of +// `Options::systems`; for a native program, the engine's launcher carrying the +// tree as a zstd frame. Returns the files written. +std::expected, Error> single_file(const Plan& plan); + +// The tree made a package: signed and described (`mcpp-package.json`). `run` +// does it before writing a container; a dispatched format whose tree was +// staged without its closure is finalized by the caller. +std::expected finalize_staged(const Plan& plan); + // Helpers used by cli.cppm to render mode names + parse `--mode`. // Canonical name shown in `--help`/diagnostics (renamed for legibility). std::string_view mode_cli_name(Mode m); @@ -540,9 +602,11 @@ namespace detail { // `.zip` whoever built it, and a `.tar.gz` full of DLLs is a package most // Windows users cannot open without installing something first. std::string default_archive_name(std::string_view name, std::string_view version, - std::string_view triple, Mode mode, bool pe) + std::string_view triple, Mode mode, ArchiveKind kind, + mcpp::pack::compress::Algorithm compression) { - std::string_view ext = pe ? ".zip" : ".tar.gz"; + std::string_view ext = kind == ArchiveKind::Zip ? std::string_view(".zip") + : mcpp::pack::compress::tar_extension(compression); auto sfx = mode_tarball_suffix(mode); if (sfx.empty()) return std::format("{}-{}-{}{}", name, version, triple, ext); @@ -556,7 +620,8 @@ std::string default_archive_name(std::string_view name, std::string_view version std::string wrapper_dirname_from_archive(const std::filesystem::path& archive) { auto name = archive.filename().string(); for (auto suffix : {std::string_view{".tar.gz"}, std::string_view{".tgz"}, - std::string_view{".zip"}}) { + std::string_view{".tar.zst"}, std::string_view{".tar.xz"}, + std::string_view{".tar"}, std::string_view{".zip"}}) { if (name.ends_with(suffix)) return name.substr(0, name.size() - suffix.size()); } // No recognised compression suffix — fall back to the bare stem. @@ -710,10 +775,22 @@ make_plan(const mcpp::manifest::Manifest& manifest, if (opts.carryToolchainRuntime) for (auto const& d : opts.toolchainRuntimeDirs) p.searchDirs.push_back(d); + // THE CONTAINER. A format left unstated follows the artifact -- a zip for + // a Windows program, which its users open without installing anything, a + // tar elsewhere; `tar` and `zip` stated are what they say (SPEC-014 §1). + p.archive = opts.format == Format::Zip ? ArchiveKind::Zip + : opts.format != Format::Tar ? ArchiveKind::None + : (p.targetIsPe && !opts.formatStated) ? ArchiveKind::Zip + : ArchiveKind::Tar; + // A universal package is named after its platform, not a triple: it is + // the same package on every system and architecture of the platform. + const std::string nameTriple = opts.universalName.empty() ? p.triple : opts.universalName; auto distDir = projectRoot / "target" / "dist"; if (opts.output.empty()) { p.archivePath = distDir / detail::default_archive_name( - p.packageName, p.packageVersion, p.triple, opts.mode, p.targetIsPe); + p.packageName, p.packageVersion, nameTriple, opts.mode, + p.archive == ArchiveKind::None ? ArchiveKind::Tar : p.archive, + opts.compression.algorithm); } else if (!opts.output.has_parent_path()) { // `-o name.tar.gz` (bare filename) → place in target/dist/. // `-o ./name.tar.gz` or `-o sub/name.tar.gz` → use as-is. @@ -727,6 +804,10 @@ make_plan(const mcpp::manifest::Manifest& manifest, // matches what GUI extractors create on click and what `tar -xzf` // produces on the CLI. p.stagingRoot = distDir / detail::wrapper_dirname_from_archive(p.archivePath); + // A single file takes the tree's name in `target/dist`; its tree is + // staged outside it. + if (opts.format == Format::SingleFile) + p.stagingRoot = projectRoot / "target" / "single-file" / "stage" / p.stagingRoot.filename(); p.includeGlobs = manifest.packConfig.include; p.excludeGlobs = manifest.packConfig.exclude; @@ -1399,27 +1480,114 @@ stage_closure(const Plan& plan, const ClosureRead& read, std::expected make_tarball(const std::filesystem::path& stagingRoot, - const std::filesystem::path& tarballPath) + const std::filesystem::path& tarballPath, + mcpp::pack::compress::Choice compression = {}) { - std::error_code ec; - if (tarballPath.has_parent_path()) - std::filesystem::create_directories(tarballPath.parent_path(), ec); // Pack with a top-level wrapper directory whose name matches the // tarball stem (computed by make_plan via wrapper_dirname_from_tarball). - // This keeps click-to-extract and `tar -xzf` aligned: both surface a - // single self-contained directory in the user's cwd. - // `--force-local` on Windows: GNU tar reads `C:/path/x.tar.gz` as the rsh - // form `host:path` and dies with `Cannot connect to C: resolve failed`, - // naming neither its argument nor the drive letter. Latent here rather than - // observed — a PE target is written as a zip — but it is the same command - // with the same hazard as the library packer's, and the two should not - // differ in whether they survive being run on Windows. - std::vector argv{ "tar" }; - if constexpr (mcpplibs::os::is_windows) argv.push_back("--force-local"); - argv.insert(argv.end(), { "-czf", tarballPath.string(), "-C", stagingRoot.parent_path().string(), - stagingRoot.filename().string() }); - auto r = run_capture(argv); - if (!r) return std::unexpected(Error{r.error()}); + // This keeps click-to-extract and `tar -xf` aligned: both surface a + // single self-contained directory in the user's cwd. The codec is mcpp's + // own (mcpp.pack.distribution::write_tar). + if (auto r = mcpp::pack::distribution::write_tar(stagingRoot, tarballPath, compression); !r) + return std::unexpected(Error{r.error()}); + return {}; +} + +std::expected +make_zip(const std::filesystem::path& stagingRoot, const std::filesystem::path& archive, + mcpp::pack::compress::Choice compression, std::string_view program = {}) +{ + if (auto r = mcpp::pack::distribution::write_zip(stagingRoot, archive, compression, program); !r) + return std::unexpected(Error{r.error()}); + return {}; +} + +// THE TREE MADE A PACKAGE (SPEC-014 §4.5, §7): each program, image, library +// and loader signed by `[pack] sign`, then every file described in +// `mcpp-package.json` with its role, its leg and its digest. Before the +// container, so the archive holds what the document says. +std::expected finalize_tree(const Plan& plan) { + namespace fs = std::filesystem; + namespace d = mcpp::pack::distribution; + std::error_code ec; + if (!fs::is_directory(plan.stagingRoot, ec)) return {}; + const auto canonical = [](std::string const& t) { + if (t.empty()) return mcpp::toolchain::triple::host_triple().str(); + auto parsed = mcpp::toolchain::triple::parse(t); + return parsed ? parsed->str() : t; + }; + d::Package pkg; + pkg.name = plan.packageName; + pkg.version = plan.packageVersion; + if (plan.platform.active) pkg.platform = std::format("{}@{}", plan.platform.package, plan.platform.version); + pkg.linkage = plan.linkageName; + pkg.mode = std::string(mode_cli_name(plan.opts.mode == Mode::Static && plan.targetIsLoaded + ? Mode::BundleAll : plan.opts.mode)); + pkg.legs.push_back(canonical(plan.triple)); + for (auto const& l : plan.loadedLegs) pkg.legs.push_back(l.triple); + for (auto const& l : plan.extraSharedLegs) pkg.legs.push_back(l.triple); + d::Stated stated; + if (!plan.opts.universalName.empty()) { + pkg.runner = plan.platform.fields.runner.cmd; + const auto stem = fs::path(plan.binaryName).stem().string(); + stated.role["bin/" + stem] = "launcher"; + stated.role["bin/" + stem + ".cmd"] = "launcher"; + std::vector> legs{ + { canonical(plan.triple), [&] { auto t = mcpp::toolchain::triple::parse(plan.triple); + return t ? t->arch : plan.triple; }() } }; + for (auto const& l : plan.loadedLegs) legs.emplace_back(l.triple, l.arch); + // Every file of `lib//` is an image of that leg. + auto state_leg = [&](std::string const& triple, std::string const& arch) { + for (auto const& e : fs::directory_iterator(plan.stagingRoot / "lib" / arch, ec)) { + const auto rel = std::format("lib/{}/{}", arch, e.path().filename().string()); + stated.role[rel] = "image"; + stated.leg[rel] = triple; + } + }; + for (auto const& [triple, arch] : legs) { + pkg.entry[arch] = std::format("lib/{}/{}", arch, plan.binaryName); + state_leg(triple, arch); + } + pkg.systems = d::loader_systems(plan.stagingRoot); + } else { + pkg.program = plan.targetIsPe ? plan.binaryName : "bin/" + plan.binaryName; + if (plan.targetIsLoaded && plan.opts.mode == Mode::BundleAll) + pkg.program = "bin/" + fs::path(plan.binaryName).stem().string() + + std::string(mcpplibs::os::exe_suffix); + } + auto described = d::describe(plan.stagingRoot, pkg, stated); + if (!described) return std::unexpected(Error{described.error()}); + if (plan.opts.sign.empty()) return {}; + for (auto const& f : described->files) { + if (f.role != "program" && f.role != "image" && f.role != "library" && f.role != "loader") + continue; + std::vector argv; + for (auto const& a : plan.opts.sign) + argv.push_back(a == "{}" ? (plan.stagingRoot / f.path).string() : a); + int spawnError = 0; + auto r = mcpp::run::capture_exec(argv, {}, {}, &spawnError); + if (spawnError != 0 || r.exit_code != 0) + return std::unexpected(Error{std::format( + "[pack] sign failed on {} (exit {}): {}\n{}", f.path, r.exit_code, + mcpp::build::command_line::display(argv), r.output)}); + } + // The digests of what was signed. + if (auto again = d::describe(plan.stagingRoot, pkg, stated); !again) + return std::unexpected(Error{again.error()}); + return {}; +} + +// The container of a built-in archive format, or nothing for a directory, a +// single file or a dispatched format; the tree is finalized first. +std::expected write_container(const Plan& plan) { + if (auto r = finalize_tree(plan); !r) return r; + switch (plan.archive) { + case ArchiveKind::None: return {}; + case ArchiveKind::Tar: return make_tarball(plan.stagingRoot, plan.archivePath, + plan.opts.compression); + case ArchiveKind::Zip: return make_zip(plan.stagingRoot, plan.archivePath, + plan.opts.compression, plan.binaryName); + } return {}; } @@ -1887,35 +2055,12 @@ run_pe(const Plan& plan) return std::unexpected(r.error()); } - if (plan.opts.format != Format::Tar) return needs; - - std::vector entries; - const auto wrapper = plan.stagingRoot.filename().string(); - for (auto const& e : - std::filesystem::recursive_directory_iterator(plan.stagingRoot, ec)) { - if (!e.is_regular_file(ec)) continue; - auto rel = std::filesystem::relative(e.path(), plan.stagingRoot, ec); - if (ec) continue; - entries.push_back({ - wrapper + "/" + rel.generic_string(), - e.path(), - e.path().filename() == plan.builtBinary.filename(), - }); - } - // Deterministic order: a directory iteration order that leaks into an - // archive is how two identical builds get two different checksums. - // - // Plain `std::sort` with an explicit comparator, not - // `std::ranges::sort(entries, {}, &Entry::name)`. A ranges projection - // naming a member of an IMPORTED type is one of three module-boundary - // shapes this function used to carry, and together they crashed the clang - // 20.1.7 frontend (0xC0000005, no diagnostic) on every Windows job. See - // the note at the top of run_pe. - std::sort(entries.begin(), entries.end(), - [](const mcpp::pack::zip::Entry& a, - const mcpp::pack::zip::Entry& b) { return a.name < b.name; }); - if (auto r = mcpp::pack::zip::write(plan.archivePath, entries); !r) - return std::unexpected(Error{r.error()}); + // The container: a zip unless `tar` was stated (`write_container`). + // Plain `std::sort` with an explicit comparator there, not a ranges + // projection onto an imported type's member: that shape was one of three + // that crashed the clang 20.1.7 frontend (0xC0000005, no diagnostic) on + // every Windows job. See the note at the top of run_pe. + if (auto r = write_container(plan); !r) return std::unexpected(r.error()); return needs; } @@ -1973,8 +2118,7 @@ run_wasm(const Plan& plan) copy_if_exists(plan.projectRoot / "README.md", plan.stagingRoot); copy_if_exists(plan.projectRoot / "LICENSE", plan.stagingRoot); - if (plan.opts.format != Format::Tar) return {}; - return make_tarball(plan.stagingRoot, plan.archivePath); + return write_container(plan); } // AN ARTIFACT A LOADER RUNS (an ok image, `-openkal`; design @@ -1992,12 +2136,19 @@ run_wasm(const Plan& plan) // with the artifacts it names beside it; the machine provides // nothing. The artifact is not recompiled: the program is the // carrier package, the carrier's entry source and the bytes, -// built by mcpp for this machine with this machine's own -// conventions -- the bootstrap boundary (§6.2), so no platform -// is selected for it. Another system's M3 is packed on that -// system. -// static refused: the artifact is loaded, never linked into one -// file with its loader by `ld`. +// built by mcpp with that system's own conventions -- the +// bootstrap boundary (§6.2), so no platform is selected for it. +// +// A UNIVERSAL PACKAGE (SPEC-014 §3, §4.2) holds every target of a platform +// that states `universal = true`: one `lib//` per leg, launchers in +// `bin/` that pick the machine's architecture and start the platform's +// runner, and under `self-contained` the runner's program for each system of +// `--systems` and each architecture -- the carrier built with nothing carried, +// which runs the image its first argument names -- under +// `libexec//-/`. +// +// `--linkage static` is a property of the image (no `.okso`), and `--mode +// static` is `--linkage static --mode self-contained`. namespace { // A reference as a dependency entry of the generated manifest. @@ -2015,16 +2166,113 @@ std::string dependency_entry(const mcpp::manifest::platform::PackageRef& r) { return std::format("[dependencies.{}]\n{} = {}\n", ns, sp.shortName, value); } +// The system names a package speaks of, and this machine's. +constexpr std::string_view host_system() { + if constexpr (mcpplibs::os::is_windows) return "windows"; + else if constexpr (mcpplibs::os::is_macos) return "macos"; + else return "linux"; +} + +// The triple a program of `system` on `arch` is built for, and the mode it +// is packed in: a Linux program is static (musl), the one form that runs on +// every distribution; a program of this machine's system and architecture on +// macOS or Windows takes this machine's own row; another system's is its +// conventional row. `single`: the program is the whole package (a single +// file), so on Windows it takes its C++ run-time from the system. +struct ProgramRow { std::string triple; std::string mode; }; +ProgramRow program_row(std::string_view system, std::string_view arch, bool single) { + if (system == "linux") return { std::format("{}-linux-musl", arch), "static" }; + const bool here = system == host_system() && arch == mcpp::toolchain::platform::host_arch; + if (system == "macos") return { here ? std::string() : std::format("{}-macos", arch), "vendored" }; + return { here ? std::string() : std::format("{}-windows-gnu", arch), single ? "system" : "vendored" }; +} + +// The bytes of `carried` as a section of a program of `system`, with the +// symbols the carrier's entry reads; `name` and `digest` as strings. An empty +// `carried` places nothing: `mcpp_carried_begin == mcpp_carried_end`. +std::string carried_assembly(std::string_view system, std::string_view sectionName, + const std::filesystem::path& carried, std::string_view name, + std::string_view digest) { + const bool macho = system == "macos"; + const bool coff = system == "windows"; + const std::string sym = macho ? "_" : ""; + std::string bare(sectionName); + while (!bare.empty() && bare.front() == '.') bare.erase(bare.begin()); + std::string upper = bare; + for (auto& ch : upper) ch = static_cast(std::toupper(static_cast(ch))); + const std::string section = macho ? std::format("\t.section __{},__carried\n", upper.substr(0, 14)) + : coff ? std::format("\t.section .{},\"dr\"\n", bare) + : std::format("\t.section .{},\"a\",@progbits\n", bare); + std::string text = std::format( + "# What the program carries, unchanged, and its name (generated by mcpp pack).\n{}" + "\t.p2align 4\n\t.globl {}mcpp_carried_begin\n{}mcpp_carried_begin:\n", + section, sym, sym); + if (!carried.empty()) + text += std::format("\t.incbin \"{}\"\n", std::filesystem::absolute(carried).generic_string()); + text += std::format( + "\t.globl {0}mcpp_carried_end\n{0}mcpp_carried_end:\n" + "\t.globl {0}mcpp_carried_name\n{0}mcpp_carried_name:\n\t.asciz \"{1}\"\n" + "\t.globl {0}mcpp_carried_digest\n{0}mcpp_carried_digest:\n\t.asciz \"{2}\"\n", + sym, name, digest); + return text; +} + +// A program generated under `dir` (its manifest, entry and carried section) +// packed for `row` as a directory; returns the directory (`bin/` holds the +// program and what it needs beside it). +std::expected +pack_generated_program(const std::filesystem::path& dir, const ProgramRow& row) { + namespace fs = std::filesystem; + std::error_code ec; + const auto out = dir / "target" / "dist" / "packed"; + fs::remove_all(out, ec); + std::vector argv{ mcpp::invocation::self_exe_path().string(), "pack", + "--pkg-format", "dir", "-o", "packed", "--platform", "none", + "--mode", row.mode }; + if (!row.triple.empty()) argv.insert(argv.end(), { "--target", row.triple }); + int spawnError = 0; + auto built = mcpp::run::capture_exec(argv, {}, dir.string(), &spawnError); + if (spawnError != 0 || built.exit_code != 0) + return std::unexpected(Error{std::format( + "packing the program under {} failed:\n{}", dir.string(), built.output)}); + if (!fs::is_directory(out / "bin", ec)) + return std::unexpected(Error{std::format("the program's package under {} has no bin/", + out.string())}); + return out; +} + +// THE STATIC ROW'S TOOLCHAIN, STATED: the carrier's own graph supplies a +// kernel interface, which would otherwise cancel the row's convention and +// leave a compiler without a musl C runtime to link with (L10, measured: clang +// took the host GCC's `crtbeginT.o`). +std::string static_row_pin(const ProgramRow& row) { + if (!row.triple.ends_with("-linux-musl")) return {}; + if (auto t = mcpp::toolchain::triple::parse(row.triple)) + if (auto* k = mcpp::toolchain::triple::find_known_target(*t); k && !k->pin.empty()) + return std::format("\n[target.{}]\ntoolchain = \"{}\"\n", row.triple, k->pin); + return {}; +} + } // namespace +// What a carrier carries and for which system it is built. +struct CarrierBuild { + std::filesystem::path carried; // empty: nothing (the runner's form) + std::string carriedName; // the artifact's name, as the carrier reports it + std::string stem; // the program's name + std::string system; // linux | macos | windows + std::string arch; + bool single = false; +}; + std::expected -carrier_build_selfcontained(const Plan& plan, const std::filesystem::path& image) +carrier_build(const Plan& plan, const CarrierBuild& b) { namespace fs = std::filesystem; auto const& c = plan.platform.fields.carrier; if (!plan.platform.active || !c.set) return std::unexpected(Error{std::format( - "--mode self-contained of {} builds a program of this machine over its platform's " + "--mode self-contained of {} builds a program of the machine over its platform's " "carrier, and {} states none ([platform] carrier = {{ package, section, main }}).", plan.binaryName, plan.platform.display())}); std::error_code ec; @@ -2034,8 +2282,9 @@ carrier_build_selfcontained(const Plan& plan, const std::filesystem::path& image plan.platform.display(), c.main.empty() ? std::string("(none)") : c.main, mcpp::platform::origin_text(c.origin))}); - const auto stem = fs::path(plan.binaryName).stem().string(); - const auto dir = plan.projectRoot / "target" / "carrier-pack" / stem; + const auto row = program_row(b.system, b.arch, b.single); + const auto dir = plan.projectRoot / "target" / "carrier-pack" + / std::format("{}-{}-{}", b.stem, b.system, b.arch); fs::create_directories(dir / "src", ec); if (ec) return std::unexpected(Error{std::format("cannot create {}: {}", dir.string(), ec.message())}); // RESOLVED AS THE BUILD THAT MADE THE ARTIFACT RESOLVED ITS PLATFORM. The @@ -2049,142 +2298,173 @@ carrier_build_selfcontained(const Plan& plan, const std::filesystem::path& image out << text; return bool(out); }; - // ON LINUX THE PROGRAM IS STATIC (musl), the one form that runs on every - // distribution, linked by the toolchain the target table pins for this - // machine's static row -- stated, because the carrier's own graph supplies - // a kernel interface, which would otherwise cancel the row's convention and - // leave a compiler without a musl C runtime to link with (L10, measured: - // clang took the host GCC's `crtbeginT.o`). - std::string staticRow; - if constexpr (mcpplibs::os::is_linux) { - const auto row = std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch); - if (auto t = mcpp::toolchain::triple::parse(row)) - if (auto* k = mcpp::toolchain::triple::find_known_target(*t); k && !k->pin.empty()) - staticRow = std::format("\n[target.{}]\ntoolchain = \"{}\"\n", row, k->pin); - } const std::string manifest = std::format( - "# Generated by `mcpp pack --mode self-contained` for {}: a program of this\n" - "# machine that carries it as a section and runs it through {}'s carrier (M3).\n" + "# Generated by `mcpp pack` for {}: a program of {}-{} that {} through {}'s\n" + "# carrier (SPEC-013 §11, SPEC-014 §4.2).\n" "[package]\nname = \"{}\"\nversion = \"{}\"\n\n" "[targets.{}]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n" "[build]\nsources = [\"src/main.cpp\", \"src/carried.S\"]\n\n" "{}{}", - plan.binaryName, plan.platform.display(), stem, - plan.packageVersion.empty() ? "0.0.0" : plan.packageVersion, stem, - dependency_entry(c.package), staticRow); - // The section, in the syntax of this machine's object format; the symbol - // spelling carries Mach-O's leading underscore. The carrier's entry reads - // the bytes through `mcpp_carried_begin` / `mcpp_carried_end` and the - // artifact's name through `mcpp_carried_name`. - const bool macho = mcpplibs::os::is_macos; - const bool coff = mcpplibs::os::is_windows; - const std::string sym = macho ? "_" : ""; - std::string bare = c.section; - while (!bare.empty() && bare.front() == '.') bare.erase(bare.begin()); - std::string upper = bare; - for (auto& ch : upper) ch = static_cast(std::toupper(static_cast(ch))); - const std::string section = macho ? std::format("\t.section __{},__image\n", upper.substr(0, 14)) - : coff ? std::format("\t.section {},\"dr\"\n", c.section) - : std::format("\t.section {},\"a\",@progbits\n", c.section); - const std::string assembly = std::format( - "# The artifact's bytes, unchanged, and its name (generated by mcpp pack).\n{}" - "\t.p2align 4\n\t.globl {}mcpp_carried_begin\n{}mcpp_carried_begin:\n" - "\t.incbin \"{}\"\n\t.globl {}mcpp_carried_end\n{}mcpp_carried_end:\n" - "\t.globl {}mcpp_carried_name\n{}mcpp_carried_name:\n\t.asciz \"{}\"\n", - section, sym, sym, fs::absolute(image).generic_string(), sym, sym, sym, sym, - plan.binaryName); + plan.binaryName, b.system, b.arch, + b.carried.empty() ? "runs the image it is given" : "carries it and runs it", + plan.platform.display(), b.stem, + plan.packageVersion.empty() ? "0.0.0" : plan.packageVersion, b.stem, + dependency_entry(c.package), static_row_pin(row)); std::string mainText; { std::ifstream in(c.main, std::ios::binary); mainText.assign(std::istreambuf_iterator(in), {}); } - if (!write(dir / "mcpp.toml", manifest) || !write(dir / "src" / "carried.S", assembly) + if (!write(dir / "mcpp.toml", manifest) + || !write(dir / "src" / "carried.S", + carried_assembly(b.system, c.section, b.carried, b.carriedName, {})) || !write(dir / "src" / "main.cpp", mainText)) return std::unexpected(Error{std::format("cannot write the program's sources under {}", dir.string())}); // PACKED, NOT MERELY BUILT: the program obeys every rule this command // keeps for a program -- no build-machine path, its closure beside it. On // macOS and Windows the system's own libraries are the only ones it names. - mcpp::ui::info("Packing", std::format("{} into a program of this machine ({}'s carrier, M3)", - plan.binaryName, plan.platform.display())); - // `-o` with a bare name: the package directory is `target/dist/`. - const auto out = dir / "target" / "dist" / "packed"; - fs::remove_all(out, ec); - std::vector argv{ mcpp::invocation::self_exe_path().string(), "pack", - "--pkg-format", "dir", "-o", "packed", "--platform", "none" }; - if constexpr (mcpplibs::os::is_linux) { - argv.insert(argv.end(), { "--target", std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch), - "--mode", "static" }); - } else { - argv.insert(argv.end(), { "--mode", "vendored" }); + mcpp::ui::info("Packing", b.carried.empty() + ? std::format("{} for {}-{} ({}'s carrier, the runner of the package)", b.stem, + b.system, b.arch, plan.platform.display()) + : std::format("{} into a program of {}-{} ({}'s carrier, M3)", plan.binaryName, + b.system, b.arch, plan.platform.display())); + return pack_generated_program(dir, row); +} + +namespace { + +// The images an image names (`DT_NEEDED`), found where its build placed them. +std::expected, Error> +needed_images(const std::filesystem::path& artifact, std::string_view binaryName) { + namespace fs = std::filesystem; + std::vector needed; + auto facts = mcpp::runtime::elf::inspect_elf_runtime(artifact); + if (!facts) return std::unexpected(Error{facts.error()}); + const auto builtDir = artifact.parent_path(); + std::error_code ec; + for (auto const& name : facts->needed) { + fs::path hit; + for (auto const& d : { builtDir, builtDir.parent_path() / "lib" }) + if (fs::is_regular_file(d / name, ec)) { hit = d / name; break; } + if (hit.empty()) return std::unexpected(Error{std::format( + "{} names the image '{}', which the build did not place beside it", binaryName, name)}); + needed.push_back(hit); } - int spawnError = 0; - auto built = mcpp::run::capture_exec(argv, {}, dir.string(), &spawnError); - if (spawnError != 0 || built.exit_code != 0) - return std::unexpected(Error{std::format( - "packing the self-contained program failed (in {}):\n{}", dir.string(), built.output)}); - if (!fs::is_directory(out / "bin", ec)) - return std::unexpected(Error{std::format( - "the self-contained program's package under {} has no bin/", out.string())}); - return out; + return needed; +} + +std::string arch_of(std::string_view triple) { + auto t = mcpp::toolchain::triple::parse(triple); + return t ? t->arch : std::string(triple.substr(0, triple.find('-'))); } +} // namespace + std::expected run_loaded(const Plan& plan) { namespace fs = std::filesystem; - if (plan.opts.mode == Mode::Static) - return std::unexpected(Error{std::format( - "--mode static does not apply to {}: it is loaded, not linked with its loader " - "into one file.\n" - " `--mode self-contained` packs a program of this machine that carries {} " - "and needs nothing installed.", plan.binaryName, plan.binaryName)}); std::error_code ec; fs::remove_all(plan.stagingRoot, ec); fs::create_directories(plan.stagingRoot / "bin", ec); if (ec) return std::unexpected(Error{std::format( "cannot create staging '{}': {}", plan.stagingRoot.string(), ec.message())}); - - // The images it names, found where the build placed them. - std::vector needed; - if (plan.opts.mode != Mode::None) { - auto facts = mcpp::runtime::elf::inspect_elf_runtime(plan.builtBinary); - if (!facts) return std::unexpected(Error{facts.error()}); - const auto builtDir = plan.builtBinary.parent_path(); - for (auto const& name : facts->needed) { - fs::path hit; - for (auto const& d : { builtDir, builtDir.parent_path() / "lib" }) - if (fs::is_regular_file(d / name, ec)) { hit = d / name; break; } - if (hit.empty()) return std::unexpected(Error{std::format( - "{} names the image '{}', which the build did not place beside it", - plan.binaryName, name)}); - needed.push_back(hit); + auto copy = [&](const fs::path& from, const fs::path& to) -> std::expected { + fs::create_directories(to.parent_path(), ec); + fs::copy_file(from, to, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format("copy {} failed: {}", from.string(), ec.message())}); + return {}; + }; + const bool bundles = plan.opts.mode != Mode::None; + + if (plan.opts.universalName.empty()) { + // ONE LEG: the image in `bin/`, the images it names beside it, or + // under `self-contained` the carrier that carries it (M3). + std::vector needed; + if (bundles) { + auto n = needed_images(plan.builtBinary, plan.binaryName); + if (!n) return std::unexpected(n.error()); + needed = std::move(*n); } - } - - fs::path staged; - if (plan.opts.mode == Mode::BundleAll) { - auto packed = carrier_build_selfcontained(plan, plan.builtBinary); - if (!packed) return std::unexpected(packed.error()); - fs::copy(*packed, plan.stagingRoot, - fs::copy_options::recursive | fs::copy_options::overwrite_existing, ec); - staged = plan.stagingRoot / "bin" - / (fs::path(plan.binaryName).stem().string() + std::string(mcpplibs::os::exe_suffix)); + fs::path staged; + // A single file is made from the images (`single_file`); the tree + // holds them as `vendored` does. + if (plan.opts.mode == Mode::BundleAll && plan.opts.format != Format::SingleFile) { + auto packed = carrier_build(plan, { plan.builtBinary, plan.binaryName, + fs::path(plan.binaryName).stem().string(), std::string(host_system()), + std::string(mcpp::toolchain::platform::host_arch), false }); + if (!packed) return std::unexpected(packed.error()); + fs::copy(*packed, plan.stagingRoot, + fs::copy_options::recursive | fs::copy_options::overwrite_existing, ec); + staged = plan.stagingRoot / "bin" + / (fs::path(plan.binaryName).stem().string() + std::string(mcpplibs::os::exe_suffix)); + } else { + staged = plan.stagingRoot / "bin" / plan.binaryName; + if (auto r = copy(plan.builtBinary, staged); !r) return r; + } + for (auto const& n : needed) + if (auto r = copy(n, plan.stagingRoot / "bin" / n.filename()); !r) return r; + if (auto r = stage_runtime_files(plan, staged.parent_path()); !r) return r; } else { - staged = plan.stagingRoot / "bin" / plan.binaryName; - fs::copy_file(plan.builtBinary, staged, fs::copy_options::overwrite_existing, ec); - } - if (ec) return std::unexpected(Error{std::format("copy {} failed: {}", staged.string(), ec.message())}); - for (auto const& n : needed) { - fs::copy_file(n, plan.stagingRoot / "bin" / n.filename(), fs::copy_options::overwrite_existing, ec); - if (ec) return std::unexpected(Error{std::format("copy {} failed: {}", n.string(), ec.message())}); + // A UNIVERSAL PACKAGE: a `lib//` per leg, the launchers, and the + // runner's program per system and architecture under `self-contained`. + auto const& runner = plan.platform.fields.runner; + if (!runner.set || runner.cmd.empty()) + return std::unexpected(Error{std::format( + "a universal package of {} starts its images through the platform's runner, and " + "{} states none.\n use: --no-universal", + plan.binaryName, plan.platform.display())}); + struct Leg { std::string triple, arch; fs::path artifact; }; + std::vector legs{ { plan.triple, arch_of(plan.triple), plan.builtBinary } }; + for (auto const& l : plan.loadedLegs) legs.push_back({ l.triple, l.arch, l.artifact }); + mcpp::pack::distribution::Launch launch; + launch.name = fs::path(plan.binaryName).stem().string(); + launch.runner = runner.cmd; + for (auto const& leg : legs) { + const auto dir = plan.stagingRoot / "lib" / leg.arch; + if (auto r = copy(leg.artifact, dir / leg.artifact.filename()); !r) return r; + launch.entry[leg.arch] = leg.artifact.filename().string(); + if (bundles) { + auto n = needed_images(leg.artifact, plan.binaryName); + if (!n) return std::unexpected(n.error()); + for (auto const& f : *n) + if (auto r = copy(f, dir / f.filename()); !r) return r; + } + if (auto r = stage_runtime_files(plan, dir); !r) return r; + } + if (plan.opts.mode == Mode::BundleAll) { + launch.loader = fs::path(runner.cmd.front()).stem().string(); + std::vector systems = plan.opts.systems; + if (systems.empty()) systems.emplace_back(host_system()); + for (auto const& system : systems) + for (auto const& leg : legs) { + auto packed = carrier_build(plan, { {}, {}, launch.loader, system, leg.arch, false }); + if (!packed) return std::unexpected(packed.error()); + const auto into = plan.stagingRoot / "libexec" / launch.loader + / std::format("{}-{}", system, leg.arch); + fs::create_directories(into, ec); + fs::copy(*packed / "bin", into, + fs::copy_options::recursive | fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format( + "cannot place the runner for {}-{}: {}", system, leg.arch, ec.message())}); + } + } + const auto posix = plan.stagingRoot / "bin" / launch.name; + { + std::ofstream out(posix, std::ios::binary | std::ios::trunc); + out << mcpp::pack::distribution::posix_launcher(launch); + } + fs::permissions(posix, fs::perms::owner_all | fs::perms::group_read | fs::perms::group_exec + | fs::perms::others_read | fs::perms::others_exec, + fs::perm_options::replace, ec); + std::ofstream(plan.stagingRoot / "bin" / (launch.name + ".cmd"), std::ios::binary | std::ios::trunc) + << mcpp::pack::distribution::windows_launcher(launch); } - if (auto r = stage_runtime_files(plan, staged.parent_path()); !r) return r; copy_if_exists(plan.projectRoot / "README.md", plan.stagingRoot); copy_if_exists(plan.projectRoot / "LICENSE", plan.stagingRoot); - if (plan.opts.format != Format::Tar) return {}; - return make_tarball(plan.stagingRoot, plan.archivePath); + return write_container(plan); } // #622 A3/A10: the Application half whose form on this row is a shared @@ -2333,9 +2613,7 @@ run_shared_program(const Plan& plan) " or link_library_dirs, or the compiler's library search path."), std::move(needs)); - if (plan.opts.format == Format::Tar) - if (auto r = make_tarball(plan.stagingRoot, plan.archivePath); !r) - return std::unexpected(r.error()); + if (auto r = write_container(plan); !r) return std::unexpected(r.error()); return ClosureResult{true, {}, std::move(needs)}; } @@ -2436,9 +2714,7 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) std::move(needs)); if (auto r = write_topentry_wrapper(plan.stagingRoot, plan.binaryName); !r) return std::unexpected(Error{r.error()}); - if (plan.opts.format == Format::Tar) - if (auto r = make_tarball(plan.stagingRoot, plan.archivePath); !r) - return std::unexpected(r.error()); + if (auto r = write_container(plan); !r) return std::unexpected(r.error()); return ClosureResult{true, {}, std::move(needs)}; } @@ -2794,12 +3070,131 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) } // Output. - if (plan.opts.format == Format::Tar) { - if (auto r = make_tarball(plan.stagingRoot, plan.archivePath); !r) - return std::unexpected(r.error()); - } + if (auto r = write_container(plan); !r) return std::unexpected(r.error()); return ClosureResult{true, {}, std::move(needs)}; } } +std::expected finalize_staged(const Plan& plan) { + return detail::finalize_tree(plan); +} + +std::expected, Error> single_file(const Plan& plan) { + namespace fs = std::filesystem; + namespace d = mcpp::pack::distribution; + std::error_code ec; + const auto distDir = plan.projectRoot / "target" / "dist"; + auto output_for = [&](std::string_view qualifier, std::string_view suffix) { + if (plan.opts.output.empty()) + return distDir / std::format("{}-{}-{}{}", plan.packageName, plan.packageVersion, + qualifier, suffix); + if (!plan.opts.output.has_parent_path()) return distDir / plan.opts.output; + return plan.opts.output; + }; + std::vector written; + if (plan.targetIsLoaded) { + // THE IMAGE, OR THE IMAGES AS ONE STORED ZIP: what the carrier's + // entry reads between `mcpp_carried_begin` and `mcpp_carried_end` + // (SPEC-013 §11): the image alone when it names no other, else every + // image of `bin/` as members of a zip whose data lies on 64 KiB + // boundaries, the entry named by `mcpp_carried_name`. + std::vector images; + for (auto const& e : fs::directory_iterator(plan.stagingRoot / "bin", ec)) + if (e.is_regular_file(ec)) images.push_back(e.path()); + std::ranges::sort(images); + fs::path carried = plan.stagingRoot / "bin" / plan.binaryName; + if (images.size() > 1) { + carried = plan.projectRoot / "target" / "carrier-pack" + / (fs::path(plan.binaryName).stem().string() + ".carried.zip"); + std::vector entries; + for (auto const& i : images) entries.push_back({ i.filename().string(), i, false, {} }); + mcpp::pack::zip::WriteOptions o; + o.align = 65536; + if (auto r = mcpp::pack::zip::write(carried, entries, o); !r) + return std::unexpected(Error{r.error()}); + } + auto t = mcpp::toolchain::triple::parse(plan.triple); + const std::string arch = t ? t->arch : std::string(mcpp::toolchain::platform::host_arch); + std::vector systems = plan.opts.systems; + if (systems.empty()) systems.emplace_back(detail::host_system()); + for (auto const& system : systems) { + const auto stem = fs::path(plan.binaryName).stem().string(); + auto packed = detail::carrier_build(plan, { carried, plan.binaryName, stem, system, arch, true }); + if (!packed) return std::unexpected(packed.error()); + const auto suffix = system == "windows" ? std::string(".exe") : std::string(); + const auto program = *packed / "bin" / (stem + suffix); + if (!fs::is_regular_file(program, ec)) + return std::unexpected(Error{std::format("the carrier built for {}-{} has no {}", + system, arch, program.filename().string())}); + const auto out = output_for(std::format("{}-{}", system, arch), suffix); + fs::create_directories(out.parent_path(), ec); + fs::copy_file(program, out, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format("cannot write {}: {}", out.string(), ec.message())}); + written.push_back(out); + } + } else { + // THE ENGINE'S LAUNCHER CARRYING THE TREE. The entry: the program a + // person runs from the unpacked tree -- a Windows program at the root, + // a self-contained Linux tree's wrapper at the root, `bin/` + // otherwise. + std::string entry = "bin/" + plan.binaryName; + if (plan.targetIsPe) entry = plan.binaryName; + else if (plan.opts.mode == Mode::BundleAll && fs::is_regular_file(plan.stagingRoot / plan.binaryName, ec)) + entry = plan.binaryName; + auto tree = d::tree_payload(plan.stagingRoot); + if (!tree) return std::unexpected(Error{tree.error()}); + auto c = plan.opts.compression; + c.algorithm = mcpp::pack::compress::Algorithm::Zstd; // `none` stores at level 1 + if (plan.opts.compression.algorithm == mcpp::pack::compress::Algorithm::None) c.level = 1; + auto payload = mcpp::pack::compress::bytes(*tree, c); + if (!payload) return std::unexpected(Error{payload.error()}); + const auto digest = d::sha256(*payload); + const auto stem = fs::path(plan.binaryName).stem().string(); + const auto dir = plan.projectRoot / "target" / "single-file" / stem; + fs::create_directories(dir / "src", ec); + fs::remove(dir / "mcpp.lock", ec); + auto write = [&](const fs::path& p, std::string_view text) -> bool { + std::ofstream out(p, std::ios::binary | std::ios::trunc); + out.write(text.data(), static_cast(text.size())); + return bool(out); + }; + auto t = mcpp::toolchain::triple::parse(plan.triple); + const std::string arch = t ? t->arch : std::string(mcpp::toolchain::platform::host_arch); + const std::string system = !t ? std::string(detail::host_system()) + : t->os == "windows" ? std::string("windows") + : t->os == "macos" ? std::string("macos") : std::string("linux"); + const auto row = detail::program_row(system, arch, true); + if (!write(dir / "payload.zst", *payload) + || !write(dir / "mcpp.toml", d::launcher_manifest(stem, plan.packageVersion) + detail::static_row_pin(row)) + || !write(dir / "src" / "main.cpp", d::launcher_main()) + || !write(dir / "src" / "carried.S", + detail::carried_assembly(system, ".mcppsf", dir / "payload.zst", entry, digest))) + return std::unexpected(Error{std::format("cannot write the launcher's sources under {}", dir.string())}); + mcpp::ui::info("Packing", std::format("{} into one file for {}-{} ({} payload, sha256 {})", + plan.binaryName, system, arch, + mcpp::pack::compress::name(c.algorithm), digest.substr(0, 12))); + auto packed = detail::pack_generated_program(dir, row); + if (!packed) return std::unexpected(packed.error()); + const auto suffix = system == "windows" ? std::string(".exe") : std::string(); + const auto program = *packed / "bin" / (stem + suffix); + const auto out = output_for(t ? t->str() : plan.triple, suffix); + fs::create_directories(out.parent_path(), ec); + fs::copy_file(program, out, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format("cannot write {}: {}", out.string(), ec.message())}); + written.push_back(out); + } + // The signer signs the file a person runs, too. + if (!plan.opts.sign.empty()) + for (auto const& f : written) { + std::vector argv; + for (auto const& a : plan.opts.sign) argv.push_back(a == "{}" ? f.string() : a); + int spawnError = 0; + auto r = mcpp::run::capture_exec(argv, {}, {}, &spawnError); + if (spawnError != 0 || r.exit_code != 0) + return std::unexpected(Error{std::format("[pack] sign failed on {} (exit {}):\n{}", + f.string(), r.exit_code, r.output)}); + } + return written; +} + } // namespace mcpp::pack diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index acd13ff0e..809500b96 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -34,6 +34,8 @@ import mcpp.log; import mcpp.ui; import mcpp.provider; import mcpp.provider.select; +import mcpp.toolchain.platform; // host_arch: the static row +import mcpp.pack.distribution; // the package's metadata, for a dispatched format namespace mcpp::pack { @@ -61,6 +63,11 @@ export struct PackOutcome { std::filesystem::path stageDir; std::filesystem::path stageManifest; std::string closure; + // The axes the package was made with (SPEC-014 §9): `linkage`, `mode`, + // and the systems whose loaders it carries. + std::string linkage; + std::string mode; + std::vector systems; // Whether a build program ran in this pack, for the envelope's `effects`. bool ranBuildPrograms = false; // What one packed member answered (member selection design 2026-09-30, @@ -77,6 +84,9 @@ export struct PackOutcome { std::filesystem::path stageDir; std::filesystem::path stageManifest; std::string closure; + std::string linkage; + std::string mode; + std::vector systems; }; std::vector members; }; @@ -559,6 +569,67 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { } } + // ─── A loaded artifact: the static row, and the universal package ── + // + // `--mode static` of an image is `--linkage static --mode self-contained` + // (SPEC-014 §1.2): one `.oke`, carried. A platform that states `universal + // = true` packs every one of its targets into one package unless the + // command or `[pack] universal` says otherwise (§3); a single file is one + // system's and one architecture's, so it is never universal. + std::vector loadedLegs; + { + auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + const bool loaded = t && t->loader_supplies_os(); + if (loaded && opts.mode == mcpp::pack::Mode::Static) opts.mode = mcpp::pack::Mode::BundleAll; + auto const& P = ctx.plan.platform; + const bool universal = loaded && P.active && P.fields.universal.set && P.fields.universal.value + && opts.universal.value_or(true) && opts.format != mcpp::pack::Format::SingleFile; + if (!universal && !opts.systems.empty() && !(loaded && opts.format == mcpp::pack::Format::SingleFile)) + return Refusal{2, about(run, m, std::format( + "--systems names the systems whose loaders a self-contained universal package " + "carries, or for which a single file of an image is made; this pack is neither.\n" + " use: --mode self-contained over a platform that states universal = true, or " + "--pkg-format single-file"))}; + if (universal) { + if (!opts.systems.empty() && opts.mode != mcpp::pack::Mode::BundleAll) + return Refusal{2, about(run, m, std::format( + "--systems names the systems whose loaders a self-contained package carries, " + "and --mode {} carries none.\n use: --mode self-contained", + mcpp::pack::mode_cli_name(opts.mode)))}; + opts.universalName = mcpp::manifest::platform::platform_name( + P.package.substr(P.package.find('.') + 1)); + for (auto const& target : P.fields.targets.value) { + auto lt = mcpp::toolchain::triple::parse(target); + if (!lt || lt->str() == t->str()) continue; + mcpp::build::BuildOverrides ov = g.ov; + ov.target_triple = lt->str(); + ov.workspace_members.clear(); + ov.workspace_request.clear(); + auto legCtx = mcpp::build::prepare_build(false, false, {}, ov); + if (!legCtx) return Refusal{1, about(run, m, std::format( + "the {} leg of the universal package: {}", lt->str(), legCtx.error()))}; + auto be = mcpp::build::make_ninja_backend(); + mcpp::build::BuildOptions bo; + if (auto br = be->build(legCtx->plan, bo); !br) { + mcpp::build::report_failed_drive(br.error()); + return Refusal{1, about(run, m, std::format("the {} leg did not build", lt->str()))}; + } + mcpp::build::populate_dependency_cache(*legCtx); + std::filesystem::path artifact; + for (auto const& lu : legCtx->plan.linkUnits) + if (lu.targetName == chosenLu->targetName && lu.kind == chosenLu->kind + && !lu.dependencyOwned && (!sharedPlan || lu.memberOf == m.name)) { + artifact = legCtx->outputDir / lu.output; + break; + } + if (artifact.empty()) return Refusal{1, about(run, m, std::format( + "the {} leg of the universal package built no '{}'", lt->str(), chosenLu->targetName))}; + mcpp::ui::status("Packed leg", lt->str()); + loadedLegs.push_back({ lt->str(), lt->arch, artifact }); + } + } + } + // ─── Build the plan ────────────────────────────────────────────── opts.baseStated = ctx.tc.base.stated(); auto plan = mcpp::pack::make_plan(ctx.manifest, *run.cfg, opts, @@ -569,6 +640,12 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { ctx.plan.runtimeRequirements, programIsSharedObject); if (!plan) return Refusal{1, about(run, m, plan.error().message)}; plan->platform = ctx.plan.platform; + plan->loadedLegs = std::move(loadedLegs); + plan->linkageName = opts.linkage == mcpp::pack::Linkage::Static ? "static" + : opts.linkage == mcpp::pack::Linkage::Shared ? "shared" + : opts.mode == mcpp::pack::Mode::Static ? "static" + : ctx.plan.manifest.buildConfig.dependencyLinkage.empty() + ? "declared" : ctx.plan.manifest.buildConfig.dependencyLinkage; // `--output ` with several members: each member's archive, or its tree // for `--format dir`, is written below the directory under the name it would // have by default. The staging tree of an archive, and of a dispatched @@ -908,14 +985,27 @@ std::optional stage_member(PackRun& run, GroupJob& g, MemberJob& m) { // closure was walked. Best-effort: see write_stage_manifest. Skipped when // no tree exists, so no manifest describes a tree that is not there. if (m.stageFailure.empty()) mcpp::pack::write_stage_manifest(plan->stagingRoot, m.closure); + // A tree a dispatched format reads holds `mcpp-package.json` whether or + // not its closure was walked (SPEC-014 §6). + if (m.stageFailure.empty() && opts.format == mcpp::pack::Format::Dispatched) { + std::error_code jec; + if (!std::filesystem::exists(plan->stagingRoot / mcpp::pack::distribution::kFileName, jec)) + if (auto r = mcpp::pack::finalize_staged(*plan); !r) + return Refusal{1, about(run, m, r.error().message)}; + } // #649 E9: the outcome a machine reader receives, from values this pass // has already answered. Every return goes through it. auto& out = m.result; out.name = m.name; - out.format = opts.format == mcpp::pack::Format::Tar ? std::string("tar") + out.format = opts.format == mcpp::pack::Format::Tar + ? (plan->archive == mcpp::pack::ArchiveKind::Zip ? std::string("zip") : std::string("tar")) : opts.format == mcpp::pack::Format::Dir ? std::string("dir") + : opts.format == mcpp::pack::Format::Zip ? std::string("zip") + : opts.format == mcpp::pack::Format::SingleFile ? std::string("single-file") : opts.formatName; + out.linkage = plan->linkageName; + out.mode = std::string(mcpp::pack::mode_cli_name(opts.mode)); const auto canonical = [](std::string const& t) { if (t.empty()) return mcpp::toolchain::triple::host_triple().str(); auto parsed = mcpp::toolchain::triple::parse(t); @@ -923,6 +1013,8 @@ std::optional stage_member(PackRun& run, GroupJob& g, MemberJob& m) { }; out.targets.push_back(canonical(plan->triple)); for (auto const& leg : plan->extraSharedLegs) out.targets.push_back(leg.triple); + for (auto const& leg : plan->loadedLegs) out.targets.push_back(leg.triple); + out.systems = mcpp::pack::distribution::loader_systems(plan->stagingRoot); if (m.stageFailure.empty()) { out.stageDir = plan->stagingRoot; out.stageManifest = mcpp::pack::stage_manifest_path(plan->stagingRoot); @@ -938,8 +1030,15 @@ std::optional stage_member(PackRun& run, GroupJob& g, MemberJob& m) { m.pathCtx = mcpp::fetcher::make_path_ctx( &cfg, run.selection ? run.selection->root : ctx.projectRoot); - if (opts.format != mcpp::pack::Format::Dispatched) { - auto outPath = (opts.format == mcpp::pack::Format::Tar) + if (opts.format == mcpp::pack::Format::SingleFile) { + auto files = mcpp::pack::single_file(*plan); + if (!files) return Refusal{1, about(run, m, files.error().message)}; + for (auto& f : *files) { + mcpp::ui::status("Packed", mcpp::ui::shorten_path(f, m.pathCtx)); + out.artifacts.push_back(std::move(f)); + } + } else if (opts.format != mcpp::pack::Format::Dispatched) { + auto outPath = plan->archive != mcpp::pack::ArchiveKind::None ? plan->archivePath : plan->stagingRoot; mcpp::ui::status("Packed", mcpp::ui::shorten_path(outPath, m.pathCtx)); out.artifacts.push_back(std::move(outPath)); @@ -1234,7 +1333,7 @@ std::optional check_destinations(PackRun& run, std::vector& g // archive for `tar`; a dispatched format's products are its // provider's, and are compared when they are submitted. std::vector writes{m.plan->stagingRoot}; - if (m.opts.format == mcpp::pack::Format::Tar) writes.push_back(m.plan->archivePath); + if (m.plan->archive != mcpp::pack::ArchiveKind::None) writes.push_back(m.plan->archivePath); for (auto const& w : writes) { auto [at, fresh] = writer.try_emplace(w.lexically_normal(), m.name); if (!fresh) @@ -1263,9 +1362,13 @@ PackOutcome run_pack(PackRun run) { // ─── Build first (pack implies a fresh build) ──────────────────── mcpp::build::BuildOverrides base; if (run.opts.mode == mcpp::pack::Mode::Static && run.opts.targetTriple.empty()) - base.target_triple = "x86_64-linux-musl"; + base.target_triple = std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch); else base.target_triple = run.opts.targetTriple; + // The linkage axis (SPEC-014 §1): the whole graph's dependency linkage for + // this build. + if (run.opts.linkage == mcpp::pack::Linkage::Static) base.dependency_linkage = "static"; + if (run.opts.linkage == mcpp::pack::Linkage::Shared) base.dependency_linkage = "shared"; // A bundled program leaves this machine: release is the fallback, not dev. // `[build] default-profile` still decides when the project states one. base.profile = run.opts.profile; @@ -1425,6 +1528,9 @@ PackOutcome run_pack(PackRun run) { out.stageDir = first.stageDir; out.stageManifest = first.stageManifest; out.closure = first.closure; + out.linkage = first.linkage; + out.mode = first.mode; + out.systems = first.systems; return out; } diff --git a/src/pack/zip.cppm b/src/pack/zip.cppm index 0460a580a..c5add48e6 100644 --- a/src/pack/zip.cppm +++ b/src/pack/zip.cppm @@ -11,12 +11,11 @@ // own OS is the thing being fixed; picking an archiver that only exists there // would reintroduce it one layer down. // -// WHY STORED, NOT DEFLATED. Entries are written with method 0 (stored). That -// is a real cost — a release zip is roughly the size of its contents — and it -// is the honest trade for now: a DEFLATE encoder is the only part of this -// file that could produce an archive which UNPACKS WRONG rather than failing -// loudly, and mcpp has no zlib to borrow one from. Adding compression later -// changes only `write()`; the archive layout below does not move. +// STORED OR DEFLATED. A member is DEFLATEd (method 8) by zlib, the reference +// encoder, when the archive asks for it and the member shrinks; otherwise it is +// stored. A package's zip is deflated by default (SPEC-014 §5); `--compress +// none` stores every member, and a member whose data is aligned (a mapped +// image) is stored whatever was asked. // // Deterministic by construction: no timestamps are read (every entry gets a // fixed DOS date), and entries are written in the order given. Two packs of @@ -26,6 +25,7 @@ export module mcpp.pack.zip; import std; +import mcpp.pack.compress; // DEFLATE for a member that shrinks export namespace mcpp::pack::zip { @@ -41,6 +41,24 @@ struct Entry { // file — which matters because a Windows artifact is routinely inspected // (and cross-tested under wine) from Linux. bool executable = false; + // A symbolic link: the member is the link (Unix mode 0120777) and its data + // is the target, as Info-ZIP writes and reads one. `source` is unread. + std::string symlinkTarget; +}; + +// How the members are written. +// +// deflate each member is DEFLATEd (method 8) when that makes it smaller, +// and stored otherwise; `level` 0 is zlib's default. +// align a STORED member's data starts at a multiple of this many bytes +// from the start of the archive, the local header's extra field +// padding the difference (as zipalign does): a reader maps the +// member where it lies. 0: no alignment. A member that is aligned +// is stored, whatever `deflate` says. +struct WriteOptions { + bool deflate = false; + int level = 0; + std::uint32_t align = 0; }; // Write `entries` to `out`, creating parent directories as needed. @@ -51,7 +69,8 @@ struct Entry { // clang 20.1.7 (MSVC ABI) segfaulted compiling the consumer — see the note at // mcpp.pack's `run_pe`. There is exactly one caller. std::expected -write(const std::filesystem::path& out, const std::vector& entries); +write(const std::filesystem::path& out, const std::vector& entries, + const WriteOptions& options = {}); // CRC-32 (IEEE), exposed for tests: an archive whose CRCs are wrong extracts // with a warning on some tools and silently on others, so this is worth @@ -97,8 +116,8 @@ constexpr std::uint16_t kDosDate = 0x0021; // (1980-1980)<<9 | 1<<5 | 1 // External attributes. The high 16 bits are the Unix mode when the // version-made-by byte says Unix (3); the low byte is the DOS attribute set. -std::uint32_t external_attrs(bool executable) { - std::uint32_t mode = executable ? 0100755u : 0100644u; +std::uint32_t external_attrs(bool executable, bool symlink = false) { + std::uint32_t mode = symlink ? 0120777u : executable ? 0100755u : 0100644u; return mode << 16; } @@ -121,22 +140,25 @@ std::uint32_t crc32(std::string_view data) { } std::expected -write(const std::filesystem::path& out, const std::vector& entries) { +write(const std::filesystem::path& out, const std::vector& entries, + const WriteOptions& options) { std::error_code ec; if (out.has_parent_path()) std::filesystem::create_directories(out.parent_path(), ec); struct Central { std::string name; - std::uint32_t crc = 0, size = 0, offset = 0; - bool executable = false; + std::uint32_t crc = 0, size = 0, packed = 0, offset = 0; + std::uint16_t method = 0; + bool executable = false, symlink = false; }; std::vector central; central.reserve(entries.size()); std::string blob; for (auto const& e : entries) { - auto data = detail::slurp(e.source); + const bool symlink = !e.symlinkTarget.empty(); + auto data = symlink ? std::optional(e.symlinkTarget) : detail::slurp(e.source); if (!data) return std::unexpected(std::format( "cannot read '{}' for the archive", e.source.string())); @@ -155,20 +177,43 @@ write(const std::filesystem::path& out, const std::vector& entries) { c.size = static_cast(data->size()); c.offset = static_cast(blob.size()); c.executable = e.executable; + c.symlink = symlink; + + // The member's bytes: DEFLATEd when asked and smaller, stored + // otherwise, and stored when its data is to be aligned. + std::string packed; + if (options.deflate && options.align == 0 && !symlink && !data->empty()) { + auto d = mcpp::pack::compress::deflate_raw(*data, options.level); + if (!d) return std::unexpected(std::format("'{}': {}", e.source.string(), d.error())); + if (d->size() < data->size()) { packed = std::move(*d); c.method = 8; } + } + const std::string& body = c.method == 8 ? packed : *data; + c.packed = static_cast(body.size()); + + // The padding that puts a stored member's data on the boundary. + std::uint16_t extra = 0; + if (options.align > 1 && c.method == 0) { + const auto dataAt = blob.size() + 30 + c.name.size(); + const auto rem = dataAt % options.align; + if (rem != 0) extra = static_cast(options.align - rem); + // An extra field holds at most 64 KiB - 1 bytes; a boundary of + // 64 KiB is reached by at most that many. + } detail::put32(blob, 0x04034b50); // local file header detail::put16(blob, 20); // version needed (2.0) detail::put16(blob, 0); // flags — no data descriptor - detail::put16(blob, 0); // method 0: stored + detail::put16(blob, c.method); // 0 stored, 8 deflated detail::put16(blob, detail::kDosTime); detail::put16(blob, detail::kDosDate); detail::put32(blob, c.crc); - detail::put32(blob, c.size); // compressed == uncompressed + detail::put32(blob, c.packed); detail::put32(blob, c.size); detail::put16(blob, static_cast(c.name.size())); - detail::put16(blob, 0); // extra field length + detail::put16(blob, extra); // extra field length blob += c.name; - blob += *data; + blob.append(extra, '\0'); + blob += body; central.push_back(std::move(c)); } @@ -181,18 +226,18 @@ write(const std::filesystem::path& out, const std::vector& entries) { detail::put16(blob, (3 << 8) | 20); detail::put16(blob, 20); // version needed detail::put16(blob, 0); - detail::put16(blob, 0); // stored + detail::put16(blob, c.method); detail::put16(blob, detail::kDosTime); detail::put16(blob, detail::kDosDate); detail::put32(blob, c.crc); - detail::put32(blob, c.size); + detail::put32(blob, c.packed); detail::put32(blob, c.size); detail::put16(blob, static_cast(c.name.size())); detail::put16(blob, 0); // extra detail::put16(blob, 0); // comment detail::put16(blob, 0); // disk number detail::put16(blob, 0); // internal attrs - detail::put32(blob, detail::external_attrs(c.executable)); + detail::put32(blob, detail::external_attrs(c.executable, c.symlink)); detail::put32(blob, c.offset); blob += c.name; } diff --git a/src/platform/platform.cppm b/src/platform/platform.cppm index 2b3abee0b..a88d695b5 100644 --- a/src/platform/platform.cppm +++ b/src/platform/platform.cppm @@ -319,6 +319,7 @@ void merge(mp::Statement& b, const mp::Statement& o) { for (auto const& [k, v] : o.runners) b.runners[k] = v; if (o.carrier.set) b.carrier = o.carrier; text(b.plugin, o.plugin); + if (o.universal.set) b.universal = o.universal; for (auto const& [k, v] : o.layers) b.layers[k] = v; for (auto const& [k, v] : o.tools) b.tools[k] = v; append(b.build.cflags, o.build.cflags); From 58fcb81f30945f0382befbcf8e58e01f7fc18420 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:00:07 +0900 Subject: [PATCH 18/54] pack: mcpp emit xpkg --package describes packages mcpp pack wrote (a universal package one asset for every system, its runner a dependency); upgrade lane: a previous release of the same BMI spelling is served from the cache; 330 lists the platform key Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .github/tools/upgrade_lane.sh | 17 ++- src/cli.cppm | 7 +- src/cli/cmd_publish.cppm | 28 +++++ src/pack/distribution.cppm | 158 +++++++++++++++++++++++++ tests/e2e/330_runner_hosted_targets.sh | 2 +- 5 files changed, 206 insertions(+), 6 deletions(-) diff --git a/.github/tools/upgrade_lane.sh b/.github/tools/upgrade_lane.sh index be23c19da..e2a0bb811 100644 --- a/.github/tools/upgrade_lane.sh +++ b/.github/tools/upgrade_lane.sh @@ -218,12 +218,21 @@ if [ "$VARIANT" = snapshot ]; then || fail "toolchain list does not offer the newer packaging" fi if [ "$VARIANT" = modules ]; then - # A BMI of the previous spelling is not read: the dependency is rebuilt - # once (its cache key carries the spelling), then served from the cache. + # A BMI of another spelling is not read: the dependency is rebuilt once (its + # cache key carries the spelling), then served from the cache. The spelling + # moved in 2026.10.26.2 (the module map's `$root`); a previous release from + # then on spells as this one, and its entry is served at once. + old_version=$("$OLD" --version | grep -Eo '[0-9]{4}\.[0-9]+\.[0-9]+\.[0-9]+' | head -1) + same_spelling=$(printf '%s\n2026.10.26.2\n' "$old_version" | sort -t. -k1,1n -k2,2n -k3,3n -k4,4n | head -1) modules_project m-new1 run new-modules1 "$W/m-new1" "$NEW" build -v || fail "the module project does not build over the previous release's cache" - grep -q "STAGE gcm.cache/mcpplibs.openkal/" "$W/new-modules1.log" \ - && fail "a dependency BMI of the previous spelling was taken from the cache" + if [ "$same_spelling" = 2026.10.26.2 ]; then + grep -q "STAGE gcm.cache/mcpplibs.openkal/" "$W/new-modules1.log" \ + || fail "a dependency BMI of the same spelling ($old_version) was not served from the cache" + else + grep -q "STAGE gcm.cache/mcpplibs.openkal/" "$W/new-modules1.log" \ + && fail "a dependency BMI of the previous spelling ($old_version) was taken from the cache" + fi modules_project m-new2 run new-modules2 "$W/m-new2" "$NEW" build -v || fail "the second module project failed" grep -q "STAGE gcm.cache/mcpplibs.openkal/" "$W/new-modules2.log" \ diff --git a/src/cli.cppm b/src/cli.cppm index 0a03425dd..50527d44b 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -978,7 +978,12 @@ int run(int argc, char** argv) { .option(cl::Option("namespace").takes_value().value_name("NS") .help("Package namespace for the emitted descriptor " "(overrides [package] namespace). Emits both " - "`namespace` and the fully-qualified `name`"))) + "`namespace` and the fully-qualified `name`")) + .option(cl::Option("package").takes_value().multiple().value_name("ARCHIVE") + .help("Describe packages `mcpp pack` wrote (repeatable): an xim-pkgindex entry " + "of the program, from their mcpp-package.json")) + .option(cl::Option("url").takes_value().value_name("PREFIX") + .help("With --package: where the packages are published (/)"))) // `sbom` belongs HERE rather than at the top level: `emit` // already means "generate a document describing this project" and // already carries `-o`. A separate `mcpp sbom` would be a second diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 8e7cc9aa1..1772636ef 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -38,6 +38,34 @@ namespace mcpp::cli { // `mcpp emit xpkg ...` — only one subcommand defined, so the action sits // directly on the `emit xpkg` nested subcommand and receives its ParsedArgs. export int cmd_emit_xpkg(const mcpplibs::cmdline::ParsedArgs& parsed) { + // `--package`: the distribution description of packages `mcpp pack` wrote + // (SPEC-014 §8), read from their mcpp-package.json. + if (parsed.option("package")) { + namespace d = mcpp::pack::distribution; + std::vector assets; + std::error_code ec; + const auto work = std::filesystem::temp_directory_path(ec) / std::format( + "mcpp-emit-{}", std::chrono::steady_clock::now().time_since_epoch().count()); + for (auto const& p : parsed.option("package")->get().values) { + const std::filesystem::path file{p}; + auto root = d::unpack(file, work / std::to_string(assets.size())); + if (!root) { mcpp::ui::error(root.error()); std::filesystem::remove_all(work, ec); return 2; } + auto pkg = d::read(*root); + if (!pkg) { mcpp::ui::error(pkg.error()); std::filesystem::remove_all(work, ec); return 2; } + assets.push_back({ file, std::move(*pkg), d::sha256_file(file) }); + } + std::filesystem::remove_all(work, ec); + auto lua = d::xpkg_descriptor(assets, parsed.value("url").value_or("")); + if (!lua) { mcpp::ui::error(lua.error()); return 2; } + if (auto out = parsed.value("output")) { + std::ofstream os(*out, std::ios::binary | std::ios::trunc); + os << *lua; + if (!os) { mcpp::ui::error(std::format("cannot write '{}'", *out)); return 1; } + return 0; + } + std::print("{}", *lua); + return 0; + } return mcpp::publish::emit_xpkg_to( parsed.option_or_empty("version").value(), std::filesystem::path{parsed.option_or_empty("output").value()}, diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 66a0db2dc..ecf62bded 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -122,6 +122,25 @@ std::expected tree_payload(const std::filesystem::path std::string launcher_manifest(std::string_view name, std::string_view version); std::string_view launcher_main(); +// ─── the distribution description (SPEC-014 §8) ─────────────────────────── + +// A package written by `mcpp pack`, as an index entry reads it. +struct Asset { + std::filesystem::path file; + Package pkg; + std::string sha256; +}; + +// The xim-pkgindex entry (spec 2) of `assets`, fetched from `/`: a universal package is one asset every system installs, and its +// runner is a dependency unless the package carries its loaders; packages of +// one system and architecture each are that system's entry, the architecture +// in the file name a `${arch}` of one URL. The install moves the archive's +// one top-level directory into place; the program is registered from its +// launcher (`bin/`, `bin/.cmd` on Windows) or the program itself. +std::expected xpkg_descriptor(const std::vector& assets, + std::string_view urlPrefix); + // ─── --merge (SPEC-014 §4.6) ────────────────────────────────────────────── struct MergeRequest { @@ -857,6 +876,145 @@ int main(int argc, char** argv) { )MCPP"; } +// ─── the distribution description ───────────────────────────────────────── + +namespace { + +// The xlings system key of a triple's system. +std::string xlings_system(std::string_view triple) { + if (triple.find("-windows") != std::string_view::npos) return "windows"; + if (triple.find("-macos") != std::string_view::npos || triple.find("-darwin") != std::string_view::npos + || triple.find("-apple") != std::string_view::npos) + return "macosx"; + return "linux"; +} + +std::string lua_list(const std::vector& v) { + std::string out = "{"; + for (std::size_t i = 0; i < v.size(); ++i) out += std::format("{}\"{}\"", i ? ", " : "", v[i]); + return out + "}"; +} + +} // namespace + +std::expected xpkg_descriptor(const std::vector& assets, + std::string_view urlPrefix) { + if (assets.empty()) return std::unexpected(std::string("no package to describe")); + auto const& first = assets.front().pkg; + std::string prefix(urlPrefix); + while (!prefix.empty() && prefix.back() == '/') prefix.pop_back(); + if (prefix.empty()) + return std::unexpected(std::string("--url names where the packages are published")); + for (auto const& a : assets) + if (a.pkg.name != first.name || a.pkg.version != first.version) + return std::unexpected(std::format("{} is {} {}, and {} is {} {}: one entry describes one " + "package", assets.front().file.filename().string(), + first.name, first.version, a.file.filename().string(), + a.pkg.name, a.pkg.version)); + const bool universal = !first.runner.empty(); + std::vector archs; + // system -> block text + std::map blocks; + if (universal) { + if (assets.size() != 1) + return std::unexpected(std::string("a universal package is one asset; merge its legs first " + "(mcpp pack --merge)")); + auto const& a = assets.front(); + for (auto const& [arch, _] : first.entry) archs.push_back(arch); + std::string deps; + if (first.mode != "self-contained") + deps = std::format(" deps = {{ \"{}\" }},\n", + std::filesystem::path(first.runner.front()).stem().string()); + for (auto system : { "linux", "macosx", "windows" }) + blocks[system] = std::format( + "{0} [\"latest\"] = {{ ref = \"{1}\" }},\n" + " [\"{1}\"] = {{\n url = \"{2}/{3}\",\n sha256 = \"{4}\",\n }},\n", + deps, first.version, prefix, a.file.filename().string(), a.sha256); + } else { + // system -> (arch -> asset) + std::map> bySystem; + for (auto const& a : assets) { + if (a.pkg.legs.size() != 1) + return std::unexpected(std::format("{} holds {} legs; an entry per system describes " + "packages of one leg", a.file.filename().string(), + a.pkg.legs.size())); + const auto& triple = a.pkg.legs.front(); + const auto arch = triple.substr(0, triple.find('-')); + bySystem[xlings_system(triple)][arch] = &a; + if (std::ranges::find(archs, arch) == archs.end()) archs.push_back(arch); + } + for (auto const& [system, byArch] : bySystem) { + // One URL with `${arch}`: the file names may differ by the + // architecture alone. + std::string tmpl; + std::string shas; + for (auto const& [arch, a] : byArch) { + auto name = a->file.filename().string(); + if (auto at = name.find(arch); at != std::string::npos) name.replace(at, arch.size(), "${arch}"); + if (!tmpl.empty() && tmpl != name) + return std::unexpected(std::format("the {} packages' names differ beyond the " + "architecture ({} and {})", system, tmpl, name)); + tmpl = name; + shas += std::format(" {} = \"{}\",\n", arch, a->sha256); + } + blocks[system] = std::format( + " [\"latest\"] = {{ ref = \"{0}\" }},\n" + " [\"{0}\"] = {{\n url = \"{1}/{2}\",\n sha256 = {{\n{3} }},\n }},\n", + first.version, prefix, tmpl, shas); + } + } + std::ranges::sort(archs); + std::string xpm; + for (auto const& [system, text] : blocks) xpm += std::format(" {} = {{\n{} }},\n", system, text); + + // Where the program a person runs lies in the installed tree. + std::string bindir = "\"bin\"", alias; + if (universal) { + alias = std::format("if os.host() == \"windows\" then opts.alias = \"{}.cmd\" end\n ", first.name); + } else { + const auto program = std::filesystem::path(first.program); + bindir = program.has_parent_path() ? std::format("\"{}\"", program.parent_path().generic_string()) + : std::string("\".\""); + } + const auto wrapper = std::format("{}-{}-", first.name, first.version); + return std::format( + "-- Generated by `mcpp emit xpkg --package` from the packages' mcpp-package.json\n" + "-- (SPEC-014 §8).\n" + "package = {{\n" + " spec = \"2\",\n" + " name = \"{0}\",\n" + " description = \"{0} {1}{2}\",\n" + " type = \"package\",\n" + " archs = {3},\n" + " programs = {{\"{0}\"}},\n" + " xvm_enable = true,\n" + " xpm = {{\n{4} }},\n" + "}}\n\n" + "import(\"xim.libxpkg.pkginfo\")\n" + "import(\"xim.libxpkg.xvm\")\n\n" + "function install()\n" + " -- The archive's one top-level directory is the package.\n" + " os.tryrm(pkginfo.install_dir())\n" + " for _, d in ipairs(os.dirs(\"{5}*\")) do\n" + " os.mv(d, pkginfo.install_dir())\n" + " return true\n" + " end\n" + " return false\n" + "end\n\n" + "function config()\n" + " local opts = {{ bindir = path.join(pkginfo.install_dir(), {6}) }}\n" + " {7}xvm.add(\"{0}\", opts)\n" + " return true\n" + "end\n\n" + "function uninstall()\n" + " xvm.remove(\"{0}\")\n" + " return true\n" + "end\n", + first.name, first.version, + universal ? std::format(", one package for every system ({})", first.platform) : std::string(), + lua_list(archs), xpm, wrapper, bindir, alias); +} + // ─── --merge ────────────────────────────────────────────────────────────── std::expected merge(const MergeRequest& req) { diff --git a/tests/e2e/330_runner_hosted_targets.sh b/tests/e2e/330_runner_hosted_targets.sh index f4eae9a69..3392503c8 100644 --- a/tests/e2e/330_runner_hosted_targets.sh +++ b/tests/e2e/330_runner_hosted_targets.sh @@ -169,7 +169,7 @@ fi printf '\n[target.%s]\nrunnerX = ["x"]\n' "$HOST" >> mcpp.toml out=$("$MCPP" build 2>&1) grep -q "unsupported key 'runnerX'" <<<"$out" || fail "array typo not reported: $out" -grep -q "Supported keys: cxx_runtime, linkage, min_api_level, run_under, runner, sysroot, toolchain" <<<"$out" \ +grep -q "Supported keys: cxx_runtime, linkage, min_api_level, platform, run_under, runner, sysroot, toolchain" <<<"$out" \ || fail "runner missing from the supported-keys list: $out" echo "PASS: 330_runner_hosted_targets" From a0978e715e14d5e835fb6a6d00c039ca5c4926ae Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:14:54 +0900 Subject: [PATCH 19/54] platform, pack: e2e 952 (a platform package of this machine's triples), 953 (the host graph), 954 (three axes), 955 (universal), 956 (single file), 957 (merge, emit xpkg); the host configuration reports its runner and tools; --systems is refused before the build; [pack] compress yields to a container it does not fit; --merge reads the format from the output's name Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/build/prepare/host_graph.cpp | 11 ++ src/cli/cmd_build.cppm | 22 +-- src/cli/cmd_publish.cppm | 17 ++- src/pack/distribution.cppm | 11 +- src/pack/pipeline.cppm | 47 +++++-- ...tform_package_states_a_class_of_targets.sh | 125 ++++++++++++++++++ ...ost_graph_runs_build_programs_as_images.sh | 96 ++++++++++++++ tests/e2e/954_pack_has_three_axes.sh | 121 +++++++++++++++++ ...versal_package_holds_every_architecture.sh | 95 +++++++++++++ ...56_a_single_file_runs_without_unpacking.sh | 96 ++++++++++++++ ..._packages_merge_and_describe_themselves.sh | 119 +++++++++++++++++ 11 files changed, 733 insertions(+), 27 deletions(-) create mode 100755 tests/e2e/952_a_platform_package_states_a_class_of_targets.sh create mode 100755 tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh create mode 100755 tests/e2e/954_pack_has_three_axes.sh create mode 100755 tests/e2e/955_a_universal_package_holds_every_architecture.sh create mode 100755 tests/e2e/956_a_single_file_runs_without_unpacking.sh create mode 100755 tests/e2e/957_packages_merge_and_describe_themselves.sh diff --git a/src/build/prepare/host_graph.cpp b/src/build/prepare/host_graph.cpp index 48f56e6ce..c041e9f65 100644 --- a/src/build/prepare/host_graph.cpp +++ b/src/build/prepare/host_graph.cpp @@ -793,6 +793,10 @@ build_group(PrepareState& state, const fs::path& dir, const std::vector if (auto it = ctx->plan.platform.tools.find(argv.front()); it != ctx->plan.platform.tools.end()) argv.front() = it->second; state.hostPlatformRunner = std::move(argv); + // The tools the host graph's build located, for the report of the + // host configuration (`self env --host`). + state.hostPlatform.tools = ctx->plan.platform.tools; + state.hostPlatform.fields.runner = pr; } std::map binaries; std::vector ninjaGoals; @@ -1028,6 +1032,13 @@ ensure_build_programs(PrepareState& state, const std::vector& packa std::vector runner; if (it->contains("runner") && (*it)["runner"].is_array()) for (auto const& a : (*it)["runner"]) runner.push_back(a.get()); + // A program built under the host platform by an earlier run: + // its runner as that run located it, for the report. + if (!runner.empty() && state.hostPlatformRunner.empty()) { + state.hostPlatformRunner = runner; + auto const& cmd = state.hostPlatform.fields.runner.cmd; + if (!cmd.empty()) state.hostPlatform.tools[cmd.front()] = runner.front(); + } state.buildProgramBinaries[p.index] = {fs::path(it->value("binary", std::string{})), p.identity, {}, std::move(runner)}; mcpp::log::verbose("build-program", std::format( diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 3f9c95797..22547d1c0 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -1653,13 +1653,19 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { return 1; } auto const& P = host ? ctx->hostPlatform : ctx->plan.platform; - auto const& runner = ctx->plan.platform.fields.runner; + auto const& runner = P.fields.runner; std::string runnerProgram; - if (!host && runner.set && !runner.cmd.empty()) { - auto found = mcpp::build::runner_lookup::locate( - runner.cmd.front(), ctx->xlingsDepBinDirs, - mcpp::invocation::get("PATH").value_or("")); - if (found.program) runnerProgram = found.program->generic_string(); + if (runner.set && !runner.cmd.empty()) { + // The platform's own tool when it is one (located by the build that + // used it), else the program the runner names on this machine. + if (auto it = P.tools.find(runner.cmd.front()); it != P.tools.end()) { + runnerProgram = it->second; + } else { + auto found = mcpp::build::runner_lookup::locate( + runner.cmd.front(), ctx->xlingsDepBinDirs, + mcpp::invocation::get("PATH").value_or("")); + if (found.program) runnerProgram = found.program->generic_string(); + } } if (json) { nlohmann::json tools = nlohmann::json::object(); @@ -1671,7 +1677,7 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { {"origin", ctx->compilerChoice.origin}}}, {"platform", nlohmann::json::parse(mcpp::platform::to_json(P), nullptr, false)}, {"tools", std::move(tools)}, - {"runner", host || !runner.set ? nlohmann::json(nullptr) : nlohmann::json{ + {"runner", !runner.set ? nlohmann::json(nullptr) : nlohmann::json{ {"argv", runner.cmd}, {"host", runner.host}, {"exclusive", runner.exclusive}, {"program", runnerProgram.empty() ? nlohmann::json(nullptr) : nlohmann::json(runnerProgram)}, {"source", mcpp::platform::origin_text(runner.origin)}}}, @@ -1695,7 +1701,7 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { std::println("{:<10} {:<17} {}", "", label, value); for (auto const& [name, path] : P.tools) std::println("{:<10} {:<17} {}", "tool", name, path); - if (!host && runner.set) { + if (runner.set) { std::string argv; for (auto const& a : runner.cmd) argv += (argv.empty() ? "" : " ") + a; std::println("{:<10} {} ({}){}", "runner", argv, mcpp::platform::origin_text(runner.origin), diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 1772636ef..842ee6b41 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -655,14 +655,27 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, } opts.linkage = mcpp::pack::Linkage::Static; } - std::string compressText = parsed.value("compress").value_or(packCfg.compress); + // `--compress` states the container's compression; `[pack] compress` + // is the project's default, which a container it does not fit (a + // directory, a zip given zstd, a single file given xz) leaves aside. + const auto compressFlag = parsed.value("compress"); + std::string compressText = compressFlag.value_or(packCfg.compress); if (!compressText.empty()) { auto c = mcpp::pack::compress::parse(compressText); - if (!c) { mcpp::ui::error(std::format("--compress: {}", c.error())); return 2; } + if (!c) { + mcpp::ui::error(std::format("{}: {}", compressFlag ? "--compress" : "[pack] compress", + c.error())); + return 2; + } opts.compression = *c; } using mcpp::pack::compress::Algorithm; const auto alg = opts.compression.algorithm; + const bool fits = + !(opts.format == mcpp::pack::Format::Dir || opts.format == mcpp::pack::Format::Dispatched) + && !(opts.format == mcpp::pack::Format::Zip && (alg == Algorithm::Zstd || alg == Algorithm::Xz)) + && !(opts.format == mcpp::pack::Format::SingleFile && (alg == Algorithm::Gzip || alg == Algorithm::Xz)); + if (opts.compression.stated && !compressFlag && !fits) opts.compression = {}; if (opts.compression.stated) { if (opts.format == mcpp::pack::Format::Dir || opts.format == mcpp::pack::Format::Dispatched) { mcpp::ui::error(std::format( diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index ecf62bded..f8a8022a9 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -1085,11 +1085,16 @@ std::expected merge(const MergeRequest& req) { std::ranges::sort(merged.systems); if (auto d = describe(out, merged, stated); !d) return std::unexpected(d.error()); - // The container: as asked, else as the first input was written. + // The container: as asked, else as the output's name says, else as the + // first input was written. std::string format = req.format; - const auto firstName = req.inputs.front().filename().string(); + const auto outName = req.output.filename().string(); + const auto firstName = !req.output.empty() && (outName.ends_with(".zip") || outName.find(".tar") != std::string::npos) + ? outName : req.inputs.front().filename().string(); if (format.empty()) - format = fs::is_directory(req.inputs.front(), ec) ? "dir" : firstName.ends_with(".zip") ? "zip" : "tar"; + format = firstName.ends_with(".zip") ? "zip" + : firstName.find(".tar") != std::string::npos ? "tar" + : fs::is_directory(req.inputs.front(), ec) ? "dir" : "tar"; auto compression = req.compression; if (!compression.stated && format == "tar") { using mcpp::pack::compress::Algorithm; diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 809500b96..10e9ff444 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -403,6 +403,17 @@ bool member_carries(const mcpp::build::BuildContext& ctx, const MemberJob& m, // // Called inside the member's view (`with_member`): `ctx.manifest`, // `ctx.projectRoot` and the plan's link group are the member's. +// Whether a pack of `ctx`'s target is a universal package (SPEC-014 §3): an +// image of a platform that states `universal = true`, not refused by the +// command or `[pack] universal`, in a format other than a single file. +bool packs_universal(const mcpp::build::BuildContext& ctx, const Options& opts) { + auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + auto const& P = ctx.plan.platform; + return t && t->loader_supplies_os() && P.active && P.fields.universal.set + && P.fields.universal.value && opts.universal.value_or(true) + && opts.format != mcpp::pack::Format::SingleFile; +} + std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { auto& ctx = *g.ctx; const bool sharedPlan = ctx.workspaceMembers.size() > 1; @@ -582,20 +593,7 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { const bool loaded = t && t->loader_supplies_os(); if (loaded && opts.mode == mcpp::pack::Mode::Static) opts.mode = mcpp::pack::Mode::BundleAll; auto const& P = ctx.plan.platform; - const bool universal = loaded && P.active && P.fields.universal.set && P.fields.universal.value - && opts.universal.value_or(true) && opts.format != mcpp::pack::Format::SingleFile; - if (!universal && !opts.systems.empty() && !(loaded && opts.format == mcpp::pack::Format::SingleFile)) - return Refusal{2, about(run, m, std::format( - "--systems names the systems whose loaders a self-contained universal package " - "carries, or for which a single file of an image is made; this pack is neither.\n" - " use: --mode self-contained over a platform that states universal = true, or " - "--pkg-format single-file"))}; - if (universal) { - if (!opts.systems.empty() && opts.mode != mcpp::pack::Mode::BundleAll) - return Refusal{2, about(run, m, std::format( - "--systems names the systems whose loaders a self-contained package carries, " - "and --mode {} carries none.\n use: --mode self-contained", - mcpp::pack::mode_cli_name(opts.mode)))}; + if (packs_universal(ctx, opts)) { opts.universalName = mcpp::manifest::platform::platform_name( P.package.substr(P.package.find('.') + 1)); for (auto const& target : P.fields.targets.value) { @@ -772,6 +770,27 @@ std::optional prepare_group(PackRun& run, GroupJob& g) { std::optional check_format(PackRun& run, GroupJob& g) { auto const& ctx = *g.ctx; const auto& formatName = run.opts.formatName; + // ─── `--systems` names the loaders a package carries ───────────── + // + // A self-contained universal package carries a loader per system, and a + // single file of an image is made per system; nothing else is made per + // system. Refused here, before the build (SPEC-014 §1.5). + if (!run.opts.systems.empty()) { + auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + const bool loaded = t && t->loader_supplies_os(); + const bool universal = packs_universal(ctx, run.opts); + const bool selfContained = run.opts.mode == mcpp::pack::Mode::BundleAll + || run.opts.mode == mcpp::pack::Mode::Static; + if (!(universal && selfContained) && !(loaded && run.opts.format == mcpp::pack::Format::SingleFile)) + return Refusal{2, universal + ? std::format("--systems names the systems whose loaders a self-contained package " + "carries, and --mode {} carries none.\n use: --mode self-contained", + mcpp::pack::mode_cli_name(run.opts.mode)) + : std::string("--systems names the systems whose loaders a self-contained universal " + "package carries, or for which a single file of an image is made; this " + "pack is neither.\n use: --mode self-contained over a platform that " + "states universal = true, or --pkg-format single-file")}; + } // ─── Is the requested format one anything provides? ────────────── // // BEFORE THE BUILD, because a refusal that arrives after a full compile is diff --git a/tests/e2e/952_a_platform_package_states_a_class_of_targets.sh b/tests/e2e/952_a_platform_package_states_a_class_of_targets.sh new file mode 100755 index 000000000..de2e32eac --- /dev/null +++ b/tests/e2e/952_a_platform_package_states_a_class_of_targets.sh @@ -0,0 +1,125 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 952 -- a platform package that has nothing to do with openkal states the +# configuration of a class of targets, and the engine carries every group of it +# (SPEC-013 §4; design 2026-10-28 §12.1, the board-support row). +# +# The platform here is a local directory for this machine's own triples. It +# states compile flags, the name of a program, a seal step and a runner; the +# project selects it by path for its system and overrides one field under +# `cfg(platform = "demo")`. +# +# Criteria: +# A. The platform's compile flag reaches the project's units; the program is +# named by the platform's pattern; the seal step ran on it before it was +# published; `mcpp run` starts it through the platform's runner. +# B. The report names the platform and the source of its selection. +# C. An override under `cfg(platform = "demo")` takes the field's place +# (`link.exe`), and the report's JSON says where each field came from. +# D. `--platform none` builds the project without the platform: the default +# name, no flag, no seal, no runner. +# E. A platform package without `[platform]` is refused, naming the package. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/plat" "$TMP/app/src" +cat > "$TMP/plat/mcpp.toml" <<'TOML' +[package] +name = "demo-platform" +version = "0.1.0" + +[platform] +targets = ["x86_64-linux-gnu", "aarch64-linux-gnu", "aarch64-macos", "x86_64-macos"] +runner = ["env", "DEMO_PLATFORM_RUNNER=yes", "{}"] + +[platform.build] +cxxflags = ["-DDEMO_PLATFORM=1"] + +[platform.link] +exe = "{name}.demo" +seal = [["sh", "-c", "printf DEMO-SEALED >> \"$0\"", "{}"]] +TOML + +cd "$TMP/app" +cat > mcpp.toml <<'TOML' +[package] +name = "app" +version = "0.1.0" + +[target.'cfg(any(linux, macos))'] +platform = { path = "../plat" } +TOML +cat > src/main.cpp <<'CPP' +#include +#include +int main() { +#ifdef DEMO_PLATFORM + const char* flag = "flag"; +#else + const char* flag = "no-flag"; +#endif + const char* runner = std::getenv("DEMO_PLATFORM_RUNNER"); + std::printf("demo: %s, runner %s\n", flag, runner ? runner : "none"); + return 0; +} +CPP + +# A +"$MCPP" build > build.log 2>&1 || fail "A: build" build.log +prog=$(ls target/*/*/bin/app.demo 2>/dev/null | head -1) +[ -n "$prog" ] || fail "A: no program named by the platform's pattern (app.demo)" build.log +[ "$(tail -c 11 "$prog")" = "DEMO-SEALED" ] || fail "A: the seal step did not run before publishing" build.log +"$MCPP" run > run.log 2>&1 || fail "A: run" run.log +grep -q "demo: flag, runner yes" run.log || fail "A: the flag or the runner did not reach the program" run.log +echo "ok: the platform's flag, name, seal and runner reach the program" + +# B +grep -qE "Platform +demo-platform 0\.1\.0 \(mcpp\.toml" build.log || fail "B: the report does not name the platform and its source" build.log +echo "ok: the report names the platform and where it was selected" + +# C +cat >> mcpp.toml <<'TOML' + +[target.'cfg(platform = "demo")'.link] +exe = "{name}.over" +TOML +"$MCPP" build > over.log 2>&1 || fail "C: build with the override" over.log +ls target/*/*/bin/app.over > /dev/null 2>&1 || fail "C: the override did not take the field's place" over.log +host=$(python3 - <<'PY' +import platform +m = platform.machine().lower() +arch = "aarch64" if m in ("arm64", "aarch64") else "x86_64" +print(f"{arch}-macos" if platform.system() == "Darwin" else f"{arch}-linux-gnu") +PY +) +"$MCPP" self env --target "$host" --format json > env.json 2> env.err || fail "C: self env" env.err env.json +python3 - env.json <<'PY' || fail "C: the JSON's sources" env.json +import json, sys +p = json.load(open(sys.argv[1]))["data"]["platform"] +assert p and p["name"].endswith("demo-platform"), p +f = p["fields"] +assert f["link.exe"]["source"].startswith("mcpp.toml"), f["link.exe"] +assert f["runner"]["source"] == "platform", f["runner"] +PY +echo "ok: an override under cfg(platform) takes the field's place, and the JSON says where" + +# D +"$MCPP" run --platform none > none.log 2>&1 || fail "D: run --platform none" none.log +grep -q "demo: no-flag, runner none" none.log || fail "D: --platform none kept a field of the platform" none.log +ls target/*/*/bin/app > /dev/null 2>&1 || fail "D: the program is not under its default name" none.log +echo "ok: --platform none builds without the platform" + +# E +mkdir -p "$TMP/notplat" +printf '[package]\nname = "notplat"\nversion = "0.1.0"\n' > "$TMP/notplat/mcpp.toml" +if "$MCPP" build --platform "path:$TMP/notplat" > notplat.log 2>&1; then + fail "E: a package without [platform] was accepted as a platform" notplat.log +fi +grep -q "has no \[platform\] section" notplat.log || fail "E: the refusal does not say why" notplat.log +grep -q "notplat" notplat.log || fail "E: the refusal does not name the package" notplat.log +echo "ok: a package without [platform] is refused as a platform" + +echo "PASS: 952 a platform package states a class of targets" diff --git a/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh b/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh new file mode 100755 index 000000000..45f7b874c --- /dev/null +++ b/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +# requires: llvm python3 +# 953 -- the host graph's platform (SPEC-013 §6): a project whose `[host]` +# selects openkal-platform builds its build program as an ok image of this +# machine's architecture and runs it through the platform's runner; the build +# program's library is an image of the same graph; the platform's own tools are +# built for this machine as before (the bootstrap boundary). +# +# Criteria: +# A. The build program runs as an image: the report names the host platform, +# and what the program defines reaches the project's unit. +# B. A library of `[build-dependencies]` the build program imports is built +# into the same image graph, and its answer reaches the program. +# C. The runner of the host graph is a program of this machine (the platform's +# `okrun`, built natively): the bootstrap boundary holds. +# D. `--host-platform none` runs the build program as a program of this +# machine: no host platform is reported, and the answer is the same. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64|aarch64|arm64) ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac + +mkdir -p "$TMP/app/src" "$TMP/app/helper/src" +cd "$TMP/app" +cat > mcpp.toml <<'TOML' +[package] +name = "hostgraph" +version = "0.1.0" + +[host] +platform = "openkal-platform@0.17.1" + +[build-dependencies] +helper = { path = "helper" } +TOML +printf '[package]\nname = "helper"\nversion = "0.1.0"\n\n[targets.helper]\nkind = "lib"\n' > helper/mcpp.toml +cat > helper/src/helper.cppm <<'CPP' +export module helper; +import std; +export namespace helper { +int answer() { return static_cast(std::string_view("forty-two").size()) * 4 + 6; } +} +CPP +cat > build.mcpp <<'CPP' +import mcpp; +import std; +import helper; +int main() { + mcpp::define("BUILT_BY_AN_IMAGE=1"); + mcpp::define(std::format("HELPER_ANSWER={}", helper::answer()).c_str()); + return 0; +} +CPP +cat > src/main.cpp <<'CPP' +#include +int main() { std::printf("host graph: %d, answer %d\n", BUILT_BY_AN_IMAGE, HELPER_ANSWER); return 0; } +CPP + +# A, B +"$MCPP" run > run.log 2>&1 || fail "A: the project did not build or run" run.log +grep -qE "Host platform +openkal-platform" run.log || fail "A: the report does not name the host platform" run.log +grep -q "host graph: 1, answer 42" run.log || fail "A/B: the build program's definitions did not arrive" run.log +echo "ok: the build program and its library run as an image" + +# C +"$MCPP" self env --host --format json > host.json 2> host.err || fail "C: self env --host" host.err host.json +python3 - host.json <<'PY' || fail "C: the host graph's runner" host.json +import json, sys +d = json.load(open(sys.argv[1]))["data"] +p = d["platform"] +assert p and p["name"].endswith("openkal-platform"), p +okrun = d["tools"].get("okrun") +assert okrun, d["tools"] +assert d["runner"] and d["runner"]["program"] == okrun, d["runner"] +# A program of this machine, not an image: an executable's magic, and no +# image name. +magic = open(okrun, "rb").read(4) +assert magic in (b"\x7fELF", b"\xcf\xfa\xed\xfe", b"MZ\x90\x00") or magic[:2] == b"MZ", magic +assert not okrun.endswith(".oke"), okrun +PY +echo "ok: the host graph's runner is a program of this machine" + +# D +rm -rf target +"$MCPP" run --host-platform none > none.log 2>&1 || fail "D: --host-platform none" none.log +grep -q "Host platform" none.log && fail "D: a host platform is reported under --host-platform none" none.log +grep -q "host graph: 1, answer 42" none.log || fail "D: the answer differs without the host platform" none.log +echo "ok: --host-platform none runs the build program natively" + +echo "PASS: 953 the host graph runs build programs as images" diff --git a/tests/e2e/954_pack_has_three_axes.sh b/tests/e2e/954_pack_has_three_axes.sh new file mode 100755 index 000000000..305a3b5ed --- /dev/null +++ b/tests/e2e/954_pack_has_three_axes.sh @@ -0,0 +1,121 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 954 -- `mcpp pack` has three axes (SPEC-014 §1): the linkage of the program, +# the closure the package carries (`--mode`), the container it is delivered in +# (`--pkg-format`, compressed as `--compress` says); every package describes +# itself in `mcpp-package.json`; impossible combinations are refused before +# anything is built. +# +# Criteria: +# A. `--compress zstd | xz | none` writes `.tar.zst`, `.tar.xz`, `.tar`, each +# holding one top-level directory; `--pkg-format zip` a zip. +# B. The tree's `mcpp-package.json` names the package, its legs and its three +# axes, and lists every file with its role and a digest that matches it. +# C. `--message-format json` states linkage, mode, pkgFormat and legs, and the +# sha256 of the archive. +# D. `[pack] format` and `[pack] compress` are the project's defaults. +# E. Refused before a build, naming both choices: single-file with --mode +# system; --compress on a directory; zstd in a zip; --systems for a package +# that carries no loader. +# F. `--format` is the earlier spelling of `--pkg-format`, with a warning. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "axes"\nversion = "0.1.0"\n' > mcpp.toml +printf 'int main() { return 0; }\n' > src/main.cpp + +# A +for c in zstd xz none; do + "$MCPP" pack --compress "$c" > "pack-$c.log" 2>&1 || fail "A: --compress $c" "pack-$c.log" +done +"$MCPP" pack --pkg-format zip > pack-zip.log 2>&1 || fail "A: --pkg-format zip" pack-zip.log +python3 - <<'PY' || fail "A: the archives" +import glob, io, lzma, tarfile, zipfile +from pathlib import Path +d = Path("target/dist") +def one_top(names): + tops = {n.split("/")[0] for n in names} + assert len(tops) == 1, tops +xz = glob.glob(str(d / "*.tar.xz")); assert xz, list(d.iterdir()) +with tarfile.open(xz[0], "r:xz") as t: one_top(t.getnames()) +plain = glob.glob(str(d / "*.tar")); assert plain +with tarfile.open(plain[0], "r:") as t: one_top(t.getnames()) +zst = glob.glob(str(d / "*.tar.zst")); assert zst +assert open(zst[0], "rb").read(4) == b"\x28\xb5\x2f\xfd", "not a zstd frame" +z = glob.glob(str(d / "*.zip")); assert z +with zipfile.ZipFile(z[0]) as f: + one_top(f.namelist()) + assert any(i.compress_type == zipfile.ZIP_DEFLATED for i in f.infolist()), "nothing deflated" +PY +echo "ok: tar compressions and zip" + +# B +python3 - <<'PY' || fail "B: mcpp-package.json" +import glob, hashlib, json, os +root = sorted(p for p in glob.glob("target/dist/*") if os.path.isdir(p))[0] +d = json.load(open(os.path.join(root, "mcpp-package.json"))) +assert d["schema"] == 1 and d["name"] == "axes" and d["version"] == "0.1.0", d +assert d["mode"] == "vendored" and d["platform"] is None and len(d["legs"]) == 1, d +listed = {f["path"]: f for f in d["files"]} +on_disk = set() +for dirpath, _, files in os.walk(root): + for n in files: + rel = os.path.relpath(os.path.join(dirpath, n), root).replace(os.sep, "/") + if rel != "mcpp-package.json" and not os.path.islink(os.path.join(dirpath, n)): + on_disk.add(rel) +assert on_disk == set(listed), (on_disk, set(listed)) +for rel, f in listed.items(): + assert f["sha256"] == hashlib.sha256(open(os.path.join(root, rel), "rb").read()).hexdigest(), rel +assert any(f["role"] == "program" for f in d["files"]), d["files"] +PY +echo "ok: mcpp-package.json describes every file" + +# C +"$MCPP" pack --compress zstd --message-format json > env.json 2> env.err || fail "C: json" env.err +python3 - env.json <<'PY' || fail "C: the envelope" env.json +import hashlib, json, sys +d = json.load(open(sys.argv[1]))["data"] +assert d["pkgFormat"] == "tar" and d["mode"] == "vendored" and d["linkage"], d +assert len(d["legs"]) == 1, d +a = d["artifacts"][0] +assert a["sha256"] == hashlib.sha256(open(a["path"], "rb").read()).hexdigest(), a +PY +echo "ok: the envelope states the axes and the digest" + +# D +cat >> mcpp.toml <<'TOML' + +[pack] +format = "tar" +compress = "xz:9" +TOML +rm -rf target/dist +"$MCPP" pack > d.log 2>&1 || fail "D: pack with [pack] defaults" d.log +ls target/dist/*.tar.xz > /dev/null 2>&1 || fail "D: [pack] compress was not the default" d.log +echo "ok: [pack] format and compress are the project's defaults" + +# E +refuse() { # label, expected text, args... + local label=$1 want=$2; shift 2 + if "$MCPP" pack "$@" > refuse.log 2>&1; then fail "E: $label was accepted" refuse.log; fi + grep -q -- "$want" refuse.log || fail "E: $label: the refusal does not say '$want'" refuse.log + grep -q "Compiling" refuse.log && fail "E: $label was refused after compiling" refuse.log + return 0 +} +refuse "single-file with --mode system" "single-file" --pkg-format single-file --mode system +refuse "--compress on a directory" "writes none" --pkg-format dir --compress zstd +refuse "zstd in a zip" "deflated" --pkg-format zip --compress zstd +refuse "--systems without loaders" "systems" --systems linux +echo "ok: impossible combinations are refused before a build" + +# F +"$MCPP" pack --format dir > f.log 2>&1 || fail "F: --format dir" f.log +grep -q "earlier spelling of --pkg-format" f.log || fail "F: no warning for the earlier spelling" f.log +echo "ok: --format is the earlier spelling, with a warning" + +echo "PASS: 954 pack has three axes" diff --git a/tests/e2e/955_a_universal_package_holds_every_architecture.sh b/tests/e2e/955_a_universal_package_holds_every_architecture.sh new file mode 100755 index 000000000..173199769 --- /dev/null +++ b/tests/e2e/955_a_universal_package_holds_every_architecture.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +# requires: llvm python3 unix-shell +# 955 -- a universal package (SPEC-014 §3, §4.2): the platform of the ok-image +# targets states `universal = true`, so `mcpp pack` of either target packs both +# into one package, named after the platform, with a launcher that picks this +# machine's architecture and starts the platform's runner. +# +# Criteria: +# A. `--openkal/` holds `lib/x86_64/` and `lib/aarch64/` (each +# the image of its architecture), `bin/` and `bin/.cmd`, and +# `mcpp-package.json` naming both legs, the runner and each entry. +# B. `bin/` runs the image of this machine's architecture through the +# runner found on PATH, with its arguments. +# C. `--mode self-contained` adds the runner's program for this system and +# each architecture under `libexec//-/`, and the launcher +# runs with no runner on PATH. +# D. `--no-universal` packs the selected target alone. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64) HOST_ARCH=x86_64; OTHER=aarch64 ;; + aarch64|arm64) HOST_ARCH=aarch64; OTHER=x86_64 ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac +case "$(uname -s)" in Linux) OS=linux ;; Darwin) OS=macos ;; *) echo "SKIP: a POSIX launcher"; exit 0 ;; esac +T="$HOST_ARCH-openkal" + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "uni"\nversion = "0.1.0"\n' > mcpp.toml +cat > src/main.cpp <<'CPP' +import std; +int main(int argc, char** argv) { + try { throw std::runtime_error("universal"); } + catch (const std::exception& e) { std::println("{} {} args", e.what(), argc); } + return 0; +} +CPP + +# A +"$MCPP" pack --target "$T" --pkg-format dir > a.log 2>&1 || fail "A: pack" a.log +P=target/dist/uni-0.1.0-openkal +for f in "lib/x86_64/uni.oke" "lib/aarch64/uni.oke" "bin/uni" "bin/uni.cmd" "mcpp-package.json"; do + [ -f "$P/$f" ] || fail "A: the package has no $f" a.log +done +python3 - "$P" <<'PY' || fail "A: the package's metadata" "$P/mcpp-package.json" +import json, sys +root = sys.argv[1] +d = json.load(open(root + "/mcpp-package.json")) +assert sorted(d["legs"]) == ["aarch64-openkal", "x86_64-openkal"], d["legs"] +assert d["launch"]["runner"][0].startswith("okrun"), d["launch"] +assert d["launch"]["entry"] == {"x86_64": "lib/x86_64/uni.oke", "aarch64": "lib/aarch64/uni.oke"}, d["launch"] +# Each image is of its architecture (e_machine 62 x86-64, 183 AArch64). +import struct +for arch, machine in (("x86_64", 62), ("aarch64", 183)): + head = open(f"{root}/lib/{arch}/uni.oke", "rb").read(20) + assert head[:4] == b"\x7fELF" and struct.unpack_from("/dev/null \ + | python3 -c 'import json,sys; print(json.load(sys.stdin)["data"]["runner"]["program"])') +[ -x "$okrun" ] || fail "B: the platform's runner was not located ($okrun)" +PATH="$(dirname "$okrun"):$PATH" "$P/bin/uni" one two > b.log 2>&1 || fail "B: the launcher" b.log +grep -q "universal 3 args" b.log || fail "B: the image's output through the launcher" b.log +echo "ok: the launcher runs this machine's image through the runner" + +# C +"$MCPP" pack --target "$T" --mode self-contained --pkg-format dir -o sc > c.log 2>&1 || fail "C: pack --mode self-contained" c.log +for a in x86_64 aarch64; do + [ -f "target/dist/sc/libexec/okrun/$OS-$a/okrun" ] || fail "C: no runner for $OS-$a" c.log +done +env -i HOME="$HOME" PATH=/usr/bin:/bin target/dist/sc/bin/uni x > c-run.log 2>&1 || fail "C: the self-contained launcher" c-run.log +grep -q "universal 2 args" c-run.log || fail "C: the self-contained package's output" c-run.log +python3 - < d.log 2>&1 || fail "D: --no-universal" d.log +[ -f target/dist/one/bin/uni.oke ] || fail "D: --no-universal did not pack the target alone" d.log +[ -d target/dist/one/lib/$OTHER ] && fail "D: --no-universal packed the other architecture" d.log +echo "ok: --no-universal packs the selected target alone" + +echo "PASS: 955 a universal package holds every architecture" diff --git a/tests/e2e/956_a_single_file_runs_without_unpacking.sh b/tests/e2e/956_a_single_file_runs_without_unpacking.sh new file mode 100755 index 000000000..36690aee8 --- /dev/null +++ b/tests/e2e/956_a_single_file_runs_without_unpacking.sh @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 956 -- a single file (SPEC-014 §5.2): `--pkg-format single-file` writes one +# executable. A native program's file is the engine's launcher carrying the +# staged tree as a zstd frame, extracted once into the user's cache under the +# payload's sha256; an image's file is the platform's carrier, which runs the +# image from memory. +# +# Criteria: +# A. The native file runs the program with its arguments and exit status; +# the first run extracts the tree once, a second run extracts nothing. +# B. On Linux, `--linkage static` makes a file with no interpreter. +# C. Refused before a build: `--compress xz` for a single file. +# D. (llvm, an ok-image row) An image's single file runs, and writes nothing +# into the cache. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +export XDG_CACHE_HOME="$TMP/cache" + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "one"\nversion = "0.1.0"\n' > mcpp.toml +cat > src/main.cpp <<'CPP' +#include +int main(int argc, char** argv) { std::printf("one file: %d args, last %s\n", argc, argv[argc - 1]); return 3; } +CPP + +# A +"$MCPP" pack --pkg-format single-file > a.log 2>&1 || fail "A: pack" a.log +f=$(python3 - <<'PY' +import glob, os +c = [p for p in glob.glob("target/dist/one-0.1.0-*") if os.path.isfile(p) and os.access(p, os.X_OK)] +print(c[0] if c else "") +PY +) +[ -n "$f" ] || fail "A: no single file under target/dist" a.log +set +e; "$f" alpha beta > run1.log 2>&1; rc=$?; set -e +[ "$rc" = 3 ] || fail "A: the exit status is $rc, not the program's 3" run1.log +grep -q "one file: 3 args, last beta" run1.log || fail "A: the program's output" run1.log +dirs=$(ls "$XDG_CACHE_HOME/mcpp/single-file" | grep -v '^\.' | wc -l) +[ "$dirs" = 1 ] || fail "A: the first run left $dirs trees in the cache" +stamp=$(ls --full-time "$XDG_CACHE_HOME/mcpp/single-file"/*/.mcpp-complete 2>/dev/null || stat "$XDG_CACHE_HOME"/mcpp/single-file/*/.mcpp-complete) +set +e; "$f" gamma > run2.log 2>&1; set -e +grep -q "last gamma" run2.log || fail "A: the second run" run2.log +stamp2=$(ls --full-time "$XDG_CACHE_HOME/mcpp/single-file"/*/.mcpp-complete 2>/dev/null || stat "$XDG_CACHE_HOME"/mcpp/single-file/*/.mcpp-complete) +[ "$stamp" = "$stamp2" ] || fail "A: the second run extracted the tree again" +echo "ok: a native single file runs, extracting once" + +# B +if [ "$(uname -s)" = Linux ]; then + "$MCPP" pack --pkg-format single-file --linkage static -o one-static > b.log 2>&1 || fail "B: --linkage static" b.log + python3 - target/dist/one-static <<'PY' || fail "B: the static file has an interpreter" b.log +import struct, sys +data = open(sys.argv[1], "rb").read() +assert data[:4] == b"\x7fELF" +phoff, = struct.unpack_from(" b-run.log 2>&1; set -e + grep -q "last z" b-run.log || fail "B: the static single file" b-run.log + echo "ok: --linkage static makes a file with no interpreter" +fi + +# C +if "$MCPP" pack --pkg-format single-file --compress xz > c.log 2>&1; then fail "C: xz was accepted" c.log; fi +grep -q "zstd" c.log || fail "C: the refusal does not name the codec" c.log +echo "ok: a single file's payload is zstd" + +# D +case "$(uname -m)" in + x86_64|amd64) T=x86_64-openkal ;; + aarch64|arm64) T=aarch64-openkal ;; + *) T= ;; +esac +if [ -n "$T" ] && command -v python3 > /dev/null && "$MCPP" toolchain list --format json 2>/dev/null | grep -q llvm; then + mkdir -p "$TMP/img/src" + cd "$TMP/img" + printf '[package]\nname = "img"\nversion = "0.1.0"\n' > mcpp.toml + printf 'import std;\nint main(int argc, char**) { std::println("image file: {} args", argc); }\n' > src/main.cpp + "$MCPP" pack --target "$T" --pkg-format single-file > d.log 2>&1 || fail "D: pack" d.log + g=$(ls target/dist/img-0.1.0-* | head -1) + before=$(find "$XDG_CACHE_HOME" -type f | wc -l) + "$g" x y > d-run.log 2>&1 || fail "D: the image's single file" d-run.log + grep -q "image file: 3 args" d-run.log || fail "D: the output" d-run.log + after=$(find "$XDG_CACHE_HOME" -type f | wc -l) + [ "$before" = "$after" ] || fail "D: the image's single file wrote into the cache" + echo "ok: an image's single file runs from memory" +fi + +echo "PASS: 956 a single file runs without unpacking" diff --git a/tests/e2e/957_packages_merge_and_describe_themselves.sh b/tests/e2e/957_packages_merge_and_describe_themselves.sh new file mode 100755 index 000000000..db6e52e1a --- /dev/null +++ b/tests/e2e/957_packages_merge_and_describe_themselves.sh @@ -0,0 +1,119 @@ +#!/usr/bin/env bash +# requires: llvm python3 unix-shell +# 957 -- packages written on several machines merge into one (SPEC-014 §4.6), +# and a package describes its own distribution entry (§8). +# +# A self-contained universal package carries the runner of the systems it was +# packed for. One packed on Linux and one packed on macOS hold the same images +# and different runners; `--merge` makes the package for both. Here the second +# is the first with its runner moved to another system's directory, which is +# what a package packed there holds. +# +# Criteria: +# A. `--merge a --merge b` writes one package whose legs and systems are the +# union, every file of both, and a `mcpp-package.json` that describes it. +# B. A file both packages hold with different digests is refused, naming the +# file and both digests; packages of different versions are refused. +# C. `mcpp emit xpkg --package` of a universal package is one asset for +# linux, macosx and windows, with the runner as a dependency when the +# package does not carry it. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64) T=x86_64-openkal ;; + aarch64|arm64) T=aarch64-openkal ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "mrg"\nversion = "0.1.0"\n' > mcpp.toml +printf 'import std;\nint main() { std::println("merged"); }\n' > src/main.cpp +"$MCPP" pack --target "$T" --mode self-contained --pkg-format dir -o here > pack.log 2>&1 || fail "pack" pack.log + +# The same package as another system packs it: its runner in that system's +# directory, and the metadata saying so. +python3 - target/dist/here "$TMP/there" <<'PY' || fail "could not shape the second package" +import json, os, shutil, sys +src, dst = sys.argv[1], sys.argv[2] +shutil.copytree(src, dst, symlinks=True) +d = json.load(open(os.path.join(dst, "mcpp-package.json"))) +here = os.path.join(dst, "libexec", "okrun") +moved = {} +for sysdir in os.listdir(here): + other = sysdir.replace("linux-", "other-").replace("macos-", "other-") + os.rename(os.path.join(here, sysdir), os.path.join(here, other)) + moved[sysdir] = other +for f in d["files"]: + for a, b in moved.items(): + f["path"] = f["path"].replace(f"libexec/okrun/{a}/", f"libexec/okrun/{b}/") +d["systems"] = sorted(moved.values()) +json.dump(d, open(os.path.join(dst, "mcpp-package.json"), "w"), indent=2) +PY + +# A +"$MCPP" pack --merge target/dist/here --merge "$TMP/there" -o "$TMP/merged.zip" > a.log 2>&1 || fail "A: --merge" a.log +python3 - "$TMP/merged.zip" <<'PY' || fail "A: the merged package" a.log +import hashlib, json, sys, zipfile +z = zipfile.ZipFile(sys.argv[1]) +names = z.namelist() +top = names[0].split("/")[0] +d = json.loads(z.read(f"{top}/mcpp-package.json")) +assert len(d["systems"]) == 4 and any(s.startswith("other-") for s in d["systems"]), d["systems"] +assert sorted(d["legs"]) == ["aarch64-openkal", "x86_64-openkal"], d["legs"] +for f in d["files"]: + data = z.read(f"{top}/{f['path']}") + assert hashlib.sha256(data).hexdigest() == f["sha256"], f["path"] +PY +echo "ok: --merge writes the union of the packages" + +# B +python3 - "$TMP/there" <<'PY' +import hashlib, json, os, sys +root = sys.argv[1] +img = os.path.join(root, "lib", "x86_64", "mrg.oke") +open(img, "ab").write(b"\0") +d = json.load(open(os.path.join(root, "mcpp-package.json"))) +for f in d["files"]: + if f["path"] == "lib/x86_64/mrg.oke": + f["sha256"] = hashlib.sha256(open(img, "rb").read()).hexdigest() +json.dump(d, open(os.path.join(root, "mcpp-package.json"), "w"), indent=2) +PY +if "$MCPP" pack --merge target/dist/here --merge "$TMP/there" -o "$TMP/bad.zip" > b.log 2>&1; then + fail "B: two digests of one image were merged" b.log +fi +grep -q "lib/x86_64/mrg.oke" b.log || fail "B: the refusal does not name the file" b.log +python3 - "$TMP/there" <<'PY' +import json, os, sys +p = os.path.join(sys.argv[1], "mcpp-package.json") +d = json.load(open(p)); d["version"] = "0.2.0"; json.dump(d, open(p, "w")) +PY +if "$MCPP" pack --merge target/dist/here --merge "$TMP/there" -o "$TMP/bad2.zip" > b2.log 2>&1; then + fail "B: two versions were merged" b2.log +fi +grep -q "version" b2.log || fail "B: the refusal does not name the version" b2.log +echo "ok: --merge refuses what is not one package" + +# C +"$MCPP" pack --target "$T" --pkg-format zip -o mrg-0.1.0-openkal.zip > c.log 2>&1 || fail "C: pack" c.log +"$MCPP" emit xpkg --package target/dist/mrg-0.1.0-openkal.zip --url https://example.org/v0.1.0 > xpkg.lua 2> c.err \ + || fail "C: emit xpkg" c.err +python3 - xpkg.lua <<'PY' || fail "C: the entry" xpkg.lua +import hashlib, re, sys +text = open(sys.argv[1]).read() +sha = hashlib.sha256(open("target/dist/mrg-0.1.0-openkal.zip", "rb").read()).hexdigest() +for system in ("linux", "macosx", "windows"): + block = re.search(system + r" = \{(.*?)\n \},", text, re.S) + assert block, system + b = block.group(1) + assert "https://example.org/v0.1.0/mrg-0.1.0-openkal.zip" in b and sha in b, (system, b) + assert 'deps = { "okrun" }' in b, (system, b) +assert 'xvm.add("mrg"' in text +PY +echo "ok: a universal package is one asset for every system" + +echo "PASS: 957 packages merge and describe themselves" From ac14201e7c57d56c1514c6bc067db609daac3c04 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:17:59 +0900 Subject: [PATCH 20/54] =?UTF-8?q?e2e=20869:=20an=20artifact=20record=20car?= =?UTF-8?q?ries=20its=20sha256=20(SPEC-014=20=C2=A79)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...ck_over_several_members_names_each_member_in_its_report.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/e2e/869_a_pack_over_several_members_names_each_member_in_its_report.sh b/tests/e2e/869_a_pack_over_several_members_names_each_member_in_its_report.sh index 8b7df095a..41e8fe0ff 100644 --- a/tests/e2e/869_a_pack_over_several_members_names_each_member_in_its_report.sh +++ b/tests/e2e/869_a_pack_over_several_members_names_each_member_in_its_report.sh @@ -144,7 +144,7 @@ for s in d["stages"]: assert set(s) == {"member", "dir", "manifest", "closure"}, s assert s["closure"] in ("walked", "not-walked"), s for a in d["artifacts"]: - assert set(a) == {"path", "type", "format", "targets", "member"}, a + assert set(a) == {"path", "type", "format", "targets", "member", "sha256"}, a EOF "$MCPP" pack --mode system -p cli --message-format json > o2.json 2> o2.log || fail "O: a pack of one member failed" o2.log python3 - <<'EOF' || fail "O: the envelope of one member changed" o2.json @@ -152,7 +152,7 @@ import json d = json.load(open("o2.json"))["data"] assert "stages" not in d, d assert isinstance(d["stage"], dict) and set(d["stage"]) == {"dir", "manifest", "closure"}, d -assert len(d["artifacts"]) == 1 and set(d["artifacts"][0]) == {"path", "type", "format", "targets"}, d +assert len(d["artifacts"]) == 1 and set(d["artifacts"][0]) == {"path", "type", "format", "targets", "sha256"}, d EOF echo "ok: O, the envelope lists every member's artifacts and keeps the single-member shape" From 579be5d87f218d8124911281892690a65d5ef5d4 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:23:55 +0900 Subject: [PATCH 21/54] docs: SPEC-014 v1.1 states the packaging as implemented (zip deflated, [pack] universal, compression refusals, the single file's payload, check and exec, the image's single file, mcpp-package.json roles, --merge, signing, emit xpkg --package; criteria e2e 954-957); SPEC-013 v1.1: universal, the carrier's three carried forms and rows, the platform phase provides pack formats, self env --host; SPEC-007 R9.10 Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/specs/build-plugins.md | 5 +- docs/specs/packaging.md | 202 +++++++++++++++++++++--------------- docs/specs/platforms.md | 44 +++++--- 3 files changed, 156 insertions(+), 95 deletions(-) diff --git a/docs/specs/build-plugins.md b/docs/specs/build-plugins.md index 42ebabdb2..d71cb18ab 100644 --- a/docs/specs/build-plugins.md +++ b/docs/specs/build-plugins.md @@ -292,7 +292,8 @@ - **R9.10** 平台包的 `[platform] plugin` 在平台阶段运行(`mcpp::phase()` 为 `"platform"`,`MCPP_PHASE=platform`)。 该阶段只能陈述编译与链接标志、`link_script`、`link_start`、协议 17 的指令、`runner` 与命名运行器、 - `run_exclusive`,以及重跑条件、警告与诊断;其他指令**必须**被拒绝(拒绝码 `platform-phase-directive`), + `run_exclusive`、平台提供的包格式(`pack-format` 与 `role = "artifact"` 的 action,SPEC-014 §6),以及重跑 + 条件、警告与诊断;其他指令(含其他角色的 action)**必须**被拒绝(拒绝码 `platform-phase-directive`), 消息点名第一条不属于该阶段的指令。陈述的内容在 SPEC-013 的 Program 层次合并;协议 17 的指令也可以由 任何构建程序在构建阶段发出。(**已实现**,mcpp 2026.10.27.1) @@ -337,4 +338,4 @@ | 0.2 | 2026-09-26 | 随 mcpp 2026.9.26.2 落地:R1.3 的警告、R2.1 的 `runtime_search_dir`、R2.4、R3.3 的 `prepare`(目录须含文件;链接边等待所有 `prepare`)、R3.5、R3.6、R4.1、R4.3、R5.2、R5.3 标为已实现。 | | 0.9 | 2026-10-11 | 随 mcpp 2026.10.11.1(mcpp#797):新增 §10 构建程序的库与构建插件(R10.1–R10.6,P1–P5);变更记录移为 §11。 | | 0.10 | 2026-10-27 | 随 mcpp 2026.10.26.2(mcpp#845):R9.2 的协议表增加协议 16(`link_start`,镜像入口)。 | -| 0.11 | 2026-10-28 | 随 mcpp 2026.10.27.1(mcpp#847):R9.2 的协议表增加协议 17(平台的链接与运行器字段);新增 R9.10 平台阶段(SPEC-013)。 | +| 0.11 | 2026-10-28 | 随 mcpp 2026.10.27.1(mcpp#847):R9.2 的协议表增加协议 17(平台的链接与运行器字段);新增 R9.10 平台阶段(SPEC-013),平台阶段可声明包格式与 artifact action(SPEC-014 §6)。 | diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 56171f33d..3914445e6 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -4,10 +4,10 @@ |---|---| | 规范编号 | SPEC-014 | | 标题 | 打包与分发:链接、闭包、容器三个维度与构建腿,暂存树与包元数据,通用包、单文件、签名、压缩、合并与分发描述 | -| 状态 | 评审中 v1.0 | -| 版本 | 1.0 | +| 状态 | 评审中 v1.1 | +| 版本 | 1.1 | | 最后修改 | 2026-10-28 | -| 对应实现 | mcpp >= 2026.10.27.1;逐条标注 | +| 对应实现 | mcpp >= 2026.10.27.1 | | 相关设计文档 | `.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`(§2–§7、§14)、`.agents/docs/2026-10-28-platforms-design.md`(§4.7) | | 相关 issue | mcpp#847、mcpp#849 | | 相关规范 | [SPEC-007](build-plugins.md)(包格式提供者、artifact action)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-013](platforms.md)(平台、载体) | @@ -33,9 +33,10 @@ ### 1.1 链接 -`--linkage` 对应 `[build] dependency_linkage`,打包按它构建。原生 Linux 目标上 `static` 选择 `-linux-musl` -行,产物为全静态程序;openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` 加其所需的 `.okso`。 -未给出时取 `[pack] linkage`,再取构建的默认值。**未实现** +`--linkage`(未给出时取 `[pack] linkage`)是这次打包的构建的依赖链接,优先于 `[build] dependency_linkage` +与构建配置中的值;都未给出时每个库按它自己的声明链接,包元数据记为 `declared`。原生 Linux 目标上 `static` +选择 `-linux-musl` 行(`-linux-gnu` 目标换成对应的 musl 行),全静态程序不论 `--mode` 为何都整体打包; +openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` 加其所需的 `.okso`。 ### 1.2 闭包 @@ -45,10 +46,10 @@ | `vendored`(缺省) | 程序与非系统共享库 | 映像与其 `.okso` | C 库、C++ 运行时与加载器;openkal 上为加载器 | | `self-contained` | 程序、全部共享库、C 与 C++ 运行时、加载器 | 映像、`.okso` 与本机程序形式的加载器(载体,SPEC-013 §11) | 无 | -`bundle-project` 与 `bundle-all` 是 `vendored` 与 `self-contained` 的永久别名。**已实现** +`bundle-project` 与 `bundle-all` 是 `vendored` 与 `self-contained` 的永久别名。 `--mode static` 是 `--linkage static --mode self-contained` 的简写:原生目标上为全静态程序;openkal 目标上为一个静态 -`.oke` 加载体。**部分实现**:原生目标已实现;openkal 目标上 `--mode static` 当前被拒绝。 +`.oke` 加载体。 ### 1.3 容器 @@ -56,18 +57,19 @@ |---|---| | `dir` | 暂存树本身 | | `tar` | 一个归档,单一顶层目录与归档同名;缺省压缩为 gzip(§5.1) | -| `zip` | 一个 ZIP 归档,条目以 stored 存放;PE 目标的 `tar` 产出 `zip` | +| `zip` | 一个 ZIP 归档,单一顶层目录;条目以 DEFLATE 压缩,`--compress none` 时存储(§5.1) | | `single-file` | 一个可执行文件(§5.2) | | 其他名字 | 依赖图中的提供者以 `pack-format` 声明的格式(§6) | -`dir`、`tar` 与提供者格式 **已实现**;`zip` 作为显式取值与 `single-file` **未实现**。 +未给出 `--pkg-format` 与 `[pack] format` 时,Windows(PE)程序的容器为 `zip`,其余为 `tar`;显式给出的 `tar` +在任何目标上都写 tar。 ### 1.4 构建腿 `--target` 可以重复。每条腿是一次构建;各腿暂存进同一棵树(§4)。平台以 `[platform] universal = true` 声明其目标 -默认打**通用包**:对该平台任一目标的打包缺省覆盖平台的全部 `targets`;`--no-universal` 只打所选目标。一个产物为 -可执行文件的目标,只有在通用包中或在产物为共享对象的行(Android 的 `kind = "app"`)上接受多条腿。 -Android 的多腿 **已实现**;`universal` 与 `--no-universal` **未实现**。 +默认打**通用包**:对该平台任一目标的打包覆盖平台的全部 `targets`,其余的腿在打包过程中构建;`--no-universal` 或 +`[pack] universal = false` 只打所选目标;单文件不是通用包(§5.2)。一个产物为可执行文件的目标,只有在通用包中或在 +产物为共享对象的行(Android 的 `kind = "app"`)上接受多条腿。 ### 1.5 组合的合法性 @@ -76,15 +78,19 @@ Android 的多腿 **已实现**;`universal` 与 `--no-universal` **未实现** | 组合 | 结果 | |---|---| | `--pkg-format single-file` 与 `--mode system` | 拒绝:单文件携带它所运行的东西;出路 `--mode vendored` 或 `self-contained` | +| `--compress` 与 `--pkg-format dir` 或提供者的格式 | 拒绝:只有容器被压缩;出路 `--pkg-format tar`、`zip` 或 `single-file` | +| `--compress zstd` 或 `xz` 与 `--pkg-format zip` | 拒绝:zip 的条目为 DEFLATE 或存储;出路 `--compress gzip`、`none`,或 `--pkg-format tar` | +| `--compress gzip` 或 `xz` 与 `--pkg-format single-file` | 拒绝:启动器解压 zstd;出路 `--compress zstd[:]` 或 `none` | | 需要宿主提供能力的程序(docs/04 §2.11)与 `--mode self-contained` 或 `static` | 拒绝:携带自己 C 库的包不能使用宿主提供的库;出路 `--mode vendored` | | `cxx_runtime = "toolchain-coupled"` 与 `--mode system` | 拒绝 | | 产物为可执行文件、非通用包,且多条 `--target` | 拒绝:一个可执行文件属于一个目标 | -| `--systems` 而非 `--mode self-contained` 的通用包 | 拒绝:只有自带加载器的包按系统区分 | -| `--merge` 的输入缺少 `mcpp-package.json`,或名字、版本、平台不同 | 拒绝,写出不一致的字段 | -| `--merge` 的两个输入含同名映像而摘要不同 | 拒绝,写出该映像与两个摘要 | +| `--systems` 而非 `--mode self-contained` 的通用包或映像的单文件 | 拒绝:只有携带加载器的包与映像的单文件按系统区分 | +| 平台未声明 `runner` 时打通用包 | 拒绝:启动器经平台的运行器启动映像;出路 `--no-universal` | +| `--merge` 的输入缺少 `mcpp-package.json`,或名字、版本、平台、链接、模式不同 | 拒绝,写出不一致的字段 | +| `--merge` 的两个输入在同一路径上有摘要不同的文件 | 拒绝,写出该路径、两个摘要与两个输入 | | 库目标(`lib`、`shared`)与提供者格式 | 拒绝:库包没有单一暂存树 | -前三行与库目标一行 **已实现**;其余 **未实现**。 +`[pack] compress` 是项目的缺省值:与所选容器不相容时在这次打包中被搁置,只有 `--compress` 选项被拒绝。 ## 2. 选项与配置 @@ -96,42 +102,43 @@ Android 的多腿 **已实现**;`universal` 与 `--no-universal` **未实现** | `--format <编码>` | 报告类命令:`self env`、`toolchain list`、`emit build-database` 等 | 输出编码 | | `--pkg-format <格式>` | `pack`、`run` | 包格式 | -`pack` 与 `run` 继续接受 `--format <格式>`,行为与 `--pkg-format` 相同,并以一条警告写出它是旧写法。**已实现** +`pack` 与 `run` 继续接受 `--format <格式>`,行为与 `--pkg-format` 相同,并以一条警告写出它是旧写法。 + +`pack` 的其余选项:`--linkage static|shared`、`--compress <算法>[:<级别>]`、`--no-universal`、 +`--systems [,...]`、`--merge <包>`(可重复,也可逗号分隔)。 ### 2.2 `[pack]` | 键 | 语义 | |---|---| | `default_mode` | 未给 `--mode` 时的闭包模式 | -| `linkage` | 未给 `--linkage` 时的链接 | +| `linkage` | 未给 `--linkage` 时的链接(`static`、`shared`) | | `format` | 未给 `--pkg-format` 时的容器 | | `compress` | 未给 `--compress` 时的压缩(§5.1) | | `sign` | 签名步骤的 argv,`{}` 为文件(§7) | +| `universal` | `false` 时等同 `--no-universal` | | `strip`、`debug_symbols`、`include`、`exclude`、`[pack.bundle-project]` | 既有键 | -`default_mode` 与既有键 **已实现**;`linkage`、`format`、`compress`、`sign` **未实现**。 - ## 3. 管线 一次打包依次经过: -1. **构建各腿**:按维度与腿构建,`release` 为缺省构建配置。 +1. **构建各腿**:按维度与腿构建,`release` 为缺省构建配置;通用包的其余腿在暂存之前构建。 2. **收集闭包**:按 `--mode` 决定每条腿携带的库;闭包记录在暂存清单(docs/50)中,未解析的名字使 `dir` 与 `tar` 拒绝。 3. **生成暂存树**:§4 的布局;通用包生成启动器;`self-contained` 加入加载器或载体。 -4. **变换**:strip、重定位、平台的封装(SPEC-013 §9.3)、签名(§7)。 +4. **变换**:strip、重定位、平台的封装(SPEC-013 §9.3)、签名(§7),然后写出 `mcpp-package.json`(§4.5)。 5. **写容器**:§5;提供者格式在第二遍准备中读取暂存树(§6)。 -6. **报告**:产物、每个产物的校验和、`mcpp.pack` 信封(§9)。 +6. **报告**:产物、每个文件产物的 sha256、`mcpp.pack` 信封(§9)。 -`--merge` 是"读入若干暂存树 → 合并 → 写容器",不经第 1、2 步。第 1、2、3、4(strip、重定位、封装)、5、6 步的 -单腿形式 **已实现**;签名、通用包与 `--merge` **未实现**。 +`--merge` 是"读入若干包 → 合并 → 描述 → 写容器",不经第 1、2、3 步(§4.6)。 ## 4. 暂存树与包元数据 ### 4.1 单腿布局 单腿包的布局由目标的二进制格式决定(ELF 的 `bin/` 与 `lib/`,PE 的扁平目录,Mach-O 的 `bin/`,Android 的 `lib/`, -WebAssembly 的同名文件族),规定见 docs/10。**已实现** +WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 ### 4.2 通用包布局 @@ -140,32 +147,35 @@ WebAssembly 的同名文件族),规定见 docs/10。**已实现** bin/ POSIX 启动器 bin/.cmd Windows 启动器 lib//.oke, *.okso 每个架构一组映像 - libexec//-/... 仅 --mode self-contained - share/... 资源,只放一份 + libexec//-/... 仅 --mode self-contained mcpp-package.json ``` -包名带平台名,不带系统与架构。**未实现** +包名带平台名(平台包名去掉 `-platform`),不带系统与架构。`[runtime] deploy` 放置的文件位于每条腿的映像旁边。 ### 4.3 启动器 启动器由引擎生成,内容只来自平台的 `runner` 与腿: -- 架构目录按 `uname -m`(POSIX)或 `%PROCESSOR_ARCHITECTURE%`(Windows)选择;映射为 `x86_64`、`amd64`、`AMD64` → - `x86_64`,`aarch64`、`arm64`、`ARM64` → `aarch64`;没有对应目录时以非零状态退出,写出本机架构与包含的架构。 -- 运行 `runner.cmd`,`{}` 替换为所选架构目录中的入口映像,其余参数原样传递;`self-contained` 时先在 - `libexec//-/` 中找加载器,再找 `PATH`。 - -**未实现** +- 包的根为启动器所在目录的上一级。 +- 架构目录按 `uname -m`(POSIX)或 `%PROCESSOR_ARCHITEW6432%`、`%PROCESSOR_ARCHITECTURE%`(Windows)选择;映射为 + `x86_64`、`amd64`、`AMD64` → `x86_64`,`aarch64`、`arm64`、`ARM64` → `aarch64`;没有对应目录时以状态 126 退出, + 写出本机架构与包含的架构。 +- 运行 `runner.cmd`,`{}` 替换为所选架构目录中的入口映像,其余参数原样传递,随后是启动器自己的参数;运行器先在 + `libexec//-/` 中找,再找 `PATH`。 ### 4.4 `--systems` -`--systems [,...]`(缺省为本机系统)为每个系统 × 平台架构构建加载器程序并放入 `libexec/`。缺少某个系统的 -SDK 载荷时拒绝,写出该系统与载荷。**未实现** +`--systems [,...]`(`linux`、`macos`、`windows`;缺省为本机系统)对 `self-contained` 的通用包,为每个系统 × +包的每个架构构建平台的载体(不携带任何内容,即运行器形式,SPEC-013 §11)并放入 +`libexec//-/`;对映像的单文件,为每个系统写一个文件(§5.2)。Linux 的程序为静态(musl);其他 +系统的按其惯常行(`-macos`、`-windows-gnu`)构建,与本机系统和架构相同时取本机的行;缺少某个系统的 +SDK 载荷时该次构建拒绝,写出该系统与载荷。 ### 4.5 `mcpp-package.json` -暂存树根的 `mcpp-package.json` 是包格式提供者、`--merge` 与 `mcpp emit xpkg` 的输入: +暂存树根的 `mcpp-package.json` 是包格式提供者、`--merge` 与 `mcpp emit xpkg` 的输入,每次打包都写出(提供者的 +格式也不例外): ```json { @@ -173,7 +183,7 @@ SDK 载荷时拒绝,写出该系统与载荷。**未实现** "name": "hello", "version": "0.1.0", "platform": "mcpplibs.openkal-platform@0.17.2", - "linkage": "shared", + "linkage": "declared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], @@ -184,48 +194,66 @@ SDK 载荷时拒绝,写出该系统与载荷。**未实现** | 字段 | 语义 | |---|---| -| `schema` | 本 schema 的版本,当前为 1 | +| `schema` | 本 schema 的版本,当前为 1;读取者拒绝其他版本 | | `name`、`version` | 包的名字与版本 | -| `platform` | 平台包与版本;无平台时为 `null` | -| `linkage`、`mode` | §1 的两个维度 | +| `platform` | 平台包与版本(`<包>@<版本>`);无平台时为 `null` | +| `linkage` | `static`、`shared`,或 `declared`(每个库按自己的声明) | +| `mode` | §1.2 的模式 | | `legs` | 腿的规范三元组 | -| `systems` | `libexec/` 中加载器所属的系统;无加载器时为空 | -| `launch` | 运行方式:`runner` 的 argv 与每个架构的入口;单腿原生包为程序的相对路径 | -| `files` | 每个文件:`path`(相对于树根,`/` 分隔)、`role`(`program`、`image`、`library`、`loader`、`launcher`、`resource`、`metadata`)、`leg`(所属的腿,共用文件为 `null`)、`sha256` | +| `systems` | `libexec//` 中加载器的 `-`;无加载器时为空 | +| `launch` | 通用包为 `runner`(平台的 argv)与每个架构的 `entry`;单腿原生包为 `program`(程序的相对路径) | +| `files` | 树中每个文件(符号链接除外):`path`(相对于树根,`/` 分隔)、`role`、`leg`(所属的腿,共用文件为 `null`)、`sha256` | -**未实现** +`role` 取 `program`、`image`、`library`、`loader`、`launcher`、`resource`、`metadata` 之一:通用包中 `lib//` +下的文件为该腿的 `image`,生成的启动器为 `launcher`;其余按位置与名字判定 —— `libexec/` 下为 `loader`; +`README*`、`LICENSE*`、`HOST-REQUIREMENTS`、`share/licenses/` 为 `metadata`;`.so*`、`.dylib`、`.dll`、`.okso` 为 +`library`;`.oke` 为 `image`;`bin/` 下的可执行文件为 `program`;`.cmd`、`.sh` 与树根的可执行文件为 `launcher`; +其他为 `resource`。 ### 4.6 合并 -`mcpp pack --merge ...` 读入若干包(目录或归档),按 `mcpp-package.json` 合并为一棵树:腿与系统取并集,同一路径 -的文件摘要**必须**相同,资源只放一份;再按 `--pkg-format` 写容器。**未实现** +`mcpp pack --merge --merge ...`(也可 `--merge ,`)读入至少两个本命令写出的包(目录,或 `.zip`、 +`.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz` 归档),按 `mcpp-package.json` 合并为一棵树:名字、版本、平台、 +链接与模式**必须**相同;腿与系统取并集;同一路径的文件摘要**必须**相同,每个文件只取一份;合并后的树重新描述。 +容器取 `--pkg-format`,否则取 `-o` 的扩展名,否则取第一个输入的容器;`-o` 缺省为当前目录下与树同名的文件。 +不进行任何构建。 ## 5. 容器 ### 5.1 压缩 -`--compress [:]` 与 `[pack] compress` 选择 `tar` 的压缩;`tar` 的缺省为 gzip,产物扩展名为 -`.tar.gz`、`.tar.zst`、`.tar.xz` 或 `.tar`。`zip` 的条目以 stored 存放。单文件的载荷使用 zstd。okb 中的映像不压缩 -(SPEC-013 平台插件的格式规定)。gzip **已实现**;`--compress` 与其余取值 **未实现**。 +`--compress [:]` 与 `[pack] compress` 选择 `tar` 的压缩:`tar` 写出未压缩的归档,mcpp 以 +自带的编解码器(zlib、zstd、liblzma)压缩,产物扩展名为 `.tar.gz`(缺省)、`.tar.zst`、`.tar.xz` 或 `.tar`。级别 +范围:gzip 1–9,zstd 1–22,xz 0–9。`zip` 的条目在变小时以 DEFLATE 压缩(`gzip[:]` 设定级别),否则存储; +`none` 时全部存储。单文件的载荷为 zstd 或不压缩。okb 中的映像不压缩(SPEC-013 平台插件的格式规定)。同一棵树的 +归档在每个宿主上字节相同:gzip 成员不带名字与时间,zip 条目使用固定日期。 ### 5.2 单文件 | 组合 | 产物 | 运行 | |---|---|---| -| 原生目标,`vendored` 或 `self-contained` | 引擎的启动桩加 zstd 压缩的暂存树 | 首次运行把载荷解到用户缓存,之后直接执行 | -| 原生 Linux 目标,`--linkage static` | 静态启动桩加静态程序 | 同上,不依赖任何系统库 | -| openkal 目标,`self-contained` | 平台的载体加本架构的映像组 | 载体经加载器库的 bundle 接口从内存加载,不写盘 | - -启动桩的规定: - -1. 源码随引擎提供,打包时为目标构建;它只做校验、解压与执行。 -2. 缓存根:`$XDG_CACHE_HOME`,未设置时 `~/.cache`(Linux);`~/Library/Caches`(macOS);`%LOCALAPPDATA%` - (Windows);其下按载荷的 sha256 寻址。 -3. 解压前**必须**校验载荷摘要;解到同一目录下的临时名,完成后原子改名;**禁止**复用不完整的目录;缓存目录只对 - 当前用户可写。 -4. 以原有的参数与环境执行暂存树中的程序,退出状态即程序的退出状态。 - -**未实现** +| 原生目标,`vendored` 或 `self-contained` | 引擎的启动器,携带暂存树的一个 zstd 帧 | 首次运行把树解到用户缓存,之后直接执行 | +| 原生 Linux 目标,`--linkage static` | 静态启动器加静态程序 | 同上,不依赖任何系统库 | +| openkal 目标 | 平台的载体,携带映像,或全部映像的一个 zip | 载体经加载器库从内存加载,不写盘 | + +原生目标的启动器: + +1. 源码随引擎提供,每次打包写入 `target/single-file//`(依赖 mcpplibs.os 与 zstd)并为目标构建:Linux 上为 + 静态(musl),macOS 上为 `vendored`,Windows 上为 `system`;它只做校验、解压与执行。树暂存在 + `target/single-file/stage/` 下;产物为 `target/dist/--`(Windows 上带 `.exe`)。 +2. 载荷是暂存树的记录流(`MCPPTREE1\n`,每项为类型字节 `d`/`f`/`x`/`l`、32 位路径长度、路径、64 位数据长度、数据; + 小端,按路径排序)的一个 zstd 帧,帧写明内容大小与内容校验和。 +3. 缓存根:`$XDG_CACHE_HOME`,未设置时 `~/.cache`(Linux);`~/Library/Caches`(macOS);`%LOCALAPPDATA%` + (Windows);其下为 `mcpp/single-file/<载荷的 sha256>/`,该摘要由 mcpp 打包时写入启动器。 +4. 解压时**必须**校验帧的内容校验和,不符时不解开;解到同一目录下的临时名(`.<摘要>-<进程号>`),全部文件写完后 + 写入 `.mcpp-complete` 并以不替换的方式改名到位;**禁止**使用没有 `.mcpp-complete` 的目录;另一次运行先改名到位时 + 使用它的结果;缓存目录在 POSIX 上只对当前用户可写。 +5. 以原有的参数执行树中的程序(`bin/`;self-contained 的 Linux 树为根部的包装脚本;Windows 树为根部的 + `.exe`),退出状态即程序的退出状态。 + +openkal 目标的单文件:载体携带映像本身(映像不引用其他映像时),否则携带全部映像的一个 stored zip,各映像数据按 +64 KiB 对齐,`mcpp_carried_name` 为入口映像(SPEC-013 §11);`--systems` 的每个系统(缺省为本机系统)各写一个文件 +`target/dist/---`(Windows 上带 `.exe`)。运行时不写盘。 ### 5.3 一个文件在多个系统上运行 @@ -235,21 +263,31 @@ SDK 载荷时拒绝,写出该系统与载荷。**未实现** 一个包以构建程序的 `mcpp::provides_pack_format("")` 声明一个格式,以 `role = "artifact"` 的 action 生成产物 (SPEC-007)。`--pkg-format ` 找到图中声明该名字的包,在第二遍准备中把暂存树交给它:暂存树目录 -(`mcpp::pack_stage_dir()`、`${mcpp.stage_dir}`)、`mcpp-package.json`、`MCPP_PACK_*` 环境;它报告的终端输出即产物。 -未知的名字被拒绝,消息列出图中可用的格式。**已实现**(`mcpp-package.json` 除外,**未实现**) +(`mcpp::pack_stage_dir()`、`${mcpp.stage_dir}`)、其中的 `mcpp-package.json`、`MCPP_PACK_*` 环境;它报告的终端 +输出即产物。未知的名字被拒绝,消息列出图中可用的格式。 平台插件(SPEC-013 §12)**可以**在平台阶段声明包格式与 artifact action;平台阶段的指令表因此包含 `pack-format` -与 artifact action。该平台的目标打包时,这些格式与引擎的格式并列。**未实现** +与 `role = "artifact"` 的 action(其他角色的 action 仍被拒绝)。该平台的目标打包时,这些格式与引擎的格式并列。 ## 7. 签名 -`[pack] sign = ["", ..., "{}"]` 对暂存树中的每个程序与映像运行一次,`{}` 为文件,在写容器之前;一个包格式 -提供者或构建插件**可以**代替它签名。任一次失败使打包失败,写出 argv 与输出。**未实现** +`[pack] sign = ["", ..., "{}"]` 在写容器之前对暂存树中角色为 `program`、`image`、`library`、`loader` 的每个 +文件运行一次,也对每个单文件运行一次,`{}` 为文件路径;一个包格式提供者或构建插件**可以**代替它签名。任一次失败 +使打包失败,写出 argv 与输出。签名后 `mcpp-package.json` 重新计算摘要。 ## 8. 分发描述 -`mcpp emit xpkg` 由 `mcpp-package.json` 生成索引条目:每个系统 × 架构一项;或通用包时一个资产对应所有系统, -URL 相同,并写出运行所需的依赖(加载器的包)。**未实现** +`mcpp emit xpkg --package <归档> [--package ...] --url <前缀> [-o <文件>]` 由本命令写出的包的 `mcpp-package.json` +生成 xim-pkgindex 条目(spec 2),资产地址为 `<前缀>/<文件名>`: + +- 通用包是一个资产,`linux`、`macosx`、`windows` 三项的 URL 与 sha256 相同;包不携带运行器(非 `self-contained`) + 时三项各以运行器为依赖(`deps = { "" }`)。 +- 各系统各架构一个的包:每个系统一项,一个以 `${arch}` 代替架构的 URL 与每个架构的 sha256;文件名除架构外不同时 + 拒绝。 +- 安装把归档的唯一顶层目录移到安装目录;配置以 xvm 注册程序(通用包为 `bin/`,Windows 上为 + `bin/.cmd`;单腿原生包为程序所在的目录)。 + +不带 `--package` 时 `mcpp emit xpkg` 照旧生成源码包的描述。 ## 9. 机器输出 @@ -259,22 +297,24 @@ URL 相同,并写出运行所需的依赖(加载器的包)。**未实现** |---|---| | `linkage`、`mode`、`pkgFormat` | 三个维度的取值 | | `legs` | 腿的规范三元组 | -| `systems` | 加载器所属的系统 | +| `systems` | 加载器所属的 `-` | | `artifacts[].sha256` | 每个文件产物的 sha256;目录产物为 `null` | -`artifacts[].format` 保留,与 `pkgFormat` 相同。**未实现** +`artifacts[].format` 保留,与 `pkgFormat` 相同。库目标的打包不带这些字段。 ## 10. 判据 -1. 每个不合法的组合在任何编译之前被拒绝,消息写出冲突的选择与出路。 -2. `--format` 在 `pack` 与 `run` 上与 `--pkg-format` 等价,并写出警告。 -3. 通用包在每个平台架构的机器上经启动器运行;`self-contained` 的通用包在未安装加载器的机器上运行。 -4. `--merge` 的结果与一次打出全部腿与系统的包逐文件相同。 -5. 单文件在载荷摘要不符、缓存目录不完整时不执行不可信的内容;openkal 单文件运行时缓存目录中无新文件。 -6. `mcpp-package.json` 的 `files` 覆盖树中每个文件,摘要与文件一致。 +1. 每个不合法的组合在任何编译之前被拒绝,消息写出冲突的选择与出路(e2e 954)。 +2. `--format` 在 `pack` 与 `run` 上与 `--pkg-format` 等价,并写出警告(e2e 954)。 +3. 通用包在本机架构上经启动器运行;`self-contained` 的通用包在 `PATH` 上没有运行器时运行(e2e 955)。 +4. `--merge` 的结果含两个输入的全部腿、系统与文件,摘要一致;冲突与不同版本被拒绝(e2e 957)。 +5. 单文件首次运行解压一次,再次运行不解压;静态单文件不带解释器;映像的单文件运行时缓存目录中无新文件(e2e 956)。 +6. `mcpp-package.json` 的 `files` 覆盖树中每个文件,摘要与文件一致(e2e 954、957)。 +7. 通用包的 `mcpp emit xpkg` 条目在三个系统上为同一资产,并以运行器为依赖(e2e 957)。 ## 变更记录 | 版本 | 日期 | 变更 | |---|---|---| | 1.0 | 2026-10-28 | 首版(mcpp#847):三个维度与构建腿、组合的合法性、选项的全局用法与 `--pkg-format`、`[pack]` 的新键、管线、单腿与通用包的布局、启动器、`--systems`、`mcpp-package.json`、`--merge`、压缩、单文件、包格式提供者与平台插件、签名、分发描述、`mcpp.pack` 的新字段。 | +| 1.1 | 2026-10-28 | 与实现对齐:zip 的条目以 DEFLATE 压缩;`[pack] universal`;压缩的拒绝与项目缺省值的搁置;`--systems` 也用于映像的单文件;启动器的根与退出状态;单文件的载荷格式、校验与执行;映像单文件的载体内容;`mcpp-package.json` 的 `linkage` 取值与角色判定;`--merge` 的输入、选项写法与容器;签名的范围;`mcpp emit xpkg --package`;判据对应 e2e 954–957。 | diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 517430e45..b44dc5a76 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -4,8 +4,8 @@ |---|---| | 规范编号 | SPEC-013 | | 标题 | 平台:一类目标的可复用配置,其声明、选择、合并、对依赖图的补全、链接与封装、运行器与载体 | -| 状态 | 评审中 v1.0 | -| 版本 | 1.0 | +| 状态 | 评审中 v1.1 | +| 版本 | 1.1 | | 最后修改 | 2026-10-28 | | 对应实现 | mcpp >= 2026.10.27.1;逐条标注 | | 相关设计文档 | `.agents/docs/2026-10-28-platforms-design.md`、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md` | @@ -68,6 +68,7 @@ plugin = "build.mcpp" | `runners` | ` = [argv]` | 命名的设备动作,每个 argv 非空 | | `carrier` | 表 | §11;`package`、`section`、`main` 三键都**必须**给出 | | `plugin` | 字符串 | 平台包中的构建程序,在平台阶段运行(§12);路径相对于平台包 | +| `universal` | 布尔 | `true` 时本平台任一目标的打包缺省为覆盖全部 `targets` 的通用包(SPEC-014 §1.4、§4.2) | | `layers` | 表 | §7.2;键只能是 §0 的四个层名 | | `tools` | 表 | §7.3 | | `build` | 表 | 四个标志列表,以及键形如 `cfg(...)` 的子表(同四个键) | @@ -301,21 +302,35 @@ plugin = "build.mcpp" ## 11. 载体(M3) -`mcpp pack --mode self-contained` 对一个依赖加载器的产物生成一个本机程序工程:依赖 `carrier.package`;入口是 -平台包中的 `carrier.main`;一个汇编文件以 `.incbin` 把产物的字节放进节 `carrier.section`(ELF 写作该名, -Mach-O 写作 `__<大写名>,__image`,COFF 写作该名并带 `"dr"` 标志),给出符号 `mcpp_carried_begin`、 -`mcpp_carried_end`、`mcpp_carried_name`(产物的名字)。该程序按宿主约定构建(自举边界),不使用平台:Linux 上 -为 `<本机架构>-linux-musl` 行的静态程序,该行的工具链被显式写出;macOS 与 Windows 上只依赖系统库。 -引擎不含任何加载器的调用。平台未声明 `carrier` 时,`--mode self-contained` 被拒绝并写出该键。**已实现** +对一个依赖加载器的产物,`mcpp pack` 在三处生成一个系统程序的工程(`target/carrier-pack/--/`): +`--mode self-contained` 的单目标包(该程序携带产物)、`self-contained` 通用包的运行器(不携带任何内容,SPEC-014 +§4.4)、映像的单文件(SPEC-014 §5.2)。工程依赖 `carrier.package`;入口是平台包中的 `carrier.main`;一个汇编文件 +以 `.incbin` 把携带的字节放进节 `carrier.section`(ELF 写作该名并带 `"a"` 标志,COFF 写作该名并带 `"dr"` 标志, +Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mcpp_carried_end`、`mcpp_carried_name` +(入口的名字)与 `mcpp_carried_digest`(字符串)。 + +携带的字节**必须**是以下之一,载体的入口按内容区分: + +| 携带 | 判别 | 载体的行为 | +|---|---|---| +| 无(`mcpp_carried_begin == mcpp_carried_end`) | 长度为 0 | 运行器形式:运行第一个参数所指的映像,其余参数传给它,如同平台的运行器程序 | +| 一个映像 | 产物本身 | 从内存运行该映像,映像以程序自己的名字为名 | +| 一组映像 | 以 `PK\x03\x04` 开头的 stored zip,各条目数据按 64 KiB 对齐 | 从内存运行名为 `mcpp_carried_name` 的映像,映像之间按名字相互解析 | + +程序按其系统的约定构建(自举边界),不使用平台:Linux 上为 `-linux-musl` 行的静态程序,该行的工具链被 +显式写出;macOS 上为 `-macos`、Windows 上为 `-windows-gnu` 行,系统与架构同本机时取本机的行,只依赖 +系统库(映像的单文件在 Windows 上以 `system` 模式打包)。引擎不含任何加载器的调用。平台未声明 `carrier` 时, +需要载体的打包被拒绝并写出该键。 ## 12. 平台阶段与协议 17 ### 12.1 平台阶段 平台包的 `plugin` 在**平台阶段**运行:`MCPP_PHASE` 为 `platform`,`mcpp::phase()` 返回 `"platform"`。该阶段只能 -陈述编译与链接标志、`link_script`、`link_start` 与协议 17 的指令、`runner` 与命名运行器、`run_exclusive`,以及 -重跑条件、警告与诊断;其他指令被拒绝(`platform-phase-directive`),消息点名第一条不属于该阶段的指令。 -陈述工具链在平台阶段被拒绝。**已实现** +陈述编译与链接标志、`link_script`、`link_start` 与协议 17 的指令、`runner` 与命名运行器、`run_exclusive`、平台 +提供的包格式(`pack-format` 与 `role = "artifact"` 的 action,SPEC-014 §6),以及重跑条件、警告与诊断;其他指令 +(含其他角色的 action)被拒绝(`platform-phase-directive`),消息点名第一条不属于该阶段的指令。陈述工具链在平台 +阶段被拒绝。**已实现** ### 12.2 协议 17 的指令 @@ -392,7 +407,7 @@ Mach-O 写作 `__<大写名>,__image`,COFF 写作该名并带 `"dr"` 标志),给 | `--platform <值>` | `build`、`run`、`test`、`why`、`pack`、`emit build-database`、`self env`、`toolchain install`;即 `MCPP_PLATFORM` | | `--host-platform <值>` | `build`、`run`、`test`、`why`、`pack`、`emit build-database`、`self env`;即 `MCPP_HOST_PLATFORM` | | `mcpp self env --target [--format json]` | 准备并报告一个配置:三元组、编译器与其来源、平台、工具、运行器与其程序、`run_under`;在工程中为该工程的配置,工程外为家目录缓存中的空程序;工程源码会被拒绝的原因(例:无 C++ 运行时而 `import std`)作为警告报告,配置照常报告 | -| `mcpp self env --host` | 宿主图的配置 | +| `mcpp self env --host` | 宿主图的配置;其构建程序已构建过时,报告宿主平台的运行器与其程序、工具 | | `mcpp toolchain install --target `(不指定编译器) | 安装该目标的工具链后准备该配置:获取平台包,构建其工具 | | `mcpp toolchain list` | 目标行有 PLATFORM 列,JSON 有 `platform` 字段 | | `mcpp test`(在平台包中,未给 `--target`) | 对 `targets` 中的每个目标以本包为平台构建测试;`host = "same-arch"` 时只运行本机架构的目标,其余报告为已构建未运行 | @@ -425,9 +440,14 @@ Mach-O 写作 `__<大写名>,__image`,COFF 写作该名并带 `"dr"` 标志),给 7. `same-arch` 的运行器不运行其他架构的产物,并给出 `run_under` 的写法。 8. 平台的工具在任何图中按宿主约定构建。 9. 锁中平台加入的包带 `origin = "platform"`;报告与 JSON 给出每个字段的来源。 +10. 一个与 openkal 无关的平台包的编译标志、命名、封装与运行器到达程序,`cfg(platform)` 下的覆盖取代对应字段, + `--platform none` 不使用平台,没有 `[platform]` 的包被拒绝为平台(e2e 952)。 +11. 宿主图选用平台时构建程序与其库作为映像构建并经平台的运行器运行,平台的工具按宿主约定构建, + `--host-platform none` 时构建程序为本机程序(e2e 953)。 ## 变更记录 | 版本 | 日期 | 变更 | |---|---|---| | 1.0 | 2026-10-28 | 首版(mcpp 2026.10.27.1,mcpp#847):`[platform]` 的七组键与 `versions`、`hosted`;三种写法与层次;目标图与宿主图的选择;`cfg(platform)`;合并;依赖图的补全、版本组与改名边界;工具链;链接、命名与封装;运行器的表形式;载体;平台阶段与协议 17;工作空间;目标表的平台列;锁、报告、机器输出、命令与拒绝码。 | +| 1.1 | 2026-10-28 | `[platform] universal`;载体的三种携带形式(运行器形式、映像、映像组的 zip)、`mcpp_carried_digest`、Mach-O 节名 `__carried`、按系统的行;平台阶段可声明包格式与 artifact action;`self env --host` 报告运行器与工具;判据 10、11(e2e 952、953)。 | From 22d16c9c3878a01dec2641fc3e1f867f57a0fa77 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:24:00 +0900 Subject: [PATCH 22/54] docs/10, 35, 50 (en, zh): the packaging as implemented -- the refusals, a zip's deflated entries, universal launchers and runners, --merge's inputs and container, the single file's launcher, cache and check, the image's single file, mcpp-package.json, emit xpkg --package, signing; the carrier's carried forms; the envelope's values; self env --host's runner and tools Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/10-pack-and-release.md | 167 +++++++++++++++++++++++---------- docs/35-platforms.md | 18 +++- docs/50-machine-output.md | 11 ++- docs/zh/10-pack-and-release.md | 107 ++++++++++++++------- docs/zh/35-platforms.md | 12 ++- docs/zh/50-machine-output.md | 10 +- 6 files changed, 230 insertions(+), 95 deletions(-) diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 7971f0726..f7945ce2b 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -63,13 +63,16 @@ A pack is decided by three independent choices and a set of legs (2026.10.27.1+) |---|---|---|---| | linkage | `--linkage` | are the libraries inside the program or beside it | `static`, `shared` | | closure | `--mode` | what the machine that runs the package provides | `system`, `vendored` (default), `self-contained` | -| container | `--pkg-format` | what is delivered | `tar` (default), `dir`, `zip`, `single-file`, a format a package provides | +| container | `--pkg-format` | what is delivered | `tar`, `zip`, `dir`, `single-file`, a format a package provides; unstated: `zip` for a Windows program, `tar` elsewhere | | legs | `--target`, repeatable | which targets one package covers | triples | -- **Linkage** is a property of the program, built as `[build] dependency_linkage` - states it. On Linux `--linkage static` selects the `-linux-musl` row and - produces a fully static program; `--target …-linux-musl` implies it. On an ok - image it is one `.oke` with no `.okso`. +- **Linkage** is a property of the program. `--linkage static|shared` (or + `[pack] linkage`) is the build's dependency linkage for the pack, over + `[build] dependency_linkage` and the profile's; unstated, each library links as + it declares itself. On Linux `--linkage static` selects the `-linux-musl` + row (a `-linux-gnu` target becomes its musl row) and packs the fully static + program whole, whatever `--mode` says; `--target …-linux-musl` implies it. On + an ok image it is one `.oke` with no `.okso`. - **Closure** is a property of the package: how much of the shared-library closure travels with the program. `--mode static` is shorthand for `--linkage static --mode self-contained`. @@ -107,10 +110,17 @@ choices: | Combination | Remedy | |---|---| | `--pkg-format single-file` with `--mode system` | `--mode vendored` or `self-contained`: a single file carries what it runs | -| several `--target` for an executable outside a universal package | one pack per target, then `--merge` | -| `--systems` on anything but a self-contained universal package | `--mode self-contained`: only a package that carries its loader differs per system | +| `--compress` with `--pkg-format dir` or a provided format | `--pkg-format tar`, `zip` or `single-file`: only a container is compressed | +| `--compress zstd` or `xz` with `--pkg-format zip` | `--compress gzip` (DEFLATE) or `none`, or `--pkg-format tar` | +| `--compress gzip` or `xz` with `--pkg-format single-file` | `--compress zstd[:]` or `none`: the launcher decompresses zstd | +| several `--target` for an executable outside a universal package | one pack per target | +| `--systems` on anything but a self-contained universal package or a single file of an image | `--mode self-contained` over a platform with `universal = true`, or `--pkg-format single-file` | +| a universal package of a platform that states no `runner` | `--no-universal` | | a dispatched format for a `lib` or `shared` target | the library route, [12](12-binary-distribution.md) | +A `[pack] compress` that does not fit the container (a directory, `zstd` for a +zip) is left aside for that pack; only the flag is refused. + ### A program that needs the HOST to provide something "Self-contained" has a floor. Some libraries can only come from the target @@ -191,7 +201,8 @@ mcpp pack --pkg-format single-file # one executable file mcpp pack --pkg-format tar --compress zstd # .tar.zst mcpp pack --target x86_64-openkal # a universal package: every architecture of the platform mcpp pack --target x86_64-openkal --mode self-contained --systems linux,macos,windows -mcpp pack --merge a.zip b.zip -o all.zip # one tree from packages built on several machines +mcpp pack --merge a.zip --merge b.zip -o all.zip # one package from packages built on several machines +mcpp emit xpkg --package all.zip --url https://example.org/v1.0.0 # its xim-pkgindex entry ``` `--format` is accepted on `pack` and `run` as the earlier spelling of @@ -659,9 +670,10 @@ for the same reason — there is no zip tool present on every host. Two consequences worth knowing: -- Entries are **stored, not deflated**, so a Windows package is roughly the - size of its contents. Compression is a size optimization, not a correctness - one, and it is not implemented yet. +- Entries are **deflated** by zlib when that makes them smaller, and stored + otherwise; `--compress gzip:` sets the level, `--compress none` stores + every entry. A format left unstated is a zip for a Windows program; + `--pkg-format tar` stated writes a tar. - The archive is **deterministic**: no timestamps are read, so two packs of the same tree are byte-identical and a published checksum means something. @@ -715,16 +727,16 @@ runs the package must provide: |---|---|---| | `system` | `bin/.oke` | `okrun` and every image it names | | `vendored` (default) | `bin/.oke` and the `.okso` images it names | `okrun` | -| `self-contained` | `bin/` — a program of this machine that carries the image as a section (`.oke`, `__OKE,__image`) and runs it through libokrun — and the `.okso` images beside it | nothing | -| `static` | `bin/`, a program of this machine that carries one static `.oke` | nothing | - -`--linkage static` builds the image with no `.okso`. The image is not -recompiled for `self-contained`: mcpp builds the platform's carrier (`[platform] -carrier`, a package and its entry source), places the image's bytes in the -carrier's section, and packs that program as it packs any other — static (musl) -on Linux, with only the system's libraries on macOS and Windows. Another -system's program is packed on that system, or by `--systems` in a universal -package. +| `self-contained` | `bin/` — a program of this machine that carries the image in the platform's carrier section (`.oke` on ELF and COFF, `__OKE,__carried` on Mach-O) and runs it through libokrun — and the `.okso` images beside it | nothing | +| `static` (`--linkage static --mode self-contained`) | `bin/`, a program of this machine that carries one static `.oke` | nothing | + +These are the packages of one target (`--no-universal`, or a platform that is +not universal). `--linkage static` builds the image with no `.okso`. The image +is not recompiled for `self-contained`: mcpp builds the platform's carrier +(`[platform] carrier`, a package and its entry source) as a generated project +under `target/carrier-pack/--/`, places the image's bytes in +the carrier's section, and packs that program as it packs any other — static +(musl) on Linux, with only the system's libraries on macOS and Windows. ### Android: the application object and its closure under `lib/` @@ -770,14 +782,37 @@ target/dist/hello-0.1.0-openkal/ └── mcpp-package.json ``` -The launchers are generated from the platform's `runner`: each picks the -architecture directory from `uname -m` or `%PROCESSOR_ARCHITECTURE%` and starts -the runner on the image there, looking first in `libexec/` and then on `PATH`. -`--systems linux,macos,windows` builds the loader for each system and -architecture into `libexec/` (the default is this machine's system); a system -whose SDK payload this machine lacks is refused, naming it. Packages built on -several machines are joined with `mcpp pack --merge a.zip b.zip …`: legs and -systems are united, and two files at one path must have the same digest. +The other legs are built during the pack (`Packed leg `). The +launchers are generated from the platform's `runner`: each resolves the package +as the parent of its own directory, picks the architecture directory from +`uname -m` (`x86_64`, `amd64` → `x86_64`; `aarch64`, `arm64` → `aarch64`) or +from `%PROCESSOR_ARCHITECTURE%` / `%PROCESSOR_ARCHITEW6432%`, and starts the +runner on the image there with its own arguments, looking for the runner first +in `libexec//-/` and then on `PATH`. A machine of an +architecture the package does not hold gets exit status 126 and a sentence +naming both. Files `[runtime] deploy` places lie beside each leg's image. +`[pack] universal = false` is `--no-universal`; a platform that states no +`runner` cannot make a universal package, and the pack is refused with +`--no-universal` as the remedy. + +Under `--mode self-contained` the runner travels: for each system of +`--systems linux,macos,windows` (default: this machine's) and each architecture +of the package, mcpp builds the platform's carrier with nothing carried — a +program that runs the image its first argument names, as `okrun` does — into +`libexec//-/`. A Linux runner is static (musl); another +system's is built for its conventional row (`-macos`, +`-windows-gnu`), and a system whose SDK payload this machine lacks is +refused by that build, naming it. + +Packages built on several machines are joined with +`mcpp pack --merge a.zip --merge b.zip` (also `--merge a.zip,b.zip`). The inputs +are package directories or archives (`.zip`, `.tar`, `.tar.gz`, `.tgz`, +`.tar.zst`, `.tar.xz`) this command wrote; nothing is built. Legs and systems +are united, each file is taken once, and the result is described anew. Inputs +of another name, version, platform, linkage or mode are refused, and so is one +path with two digests, naming the path, both digests and both inputs. The +container is `--pkg-format`, else the extension of `-o`, else the first +input's. ### A single-file package @@ -785,15 +820,32 @@ systems are united, and two files at one path must have the same digest. | Target and mode | The file | On start | |---|---|---| -| native, `vendored` or `self-contained` | the engine's stub and the staged tree, zstd-compressed | the first run unpacks the tree into the user's cache, and later runs start it from there | -| native Linux, `--linkage static` | a static stub and the static program | as above, with no system library | -| ok image, `self-contained` | the platform's carrier and the images of its architecture | the carrier loads the images from memory and writes nothing to disk | - -The cache is `$XDG_CACHE_HOME` (else `~/.cache`) on Linux, `~/Library/Caches` -on macOS and `%LOCALAPPDATA%` on Windows, below a directory named by the -payload's sha256. The payload's digest is checked before it is unpacked, it is -unpacked under a temporary name and renamed into place, and an incomplete -directory is never used. +| native, `vendored` or `self-contained` | the engine's launcher carrying the staged tree as one zstd frame | the first run unpacks the tree into the user's cache, and later runs start it from there | +| native Linux, `--linkage static` | a static launcher and the static program | as above, with no system library | +| ok image | the platform's carrier and the image, or a zip of all its images | the carrier loads the images from memory and writes nothing to disk | + +A native program's file is `target/dist/--` (`.exe` on +Windows); its tree is staged under `target/single-file/stage/`. The launcher is +a small program whose source mcpp writes with each pack (under +`target/single-file//`, over mcpplibs.os and zstd) and builds for the +target: static (musl) on Linux, with the system's libraries only on macOS and +Windows. The payload is one zstd frame (`--compress zstd[:]`, the +default, or `none`) of the tree. The first run unpacks it into +`/mcpp/single-file//`, where `` is +`$XDG_CACHE_HOME` (else `~/.cache`) on Linux, `~/Library/Caches` on macOS and +`%LOCALAPPDATA%` on Windows: into a name of its own beside it, renamed into +place without replacing when every file is written (`.mcpp-complete` marks a +complete tree; the directory is the user's alone on Linux and macOS). The frame +carries its content's checksum, and a payload that does not match it is never +unpacked. The launcher then becomes the program — `bin/`, the wrapper at +the root of a self-contained Linux tree, or `.exe` at the root of a +Windows tree — with its arguments, and its exit status is the program's. + +An ok image's file is `target/dist/---` (`.exe` on +Windows), one per system of `--systems` (default: this machine's). The carrier +carries the image alone when it names no other image, and otherwise a stored +zip of all the images, each aligned to 64 KiB, which the carrier's entry reads +through openkal-run's bundle interface (openkal-platform 0.17.2). ### `mcpp-package.json` @@ -809,18 +861,37 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} ``` -`mcpp emit xpkg` writes one index entry per system and architecture, or, for a -universal package, one asset for every system with the loader's package as a -dependency. +The file is written for every pack, a provided format's included. Each file's +`role` is `program`, `image`, `library`, `loader`, `launcher`, `resource` or +`metadata`; in a universal package every file of `lib//` is an image of +that leg. Symbolic links are not listed. `linkage` is `static`, `shared` or +`declared` (each library as it declares itself); `launch` is `program` for one +native leg and `runner` with an `entry` per architecture for a universal +package. + +`mcpp emit xpkg --package [--package …] --url [-o ]` +writes the xim-pkgindex entry (spec 2) of packages this command wrote, from +their `mcpp-package.json`. A universal package is one asset that `linux`, +`macosx` and `windows` install alike, with the runner as a dependency +(`deps = { "okrun" }`) unless the package carries it; packages of one system +and architecture each make that system's entry, one URL with `${arch}` and a +sha256 per architecture. The install moves the archive's top directory into +place and registers the program with xvm (the `.cmd` launcher on Windows). ### Compression and signing `--compress zstd|gzip|xz|none[:level]` (or `[pack] compress`) chooses the -compression of `tar`; the default is gzip. A `zip` stores its entries -uncompressed, and the images inside an okb are stored so that the loader maps -them in place. `[pack] sign = ["", …, "{}"]` runs once on each program and -image of the staged tree before the container is written; a failure stops the -pack. +compression of `tar` (`.tar.gz`, the default, `.tar.zst`, `.tar.xz`, `.tar`). +`tar` writes the archive and mcpp compresses it with its own codecs (zlib, zstd +and liblzma), so no compressor is looked for on `PATH` and an archive is the +same bytes on every host. A `zip` deflates its entries (`gzip[:level]`) or +stores them (`none`); a single file's payload is `zstd` or `none`; the images +inside an okb are stored so that the loader maps them in place. + +`[pack] sign = ["", …, "{}"]` runs once on each program, image, library +and loader of the staged tree before the container is written, and on each +single file; `{}` is the file. A failure stops the pack, naming the argv and its +output, and `mcpp-package.json` states the digests of the signed files. ## Configuration @@ -833,7 +904,8 @@ default_mode = "static" # override the normal vendored default for b linkage = "shared" # the default of --linkage format = "tar" # the default of --pkg-format compress = "zstd" # the default of --compress -sign = ["signtool", "sign", "{}"] # run on each program and image before the container is written +sign = ["signtool", "sign", "{}"] # run on each program, image, library and loader before the container +universal = false # = --no-universal strip = true # default. false ships the artifacts as built debug_symbols = "dist/debug" # separate the debug info here instead of discarding it include = ["share/**", "config/*.toml"] # extra files to bundle @@ -860,7 +932,6 @@ The `static` mode additionally requires a musl toolchain configured under - Distribution formats such as `.deb`, `.rpm`, AppImage and `.msi` come from packages through `--pkg-format `; the engine provides `tar`, `dir`, `zip` and `single-file`. -- A `zip` stores its entries uncompressed. - One file that runs on several systems is not produced; a single-file package serves one system and one architecture. - An ELF program's closure is resolved by running its dynamic linker, so a Linux diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 8f6ac08de..0b5c4714f 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -67,7 +67,8 @@ the platform added with `origin = "platform"`. | the compile | `[platform.build]`, `[platform.build.'cfg(…)']` | flags for every unit of the target | | the link | `[platform.link]` | `driver`, `flags`, `start`, `script`, `inputs`, the names `exe` and `shared`, the `seal` steps | | the run | `runner`, `runners` | how `mcpp run` and `mcpp test` start an artifact, and named device actions | -| the carrier | `carrier` | the program `mcpp pack --mode self-contained` builds around a loaded artifact | +| the carrier | `carrier` | the program `mcpp pack` builds of a loaded artifact for a system: the self-contained program, a universal package's runner, a single file | +| the package | `universal` | a pack of any target covers all of them, in one package ([10](10-pack-and-release.md)) | | the code | `plugin` | a build program run in the platform phase, for what depends on the graph | | the group | `[platform.versions]` | one version of each package of a family | | the host part | `[platform.hosted]` | what a host triple takes when it selects the platform | @@ -224,8 +225,17 @@ seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] platform phase (`mcpp::phase()` is `"platform"`) and may state flags, link fields and runners (`mcpp::link_driver`, `mcpp::link_input_from`, `mcpp::artifact_name`, `mcpp::seal`, `mcpp::run_host`); any other directive is - refused. A board-support package's build program states the same fields in - its build phase ([34](34-authoring-a-bsp.md)). + refused, except the package formats the platform provides (`pack-format` and + `artifact` actions, which make e.g. an `okb` from the staged tree). A + board-support package's build program states the same fields in its build + phase ([34](34-authoring-a-bsp.md)). +- **The carrier's entry reads what it carries.** mcpp places the carried bytes + between `mcpp_carried_begin` and `mcpp_carried_end` (the entry's name at + `mcpp_carried_name`): nothing, when the program is a universal package's + runner and runs the image its first argument names; the image itself; or a + stored zip of the images (`PK\x03\x04`, each aligned to 64 KiB) for a + single file of an image that names others ([SPEC-013](specs/platforms.md) + §11). During development a project selects the working tree with `--platform path:` or `platform = { path = "" }`. `mcpp test` in the @@ -238,7 +248,7 @@ architecture, and reports the others as built and not run. ```sh mcpp self env --target x86_64-openkal # toolchain, platform, tools, runner mcpp self env --target x86_64-openkal --format json -mcpp self env --host # the host graph +mcpp self env --host # the host graph, its runner and tools once its build programs were built mcpp toolchain list # the PLATFORM column mcpp toolchain install --target x86_64-openkal # fetch and build what the target needs ``` diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 168926358..e32284bae 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -359,7 +359,10 @@ select as they do for a build ([35](35-platforms.md)). ``` `platform` is `null` when no platform is selected; its shape is the `plan` -event's (§9). A reason the project's sources would be refused for (an `import +event's (§9). `runner.program` is the platform's own tool when the runner names +one, else the program found on this machine. For `--host` the host platform's +`runner` and `tools` are reported once the project's build programs have been +built under it. A reason the project's sources would be refused for (an `import std` with no C++ run-time in the graph) is a `warning` diagnostic with the code `MCPP_CONFIGURATION_SOURCES_REFUSED`, and the configuration is still reported. A configuration that cannot be prepared carries `MCPP_CONFIGURATION_FAILED` and @@ -715,10 +718,10 @@ to stderr, including what the build programs and tools the pack starts print. | field | | |---|---| -| `artifacts` | one record per produced artifact: `path` (absolute), `type` (`file` or `directory`), `format` (the `--pkg-format` value, `tar` when omitted), `targets` (the canonical triple of each leg that went into it) and `sha256` (a file's digest; `null` for a directory; 2026.10.27.1+). A dispatched format reports the terminal outputs of the actions the request introduced; a several-`--target` Android pack reports one artifact whose `targets` lists every leg | +| `artifacts` | one record per produced artifact: `path` (absolute), `type` (`file` or `directory`), `format` (the container: the `--pkg-format` value; when none is given, `zip` for a Windows program and `tar` otherwise), `targets` (the canonical triple of each leg that went into it) and `sha256` (a file's digest; `null` for a directory; 2026.10.27.1+). A dispatched format reports the terminal outputs of the actions the request introduced; a several-`--target` Android pack reports one artifact whose `targets` lists every leg | | `stage` | the tree the artifact was made from: `dir`, `manifest` (the stage manifest below) and `closure` (`walked` or `not-walked`); `null` for a library package, when no tree was staged, and for a pack of several members | -| `linkage`, `mode`, `pkgFormat` *(2026.10.27.1+)* | the three choices of the pack ([10](10-pack-and-release.md)) | -| `legs`, `systems` *(2026.10.27.1+)* | the canonical triple of each leg, and the systems whose loaders a self-contained universal package carries | +| `linkage`, `mode`, `pkgFormat` *(2026.10.27.1+)* | the three choices of the pack ([10](10-pack-and-release.md)): `linkage` is `static`, `shared` or `declared`; `pkgFormat` is the `format` of the artifacts (`tar`, `zip`, `dir`, `single-file`, or a provided format) | +| `legs`, `systems` *(2026.10.27.1+)* | the canonical triple of each leg (a universal package lists every target of its platform), and the `-` of each runner a self-contained universal package carries | | `stages` *(2026.10.1.1+)* | present only for a pack of several members (`--workspace`, or `-p` repeated): one record per member in `[workspace] members` order, with `member` (the qualified package name) and the `dir`, `manifest` and `closure` of `stage` | For a pack of several members `artifacts` lists every member's artifacts, in the diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 37b15ce77..724f3bd78 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -57,12 +57,13 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 |---|---|---|---| | 链接 | `--linkage` | 库在程序里，还是在程序旁边 | `static`、`shared` | | 闭包 | `--mode` | 运行该包的机器提供什么 | `system`、`vendored`（默认）、`self-contained` | -| 容器 | `--pkg-format` | 交付成什么 | `tar`（默认）、`dir`、`zip`、`single-file`、某个包提供的格式 | +| 容器 | `--pkg-format` | 交付成什么 | `tar`、`zip`、`dir`、`single-file`、某个包提供的格式；未给出时 Windows 程序为 `zip`，其余为 `tar` | | 腿 | `--target`，可重复 | 一个包覆盖哪些目标 | 三元组 | -- **链接**是程序的属性，按 `[build] dependency_linkage` 构建。Linux 上 `--linkage static` - 选择 `-linux-musl` 行，产出全静态程序；`--target …-linux-musl` 隐含它。ok 映像上 - 它是一个 `.oke`、没有 `.okso`。 +- **链接**是程序的属性。`--linkage static|shared`（或 `[pack] linkage`）是这次打包的构建的依赖链接， + 优先于 `[build] dependency_linkage` 与构建配置中的值；未给出时每个库按它自己的声明链接。Linux 上 + `--linkage static` 选择 `-linux-musl` 行（`-linux-gnu` 目标换成对应的 musl 行），全静态程序 + 不论 `--mode` 为何都整体打包；`--target …-linux-musl` 隐含它。ok 映像上它是一个 `.oke`、没有 `.okso`。 - **闭包**是包的属性：共享库闭包中有多少随程序携带。`--mode static` 是 `--linkage static --mode self-contained` 的简写。 - **容器**是交付的形状，见下文。 @@ -97,10 +98,16 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 | 组合 | 出路 | |---|---| | `--pkg-format single-file` 与 `--mode system` | `--mode vendored` 或 `self-contained`：单文件携带它所运行的东西 | -| 通用包之外、对一个可执行文件给出多个 `--target` | 每个目标打一次，再 `--merge` | -| 自带加载器的通用包之外使用 `--systems` | `--mode self-contained`：只有携带加载器的包随系统不同 | +| `--compress` 与 `--pkg-format dir` 或提供者的格式 | `--pkg-format tar`、`zip` 或 `single-file`：只有容器被压缩 | +| `--compress zstd` 或 `xz` 与 `--pkg-format zip` | `--compress gzip`（DEFLATE）或 `none`，或 `--pkg-format tar` | +| `--compress gzip` 或 `xz` 与 `--pkg-format single-file` | `--compress zstd[:]` 或 `none`：启动器解压 zstd | +| 通用包之外、对一个可执行文件给出多个 `--target` | 每个目标打一次 | +| 自带加载器的通用包与映像的单文件之外使用 `--systems` | 在声明 `universal = true` 的平台上用 `--mode self-contained`，或 `--pkg-format single-file` | +| 平台未声明 `runner` 时打通用包 | `--no-universal` | | 对 `lib` 或 `shared` 目标使用被分派的格式 | 库的路线，[12](12-binary-distribution.md) | +与容器不相容的 `[pack] compress`（目录，或 zip 的 `zstd`）在这次打包中被搁置；只有选项被拒绝。 + ### 需要宿主提供某种能力的程序 「自包含」有一个下限。有些库只能来自目标机器：图形驱动的用户态部分与正在 @@ -172,7 +179,8 @@ mcpp pack --pkg-format single-file # one executable file mcpp pack --pkg-format tar --compress zstd # .tar.zst mcpp pack --target x86_64-openkal # a universal package: every architecture of the platform mcpp pack --target x86_64-openkal --mode self-contained --systems linux,macos,windows -mcpp pack --merge a.zip b.zip -o all.zip # one tree from packages built on several machines +mcpp pack --merge a.zip --merge b.zip -o all.zip # one package from packages built on several machines +mcpp emit xpkg --package all.zip --url https://example.org/v1.0.0 # its xim-pkgindex entry ``` `pack` 与 `run` 接受 `--format` 作为 `--pkg-format` 的旧写法，并打印一条警告（2026.10.27.1+）。 @@ -576,8 +584,9 @@ mcpp pack --target x86_64-windows-gnu # from a Linux host 有两点值得知道： -- 条目是**存储而非压缩**的，因此 Windows 包的体积大致等于其内容之和。压缩是 - 体积上的优化，不是正确性问题，目前尚未实现。 +- 条目在变小时由 zlib **DEFLATE 压缩**，否则存储；`--compress gzip:` 设定级别， + `--compress none` 存储全部条目。未给出格式时 Windows 程序写 zip；显式给出 `--pkg-format tar` + 则写 tar。 - 压缩包是**确定性的**：不读取任何时间戳，因此同一棵树打两次包字节完全一致， 公布出去的校验和才有意义。 @@ -625,13 +634,13 @@ ok 映像经 `okrun` 在其架构的每个系统上运行（见 [24](24-openkal- |---|---|---| | `system` | `bin/.oke` | `okrun` 与它所需的每个映像 | | `vendored`（默认） | `bin/.oke` 与它所需的 `.okso` 映像 | `okrun` | -| `self-contained` | `bin/`：本机的程序，以段（`.oke`、`__OKE,__image`）携带映像并经 libokrun 运行，旁边是所需的 `.okso` 映像 | 无 | -| `static` | `bin/`：本机的程序，携带一个静态 `.oke` | 无 | +| `self-contained` | `bin/`：本机的程序，以平台载体的段（ELF 与 COFF 上为 `.oke`，Mach-O 上为 `__OKE,__carried`）携带映像并经 libokrun 运行，旁边是所需的 `.okso` 映像 | 无 | +| `static`（`--linkage static --mode self-contained`） | `bin/`：本机的程序，携带一个静态 `.oke` | 无 | -`--linkage static` 构建不带 `.okso` 的映像。`self-contained` 不重新编译映像：mcpp 构建平台的载体 -（`[platform] carrier`，一个包与它的入口源码），把映像的字节放进载体的段，再按任何程序的方式打包该程序 —— -Linux 上为静态（musl），macOS 与 Windows 上只引用系统自带的库。其他系统的程序在该系统上打包，或在通用包中 -由 `--systems` 构建。 +以上是单个目标的包（`--no-universal`，或平台不声明通用包）。`--linkage static` 构建不带 `.okso` 的映像。 +`self-contained` 不重新编译映像：mcpp 把平台的载体（`[platform] carrier`，一个包与它的入口源码）生成为 +`target/carrier-pack/--/` 下的工程，把映像的字节放进载体的段，再按任何程序的方式打包该程序 —— +Linux 上为静态（musl），macOS 与 Windows 上只引用系统自带的库。 ### Android：应用目标文件与它的闭包在 `lib/` 下 @@ -673,10 +682,22 @@ target/dist/hello-0.1.0-openkal/ └── mcpp-package.json ``` -启动器由平台的 `runner` 生成：按 `uname -m` 或 `%PROCESSOR_ARCHITECTURE%` 选择架构目录，以该目录中的映像 -启动运行器，先在 `libexec/` 中、再在 `PATH` 上查找运行器。`--systems linux,macos,windows` 为每个系统与架构构建 -加载器放入 `libexec/`（缺省为本机系统）；本机缺少某个系统的 SDK 载荷时拒绝并点名。多台机器打出的包以 -`mcpp pack --merge a.zip b.zip …` 合并：腿与系统取并集，同一路径上的两个文件摘要必须相同。 +其余的腿在打包过程中构建（`Packed leg `）。启动器由平台的 `runner` 生成：以自身目录的上一级为包的 +根，按 `uname -m`（`x86_64`、`amd64` → `x86_64`；`aarch64`、`arm64` → `aarch64`）或 +`%PROCESSOR_ARCHITECTURE%` / `%PROCESSOR_ARCHITEW6432%` 选择架构目录，以该目录中的映像与自身的参数启动运行器， +先在 `libexec//-/` 中、再在 `PATH` 上查找运行器。包中没有本机架构时以状态 126 退出，写出两者。 +`[runtime] deploy` 放置的文件位于每条腿的映像旁边。`[pack] universal = false` 等同 `--no-universal`；平台未声明 +`runner` 时不能打通用包，打包被拒绝，出路为 `--no-universal`。 + +`--mode self-contained` 时运行器随包携带：对 `--systems linux,macos,windows`（缺省为本机系统）的每个系统与包的 +每个架构，mcpp 构建不携带任何内容的平台载体 —— 一个像 `okrun` 一样运行其第一个参数所指映像的程序 —— 放入 +`libexec//-/`。Linux 的运行器为静态（musl）；其他系统的按其惯常行（`-macos`、 +`-windows-gnu`）构建，本机缺少该系统的 SDK 载荷时该次构建拒绝并点名。 + +多台机器打出的包以 `mcpp pack --merge a.zip --merge b.zip`（也可 `--merge a.zip,b.zip`）合并。输入是本命令写出的 +包目录或归档（`.zip`、`.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz`），不进行任何构建。腿与系统取并集，每个 +文件只取一份，结果重新描述。名字、版本、平台、链接或模式不同的输入被拒绝；同一路径有两个摘要时也被拒绝， +写出该路径、两个摘要与两个输入。容器取 `--pkg-format`，否则取 `-o` 的扩展名，否则取第一个输入的容器。 ### 单文件包 @@ -684,13 +705,23 @@ target/dist/hello-0.1.0-openkal/ | 目标与模式 | 文件 | 启动时 | |---|---|---| -| 原生，`vendored` 或 `self-contained` | 引擎的启动桩加 zstd 压缩的暂存树 | 首次运行把树解到用户缓存，之后从那里启动 | -| 原生 Linux，`--linkage static` | 静态启动桩加静态程序 | 同上，不依赖任何系统库 | -| ok 映像，`self-contained` | 平台的载体加本架构的映像 | 载体从内存加载映像，不写盘 | - -缓存在 Linux 上为 `$XDG_CACHE_HOME`（否则 `~/.cache`），macOS 上为 `~/Library/Caches`，Windows 上为 -`%LOCALAPPDATA%`，其下以载荷的 sha256 命名目录。解压前校验载荷摘要，以临时名解压后原子改名，不完整的目录 -从不使用。 +| 原生，`vendored` 或 `self-contained` | 引擎的启动器，以一个 zstd 帧携带暂存树 | 首次运行把树解到用户缓存，之后从那里启动 | +| 原生 Linux，`--linkage static` | 静态启动器加静态程序 | 同上，不依赖任何系统库 | +| ok 映像 | 平台的载体加映像，或全部映像的一个 zip | 载体从内存加载映像，不写盘 | + +原生程序的文件为 `target/dist/--`（Windows 上带 `.exe`），其树暂存在 +`target/single-file/stage/` 下。启动器是一个小程序，mcpp 每次打包写出它的源码（`target/single-file//`， +基于 mcpplibs.os 与 zstd）并为目标构建：Linux 上为静态（musl），macOS 与 Windows 上只引用系统自带的库。载荷是 +树的一个 zstd 帧（`--compress zstd[:]`，缺省，或 `none`）。首次运行把它解到 +`/mcpp/single-file/<载荷的 sha256>/`，`` 在 Linux 上为 `$XDG_CACHE_HOME`（否则 `~/.cache`）， +macOS 上为 `~/Library/Caches`，Windows 上为 `%LOCALAPPDATA%`：先解到旁边一个自己的名字，全部文件写完后以不替换 +的方式改名到位（`.mcpp-complete` 标记完整的树；Linux 与 macOS 上该目录只对当前用户可写）。帧携带内容的校验和， +不符的载荷从不解开。随后启动器成为程序 —— `bin/`、self-contained 的 Linux 树根部的包装脚本，或 Windows +树根部的 `.exe` —— 带着它的参数，退出状态即程序的退出状态。 + +ok 映像的文件为 `target/dist/---`（Windows 上带 `.exe`），`--systems` 的每个系统 +（缺省为本机系统）各一个。映像不引用其他映像时载体只携带它，否则携带全部映像的一个 stored zip（每个映像对齐到 +64 KiB），载体的入口经 openkal-run 的 bundle 接口读取（openkal-platform 0.17.2）。 ### `mcpp-package.json` @@ -704,13 +735,25 @@ target/dist/hello-0.1.0-openkal/ "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} ``` -`mcpp emit xpkg` 为每个系统与架构写一个索引条目；通用包则写一个对应所有系统的资产，并以加载器的包为依赖。 +每次打包都写出该文件，提供者的格式也不例外。每个文件的 `role` 为 `program`、`image`、`library`、`loader`、 +`launcher`、`resource` 或 `metadata`；通用包中 `lib//` 下的每个文件都是该腿的映像。符号链接不列出。 +`linkage` 为 `static`、`shared` 或 `declared`（每个库按它自己的声明）；`launch` 对单个原生腿为 `program`， +对通用包为 `runner` 加每个架构的 `entry`。 + +`mcpp emit xpkg --package <归档> [--package …] --url <前缀> [-o <文件>]` 由本命令写出的包的 +`mcpp-package.json` 生成 xim-pkgindex 条目（spec 2）。通用包是 `linux`、`macosx`、`windows` 同样安装的一个资产， +包不携带运行器时以运行器为依赖（`deps = { "okrun" }`）；各系统各架构一个的包，每个系统一项，一个带 `${arch}` 的 +URL 与每个架构的 sha256。安装把归档的顶层目录移到位，并向 xvm 注册程序（Windows 上为 `.cmd` 启动器）。 ### 压缩与签名 -`--compress zstd|gzip|xz|none[:level]`（或 `[pack] compress`）选择 `tar` 的压缩；缺省为 gzip。`zip` 的条目 -不压缩存放；okb 中的映像以 stored 存放，使加载器原地映射。`[pack] sign = ["", …, "{}"]` 在写容器之前 -对暂存树中的每个程序与映像各运行一次；失败使打包停止。 +`--compress zstd|gzip|xz|none[:level]`（或 `[pack] compress`）选择 `tar` 的压缩（`.tar.gz` 为缺省，另有 +`.tar.zst`、`.tar.xz`、`.tar`）。`tar` 写出归档，mcpp 用自己的编解码器（zlib、zstd、liblzma）压缩，因此不在 +`PATH` 上查找压缩程序，同一归档在每个宿主上字节相同。`zip` 的条目以 DEFLATE 压缩（`gzip[:level]`）或存储 +（`none`）；单文件的载荷为 `zstd` 或 `none`；okb 中的映像以 stored 存放，使加载器原地映射。 + +`[pack] sign = ["", …, "{}"]` 在写容器之前对暂存树中的每个程序、映像、库与加载器各运行一次，也对每个单文件 +运行一次；`{}` 为该文件。失败使打包停止，写出 argv 与其输出；`mcpp-package.json` 写出签名后文件的摘要。 ## 配置项 @@ -722,7 +765,8 @@ default_mode = "static" # override the normal vendored default for b linkage = "shared" # the default of --linkage format = "tar" # the default of --pkg-format compress = "zstd" # the default of --compress -sign = ["signtool", "sign", "{}"] # run on each program and image before the container is written +sign = ["signtool", "sign", "{}"] # run on each program, image, library and loader before the container +universal = false # = --no-universal strip = true # default. false ships the artifacts as built debug_symbols = "dist/debug" # separate the debug info here instead of discarding it include = ["share/**", "config/*.toml"] # extra files to bundle @@ -748,6 +792,5 @@ force_bundle = ["libfoo.so"] # bundle even if matched by the PEP 600 list - `.deb`、`.rpm`、AppImage、`.msi` 等分发格式经 `--pkg-format ` 由包提供；引擎提供 `tar`、`dir`、 `zip` 与 `single-file`。 -- `zip` 的条目不压缩存放。 - 不产出在多个系统上运行的单个文件；一个单文件包服务一个系统与一个架构。 - ELF 程序的闭包由其动态链接器解析，因此不能在 Windows 上打包 Linux 程序。 diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index cc1d768b1..f06eaf4e3 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -58,7 +58,8 @@ mcpp run --target x86_64-openkal | 编译 | `[platform.build]`、`[platform.build.'cfg(…)']` | 目标每个翻译单元的标志 | | 链接 | `[platform.link]` | `driver`、`flags`、`start`、`script`、`inputs`、名字 `exe` 与 `shared`、封装步骤 `seal` | | 运行 | `runner`、`runners` | `mcpp run` 与 `mcpp test` 怎样启动产物,以及具名的设备动作 | -| 载体 | `carrier` | `mcpp pack --mode self-contained` 为依赖加载器的产物构建的程序 | +| 载体 | `carrier` | `mcpp pack` 为依赖加载器的产物按系统构建的程序:self-contained 的程序、通用包的运行器、单文件 | +| 包 | `universal` | 任一目标的打包覆盖全部目标,成为一个包([10](10-pack-and-release.md)) | | 代码 | `plugin` | 在平台阶段运行的构建程序,陈述随依赖图而变的部分 | | 版本组 | `[platform.versions]` | 一个家族中每个包的一个版本 | | 宿主部分 | `[platform.hosted]` | 宿主三元组选用该平台时适用的部分 | @@ -196,8 +197,13 @@ seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] - **封装步骤失败时不发布。** 上一次成功构建发布的产物保留原位。 - **随依赖图而变的部分写成代码。** `plugin = "build.mcpp"` 在平台阶段运行(`mcpp::phase()` 为 `"platform"`), 可以陈述标志、链接字段与运行器(`mcpp::link_driver`、`mcpp::link_input_from`、`mcpp::artifact_name`、 - `mcpp::seal`、`mcpp::run_host`);其他指令被拒绝。板级支持包的构建程序在构建阶段陈述同样的字段 + `mcpp::seal`、`mcpp::run_host`);其他指令被拒绝,平台提供的包格式除外(`pack-format` 与 `artifact` + action,例如由暂存树生成 `okb`)。板级支持包的构建程序在构建阶段陈述同样的字段 ([34](34-authoring-a-bsp.md))。 +- **载体的入口读取它携带的内容。** mcpp 把携带的字节放在 `mcpp_carried_begin` 与 `mcpp_carried_end` 之间 + (入口的名字在 `mcpp_carried_name`):程序为通用包的运行器时不携带任何内容,运行其第一个参数所指的映像; + 或映像本身;或映像引用其他映像时的单文件所携带的全部映像的一个 stored zip(`PK\x03\x04`,每个映像对齐到 + 64 KiB)([SPEC-013](../specs/platforms.md) §11)。 开发期间,工程以 `--platform path:` 或 `platform = { path = "" }` 选择工作树。在平台包中执行 `mcpp test` 而不给 `--target` 时,以该包为平台为其每个 `targets` 构建测试,运行本机架构的那些,其余报告为 @@ -208,7 +214,7 @@ seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] ```sh mcpp self env --target x86_64-openkal # toolchain, platform, tools, runner mcpp self env --target x86_64-openkal --format json -mcpp self env --host # the host graph +mcpp self env --host # the host graph, its runner and tools once its build programs were built mcpp toolchain list # the PLATFORM column mcpp toolchain install --target x86_64-openkal # fetch and build what the target needs ``` diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index f532fad18..57853a495 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -316,7 +316,9 @@ mcpp self env --format json } ``` -未选择平台时 `platform` 为 `null`；其形状与 `plan` 事件的相同（§9）。工程源码会被拒绝的原因 +未选择平台时 `platform` 为 `null`；其形状与 `plan` 事件的相同（§9）。运行器指名平台自己的工具时 +`runner.program` 为该工具，否则为本机找到的程序。`--host` 时，工程的构建程序已在宿主平台下构建过，才报告宿主平台 +的 `runner` 与 `tools`。工程源码会被拒绝的原因 （图中没有 C++ 运行时而 `import std`）是一条代码为 `MCPP_CONFIGURATION_SOURCES_REFUSED` 的 `warning` 诊断，配置照常报告。无法准备的配置带 `MCPP_CONFIGURATION_FAILED`，没有 `data`。 @@ -641,10 +643,10 @@ mcpp pack [target] [--pkg-format ] [--target ...] --message-format js | 字段 | | |---|---| -| `artifacts` | 每个产出的产物一条记录：`path`（绝对路径）、`type`（`file` 或 `directory`）、`format`（`--pkg-format` 的取值，省略时为 `tar`）、`targets`（进入该产物的每条腿的规范三元组），以及 `sha256`（文件的摘要；目录为 `null`；2026.10.27.1+）。被分派的格式报告本次请求引入的那些 action 的终端输出；一个多 `--target` 的 Android 打包报告一个产物，其 `targets` 列出每一条腿 | +| `artifacts` | 每个产出的产物一条记录：`path`（绝对路径）、`type`（`file` 或 `directory`）、`format`（容器：`--pkg-format` 的取值；未给出时 Windows 程序为 `zip`，其余为 `tar`）、`targets`（进入该产物的每条腿的规范三元组），以及 `sha256`（文件的摘要；目录为 `null`；2026.10.27.1+）。被分派的格式报告本次请求引入的那些 action 的终端输出；一个多 `--target` 的 Android 打包报告一个产物，其 `targets` 列出每一条腿 | | `stage` | 该产物所来自的那棵树：`dir`、`manifest`（即下文的暂存清单）与 `closure`（`walked` 或 `not-walked`）；库包、没有暂存任何树的情形，以及对多个成员的打包，此字段为 `null` | -| `linkage`、`mode`、`pkgFormat` *（2026.10.27.1+）* | 本次打包的三个选择（[10](10-pack-and-release.md)） | -| `legs`、`systems` *（2026.10.27.1+）* | 每条腿的规范三元组，以及自带加载器的通用包所携带的加载器所属的系统 | +| `linkage`、`mode`、`pkgFormat` *（2026.10.27.1+）* | 本次打包的三个选择（[10](10-pack-and-release.md)）：`linkage` 为 `static`、`shared` 或 `declared`；`pkgFormat` 为产物的 `format`（`tar`、`zip`、`dir`、`single-file` 或提供者的格式） | +| `legs`、`systems` *（2026.10.27.1+）* | 每条腿的规范三元组（通用包列出其平台的每个目标），以及自带加载器的通用包所携带的每个运行器的 `-` | | `stages` *（2026.10.1.1+）* | 只在对多个成员的打包（`--workspace`，或重复的 `-p`）中出现：每个成员一条记录，按 `[workspace] members` 的顺序，带 `member`（限定的包名），以及与 `stage` 相同的 `dir`、`manifest`、`closure` | 对多个成员的打包，`artifacts` 按同样的成员顺序列出每个成员的产物，每条记录多出 From 3ab923942722fc38f35b2696397649c7f82033fb Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:24:07 +0900 Subject: [PATCH 23/54] docs: CHANGELOG 2026.10.27.1 packaging entries (three axes, universal packages, single files, codecs, mcpp-package.json, --merge, emit xpkg --package, signing; e2e 952-957); docs/90 counts (155 unit, 644 e2e) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 49 +++++++++++++++++++++++++-------- docs/90-build-from-source.md | 8 +++--- docs/zh/90-build-from-source.md | 8 +++--- 3 files changed, 46 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 70dcedbd4..69891fa01 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,7 +37,8 @@ - **The host graph's platform.** With `[host] platform` build programs and their libraries are compiled as ok images of this machine's architecture, with one toolchain on Linux, macOS and Windows, and run by the platform's - runner; a platform's own tools are always built natively. + runner; a platform's own tools are always built natively. e2e 953; a + platform package unrelated to openkal is e2e 952. - **Protocol 17** (SPEC-007 R9.10): `link_driver`, `link_input`, `link_input_from`, `artifact_name`, `seal` and `run_host`, stated by a platform's plugin in the platform phase (`mcpp::phase()` is `"platform"`) @@ -52,16 +53,42 @@ with no `--target`, it builds the tests for each of the platform's targets and runs those of this machine's architecture. - **Packaging along three axes** (SPEC-014; docs/10): `--linkage - static|shared` (the program), `--mode system|vendored|self-contained` (what - the machine provides; `--mode static` is `--linkage static --mode - self-contained`), `--pkg-format` (what is delivered) and the legs - (`--target`, repeatable). A platform with `universal = true` packs every - architecture into one tree with generated launchers (`--no-universal`, - `--systems`, `--merge`); `--pkg-format single-file` delivers one executable; - `--compress zstd|gzip|xz|none`; `[pack] linkage`, `format`, `compress`, - `sign`; every staged tree carries `mcpp-package.json`; the `mcpp.pack` - envelope gains `linkage`, `mode`, `pkgFormat`, `legs`, `systems` and each - artifact's `sha256`. + static|shared` (the program's dependency linkage; on Linux `static` is the + musl row), `--mode system|vendored|self-contained` (what the machine + provides; `--mode static` is `--linkage static --mode self-contained`), + `--pkg-format tar|zip|dir|single-file|` (what is delivered; a + format left unstated is a zip for a Windows program and a tar elsewhere) and + the legs (`--target`, repeatable). Impossible combinations are refused before + the build. `[pack] linkage`, `format`, `compress`, `sign` and `universal` are + the project's defaults. e2e 954. +- **Universal packages.** A platform with `universal = true` (openkal-platform) + packs every one of its targets into one package, `--`, + with `lib//` per leg and generated launchers (`bin/`, + `bin/.cmd`) that pick the machine's architecture and start the + platform's runner; `--mode self-contained` adds the runner of each system of + `--systems` and each architecture under `libexec/`; `--no-universal` packs the + selected target alone. e2e 955. +- **Single files.** `--pkg-format single-file` writes one executable: for a + native program the engine's launcher carrying the staged tree as a zstd + frame, extracted once into the user's cache under the payload's sha256 and + run from there; for an ok image the platform's carrier running the image + from memory, one file per system of `--systems`. e2e 956. +- **Compression in mcpp's own codecs.** `--compress gzip|zstd|xz|none[:level]` + (`.tar.gz`, `.tar.zst`, `.tar.xz`, `.tar`) with zlib, zstd and liblzma as + dependencies of mcpp: no compressor is looked for on `PATH`, and an archive + is the same bytes on every host. A zip's entries are deflated (`gzip`) or + stored (`none`). +- **`mcpp-package.json` and what reads it.** Every staged tree describes its + package: platform, the three choices, legs, systems, how it starts, and each + file with its role, leg and sha256. `mcpp pack --merge a --merge b` joins + packages written on several machines (legs and systems united, one digest + per path); `mcpp emit xpkg --package --url ` writes the + xim-pkgindex entry, one asset for every system for a universal package with + its runner as a dependency. `[pack] sign = [argv]` signs each program, image, + library and loader before the container, and each single file. e2e 957. +- The `mcpp.pack` envelope gains `linkage`, `mode`, `pkgFormat`, `legs`, + `systems` and each file artifact's `sha256`. A platform's plugin may provide + package formats (`pack-format`, `artifact` actions) in the platform phase. - `mcpp.lock` records the packages a platform added with `origin = "platform"`; the `plan` event and `result.json` carry `configuration.platform` and `hostPlatform`. diff --git a/docs/90-build-from-source.md b/docs/90-build-from-source.md index f1f503e37..c91a75101 100644 --- a/docs/90-build-from-source.md +++ b/docs/90-build-from-source.md @@ -81,19 +81,19 @@ src/ └── scaffold/ `mcpp new` and templates tests/ -├── unit/ 154 C++ tests, discovered by `mcpp test` -└── e2e/ 638 shell scripts against a real binary +├── unit/ 155 C++ tests, discovered by `mcpp test` +└── e2e/ 644 shell scripts against a real binary ``` ## Test Organization Two layers, and they answer different questions. -**Unit tests** (`tests/unit/`, 154 files) are C++ programs `mcpp test` +**Unit tests** (`tests/unit/`, 155 files) are C++ programs `mcpp test` discovers. They exercise a module's contract with no binary and no filesystem state. -**End-to-end tests** (`tests/e2e/NN_.sh`, 638 files) run a real `mcpp` +**End-to-end tests** (`tests/e2e/NN_.sh`, 644 files) run a real `mcpp` binary against a real project. `run_all.sh` is the CI entry point. `mcpp test` does **not** run them. diff --git a/docs/zh/90-build-from-source.md b/docs/zh/90-build-from-source.md index 74d38c85b..e01f3bc31 100644 --- a/docs/zh/90-build-from-source.md +++ b/docs/zh/90-build-from-source.md @@ -83,18 +83,18 @@ src/ └── scaffold/ `mcpp new` and templates tests/ -├── unit/ 154 C++ tests, discovered by `mcpp test` -└── e2e/ 638 shell scripts against a real binary +├── unit/ 155 C++ tests, discovered by `mcpp test` +└── e2e/ 644 shell scripts against a real binary ``` ## 测试组织 两层，回答的是不同的问题。 -**单元测试**（`tests/unit/`，154 个文件）是 `mcpp test` 发现的 C++ +**单元测试**（`tests/unit/`，155 个文件）是 `mcpp test` 发现的 C++ 程序。它们检验一个模块的契约，不需要二进制、也不需要文件系统状态。 -**端到端测试**（`tests/e2e/NN_.sh`，638 个文件）拿真实的 `mcpp` +**端到端测试**（`tests/e2e/NN_.sh`，644 个文件）拿真实的 `mcpp` 二进制跑真实工程。`run_all.sh` 是 CI 的入口。`mcpp test` **不**运行它们。 ```bash From 0dec0958cdfbe5185215590e5bcd9b3870f09504 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:32:20 +0900 Subject: [PATCH 24/54] prepare: the state on the heap (a nested prepare overflowed a Windows main thread's stack); pack: [pack] include / exclude staged once at the project's paths; docs: the platform's spelling in mcpp-package.json, include Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/10-pack-and-release.md | 4 ++-- docs/specs/packaging.md | 5 +++-- docs/zh/10-pack-and-release.md | 4 ++-- src/build/prepare/driver.cpp | 10 +++++++-- src/pack/pack.cppm | 33 ++++++++++++++++++++++++++++ tests/e2e/954_pack_has_three_axes.sh | 21 ++++++++++++++++++ 6 files changed, 69 insertions(+), 8 deletions(-) diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index f7945ce2b..24f0068f2 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -855,7 +855,7 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it ([SPEC-014](specs/packaging.md) §4.5): ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "mcpplibs.openkal-platform@0.17.1", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.1", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} @@ -908,7 +908,7 @@ sign = ["signtool", "sign", "{}"] # run on each program, image, libra universal = false # = --no-universal strip = true # default. false ships the artifacts as built debug_symbols = "dist/debug" # separate the debug info here instead of discarding it -include = ["share/**", "config/*.toml"] # extra files to bundle +include = ["share/**", "config/*.toml"] # project files staged at the same path, once exclude = ["debug/**"] # Fine-tune the vendored filtering policy. The configuration key keeps its diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 3914445e6..ed3e6017b 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -117,7 +117,8 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` | `compress` | 未给 `--compress` 时的压缩(§5.1) | | `sign` | 签名步骤的 argv,`{}` 为文件(§7) | | `universal` | `false` 时等同 `--no-universal` | -| `strip`、`debug_symbols`、`include`、`exclude`、`[pack.bundle-project]` | 既有键 | +| `include`、`exclude` | 工程中按 glob 选出的文件（相对工程根），放在暂存树中同样的相对路径下，一个包只放一份（通用包亦然）；`exclude` 去掉其中匹配的文件；`mcpp-package.json` 中角色为 `resource` | +| `strip`、`debug_symbols`、`[pack.bundle-project]` | 既有键 | ## 3. 管线 @@ -182,7 +183,7 @@ SDK 载荷时该次构建拒绝,写出该系统与载荷。 "schema": 1, "name": "hello", "version": "0.1.0", - "platform": "mcpplibs.openkal-platform@0.17.2", + "platform": "openkal-platform@0.17.2", "linkage": "declared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 724f3bd78..c93bbfbe1 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -729,7 +729,7 @@ ok 映像的文件为 `target/dist/---`（Windows 上 提供的格式、`--merge` 与 `mcpp emit xpkg` 读取它（[SPEC-014](../specs/packaging.md) §4.5）： ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "mcpplibs.openkal-platform@0.17.1", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.1", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} @@ -769,7 +769,7 @@ sign = ["signtool", "sign", "{}"] # run on each program, image, libra universal = false # = --no-universal strip = true # default. false ships the artifacts as built debug_symbols = "dist/debug" # separate the debug info here instead of discarding it -include = ["share/**", "config/*.toml"] # extra files to bundle +include = ["share/**", "config/*.toml"] # 工程文件，放在包中同样的相对路径下，只放一份 exclude = ["debug/**"] # Fine-tune the vendored filtering policy. The configuration key keeps its diff --git a/src/build/prepare/driver.cpp b/src/build/prepare/driver.cpp index 72b27dc53..c3213ce92 100644 --- a/src/build/prepare/driver.cpp +++ b/src/build/prepare/driver.cpp @@ -150,8 +150,14 @@ prepare_build_pass(bool print_fingerprint, std::vector extraTargets, BuildOverrides overrides) { read_build_programs_marker(overrides); - PrepareState state(print_fingerprint, includeDevDeps, - std::move(extraTargets), std::move(overrides)); + // ON THE HEAP. The state is over 100 KiB, and a prepare nests: a host + // tool's sub-build, then its build programs' graph, each its own prepare. + // Three of them on a Windows main thread's 1 MiB stack overflowed it + // (measured: openkal-platform's tests on windows-2022 ended with exit 127 + // and no word, in the host graph of `okld`'s build). + auto statePtr = std::make_unique(print_fingerprint, includeDevDeps, + std::move(extraTargets), std::move(overrides)); + PrepareState& state = *statePtr; pending_flag_words_notes().clear(); g_notesOnFailure.clear(); diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index ae4e8877e..4e773abf6 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -49,6 +49,7 @@ import mcpp.pack.strip; import mcpp.pack.zip; import mcpp.pack.compress; import mcpp.pack.distribution; +import mcpp.modgraph.glob; // [pack] include / exclude import mcpplibs.os.platform; import mcpp.toolchain.platform; import mcpp.invocation; @@ -1506,11 +1507,43 @@ make_zip(const std::filesystem::path& stagingRoot, const std::filesystem::path& // and loader signed by `[pack] sign`, then every file described in // `mcpp-package.json` with its role, its leg and its digest. Before the // container, so the archive holds what the document says. +// `[pack] include`: the project's files the globs name, at the same path +// below the tree's root, once whatever the legs (a `share/**` is the +// package's, not an architecture's); `[pack] exclude` drops what it names. +std::expected stage_included(const Plan& plan) { + namespace fs = std::filesystem; + if (plan.includeGlobs.empty()) return {}; + std::vector files; + auto excluded = [&](const fs::path& f) { + return std::ranges::any_of(plan.excludeGlobs, [&](std::string const& g) { + return mcpp::modgraph::path_matches_glob(f, plan.projectRoot, g); + }); + }; + auto onFile = [&](const fs::path& f) { + const bool named = std::ranges::any_of(plan.includeGlobs, [&](std::string const& g) { + return mcpp::modgraph::path_matches_glob(f, plan.projectRoot, g); + }); + if (named && !excluded(f)) files.push_back(f); + }; + mcpp::modgraph::walk_glob_tree(plan.projectRoot, plan.projectRoot, + [](const fs::path&) { return true; }, onFile); + std::error_code ec; + for (auto const& f : files) { + const auto to = plan.stagingRoot / f.lexically_relative(plan.projectRoot); + fs::create_directories(to.parent_path(), ec); + fs::copy_file(f, to, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format("[pack] include: cannot stage {}: {}", + f.string(), ec.message())}); + } + return {}; +} + std::expected finalize_tree(const Plan& plan) { namespace fs = std::filesystem; namespace d = mcpp::pack::distribution; std::error_code ec; if (!fs::is_directory(plan.stagingRoot, ec)) return {}; + if (auto r = stage_included(plan); !r) return r; const auto canonical = [](std::string const& t) { if (t.empty()) return mcpp::toolchain::triple::host_triple().str(); auto parsed = mcpp::toolchain::triple::parse(t); diff --git a/tests/e2e/954_pack_has_three_axes.sh b/tests/e2e/954_pack_has_three_axes.sh index 305a3b5ed..b0bef9205 100755 --- a/tests/e2e/954_pack_has_three_axes.sh +++ b/tests/e2e/954_pack_has_three_axes.sh @@ -18,6 +18,8 @@ # system; --compress on a directory; zstd in a zip; --systems for a package # that carries no loader. # F. `--format` is the earlier spelling of `--pkg-format`, with a warning. +# G. `[pack] include` stages the files it names at their paths, less what +# `[pack] exclude` names, and the metadata lists them as resources. set -e TMP=$(mktemp -d) @@ -118,4 +120,23 @@ echo "ok: impossible combinations are refused before a build" grep -q "earlier spelling of --pkg-format" f.log || fail "F: no warning for the earlier spelling" f.log echo "ok: --format is the earlier spelling, with a warning" +# G +mkdir -p share/data +printf 'kept\n' > share/data/keep.txt +printf 'dropped\n' > share/data/drop.tmp +cat >> mcpp.toml <<'TOML' +include = ["share/**"] +exclude = ["share/**/*.tmp"] +TOML +"$MCPP" pack --pkg-format dir -o inc > g.log 2>&1 || fail "G: pack with [pack] include" g.log +[ -f target/dist/inc/share/data/keep.txt ] || fail "G: the included file is not staged" g.log +[ -f target/dist/inc/share/data/drop.tmp ] && fail "G: the excluded file was staged" g.log +python3 - <<'PY' || fail "G: the metadata" target/dist/inc/mcpp-package.json +import json +d = json.load(open("target/dist/inc/mcpp-package.json")) +roles = {f["path"]: f["role"] for f in d["files"]} +assert roles.get("share/data/keep.txt") == "resource", roles +PY +echo "ok: [pack] include stages what it names, less what exclude names" + echo "PASS: 954 pack has three axes" From 907589954322348a939171a5ea1acd038d82f921 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:38:14 +0900 Subject: [PATCH 25/54] pack: an aligned zip member's padding is zipalign's extra field (0xD935), or a gap before a short one; unit tests of the codecs, the zip writer, sha256, the launchers, mcpp-package.json and the single file's payload Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/zip.cppm | 35 +++-- tests/unit/test_pack_distribution.cpp | 187 ++++++++++++++++++++++++++ 2 files changed, 213 insertions(+), 9 deletions(-) create mode 100644 tests/unit/test_pack_distribution.cpp diff --git a/src/pack/zip.cppm b/src/pack/zip.cppm index c5add48e6..3a85456ff 100644 --- a/src/pack/zip.cppm +++ b/src/pack/zip.cppm @@ -190,14 +190,31 @@ write(const std::filesystem::path& out, const std::vector& entries, const std::string& body = c.method == 8 ? packed : *data; c.packed = static_cast(body.size()); - // The padding that puts a stored member's data on the boundary. - std::uint16_t extra = 0; + // The padding that puts a stored member's data on the boundary, as + // zipalign writes it: an extra field (id 0xD935) holding the alignment + // and zero bytes. A gap too short for a field (under six bytes) lies + // before the local header instead, where no directory entry names it + // and a reader through the central directory never looks. + std::string extraField; if (options.align > 1 && c.method == 0) { - const auto dataAt = blob.size() + 30 + c.name.size(); - const auto rem = dataAt % options.align; - if (rem != 0) extra = static_cast(options.align - rem); - // An extra field holds at most 64 KiB - 1 bytes; a boundary of - // 64 KiB is reached by at most that many. + auto pad_at = [&](std::size_t headerAt) { + const auto dataAt = headerAt + 30 + c.name.size(); + return (options.align - dataAt % options.align) % options.align; + }; + auto need = pad_at(blob.size()); + if (need != 0 && need < 6) { + blob.append(need, '\0'); + need = pad_at(blob.size()); + // After the gap the data is on the boundary; nothing more. + } + if (need >= 6) { + detail::put16(extraField, 0xD935); + detail::put16(extraField, static_cast(need - 4)); + detail::put16(extraField, static_cast( + options.align > 0xFFFF ? 0 : options.align)); + extraField.append(need - 6, '\0'); + } + c.offset = static_cast(blob.size()); } detail::put32(blob, 0x04034b50); // local file header @@ -210,9 +227,9 @@ write(const std::filesystem::path& out, const std::vector& entries, detail::put32(blob, c.packed); detail::put32(blob, c.size); detail::put16(blob, static_cast(c.name.size())); - detail::put16(blob, extra); // extra field length + detail::put16(blob, static_cast(extraField.size())); blob += c.name; - blob.append(extra, '\0'); + blob += extraField; blob += body; central.push_back(std::move(c)); diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp new file mode 100644 index 000000000..b251d37a5 --- /dev/null +++ b/tests/unit/test_pack_distribution.cpp @@ -0,0 +1,187 @@ +// The distribution half of `mcpp pack` (SPEC-014): the codecs, the zip writer's +// deflated and aligned members, SHA-256, the launchers, the package metadata +// and the single file's payload. +#include + +import std; +import mcpp.pack.compress; +import mcpp.pack.distribution; +import mcpp.pack.zip; + +namespace fs = std::filesystem; +namespace c = mcpp::pack::compress; +namespace d = mcpp::pack::distribution; + +namespace { + +struct Tmp { + fs::path path; + Tmp() { + path = fs::temp_directory_path() / std::format("mcpp-dist-{}", + std::chrono::steady_clock::now().time_since_epoch().count()); + fs::create_directories(path); + } + ~Tmp() { std::error_code ec; fs::remove_all(path, ec); } +}; + +void write(const fs::path& p, std::string_view text) { + fs::create_directories(p.parent_path()); + std::ofstream(p, std::ios::binary) << text; +} + +std::uint32_t le32(std::string_view b, std::size_t at) { + return static_cast(static_cast(b[at])) + | static_cast(static_cast(b[at + 1])) << 8 + | static_cast(static_cast(b[at + 2])) << 16 + | static_cast(static_cast(b[at + 3])) << 24; +} + +} // namespace + +TEST(PackCompress, ParsesAlgorithmsAndLevels) { + auto z = c::parse("zstd:19"); + ASSERT_TRUE(z); + EXPECT_EQ(z->algorithm, c::Algorithm::Zstd); + EXPECT_EQ(z->level, 19); + EXPECT_TRUE(z->stated); + EXPECT_TRUE(c::parse("none")); + EXPECT_FALSE(c::parse("none:3")); + EXPECT_FALSE(c::parse("gzip:10")); + EXPECT_FALSE(c::parse("brotli")); + EXPECT_EQ(c::tar_extension(c::Algorithm::Xz), ".tar.xz"); + EXPECT_EQ(c::tar_extension(c::Algorithm::None), ".tar"); +} + +TEST(PackCompress, EveryCodecRoundTrips) { + std::string text; + for (int i = 0; i < 4000; ++i) text += std::format("line {} of a package\n", i % 37); + for (auto a : { c::Algorithm::Gzip, c::Algorithm::Zstd, c::Algorithm::Xz, c::Algorithm::None }) { + c::Choice ch; + ch.algorithm = a; + auto packed = c::bytes(text, ch); + ASSERT_TRUE(packed) << c::name(a); + if (a != c::Algorithm::None) EXPECT_LT(packed->size(), text.size()) << c::name(a); + auto back = c::decompress(*packed, a); + ASSERT_TRUE(back) << c::name(a); + EXPECT_EQ(*back, text) << c::name(a); + } + auto raw = c::deflate_raw(text, 0); + ASSERT_TRUE(raw); + auto back = c::inflate_raw(*raw, text.size()); + ASSERT_TRUE(back); + EXPECT_EQ(*back, text); +} + +TEST(PackDistribution, Sha256OfKnownVectors) { + EXPECT_EQ(d::sha256(""), "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"); + EXPECT_EQ(d::sha256("abc"), "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"); + EXPECT_EQ(d::sha256(std::string(1000, 'a')), + "41edece42d63e8d9bf515a9ba6932e1c20cbc9f5a5d134645adb5db1b9737ea3"); +} + +TEST(PackZip, AnAlignedMemberIsStoredOnItsBoundary) { + Tmp t; + write(t.path / "a.oke", std::string(3000, 'x')); + write(t.path / "libb.okso", std::string(70000, 'y')); + std::vector entries{ + { "a.oke", t.path / "a.oke", false, {} }, + { "libb.okso", t.path / "libb.okso", false, {} }, + }; + mcpp::pack::zip::WriteOptions o; + o.align = 65536; + ASSERT_TRUE(mcpp::pack::zip::write(t.path / "out.zip", entries, o)); + std::ifstream in(t.path / "out.zip", std::ios::binary); + std::string b{std::istreambuf_iterator(in), {}}; + // Each member's data, found through the central directory, starts on a + // 64 KiB boundary and is stored. + std::size_t eocd = b.rfind(std::string("PK\x05\x06", 4)); + ASSERT_NE(eocd, std::string::npos); + std::size_t at = le32(b, eocd + 16); + for (int n = 0; n < 2; ++n) { + ASSERT_EQ(le32(b, at), 0x02014b50u); + const auto method = static_cast(b[at + 10]); + const auto nameLen = static_cast(static_cast(b[at + 28])) + | static_cast(static_cast(b[at + 29])) << 8; + const auto local = le32(b, at + 42); + const auto lname = static_cast(static_cast(b[local + 26])) + | static_cast(static_cast(b[local + 27])) << 8; + const auto lextra = static_cast(static_cast(b[local + 28])) + | static_cast(static_cast(b[local + 29])) << 8; + EXPECT_EQ(method, 0); + EXPECT_EQ((local + 30 + lname + lextra) % 65536, 0u); + at += 46 + nameLen; + } +} + +TEST(PackZip, DeflateWhenItShrinks) { + Tmp t; + write(t.path / "text", std::string(10000, 'z')); + mcpp::pack::zip::WriteOptions o; + o.deflate = true; + ASSERT_TRUE(mcpp::pack::zip::write(t.path / "out.zip", { { "text", t.path / "text", false, {} } }, o)); + EXPECT_LT(fs::file_size(t.path / "out.zip"), 1000u); + // And it reads back. + auto root = d::unpack(t.path / "out.zip", t.path / "x"); + ASSERT_FALSE(root) << "a zip without one top-level directory is not a package"; +} + +TEST(PackDistribution, LaunchersPickTheArchitectureAndTheRunner) { + d::Launch l; + l.name = "hello"; + l.runner = { "okrun", "{}" }; + l.entry = { { "x86_64", "hello.oke" }, { "aarch64", "hello.oke" } }; + l.loader = "okrun"; + const auto sh = d::posix_launcher(l); + EXPECT_TRUE(sh.starts_with("#!/bin/sh\n")); + EXPECT_NE(sh.find("x86_64|amd64|AMD64) arch=x86_64"), std::string::npos); + EXPECT_NE(sh.find("image=\"$top/lib/aarch64/hello.oke\""), std::string::npos); + EXPECT_NE(sh.find("runner=\"$top/libexec/okrun/$os-$arch/okrun\""), std::string::npos); + EXPECT_NE(sh.find("exec \"$runner\" \"$image\" \"$@\""), std::string::npos); + const auto cmd = d::windows_launcher(l); + EXPECT_NE(cmd.find("PROCESSOR_ARCHITEW6432"), std::string::npos); + EXPECT_NE(cmd.find("libexec\\okrun\\windows-%arch%\\okrun.exe"), std::string::npos); + EXPECT_EQ(d::arch_dir("arm64"), "aarch64"); + EXPECT_EQ(d::arch_dir("AMD64"), "x86_64"); +} + +TEST(PackDistribution, PackageMetadataDescribesEveryFile) { + Tmp t; + const auto root = t.path / "app-1.0-openkal"; + write(root / "bin/app", "#!/bin/sh\n"); + write(root / "lib/x86_64/app.oke", "image"); + write(root / "share/data.txt", "data"); + write(root / "LICENSE", "license"); + d::Package p; + p.name = "app"; + p.version = "1.0"; + p.legs = { "x86_64-openkal" }; + p.runner = { "okrun", "{}" }; + p.entry = { { "x86_64", "lib/x86_64/app.oke" } }; + d::Stated s; + s.role["bin/app"] = "launcher"; + auto described = d::describe(root, p, s); + ASSERT_TRUE(described); + auto back = d::read(root); + ASSERT_TRUE(back) << back.error(); + std::map roles; + for (auto const& f : back->files) roles[f.path] = f.role; + EXPECT_EQ(roles["bin/app"], "launcher"); + EXPECT_EQ(roles["lib/x86_64/app.oke"], "image"); + EXPECT_EQ(roles["share/data.txt"], "resource"); + EXPECT_EQ(roles["LICENSE"], "metadata"); + EXPECT_EQ(back->entry.at("x86_64"), "lib/x86_64/app.oke"); + for (auto const& f : back->files) + EXPECT_EQ(f.sha256, d::sha256_file(root / f.path)) << f.path; +} + +TEST(PackDistribution, TreePayloadIsOrderedRecords) { + Tmp t; + write(t.path / "b/x", "12"); + write(t.path / "a", "3"); + auto payload = d::tree_payload(t.path); + ASSERT_TRUE(payload); + EXPECT_TRUE(payload->starts_with("MCPPTREE1\n")); + // `a` before `b` before `b/x`. + const auto a = payload->find("a"), b = payload->find("b/x"); + EXPECT_LT(a, b); +} From 3c0ec4f2cf4ec09524704f76e34eb84cee394a95 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:44:05 +0900 Subject: [PATCH 26/54] platform: the ELF linker of a payload under any of lld's names (ld.lld, lld, lld-link with -flavor gnu): the llvm payload for Windows ships lld-link alone Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/build/flags.cppm | 9 +++++---- src/build/prepare/platform.cpp | 2 +- src/freestanding/linkline.cppm | 16 ++++++++++++++++ 3 files changed, 22 insertions(+), 5 deletions(-) diff --git a/src/build/flags.cppm b/src/build/flags.cppm index e305fe53b..fa3153e1b 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -2508,12 +2508,13 @@ CompileFlags compute_flags(const BuildPlan& plan) { f.cxx += words(p.flags.cxxflags); f.as += words(p.flags.asmflags); if (p.fields.link.driver.value == "lld") { - f.ldDriver = mcpp::freestanding::resolve_lld(plan.toolchain.binaryPath); + f.ldDriver = mcpp::freestanding::resolve_lld_any(plan.toolchain.binaryPath); f.ld = words(p.fields.link.flags.value) + words(p.flags.ldflags) + user_ldflags; - // `lld` by that name is the generic driver, which chooses its + // `lld` and `lld-link` are the same program, which chooses its // flavour from its name or from `-flavor`; a payload that ships - // only it is asked for the ELF linker explicitly. - if (f.ldDriver.stem() == "lld") f.ld = " -flavor gnu" + f.ld; + // one of them and not `ld.lld` is asked for the ELF linker + // explicitly. + if (f.ldDriver.stem() != "ld.lld") f.ld = " -flavor gnu" + f.ld; f.ldC = f.ld; f.sysroot.clear(); f.bFlag.clear(); diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 1e8633a94..8cc53f6b3 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -592,7 +592,7 @@ std::expected step13_finish_platform(PrepareState& state, Bui if (P.active && P.fields.link.driver.value == "lld" && !state.overrides.plan_only && std::ranges::any_of(ctx.plan.linkUnits, [](auto const& lu) { return lu.kind != mcpp::build::LinkUnit::StaticLibrary; }) - && mcpp::freestanding::resolve_lld(state.tc->binaryPath).empty()) { + && mcpp::freestanding::resolve_lld_any(state.tc->binaryPath).empty()) { refusal::record(refusal::Code::LldRequiredAbsent); return std::unexpected(std::format( "the platform {} links {} with ld.lld itself ({}), and the toolchain has no ld.lld " diff --git a/src/freestanding/linkline.cppm b/src/freestanding/linkline.cppm index 5c6a10228..4e688d7a1 100644 --- a/src/freestanding/linkline.cppm +++ b/src/freestanding/linkline.cppm @@ -185,6 +185,22 @@ inline std::filesystem::path resolve_lld(const std::filesystem::path& driver) { return {}; } +// THE ELF LINKER OF A PAYLOAD, under any of the names lld answers to: +// `ld.lld`, the generic `lld`, or `lld-link` -- one program that takes its +// flavour from its name or from a leading `-flavor`. A payload that ships the +// COFF name alone (the llvm payload for Windows) still links an ELF file with +// `-flavor gnu`, which the caller adds unless the name is `ld.lld`. +inline std::filesystem::path resolve_lld_any(const std::filesystem::path& driver) { + if (auto p = resolve_lld(driver); !p.empty()) return p; + if (driver.empty()) return {}; + std::error_code ec; + for (const char* name : { "lld-link", "lld-link.exe" }) { + auto p = driver.parent_path() / name; + if (std::filesystem::exists(p, ec)) return p; + } + return {}; +} + // llvm-objcopy inside the same payload as the driver, for the flat image a // flasher takes. Derived from the driver's path for the same reason // `resolve_lld` is: a search is how the wrong tool gets picked. Returns "" for From e99974e8da9e1b74710133557bac6fe05312bdf5 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:47:08 +0900 Subject: [PATCH 27/54] explain: the platform's refusals by their reason tokens Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/cli.cppm | 4 ++-- src/doctor.cppm | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/src/cli.cppm b/src/cli.cppm index 50527d44b..acb7164e2 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -1226,7 +1226,7 @@ int run(int argc, char** argv) { .description("Show mcpp version")) .subcommand(cl::App("explain") .description("Show extended description for an error code") - .arg(cl::Arg("code").help("Error code such as E0001").required())) + .arg(cl::Arg("code").help("Error code such as E0001, or a refusal's reason token such as platform-version-group").required())) .subcommand(cl::App("store") .description("Compare this home's payloads with its upstreams") .subcommand(cl::App("dedupe") @@ -1249,7 +1249,7 @@ int run(int argc, char** argv) { // `mcpp self explain E0001`. .subcommand(cl::App("explain") .description("Show extended description for an error code") - .arg(cl::Arg("code").help("Error code such as E0001").required()) + .arg(cl::Arg("code").help("Error code such as E0001, or a refusal's reason token such as platform-version-group").required()) .action(wrap_rc(cmd_explain_action))) // ─── bareword `version` alias ───────────────────────────────── diff --git a/src/doctor.cppm b/src/doctor.cppm index cefc31c27..fecbaba81 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -1801,6 +1801,30 @@ export int explain_code(std::string_view code) { "silently misbehave, so resolution stops instead. Upgrade mcpp:\n" " curl -fsSL https://github.com/mcpp-community/mcpp/releases/latest/download/install.sh | bash\n" "To bypass for debugging only: MCPP_INDEX_FLOOR=ignore mcpp build"}, + // The platform's refusals (SPEC-013 §14), by their reason token. + {"platform-not-a-platform", "the selected package states no platform", + "The package selected as the target's platform (--platform, MCPP_PLATFORM,\n" + "[target.] platform, config.toml, the target table) has no [platform]\n" + "section. Select a platform package, or none: --platform none."}, + {"platform-target-unserved", "the platform does not state this target", + "The platform's [platform] targets (and [platform.hosted] targets) do not name\n" + "the target being built. Build one of the targets it names, or select another\n" + "platform for this target under [target.] platform."}, + {"platform-tool-absent", "a tool the platform names is not available", + "The platform links or seals with a tool ([platform.tools]) that the graph did\n" + "not build and this machine does not have. Name the package that builds it:\n" + " [target.'cfg(platform = \"\")'.tools]\n" + " = \"@\""}, + {"platform-version-group", "a family package outside the platform's version group", + "The platform's [platform.versions] names one version of each package of its\n" + "family, whose pins are exact; the graph holds another version of one. Select\n" + "the platform release of that version, or write the group's version where the\n" + "package is requested. `mcpp why` names who requested it."}, + {"platform-phase-directive", "a platform plugin stated what a platform cannot", + "A platform's build.mcpp runs in the platform phase (mcpp::phase() == \"platform\")\n" + "and states the platform's build, link and runner fields and the package formats\n" + "it provides (pack-format, artifact actions). Move any other directive to a\n" + "package of the graph."}, }; for (auto& e : table) { if (e.code == code) { From 10e2d8b47d0b43d9989bbc840ee4d9930f2b5ad4 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 06:58:08 +0900 Subject: [PATCH 28/54] run, test: a runner the platform names as one of its tools is that tool's program; a bare runner name on Windows takes the system's suffix Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/build/execute.cppm | 12 ++++++++++++ src/build/runner_lookup.cppm | 15 +++++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index b383bcf19..c0dcb70cd 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1073,6 +1073,16 @@ struct ExecutionHost { std::string error; }; +// The platform's own tool by the name a runner gives it (SPEC-013 §10): the +// program the build located for it, before any search. +std::optional platform_tool(const mcpp::build::BuildContext& ctx, + std::string_view name) { + auto const& tools = ctx.plan.platform.tools; + if (auto it = tools.find(std::string(name)); it != tools.end() && !it->second.empty()) + return std::filesystem::path(it->second); + return std::nullopt; +} + ExecutionHost locate_run_under(const mcpp::build::BuildContext& ctx, std::string_view tripleKey) { ExecutionHost h; if (ctx.runUnder.empty()) return h; @@ -2863,6 +2873,7 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, auto tmpl = choice.tmpl; auto found = mcpp::build::runner_lookup::locate( tmpl.front(), ctx.xlingsDepBinDirs, mcpp::invocation::get("PATH").value_or("")); + if (auto tool = platform_tool(ctx, tmpl.front())) found.program = *tool; if (!found.program) { std::println(stderr, "error: {}", mcpp::build::runner_lookup::not_found_message( @@ -3702,6 +3713,7 @@ static void test_resolve_runner(TestBuild& tb, const TestOptions& testOpts, bool if (!runnerTmpl.empty()) { auto found = mcpp::build::runner_lookup::locate( runnerTmpl.front(), ctx->xlingsDepBinDirs, mcpp::invocation::get("PATH").value_or("")); + if (auto tool = platform_tool(*ctx, runnerTmpl.front())) found.program = *tool; if (found.program) runnerTmpl.front() = found.program->string(); else tb.invocationNotRunReason = mcpp::build::runner_lookup::not_found_message( runnerChoice.tripleKey, runnerTmpl.front(), found.searched); diff --git a/src/build/runner_lookup.cppm b/src/build/runner_lookup.cppm index 7b9b0587b..f0bb3ac39 100644 --- a/src/build/runner_lookup.cppm +++ b/src/build/runner_lookup.cppm @@ -88,9 +88,20 @@ inline Lookup locate(std::string_view argv0, out.searched.push_back(a0.parent_path()); return out; } + // A bare name on Windows is a program with the system's suffix, as the + // shell finds it: `okrun` is `okrun.exe`. + auto in_dir = [&](const std::filesystem::path& d) -> std::optional { + if (auto c = d / a0; detail::executable_file(c)) return c; + if constexpr (mcpplibs::os::is_windows) { + if (!a0.has_extension()) + for (auto ext : { ".exe", ".cmd", ".bat" }) + if (auto c = d / (a0.string() + ext); detail::executable_file(c)) return c; + } + return std::nullopt; + }; for (auto const& d : depBinDirs) { out.searched.push_back(d); - if (auto c = d / a0; detail::executable_file(c)) { out.program = c; return out; } + if (auto c = in_dir(d)) { out.program = *c; return out; } } constexpr char sep = mcpplibs::os::is_windows ? ';' : ':'; for (auto part : std::views::split(pathEnv, sep)) { @@ -98,7 +109,7 @@ inline Lookup locate(std::string_view argv0, if (sv.empty()) continue; std::filesystem::path d(sv); out.searched.push_back(d); - if (auto c = d / a0; detail::executable_file(c)) { out.program = c; return out; } + if (auto c = in_dir(d)) { out.program = *c; return out; } } return out; } From 752c64c5d579506605c3c1bd6d5d63b2142f9610 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 07:16:34 +0900 Subject: [PATCH 29/54] pack: a generated program's tree is its bin/, or the tree itself for a Windows program's flat one (M3, loaders and single files on Windows) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/pack.cppm | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 4e773abf6..c7894564f 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -2251,8 +2251,9 @@ std::string carried_assembly(std::string_view system, std::string_view sectionNa } // A program generated under `dir` (its manifest, entry and carried section) -// packed for `row` as a directory; returns the directory (`bin/` holds the -// program and what it needs beside it). +// packed for `row` as a directory; returns the directory that holds the +// program and what it needs beside it: the tree's `bin/`, or, for a Windows +// program, whose tree is flat, the tree itself. std::expected pack_generated_program(const std::filesystem::path& dir, const ProgramRow& row) { namespace fs = std::filesystem; @@ -2268,10 +2269,10 @@ pack_generated_program(const std::filesystem::path& dir, const ProgramRow& row) if (spawnError != 0 || built.exit_code != 0) return std::unexpected(Error{std::format( "packing the program under {} failed:\n{}", dir.string(), built.output)}); - if (!fs::is_directory(out / "bin", ec)) - return std::unexpected(Error{std::format("the program's package under {} has no bin/", - out.string())}); - return out; + if (fs::is_directory(out / "bin", ec)) return out / "bin"; + if (fs::is_directory(out, ec)) return out; + return std::unexpected(Error{std::format("the program's package under {} was not written", + out.string())}); } // THE STATIC ROW'S TOOLCHAIN, STATED: the carrier's own graph supplies a @@ -2428,7 +2429,7 @@ run_loaded(const Plan& plan) fs::path(plan.binaryName).stem().string(), std::string(host_system()), std::string(mcpp::toolchain::platform::host_arch), false }); if (!packed) return std::unexpected(packed.error()); - fs::copy(*packed, plan.stagingRoot, + fs::copy(*packed, plan.stagingRoot / "bin", fs::copy_options::recursive | fs::copy_options::overwrite_existing, ec); staged = plan.stagingRoot / "bin" / (fs::path(plan.binaryName).stem().string() + std::string(mcpplibs::os::exe_suffix)); @@ -2477,7 +2478,7 @@ run_loaded(const Plan& plan) const auto into = plan.stagingRoot / "libexec" / launch.loader / std::format("{}-{}", system, leg.arch); fs::create_directories(into, ec); - fs::copy(*packed / "bin", into, + fs::copy(*packed, into, fs::copy_options::recursive | fs::copy_options::overwrite_existing, ec); if (ec) return std::unexpected(Error{std::format( "cannot place the runner for {}-{}: {}", system, leg.arch, ec.message())}); @@ -3155,7 +3156,7 @@ std::expected, Error> single_file(const Plan& auto packed = detail::carrier_build(plan, { carried, plan.binaryName, stem, system, arch, true }); if (!packed) return std::unexpected(packed.error()); const auto suffix = system == "windows" ? std::string(".exe") : std::string(); - const auto program = *packed / "bin" / (stem + suffix); + const auto program = *packed / (stem + suffix); if (!fs::is_regular_file(program, ec)) return std::unexpected(Error{std::format("the carrier built for {}-{} has no {}", system, arch, program.filename().string())}); @@ -3209,7 +3210,7 @@ std::expected, Error> single_file(const Plan& auto packed = detail::pack_generated_program(dir, row); if (!packed) return std::unexpected(packed.error()); const auto suffix = system == "windows" ? std::string(".exe") : std::string(); - const auto program = *packed / "bin" / (stem + suffix); + const auto program = *packed / (stem + suffix); const auto out = output_for(t ? t->str() : plan.triple, suffix); fs::create_directories(out.parent_path(), ec); fs::copy_file(program, out, fs::copy_options::overwrite_existing, ec); From d14a53cf0baf090b2faf71e121e3b510313c0157 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 07:20:22 +0900 Subject: [PATCH 30/54] =?UTF-8?q?design=20part=202=20=C2=A79:=20the=20meas?= =?UTF-8?q?ured=20sizes=20(image=20flags,=20-Oz,=20containers,=20okb,=20si?= =?UTF-8?q?ngle=20file)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...part2-packaging-and-distribution-design.md | 21 ++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md b/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md index 760c1229f..2a5e8f376 100644 --- a/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md +++ b/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md @@ -151,7 +151,7 @@ xim-pkgindex 的包描述可以表达 openkal 程序： ## 9. 体积 -本批全部实施，并在 hello（C++，`import std`）与 mcppls 服务器两个映像、一个原生 hello 上实测前后，表写入本节（实施时填写）。 +本批全部实施，实测见本节末表（hello 映像；mcppls 服务器与 mcpp-ide 后端的数字随其 PR 写入 §15）。 | 处 | 做法 | |---|---| @@ -161,6 +161,25 @@ xim-pkgindex 的包描述可以表达 openkal 程序： | 去重 | 通用包与 okb 中资源只放一份，只有映像按架构；分发矩阵从"系统 × 架构"降为"架构" | | 共享 | 共享 `.okso` 作为独立 xlings 包，一个架构装一次，多个程序共用（§7） | +**实测（2026-10-28，x86_64 Linux 主机，`import std` 的 hello，release 构建，已 strip）。** + +| 项 | 0.17.1 平台 | 0.17.2 平台 | 变化 | +|---|---|---|---| +| x86_64 映像 `hello.oke` | 1 149 944 B | 1 098 872 B | −4.4%（`-ffunction-sections -fdata-sections`、`--icf=safe`） | +| 同上，`[profile.release] opt = "z"` | — | 998 928 B | 再 −9.1% | + +| 容器（通用包：x86_64 与 aarch64 两组映像） | 大小 | 相对 `.tar.gz` | +|---|---|---| +| `tar`（gzip，缺省） | 787 266 B | — | +| `zip`（deflate） | 787 669 B | +0.05% | +| `tar --compress zstd`（19 级） | 594 457 B | −24.5% | +| `tar --compress xz`（6 级） | 532 196 B | −32.4% | +| `okb`（映像 stored、64 KiB 对齐，为原地映射） | 2 475 329 B | 未压缩：分发时由外层容器或传输压缩 | +| 单文件（openkal，Linux 静态载体加 x86_64 映像） | 1 164 664 B | 一个可执行文件 | + +结论：分发矩阵从"系统 × 架构"降为"架构"后，再以 zstd/xz 压缩容器得到最大收益；映像本身的收益来自段级回收与 `-Oz`。 +okb 不压缩是格式的选择（原地加载），其传输体积由外层（xlings 下载的压缩包或 HTTP 压缩）决定。 + ## 10. openkal 0.17.2 轮（本批最后） | 包 | 版本 | 内容 | From 91b2e3283a7c5398fd202b31b3431feaa5304625 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 07:24:37 +0900 Subject: [PATCH 31/54] build programs: a cached run is replayed only while the paths it named below its output directory exist (e2e 958) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/build/build_program.cppm | 28 +++++++++- ...gram_runs_again_when_its_output_is_gone.sh | 53 +++++++++++++++++++ 2 files changed, 79 insertions(+), 2 deletions(-) create mode 100755 tests/e2e/958_a_build_program_runs_again_when_its_output_is_gone.sh diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 47fbec886..719276126 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -968,7 +968,7 @@ CacheRecord read_cache(const fs::path& bdir) { // Why the cached run is not valid, for the log; empty when it is. std::string cache_stale_reason(const fs::path& root, const fs::path& bdir, const CacheRecord& c, const std::string& programHash, const std::string& compilerHash, - const std::string& ctxHash) { + const std::string& ctxHash, const fs::path& outDir = {}) { if (!c.loaded) return "no record"; if (c.epoch != dirs::kCacheEpoch) return "record epoch"; // pre-epoch entries rerun once if (c.unknownRecord) return "unknown record line"; @@ -995,6 +995,29 @@ std::string cache_stale_reason(const fs::path& root, const fs::path& bdir, const for (auto const& p : c.directives.at(def.slot)) if (!fs::exists(abs_against_root(root, p))) return "output " + p; } + // WHAT THE PROGRAM WROTE INTO ITS OUTPUT DIRECTORY AND NAMED: an include + // directory, a library directory, a file to deploy. A directive that names + // a path below `MCPP_OUT_DIR` which is no longer there was written by a + // run whose output was deleted since (`rm -rf target/.build-mcpp/out`), + // and replaying it hands the build a directory with nothing in it. + if (!outDir.empty()) { + const std::array spellings{ outDir.generic_string(), outDir.string() }; + // The path a value names below the output directory, when it is gone. + auto vanished = [&](const std::string& v) -> std::optional { + std::error_code ec; + for (auto const& text : spellings) { + const auto at = v.find(text); + if (at == std::string::npos) continue; + const auto tab = v.find('\t', at); + auto path = v.substr(at, tab == std::string::npos ? std::string::npos : tab - at); + if (!fs::exists(path, ec)) return path; + } + return std::nullopt; + }; + for (auto const& slot : c.directives.slots) + for (auto const& v : slot) + if (auto path = vanished(v)) return "output " + *path; + } return {}; } @@ -1108,7 +1131,8 @@ std::expected run_build_program_impl( // Fast path: declared inputs + contract unchanged → reapply cached // directives, no run. CacheRecord cache = read_cache(bdir); - const auto stale = cache_stale_reason(root, bdir, cache, programHash, compilerHash, ctxHash); + const auto stale = cache_stale_reason(root, bdir, cache, programHash, compilerHash, ctxHash, + outDir); if (!stale.empty()) { std::string keys; if (stale == "environment contract") keys = changed_contract_keys(bdir, contractEnv); diff --git a/tests/e2e/958_a_build_program_runs_again_when_its_output_is_gone.sh b/tests/e2e/958_a_build_program_runs_again_when_its_output_is_gone.sh new file mode 100755 index 000000000..fb14b4339 --- /dev/null +++ b/tests/e2e/958_a_build_program_runs_again_when_its_output_is_gone.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +# requires: gcc +# 958 -- a build program's cached run is replayed only while what it wrote is +# there. The program writes a header below `MCPP_OUT_DIR` and names the +# directory with `mcpp::include_dir`; once that directory is deleted, a replay +# would hand the compile an include directory with nothing in it. The program +# runs again instead (measured 2026-10-28 on openkal-image's build program: a +# deleted output directory, then generated includes missing). +# +# Criteria: +# A. A second build replays the program (cached) and compiles nothing. +# B. With the program's output deleted, the build runs the program again and +# the unit finds its header. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +cat > build.mcpp <<'CPP' +#include +#include +#include +import mcpp; +int main() { + const std::string inc = std::string(mcpp::out_dir()) + "/inc"; + std::filesystem::create_directories(inc); + std::ofstream(inc + "/generated.h") << "#define GENERATED 7\n"; + mcpp::include_dir(inc.c_str()); + return 0; +} +CPP +printf '#include \n#include \nint main() { std::printf("generated %%d\\n", GENERATED); return 0; }\n' > src/main.cpp +printf '[package]\nname = "regen"\nversion = "0.1.0"\n' > mcpp.toml + +"$MCPP" run > a1.log 2>&1 || fail "A: the first build" a1.log +grep -q "generated 7" a1.log || fail "A: the header did not arrive" a1.log +"$MCPP" build -v > a2.log 2>&1 || fail "A: the second build" a2.log +grep -q "Compiling regen" a2.log && fail "A: the second build compiled" a2.log +echo "ok: a second build replays the program" + +# B +out=$(find target -type f -name generated.h | head -1) +[ -n "$out" ] || fail "B: the generated header is not under target/" +rm -rf "$(dirname "$out")" +rm -rf target/*/*/bin target/*/*/obj +"$MCPP" run -v > b.log 2>&1 || fail "B: the build after the output was deleted" b.log +grep -q "generated 7" b.log || fail "B: the program did not run again" b.log +echo "ok: the program runs again when its output is gone" + +echo "PASS: 958 a build program runs again when its output is gone" From 5598567575959ad5f5c6666824746437d380c32c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 07:42:10 +0900 Subject: [PATCH 32/54] e2e 956: the single file's cache on macOS is ~/Library/Caches; it runs with a home of its own Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...56_a_single_file_runs_without_unpacking.sh | 29 +++++++++++++------ 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/tests/e2e/956_a_single_file_runs_without_unpacking.sh b/tests/e2e/956_a_single_file_runs_without_unpacking.sh index 36690aee8..e8ae5dd56 100755 --- a/tests/e2e/956_a_single_file_runs_without_unpacking.sh +++ b/tests/e2e/956_a_single_file_runs_without_unpacking.sh @@ -18,7 +18,18 @@ set -e TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } -export XDG_CACHE_HOME="$TMP/cache" +# The user's cache, isolated: `$XDG_CACHE_HOME` on Linux; on macOS the cache is +# `~/Library/Caches`, so the files run with a home of their own (mcpp keeps +# the real one). +if [ "$(uname -s)" = Darwin ]; then + CACHE="$TMP/home/Library/Caches" + mkdir -p "$CACHE" + run_file() { HOME="$TMP/home" "$@"; } +else + export XDG_CACHE_HOME="$TMP/cache" + CACHE="$XDG_CACHE_HOME" + run_file() { "$@"; } +fi mkdir -p "$TMP/app/src" cd "$TMP/app" @@ -37,15 +48,15 @@ print(c[0] if c else "") PY ) [ -n "$f" ] || fail "A: no single file under target/dist" a.log -set +e; "$f" alpha beta > run1.log 2>&1; rc=$?; set -e +set +e; run_file "$f" alpha beta > run1.log 2>&1; rc=$?; set -e [ "$rc" = 3 ] || fail "A: the exit status is $rc, not the program's 3" run1.log grep -q "one file: 3 args, last beta" run1.log || fail "A: the program's output" run1.log -dirs=$(ls "$XDG_CACHE_HOME/mcpp/single-file" | grep -v '^\.' | wc -l) +dirs=$(ls "$CACHE/mcpp/single-file" | grep -v '^\.' | wc -l | tr -d ' ') [ "$dirs" = 1 ] || fail "A: the first run left $dirs trees in the cache" -stamp=$(ls --full-time "$XDG_CACHE_HOME/mcpp/single-file"/*/.mcpp-complete 2>/dev/null || stat "$XDG_CACHE_HOME"/mcpp/single-file/*/.mcpp-complete) -set +e; "$f" gamma > run2.log 2>&1; set -e +stamp=$(ls --full-time "$CACHE/mcpp/single-file"/*/.mcpp-complete 2>/dev/null || stat "$CACHE"/mcpp/single-file/*/.mcpp-complete) +set +e; run_file "$f" gamma > run2.log 2>&1; set -e grep -q "last gamma" run2.log || fail "A: the second run" run2.log -stamp2=$(ls --full-time "$XDG_CACHE_HOME/mcpp/single-file"/*/.mcpp-complete 2>/dev/null || stat "$XDG_CACHE_HOME"/mcpp/single-file/*/.mcpp-complete) +stamp2=$(ls --full-time "$CACHE/mcpp/single-file"/*/.mcpp-complete 2>/dev/null || stat "$CACHE"/mcpp/single-file/*/.mcpp-complete) [ "$stamp" = "$stamp2" ] || fail "A: the second run extracted the tree again" echo "ok: a native single file runs, extracting once" @@ -85,10 +96,10 @@ if [ -n "$T" ] && command -v python3 > /dev/null && "$MCPP" toolchain list --for printf 'import std;\nint main(int argc, char**) { std::println("image file: {} args", argc); }\n' > src/main.cpp "$MCPP" pack --target "$T" --pkg-format single-file > d.log 2>&1 || fail "D: pack" d.log g=$(ls target/dist/img-0.1.0-* | head -1) - before=$(find "$XDG_CACHE_HOME" -type f | wc -l) - "$g" x y > d-run.log 2>&1 || fail "D: the image's single file" d-run.log + before=$(find "$CACHE" -type f | wc -l) + run_file "$g" x y > d-run.log 2>&1 || fail "D: the image's single file" d-run.log grep -q "image file: 3 args" d-run.log || fail "D: the output" d-run.log - after=$(find "$XDG_CACHE_HOME" -type f | wc -l) + after=$(find "$CACHE" -type f | wc -l) [ "$before" = "$after" ] || fail "D: the image's single file wrote into the cache" echo "ok: an image's single file runs from memory" fi From d593fe0176f73ed0e62be63afbc757f19e7960eb Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 07:44:23 +0900 Subject: [PATCH 33/54] e2e 951: toolchain install --target prepares an ok-image project for --offline Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- tests/e2e/951_an_ok_image_needs_no_declaration.sh | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/tests/e2e/951_an_ok_image_needs_no_declaration.sh b/tests/e2e/951_an_ok_image_needs_no_declaration.sh index 2ec1e672c..da5c6f144 100755 --- a/tests/e2e/951_an_ok_image_needs_no_declaration.sh +++ b/tests/e2e/951_an_ok_image_needs_no_declaration.sh @@ -26,6 +26,8 @@ # (reported `graph`); a version of the family the platform's group does not # name is refused, naming the group and both remedies. # G. `--platform none` selects no platform for the target. +# H. After `mcpp toolchain install --target`, a project for the target +# builds and runs with `--offline`. set -e TMP=$(mktemp -d) @@ -135,4 +137,15 @@ assert d["platform"] is None, d["platform"] PY echo "ok: --platform none selects no platform" +# H +mkdir -p "$TMP/off/src" +cd "$TMP/off" +printf '[package]\nname = "off"\nversion = "0.1.0"\n' > mcpp.toml +printf 'import std;\nint main() { std::println("offline image"); }\n' > src/main.cpp +"$MCPP" toolchain install --target "$T" > install.log 2>&1 || fail "H: toolchain install --target" install.log +grep -q "Installed openkal-platform" install.log || fail "H: the install does not name the platform" install.log +"$MCPP" run --offline --target "$T" > off.log 2>&1 || fail "H: run --offline after the install" off.log +grep -q "offline image" off.log || fail "H: the offline run's output" off.log +echo "ok: toolchain install --target prepares the target for --offline" + echo "PASS: 951 an ok image needs no declaration" From ef650ca735da0b25ed4cd4bbd4e15a9cbeed51fd Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 08:03:53 +0900 Subject: [PATCH 34/54] pack: a Linux runner or carrier is built over the platform's hosted arrangement where it serves the triple (static on every architecture, no cross GCC); a row this mcpp builds no program for is said and left out; e2e: 942 and 951 pack one leg, 951's group version is the platform's Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/pack.cppm | 83 ++++++++++++++++--- ...ok_image_is_built_once_and_run_by_okrun.sh | 4 +- .../951_an_ok_image_needs_no_declaration.sh | 4 +- ...versal_package_holds_every_architecture.sh | 14 ++-- 4 files changed, 83 insertions(+), 22 deletions(-) diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index c7894564f..f120ed25e 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -2212,9 +2212,34 @@ constexpr std::string_view host_system() { // macOS or Windows takes this machine's own row; another system's is its // conventional row. `single`: the program is the whole package (a single // file), so on Windows it takes its C++ run-time from the system. -struct ProgramRow { std::string triple; std::string mode; }; -ProgramRow program_row(std::string_view system, std::string_view arch, bool single) { - if (system == "linux") return { std::format("{}-linux-musl", arch), "static" }; +struct ProgramRow { std::string triple; std::string mode; std::string platform; }; + +// The platform a generated program of `triple` is built over, as `--platform` +// names it: the target's platform where its hosted arrangement serves the +// triple (SPEC-013 §5.4), else none. A Linux program over the hosted +// arrangement is static on every architecture, its C library and C++ run-time +// built by the graph with the platform's toolchain; the musl row's own payload +// is a cross GCC published for an x86_64 host only. +std::string hosted_platform_for(const mcpp::platform::Resolved* P, std::string_view triple) { + if (!P || !P->active || !P->fields.hosted.set) return {}; + auto t = mcpp::toolchain::triple::parse(triple); + if (!t) return {}; + auto const& targets = P->fields.hosted.targets; + const bool listed = !targets.set || std::ranges::any_of(targets.value, [&](std::string const& x) { + auto h = mcpp::toolchain::triple::parse(x); + return h && h->str() == t->str(); + }); + if (!listed) return {}; + if (!P->selection.ref.spec.path.empty()) return "path:" + P->root.generic_string(); + return std::format("{}@{}", P->package, P->version); +} + +ProgramRow program_row(std::string_view system, std::string_view arch, bool single, + const mcpp::platform::Resolved* P = nullptr) { + if (system == "linux") { + const auto triple = std::format("{}-linux-musl", arch); + return { triple, "static", hosted_platform_for(P, triple) }; + } const bool here = system == host_system() && arch == mcpp::toolchain::platform::host_arch; if (system == "macos") return { here ? std::string() : std::format("{}-macos", arch), "vendored" }; return { here ? std::string() : std::format("{}-windows-gnu", arch), single ? "system" : "vendored" }; @@ -2261,7 +2286,8 @@ pack_generated_program(const std::filesystem::path& dir, const ProgramRow& row) const auto out = dir / "target" / "dist" / "packed"; fs::remove_all(out, ec); std::vector argv{ mcpp::invocation::self_exe_path().string(), "pack", - "--pkg-format", "dir", "-o", "packed", "--platform", "none", + "--pkg-format", "dir", "-o", "packed", + "--platform", row.platform.empty() ? std::string("none") : row.platform, "--mode", row.mode }; if (!row.triple.empty()) argv.insert(argv.end(), { "--target", row.triple }); int spawnError = 0; @@ -2278,13 +2304,32 @@ pack_generated_program(const std::filesystem::path& dir, const ProgramRow& row) // THE STATIC ROW'S TOOLCHAIN, STATED: the carrier's own graph supplies a // kernel interface, which would otherwise cancel the row's convention and // leave a compiler without a musl C runtime to link with (L10, measured: clang -// took the host GCC's `crtbeginT.o`). -std::string static_row_pin(const ProgramRow& row) { - if (!row.triple.ends_with("-linux-musl")) return {}; - if (auto t = mcpp::toolchain::triple::parse(row.triple)) - if (auto* k = mcpp::toolchain::triple::find_known_target(*t); k && !k->pin.empty()) - return std::format("\n[target.{}]\ntoolchain = \"{}\"\n", row.triple, k->pin); - return {}; +// took the host GCC's `crtbeginT.o`). The row's own pin on this machine's +// architecture; for another architecture's row the platform's toolchain, +// which every host carries (the row's cross GCC is published for an x86_64 +// host only: measured, `x86_64-linux-musl-gcc@16.1.0` has no build for an +// aarch64 Linux host). +std::string static_row_pin(const ProgramRow& row, std::string_view platformToolchain = {}) { + if (!row.triple.ends_with("-linux-musl") || !row.platform.empty()) return {}; + auto t = mcpp::toolchain::triple::parse(row.triple); + if (!t) return {}; + std::string pin; + if (t->arch != mcpp::toolchain::platform::host_arch && !platformToolchain.empty()) + pin = std::string(platformToolchain); + else if (auto* k = mcpp::toolchain::triple::find_known_target(*t); k && !k->pin.empty()) + pin = std::string(k->pin); + if (pin.empty()) return {}; + return std::format("\n[target.{}]\ntoolchain = \"{}\"\n", row.triple, pin); +} + +// Whether this mcpp builds programs for `row`: a target the table states as +// planned is not built (its toolchain is not published). +bool row_is_built(const ProgramRow& row) { + if (row.triple.empty()) return true; // this machine's own row + auto t = mcpp::toolchain::triple::parse(row.triple); + if (!t) return false; + auto* k = mcpp::toolchain::triple::find_known_target(*t); + return !k || k->tier != "planned"; } } // namespace @@ -2316,7 +2361,7 @@ carrier_build(const Plan& plan, const CarrierBuild& b) plan.platform.display(), c.main.empty() ? std::string("(none)") : c.main, mcpp::platform::origin_text(c.origin))}); - const auto row = program_row(b.system, b.arch, b.single); + const auto row = program_row(b.system, b.arch, b.single, &plan.platform); const auto dir = plan.projectRoot / "target" / "carrier-pack" / std::format("{}-{}-{}", b.stem, b.system, b.arch); fs::create_directories(dir / "src", ec); @@ -2343,7 +2388,9 @@ carrier_build(const Plan& plan, const CarrierBuild& b) b.carried.empty() ? "runs the image it is given" : "carries it and runs it", plan.platform.display(), b.stem, plan.packageVersion.empty() ? "0.0.0" : plan.packageVersion, b.stem, - dependency_entry(c.package), static_row_pin(row)); + dependency_entry(c.package), + static_row_pin(row, plan.platform.fields.toolchain.set ? plan.platform.fields.toolchain.value + : std::string())); std::string mainText; { std::ifstream in(c.main, std::ios::binary); @@ -2473,6 +2520,16 @@ run_loaded(const Plan& plan) if (systems.empty()) systems.emplace_back(host_system()); for (auto const& system : systems) for (auto const& leg : legs) { + // A row this mcpp builds no program for (a planned + // target, `x86_64-macos`) carries no runner: said, and + // the package's metadata lists the systems it carries. + if (!row_is_built(program_row(system, leg.arch, false, &plan.platform))) { + mcpp::ui::warning(std::format( + "no runner for {}-{} in the package: this mcpp builds no program for " + "that system and architecture; the launcher there looks for `{}` on PATH", + system, leg.arch, launch.loader)); + continue; + } auto packed = carrier_build(plan, { {}, {}, launch.loader, system, leg.arch, false }); if (!packed) return std::unexpected(packed.error()); const auto into = plan.stagingRoot / "libexec" / launch.loader diff --git a/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh b/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh index 3cd77617d..04ae8ba5b 100755 --- a/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh +++ b/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh @@ -156,9 +156,9 @@ grep -q "Compiling" again.log && fail "E: the second build compiled again" again echo "ok: a second build is up to date" # F -"$MCPP" pack --target "$HOST_ARCH-openkal" --pkg-format dir -o m2 > m2.log 2>&1 || fail "F: pack (M2)" m2.log +"$MCPP" pack --target "$HOST_ARCH-openkal" --no-universal --pkg-format dir -o m2 > m2.log 2>&1 || fail "F: pack (M2)" m2.log [ -f target/dist/m2/bin/app.oke ] || fail "F: the M2 package has no bin/app.oke" m2.log -"$MCPP" pack --target "$HOST_ARCH-openkal" --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 \ +"$MCPP" pack --target "$HOST_ARCH-openkal" --no-universal --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 \ || fail "F: pack --mode self-contained (M3)" m3.log m3=$(ls target/dist/m3/bin/app target/dist/m3/bin/app.exe 2>/dev/null | head -1) [ -n "$m3" ] || fail "F: the M3 package has no program" m3.log diff --git a/tests/e2e/951_an_ok_image_needs_no_declaration.sh b/tests/e2e/951_an_ok_image_needs_no_declaration.sh index da5c6f144..cba6ef2b0 100755 --- a/tests/e2e/951_an_ok_image_needs_no_declaration.sh +++ b/tests/e2e/951_an_ok_image_needs_no_declaration.sh @@ -82,7 +82,7 @@ echo "ok: the lock says which packages the platform added" # D "$MCPP" test --target "$T" > test.log 2>&1 || fail "D: mcpp test" test.log grep -q "smoke ... ok" test.log || fail "D: the test image did not pass" test.log -"$MCPP" pack --target "$T" --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 || fail "D: pack (M3)" m3.log +"$MCPP" pack --target "$T" --no-universal --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 || fail "D: pack (M3)" m3.log m3=$(ls target/dist/m3/bin/app target/dist/m3/bin/app.exe 2>/dev/null | head -1) [ -n "$m3" ] || fail "D: the M3 package has no program" m3.log "$m3" x > m3run.log 2>&1 || fail "D: the M3 program did not run" m3run.log @@ -125,7 +125,7 @@ if "$MCPP" build --target "$T" > group.log 2>&1; then fail "F: a version outside the platform's group was built" group.log fi grep -q "version group" group.log || fail "F: the refusal does not name the group" group.log -grep -q "0.22.3" group.log || fail "F: the refusal does not name the group's version" group.log +grep -qE "names openkal-musl [0-9]+\.[0-9]+\.[0-9]+" group.log || fail "F: the refusal does not name the group's version" group.log echo "ok: a version outside the platform's group is refused" # G diff --git a/tests/e2e/955_a_universal_package_holds_every_architecture.sh b/tests/e2e/955_a_universal_package_holds_every_architecture.sh index 173199769..9ee0554c2 100755 --- a/tests/e2e/955_a_universal_package_holds_every_architecture.sh +++ b/tests/e2e/955_a_universal_package_holds_every_architecture.sh @@ -74,15 +74,19 @@ echo "ok: the launcher runs this machine's image through the runner" # C "$MCPP" pack --target "$T" --mode self-contained --pkg-format dir -o sc > c.log 2>&1 || fail "C: pack --mode self-contained" c.log -for a in x86_64 aarch64; do - [ -f "target/dist/sc/libexec/okrun/$OS-$a/okrun" ] || fail "C: no runner for $OS-$a" c.log -done +# This machine's architecture's runner is always there; the other one is there +# where this mcpp builds programs for it (not x86_64-macos, a planned row). +[ -f "target/dist/sc/libexec/okrun/$OS-$HOST_ARCH/okrun" ] || fail "C: no runner for $OS-$HOST_ARCH" c.log +if [ ! -f "target/dist/sc/libexec/okrun/$OS-$OTHER/okrun" ]; then + grep -q "no runner for $OS-$OTHER" c.log || fail "C: the runner for $OS-$OTHER is missing and not said" c.log +fi env -i HOME="$HOME" PATH=/usr/bin:/bin target/dist/sc/bin/uni x > c-run.log 2>&1 || fail "C: the self-contained launcher" c-run.log grep -q "universal 2 args" c-run.log || fail "C: the self-contained package's output" c-run.log python3 - < Date: Wed, 28 Oct 2026 09:01:44 +0900 Subject: [PATCH 35/54] fixes from CI: a build program's replay ignores its actions' outputs and reads a path to its end; offline, a miss in a namespace a project index serves is not an unfetched index; the unit test's absolute path on Windows; the aarch64 self-host builds two compiles at a time Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .github/workflows/ci-aarch64-fresh-install.yml | 9 +++++++-- src/build/build_program.cppm | 12 ++++++++---- src/build/prepare/graph_load.cpp | 10 ++++++++-- tests/unit/test_platform.cpp | 3 ++- 4 files changed, 25 insertions(+), 9 deletions(-) diff --git a/.github/workflows/ci-aarch64-fresh-install.yml b/.github/workflows/ci-aarch64-fresh-install.yml index 90da06984..9625fc3ad 100644 --- a/.github/workflows/ci-aarch64-fresh-install.yml +++ b/.github/workflows/ci-aarch64-fresh-install.yml @@ -168,7 +168,11 @@ jobs: # leg was written separately and never got it. rm -f .xlings.json mcpp self config --mirror GLOBAL 2>/dev/null || true - mcpp build --target aarch64-linux-musl + # Two compiles at a time: the runner (4 vCPU, 16 GB) received a + # shutdown signal twice at 656/689, with four of the engine's largest + # module units compiling at once with debug information + # (2026-10-28). The build is the same; only its width is not. + MCPP_JOBS=2 mcpp build --target aarch64-linux-musl # Absolute: it is used again after `cd /tmp/xlings-src` below. m=$(find "$PWD/target/aarch64-linux-musl" -type f -path '*/bin/mcpp' | head -1) file "$m" | grep -q "ARM aarch64" || { echo "expected aarch64 mcpp"; exit 1; } @@ -239,7 +243,8 @@ jobs: # selects one that does not exist. Same removal as the self-host step # above, same reason as install_released_mcpp.sh point 1. rm -f .xlings.json - mcpp build --target aarch64-linux-musl + # Two compiles at a time, for the reason of the self-host step above. + MCPP_JOBS=2 mcpp build --target aarch64-linux-musl self=$(find target/aarch64-linux-musl -type f -path '*/bin/mcpp' | head -1) test -x "$self" self=$(realpath "$self") diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 719276126..39aaed896 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -1008,15 +1008,19 @@ std::string cache_stale_reason(const fs::path& root, const fs::path& bdir, const for (auto const& text : spellings) { const auto at = v.find(text); if (at == std::string::npos) continue; - const auto tab = v.find('\t', at); - auto path = v.substr(at, tab == std::string::npos ? std::string::npos : tab - at); + const auto end = v.find_first_of("\t\"", at); + auto path = v.substr(at, end == std::string::npos ? std::string::npos : end - at); if (!fs::exists(path, ec)) return path; } return std::nullopt; }; - for (auto const& slot : c.directives.slots) - for (auto const& v : slot) + // Not an action's: its outputs are the build's products, made again by + // the build whenever they are missing. + for (std::size_t i = 0; i < c.directives.slots.size(); ++i) { + if (static_cast(i) == Slot::Actions) continue; + for (auto const& v : c.directives.slots[i]) if (auto path = vanished(v)) return "output " + *path; + } } return {}; } diff --git a/src/build/prepare/graph_load.cpp b/src/build/prepare/graph_load.cpp index 2b9e5bb5d..13818af10 100644 --- a/src/build/prepare/graph_load.cpp +++ b/src/build/prepare/graph_load.cpp @@ -682,8 +682,14 @@ static void step4a_define_candidate_selection_closures(PrepareState& state) { } // Offline with no local copy of the index at all, the miss says // nothing about the package: nothing has been downloaded to look - // in. That is a download the run needs, not a wrong selector. - if (mcpp::effects::offline_mode()) { + // in. That is a download the run needs, not a wrong selector -- + // unless every candidate's namespace is served by an index of + // the project's own (`[indices]`), which is on disk and answered. + const bool servedByProjectIndex = std::ranges::all_of(candidates, + [&](const mcpp::pm::DependencyCoordinate& c) { + return state.index_route(nullptr).find_for_ns(c.namespace_) != nullptr; + }); + if (mcpp::effects::offline_mode() && !servedByProjectIndex) { if (auto cfgO = state.get_cfg(true); cfgO && !mcpp::provider::open(mcpp::config::make_xlings_env(**cfgO))->default_index_status(0).present) { hint += "\n offline: the package index has never been fetched; " diff --git a/tests/unit/test_platform.cpp b/tests/unit/test_platform.cpp index 75730dc9d..f098d0b54 100644 --- a/tests/unit/test_platform.cpp +++ b/tests/unit/test_platform.cpp @@ -287,7 +287,8 @@ TEST(PlatformSelect, TheHostGraph) { TEST(PlatformSelect, ADirectoryIsAPath) { auto s = pf::parse_choice("path:/opt/board", {}); ASSERT_TRUE(s); - EXPECT_EQ(s->ref.spec.path, "/opt/board"); + // Absolute on this machine: `D:/opt/board` on a Windows one. + EXPECT_TRUE(s->ref.spec.path.ends_with("/opt/board")) << s->ref.spec.path; s = pf::parse_choice("../board", {}); ASSERT_TRUE(s); EXPECT_FALSE(s->ref.spec.path.empty()); From 751865e13718dacdaf33b73165d24c204fceacbe Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 09:10:15 +0900 Subject: [PATCH 36/54] pack: a runner row whose triple the target table does not state (aarch64-windows-gnu) is said and left out, as a planned one is Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/pack.cppm | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index f120ed25e..096a451ca 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -2322,14 +2322,15 @@ std::string static_row_pin(const ProgramRow& row, std::string_view platformToolc return std::format("\n[target.{}]\ntoolchain = \"{}\"\n", row.triple, pin); } -// Whether this mcpp builds programs for `row`: a target the table states as -// planned is not built (its toolchain is not published). +// Whether this mcpp builds programs for `row`: a target its table does not +// state (`aarch64-windows-gnu`), or states as planned (`x86_64-macos`, its +// toolchain not published), is not built. bool row_is_built(const ProgramRow& row) { if (row.triple.empty()) return true; // this machine's own row auto t = mcpp::toolchain::triple::parse(row.triple); if (!t) return false; auto* k = mcpp::toolchain::triple::find_known_target(*t); - return !k || k->tier != "planned"; + return k && k->tier != "planned"; } } // namespace From 035675e4221731d5e22090e485d5c14187ae07f7 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 09:41:59 +0900 Subject: [PATCH 37/54] platform: the ok-image rows select openkal-platform 0.17.2 (universal packages, the okb format, the carrier as a runner; the 0.17.2 round); docs and e2e 953 follow Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/04-mcpp-toml.md | 2 +- docs/10-pack-and-release.md | 2 +- docs/24-openkal-cross.md | 2 +- docs/35-platforms.md | 44 ++++++++++--------- docs/50-machine-output.md | 2 +- docs/specs/platforms.md | 6 +-- docs/zh/04-mcpp-toml.md | 2 +- docs/zh/10-pack-and-release.md | 2 +- docs/zh/24-openkal-cross.md | 2 +- docs/zh/35-platforms.md | 44 ++++++++++--------- docs/zh/50-machine-output.md | 2 +- modules/toolchain-model/src/triple.cppm | 4 +- ...ost_graph_runs_build_programs_as_images.sh | 2 +- 13 files changed, 60 insertions(+), 56 deletions(-) diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 6eef022ce..53113efe4 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1496,7 +1496,7 @@ the platform's own key names ([35 — Platforms](35-platforms.md)): ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.17.1" # or { path = "…" }, or "none" +platform = "openkal-platform@0.17.2" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 24f0068f2..38d88dc80 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -855,7 +855,7 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it ([SPEC-014](specs/packaging.md) §4.5): ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.1", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.2", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index cd13d4800..a6462cb7b 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -111,7 +111,7 @@ practice. ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.1" +platform = "openkal-platform@0.17.2" ``` One table (2026.10.27.1+). It selects the openkal platform for the host triples, diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 0b5c4714f..d18dffaa9 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -40,13 +40,13 @@ mcpp run --target x86_64-openkal ``` Resolved llvm@23.1.3 → x86_64-openkal → …/xim-x-llvm/23.1.3/bin/clang++ - the platform's (openkal-platform 0.17.1), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` + the platform's (openkal-platform 0.17.2), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` Target x86_64-openkal → x86_64-unknown-none-elf - compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.3, platform) - c-abi musl (openkal-musl@0.22.3, platform) - c++-abi libc++ (openkal-llvm-runtime@0.16.3, platform) - Platform openkal-platform 0.17.1 (built-in) - tools okld, okrun (openkal-tools@0.1.1, platform) + compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.5, platform) + c-abi musl (openkal-musl@0.22.5, platform) + c++-abi libc++ (openkal-llvm-runtime@0.16.5, platform) + Platform openkal-platform 0.17.2 (built-in) + tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) Compiling app v0.1.0 (.) @@ -109,9 +109,9 @@ The first of these that answers selects the target's platform: | Source | Spelling | |---|---| -| the command line | `--platform openkal-platform@0.17.1`, `--platform path:../openkal-platform`, `--platform none` | +| the command line | `--platform openkal-platform@0.17.2`, `--platform path:../openkal-platform`, `--platform none` | | the environment | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| the project | `[target.] platform = "openkal-platform@0.17.1"`, or `{ path = "…" }` | +| the project | `[target.] platform = "openkal-platform@0.17.2"`, or `{ path = "…" }` | | this machine | the same key in `~/.mcpp/config.toml` | | the target table | the platform column (`mcpp toolchain list`) | @@ -125,11 +125,11 @@ its own conventions. `[host] platform` selects a platform for them instead: ```toml [host] -platform = "openkal-platform@0.17.1" +platform = "openkal-platform@0.17.2" ``` ``` - Host platform openkal-platform 0.17.1 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun + Host platform openkal-platform 0.17.2 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun ``` Every build program then compiles as an ok image of this machine's @@ -147,7 +147,7 @@ implementation: ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.1" +platform = "openkal-platform@0.17.2" ``` On Windows the arrangement is the GNU ABI's row (`--target x86_64-windows-gnu`); @@ -161,12 +161,12 @@ packages pin one another exactly. A graph that holds another version of one of them is refused before anything compiles: ``` -error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.1 is a version group that names openkal-musl 0.22.3. +error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.2 is a version group that names openkal-musl 0.22.5. requested by: own The family's packages pin one another exactly, so two versions of one of them cannot be linked into one program. Either select the platform release of that version ([target.] platform = "openkal-platform@"), or write - openkal-musl = "0.22.3" where it is requested. + openkal-musl = "0.22.5" where it is requested. ``` A package taken from a directory or a repository is not bound by the group. @@ -180,23 +180,25 @@ abridged: [package] namespace = "mcpplibs" name = "openkal-platform" -version = "0.17.1" +version = "0.17.2" mcpp = ">=2026.10.27.1" [platform] targets = ["x86_64-openkal", "aarch64-openkal"] toolchain = "llvm@23.1.3" runner = { cmd = ["okrun", "{}"], host = "same-arch" } -carrier = { package = "openkal-run@0.1.1", section = ".oke", main = "carrier/main.cpp" } +carrier = { package = "openkal-run@0.1.3", section = ".oke", main = "carrier/main.cpp" } +universal = true +plugin = "build.mcpp" [platform.layers] -compiler-runtime = "openkal-llvm-runtime@0.16.3" -"c++-abi" = "openkal-llvm-runtime@0.16.3" -c-abi = { name = "openkal-musl", version = "0.22.3", default-features = false, features = ["posix"] } +compiler-runtime = "openkal-llvm-runtime@0.16.5" +"c++-abi" = "openkal-llvm-runtime@0.16.5" +c-abi = { name = "openkal-musl", version = "0.22.5", default-features = false, features = ["posix"] } [platform.tools] -okld = "openkal-tools@0.1.1" -okrun = "openkal-tools@0.1.1" +okld = "openkal-tools@0.1.3" +okrun = "openkal-tools@0.1.3" [platform.build] cxxflags = ["-fPIC", "-femulated-tls", "-fvisibility=hidden", "-fasynchronous-unwind-tables"] @@ -207,7 +209,7 @@ cxxflags = ["-ffixed-x18", "-mno-outline-atomics"] [platform.link] driver = "lld" start = "ok_main" -flags = ["-shared", "-z", "max-page-size=65536", "-z", "now", "--gc-sections", "--eh-frame-hdr"] +flags = ["-shared", "-z", "max-page-size=65536", "-z", "now", "--gc-sections", "--icf=safe", "--eh-frame-hdr"] inputs = [{ from = ["okld", "note-source"], as = "ok-notes.S" }] exe = "{name}.oke" shared = "lib{name}.okso" diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index e32284bae..97c9ede5c 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -486,7 +486,7 @@ installation, which is located on the machine rather than installed by mcpp. A target row is `{target, note, toolchain, pin, status, default}`, and `status` is one of `installed` / `available` / `via dependency graph` / `planned`. A target row also carries `platform` (2026.10.27.1+): the platform package the -target table selects for it (`mcpplibs.openkal-platform@0.17.1`), or `null`. +target table selects for it (`mcpplibs.openkal-platform@0.17.2`), or `null`. **`toolchain` and `pin` are not the same field twice.** `toolchain` is what the row is associated with — the installed payload on an installed row, the diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index b44dc5a76..88d25efb3 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -377,8 +377,8 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc 目标报告中,平台加入的层的来源写作 `platform`,图已提供的写作 `graph`。层之后为平台块: ``` - Platform openkal-platform 0.17.1 (built-in) - tools okld, okrun (openkal-tools@0.1.1, platform) + Platform openkal-platform 0.17.2 (built-in) + tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) ``` @@ -394,7 +394,7 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc ```json {"name": "mcpplibs.openkal-platform", "version": "0.17.1", "source": "built-in", "hosted": false, "fields": {"toolchain": {"value": "llvm@23.1.3", "source": "platform"}, - "layers.c-abi": {"value": "openkal-musl@0.22.3", "source": "platform"}}} + "layers.c-abi": {"value": "openkal-musl@0.22.5", "source": "platform"}}} ``` `fields` 的键为 `toolchain`、`runner`、`runners.`、`layers.`、`tools.`(值为所用程序的路径)、 diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 119d4679f..f04a91df5 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1380,7 +1380,7 @@ runner 与 `run_under`。 ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.17.1" # or { path = "…" }, or "none" +platform = "openkal-platform@0.17.2" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index c93bbfbe1..b9a3a547a 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -729,7 +729,7 @@ ok 映像的文件为 `target/dist/---`（Windows 上 提供的格式、`--merge` 与 `mcpp emit xpkg` 读取它（[SPEC-014](../specs/packaging.md) §4.5）： ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.1", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.2", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index b14747061..59b352687 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -93,7 +93,7 @@ SDK，或者区分 `linux`/`windows`/`macos`。那些是 C 环境层或平台层 ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.1" +platform = "openkal-platform@0.17.2" ``` 一张表（2026.10.27.1+）。它为宿主三元组选择 openkal 平台，平台的宿主部分命名编译器与目标侧三个层的包； diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index f06eaf4e3..a72facffd 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -32,13 +32,13 @@ mcpp run --target x86_64-openkal ``` Resolved llvm@23.1.3 → x86_64-openkal → …/xim-x-llvm/23.1.3/bin/clang++ - the platform's (openkal-platform 0.17.1), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` + the platform's (openkal-platform 0.17.2), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` Target x86_64-openkal → x86_64-unknown-none-elf - compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.3, platform) - c-abi musl (openkal-musl@0.22.3, platform) - c++-abi libc++ (openkal-llvm-runtime@0.16.3, platform) - Platform openkal-platform 0.17.1 (built-in) - tools okld, okrun (openkal-tools@0.1.1, platform) + compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.5, platform) + c-abi musl (openkal-musl@0.22.5, platform) + c++-abi libc++ (openkal-llvm-runtime@0.16.5, platform) + Platform openkal-platform 0.17.2 (built-in) + tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) Compiling app v0.1.0 (.) @@ -96,9 +96,9 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 | 来源 | 写法 | |---|---| -| 命令行 | `--platform openkal-platform@0.17.1`、`--platform path:../openkal-platform`、`--platform none` | +| 命令行 | `--platform openkal-platform@0.17.2`、`--platform path:../openkal-platform`、`--platform none` | | 环境变量 | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| 工程 | `[target.] platform = "openkal-platform@0.17.1"`,或 `{ path = "…" }` | +| 工程 | `[target.] platform = "openkal-platform@0.17.2"`,或 `{ path = "…" }` | | 本机 | `~/.mcpp/config.toml` 中的同一个键 | | 目标表 | 平台列(`mcpp toolchain list`) | @@ -110,11 +110,11 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 ```toml [host] -platform = "openkal-platform@0.17.1" +platform = "openkal-platform@0.17.2" ``` ``` - Host platform openkal-platform 0.17.1 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun + Host platform openkal-platform 0.17.2 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun ``` 此后每个构建程序都编译为本机架构的 ok 映像,在 Linux、macOS 与 Windows 上使用同一个工具链,经平台的运行器 @@ -128,7 +128,7 @@ platform = "openkal-platform@0.17.1" ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.1" +platform = "openkal-platform@0.17.2" ``` Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[platform.hosted] targets` 之外的 @@ -140,12 +140,12 @@ Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[pl 另一个版本时,在任何编译之前拒绝: ``` -error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.1 is a version group that names openkal-musl 0.22.3. +error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.2 is a version group that names openkal-musl 0.22.5. requested by: own The family's packages pin one another exactly, so two versions of one of them cannot be linked into one program. Either select the platform release of that version ([target.] platform = "openkal-platform@"), or write - openkal-musl = "0.22.3" where it is requested. + openkal-musl = "0.22.5" where it is requested. ``` 从目录或仓库取得的包不受版本组约束。 @@ -158,23 +158,25 @@ error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0 [package] namespace = "mcpplibs" name = "openkal-platform" -version = "0.17.1" +version = "0.17.2" mcpp = ">=2026.10.27.1" [platform] targets = ["x86_64-openkal", "aarch64-openkal"] toolchain = "llvm@23.1.3" runner = { cmd = ["okrun", "{}"], host = "same-arch" } -carrier = { package = "openkal-run@0.1.1", section = ".oke", main = "carrier/main.cpp" } +carrier = { package = "openkal-run@0.1.3", section = ".oke", main = "carrier/main.cpp" } +universal = true +plugin = "build.mcpp" [platform.layers] -compiler-runtime = "openkal-llvm-runtime@0.16.3" -"c++-abi" = "openkal-llvm-runtime@0.16.3" -c-abi = { name = "openkal-musl", version = "0.22.3", default-features = false, features = ["posix"] } +compiler-runtime = "openkal-llvm-runtime@0.16.5" +"c++-abi" = "openkal-llvm-runtime@0.16.5" +c-abi = { name = "openkal-musl", version = "0.22.5", default-features = false, features = ["posix"] } [platform.tools] -okld = "openkal-tools@0.1.1" -okrun = "openkal-tools@0.1.1" +okld = "openkal-tools@0.1.3" +okrun = "openkal-tools@0.1.3" [platform.build] cxxflags = ["-fPIC", "-femulated-tls", "-fvisibility=hidden", "-fasynchronous-unwind-tables"] @@ -185,7 +187,7 @@ cxxflags = ["-ffixed-x18", "-mno-outline-atomics"] [platform.link] driver = "lld" start = "ok_main" -flags = ["-shared", "-z", "max-page-size=65536", "-z", "now", "--gc-sections", "--eh-frame-hdr"] +flags = ["-shared", "-z", "max-page-size=65536", "-z", "now", "--gc-sections", "--icf=safe", "--eh-frame-hdr"] inputs = [{ from = ["okld", "note-source"], as = "ok-notes.S" }] exe = "{name}.oke" shared = "lib{name}.okso" diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 57853a495..6160a7a5f 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -433,7 +433,7 @@ mcpp toolchain list --format json 的形状是 `{target, note, toolchain, pin, status, default}`，`status` 取 `installed` / `available` / `via dependency graph` / `planned` 之一。 目标行另有 `platform`（2026.10.27.1+）：目标表为该目标选择的平台包 -（`mcpplibs.openkal-platform@0.17.1`），或 `null`。 +（`mcpplibs.openkal-platform@0.17.2`），或 `null`。 **`toolchain` 与 `pin` 不是把同一个字段写了两遍。** `toolchain` 表示这一行 关联到什么 —— 已装的行是装好的载荷，词表行是它自己的约定。`pin` 只承载目标 diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index e4771b3a6..6da65beb7 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -906,9 +906,9 @@ inline constexpr TargetInfo kKnownTargets[] = { // `openkal-platform` (SPEC-013), pinned here. The two architectures the // loader relocates. { "x86_64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.17.1" }, + "mcpplibs.openkal-platform@0.17.2" }, { "aarch64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.17.1" }, + "mcpplibs.openkal-platform@0.17.2" }, }; inline std::span known_targets() { return kKnownTargets; } diff --git a/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh b/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh index 45f7b874c..8ed6dc437 100755 --- a/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh +++ b/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh @@ -34,7 +34,7 @@ name = "hostgraph" version = "0.1.0" [host] -platform = "openkal-platform@0.17.1" +platform = "openkal-platform@0.17.2" [build-dependencies] helper = { path = "helper" } From d6fa98ecebb89bbde3183ac4344d053d12231b9e Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 10:47:45 +0900 Subject: [PATCH 38/54] review: platform manifests refuse what they cannot honour and warn on what they do not know; replays check only what reaches a build; a member's program states platform fields; a kernel-abi package on a hosted target leaves the row's toolchain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - [platform.link] shared names {name} exactly once and no directory; plugin is "build.mcpp"; a git reference names its package; unknown keys of [platform]'s tables, [host] and references are warnings (SPEC-013 §18.1) - the build program replay check reads only directives that reach a compile, a link or a run - the version group asks the edges which packages are build-time only (package_instances) - a workspace member's program states platform fields as a dependency's does (merge_program_platform) - workspace override layering stated in the spec; [host] is not a workspace table - --platform / --host-platform reported as flag or environment; self env --host reports the host graph's triple and compiler; a configuration's target is checked before it names a directory - platform-step passes a step's standard error on and, when it refuses, its output (okld check's reason) - a kernel-abi package supplies the system only where the target has none (freestanding or loader-supplied); on linux-aarch64 `--linkage static` compiled the musl row with llvm against glibc's headers - -o with several --systems refused before the build - three-level loops flattened (member layer edges, link inputs, programs.json runner) - e2e: 952 F/G (a flag with $, reseal once), 954 H (sign), 956 E, 957 D (archive merge), 292 half three; unit: refusals and warnings - docs: packaging spec (mode static, single-file none, runner deps), CHANGELOG Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 11 ++ docs/specs/packaging.md | 7 +- docs/specs/platforms.md | 11 +- modules/buildmcpp/src/directives.cppm | 15 +- modules/manifest/src/platform_toml.cppm | 149 +++++++++++------- modules/toolchain-model/src/triple.cppm | 5 + src/build/build_program.cppm | 23 ++- src/build/flags.cppm | 5 +- src/build/ninja_backend.cppm | 17 +- src/build/plan.cppm | 8 +- src/build/prepare.cppm | 7 + src/build/prepare/configuration.cpp | 7 +- src/build/prepare/features.cpp | 38 +++-- src/build/prepare/host_graph.cpp | 2 +- src/build/prepare/platform.cpp | 86 ++++++---- src/build/prepare/state.cppm | 12 ++ src/build/prepare/target_side.cpp | 4 + src/build/prepare/toolchain_decision.cpp | 32 +++- src/cli.cppm | 3 +- src/cli/cmd_build.cppm | 68 +++++--- src/cli/cmd_publish.cppm | 72 ++++++++- src/pack/pack.cppm | 53 +++++-- src/pack/pipeline.cppm | 36 ++++- src/platform/platform.cppm | 3 +- src/project.cppm | 9 +- ...ayer_does_not_lose_the_targets_compiler.sh | 30 ++++ ...tform_package_states_a_class_of_targets.sh | 26 ++- tests/e2e/954_pack_has_three_axes.sh | 23 +++ ...56_a_single_file_runs_without_unpacking.sh | 10 ++ ..._packages_merge_and_describe_themselves.sh | 19 +++ tests/unit/test_platform.cpp | 44 +++++- 31 files changed, 642 insertions(+), 193 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 69891fa01..406aad5fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -107,6 +107,9 @@ index, its revision and age, and whether it was refreshed now and why not; the provider's own report is printed under `-v`. - A package shared by several configurations of one command is announced once. +- **`--pkg-format tar` writes `.tar.gz` on every host, Windows included**, with + mcpp's own zlib; no `tar` or `gzip` program is run. `--compress none` writes + `.tar`. - **A version the local index lacks is refreshed for when its install is @@ -127,6 +130,14 @@ undefined, and CPython could not load an extension built on `mcpplibs.os` (`undefined symbol: kal_process_close`). A `kernel-abi` provider is now a boundary only when a C library of the graph sits on it. e2e 950. +- **A kernel-abi package over the system's C library leaves a hosted row's + toolchain in place.** The musl and windows-gnu rows name the payload that + supplies the target's C library, and a graph that supplies a C library + replaces it; a kernel-abi package counted as one. On linux-aarch64 a project + reaching openkal-linux through `mcpplibs.os` then compiled `--linkage + static` (the musl row) with the machine's llvm against glibc's headers. A + kernel-abi package now supplies the system only on a target that has none of + its own (a freestanding or a loader-supplied row). e2e 292. - **A package whose descriptor was read is the only one asked for.** A refused install of a namespace-omitted dependency was tried again as `compat:` and `compat.`, shown as downloading, and reported with diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index ed3e6017b..f5a457dff 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -199,7 +199,7 @@ SDK 载荷时该次构建拒绝,写出该系统与载荷。 | `name`、`version` | 包的名字与版本 | | `platform` | 平台包与版本(`<包>@<版本>`);无平台时为 `null` | | `linkage` | `static`、`shared`,或 `declared`(每个库按自己的声明) | -| `mode` | §1.2 的模式 | +| `mode` | §1.2 的模式:`system`、`vendored`、`self-contained`;`--mode static` 的原生包为 `static`,映像为 `self-contained` | | `legs` | 腿的规范三元组 | | `systems` | `libexec//` 中加载器的 `-`;无加载器时为空 | | `launch` | 通用包为 `runner`(平台的 argv)与每个架构的 `entry`;单腿原生包为 `program`(程序的相对路径) | @@ -226,7 +226,7 @@ SDK 载荷时该次构建拒绝,写出该系统与载荷。 `--compress [:]` 与 `[pack] compress` 选择 `tar` 的压缩:`tar` 写出未压缩的归档,mcpp 以 自带的编解码器(zlib、zstd、liblzma)压缩,产物扩展名为 `.tar.gz`(缺省)、`.tar.zst`、`.tar.xz` 或 `.tar`。级别 范围:gzip 1–9,zstd 1–22,xz 0–9。`zip` 的条目在变小时以 DEFLATE 压缩(`gzip[:]` 设定级别),否则存储; -`none` 时全部存储。单文件的载荷为 zstd 或不压缩。okb 中的映像不压缩(SPEC-013 平台插件的格式规定)。同一棵树的 +`none` 时全部存储。单文件的载荷总是一个 zstd 帧(启动器只解 zstd),`none` 取级别 1。okb 中的映像不压缩(SPEC-013 平台插件的格式规定)。同一棵树的 归档在每个宿主上字节相同:gzip 成员不带名字与时间,zip 条目使用固定日期。 ### 5.2 单文件 @@ -282,7 +282,8 @@ openkal 目标的单文件:载体携带映像本身(映像不引用其他映像 生成 xim-pkgindex 条目(spec 2),资产地址为 `<前缀>/<文件名>`: - 通用包是一个资产,`linux`、`macosx`、`windows` 三项的 URL 与 sha256 相同;包不携带运行器(非 `self-contained`) - 时三项各以运行器为依赖(`deps = { "" }`)。 + 时三项各以运行器为依赖(`deps = { "" }`,`` 为平台运行器 argv 首项的程序名,即 xim-pkgindex + 中同名的包;openkal 为 `okrun`)。 - 各系统各架构一个的包:每个系统一项,一个以 `${arch}` 代替架构的 URL 与每个架构的 sha256;文件名除架构外不同时 拒绝。 - 安装把归档的唯一顶层目录移到安装目录;配置以 xvm 注册程序(通用包为 `bin/`,Windows 上为 diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 88d25efb3..5f2ca1547 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -67,7 +67,7 @@ plugin = "build.mcpp" | `runner` | 字符串数组或表 | §10 | | `runners` | ` = [argv]` | 命名的设备动作,每个 argv 非空 | | `carrier` | 表 | §11;`package`、`section`、`main` 三键都**必须**给出 | -| `plugin` | 字符串 | 平台包中的构建程序,在平台阶段运行(§12);路径相对于平台包 | +| `plugin` | 字符串 | 只能为 `"build.mcpp"`:平台包自身的构建程序(SPEC-007)在平台阶段运行(§12);其他值拒绝 | | `universal` | 布尔 | `true` 时本平台任一目标的打包缺省为覆盖全部 `targets` 的通用包(SPEC-014 §1.4、§4.2) | | `layers` | 表 | §7.2;键只能是 §0 的四个层名 | | `tools` | 表 | §7.3 | @@ -79,8 +79,8 @@ plugin = "build.mcpp" **包引用。** `layers`、`tools`、`carrier.package` 与选择平台的值都是包引用:字符串 `"[.]@"`,或表。表的键为 `name`、`version`、`path`、`git` 与 `rev` / `tag` / `branch` 之一、`features`、`default-features`;**必须**恰好给出 `version`、`path`、 -`git` 之一;`version` 与 `name` 同时出现;`path` 相对于写它的文件;`path` 或 `git` 的引用**可以** -省略 `name`,此时由该包自己的清单命名。`false` 只在 `layers` 中有效;`{ program = "" }` 只在 +`git` 之一;`version` 与 `name` 同时出现;`path` 相对于写它的文件;`git` 的引用同时给出 `name`;`path` 的引用**可以** +省略 `name`,此时由该目录中包自己的清单命名。`false` 只在 `layers` 中有效;`{ program = "" }` 只在 `tools` 中有效,且不带其他键。**已实现** **argv 约定。** argv 中的 `{}` 为产物;程序以名字书写,按"图中为本机构建的程序 → 声明的载荷 → @@ -349,7 +349,8 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc ## 13. 工作空间 `[target.]` 的平台键与 `[host]` 是根位置的键,从计划的根读取。工作空间层的 -`[workspace.target.]` 先于成员自己的表合并;`[host]` 不是工作空间层的表。虚拟根携带第一个被选中 +`[workspace.target.]` 先于成员自己的表合并:同一选择器的两张表,成员的字段胜出;不同选择器之间 +按具体度,与 `[target.]` 其他键的合并相同。`[host]` 不是工作空间层的表。虚拟根携带第一个被选中 成员的平台键与 `[host]`;根位置的比较键包含适用的平台表与 `[host]`,使二者不同的成员不共用一个计划。 **已实现** @@ -431,7 +432,7 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc ## 18. 判据 -1. `[platform]` 的每个键的类型错误在读清单时拒绝,消息写出键与文件;未知键为警告。 +1. `[platform]` 及其子表、`[host]` 与包引用中每个键的类型错误在读清单时拒绝,消息写出键与文件;未知键为警告并被忽略。 2. 选择按 §3.1、§4.1 的顺序;`none` 不使用平台;目录形式读取目录中的清单。 3. `cfg(platform = ...)` 下写 `platform` 被拒绝。 4. 图已提供某层时平台不加入该层的包,报告为 `graph`;`false` 不加入。 diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index da5257262..b4141a79a 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -659,7 +659,8 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d); // `link-input`, `artifact-name`, `seal`, `run-host`), as a platform statement // at the build-program tier (SPEC-013 §4.1). `apply` calls it; the platform // phase reads it for the platform's own program. -void apply_platform(mcpp::manifest::platform::Statement& s, const Directives& d); +void apply_platform(mcpp::manifest::platform::Statement& s, const Directives& d, + const std::filesystem::path& root = {}); // Decode one `mcpp:action=` JSON payload. nullopt = malformed. std::optional decode_action(std::string_view payload); @@ -1215,7 +1216,7 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { // The platform's fields (protocol 17), at the build-program tier; // `platform_directive_error` has refused a malformed line. - apply_platform(m.buildConfig.programPlatform, d); + apply_platform(m.buildConfig.programPlatform, d, m.sourcePath.parent_path()); } // The five roles `mcpp::action::role` may name, in the order @@ -1367,7 +1368,8 @@ static std::string named_target_error(const mcpp::manifest::Manifest& m, return {}; } -void apply_platform(mcpp::manifest::platform::Statement& s, const Directives& d) { +void apply_platform(mcpp::manifest::platform::Statement& s, const Directives& d, + const std::filesystem::path& root) { namespace mp = mcpp::manifest::platform; auto origin = [](std::string_view wire) { return mp::Origin{ mp::Tier::Program, {}, 0, std::format("mcpp:{}", wire) }; @@ -1378,7 +1380,9 @@ void apply_platform(mcpp::manifest::platform::Statement& s, const Directives& d) std::vector inputs; for (auto const& v : d.at(Slot::LinkInputs)) { if (v.starts_with("file:")) { - inputs.push_back({ .file = v.substr(5) }); + // A relative file is the stating package's own, as in the + // TOML form (`[platform.link] inputs`). + inputs.push_back({ .file = root.empty() ? v.substr(5) : abs_against(root, v.substr(5)) }); continue; } const auto colon = v.find(':', 5); @@ -1438,6 +1442,9 @@ std::string target_directive_error(const mcpp::manifest::Manifest& m, const Dire if (!file && !from) return std::format("build.mcpp emitted `mcpp:link-input={}`; an input is " "`file:` or `from::`", v); + if (from && v.substr(5, colon - 5).find_first_of("/\\") != std::string::npos) + return std::format("build.mcpp emitted `mcpp:link-input={}`; `` names the file " + "the output is saved as, without a directory", v); } for (auto const& v : d.at(Slot::ArtifactNames)) { const auto colon = v.find(':'); diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm index cf7b371c4..c9618e650 100644 --- a/modules/manifest/src/platform_toml.cppm +++ b/modules/manifest/src/platform_toml.cppm @@ -46,6 +46,10 @@ struct Reader { std::expected link(const mcpp::libs::toml::Table& t, std::string_view where, Link& out) const; + // One entry of `link.inputs`: `{ file = "..." }` or `{ from = [...], as = "..." }`. + std::expected + link_input(const mcpp::libs::toml::Value& e, std::string_view key) const; + std::expected layers(const mcpp::libs::toml::Value& v, std::string_view where, std::map& out) const; @@ -197,8 +201,8 @@ Reader::ref(const toml::Value& v, std::string_view where, bool allowOff, bool al r.spec.defaultFeatures = val.as_bool(); r.spec.defaultFeaturesDeclared = true; } else { - return std::unexpected(std::format( - "{}: '{}' is not a key of a package reference; the keys are name, version, " + warn(std::format( + "{}: '{}' is not a key of a package reference (ignored); the keys are name, version, " "path, git, rev, tag, branch, features, default-features{}", where, k, allowProgram ? ", or program alone" : "")); } @@ -216,6 +220,12 @@ Reader::ref(const toml::Value& v, std::string_view where, bool allowOff, bool al return std::unexpected(std::format( "{}: a version names the package too: {{ name = \"\", version = \"\" }}, " "or \"@\"", where)); + // A repository is not read before its package is fetched, and its URL does + // not say the package's name; a git reference names it. + if (!r.spec.git.empty() && r.name.empty()) + return std::unexpected(std::format( + "{}: a git source names the package too: {{ name = \"\", git = \"\", ... }}", + where)); split_name(r); return r; } @@ -246,8 +256,9 @@ std::expected Reader::runner(const toml::Value& v, std::str r.exclusive = val.as_bool(); r.exclusiveSet = true; } else { - return std::unexpected(std::format( - "{}: '{}' is not a key of a runner; the keys are cmd, host, exclusive", where, k)); + warn(std::format( + "{}: '{}' is not a key of a runner (ignored); the keys are cmd, host, exclusive", + where, k)); } } } else { @@ -280,10 +291,12 @@ Reader::flags(const toml::Table& t, std::string_view where, Flags& out, : k == "asmflags" ? &out.asmflags : k == "ldflags" ? &out.ldflags : nullptr; - if (!dst) - return std::unexpected(std::format( - "{} has no key '{}'; the keys are {}{}", where, k, list_of(kBuildKeys), + if (!dst) { + warn(std::format( + "{} has unsupported key '{}' (ignored); the keys are {}{}", where, k, list_of(kBuildKeys), cfg ? ", and 'cfg(...)' tables of the same keys" : "")); + continue; + } auto s = strings(v, key); if (!s) return std::unexpected(s.error()); dst->insert(dst->end(), s->begin(), s->end()); @@ -291,6 +304,45 @@ Reader::flags(const toml::Table& t, std::string_view where, Flags& out, return {}; } +std::expected +Reader::link_input(const toml::Value& e, std::string_view key) const { + if (!e.is_table()) + return std::unexpected(std::format( + "{} must contain only tables: {{ file = \"...\" }} or " + "{{ from = [...], as = \"...\" }}", key)); + LinkInput in; + for (auto const& [ik, iv] : e.as_table()) { + auto ikey = std::format("{}.{}", key, ik); + if (ik == "file") { + if (!iv.is_string() || iv.as_string().empty()) + return std::unexpected(std::format("{} must be a path", ikey)); + in.file = absolute(iv.as_string()).generic_string(); + } else if (ik == "from") { + auto s = strings(iv, ikey); + if (!s) return std::unexpected(s.error()); + if (s->empty()) + return std::unexpected(std::format("{} is empty; it names a program", ikey)); + in.from = std::move(*s); + } else if (ik == "as") { + if (!iv.is_string() || iv.as_string().empty() + || iv.as_string().find_first_of("/\\") != std::string::npos) + return std::unexpected(std::format("{} must be a file name", ikey)); + in.as = iv.as_string(); + } else { + warn(std::format( + "{}: '{}' is not a key of a link input (ignored); the keys are file, " + "or from and as", key, ik)); + } + } + if (in.file.empty() == in.from.empty()) + return std::unexpected(std::format( + "{}: an input is {{ file = \"...\" }} or {{ from = [...], as = \"...\" }}", key)); + if (!in.from.empty() && in.as.empty()) + return std::unexpected(std::format( + "{}: an input made by a program names the file it is saved as (`as`)", key)); + return in; +} + std::expected Reader::link(const toml::Table& t, std::string_view where, Link& out) const { for (auto const& [k, v] : t) { @@ -298,10 +350,18 @@ Reader::link(const toml::Table& t, std::string_view where, Link& out) const { auto text = [&](Text& dst, bool needsName) -> std::expected { if (!v.is_string() || v.as_string().empty()) return std::unexpected(std::format("{} must be a non-empty string", key)); - if (needsName && v.as_string().find("{name}") == std::string::npos) + // A name pattern holds `{name}` exactly once and names no directory + // (the rule `mcpp::platform::split_name_pattern` applies). + const std::string_view s = v.as_string(); + const auto first = s.find("{name}"); + if (needsName && (first == std::string_view::npos + || s.find("{name}", first + 1) != std::string_view::npos)) + return std::unexpected(std::format( + "{} = \"{}\" names the artifact as {{name}} exactly once; " + "for example \"{{name}}.oke\"", key, s)); + if (needsName && s.find_first_of("/\\") != std::string_view::npos) return std::unexpected(std::format( - "{} = \"{}\" names no artifact; write the pattern with {{name}}, " - "for example \"{{name}}.oke\"", key, v.as_string())); + "{} = \"{}\" is a file name pattern; it names no directory", key, s)); dst = Text{ v.as_string(), at(v, key), true }; return {}; }; @@ -334,41 +394,9 @@ Reader::link(const toml::Table& t, std::string_view where, Link& out) const { "{} must be an array of {{ file = \"...\" }} or {{ from = [...], as = \"...\" }}", key)); out.inputs.clear(); for (auto const& e : v.as_array()) { - if (!e.is_table()) - return std::unexpected(std::format( - "{} must contain only tables: {{ file = \"...\" }} or " - "{{ from = [...], as = \"...\" }}", key)); - LinkInput in; - for (auto const& [ik, iv] : e.as_table()) { - auto ikey = std::format("{}.{}", key, ik); - if (ik == "file") { - if (!iv.is_string() || iv.as_string().empty()) - return std::unexpected(std::format("{} must be a path", ikey)); - in.file = absolute(iv.as_string()).generic_string(); - } else if (ik == "from") { - auto s = strings(iv, ikey); - if (!s) return std::unexpected(s.error()); - if (s->empty()) - return std::unexpected(std::format("{} is empty; it names a program", ikey)); - in.from = std::move(*s); - } else if (ik == "as") { - if (!iv.is_string() || iv.as_string().empty() - || iv.as_string().find_first_of("/\\") != std::string::npos) - return std::unexpected(std::format("{} must be a file name", ikey)); - in.as = iv.as_string(); - } else { - return std::unexpected(std::format( - "{}: '{}' is not a key of a link input; the keys are file, or from and as", - key, ik)); - } - } - if (in.file.empty() == in.from.empty()) - return std::unexpected(std::format( - "{}: an input is {{ file = \"...\" }} or {{ from = [...], as = \"...\" }}", key)); - if (!in.from.empty() && in.as.empty()) - return std::unexpected(std::format( - "{}: an input made by a program names the file it is saved as (`as`)", key)); - out.inputs.push_back(std::move(in)); + auto in = link_input(e, key); + if (!in) return std::unexpected(in.error()); + out.inputs.push_back(std::move(*in)); } out.inputsOrigin = at(v, key); out.inputsSet = true; @@ -388,8 +416,8 @@ Reader::link(const toml::Table& t, std::string_view where, Link& out) const { out.sealOrigin = at(v, key); out.sealSet = true; } else { - return std::unexpected(std::format( - "{} has no key '{}'; the keys are {}", where, k, list_of(kLinkKeys))); + warn(std::format( + "{} has unsupported key '{}' (ignored); the keys are {}", where, k, list_of(kLinkKeys))); } } return {}; @@ -451,8 +479,9 @@ std::expected Reader::carrier(const toml::Value& v, std::s return std::unexpected(std::format("{} must be a source file of the package", key)); c.main = absolute(val.as_string()).generic_string(); } else { - return std::unexpected(std::format( - "{}: '{}' is not a key of a carrier; the keys are package, section, main", where, k)); + warn(std::format( + "{}: '{}' is not a key of a carrier (ignored); the keys are package, section, main", + where, k)); } } if (c.package.empty() || c.section.empty() || c.main.empty()) @@ -486,8 +515,9 @@ std::expected Reader::statement(const toml::Table& t) co auto p = strings(sv, key + ".provides"); if (!p) return std::unexpected(p.error()); s.sysrootProvides = std::move(*p); - } else return std::unexpected(std::format( - "{}: '{}' is not a key of a sysroot table; the keys are path, provides", key, sk)); + } else warn(std::format( + "{}: '{}' is not a key of a sysroot table (ignored); the keys are path, provides", + key, sk)); } if (path.empty()) return std::unexpected(std::format("{} is a table without `path`", key)); @@ -514,9 +544,11 @@ std::expected Reader::statement(const toml::Table& t) co if (!c) return std::unexpected(c.error()); s.carrier = std::move(*c); } else if (k == "plugin") { - if (!v.is_string() || v.as_string().empty()) + // The platform's program is its package's build program, which a + // package names `build.mcpp` (SPEC-007); no other file is run. + if (!v.is_string() || v.as_string() != "build.mcpp") return std::unexpected(std::format( - "{} must name the platform's build program, for example \"build.mcpp\"", key)); + "{} must be \"build.mcpp\": the platform's program is its package's build program", key)); s.plugin = Text{ absolute(v.as_string()).generic_string(), at(v, key), true }; } else if (k == "universal") { if (!v.is_bool()) @@ -561,9 +593,9 @@ std::expected Reader::statement(const toml::Table& t) co if (auto r = layers(hv, "[platform.hosted.layers]", s.hosted.layers); !r) return std::unexpected(r.error()); } else { - return std::unexpected(std::format( - "[platform.hosted] has no key '{}'; a host triple takes the platform's {} only", - hk, list_of(kHostedKeys))); + warn(std::format( + "[platform.hosted] has unsupported key '{}' (ignored); a host triple takes " + "the platform's {} only", hk, list_of(kHostedKeys))); } } } else { @@ -594,8 +626,9 @@ Reader::override_(const toml::Table& t, std::string_view selector) const { o.platformSet = true; } else if (selector.empty()) { // `[host]` selects the host graph's platform and nothing else. - return std::unexpected(std::format( - "[host] has no key '{}'; it holds `platform`, the host graph's platform", k)); + warn(std::format( + "[host] has unsupported key '{}' (ignored); it holds `platform`, the host graph's platform", + k)); } else if (k == "layers") { if (auto r = layers(v, key, o.fields.layers); !r) return std::unexpected(r.error()); } else if (k == "tools") { diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 6da65beb7..65fba50ba 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -1194,6 +1194,11 @@ struct ArtifactNaming { // produced artifact. Shared libraries have never been verified end-to-end // on PE or Mach-O — every shared-library e2e declares `# requires: elf`. bool sharedNeedsImportLib; + // A shared library's own prefix where a platform names it + // (`[platform.link] shared = "lib{name}.okso"`); the static library keeps + // `libPrefix`. + std::string_view sharedLibPrefix = {}; + bool sharedLibPrefixSet = false; }; // Naming for an explicit target triple. An EMPTY triple means "build for this diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 39aaed896..a8b0e068b 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -965,6 +965,14 @@ CacheRecord read_cache(const fs::path& bdir) { return r; } +// The first value `vanished` answers for. +template +std::optional first_vanished(const std::vector& values, F&& vanished) { + for (auto const& v : values) + if (auto path = vanished(v)) return path; + return std::nullopt; +} + // Why the cached run is not valid, for the log; empty when it is. std::string cache_stale_reason(const fs::path& root, const fs::path& bdir, const CacheRecord& c, const std::string& programHash, const std::string& compilerHash, @@ -1014,12 +1022,15 @@ std::string cache_stale_reason(const fs::path& root, const fs::path& bdir, const } return std::nullopt; }; - // Not an action's: its outputs are the build's products, made again by - // the build whenever they are missing. - for (std::size_t i = 0; i < c.directives.slots.size(); ++i) { - if (static_cast(i) == Slot::Actions) continue; - for (auto const& v : c.directives.slots[i]) - if (auto path = vanished(v)) return "output " + *path; + // Only what reaches a compile, a link or a run: a warning or a claim + // that mentions a path is text, and an action's outputs are the + // build's products, made again by the build whenever they are missing. + for (auto const& def : dirs::kTable) { + if (def.scope == dirs::Scope::Advisory || def.scope == dirs::Scope::Claim + || def.scope == dirs::Scope::RerunKey || def.scope == dirs::Scope::GraphNode) + continue; + if (auto path = first_vanished(c.directives.at(def.slot), vanished)) + return "output " + *path; } } return {}; diff --git a/src/build/flags.cppm b/src/build/flags.cppm index fa3153e1b..a910e57b3 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -2498,10 +2498,11 @@ CompileFlags compute_flags(const BuildPlan& plan) { // and no system library of the toolchain's, and what it holds is the // graph's, reached through the graph's own `-L`/`-l`. if (auto const& p = plan.platform; p.active) { + // Each element one word, quoted for the shell and `$` doubled for + // ninja, as every other word of a command is. auto words = [](const std::vector& v) { std::string out; - for (auto const& w : v) - out += " " + (w.find(' ') == std::string::npos ? w : "\"" + w + "\""); + for (auto const& w : v) out += " " + ninja_command_word(w); return out; }; f.cc += words(p.flags.cflags); diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 847cd0748..1f32a5a40 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -1610,15 +1610,18 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, } steps += line + "\n"; } - const auto stepsFile = std::filesystem::path("obj") / "platform" / "seal.steps"; - std::error_code mkec; - std::filesystem::create_directories((plan.outputDir / stepsFile).parent_path(), mkec); - write_file(plan.outputDir / stepsFile, steps); + // IN THE COMMAND, HEX-ENCODED, NOT IN A FILE BESIDE IT: a file under + // `obj/` is gone after `rm -rf obj`, and a build that does not plan + // again (its graph is current) then ran the seal with no steps and + // published the artifact unsealed. The command is the steps, so a + // changed step reseals by the command's own change. + static constexpr char kHex[] = "0123456789abcdef"; + std::string hex; + for (unsigned char ch : steps) { hex += kHex[ch >> 4]; hex += kHex[ch & 15]; } append("seal_with =\n"); append("rule platform_seal\n"); - append(std::format(" command = $mcpp platform-step seal --steps {} --digest {} -- $in $out $seal_with\n", - ninja_command_word(stepsFile.generic_string()), - mcpp::toolchain::hash_string(steps))); + append(std::format(" command = $mcpp platform-step seal --steps-hex {} -- $in $out $seal_with\n", + hex)); append(" description = SEAL $out\n\n"); } diff --git a/src/build/plan.cppm b/src/build/plan.cppm index fbb2ef7ab..3af12d2c7 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -1011,7 +1011,8 @@ std::filesystem::path target_output(const mcpp::manifest::Target& t, if (t.kind == mcpp::manifest::Target::SharedLibrary || (t.kind == mcpp::manifest::Target::Application && asSharedObject)) { return std::filesystem::path("bin") / - std::format("{}{}{}", n.libPrefix, t.name, n.sharedLibExt); + std::format("{}{}{}", n.sharedLibPrefixSet ? n.sharedLibPrefix : n.libPrefix, + t.name, n.sharedLibExt); } return std::filesystem::path("bin") / std::format("{}{}", t.name, n.exeSuffix); @@ -1686,8 +1687,9 @@ make_plan(const mcpp::manifest::Manifest& manifest, if (l.shared.set && mcpp::platform::split_name_pattern(l.shared.value)) { const std::string_view v = l.shared.value; const auto at = v.find(kName); - naming.libPrefix = v.substr(0, at); - naming.sharedLibExt = v.substr(at + kName.size()); + naming.sharedLibPrefix = v.substr(0, at); + naming.sharedLibPrefixSet = true; + naming.sharedLibExt = v.substr(at + kName.size()); } } diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 385feac65..47e68d2af 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -529,6 +529,13 @@ export struct BuildContext { // tools of this build are built for. Inactive = the host conventions. // The target's is `plan.platform`. mcpp::platform::Resolved hostPlatform; + // The host graph's triple and compiler, prepared for `mcpp self env --host` + // only: under a host platform, the platform's target of this machine's + // architecture and the platform's toolchain; else the toolchain the build + // programs are compiled with. + std::string hostTriple; + std::string hostCompiler; + std::string hostCompilerOrigin; // The payload directory of every `[xlings]` address resolved above, for // the fast path's presence check (#716). See BuildCacheEntry::xlingsPayloads. std::vector xlingsPayloads; diff --git a/src/build/prepare/configuration.cpp b/src/build/prepare/configuration.cpp index 9a58af88a..ca23b6f3d 100644 --- a/src/build/prepare/configuration.cpp +++ b/src/build/prepare/configuration.cpp @@ -26,10 +26,15 @@ std::expected prepare_configuration(std::string_view // No project here: an empty program in the home's cache stands for one, // one directory per target, so its lock and its build directory persist // and a second run finds everything on disk. + // The directory is named after the triple, which is checked first: the + // text names a directory under the home. std::string name = "host"; if (!targetTriple.empty()) { auto t = mcpp::toolchain::triple::parse(targetTriple); - name = t ? t->str() : std::string(targetTriple); + if (!t) return std::unexpected(std::format( + "'{}' is not a target triple (-[-], for example x86_64-openkal)", + targetTriple)); + name = t->str(); } const auto dir = mcpp::home::root() / "cache" / "configurations" / name; fs::create_directories(dir / "src", ec); diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index fc9bbadd2..f0eb91442 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -182,10 +182,23 @@ static void step6_compat_library_edges(PrepareState& state) { // Nothing is cleared from a manifest: `compilesHere` reads the result, and the // link flags a package without a target instance contributed during // resolution are withdrawn from the root's link line. -static void step6_instances(PrepareState& state) { +void merge_program_platform(PrepareState& state, mcpp::manifest::platform::Statement stated, + std::string_view who) { + for (auto* o : { &stated.link.driver.origin, &stated.link.inputsOrigin, + &stated.link.exe.origin, &stated.link.shared.origin, + &stated.link.sealOrigin, &stated.runner.origin }) + if (o->tier == mcpp::manifest::platform::Tier::Program) + o->key = std::format("{} ({})", who, o->key); + if (!stated.empty() || !stated.runner.host.empty()) + mcpp::platform::merge(state.programPlatform, stated); +} + +PackageInstances package_instances(const PrepareState& state) { const auto n = state.packages.size(); - if (n == 0) return; - std::vector target(n, false), host(n, false); + PackageInstances out{ std::vector(n, false), std::vector(n, false) }; + if (n == 0) return out; + auto& target = out.target; + auto& host = out.host; std::vector stack{0}; target[0] = true; while (!stack.empty()) { @@ -215,6 +228,13 @@ static void step6_instances(PrepareState& state) { stack.push_back(d); } } + return out; +} + +static void step6_instances(PrepareState& state) { + const auto n = state.packages.size(); + if (n == 0) return; + const auto [target, host] = package_instances(state); for (std::size_t d = 1; d < n; ++d) { state.packages[d].hostInstance = host[d]; // A package no edge mentions is a bookkeeping gap, not a build @@ -1951,22 +1971,14 @@ static std::expected step6_dependency_build_programs(PrepareS // after the package (SPEC-013 §4.1). A plugin's compile flags are // the platform's, for every unit of the target, not its own. { - auto stated = std::move(bcDep.programPlatform); - bcDep.programPlatform = {}; + auto stated = std::exchange(bcDep.programPlatform, {}); if (platformPhase) { stated.build.cxxflags.assign(bcDep.cxxflags.begin() + static_cast(cxxN), bcDep.cxxflags.end()); stated.build.cflags.assign(bcDep.cflags.begin() + static_cast(cN), bcDep.cflags.end()); } - const auto who = pkg.manifest.package.name; - for (auto* o : { &stated.link.driver.origin, &stated.link.inputsOrigin, - &stated.link.exe.origin, &stated.link.shared.origin, - &stated.link.sealOrigin, &stated.runner.origin }) - if (o->tier == mcpp::manifest::platform::Tier::Program) - o->key = std::format("{} ({})", who, o->key); - if (!stated.empty() || !stated.runner.host.empty()) - mcpp::platform::merge(state.programPlatform, stated); + merge_program_platform(state, std::move(stated), pkg.manifest.package.name); } } diff --git a/src/build/prepare/host_graph.cpp b/src/build/prepare/host_graph.cpp index c041e9f65..ff5813f08 100644 --- a/src/build/prepare/host_graph.cpp +++ b/src/build/prepare/host_graph.cpp @@ -1031,7 +1031,7 @@ ensure_build_programs(PrepareState& state, const std::vector& packa place_program_copy(state, p.index, fs::path(it->value("binary", std::string{}))); std::vector runner; if (it->contains("runner") && (*it)["runner"].is_array()) - for (auto const& a : (*it)["runner"]) runner.push_back(a.get()); + runner = (*it)["runner"].get>(); // A program built under the host platform by an earlier run: // its runner as that run located it, for the report. if (!runner.empty() && state.hostPlatformRunner.empty()) { diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 8cc53f6b3..f290411f5 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -46,8 +46,8 @@ mcpp::toolchain::triple::Triple target_of(const PrepareState& state) { } // The package short name a reference names, before the package is fetched: -// the reference's own name, the `[package]` of a directory, or the last -// segment of a repository's URL. +// the reference's own name (a git reference always has one), or the +// `[package]` of a directory. std::string known_short_name(const mp::PackageRef& r) { if (!r.spec.shortName.empty()) return r.spec.shortName; if (!r.spec.path.empty()) { @@ -56,16 +56,17 @@ std::string known_short_name(const mp::PackageRef& r) { .shortName; return {}; } - if (!r.spec.git.empty()) { - std::string_view u = r.spec.git; - while (!u.empty() && u.back() == '/') u.remove_suffix(1); - if (auto s = u.find_last_of("/:"); s != std::string_view::npos) u = u.substr(s + 1); - if (u.ends_with(".git")) u.remove_suffix(4); - return std::string(u); - } return {}; } +// Whether this invocation's command line stated a variable (`--platform` is +// `MCPP_PLATFORM` for the invocation), rather than the environment. +bool stated_on_command_line(std::string_view name) { + return std::ranges::any_of(mcpp::invocation::overrides(), [&](auto const& o) { + return o.first == name && o.second.has_value(); + }); +} + // The project's override tables: the plan root's, which hold the workspace // layer's first (`merge_layer_root_position`). std::vector project_tables(const mcpp::manifest::Manifest& m) { @@ -128,7 +129,8 @@ std::expected step1_select_platform(PrepareState& state) { if (auto cfg = state.get_cfg(false); cfg) config = (*cfg)->hostPlatform; const auto hostFlag = mcpp::invocation::get("MCPP_HOST_PLATFORM").value_or(""); mp::Override project = state.m->hostPlatform; - auto hs = mcpp::platform::select_host(hostFlag, false, project, config); + auto hs = mcpp::platform::select_host(hostFlag, stated_on_command_line("MCPP_HOST_PLATFORM"), + project, config); if (!hs) return std::unexpected(hs.error()); state.hostPlatformSel = std::move(*hs); } @@ -144,6 +146,8 @@ std::expected step1_select_platform(PrepareState& state) { std::string column; if (auto* row = mcpp::toolchain::triple::find_known_target(t)) column = std::string(row->platform); mcpp::platform::ChoiceInputs in{ flag, env, envKey, project, config, column }; + if (state.overrides.platform_choice.empty() && !stated_on_command_line("MCPP_PLATFORM")) + in.flagKey = "MCPP_PLATFORM"; auto sel = mcpp::platform::select_target(in, matcher(cfgpred::context_for(t.str()))); if (!sel) return std::unexpected(sel.error()); state.platform = mcpp::platform::Resolved{}; @@ -391,10 +395,13 @@ std::expected step4b_apply_platform(PrepareState& state) { std::expected step4b_check_platform_versions(PrepareState& state) { auto const& P = state.platform; if (!P.active || P.fields.versions.empty()) return {}; + // Asked of the edges as they stand: the check runs before each round's + // drain, before `buildTimeOnly` is decided. + const auto instances = package_instances(state); for (std::size_t i = 1; i < state.packages.size(); ++i) { - // A package built for this machine (a tool's) is not linked with the - // target's, and a group's exact pins are about one link. - if (state.packages[i].buildTimeOnly) continue; + // A package built for this machine only (a tool's) is not linked with + // the target's, and a group's exact pins are about one link. + if (!instances.target[i] && instances.host[i]) continue; auto const& pm = state.packages[i].manifest; auto n = mcpp::pm::compat::resolve_package_name(pm.package.name, pm.package.namespace_); auto it = P.fields.versions.find(n.shortName); @@ -431,24 +438,32 @@ std::expected step4b_check_platform_versions(PrepareState& st // the C library and the C++ run-time, and `okld check` refused `free`.) void step4b_platform_member_edges(PrepareState& state) { if (!state.platform.active || !state.workspacePlan() || state.platformLayerSpecs.empty()) return; + // The packages of the plan by short name, so a member's edge to a layer + // provider is a lookup. + std::map, std::less<>> byName; + for (std::size_t l = 1; l < state.packages.size(); ++l) { + auto const& pm = state.packages[l].manifest; + byName[mcpp::pm::compat::resolve_package_name(pm.package.name, pm.package.namespace_) + .shortName].push_back(l); + } + auto give = [&](std::size_t member, const std::string& key, const mcpp::manifest::DependencySpec& spec) { + auto& deps = state.packages[member].manifest.dependencies; + if (deps.contains(key)) return; + deps[key] = spec; + auto it = byName.find(spec.shortName); + if (it == byName.end()) return; + for (auto const l : it->second) { + DependencyEdge e; + e.consumerPackageIndex = member; + e.dependencyPackageIndex = l; + e.requestedFeatures = spec.features; + e.defaultFeatures = spec.defaultFeatures; + state.dependencyEdges.push_back(std::move(e)); + } + }; for (std::size_t m = 1; m < state.packages.size(); ++m) { if (!state.packages[m].selectedMember) continue; - auto& deps = state.packages[m].manifest.dependencies; - for (auto const& [key, spec] : state.platformLayerSpecs) { - if (deps.contains(key)) continue; - deps[key] = spec; - for (std::size_t l = 1; l < state.packages.size(); ++l) { - auto const& pm = state.packages[l].manifest; - auto n = mcpp::pm::compat::resolve_package_name(pm.package.name, pm.package.namespace_); - if (n.shortName != spec.shortName) continue; - DependencyEdge e; - e.consumerPackageIndex = m; - e.dependencyPackageIndex = l; - e.requestedFeatures = spec.features; - e.defaultFeatures = spec.defaultFeatures; - state.dependencyEdges.push_back(std::move(e)); - } - } + for (auto const& [key, spec] : state.platformLayerSpecs) give(m, key, spec); } } @@ -622,6 +637,19 @@ std::expected step13_finish_platform(PrepareState& state, Bui state.hostPlatformTarget, state.hostPlatformRunner.empty() ? std::string("its runner") : std::filesystem::path(state.hostPlatformRunner.front()).filename().string())); + if (state.overrides.configuration_only) { + if (ctx.hostPlatform.active) { + ctx.hostTriple = state.hostPlatformTarget; + ctx.hostCompiler = ctx.hostPlatform.fields.toolchain.value; + ctx.hostCompilerOrigin = "platform"; + } else if (auto h = state.host_tc_for_build_program()) { + ctx.hostTriple = h->second.targetTriple; + ctx.hostCompiler = h->second.binaryPath.generic_string(); + ctx.hostCompilerOrigin = "host"; + } else { + return std::unexpected(h.error()); + } + } return {}; } diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index accfb7d51..3ac8c8b15 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -1098,6 +1098,18 @@ std::expected step2_use_local_toolchain(PrepareState& state, const mcpp::toolchain::ToolchainSpec& spec); std::expected step2_apply_local_toolchain(PrepareState& state); std::expected step6_toolchain_phase(PrepareState& state); +// Which instances each package has, by index (features.cpp): a target instance +// where ordinary edges reach it from the root, a build-program instance where +// a `[build-dependencies]` edge reaches it. +struct PackageInstances { + std::vector target; + std::vector host; +}; +PackageInstances package_instances(const PrepareState& state); +// The platform's fields a package's build program stated (protocol 17), at the +// build-program tier of the build's platform statement, named after `who`. +void merge_program_platform(PrepareState& state, mcpp::manifest::platform::Statement stated, + std::string_view who); // The platform through the phases (prepare/platform.cpp). mcpp::build::cfgpred::Ctx platform_cfg_context(const PrepareState& state); std::expected step1_select_platform(PrepareState& state); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index edad07d4a..639b61a7b 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -2222,6 +2222,10 @@ static std::expected step9_member_build_programs(PrepareState who = pkg.manifest.package.name; } if (bc.runExclusive && !exclusiveBefore) rootBc.runExclusive = true; + // The platform's fields the member's program stated, as a dependency's + // program's are (SPEC-013 §4.1). + merge_program_platform(state, std::exchange(bc.programPlatform, {}), + pkg.manifest.package.name); } if (auto err = state.checkVersionFloors(); err) return std::unexpected(*err); // What the programs published (an include directory, an interface diff --git a/src/build/prepare/toolchain_decision.cpp b/src/build/prepare/toolchain_decision.cpp index 88a359d7a..7a3b435ec 100644 --- a/src/build/prepare/toolchain_decision.cpp +++ b/src/build/prepare/toolchain_decision.cpp @@ -19,6 +19,7 @@ import mcpp.toolchain.dialect; import mcpp.toolchain.fingerprint; import mcpp.toolchain.registry; import mcpp.toolchain.linkmodel; +import mcpp.toolchain.triple; // For `resolve_version_match` / `list_installed_versions`: a bare compiler // family named by the dependency graph resolves to a concrete version through // exactly the path `mcpp toolchain default ` uses. @@ -160,14 +161,28 @@ std::expected phase5_toolchain_after_graph(PrepareState& stat reqCompilerBy, family, family, family)); }; + // WHAT "THE GRAPH SUPPLIES THE SYSTEM" MEANS FOR THIS TARGET. A C + // library of the graph does, on any target. A kernel-abi package does + // only where the target has no system of its own (a freestanding or a + // loader-supplied row): on a hosted target it sits over the system's C + // library, which the row's payload still supplies. Measured on + // linux-aarch64: openkal-tools reaches openkal-linux (kernel-abi over + // the system's C library), the musl row's convention was cancelled by + // it, and `mcpp pack --linkage static` compiled the musl target with + // the machine's llvm against glibc's headers. + const auto targetTriple = mcpp::toolchain::triple::parse( + state.overrides.target_triple.empty() ? mcpp::toolchain::triple::host_triple().str() + : state.overrides.target_triple); + const bool targetHasNoSystem = targetTriple + && (targetTriple->is_freestanding() || targetTriple->loader_supplies_os()); + auto supplies_system = [&](mcpp::targetside::CapLayer layer) { + return layer == mcpp::targetside::CapLayer::CAbi + || (layer == mcpp::targetside::CapLayer::KernelAbi && targetHasNoSystem); + }; for (auto const& pkg : state.packages) { for (auto const& entry : pkg.manifest.provides) { auto cap = mcpp::targetside::parse_capability(entry); - if (!cap || !*cap) continue; - if ((*cap)->layer == mcpp::targetside::CapLayer::KernelAbi - || (*cap)->layer == mcpp::targetside::CapLayer::CAbi) { - graphSuppliesSystem = true; - } + if (cap && *cap && supplies_system((*cap)->layer)) graphSuppliesSystem = true; } for (auto const& entry : pkg.manifest.requires_) { auto cap = mcpp::targetside::parse_capability(entry); @@ -205,9 +220,10 @@ std::expected phase5_toolchain_after_graph(PrepareState& stat } } } - // AND A FREESTANDING PIN SURVIVES IT. `graphSuppliesSystem` spans - // kernel-abi and c-abi, and it correctly cancels a HOSTED row's - // convention — that row names the payload the graph is replacing. + // AND A FREESTANDING PIN SURVIVES IT. `graphSuppliesSystem` (a C + // library of the graph, or a kernel-abi package where the target has no + // system) correctly cancels a HOSTED row's convention — that row names + // the payload the graph is replacing. // A bare-metal row names the only compiler that emits the target. // // Measured 2026-08-25, on a three-line manifest: diff --git a/src/cli.cppm b/src/cli.cppm index acb7164e2..7c366fb5d 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -1307,8 +1307,7 @@ int run(int argc, char** argv) { .description("(internal: invoked by ninja for a platform's link) `capture --out F -- argv…` saves a program's output; `seal --steps F -- [needed…]` runs the platform's seal steps, then publishes") .arg(cl::Arg("mode").help("capture | seal")) .option(cl::Option("out").takes_value().value_name("PATH").help("capture: the file the output is saved as")) - .option(cl::Option("steps").takes_value().value_name("PATH").help("seal: the steps, one per line, words separated by a tab")) - .option(cl::Option("digest").takes_value().value_name("HEX").help("seal: the steps' digest (part of the command, so a changed step reseals)")) + .option(cl::Option("steps-hex").takes_value().value_name("HEX").help("seal: the steps, one per line, words separated by a tab, hex-encoded (the command is the steps, so a changed step reseals)")) .action(wrap_rc([&passthrough](const cl::ParsedArgs& p) { return cmd_platform_step(p, std::span(passthrough)); }))) diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 22547d1c0..940eb604f 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -40,6 +40,7 @@ import mcpp.hooks; import mcpp.libs.json; import mcpp.log; import mcpplibs.os.fs; +import mcpplibs.os.process; // platform-step: a program's output alone import mcpp.invocation; import mcpp.effects; import mcpp.run; @@ -1531,22 +1532,30 @@ export int cmd_platform_step(const mcpplibs::cmdline::ParsedArgs& parsed, std::span rest) { namespace fs = std::filesystem; const std::string mode = parsed.positional_count() > 0 ? parsed.positional(0) : std::string(); + // The program's standard output, kept apart: its standard error is the + // build's, passed on (a warning it writes there is not part of what it + // produces, which `capture` saves as a source file). auto run = [&](const std::vector& argv, std::string* out) { - int spawnError = 0; - auto r = mcpp::run::capture_exec(argv, {}, {}, &spawnError); - if (spawnError != 0) { - std::println(stderr, "error: cannot start {}: {}", argv.front(), - std::generic_category().message(spawnError)); + namespace proc = mcpplibs::os::process; + proc::spawn_options o; + o.program = argv.front(); + o.args.assign(argv.begin() + 1, argv.end()); + std::string shown; + for (auto const& a : argv) shown += (shown.empty() ? "" : " ") + a; + auto r = proc::run(std::move(o)); + if (!r) { + std::println(stderr, "error: cannot start {}", shown); return false; } - if (r.exit_code != 0) { - std::string shown; - for (auto const& a : argv) shown += (shown.empty() ? "" : " ") + a; - std::println(stderr, "error: `{}` exited {}", shown, r.exit_code); - std::print(stderr, "{}", r.output); + std::print(stderr, "{}", r->err); + if (!r->status.success()) { + // What a refusing step wrote is its reason (`okld check` names the + // import it refused), on either stream. + std::print(stderr, "{}", r->out); + std::println(stderr, "error: `{}` exited {}", shown, r->status.shell_code()); return false; } - if (out) *out = std::move(r.output); + if (out) *out = std::move(r->out); return true; }; std::error_code ec; @@ -1571,14 +1580,24 @@ export int cmd_platform_step(const mcpplibs::cmdline::ParsedArgs& parsed, return 0; } if (mode == "seal") { - const std::string stepsFile = parsed.option_or_empty("steps").value(); - if (stepsFile.empty() || rest.size() < 2) { - std::println(stderr, "error: platform-step seal needs --steps -- [needed…]"); + const std::string hex = parsed.option_or_empty("steps-hex").value(); + if (hex.empty() || hex.size() % 2 != 0 || rest.size() < 2) { + std::println(stderr, "error: platform-step seal needs --steps-hex -- [needed…]"); return 2; } + std::string text; + for (std::size_t i = 0; i < hex.size(); i += 2) { + unsigned v = 0; + auto [p, e] = std::from_chars(hex.data() + i, hex.data() + i + 2, v, 16); + if (e != std::errc{} || p != hex.data() + i + 2) { + std::println(stderr, "error: platform-step seal: the steps are not hexadecimal"); + return 2; + } + text.push_back(static_cast(v)); + } std::vector> steps; { - std::ifstream in(stepsFile, std::ios::binary); + std::istringstream in(text); std::string line; while (std::getline(in, line)) { if (!line.empty() && line.back() == '\r') line.pop_back(); @@ -1594,6 +1613,11 @@ export int cmd_platform_step(const mcpplibs::cmdline::ParsedArgs& parsed, steps.push_back(std::move(step)); } } + // A seal with no step would publish what it was asked to seal. + if (steps.empty()) { + std::println(stderr, "error: platform-step seal was given no step"); + return 2; + } const fs::path linked = rest[0], artifact = rest[1]; const std::vector needed(rest.begin() + 2, rest.end()); auto tmp = artifact; @@ -1653,6 +1677,10 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { return 1; } auto const& P = host ? ctx->hostPlatform : ctx->plan.platform; + // The graph reported: the host graph's triple and compiler with --host. + const std::string triple = host ? ctx->hostTriple : ctx->tc.targetTriple; + const std::string compiler = host ? ctx->hostCompiler : ctx->tc.binaryPath.generic_string(); + const std::string compilerOrigin = host ? ctx->hostCompilerOrigin : ctx->compilerChoice.origin; auto const& runner = P.fields.runner; std::string runnerProgram; if (runner.set && !runner.cmd.empty()) { @@ -1671,10 +1699,9 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { nlohmann::json tools = nlohmann::json::object(); for (auto const& [name, path] : P.tools) tools[name] = path; nlohmann::json data{ - {"triple", ctx->tc.targetTriple}, + {"triple", triple}, {"graph", host ? "host" : "target"}, - {"toolchain", {{"compiler", ctx->tc.binaryPath.generic_string()}, - {"origin", ctx->compilerChoice.origin}}}, + {"toolchain", {{"compiler", compiler}, {"origin", compilerOrigin}}}, {"platform", nlohmann::json::parse(mcpp::platform::to_json(P), nullptr, false)}, {"tools", std::move(tools)}, {"runner", !runner.set ? nlohmann::json(nullptr) : nlohmann::json{ @@ -1691,9 +1718,8 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { return 0; } for (auto const& n : ctx->planNotes) mcpp::ui::warning(n.message); - std::println("{:<10} {}", "target", ctx->tc.targetTriple); - std::println("{:<10} {} ({})", "compiler", ctx->tc.binaryPath.generic_string(), - ctx->compilerChoice.origin); + std::println("{:<10} {}", host ? "host" : "target", triple); + std::println("{:<10} {} ({})", "compiler", compiler, compilerOrigin); std::println("{:<10} {}{}", "platform", P.display(), P.active ? std::format(" ({})", mcpp::platform::origin_text(P.selection.origin)) : std::string()); diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 842ee6b41..f97083520 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -30,6 +30,7 @@ import mcpp.pack.stage_tree; // the built-in formats import mcpp.toolchain.fingerprint; // hash_string import mcpp.toolchain.platform; // host_arch: the static row of `--linkage static` import mcpp.pack.compress; // --compress +import mcpp.toolchain.triple; // which --target values are images' rows import mcpp.pack.distribution; // --merge import mcpplibs.os.platform; @@ -48,6 +49,13 @@ export int cmd_emit_xpkg(const mcpplibs::cmdline::ParsedArgs& parsed) { "mcpp-emit-{}", std::chrono::steady_clock::now().time_since_epoch().count()); for (auto const& p : parsed.option("package")->get().values) { const std::filesystem::path file{p}; + // An entry names a published file, which a directory is not. + if (std::filesystem::is_directory(file, ec)) { + mcpp::ui::error(std::format("--package {} is a directory; an index entry names the " + "archive that is published (--pkg-format zip | tar)", p)); + std::filesystem::remove_all(work, ec); + return 2; + } auto root = d::unpack(file, work / std::to_string(assets.size())); if (!root) { mcpp::ui::error(root.error()); std::filesystem::remove_all(work, ec); return 2; } auto pkg = d::read(*root); @@ -214,7 +222,9 @@ namespace { std::optional pack_format_option(const mcpplibs::cmdline::ParsedArgs& parsed) { auto pkg = parsed.value("pkg-format"); auto old = parsed.value("format"); - if (old) { + // Said once per command, however often the value is read. + static bool warned = false; + if (old && !std::exchange(warned, true)) { if (pkg && *pkg != *old) { mcpp::ui::warning(std::format( "--format '{}' and --pkg-format '{}' disagree; --pkg-format is used", *old, *pkg)); @@ -574,9 +584,12 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, // THE PROJECT'S DEFAULTS OF THE THREE AXES (`[pack]`, SPEC-014 §1), read // from the package this command packs; a flag states over each. mcpp::manifest::PackConfig packCfg; + std::string projectTarget; // `[build] target`: the project's own default triple if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path())) - if (auto eff = mcpp::project::load_effective_manifest(*root)) + if (auto eff = mcpp::project::load_effective_manifest(*root)) { packCfg = eff->manifest.packConfig; + projectTarget = eff->manifest.buildConfig.target; + } if (parsed.option("merge")) { std::vector inputs; for (auto const& v : parsed.option("merge")->get().values) { @@ -593,10 +606,26 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, if (auto v = parsed.value("output")) req.output = *v; if (auto f = pack_format_option(parsed)) req.format = *f; else if (!packCfg.format.empty()) req.format = packCfg.format; - std::string compressText = parsed.value("compress").value_or(packCfg.compress); - if (!compressText.empty()) { - auto c = mcpp::pack::compress::parse(compressText); + // The container: as stated, else as the output's name says. Only the + // flag's compression is the command's (a project's `[pack] compress` + // describes that project's packs, not a merge of packages). + std::string container = req.format; + const auto outName = req.output.filename().string(); + if (container.empty() && outName.ends_with(".zip")) container = "zip"; + else if (container.empty() && outName.find(".tar") != std::string::npos) container = "tar"; + if (auto text = parsed.value("compress")) { + auto c = mcpp::pack::compress::parse(*text); if (!c) { mcpp::ui::error(std::format("--compress: {}", c.error())); return 2; } + using mcpp::pack::compress::Algorithm; + if (container == "dir") { + mcpp::ui::error("--compress compresses a container, and the merge writes a directory"); + return 2; + } + if (container == "zip" && (c->algorithm == Algorithm::Zstd || c->algorithm == Algorithm::Xz)) { + mcpp::ui::error(std::format("a zip's members are deflated (gzip) or stored (none); " + "--compress {} is a tar's", mcpp::pack::compress::name(c->algorithm))); + return 2; + } req.compression = *c; } auto merged = mcpp::pack::distribution::merge(req); @@ -605,7 +634,21 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, if (report) { report->rc = 0; report->artifacts.push_back(*merged); - report->format = req.format.empty() ? std::string("tar") : req.format; + std::error_code mec; + const auto name = merged->filename().string(); + report->format = std::filesystem::is_directory(*merged, mec) ? std::string("dir") + : name.ends_with(".zip") ? std::string("zip") : std::string("tar"); + // What the merged package says of itself. + const auto work = std::filesystem::temp_directory_path(mec) + / std::format("mcpp-merged-{}", std::chrono::steady_clock::now().time_since_epoch().count()); + if (auto root = mcpp::pack::distribution::unpack(*merged, work)) + if (auto pkg = mcpp::pack::distribution::read(*root)) { + report->targets = pkg->legs; + report->systems = pkg->systems; + report->linkage = pkg->linkage; + report->mode = pkg->mode; + } + std::filesystem::remove_all(work, mec); } return 0; } @@ -668,6 +711,7 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, return 2; } opts.compression = *c; + opts.compressionFromFlag = compressFlag.has_value(); } using mcpp::pack::compress::Algorithm; const auto alg = opts.compression.algorithm; @@ -760,7 +804,8 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, // is what `--mode static` has always built. On another system the // libraries of the graph are linked in and the system's stay its own. if (opts.linkage == mcpp::pack::Linkage::Static) { - if (triples.empty()) { + // This machine's row only when the project names no target of its own. + if (triples.empty() && projectTarget.empty()) { if constexpr (mcpplibs::os::is_linux) triples.push_back(std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch)); } else { @@ -856,6 +901,19 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, return mcpp::pack::build_and_pack_library(route->targetName, triples, opts, libraryReport); } + // SEVERAL IMAGES' TARGETS ARE ONE UNIVERSAL PACKAGE'S LEGS (SPEC-014 + // §1.4): the pack builds the first and the others as its legs; whether the + // platform packs them into one package is decided once it is known. + if (triples.size() > 1 && std::ranges::all_of(triples, [](std::string const& t) { + auto p = mcpp::toolchain::triple::parse(t); + return p && p->loader_supplies_os(); + })) { + opts.legTargets = triples; + opts.targetTriple = triples.front(); + auto out = mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName); + if (report) *report = out; + return out.rc; + } if (triples.size() > 1) { // #630 A9: THE ROUTE IS CHOSEN BY THE ARTIFACT'S FORM, NOT BY THE // TARGET'S KIND. An `app` whose resolved link form is a shared diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 096a451ca..abb3bed29 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -137,6 +137,7 @@ struct Options { Linkage linkage = Linkage::Unstated; bool formatStated = false; mcpp::pack::compress::Choice compression; + bool compressionFromFlag = false; // --compress, not [pack] // Universal packages (SPEC-014 §3). `universal`: nullopt follows the // platform's `universal`; false packs the selected target alone. When the // pipeline decides a universal package it sets `universalName` (the @@ -146,6 +147,9 @@ struct Options { std::optional universal; std::string universalName; std::vector systems; + // Several `--target` naming a universal platform's targets: the legs, in + // place of the platform's whole `targets` (SPEC-014 §1.4). + std::vector legTargets; // `[pack] sign`: run on each program, image, library and loader of the // staged tree before the container is written, `{}` the file. std::vector sign; @@ -783,6 +787,21 @@ make_plan(const mcpp::manifest::Manifest& manifest, : opts.format != Format::Tar ? ArchiveKind::None : (p.targetIsPe && !opts.formatStated) ? ArchiveKind::Zip : ArchiveKind::Tar; + // A zip deflates or stores: a zstd or xz the format left to the artifact + // cannot honour is refused when the command stated it, and the project's + // default leaves it aside (SPEC-014 §5.1). + if (p.archive == ArchiveKind::Zip + && (opts.compression.algorithm == mcpp::pack::compress::Algorithm::Zstd + || opts.compression.algorithm == mcpp::pack::compress::Algorithm::Xz)) { + if (opts.compressionFromFlag) + return std::unexpected(Error{std::format( + "{} is packed as a zip (a Windows program's archive), whose members are deflated " + "or stored; --compress {} is a tar's.\n use: --pkg-format tar --compress {}, or " + "--compress gzip | none", p.binaryName, + mcpp::pack::compress::name(opts.compression.algorithm), + mcpp::pack::compress::name(opts.compression.algorithm))}); + p.opts.compression = {}; + } // A universal package is named after its platform, not a triple: it is // the same package on every system and architecture of the platform. const std::string nameTriple = opts.universalName.empty() ? p.triple : opts.universalName; @@ -1299,9 +1318,12 @@ void copy_if_exists(const std::filesystem::path& src, // every one of them, so a missing file is an error naming it rather than a // bundle that silently lacks it. std::expected -stage_runtime_files(const Plan& plan, const std::filesystem::path& stagedExeDir) +stage_runtime_files(const Plan& plan, const std::filesystem::path& stagedExeDir, + const std::filesystem::path& builtExeDir = {}) { - const auto builtDir = plan.builtBinary.parent_path(); + // The leg's own build directory: another architecture's leg deployed its + // own files beside its own artifact. + const auto builtDir = builtExeDir.empty() ? plan.builtBinary.parent_path() : builtExeDir; for (auto const& rel : plan.opts.runtimeFiles) { const auto src = builtDir / rel; const auto dst = stagedExeDir / rel; @@ -1510,9 +1532,10 @@ make_zip(const std::filesystem::path& stagingRoot, const std::filesystem::path& // `[pack] include`: the project's files the globs name, at the same path // below the tree's root, once whatever the legs (a `share/**` is the // package's, not an architecture's); `[pack] exclude` drops what it names. -std::expected stage_included(const Plan& plan) { +std::expected, Error> stage_included(const Plan& plan) { namespace fs = std::filesystem; - if (plan.includeGlobs.empty()) return {}; + std::vector staged; + if (plan.includeGlobs.empty()) return staged; std::vector files; auto excluded = [&](const fs::path& f) { return std::ranges::any_of(plan.excludeGlobs, [&](std::string const& g) { @@ -1534,8 +1557,9 @@ std::expected stage_included(const Plan& plan) { fs::copy_file(f, to, fs::copy_options::overwrite_existing, ec); if (ec) return std::unexpected(Error{std::format("[pack] include: cannot stage {}: {}", f.string(), ec.message())}); + staged.push_back(f.lexically_relative(plan.projectRoot).generic_string()); } - return {}; + return staged; } std::expected finalize_tree(const Plan& plan) { @@ -1543,7 +1567,8 @@ std::expected finalize_tree(const Plan& plan) { namespace d = mcpp::pack::distribution; std::error_code ec; if (!fs::is_directory(plan.stagingRoot, ec)) return {}; - if (auto r = stage_included(plan); !r) return r; + auto included = stage_included(plan); + if (!included) return std::unexpected(included.error()); const auto canonical = [](std::string const& t) { if (t.empty()) return mcpp::toolchain::triple::host_triple().str(); auto parsed = mcpp::toolchain::triple::parse(t); @@ -1587,7 +1612,10 @@ std::expected finalize_tree(const Plan& plan) { if (plan.targetIsLoaded && plan.opts.mode == Mode::BundleAll) pkg.program = "bin/" + fs::path(plan.binaryName).stem().string() + std::string(mcpplibs::os::exe_suffix); + stated.role[pkg.program] = "program"; } + // What `[pack] include` named is the package's resource, whatever its name. + for (auto const& rel : *included) stated.role[rel] = "resource"; auto described = d::describe(plan.stagingRoot, pkg, stated); if (!described) return std::unexpected(Error{described.error()}); if (plan.opts.sign.empty()) return {}; @@ -2513,7 +2541,7 @@ run_loaded(const Plan& plan) for (auto const& f : *n) if (auto r = copy(f, dir / f.filename()); !r) return r; } - if (auto r = stage_runtime_files(plan, dir); !r) return r; + if (auto r = stage_runtime_files(plan, dir, leg.artifact.parent_path()); !r) return r; } if (plan.opts.mode == Mode::BundleAll) { launch.loader = fs::path(runner.cmd.front()).stem().string(); @@ -3209,6 +3237,11 @@ std::expected, Error> single_file(const Plan& const std::string arch = t ? t->arch : std::string(mcpp::toolchain::platform::host_arch); std::vector systems = plan.opts.systems; if (systems.empty()) systems.emplace_back(detail::host_system()); + if (systems.size() > 1 && !plan.opts.output.empty()) + return std::unexpected(Error{std::format( + "--output names one file, and --systems makes one single file per system ({}).\n" + " use: leave --output out; each file is named ---", + systems.size())}); for (auto const& system : systems) { const auto stem = fs::path(plan.binaryName).stem().string(); auto packed = detail::carrier_build(plan, { carried, plan.binaryName, stem, system, arch, true }); @@ -3283,8 +3316,10 @@ std::expected, Error> single_file(const Plan& int spawnError = 0; auto r = mcpp::run::capture_exec(argv, {}, {}, &spawnError); if (spawnError != 0 || r.exit_code != 0) - return std::unexpected(Error{std::format("[pack] sign failed on {} (exit {}):\n{}", - f.string(), r.exit_code, r.output)}); + return std::unexpected(Error{std::format("[pack] sign failed on {} (exit {}): {}\n{}", + f.string(), r.exit_code, + mcpp::build::command_line::display(argv), + r.output)}); } return written; } diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 10e9ff444..aa55872ca 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -36,6 +36,7 @@ import mcpp.provider; import mcpp.provider.select; import mcpp.toolchain.platform; // host_arch: the static row import mcpp.pack.distribution; // the package's metadata, for a dispatched format +import mcpp.platform; // serves: a leg the platform states namespace mcpp::pack { @@ -593,16 +594,37 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { const bool loaded = t && t->loader_supplies_os(); if (loaded && opts.mode == mcpp::pack::Mode::Static) opts.mode = mcpp::pack::Mode::BundleAll; auto const& P = ctx.plan.platform; + // A single file carries what it runs, whichever source stated the mode. + if (opts.format == mcpp::pack::Format::SingleFile && opts.mode == mcpp::pack::Mode::None) + return Refusal{2, about(run, m, std::string( + "--pkg-format single-file carries what the program runs, and the mode is system " + "([pack] default_mode).\n use: --mode vendored | self-contained"))}; + if (!opts.legTargets.empty() && !packs_universal(ctx, opts)) + return Refusal{2, about(run, m, std::string( + "several --target name one package's legs only where the target's platform packs " + "them into one universal package (`[platform] universal = true`).\n" + " use: one pack per target"))}; if (packs_universal(ctx, opts)) { opts.universalName = mcpp::manifest::platform::platform_name( P.package.substr(P.package.find('.') + 1)); - for (auto const& target : P.fields.targets.value) { + // The legs: those the command named, else every target of the + // platform. + auto const& legs = opts.legTargets.empty() ? P.fields.targets.value : opts.legTargets; + for (auto const& target : legs) { auto lt = mcpp::toolchain::triple::parse(target); if (!lt || lt->str() == t->str()) continue; + if (!mcpp::platform::serves(P.fields, P.package, *lt)) + return Refusal{2, about(run, m, std::format( + "--target {} is not a target of {}", lt->str(), P.display()))}; mcpp::build::BuildOverrides ov = g.ov; ov.target_triple = lt->str(); + // The member's own plan, in a workspace; the project's otherwise. ov.workspace_members.clear(); ov.workspace_request.clear(); + if (m.inWorkspace && !m.path.empty()) { + ov.workspace_members = { m.path }; + ov.workspace_request = { m.path }; + } auto legCtx = mcpp::build::prepare_build(false, false, {}, ov); if (!legCtx) return Refusal{1, about(run, m, std::format( "the {} leg of the universal package: {}", lt->str(), legCtx.error()))}; @@ -616,7 +638,8 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { std::filesystem::path artifact; for (auto const& lu : legCtx->plan.linkUnits) if (lu.targetName == chosenLu->targetName && lu.kind == chosenLu->kind - && !lu.dependencyOwned && (!sharedPlan || lu.memberOf == m.name)) { + && !lu.dependencyOwned + && (legCtx->workspaceMembers.size() < 2 || lu.memberOf == m.name)) { artifact = legCtx->outputDir / lu.output; break; } @@ -755,7 +778,7 @@ std::optional prepare_group(PackRun& run, GroupJob& g) { // prepare with the same overrides plus two fields -- and a second // overrides object left behind here would make that pass differ from // this build in a way nothing states. - g.ov.target_triple = "x86_64-linux-musl"; + g.ov.target_triple = std::format("{}-linux-musl", mcpp::toolchain::platform::host_arch); // Quiet on the same grounds as the first prepare: this one also runs // before `--format` has been validated. auto ctx2 = prepare(); @@ -790,6 +813,13 @@ std::optional check_format(PackRun& run, GroupJob& g) { "package carries, or for which a single file of an image is made; this " "pack is neither.\n use: --mode self-contained over a platform that " "states universal = true, or --pkg-format single-file")}; + // A single file is made per system, and `-o` names one file. + if (run.opts.format == mcpp::pack::Format::SingleFile && run.opts.systems.size() > 1 + && !run.opts.output.empty()) + return Refusal{2, std::format( + "--output names one file, and --systems makes one single file per system ({}).\n" + " use: leave --output out; each file is named ---", + run.opts.systems.size())}; } // ─── Is the requested format one anything provides? ────────────── // diff --git a/src/platform/platform.cppm b/src/platform/platform.cppm index a88d695b5..a00609753 100644 --- a/src/platform/platform.cppm +++ b/src/platform/platform.cppm @@ -62,6 +62,7 @@ struct ChoiceInputs { std::span project; // the plan root's tables, the workspace layer's first std::span config; // config.toml's std::string_view column; // the target table's platform column + std::string flagKey = "--platform"; // or MCPP_PLATFORM, from the environment }; // The target graph's platform: the flag, the environment, the project, this @@ -250,7 +251,7 @@ std::vector matching(std::span tables, std::expected select_target(const ChoiceInputs& in, const Matches& matches) { if (!in.flag.empty()) - return parse_choice(in.flag, mp::Origin{mp::Tier::Flag, {}, 0, "--platform"}); + return parse_choice(in.flag, mp::Origin{mp::Tier::Flag, {}, 0, in.flagKey}); if (!in.env.empty()) return parse_choice(in.env, mp::Origin{mp::Tier::Env, {}, 0, in.envKey}); for (auto span : {in.project, in.config}) { diff --git a/src/project.cppm b/src/project.cppm index fe663461e..0d0b4e1d2 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -399,14 +399,13 @@ void merge_layer_root_position(mcpp::manifest::Manifest& member, for (auto const& [n, argv] : from.namedRunners) to.namedRunners.try_emplace(n, argv); } - // The platform part of `[workspace.target.]` and `[workspace.host]` - // (SPEC-013 §4.5): the workspace's tables merge first, so the member's own - // come after them and win field by field. + // The platform part of `[workspace.target.]` (SPEC-013 §13): the + // workspace's tables merge first, so of two tables of one selector the + // member's wins field by field; between selectors the more specific wins, + // as for the other keys above. `[host]` is not a workspace table. member.platformOverrides.insert(member.platformOverrides.begin(), layer.platformOverrides.begin(), layer.platformOverrides.end()); - if (!member.hostPlatform.platformSet && layer.hostPlatform.platformSet) - member.hostPlatform = layer.hostPlatform; } } // namespace detail diff --git a/tests/e2e/292_a_package_that_names_a_layer_does_not_lose_the_targets_compiler.sh b/tests/e2e/292_a_package_that_names_a_layer_does_not_lose_the_targets_compiler.sh index b3ea0ba94..0307579d1 100755 --- a/tests/e2e/292_a_package_that_names_a_layer_does_not_lose_the_targets_compiler.sh +++ b/tests/e2e/292_a_package_that_names_a_layer_does_not_lose_the_targets_compiler.sh @@ -96,4 +96,34 @@ case "$hosted" in exit 1 ;; esac +# ── Half three: a kernel-abi package over the system's C library ────────── +# +# On a hosted target a kernel-abi package (openkal-linux under mcpplibs.os) +# sits over the system's C library, which the row's payload still supplies: +# it supplies no system, and the row's convention stands. Measured on +# linux-aarch64: it cancelled the musl row's convention, and the machine's +# llvm compiled the musl target against glibc's headers. With a compiler of +# another family named and nothing in the payload's place, the decision is +# the refusal half one of e2e 298 states. +mkdir -p "$work/kabi/src" +cat > "$work/kabi/mcpp.toml" < "$work/kabi/src/main.cpp" +kabi="$(cd "$work/kabi" && "$MCPP" build --target x86_64-linux-musl 2>&1 || true)" +case "$kabi" in + *"takes its C library from"*) + echo " ok a kernel-abi package on a hosted target leaves the row's convention standing" ;; + *) + echo "FAIL: a kernel-abi package over the system's C library cancelled the row's convention" + printf '%s\n' "$kabi" | grep -iE 'Resolved|error' | head -4 | sed 's/^/ /' + exit 1 ;; +esac + echo "OK: naming a layer changes the system, not the compiler that emits the target" diff --git a/tests/e2e/952_a_platform_package_states_a_class_of_targets.sh b/tests/e2e/952_a_platform_package_states_a_class_of_targets.sh index de2e32eac..1e2d1027d 100755 --- a/tests/e2e/952_a_platform_package_states_a_class_of_targets.sh +++ b/tests/e2e/952_a_platform_package_states_a_class_of_targets.sh @@ -19,6 +19,8 @@ # D. `--platform none` builds the project without the platform: the default # name, no flag, no seal, no runner. # E. A platform package without `[platform]` is refused, naming the package. +# F. A flag with `$`, quotes and a space reaches the compile as written. +# G. With the objects removed, the program is linked and sealed again, once. set -e TMP=$(mktemp -d) @@ -36,7 +38,7 @@ targets = ["x86_64-linux-gnu", "aarch64-linux-gnu", "aarch64-macos", "x86_64-mac runner = ["env", "DEMO_PLATFORM_RUNNER=yes", "{}"] [platform.build] -cxxflags = ["-DDEMO_PLATFORM=1"] +cxxflags = ["-DDEMO_PLATFORM=1", "-DDEMO_TAG=\"a$b c\""] [platform.link] exe = "{name}.demo" @@ -60,9 +62,14 @@ int main() { const char* flag = "flag"; #else const char* flag = "no-flag"; +#endif +#ifdef DEMO_TAG + const char* tag = DEMO_TAG; +#else + const char* tag = "none"; #endif const char* runner = std::getenv("DEMO_PLATFORM_RUNNER"); - std::printf("demo: %s, runner %s\n", flag, runner ? runner : "none"); + std::printf("demo: %s, runner %s, tag [%s]\n", flag, runner ? runner : "none", tag); return 0; } CPP @@ -122,4 +129,19 @@ grep -q "has no \[platform\] section" notplat.log || fail "E: the refusal does n grep -q "notplat" notplat.log || fail "E: the refusal does not name the package" notplat.log echo "ok: a package without [platform] is refused as a platform" +# F +grep -qF 'tag [a$b c]' run.log || fail "F: the flag with \$ did not reach the compile as written" run.log +echo "ok: a flag with \$, quotes and a space reaches the compile as written" + +# G +"$MCPP" build > g0.log 2>&1 || fail "G: build" g0.log +prog=$(ls target/*/*/bin/app.over | head -1) +[ -n "$prog" ] || fail "G: no program" g0.log +rm -rf "$(dirname "$(dirname "$prog")")/obj" +"$MCPP" build > g.log 2>&1 || fail "G: build after the objects were removed" g.log +[ "$(tail -c 11 "$prog")" = "DEMO-SEALED" ] || fail "G: the program was not sealed again" g.log +[ "$(tail -c 22 "$prog")" = "DEMO-SEALEDDEMO-SEALED" ] && fail "G: the program was sealed twice" g.log +"$MCPP" run > g-run.log 2>&1 || fail "G: run" g-run.log +echo "ok: a program linked again is sealed again, once" + echo "PASS: 952 a platform package states a class of targets" diff --git a/tests/e2e/954_pack_has_three_axes.sh b/tests/e2e/954_pack_has_three_axes.sh index b0bef9205..466e752da 100755 --- a/tests/e2e/954_pack_has_three_axes.sh +++ b/tests/e2e/954_pack_has_three_axes.sh @@ -20,6 +20,8 @@ # F. `--format` is the earlier spelling of `--pkg-format`, with a warning. # G. `[pack] include` stages the files it names at their paths, less what # `[pack] exclude` names, and the metadata lists them as resources. +# H. `[pack] sign` runs on each program before the container, and the +# metadata's digests are those of the signed files. set -e TMP=$(mktemp -d) @@ -139,4 +141,25 @@ assert roles.get("share/data/keep.txt") == "resource", roles PY echo "ok: [pack] include stages what it names, less what exclude names" +# H +cat >> mcpp.toml <<'TOML' +sign = ["sh", "-c", "printf SIGNED >> \"$0\"", "{}"] +TOML +"$MCPP" pack --pkg-format dir -o signed > h.log 2>&1 || fail "H: pack with [pack] sign" h.log +python3 - <<'PY' || fail "H: the signed tree" target/dist/signed/mcpp-package.json +import hashlib, json, os +root = "target/dist/signed" +d = json.load(open(os.path.join(root, "mcpp-package.json"))) +programs = [f for f in d["files"] if f["role"] == "program"] +assert programs, d["files"] +for f in programs: + data = open(os.path.join(root, f["path"]), "rb").read() + assert data.endswith(b"SIGNED"), f["path"] + " was not signed" + assert hashlib.sha256(data).hexdigest() == f["sha256"], f["path"] + ": the digest is the unsigned file's" +for f in d["files"]: + if f["role"] in ("resource", "metadata"): + assert not open(os.path.join(root, f["path"]), "rb").read().endswith(b"SIGNED"), f["path"] +PY +echo "ok: [pack] sign signs each program, and the digests are the signed files'" + echo "PASS: 954 pack has three axes" diff --git a/tests/e2e/956_a_single_file_runs_without_unpacking.sh b/tests/e2e/956_a_single_file_runs_without_unpacking.sh index e8ae5dd56..1dab46acc 100755 --- a/tests/e2e/956_a_single_file_runs_without_unpacking.sh +++ b/tests/e2e/956_a_single_file_runs_without_unpacking.sh @@ -13,6 +13,8 @@ # C. Refused before a build: `--compress xz` for a single file. # D. (llvm, an ok-image row) An image's single file runs, and writes nothing # into the cache. +# E. (an ok-image row) `-o` with several `--systems` is refused before a +# build: one file is made per system. set -e TMP=$(mktemp -d) @@ -102,6 +104,14 @@ if [ -n "$T" ] && command -v python3 > /dev/null && "$MCPP" toolchain list --for after=$(find "$CACHE" -type f | wc -l) [ "$before" = "$after" ] || fail "D: the image's single file wrote into the cache" echo "ok: an image's single file runs from memory" + + # E + if "$MCPP" pack --target "$T" --pkg-format single-file --systems linux,macos -o one-file > e.log 2>&1; then + fail "E: -o with two systems was accepted" e.log + fi + grep -q "one single file per system" e.log || fail "E: the refusal" e.log + grep -q "Compiling" e.log && fail "E: refused after compiling" e.log + echo "ok: -o with several systems is refused before a build" fi echo "PASS: 956 a single file runs without unpacking" diff --git a/tests/e2e/957_packages_merge_and_describe_themselves.sh b/tests/e2e/957_packages_merge_and_describe_themselves.sh index db6e52e1a..275d63262 100755 --- a/tests/e2e/957_packages_merge_and_describe_themselves.sh +++ b/tests/e2e/957_packages_merge_and_describe_themselves.sh @@ -17,6 +17,7 @@ # C. `mcpp emit xpkg --package` of a universal package is one asset for # linux, macosx and windows, with the runner as a dependency when the # package does not carry it. +# D. An archive merges as the directory it holds does. set -e TMP=$(mktemp -d) @@ -71,6 +72,24 @@ for f in d["files"]: PY echo "ok: --merge writes the union of the packages" +# D (before B changes the second package) +python3 - "$TMP/there" "$TMP/there.tar.gz" <<'PY' +import sys, tarfile +with tarfile.open(sys.argv[2], "w:gz") as t: + t.add(sys.argv[1], arcname="mrg-0.1.0-openkal") +PY +"$MCPP" pack --merge target/dist/here --merge "$TMP/there.tar.gz" --pkg-format dir -o "$TMP/merged-dir" > d.log 2>&1 \ + || fail "D: --merge of a directory and an archive" d.log +python3 - "$TMP/merged-dir" <<'PY' || fail "D: the merged tree" d.log +import hashlib, json, os, sys +root = sys.argv[1] +d = json.load(open(os.path.join(root, "mcpp-package.json"))) +assert len(d["systems"]) == 4, d["systems"] +for f in d["files"]: + assert hashlib.sha256(open(os.path.join(root, f["path"]), "rb").read()).hexdigest() == f["sha256"], f["path"] +PY +echo "ok: an archive merges as its directory does" + # B python3 - "$TMP/there" <<'PY' import hashlib, json, os, sys diff --git a/tests/unit/test_platform.cpp b/tests/unit/test_platform.cpp index f098d0b54..b84b7e6e3 100644 --- a/tests/unit/test_platform.cpp +++ b/tests/unit/test_platform.cpp @@ -135,12 +135,18 @@ TEST(PlatformManifest, RefusalsNameTheKey) { "names the package too" }, { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.layers]\nc-abi = { program = \"x\" }\n", "for a tool only" }, - { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.hosted]\nrunner = [\"x\"]\n", - "a host triple takes" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.link]\nshared = \"{name}{name}\"\n", + "exactly once" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.link]\nshared = \"lib/{name}.okso\"\n", + "names no directory" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\nplugin = \"gen.mcpp\"\n", + "must be \"build.mcpp\"" }, + { "[platform]\ntargets = [\"x86_64-openkal\"]\n[platform.layers]\n" + "c-abi = { git = \"https://x/y.git\", tag = \"v1\" }\n", + "a git source names the package too" }, { "[target.'cfg(platform = \"openkal\")']\nplatform = \"x@1.0.0\"\n", "cannot be selected under a selector" }, { "[target.x86_64-openkal]\nlayers = \"x\"\n", "must be a table" }, - { "[host]\nrunner = [\"x\"]\n", "[host] has no key" }, }; for (auto const& r : rows) { auto e = parse_error(head + r.body); @@ -148,6 +154,38 @@ TEST(PlatformManifest, RefusalsNameTheKey) { } } +// An unknown key, in `[platform]`, its tables or `[host]`, is a warning: a +// manifest written for a later mcpp still reads (SPEC-013 §18.1). +TEST(PlatformManifest, UnknownKeysWarn) { + auto m = parse(R"( +[package] +name = "p" +version = "1.0.0" + +[platform] +targets = ["x86_64-openkal"] +runner = { cmd = ["okrun", "{}"], later = true } +[platform.hosted] +runner = ["x"] +[platform.link] +exe = "{name}.oke" +later = "x" +[platform.build] +rustflags = ["-x"] + +[host] +runner = ["x"] +)"); + std::string all; + for (auto const& w : m.schemaWarnings) all += w + "\n"; + for (std::string_view key : { "'later' is not a key of a runner", "[platform.hosted] has unsupported key 'runner'", + "[platform.link] has unsupported key 'later'", + "[platform.build] has unsupported key 'rustflags'", + "[host] has unsupported key 'runner'" }) + EXPECT_NE(all.find(key), std::string::npos) << key << "\n=> " << all; + EXPECT_EQ(m.platform.link.exe.value, "{name}.oke"); +} + TEST(PlatformManifest, OverridesAndHostAreRead) { auto m = parse(R"( [package] From c3d5ecceb886fdc7d856468469409199a846fda4 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 10:46:54 +0900 Subject: [PATCH 39/54] pack.compress: an unstated level is -1 and every stated one is honoured (xz:0, zstd:1); a gzip member states its header (no name, no time, system 255) so a .tar.gz is the same bytes on every host; a raw DEFLATE member is never inflated past its stated size; zstd_decompress, used by nothing, is gone Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/compress.cppm | 25 +++++++++---------- src/pack/compress_codecs.cpp | 48 ++++++++++++++++++++++++++---------- src/pack/zip.cppm | 4 +-- 3 files changed, 49 insertions(+), 28 deletions(-) diff --git a/src/pack/compress.cppm b/src/pack/compress.cppm index a32baa5eb..3e01ace24 100644 --- a/src/pack/compress.cppm +++ b/src/pack/compress.cppm @@ -8,9 +8,10 @@ // package is written the same way on every host and nothing is looked for on // PATH. // -// Deterministic by construction: a gzip member carries no name and no time, a -// zstd frame states its content size and its checksum, an xz stream checks -// with CRC64. Two packs of one tree produce the same bytes. +// Deterministic by construction: a gzip member carries no name, no time and the +// system byte 255 (unknown) whatever host wrote it, a zstd frame states its +// content size and its checksum, an xz stream checks with CRC64. Two packs of +// one tree produce the same bytes, on every host. export module mcpp.pack.compress; @@ -22,7 +23,9 @@ enum class Algorithm { None, Gzip, Zstd, Xz }; struct Choice { Algorithm algorithm = Algorithm::Gzip; - int level = 0; // 0: the codec's own default + // -1: the codec's own default (gzip 6, zstd 19, xz 6). Any other value is + // the level stated, honoured as it is: `xz:0` is xz's fastest preset. + int level = -1; bool stated = false; // written by the user (flag or manifest) }; @@ -44,18 +47,18 @@ std::expected file(const std::filesystem::path& in, // `in`, compressed under `c`. std::expected bytes(std::string_view in, Choice c); -// `in` as raw DEFLATE (a zip member's method 8), at `level` (0: default). +// `in` as raw DEFLATE (a zip member's method 8), at `level` (-1: zlib's +// default). std::expected deflate_raw(std::string_view in, int level); -// A zstd frame's content, for the readers of what mcpp wrote (the tests, and -// `--merge`, which reads a package it wrote). -std::expected zstd_decompress(std::string_view frame); - // What `bytes` compressed under `a`: a gzip member, a zstd frame, an xz // stream, or the bytes themselves. `--merge` reads the archives it merges so. std::expected decompress(std::string_view in, Algorithm a); // A raw DEFLATE stream's content, `size` bytes long (a zip member's method 8). +// A stream that holds more than `size` bytes is refused once it passes them, +// not inflated to its end: the size is the archive's word for the member, and +// a reader trusts no more of it than that. std::expected inflate_raw(std::string_view in, std::size_t size); } // namespace mcpp::pack::compress @@ -164,10 +167,6 @@ std::expected deflate_raw(std::string_view in, int lev return codec::deflate(in, level); } -std::expected zstd_decompress(std::string_view frame) { - return codec::unzstd(frame); -} - std::expected decompress(std::string_view in, Algorithm a) { switch (a) { case Algorithm::None: return std::string(in); diff --git a/src/pack/compress_codecs.cpp b/src/pack/compress_codecs.cpp index 93ca65883..3e41cddae 100644 --- a/src/pack/compress_codecs.cpp +++ b/src/pack/compress_codecs.cpp @@ -14,14 +14,23 @@ namespace mcpp::pack::compress::codec { namespace { -// zlib's deflate with `windowBits`: 15 + 16 is a gzip member (no name, no -// time: deflate writes neither unless a header is set), -15 is raw DEFLATE. +// zlib's deflate with `windowBits`: 15 + 16 is a gzip member, -15 is raw +// DEFLATE. A gzip member's header is stated rather than left to zlib, whose own +// header names the system zlib was built for (3 on Unix, 10 or 11 on Windows): +// no name, no time, the system 255 (unknown), so the same tree is the same +// bytes on every host. std::expected zlib_deflate(std::string_view in, int level, int windowBits) { z_stream s{}; - const int lvl = level == 0 ? Z_DEFAULT_COMPRESSION : level; + const int lvl = level < 0 ? Z_DEFAULT_COMPRESSION : level; if (deflateInit2(&s, lvl, Z_DEFLATED, windowBits, 8, Z_DEFAULT_STRATEGY) != Z_OK) return std::unexpected(std::string("zlib: deflateInit2 failed")); + gz_header header{}; + header.os = 255; + if (windowBits > 15 && deflateSetHeader(&s, &header) != Z_OK) { + deflateEnd(&s); + return std::unexpected(std::string("zlib: deflateSetHeader failed")); + } std::string out; out.resize(deflateBound(&s, static_cast(in.size())) + 64); s.next_in = reinterpret_cast(const_cast(in.data())); @@ -53,7 +62,7 @@ std::expected deflate(std::string_view in, int level) std::expected zstd(std::string_view in, int level) { ZSTD_CCtx* cx = ZSTD_createCCtx(); if (!cx) return std::unexpected(std::string("zstd: cannot create a context")); - ZSTD_CCtx_setParameter(cx, ZSTD_c_compressionLevel, level == 0 ? 19 : level); + ZSTD_CCtx_setParameter(cx, ZSTD_c_compressionLevel, level < 0 ? 19 : level); ZSTD_CCtx_setParameter(cx, ZSTD_c_checksumFlag, 1); ZSTD_CCtx_setParameter(cx, ZSTD_c_contentSizeFlag, 1); std::string out; @@ -82,7 +91,7 @@ std::expected xz(std::string_view in, int level) { out.resize(lzma_stream_buffer_bound(in.size())); std::size_t pos = 0; const lzma_ret rc = lzma_easy_buffer_encode( - static_cast(level == 0 ? 6 : level), LZMA_CHECK_CRC64, nullptr, + static_cast(level < 0 ? 6 : level), LZMA_CHECK_CRC64, nullptr, reinterpret_cast(in.data()), in.size(), reinterpret_cast(out.data()), &pos, out.size()); if (rc != LZMA_OK) return std::unexpected(std::format("xz: lzma_easy_buffer_encode returned {}", @@ -93,22 +102,35 @@ std::expected xz(std::string_view in, int level) { namespace { -// zlib's inflate until the stream ends, growing the output: a gzip member -// (windowBits 15 + 32 detects the header) or raw DEFLATE (-15). +// zlib's inflate until the stream ends: a gzip member (windowBits 15 + 32 +// detects the header), its output growing, or raw DEFLATE (-15) of a stated +// `size`, its output never more than that. std::expected zlib_inflate(std::string_view in, int windowBits, - std::size_t hint) { + std::optional size) { + if (in.size() > std::numeric_limits::max()) + return std::unexpected(std::string("zlib: an input over 4 GiB")); z_stream s{}; if (inflateInit2(&s, windowBits) != Z_OK) return std::unexpected(std::string("zlib: inflateInit2 failed")); std::string out; - out.resize(hint > 0 ? hint : in.size() * 4 + 1024); + // One byte past a stated size: a stream that fills it holds more than it + // said. + out.resize(size ? *size + 1 : in.size() * 4 + 1024); s.next_in = reinterpret_cast(const_cast(in.data())); s.avail_in = static_cast(in.size()); int rc = Z_OK; while (rc == Z_OK) { - if (s.total_out >= out.size()) out.resize(out.size() * 2); + if (s.total_out >= out.size()) { + if (size) { + inflateEnd(&s); + return std::unexpected(std::format("zlib: the stream holds more than its {} bytes", + *size)); + } + out.resize(out.size() * 2); + } s.next_out = reinterpret_cast(out.data() + s.total_out); - s.avail_out = static_cast(out.size() - s.total_out); + s.avail_out = static_cast(std::min(out.size() - s.total_out, + std::numeric_limits::max())); rc = ::inflate(&s, Z_NO_FLUSH); } const auto written = s.total_out; @@ -121,11 +143,11 @@ std::expected zlib_inflate(std::string_view in, int wi } // namespace std::expected gunzip(std::string_view in) { - return zlib_inflate(in, 15 + 32, 0); + return zlib_inflate(in, 15 + 32, std::nullopt); } std::expected inflate(std::string_view in, std::size_t size) { - return zlib_inflate(in, -15, size + 1); + return zlib_inflate(in, -15, size); } std::expected unxz(std::string_view in) { diff --git a/src/pack/zip.cppm b/src/pack/zip.cppm index 3a85456ff..4cd2a03ca 100644 --- a/src/pack/zip.cppm +++ b/src/pack/zip.cppm @@ -49,7 +49,7 @@ struct Entry { // How the members are written. // // deflate each member is DEFLATEd (method 8) when that makes it smaller, -// and stored otherwise; `level` 0 is zlib's default. +// and stored otherwise; `level` -1 is zlib's default. // align a STORED member's data starts at a multiple of this many bytes // from the start of the archive, the local header's extra field // padding the difference (as zipalign does): a reader maps the @@ -57,7 +57,7 @@ struct Entry { // is stored, whatever `deflate` says. struct WriteOptions { bool deflate = false; - int level = 0; + int level = -1; std::uint32_t align = 0; }; From 14305640566759ca814b434c4e9567a60031dbae Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 10:47:03 +0900 Subject: [PATCH 40/54] pack.distribution: unpack creates nothing outside its root and writes the tar itself; launchers expand the runner as mcpp run does; --merge trusts no recorded digest and replaces nothing that is not a package - unpack reads zip and tar in-process: every header within the archive, every zip member's CRC-32 checked, a member that is absolute or holds `..`, one under a link member, and a link whose target leaves the tree refused before anything is written; links are made after every file. Reads ustar, pax and GNU long names (what earlier releases wrote through GNU tar) and `./` roots. - write_tar writes ustar (pax for long names and targets) in path order, owner 0 and unnamed, time 0, modes 0755/0644/0777: two packs of one tree are the same bytes. One tree walk serves the tar, the zip, the metadata and the single file's payload, and reports what it cannot read. - Launchers: a word's first `{}` is the image (`--image={}`), appended when no word holds one; POSIX words single-quoted, `$0` followed through links by a readlink loop; a .cmd doubles `%` and `launchable` refuses a word holding a double quote or a line break. arch_dir, used by nothing, is gone. - role_of: a library is `.so` or `.so.`, not any name holding `.so`. - read() refuses a path that is not a normalised one inside the package; merge() checks each file against its recorded sha256 before copying and compares the computed digests; the output is new or an archive that is a package and none of the inputs; `pkg/` names the directory `pkg`. - The single file's launcher runs nothing from a cache directory (mcpp/, single-file/, the digest's tree) that is a link or another user's. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/distribution.cppm | 1044 +++++++++++++++++++------ tests/unit/test_pack_distribution.cpp | 407 +++++++++- 2 files changed, 1197 insertions(+), 254 deletions(-) diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index f8a8022a9..44bf50b6d 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -15,8 +15,6 @@ import std; import mcpp.libs.json; import mcpp.pack.compress; import mcpp.pack.zip; -import mcpp.run; -import mcpplibs.os.platform; export namespace mcpp::pack::distribution { @@ -29,7 +27,8 @@ std::string sha256_file(const std::filesystem::path& file); // ─── containers ─────────────────────────────────────────────────────────── // `root` (a staged tree) as a tar whose one top-level directory is -// `root.filename()`, compressed as `c` says. +// `root.filename()`, compressed as `c` says. The tar is written here, the same +// bytes for the same tree on every host (see `tar_bytes`). std::expected write_tar(const std::filesystem::path& root, const std::filesystem::path& archive, mcpp::pack::compress::Choice c); @@ -42,7 +41,14 @@ std::expected write_zip(const std::filesystem::path& root, // A package this command wrote, as a tree under `into`: a directory as it is, // an archive (`.zip`, `.tar`, `.tar.gz`, `.tgz`, `.tar.zst`, `.tar.xz`) -// extracted. Returns the tree's root, the archive's one top-level directory. +// extracted into `into`, which is new or empty. Returns the tree's root, the +// archive's one top-level directory. +// +// An archive is read here, not by a program on PATH, and creates nothing +// outside `into`: every header is read within the archive, every zip member's +// CRC-32 checked; a member whose path is absolute or holds `..`, one that lies +// under a link member, and a link whose target leaves the tree are refused, +// before anything is written; the links are made after every file. std::expected unpack(const std::filesystem::path& package, const std::filesystem::path& into); @@ -50,19 +56,25 @@ unpack(const std::filesystem::path& package, const std::filesystem::path& into); struct Launch { std::string name; // bin/, bin/.cmd - std::vector runner; // the platform's `runner.cmd`, `{}` the image + // The platform's `runner.cmd`, expanded as `mcpp run` expands it: the + // first `{}` of a word is the image (`{}`, `--image={}`), and the image is + // appended when no word holds one. + std::vector runner; std::map entry; // arch -> the entry image's file name // The loader a self-contained package carries under // `libexec//-/`; empty when it carries none. std::string loader; }; +// Whether both launchers can state `l`: a `.cmd` keeps a word literal inside +// double quotes, so a word holding a double quote or a line break cannot be +// written in one. Checked before a launcher is written; `windows_launcher` of +// a `Launch` this refuses is a `.cmd` that says so and runs nothing. +std::expected launchable(const Launch& l); + std::string posix_launcher(const Launch& l); std::string windows_launcher(const Launch& l); -// The architecture directory a machine name selects, as the launchers map it. -std::string_view arch_dir(std::string_view machine); - // ─── mcpp-package.json (SPEC-014 §4.5) ──────────────────────────────────── inline constexpr std::string_view kFileName = "mcpp-package.json"; @@ -99,6 +111,8 @@ struct Stated { std::expected describe(const std::filesystem::path& root, Package pkg, const Stated& stated); std::string to_json(const Package& p); +// `root/mcpp-package.json`, every path it names (`files[].path`, the entries, +// the program) checked to be a normalised path inside the package. std::expected read(const std::filesystem::path& root); // The `-` directories of `root/libexec//`: the systems whose @@ -118,7 +132,8 @@ std::expected tree_payload(const std::filesystem::path // carried payload (`mcpp_carried_begin` .. `mcpp_carried_end`, a zstd frame // of `tree_payload`), extracts it once into the user's cache under the // payload's sha256 (`mcpp_carried_digest`), and executes the program at -// `mcpp_carried_name` with its own arguments. +// `mcpp_carried_name` with its own arguments. On POSIX it runs nothing from a +// cache directory another user owns. std::string launcher_manifest(std::string_view name, std::string_view version); std::string_view launcher_main(); @@ -150,6 +165,9 @@ struct MergeRequest { mcpp::pack::compress::Choice compression; }; +// The inputs' trees as one package. Every file is checked against the digest +// its package states before it is copied; the output is new, or an archive +// that is a package and none of the inputs (`dir` writes a new directory only). std::expected merge(const MergeRequest& req); } // namespace mcpp::pack::distribution @@ -244,6 +262,45 @@ bool executable_file(const fs::path& p) { return (perms & fs::perms::owner_exec) != fs::perms::none; } +// One entry of a staged tree: its path under the root (`/`-separated), what it +// is (`d` a directory, `f` a file, `x` an executable file, `l` a link) and a +// link's target. +struct Node { + std::string path; + char kind = 'f'; + std::string target; +}; + +// Every entry under `root`, links not followed, in path order (bytewise: a +// directory before what it holds). Whatever cannot be read is an error, not +// an entry left out. +std::expected, std::string> walk(const fs::path& root) { + std::vector out; + std::error_code ec; + for (auto it = fs::recursive_directory_iterator(root, ec); + !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { + std::error_code entry; + const auto st = it->symlink_status(entry); + Node n; + n.path = it->path().lexically_relative(root).generic_string(); + if (fs::is_symlink(st)) { + n.kind = 'l'; + n.target = fs::read_symlink(it->path(), entry).generic_string(); + } else if (fs::is_directory(st)) { + n.kind = 'd'; + } else if (fs::is_regular_file(st)) { + n.kind = executable_file(it->path()) ? 'x' : 'f'; + } else if (!entry) { + continue; // a socket, a device: nothing a package holds + } + if (entry) return std::unexpected(std::format("cannot read '{}': {}", it->path().string(), entry.message())); + out.push_back(std::move(n)); + } + if (ec) return std::unexpected(std::format("cannot walk '{}': {}", root.string(), ec.message())); + std::sort(out.begin(), out.end(), [](const Node& a, const Node& b) { return a.path < b.path; }); + return out; +} + } // namespace std::string sha256(std::string_view bytes) { @@ -266,69 +323,135 @@ std::string sha256_file(const fs::path& file) { // ─── containers ─────────────────────────────────────────────────────────── +namespace { + +// ── the tar mcpp writes ── +// +// POSIX ustar, written here rather than by a `tar` on PATH, whose bytes are +// its own: GNU tar records the owner, the group and the times of the files, +// and the order the file system lists a directory in, so two packs of one +// tree differed. These bytes are the same on every host: the entries in path +// order, every owner 0 and unnamed, every time 0, the mode 0755 (a directory, +// an executable file), 0644 (another file) or 0777 (a link). A name or a link +// target over the 100 bytes of its field, or a size over the 8 GiB of its, is +// stated in a pax extended header ahead of the entry (POSIX.1-2001), which +// every tar reads. + +constexpr std::size_t kBlock = 512; + +std::size_t padding(std::size_t n) { return (kBlock - n % kBlock) % kBlock; } + +// `v` in octal, `width - 1` digits and a NUL. +void octal(char* field, std::size_t width, std::uint64_t v) { + for (std::size_t i = width - 1; i-- > 0; v >>= 3) field[i] = static_cast('0' + (v & 7)); + field[width - 1] = '\0'; +} + +void tar_header(std::string& out, std::string_view name, char type, unsigned mode, + std::uint64_t size, std::string_view link) { + std::array h{}; + std::copy_n(name.data(), std::min(name.size(), 100), h.data()); + octal(h.data() + 100, 8, mode); + octal(h.data() + 108, 8, 0); // uid + octal(h.data() + 116, 8, 0); // gid + octal(h.data() + 124, 12, size <= 077777777777ull ? size : 0); + octal(h.data() + 136, 12, 0); // mtime + std::fill_n(h.data() + 148, 8, ' '); + h[156] = type; + std::copy_n(link.data(), std::min(link.size(), 100), h.data() + 157); + std::copy_n("ustar\0" "00", 8, h.data() + 257); + octal(h.data() + 329, 8, 0); // devmajor + octal(h.data() + 337, 8, 0); // devminor + unsigned sum = 0; + for (char ch : h) sum += static_cast(ch); + octal(h.data() + 148, 7, sum); // six digits and a NUL; the space stays + out.append(h.data(), h.size()); +} + +// One pax record: " =\n", the length counting its own +// digits. +std::string pax_record(std::string_view key, std::string_view value) { + const auto rest = key.size() + value.size() + 3; + auto length = rest + 1; + while (length != rest + std::to_string(length).size()) length = rest + std::to_string(length).size(); + return std::format("{} {}={}\n", length, key, value); +} + +void tar_entry(std::string& out, std::string_view name, char type, unsigned mode, + std::string_view data, std::string_view link) { + std::string pax; + if (name.size() > 100) pax += pax_record("path", name); + if (link.size() > 100) pax += pax_record("linkpath", link); + if (data.size() > 077777777777ull) pax += pax_record("size", std::to_string(data.size())); + if (!pax.empty()) { + tar_header(out, "././@PaxHeader", 'x', 0644, pax.size(), {}); + out += pax; + out.append(padding(pax.size()), '\0'); + } + tar_header(out, name, type, mode, data.size(), link); + out += data; + out.append(padding(data.size()), '\0'); +} + +// The tar of `root`: its directory, then every entry under it. +std::expected tar_bytes(const fs::path& root) { + auto nodes = walk(root); + if (!nodes) return std::unexpected(nodes.error()); + const auto top = (root.has_filename() ? root : root.parent_path()).filename().generic_string(); + std::string out; + tar_entry(out, top + "/", '5', 0755, {}, {}); + for (auto const& n : *nodes) { + const auto name = top + "/" + n.path; + if (n.kind == 'd') { tar_entry(out, name + "/", '5', 0755, {}, {}); continue; } + if (n.kind == 'l') { tar_entry(out, name, '2', 0777, {}, n.target); continue; } + auto data = slurp(root / fs::path(n.path)); + if (!data) return std::unexpected(std::format("cannot read '{}'", (root / n.path).string())); + tar_entry(out, name, '0', n.kind == 'x' ? 0755 : 0644, *data, {}); + } + out.append(2 * kBlock, '\0'); + return out; +} + +} // namespace + std::expected write_tar(const fs::path& root, const fs::path& archive, mcpp::pack::compress::Choice c) { - std::error_code ec; - if (archive.has_parent_path()) fs::create_directories(archive.parent_path(), ec); - // Written uncompressed by tar, compressed by mcpp's codec: `.tar.zst` and - // `.tar.xz` need no program on PATH, and a `.tar.gz` is the same bytes on - // every host. `--force-local` on Windows: GNU tar reads `C:/x.tar` as the - // rsh form `host:path`. - auto plain = archive; - plain += ".mcpp-tar"; - std::vector argv{"tar"}; - if constexpr (mcpplibs::os::is_windows) argv.push_back("--force-local"); - argv.insert(argv.end(), {"-cf", plain.string(), "-C", root.parent_path().string(), - root.filename().string()}); - auto r = mcpp::run::execute(argv); - if (r.exit_code != 0) { - fs::remove(plain, ec); - return std::unexpected(std::format("tar exited with {}: {}", r.exit_code, r.output)); - } - auto packed = mcpp::pack::compress::file(plain, archive, c); - fs::remove(plain, ec); - if (!packed) return std::unexpected(packed.error()); + auto tar = tar_bytes(root); + if (!tar) return std::unexpected(tar.error()); + auto packed = mcpp::pack::compress::bytes(*tar, c); + if (!packed) return std::unexpected(std::format("{}: {}", archive.string(), packed.error())); + if (!spit(archive, *packed)) return std::unexpected(std::format("cannot write '{}'", archive.string())); return {}; } std::expected write_zip(const fs::path& root, const fs::path& archive, mcpp::pack::compress::Choice c, std::string_view program) { - std::error_code ec; + auto nodes = walk(root); + if (!nodes) return std::unexpected(nodes.error()); + const auto top = (root.has_filename() ? root : root.parent_path()).filename().generic_string(); + // In path order, as `walk` returns the tree. std::vector entries; - const auto wrapper = root.filename().string(); - for (auto it = fs::recursive_directory_iterator(root, ec); - !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { - const auto& e = *it; + for (auto const& n : *nodes) { + if (n.kind == 'd' || (n.kind == 'l' && n.target.empty())) continue; mcpp::pack::zip::Entry entry; - entry.name = wrapper + "/" + e.path().lexically_relative(root).generic_string(); - entry.source = e.path(); - if (e.is_symlink(ec)) { - entry.symlinkTarget = fs::read_symlink(e.path(), ec).generic_string(); - if (entry.symlinkTarget.empty()) continue; - if (e.is_directory(ec)) it.disable_recursion_pending(); - } else if (!e.is_regular_file(ec)) { - continue; - } else { - entry.executable = executable_file(e.path()) - || (!program.empty() && e.path().filename() == program); - } + entry.name = top + "/" + n.path; + entry.source = root / fs::path(n.path); + entry.symlinkTarget = n.target; + entry.executable = n.kind == 'x' + || (n.kind == 'f' && !program.empty() && entry.source.filename() == program); entries.push_back(std::move(entry)); } - // Deterministic order; a comparator rather than a projection onto an - // imported type's member (mcpp.pack's run_pe says why). - std::sort(entries.begin(), entries.end(), - [](const mcpp::pack::zip::Entry& a, const mcpp::pack::zip::Entry& b) { - return a.name < b.name; - }); mcpp::pack::zip::WriteOptions o; o.deflate = c.algorithm != mcpp::pack::compress::Algorithm::None; - o.level = c.algorithm == mcpp::pack::compress::Algorithm::Gzip ? c.level : 0; + o.level = c.algorithm == mcpp::pack::compress::Algorithm::Gzip ? c.level : -1; return mcpp::pack::zip::write(archive, entries, o); } namespace { +// ── what an archive holds ── + std::uint16_t get16(std::string_view b, std::size_t at) { return static_cast(static_cast(b[at]) | (static_cast(b[at + 1]) << 8)); @@ -337,70 +460,350 @@ std::uint32_t get32(std::string_view b, std::size_t at) { return static_cast(get16(b, at)) | (static_cast(get16(b, at + 2)) << 16); } -// A relative, `/`-separated name with no `..`: what an archive may create. -bool safe_member(std::string_view name) { - if (name.empty() || name.front() == '/' || name.find('\\') != std::string_view::npos - || (name.size() > 1 && name[1] == ':')) - return false; - std::size_t at = 0; - while (at <= name.size()) { - auto slash = name.find('/', at); - if (slash == std::string_view::npos) slash = name.size(); - if (name.substr(at, slash - at) == "..") return false; +// `s` split at every `/`, empty parts kept. +std::vector parts(std::string_view s) { + std::vector out; + for (std::size_t at = 0; at <= s.size();) { + auto slash = s.find('/', at); + if (slash == std::string_view::npos) slash = s.size(); + out.push_back(s.substr(at, slash - at)); at = slash + 1; } + return out; +} + +// What can never be a path inside a tree on every system: empty, absolute +// (`/x`, `C:`), with a backslash or a NUL. +bool foreign(std::string_view p) { + return p.empty() || p.front() == '/' || p.find('\\') != std::string_view::npos + || p.find('\0') != std::string_view::npos || (p.size() > 1 && p[1] == ':'); +} + +// The path inside the extraction root a member's name names, normalised: +// relative, `/`-separated, without `.` or empty parts or a trailing `/`; empty +// for the root itself (`./`, which `tar -C dir .` writes). None when it would +// leave the root. A `..` is refused wherever it is, `a/../b` too, which stays +// inside as words: through a link member `a` it would not, and no archive mcpp +// writes holds one. +std::optional member_path(std::string_view name) { + if (foreign(name)) return std::nullopt; + std::string out; + for (auto part : parts(name)) { + if (part.empty() || part == ".") continue; + if (part == "..") return std::nullopt; + if (!out.empty()) out += '/'; + out += part; + } + return out; +} + +// Whether the link member at `path` whose target is `target` resolves inside +// the extraction root: the target is relative and, walked from the link's +// directory, never climbs above the root and goes on from no link member (one +// may be where it ends: that link's own target is checked the same way). +bool link_inside(std::string_view path, std::string_view target, const std::set& links) { + if (foreign(target)) return false; + // Where the walk is, a path under the root (empty: the root itself); the + // link's directory first. + const auto slash = path.rfind('/'); + std::string here(slash == std::string_view::npos ? std::string_view() : path.substr(0, slash)); + for (auto part : parts(target)) { + if (part.empty() || part == ".") continue; + if (!here.empty() && links.contains(here)) return false; + if (part == "..") { + if (here.empty()) return false; + const auto up = here.rfind('/'); + here.resize(up == std::string::npos ? 0 : up); + } else { + here += (here.empty() ? "" : "/") + std::string(part); + } + } return true; } +// One member of an archive: its path (`member_path`) and what it is. A file's +// bytes are as stored (`method` 8: DEFLATEd); a link's target is read at once. +struct Member { + std::string path; + char kind = 'f'; // 'd' directory, 'f' file, 'l' link + bool executable = false; + std::string_view stored; + std::uint16_t method = 0; + std::uint64_t size = 0; + std::optional crc; // a zip member's CRC-32 + std::string target; +}; + +// A file member's content: inflated if it was deflated, its length and its +// CRC-32 (a zip's) checked against the archive's. +std::expected contents(const Member& m) { + std::string data; + if (m.method == 8) { + auto d = mcpp::pack::compress::inflate_raw(m.stored, static_cast(m.size)); + if (!d) return std::unexpected(std::format("'{}': {}", m.path, d.error())); + data = std::move(*d); + } else { + data.assign(m.stored); + } + if (data.size() != m.size) + return std::unexpected(std::format("'{}' holds {} bytes; the archive states {}", m.path, + data.size(), m.size)); + if (m.crc && mcpp::pack::zip::crc32(data) != *m.crc) + return std::unexpected(std::format("'{}' fails its CRC-32: {:08x}, and the archive states {:08x}", + m.path, mcpp::pack::zip::crc32(data), *m.crc)); + return data; +} + // The members of a zip, read through its central directory (stored and -// deflated, as mcpp writes them). -std::expected unzip(std::string_view b, const fs::path& into) { +// deflated, as mcpp writes them). Every field is read within the archive: an +// offset or a length that points past it is refused, never followed. +std::expected, std::string> zip_members(std::string_view b) { + auto within = [&](std::uint64_t at, std::uint64_t n) { return at <= b.size() && n <= b.size() - at; }; + if (b.size() < 22) return std::unexpected(std::string("not a zip: shorter than its end record")); + // The end of the central directory: the last 22 bytes, or as far before + // them as its comment (65535 bytes at most) reaches. std::size_t eocd = std::string_view::npos; - for (std::size_t i = b.size() >= 22 ? b.size() - 22 : 0; ; --i) { - if (get32(b, i) == 0x06054b50) { eocd = i; break; } - if (i == 0 || b.size() - i > 22 + 65535) break; + const std::size_t lowest = b.size() - 22 > 65535 ? b.size() - 22 - 65535 : 0; + for (std::size_t i = b.size() - 22;; --i) { + if (get32(b, i) == 0x06054b50 && within(i + 22, get16(b, i + 20))) { eocd = i; break; } + if (i == lowest) break; } if (eocd == std::string_view::npos) return std::unexpected(std::string("not a zip: no end of central directory")); const auto count = get16(b, eocd + 10); std::size_t at = get32(b, eocd + 16); + if (count == 0xFFFF || at == 0xFFFFFFFF) + return std::unexpected(std::string("a ZIP64 archive, which mcpp neither writes nor reads")); + std::vector out; for (std::uint16_t n = 0; n < count; ++n) { - if (at + 46 > b.size() || get32(b, at) != 0x02014b50) - return std::unexpected(std::string("a zip's central directory is malformed")); - const auto method = get16(b, at + 10); - const auto packed = get32(b, at + 20); - const auto size = get32(b, at + 24); + if (!within(at, 46) || get32(b, at) != 0x02014b50) + return std::unexpected(std::format("the central directory is malformed at {}", at)); + const auto flags = get16(b, at + 8), method = get16(b, at + 10); + const auto crc = get32(b, at + 16), packed = get32(b, at + 20), size = get32(b, at + 24); const auto nameLen = get16(b, at + 28), extraLen = get16(b, at + 30), commentLen = get16(b, at + 32); - const auto attrs = get32(b, at + 38); + const auto mode = get32(b, at + 38) >> 16; const auto local = get32(b, at + 42); - const std::string name(b.substr(at + 46, nameLen)); - at += 46 + nameLen + extraLen + commentLen; - if (!safe_member(name)) return std::unexpected(std::format("a zip member names '{}'", name)); - if (name.ends_with('/')) { std::error_code ec; fs::create_directories(into / name, ec); continue; } - const auto lnameLen = get16(b, local + 26), lextraLen = get16(b, local + 28); - const auto dataAt = static_cast(local) + 30 + lnameLen + lextraLen; - if (dataAt + packed > b.size()) return std::unexpected(std::format("zip member '{}' is truncated", name)); - std::string data(b.substr(dataAt, packed)); - if (method == 8) { - auto d = mcpp::pack::compress::inflate_raw(data, size); - if (!d) return std::unexpected(std::format("zip member '{}': {}", name, d.error())); - data = std::move(*d); - } else if (method != 0) { - return std::unexpected(std::format("zip member '{}' uses method {}", name, method)); + if (!within(at + 46, std::uint64_t(nameLen) + extraLen + commentLen)) + return std::unexpected(std::string("the central directory is truncated")); + const auto name = b.substr(at + 46, nameLen); + at += 46 + std::size_t(nameLen) + extraLen + commentLen; + const bool dir = name.ends_with('/') || (mode & 0170000u) == 0040000u; + auto path = member_path(name); + if (!path || (path->empty() && !dir)) + return std::unexpected(std::format("a member names '{}', which is not a path inside the archive", + name)); + if (path->empty()) continue; // the root, there already + if (flags & 1) return std::unexpected(std::format("'{}' is encrypted", name)); + if (method != 0 && method != 8) + return std::unexpected(std::format("'{}' uses method {}", name, method)); + if (packed == 0xFFFFFFFF || size == 0xFFFFFFFF || local == 0xFFFFFFFF) + return std::unexpected(std::format("'{}' is a ZIP64 member, which mcpp neither writes nor reads", + name)); + Member m; + m.path = std::move(*path); + if (dir) { + m.kind = 'd'; + out.push_back(std::move(m)); + continue; } - const auto mode = attrs >> 16; - const auto target = into / fs::path(name); - std::error_code ec; - fs::create_directories(target.parent_path(), ec); + if (!within(local, 30) || get32(b, local) != 0x04034b50) + return std::unexpected(std::format("'{}' has no local header at {}", name, local)); + const std::uint64_t dataAt = std::uint64_t(local) + 30 + get16(b, local + 26) + get16(b, local + 28); + if (!within(dataAt, packed)) return std::unexpected(std::format("'{}' is truncated", name)); + m.stored = b.substr(static_cast(dataAt), packed); + m.method = method; + m.size = size; + m.crc = crc; + m.executable = (mode & 0100u) != 0; if ((mode & 0170000u) == 0120000u) { - fs::create_symlink(data, target, ec); - if (ec) return std::unexpected(std::format("cannot link '{}': {}", target.string(), ec.message())); + auto target = contents(m); + if (!target) return std::unexpected(target.error()); + m.kind = 'l'; + m.target = std::move(*target); + m.stored = {}; + } + out.push_back(std::move(m)); + } + return out; +} + +// A numeric field of a tar header: octal digits (after spaces, ended by a +// space or a NUL), or base-256 when the first byte's high bit is set. +std::optional tar_number(std::string_view f) { + std::uint64_t v = 0; + if (!f.empty() && (static_cast(f.front()) & 0x80)) { + if (static_cast(f.front()) != 0x80) return std::nullopt; // negative, or too large + for (char ch : f.substr(1)) { + if (v >> 56) return std::nullopt; + v = v << 8 | static_cast(ch); + } + return v; + } + std::size_t i = f.find_first_not_of(' '); + if (i == std::string_view::npos) return 0; + for (; i < f.size() && f[i] >= '0' && f[i] <= '7'; ++i) { + if (v >> 60) return std::nullopt; + v = v * 8 + static_cast(f[i] - '0'); + } + if (i < f.size() && f[i] != ' ' && f[i] != '\0') return std::nullopt; + return v; +} + +// A text field of a tar header, up to its first NUL. +std::string_view tar_text(std::string_view f) { return f.substr(0, f.find('\0')); } + +// The records of a pax extended header that name the next entry: its path, +// its link's target, its size. The others (times, owners) are read past. +std::expected pax_records(std::string_view data, std::string& path, std::string& link, + std::optional& size) { + while (!data.empty()) { + const auto space = data.find(' '); + std::uint64_t length = 0; + if (space == std::string_view::npos + || std::from_chars(data.data(), data.data() + space, length).ec != std::errc{} + || length <= space + 1 || length > data.size() || data[length - 1] != '\n') + return std::unexpected(std::string("a pax header is malformed")); + const auto record = data.substr(space + 1, length - space - 2); + data.remove_prefix(length); + const auto eq = record.find('='); + if (eq == std::string_view::npos) return std::unexpected(std::string("a pax record has no '='")); + const auto key = record.substr(0, eq), value = record.substr(eq + 1); + if (key == "path") path = value; + else if (key == "linkpath") link = value; + else if (key == "size") { + std::uint64_t v = 0; + if (std::from_chars(value.data(), value.data() + value.size(), v).ec != std::errc{}) + return std::unexpected(std::format("a pax size '{}' is not a number", value)); + size = v; + } + } + return {}; +} + +// The members of a tar: ustar, its pax extended headers and GNU's long names +// (the archives earlier releases wrote through GNU tar). Every header's +// checksum is checked and every length read within the archive. +std::expected, std::string> tar_members(std::string_view b) { + std::vector out; + std::string longName, longLink; // the next entry's, from a pax or GNU header + std::optional longSize; + std::size_t at = 0; + while (true) { + if (at == b.size()) break; // no end blocks: as GNU tar, the end of the archive + if (b.size() - at < kBlock) return std::unexpected(std::string("the tar ends inside a header")); + const auto h = b.substr(at, kBlock); + if (h.find_first_not_of('\0') == std::string_view::npos) break; // the end of the archive + unsigned sum = 0; + int signedSum = 0; + for (std::size_t i = 0; i < kBlock; ++i) { + const char ch = i >= 148 && i < 156 ? ' ' : h[i]; + sum += static_cast(ch); + signedSum += static_cast(ch); + } + const auto stated = tar_number(h.substr(148, 8)); + if (!stated || (*stated != sum && *stated != static_cast(signedSum))) + return std::unexpected(std::format("the tar header at {} fails its checksum", at)); + const char type = h[156]; + const bool meta = type == 'x' || type == 'g' || type == 'L' || type == 'K'; + auto size = tar_number(h.substr(124, 12)); + if (!size) return std::unexpected(std::format("the tar header at {} states no size", at)); + if (longSize && !meta) size = longSize; + at += kBlock; + if (*size > b.size() - at || padding(static_cast(*size)) > b.size() - at - *size) + return std::unexpected(std::format("the tar entry at {} is truncated", at - kBlock)); + const auto data = b.substr(at, static_cast(*size)); + at += static_cast(*size) + padding(static_cast(*size)); + if (type == 'g') continue; + if (type == 'x') { + if (auto r = pax_records(data, longName, longLink, longSize); !r) return std::unexpected(r.error()); + continue; + } + if (type == 'L') { longName = tar_text(data); continue; } + if (type == 'K') { longLink = tar_text(data); continue; } + std::string name(longName); + if (name.empty()) { + name = tar_text(h.substr(0, 100)); + // POSIX ustar's prefix; GNU's own magic ("ustar ") puts other + // things there. + if (const auto prefix = tar_text(h.substr(345, 155)); h.substr(257, 6) == std::string_view("ustar\0", 6) + && !prefix.empty()) + name = std::string(prefix) + "/" + name; + } + const std::string link = longLink.empty() ? std::string(tar_text(h.substr(157, 100))) : longLink; + longName.clear(); + longLink.clear(); + longSize.reset(); + const bool dir = type == '5' || ((type == '0' || type == '\0') && name.ends_with('/')); + auto path = member_path(name); + if (!path || (path->empty() && !dir)) + return std::unexpected(std::format("a member names '{}', which is not a path inside the archive", + name)); + if (path->empty()) continue; // the root, there already + Member m; + m.path = std::move(*path); + if (dir) { + m.kind = 'd'; + } else if (type == '0' || type == '\0' || type == '7') { + m.stored = data; + m.size = data.size(); + m.executable = (tar_number(h.substr(100, 8)).value_or(0) & 0100u) != 0; + } else if (type == '2') { + m.kind = 'l'; + m.target = link; + } else if (type == '1') { + return std::unexpected(std::format("'{}' is a hard link, which mcpp does not write", name)); + } else { + return std::unexpected(std::format("'{}' is a tar entry of type '{}', which a package does not " + "hold", name, type)); + } + out.push_back(std::move(m)); + } + return out; +} + +// The members created under `into`, and nothing outside it. Every path is +// checked before anything is written: none twice, none under a link member, +// no link whose target leaves the tree. Then the directories and the files +// are written, and the links made last, so no member is written through one. +std::expected place(const std::vector& members, const fs::path& into) { + std::set paths, links; + for (auto const& m : members) { + if (!paths.insert(m.path).second) return std::unexpected(std::format("'{}' is in the archive twice", m.path)); + if (m.kind == 'l') links.insert(m.path); + } + for (auto const& m : members) { + for (auto slash = m.path.find('/'); slash != std::string::npos; slash = m.path.find('/', slash + 1)) + if (links.contains(m.path.substr(0, slash))) + return std::unexpected(std::format("'{}' lies under the link '{}'", m.path, m.path.substr(0, slash))); + if (m.kind == 'l' && !link_inside(m.path, m.target, links)) + return std::unexpected(std::format("the link '{}' points to '{}', outside the package", m.path, + m.target)); + } + std::error_code ec; + for (auto const& m : members) { + const auto target = into / fs::path(m.path); + if (m.kind == 'd') { + fs::create_directories(target, ec); + if (ec) return std::unexpected(std::format("cannot create '{}': {}", target.string(), ec.message())); continue; } - if (!spit(target, data)) return std::unexpected(std::format("cannot write '{}'", target.string())); - if (mode & 0100u) + if (m.kind != 'f') continue; + auto data = contents(m); + if (!data) return std::unexpected(data.error()); + if (!spit(target, *data)) return std::unexpected(std::format("cannot write '{}'", target.string())); + if (m.executable) fs::permissions(target, fs::perms::owner_exec | fs::perms::group_exec | fs::perms::others_exec, fs::perm_options::add, ec); } + for (auto const& m : members) { + if (m.kind != 'l') continue; + const auto link = into / fs::path(m.path); + fs::create_directories(link.parent_path(), ec); + if (fs::is_directory(link.parent_path() / fs::path(m.target), ec)) + fs::create_directory_symlink(fs::path(m.target), link, ec); + else + fs::create_symlink(fs::path(m.target), link, ec); + if (ec) return std::unexpected(std::format("cannot link '{}': {}", link.string(), ec.message())); + } return {}; } @@ -412,39 +815,42 @@ std::expected unpack(const fs::path& package, const fs::p if (!fs::is_regular_file(package, ec)) return std::unexpected(std::format("'{}' is neither a package directory nor an archive", package.string())); - fs::create_directories(into, ec); const auto name = package.filename().string(); + using mcpp::pack::compress::Algorithm; + const bool zip = name.ends_with(".zip"); + const std::optional a = name.ends_with(".tar.gz") || name.ends_with(".tgz") ? Algorithm::Gzip + : name.ends_with(".tar.zst") ? Algorithm::Zstd + : name.ends_with(".tar.xz") ? Algorithm::Xz + : name.ends_with(".tar") ? std::optional(Algorithm::None) + : std::nullopt; + if (!zip && !a) + return std::unexpected(std::format( + "'{}' is not a package this command writes (.zip, .tar, .tar.gz, .tar.zst, .tar.xz, " + "or a directory)", name)); + if (fs::exists(fs::symlink_status(into, ec)) && !(fs::is_directory(fs::symlink_status(into, ec)) + && fs::is_empty(into, ec))) + return std::unexpected(std::format("'{}' is not an empty directory to unpack {} into", + into.string(), name)); + fs::create_directories(into, ec); auto bytes = slurp(package); if (!bytes) return std::unexpected(std::format("cannot read '{}'", package.string())); - if (name.ends_with(".zip")) { - if (auto r = unzip(*bytes, into); !r) return std::unexpected(std::format("{}: {}", name, r.error())); + // The members point into the archive's bytes, or into its tar. + std::string plain; + std::expected, std::string> members; + if (zip) { + members = zip_members(*bytes); } else { - using mcpp::pack::compress::Algorithm; - Algorithm a = name.ends_with(".tar.gz") || name.ends_with(".tgz") ? Algorithm::Gzip - : name.ends_with(".tar.zst") ? Algorithm::Zstd - : name.ends_with(".tar.xz") ? Algorithm::Xz - : name.ends_with(".tar") ? Algorithm::None - : static_cast(-1); - if (static_cast(a) == -1) - return std::unexpected(std::format( - "'{}' is not a package this command writes (.zip, .tar, .tar.gz, .tar.zst, .tar.xz, " - "or a directory)", name)); - auto plain = mcpp::pack::compress::decompress(*bytes, a); - if (!plain) return std::unexpected(std::format("{}: {}", name, plain.error())); - const auto tarFile = into / ".mcpp-merge.tar"; - if (!spit(tarFile, *plain)) return std::unexpected(std::format("cannot write '{}'", tarFile.string())); - std::vector argv{"tar"}; - if constexpr (mcpplibs::os::is_windows) argv.push_back("--force-local"); - argv.insert(argv.end(), {"-xf", tarFile.string(), "-C", into.string()}); - auto r = mcpp::run::execute(argv); - fs::remove(tarFile, ec); - if (r.exit_code != 0) return std::unexpected(std::format("tar -xf {} exited with {}: {}", name, - r.exit_code, r.output)); + auto t = mcpp::pack::compress::decompress(*bytes, *a); + if (!t) return std::unexpected(std::format("{}: {}", name, t.error())); + plain = std::move(*t); + members = tar_members(plain); } - // The archive's one top-level directory. + if (!members) return std::unexpected(std::format("{}: {}", name, members.error())); + if (auto r = place(*members, into); !r) return std::unexpected(std::format("{}: {}", name, r.error())); + // The archive's one top-level directory (a link there is none). std::vector tops; for (auto const& e : fs::directory_iterator(into, ec)) - if (e.is_directory(ec)) tops.push_back(e.path()); + if (fs::is_directory(e.symlink_status(ec))) tops.push_back(e.path()); if (tops.size() != 1) return std::unexpected(std::format("'{}' has {} top-level directories; a package has one", name, tops.size())); @@ -453,12 +859,6 @@ std::expected unpack(const fs::path& package, const fs::p // ─── launchers ──────────────────────────────────────────────────────────── -std::string_view arch_dir(std::string_view machine) { - if (machine == "x86_64" || machine == "amd64" || machine == "AMD64") return "x86_64"; - if (machine == "aarch64" || machine == "arm64" || machine == "ARM64") return "aarch64"; - return {}; -} - namespace { // A word of a POSIX shell command, single-quoted. @@ -472,32 +872,118 @@ std::string sh_word(std::string_view w) { return out; } -} // namespace +// Text of a .cmd inside double quotes: `%` doubled, the one character a batch +// file reads there. +std::string cmd_quoted(std::string_view w) { + std::string out; + for (char ch : w) { + if (ch == '%') out += "%%"; + else out.push_back(ch); + } + return out; +} -std::string posix_launcher(const Launch& l) { +// Text of a .cmd outside quotes (an `echo`): `%` doubled and every character +// the command line reads escaped with `^`. +std::string cmd_bare(std::string_view w) { + std::string out; + for (char ch : w) { + if (ch == '%') out += "%%"; + else if (std::string_view("&|<>^()").find(ch) != std::string_view::npos) out += {'^', ch}; + else out.push_back(ch); + } + return out; +} + +// How a launcher's language writes the runner's words: `literal` a piece of +// a word as text, `image` the image's variable, `around` what encloses each +// word. +struct Quoting { + std::string (*literal)(std::string_view); + std::string_view image; + std::string_view around; +}; + +// The runner's arguments, its words after the program, by `mcpp run`'s rule +// (mcpp.freestanding's `expand`): a word's first `{}` is the image, in +// `--image={}` as well as in `{}`, and the image is appended when no word +// holds one. +std::string runner_args(const std::vector& runner, const Quoting& q) { + std::string out; + bool placed = false; + for (std::size_t i = 1; i < runner.size(); ++i) { + const std::string_view w = runner[i]; + out += std::format(" {}", q.around); + if (const auto at = w.find("{}"); at == std::string_view::npos) { + out += q.literal(w); + } else { + placed = true; + if (at > 0) out += q.literal(w.substr(0, at)); + out += q.image; + if (at + 2 < w.size()) out += q.literal(w.substr(at + 2)); + } + out += q.around; + } + if (!placed) out += std::format(" {0}{1}{0}", q.around, q.image); + return out; +} + +std::string arch_list(const Launch& l) { std::string arches; for (auto const& [arch, _] : l.entry) arches += (arches.empty() ? "" : " ") + arch; + return arches; +} + +} // namespace + +std::expected launchable(const Launch& l) { + std::vector words{l.name, l.loader}; + words.insert(words.end(), l.runner.begin(), l.runner.end()); + for (auto const& [arch, file] : l.entry) words.insert(words.end(), {arch, file}); + for (auto w : words) + if (w.find_first_of(std::string_view("\"\r\n\0", 4)) != std::string_view::npos) + return std::unexpected(std::format( + "the launcher of {} cannot state '{}': a word of a .cmd holds no double quote and no " + "line break", l.name, w)); + return {}; +} + +std::string posix_launcher(const Launch& l) { std::string cases; for (auto const& [arch, file] : l.entry) - cases += std::format(" {}) image=\"$top/lib/{}/{}\" ;;\n", arch, arch, file); - // The runner's argv: its program found in the package, then on PATH; `{}` - // the image; every other word as written. - std::string args; - for (std::size_t i = 1; i < l.runner.size(); ++i) - args += l.runner[i] == "{}" ? std::string(" \"$image\"") : " " + sh_word(l.runner[i]); - const std::string program = l.runner.empty() ? std::string("{}") : l.runner.front(); - std::string find; - if (!l.loader.empty()) - find = std::format( - "runner=\"$top/libexec/{0}/$os-$arch/{0}\"\n" - "[ -x \"$runner\" ] || runner={1}\n", l.loader, sh_word(program)); - else - find = std::format("runner={}\n", sh_word(program)); + cases += std::format(" {}) entry={} ;;\n", sh_word(arch), sh_word(file)); + // The runner: its program found in the package, then on PATH; none, the + // image itself. + std::string find, args = " \"$@\""; + if (l.runner.empty()) { + find = "runner=$image\n"; + } else { + args = runner_args(l.runner, { sh_word, "\"$image\"", "" }) + args; + find = std::format("runner={}\n", sh_word(l.runner.front())); + if (!l.loader.empty()) + find = std::format( + "loader={0}\n" + "runner=\"$top/libexec/$loader/$os-$arch/$loader\"\n" + "[ -x \"$runner\" ] || runner={1}\n", sh_word(l.loader), sh_word(l.runner.front())); + } return std::format( "#!/bin/sh\n" - "# Generated by mcpp pack (SPEC-014 §4.3): runs {0} on this machine's\n" - "# architecture through the platform's runner.\n" - "top=$(CDPATH= cd -- \"$(dirname -- \"$0\")/..\" && pwd)\n" + "# Generated by mcpp pack (SPEC-014 §4.3): runs the package's program on this\n" + "# machine's architecture through the platform's runner.\n" + "#\n" + "# The package's directory is found through the links a person may have made\n" + "# to this file, by `readlink` without -f, which not every system has.\n" + "self=$0\n" + "n=0\n" + "while [ -h \"$self\" ] && [ \"$n\" -lt 40 ]; do\n" + " link=$(readlink -- \"$self\") || break\n" + " case $link in\n" + " /*) self=$link ;;\n" + " *) self=$(dirname -- \"$self\")/$link ;;\n" + " esac\n" + " n=$((n + 1))\n" + "done\n" + "top=$(CDPATH= cd -- \"$(dirname -- \"$self\")/..\" && pwd)\n" "case \"$(uname -m)\" in\n" " x86_64|amd64|AMD64) arch=x86_64 ;;\n" " aarch64|arm64|ARM64) arch=aarch64 ;;\n" @@ -510,32 +996,38 @@ std::string posix_launcher(const Launch& l) { "esac\n" "case \"$arch\" in\n" "{1}" - " *) echo \"{0}: this machine is $arch; the package holds {2}\" >&2; exit 126 ;;\n" + " *) printf '%s: this machine is %s; the package holds %s\\n' {0} \"$arch\" {2} >&2; exit 126 ;;\n" "esac\n" + "image=\"$top/lib/$arch/$entry\"\n" "{3}" - "exec \"$runner\"{4} \"$@\"\n", - l.name, cases, arches, find, args); + "exec \"$runner\"{4}\n", + sh_word(l.name), cases, sh_word(arch_list(l)), find, args); } std::string windows_launcher(const Launch& l) { - std::string arches; - for (auto const& [arch, _] : l.entry) arches += (arches.empty() ? "" : " ") + arch; + if (!launchable(l)) + return "@echo off\r\n" + "rem Generated by mcpp pack (SPEC-014 §4.3).\r\n" + "echo This package's runner has a word a .cmd cannot state; run it from a POSIX shell. 1>&2\r\n" + "exit /b 126\r\n"; std::string picks; for (auto const& [arch, file] : l.entry) - picks += std::format("if \"%arch%\"==\"{0}\" set \"image=%top%\\lib\\{0}\\{1}\"\r\n", arch, file); - std::string args; - for (std::size_t i = 1; i < l.runner.size(); ++i) - args += l.runner[i] == "{}" ? std::string(" \"%image%\"") : " \"" + l.runner[i] + "\""; - const std::string program = l.runner.empty() ? std::string("{}") : l.runner.front(); - std::string find = std::format("set \"runner={}\"\r\n", program); - if (!l.loader.empty()) - find = std::format( - "set \"runner=%top%\\libexec\\{0}\\windows-%arch%\\{0}.exe\"\r\n" - "if not exist \"%runner%\" set \"runner={1}\"\r\n", l.loader, program); + picks += std::format("if \"%arch%\"==\"{0}\" set \"image=%top%\\lib\\{0}\\{1}\"\r\n", + cmd_quoted(arch), cmd_quoted(file)); + std::string find = "set \"runner=%image%\"\r\n", args; + if (!l.runner.empty()) { + args = runner_args(l.runner, { cmd_quoted, "%image%", "\"" }); + const auto program = cmd_quoted(l.runner.front()); + find = std::format("set \"runner={}\"\r\n", program); + if (!l.loader.empty()) + find = std::format( + "set \"runner=%top%\\libexec\\{0}\\windows-%arch%\\{0}.exe\"\r\n" + "if not exist \"%runner%\" set \"runner={1}\"\r\n", cmd_quoted(l.loader), program); + } return std::format( "@echo off\r\n" - "rem Generated by mcpp pack (SPEC-014 §4.3): runs {0} on this machine's\r\n" - "rem architecture through the platform's runner.\r\n" + "rem Generated by mcpp pack (SPEC-014 §4.3): runs the package's program on this\r\n" + "rem machine's architecture through the platform's runner.\r\n" "setlocal\r\n" "for %%I in (\"%~dp0..\") do set \"top=%%~fI\"\r\n" "set \"machine=%PROCESSOR_ARCHITECTURE%\"\r\n" @@ -552,13 +1044,23 @@ std::string windows_launcher(const Launch& l) { "{3}" "\"%runner%\"{4} %*\r\n" "exit /b %ERRORLEVEL%\r\n", - l.name, picks, arches, find, args); + cmd_bare(l.name), picks, cmd_bare(arch_list(l)), find, args); } // ─── mcpp-package.json ──────────────────────────────────────────────────── namespace { +// A shared object's name: `.so`, or `.so.` and a version of digits and dots +// (`libz.so.1.3`); not a name that merely holds `.so` (`notes.sol`). +bool shared_object(std::string_view file) { + const auto at = file.rfind(".so"); + if (at == std::string_view::npos) return false; + const auto rest = file.substr(at + 3); + return rest.empty() + || (rest.size() > 1 && rest.front() == '.' && rest.find_first_not_of("0123456789.") == std::string_view::npos); +} + std::string role_of(std::string_view path, bool executable) { auto ends = [&](std::string_view s) { return path.ends_with(s); }; const auto file = path.substr(path.rfind('/') == std::string_view::npos ? 0 : path.rfind('/') + 1); @@ -566,8 +1068,7 @@ std::string role_of(std::string_view path, bool executable) { if (file.starts_with("README") || file.starts_with("LICENSE") || file == "HOST-REQUIREMENTS" || path.starts_with("share/licenses/")) return "metadata"; - if (ends(".dll") || ends(".dylib") || path.find(".so") != std::string_view::npos - || ends(".okso")) + if (ends(".dll") || ends(".dylib") || shared_object(file) || ends(".okso")) return "library"; if (ends(".oke")) return "image"; if (path.starts_with("bin/") && (executable || ends(".exe") || ends(".js") || ends(".wasm"))) @@ -610,26 +1111,21 @@ std::string to_json(const Package& p) { } std::expected describe(const fs::path& root, Package pkg, const Stated& stated) { - std::error_code ec; pkg.files.clear(); - for (auto it = fs::recursive_directory_iterator(root, ec); - !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { - const auto& e = *it; - if (e.is_symlink(ec)) { if (e.is_directory(ec)) it.disable_recursion_pending(); continue; } - if (!e.is_regular_file(ec)) continue; - const auto rel = e.path().lexically_relative(root).generic_string(); - if (rel == kFileName) continue; + auto nodes = walk(root); + if (!nodes) return std::unexpected(nodes.error()); + // The files, in path order; a link is not a file of the package. + for (auto const& n : *nodes) { + if ((n.kind != 'f' && n.kind != 'x') || n.path == kFileName) continue; File f; - f.path = rel; - if (auto r = stated.role.find(rel); r != stated.role.end()) f.role = r->second; - else f.role = role_of(rel, executable_file(e.path())); - if (auto l = stated.leg.find(rel); l != stated.leg.end()) f.leg = l->second; + f.path = n.path; + if (auto r = stated.role.find(n.path); r != stated.role.end()) f.role = r->second; + else f.role = role_of(n.path, n.kind == 'x'); + if (auto l = stated.leg.find(n.path); l != stated.leg.end()) f.leg = l->second; else if (pkg.legs.size() == 1 && f.role != "metadata" && f.role != "resource") f.leg = pkg.legs.front(); - f.sha256 = sha256_file(e.path()); + f.sha256 = sha256_file(root / fs::path(n.path)); pkg.files.push_back(std::move(f)); } - std::sort(pkg.files.begin(), pkg.files.end(), - [](const File& a, const File& b) { return a.path < b.path; }); if (!spit(root / kFileName, to_json(pkg))) return std::unexpected(std::format("cannot write '{}'", (root / kFileName).string())); return pkg; @@ -671,6 +1167,16 @@ std::expected read(const fs::path& root) { if (p.schema != 1) return std::unexpected(std::format("{} states schema {}; this mcpp reads schema 1", (root / kFileName).string(), p.schema)); + // Every path it names is one inside the package, written as `describe` + // writes it: what reads the package (--merge, the index entry) opens them. + std::vector named; + for (auto const& f : p.files) named.push_back(f.path); + for (auto const& [_, e] : p.entry) named.push_back(e); + if (!p.program.empty()) named.push_back(p.program); + for (auto n : named) + if (n.empty() || member_path(n) != n) + return std::unexpected(std::format("{} names '{}', which is not a path inside the package", + (root / kFileName).string(), n)); return p; } @@ -694,42 +1200,24 @@ std::vector loader_systems(const fs::path& root) { // ─── a single file ──────────────────────────────────────────────────────── std::expected tree_payload(const fs::path& root) { - struct Item { std::string path; char kind; std::string data; }; - std::vector items; - std::error_code ec; - for (auto it = fs::recursive_directory_iterator(root, ec); - !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { - const auto& e = *it; - Item item; - item.path = e.path().lexically_relative(root).generic_string(); - if (e.is_symlink(ec)) { - item.kind = 'l'; - item.data = fs::read_symlink(e.path(), ec).generic_string(); - if (e.is_directory(ec)) it.disable_recursion_pending(); - } else if (e.is_directory(ec)) { - item.kind = 'd'; - } else if (e.is_regular_file(ec)) { - item.kind = executable_file(e.path()) ? 'x' : 'f'; - auto data = slurp(e.path()); - if (!data) return std::unexpected(std::format("cannot read '{}'", e.path().string())); - item.data = std::move(*data); - } else { - continue; - } - items.push_back(std::move(item)); - } - if (ec) return std::unexpected(std::format("cannot walk '{}': {}", root.string(), ec.message())); - std::sort(items.begin(), items.end(), [](const Item& a, const Item& b) { return a.path < b.path; }); + auto nodes = walk(root); + if (!nodes) return std::unexpected(nodes.error()); std::string out = "MCPPTREE1\n"; auto put = [&](std::uint64_t v, int bytes) { for (int i = 0; i < bytes; ++i) out.push_back(static_cast((v >> (8 * i)) & 0xff)); }; - for (auto const& i : items) { - out.push_back(i.kind); - put(i.path.size(), 4); - out += i.path; - put(i.data.size(), 8); - out += i.data; + for (auto const& n : *nodes) { + std::string data = n.target; + if (n.kind == 'f' || n.kind == 'x') { + auto bytes = slurp(root / fs::path(n.path)); + if (!bytes) return std::unexpected(std::format("cannot read '{}'", (root / n.path).string())); + data = std::move(*bytes); + } + out.push_back(n.kind); + put(n.path.size(), 4); + out += n.path; + put(data.size(), 8); + out += data; } return out; } @@ -756,6 +1244,11 @@ std::string_view launcher_main() { // file is written, so a directory that is there is complete. Every later run // executes the program at once. The frame carries its content's checksum, and // zstd refuses a payload that does not match it. +// +// Nothing runs from a directory another user owns. On POSIX the cache's +// directories this program makes (`mcpp/`, `single-file/`) and the tree it runs +// from are this user's and no links, or the program refuses: another user who +// owned one could have put a program of theirs under the payload's name. #include import std; @@ -841,6 +1334,18 @@ std::optional extract(const fs::path& into) { return std::nullopt; } +// Whether `dir` is a directory, not a link, and (on POSIX) this user's. +bool own(const fs::path& dir) { + std::error_code ec; + if (!fs::is_directory(fs::symlink_status(dir, ec))) return false; + if constexpr (os::is_posix) return os::fs::owner_of(dir) == os::fs::owner::this_user; + else return true; +} + +int foreign(const fs::path& dir) { + return fail(std::format("{} is not a directory of this user's; nothing is run from it", dir.string())); +} + } // namespace int main(int argc, char** argv) { @@ -849,8 +1354,10 @@ int main(int argc, char** argv) { const fs::path base = *cache / "mcpp" / "single-file"; const fs::path root = base / std::string(kDigest); std::error_code ec; + fs::create_directories(base, ec); + if (!own(base.parent_path())) return foreign(base.parent_path()); + if (!own(base)) return foreign(base); if (!fs::exists(root / ".mcpp-complete", ec)) { - fs::create_directories(base, ec); if constexpr (os::is_posix) fs::permissions(base, fs::perms::owner_all, fs::perm_options::replace, ec); const auto self = os::process::self(); @@ -865,6 +1372,7 @@ int main(int argc, char** argv) { return fail(std::format("cannot place the program under {}", root.string())); } } + if (!own(root)) return foreign(root); const fs::path program = root / fs::path(std::string(kName)); os::process::spawn_options o; o.program = program; @@ -1017,6 +1525,19 @@ std::expected xpkg_descriptor(const std::vector // ─── --merge ────────────────────────────────────────────────────────────── +namespace { + +// `p` absolute, normalised, and with a name: `pkg/` is `pkg`, `.` the current +// directory. +fs::path named(const fs::path& p) { + std::error_code ec; + auto out = fs::absolute(p, ec).lexically_normal(); + if (!out.has_filename() && out.has_relative_path()) out = out.parent_path(); + return out; +} + +} // namespace + std::expected merge(const MergeRequest& req) { if (req.inputs.size() < 2) return std::unexpected(std::string("--merge reads two packages or more")); @@ -1026,9 +1547,11 @@ std::expected merge(const MergeRequest& req) { fs::create_directories(work, ec); struct Cleanup { fs::path p; ~Cleanup() { std::error_code e; fs::remove_all(p, e); } } cleanup{work}; + std::vector inputs; + for (auto const& i : req.inputs) inputs.push_back(named(i)); std::vector> trees; - for (std::size_t i = 0; i < req.inputs.size(); ++i) { - auto root = unpack(req.inputs[i], work / std::format("in{}", i)); + for (std::size_t i = 0; i < inputs.size(); ++i) { + auto root = unpack(inputs[i], work / std::format("in{}", i)); if (!root) return std::unexpected(root.error()); auto pkg = read(*root); if (!pkg) return std::unexpected(pkg.error()); @@ -1049,6 +1572,51 @@ std::expected merge(const MergeRequest& req) { if (r) return std::unexpected(*r); } const auto wrapper = trees.front().first.filename(); + + // The container: as asked, else as the output's name says, else as the + // first input was written. + std::string format = req.format; + const auto outName = req.output.filename().string(); + const auto firstName = !req.output.empty() && (outName.ends_with(".zip") || outName.find(".tar") != std::string::npos) + ? outName : inputs.front().filename().string(); + if (format.empty()) + format = firstName.ends_with(".zip") ? "zip" + : firstName.find(".tar") != std::string::npos ? "tar" + : fs::is_directory(inputs.front(), ec) ? "dir" : "tar"; + if (format != "tar" && format != "zip" && format != "dir") + return std::unexpected(std::format("--merge writes tar, zip or dir; '{}' is a format a package " + "provides, which reads a staged tree: merge to dir first", + format)); + auto compression = req.compression; + if (!compression.stated && format == "tar") { + using mcpp::pack::compress::Algorithm; + compression.algorithm = firstName.ends_with(".tar.zst") ? Algorithm::Zstd + : firstName.ends_with(".tar.xz") ? Algorithm::Xz + : firstName.ends_with(".tar") ? Algorithm::None + : Algorithm::Gzip; + } + const auto ext = format == "zip" ? std::string(".zip") : format == "dir" ? std::string() + : std::string(mcpp::pack::compress::tar_extension(compression.algorithm)); + const fs::path dest = named(req.output.empty() ? fs::path(wrapper.string() + ext) : req.output); + + // What is there already is replaced only when it is an archive this + // command wrote and none of the inputs: --merge never removes a directory, + // nor a file it cannot read as a package. + for (std::size_t i = 0; i < inputs.size(); ++i) + if (dest == inputs[i] || fs::equivalent(dest, inputs[i], ec)) + return std::unexpected(std::format("--merge: the output {} is the input {}", dest.string(), + req.inputs[i].string())); + if (const auto st = fs::symlink_status(dest, ec); fs::exists(st)) { + if (format == "dir") + return std::unexpected(std::format("--merge: {} exists; a merged directory is written as a new " + "one", dest.string())); + auto old = fs::is_regular_file(st) ? unpack(dest, work / "existing") + : std::unexpected(std::string("not a file")); + if (!old || !read(*old)) + return std::unexpected(std::format("--merge: {} exists and is not a package; it is not replaced", + dest.string())); + } + const auto out = work / "merged" / wrapper; fs::create_directories(out, ec); Package merged = first; @@ -1064,16 +1632,25 @@ std::expected merge(const MergeRequest& req) { for (auto const& s : pkg.systems) if (std::ranges::find(merged.systems, s) == merged.systems.end()) merged.systems.push_back(s); for (auto const& [arch, e] : pkg.entry) merged.entry[arch] = e; + // Each file as it is, not as its package says: a file changed after the + // package was written is refused, and two inputs agree on a file when + // its bytes do. for (auto const& f : pkg.files) { + const auto digest = sha256_file(root / fs::path(f.path)); + if (digest != f.sha256) + return std::unexpected(std::format( + "--merge: '{}' in {} has sha256 {}, and its {} states {}", f.path, + req.inputs[i].string(), digest.empty() ? std::string("(no such file)") : digest, kFileName, + f.sha256)); if (auto it = seen.find(f.path); it != seen.end()) { - if (it->second.first != f.sha256) + if (it->second.first != digest) return std::unexpected(std::format( "--merge: '{}' is in {} with sha256 {} and in {} with sha256 {}; one package " "holds one of each file", f.path, it->second.second, it->second.first, - req.inputs[i].string(), f.sha256)); + req.inputs[i].string(), digest)); continue; } - seen[f.path] = {f.sha256, req.inputs[i].string()}; + seen[f.path] = {digest, req.inputs[i].string()}; stated.role[f.path] = f.role; if (!f.leg.empty()) stated.leg[f.path] = f.leg; fs::create_directories((out / f.path).parent_path(), ec); @@ -1085,39 +1662,12 @@ std::expected merge(const MergeRequest& req) { std::ranges::sort(merged.systems); if (auto d = describe(out, merged, stated); !d) return std::unexpected(d.error()); - // The container: as asked, else as the output's name says, else as the - // first input was written. - std::string format = req.format; - const auto outName = req.output.filename().string(); - const auto firstName = !req.output.empty() && (outName.ends_with(".zip") || outName.find(".tar") != std::string::npos) - ? outName : req.inputs.front().filename().string(); - if (format.empty()) - format = firstName.ends_with(".zip") ? "zip" - : firstName.find(".tar") != std::string::npos ? "tar" - : fs::is_directory(req.inputs.front(), ec) ? "dir" : "tar"; - auto compression = req.compression; - if (!compression.stated && format == "tar") { - using mcpp::pack::compress::Algorithm; - compression.algorithm = firstName.ends_with(".tar.zst") ? Algorithm::Zstd - : firstName.ends_with(".tar.xz") ? Algorithm::Xz - : firstName.ends_with(".tar") ? Algorithm::None - : Algorithm::Gzip; - } - fs::path dest = req.output; - const auto ext = format == "zip" ? std::string(".zip") : format == "dir" ? std::string() - : std::string(mcpp::pack::compress::tar_extension(compression.algorithm)); - if (dest.empty()) dest = fs::current_path(ec) / (wrapper.string() + ext); if (format == "dir") { - fs::remove_all(dest, ec); fs::create_directories(dest.parent_path(), ec); fs::copy(out, dest, fs::copy_options::recursive | fs::copy_options::copy_symlinks, ec); if (ec) return std::unexpected(std::format("cannot write '{}': {}", dest.string(), ec.message())); return dest; } - if (format != "tar" && format != "zip") - return std::unexpected(std::format("--merge writes tar, zip or dir; '{}' is a format a package " - "provides, which reads a staged tree: merge to dir first", - format)); auto r = format == "zip" ? write_zip(out, dest, compression) : write_tar(out, dest, compression); if (!r) return std::unexpected(r.error()); return dest; diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index b251d37a5..b891aa0e7 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -1,6 +1,7 @@ // The distribution half of `mcpp pack` (SPEC-014): the codecs, the zip writer's -// deflated and aligned members, SHA-256, the launchers, the package metadata -// and the single file's payload. +// deflated and aligned members, SHA-256, the tar mcpp writes, what `unpack` +// refuses to create, the launchers, the package metadata, `--merge` and the +// single file's payload. #include import std; @@ -29,6 +30,11 @@ void write(const fs::path& p, std::string_view text) { std::ofstream(p, std::ios::binary) << text; } +std::string slurp(const fs::path& p) { + std::ifstream in(p, std::ios::binary); + return std::string(std::istreambuf_iterator(in), {}); +} + std::uint32_t le32(std::string_view b, std::size_t at) { return static_cast(static_cast(b[at])) | static_cast(static_cast(b[at + 1])) << 8 @@ -36,6 +42,13 @@ std::uint32_t le32(std::string_view b, std::size_t at) { | static_cast(static_cast(b[at + 3])) << 24; } +// A zip of `entries` at `dir/name`, members stored. +fs::path zip_of(const fs::path& dir, std::string_view name, const std::vector& entries) { + const auto out = dir / name; + EXPECT_TRUE(mcpp::pack::zip::write(out, entries)); + return out; +} + } // namespace TEST(PackCompress, ParsesAlgorithmsAndLevels) { @@ -52,6 +65,37 @@ TEST(PackCompress, ParsesAlgorithmsAndLevels) { EXPECT_EQ(c::tar_extension(c::Algorithm::None), ".tar"); } +// An unstated level is -1, the codec's default; every stated one is honoured, +// `xz:0` and `zstd:1` too. +TEST(PackCompress, AStatedLevelIsHonoured) { + EXPECT_EQ(c::Choice{}.level, -1); + EXPECT_EQ(c::parse("xz")->level, -1); + EXPECT_EQ(c::parse("xz:0")->level, 0); + EXPECT_EQ(c::parse("zstd:1")->level, 1); + std::string text; + for (int i = 0; i < 4000; ++i) text += std::format("{} ", i * 7919 % 10007); + auto fastest = c::bytes(text, *c::parse("xz:0")); + auto preset = c::bytes(text, *c::parse("xz")); + ASSERT_TRUE(fastest && preset); + EXPECT_NE(*fastest, *preset) << "xz:0 was written at the default preset"; + auto z1 = c::bytes(text, *c::parse("zstd:1")); + auto z19 = c::bytes(text, *c::parse("zstd")); + ASSERT_TRUE(z1 && z19); + EXPECT_NE(*z1, *z19) << "zstd:1 was written at the default level"; +} + +// A gzip member names no file, no time and the system 255, whatever host +// wrote it. +TEST(PackCompress, AGzipHeaderIsTheSameOnEveryHost) { + auto gz = c::bytes("the same bytes", c::Choice{}); + ASSERT_TRUE(gz); + ASSERT_GE(gz->size(), 10u); + EXPECT_EQ(static_cast((*gz)[0]), 0x1f); + EXPECT_EQ(static_cast((*gz)[3]), 0) << "a name, a comment or an extra field"; + EXPECT_EQ(le32(*gz, 4), 0u) << "a modification time"; + EXPECT_EQ(static_cast((*gz)[9]), 255) << "the system byte"; +} + TEST(PackCompress, EveryCodecRoundTrips) { std::string text; for (int i = 0; i < 4000; ++i) text += std::format("line {} of a package\n", i % 37); @@ -65,7 +109,7 @@ TEST(PackCompress, EveryCodecRoundTrips) { ASSERT_TRUE(back) << c::name(a); EXPECT_EQ(*back, text) << c::name(a); } - auto raw = c::deflate_raw(text, 0); + auto raw = c::deflate_raw(text, -1); ASSERT_TRUE(raw); auto back = c::inflate_raw(*raw, text.size()); ASSERT_TRUE(back); @@ -131,17 +175,127 @@ TEST(PackDistribution, LaunchersPickTheArchitectureAndTheRunner) { l.runner = { "okrun", "{}" }; l.entry = { { "x86_64", "hello.oke" }, { "aarch64", "hello.oke" } }; l.loader = "okrun"; + ASSERT_TRUE(d::launchable(l)); const auto sh = d::posix_launcher(l); EXPECT_TRUE(sh.starts_with("#!/bin/sh\n")); EXPECT_NE(sh.find("x86_64|amd64|AMD64) arch=x86_64"), std::string::npos); - EXPECT_NE(sh.find("image=\"$top/lib/aarch64/hello.oke\""), std::string::npos); - EXPECT_NE(sh.find("runner=\"$top/libexec/okrun/$os-$arch/okrun\""), std::string::npos); + EXPECT_NE(sh.find("'aarch64') entry='hello.oke' ;;"), std::string::npos); + EXPECT_NE(sh.find("image=\"$top/lib/$arch/$entry\""), std::string::npos); + EXPECT_NE(sh.find("runner=\"$top/libexec/$loader/$os-$arch/$loader\""), std::string::npos); EXPECT_NE(sh.find("exec \"$runner\" \"$image\" \"$@\""), std::string::npos); + // The package is found through a link to the launcher, without readlink -f. + EXPECT_NE(sh.find("readlink -- \"$self\""), std::string::npos); + EXPECT_EQ(sh.find("readlink -f"), std::string::npos); const auto cmd = d::windows_launcher(l); EXPECT_NE(cmd.find("PROCESSOR_ARCHITEW6432"), std::string::npos); EXPECT_NE(cmd.find("libexec\\okrun\\windows-%arch%\\okrun.exe"), std::string::npos); - EXPECT_EQ(d::arch_dir("arm64"), "aarch64"); - EXPECT_EQ(d::arch_dir("AMD64"), "x86_64"); + EXPECT_NE(cmd.find("\"%runner%\" \"%image%\" %*"), std::string::npos); +} + +// The runner's words expand as `mcpp run` expands them: a word's `{}` is the +// image inside the word as well as alone; with no `{}` the image is appended. +TEST(PackDistribution, LaunchersExpandTheRunnerAsMcppRunDoes) { + d::Launch l; + l.name = "hello"; + l.entry = { { "x86_64", "hello.oke" } }; + l.runner = { "okrun", "--image={}", "it's" }; + EXPECT_NE(d::posix_launcher(l).find("exec \"$runner\" '--image='\"$image\" 'it'\\''s' \"$@\""), + std::string::npos) << d::posix_launcher(l); + EXPECT_NE(d::windows_launcher(l).find("\"%runner%\" \"--image=%image%\" \"it's\" %*"), std::string::npos) + << d::windows_launcher(l); + l.runner = { "okrun", "--quiet" }; + EXPECT_NE(d::posix_launcher(l).find("exec \"$runner\" '--quiet' \"$image\" \"$@\""), std::string::npos) + << d::posix_launcher(l); + EXPECT_NE(d::windows_launcher(l).find("\"%runner%\" \"--quiet\" \"%image%\" %*"), std::string::npos) + << d::windows_launcher(l); +} + +// A .cmd doubles `%` in what it states literally, and a word it cannot state +// (a double quote) is refused: the .cmd written for it runs nothing. +TEST(PackDistribution, AWindowsLauncherEscapesWhatItStates) { + d::Launch l; + l.name = "hello"; + l.entry = { { "x86_64", "hello.oke" } }; + l.runner = { "ok%run", "--load=50%", "{}" }; + const auto cmd = d::windows_launcher(l); + EXPECT_NE(cmd.find("set \"runner=ok%%run\""), std::string::npos) << cmd; + EXPECT_NE(cmd.find("\"%runner%\" \"--load=50%%\" \"%image%\" %*"), std::string::npos) << cmd; + l.runner = { "okrun", "--say=\"hi\"", "{}" }; + auto ok = d::launchable(l); + ASSERT_FALSE(ok); + EXPECT_NE(ok.error().find("double quote"), std::string::npos) << ok.error(); + const auto refused = d::windows_launcher(l); + EXPECT_EQ(refused.find("%runner%"), std::string::npos) << refused; + EXPECT_NE(refused.find("exit /b 126"), std::string::npos) << refused; +} + +// The POSIX launcher, run through a link to it from another directory, finds +// its package and starts the runner with the image spliced into its word. +TEST(PackDistribution, APosixLauncherRunsThroughALink) { + if (!fs::exists("/bin/sh")) GTEST_SKIP() << "no /bin/sh"; + std::string arch; + { + Tmp probe; + if (std::system(std::format("uname -m > '{}'", (probe.path / "m").string()).c_str()) != 0) + GTEST_SKIP() << "no uname"; + arch = slurp(probe.path / "m"); + } + arch = arch.starts_with("x86_64") || arch.starts_with("amd64") ? "x86_64" + : arch.starts_with("aarch64") || arch.starts_with("arm64") ? "aarch64" : ""; + if (arch.empty()) GTEST_SKIP() << "no launcher row for this machine"; + Tmp t; + const auto pkg = t.path / "my pkg"; + d::Launch l; + l.name = "hello"; + l.entry = { { "x86_64", "hello.oke" }, { "aarch64", "hello.oke" } }; + l.runner = { "fakerun", "--image={}" }; + write(pkg / "bin/hello", d::posix_launcher(l)); + write(pkg / "lib" / arch / "hello.oke", "image"); + write(t.path / "path/fakerun", "#!/bin/sh\nprintf '%s|' \"$@\"\n"); + for (auto f : { pkg / "bin/hello", t.path / "path/fakerun" }) + fs::permissions(f, fs::perms::owner_all, fs::perm_options::add); + fs::create_directories(t.path / "elsewhere"); + fs::create_symlink(pkg / "bin/hello", t.path / "elsewhere/hello"); + const auto out = t.path / "out"; + const auto rc = std::system(std::format("PATH='{}':\"$PATH\" '{}' 'a b' c > '{}' 2>&1", + (t.path / "path").string(), (t.path / "elsewhere/hello").string(), + out.string()).c_str()); + EXPECT_EQ(rc, 0) << slurp(out); + EXPECT_EQ(slurp(out), std::format("--image={}|a b|c|", (pkg / "lib" / arch / "hello.oke").string())); +} + +// A shared object is `.so` or `.so.`; a name that merely holds `.so` +// is not one. +TEST(PackDistribution, ALibraryIsNamedSoOrSoVersion) { + Tmp t; + const auto root = t.path / "app-1.0"; + for (auto f : { "lib/libz.so", "lib/libz.so.1.3", "share/notes.sol", "share/a.so.x", "share/b.socket" }) + write(root / f, "x"); + d::Package p; + p.name = "app"; + p.version = "1.0"; + auto described = d::describe(root, p, {}); + ASSERT_TRUE(described) << described.error(); + std::map roles; + for (auto const& f : described->files) roles[f.path] = f.role; + EXPECT_EQ(roles["lib/libz.so"], "library"); + EXPECT_EQ(roles["lib/libz.so.1.3"], "library"); + EXPECT_EQ(roles["share/notes.sol"], "resource"); + EXPECT_EQ(roles["share/a.so.x"], "resource"); + EXPECT_EQ(roles["share/b.socket"], "resource"); +} + +// `read` refuses a document that names a path outside the package. +TEST(PackDistribution, ReadRefusesAPathOutsideThePackage) { + Tmp t; + for (auto bad : { "../outside", "/etc/passwd", "a/../../b", "./bin/x", "bin\\\\x", "" }) { + write(t.path / "mcpp-package.json", + std::format(R"({{"schema": 1, "name": "x", "version": "1", "files": [{{"path": "{}", "role": "resource", "sha256": ""}}]}})", + bad)); + auto r = d::read(t.path); + ASSERT_FALSE(r) << bad; + EXPECT_NE(r.error().find("not a path inside the package"), std::string::npos) << r.error(); + } } TEST(PackDistribution, PackageMetadataDescribesEveryFile) { @@ -185,3 +339,242 @@ TEST(PackDistribution, TreePayloadIsOrderedRecords) { const auto a = payload->find("a"), b = payload->find("b/x"); EXPECT_LT(a, b); } + +// ─── the tar mcpp writes ────────────────────────────────────────────────── + +namespace { + +// A tree with a directory, an executable, a link, and a name and a link target +// past the 100 bytes of a ustar field. +fs::path sample_tree(const fs::path& dir) { + const auto root = dir / "app-1.0"; + write(root / "bin/app", "#!/bin/sh\necho app\n"); + fs::permissions(root / "bin/app", fs::perms::owner_exec, fs::perm_options::add); + write(root / "share/data.txt", "data"); + write(root / "share" / std::string(60, 'd') / (std::string(60, 'e') + ".txt"), "deep"); + fs::create_directories(root / "empty"); + fs::create_symlink("data.txt", root / "share/link"); + fs::create_symlink(std::string(60, 'd') + "/" + std::string(60, 'e') + ".txt", root / "share/far"); + return root; +} + +const c::Choice kPlain{ c::Algorithm::None, -1, false }; + +} // namespace + +// Two writes of one tree are the same bytes, its times and modes changed in +// between: no owner, no time, the modes normalised. +TEST(PackTar, TheSameTreeIsTheSameBytes) { + Tmp t; + const auto root = sample_tree(t.path); + for (auto ch : { kPlain, c::Choice{} }) { + ASSERT_TRUE(d::write_tar(root, t.path / "one", ch)); + fs::last_write_time(root / "share/data.txt", fs::file_time_type::clock::now() - std::chrono::hours(48)); + fs::permissions(root / "share/data.txt", fs::perms::group_write, fs::perm_options::add); + ASSERT_TRUE(d::write_tar(root, t.path / "two", ch)); + EXPECT_EQ(slurp(t.path / "one"), slurp(t.path / "two")) << c::name(ch.algorithm); + } + ASSERT_TRUE(d::write_tar(root, t.path / "plain.tar", kPlain)); + const auto tar = slurp(t.path / "plain.tar"); + ASSERT_GE(tar.size(), 1024u); + EXPECT_EQ(tar.substr(0, 9), std::string("app-1.0/\0", 9)); + EXPECT_EQ(tar.substr(108, 8), std::string("0000000\0", 8)) << "uid"; + EXPECT_EQ(tar.substr(116, 8), std::string("0000000\0", 8)) << "gid"; + EXPECT_EQ(tar.substr(136, 12), std::string("00000000000\0", 12)) << "mtime"; + EXPECT_EQ(tar.substr(257, 6), std::string("ustar\0", 6)); + EXPECT_EQ(tar.substr(265, 32), std::string(32, '\0')) << "uname"; + EXPECT_EQ(tar.size() % 512, 0u); +} + +// What mcpp writes, `unpack` reads back: files, modes, links, long names. +TEST(PackTar, UnpackReadsWhatWriteTarWrote) { + Tmp t; + const auto root = sample_tree(t.path); + for (auto ch : { kPlain, c::Choice{}, c::Choice{ c::Algorithm::Zstd, -1, false }, + c::Choice{ c::Algorithm::Xz, -1, false } }) { + const auto archive = t.path / ("app" + std::string(c::tar_extension(ch.algorithm))); + ASSERT_TRUE(d::write_tar(root, archive, ch)); + const auto into = t.path / ("x-" + std::string(c::name(ch.algorithm))); + auto back = d::unpack(archive, into); + ASSERT_TRUE(back) << back.error(); + EXPECT_EQ(back->filename(), "app-1.0"); + EXPECT_EQ(slurp(*back / "share/data.txt"), "data"); + EXPECT_EQ(slurp(*back / "share" / std::string(60, 'd') / (std::string(60, 'e') + ".txt")), "deep"); + EXPECT_NE(fs::status(*back / "bin/app").permissions() & fs::perms::owner_exec, fs::perms::none); + EXPECT_TRUE(fs::is_directory(*back / "empty")); + EXPECT_TRUE(fs::is_symlink(*back / "share/link")); + EXPECT_EQ(fs::read_symlink(*back / "share/far").generic_string(), + std::string(60, 'd') + "/" + std::string(60, 'e') + ".txt"); + EXPECT_EQ(slurp(*back / "share/far"), "deep"); + } +} + +// A tar holding a link out of its tree is refused, and nothing is written. +TEST(PackTar, UnpackRefusesALinkOutOfTheTree) { + Tmp t; + const auto root = t.path / "app-1.0"; + write(root / "a", "a"); + fs::create_symlink("../../outside", root / "escape"); + ASSERT_TRUE(d::write_tar(root, t.path / "app.tar", kPlain)); + auto r = d::unpack(t.path / "app.tar", t.path / "x"); + ASSERT_FALSE(r); + EXPECT_NE(r.error().find("outside the package"), std::string::npos) << r.error(); + EXPECT_TRUE(fs::is_empty(t.path / "x")); +} + +// ─── what unpack refuses ────────────────────────────────────────────────── + +// A member that would land outside the extraction root, or be written through +// a link member, is refused before anything is written. +TEST(PackUnpack, AZipCreatesNothingOutsideItsRoot) { + Tmp t; + write(t.path / "f", "content"); + using E = mcpp::pack::zip::Entry; + const auto f = t.path / "f"; + struct Case { std::string_view label, why; std::vector entries; }; + const std::vector cases{ + { "dot-dot", "not a path inside", { { "pkg/a", f, false, {} }, { "pkg/../evil", f, false, {} } } }, + { "absolute", "not a path inside", { { "/tmp/mcpp-evil", f, false, {} } } }, + { "drive", "not a path inside", { { "C:/evil", f, false, {} } } }, + { "the root as a file", "not a path inside", { { "pkg/.", f, false, {} }, { ".", f, false, {} } } }, + { "backslash", "not a path inside", { { "pkg\\..\\evil", f, false, {} } } }, + { "link out", "outside the package", { { "pkg/a", f, false, {} }, { "pkg/l", {}, false, "../../outside" } } }, + { "absolute link", "outside the package", { { "pkg/a", f, false, {} }, { "pkg/l", {}, false, "/etc" } } }, + { "link through", "lies under the link", { { "pkg/l", {}, false, "sub" }, { "pkg/l/x", f, false, {} } } }, + { "link chain out", "outside the package", + { { "pkg/d/up", {}, false, ".." }, { "pkg/l", {}, false, "d/up/../.." }, { "pkg/a", f, false, {} } } }, + { "twice", "twice", { { "pkg/a", f, false, {} }, { "pkg/a", {}, false, "b" } } }, + }; + for (std::size_t i = 0; i < cases.size(); ++i) { + const auto into = t.path / std::format("x{}", i); + auto r = d::unpack(zip_of(t.path, std::format("c{}.zip", i), cases[i].entries), into); + ASSERT_FALSE(r) << cases[i].label; + EXPECT_NE(r.error().find(cases[i].why), std::string::npos) << cases[i].label << ": " << r.error(); + EXPECT_TRUE(fs::is_empty(into)) << cases[i].label << ": something was written"; + } + EXPECT_FALSE(fs::exists(t.path / "evil")); + // A link inside the tree is made, after the files. + auto ok = d::unpack(zip_of(t.path, "ok.zip", { { "pkg/l", {}, false, "sub" }, { "pkg/sub/x", f, false, {} } }), + t.path / "ok"); + ASSERT_TRUE(ok) << ok.error(); + EXPECT_TRUE(fs::is_symlink(*ok / "l")); + EXPECT_EQ(slurp(*ok / "l/x"), "content"); +} + +// Every truncation of a zip, and every byte of it changed, is an error or a +// tree, never a read past the archive. +TEST(PackUnpack, ATruncatedOrCorruptZipIsAnError) { + Tmp t; + write(t.path / "f", std::string(300, 'q')); + write(t.path / "g", "small"); + mcpp::pack::zip::WriteOptions o; + o.deflate = true; + ASSERT_TRUE(mcpp::pack::zip::write(t.path / "a.zip", { { "pkg/f", t.path / "f", false, {} }, + { "pkg/g", t.path / "g", true, {} } }, o)); + const auto whole = slurp(t.path / "a.zip"); + for (std::size_t n = 0; n < whole.size(); ++n) { + write(t.path / "cut.zip", whole.substr(0, n)); + const auto into = t.path / std::format("cut{}", n); + EXPECT_FALSE(d::unpack(t.path / "cut.zip", into)) << n << " bytes"; + fs::remove_all(into); + } + for (std::size_t n = 0; n < whole.size(); ++n) { + auto bad = whole; + bad[n] = static_cast(bad[n] ^ 0x5a); + write(t.path / "bad.zip", bad); + const auto into = t.path / std::format("bad{}", n); + (void)d::unpack(t.path / "bad.zip", into); + fs::remove_all(into); + } +} + +// A member whose bytes do not match its CRC-32 is refused. +TEST(PackUnpack, ACrcMismatchIsRefused) { + Tmp t; + write(t.path / "f", "the member's content"); + auto zip = slurp(zip_of(t.path, "a.zip", { { "pkg/f", t.path / "f", false, {} } })); + const auto at = zip.find("the member's content"); + ASSERT_NE(at, std::string::npos); + zip[at] = 'T'; + write(t.path / "a.zip", zip); + auto r = d::unpack(t.path / "a.zip", t.path / "x"); + ASSERT_FALSE(r); + EXPECT_NE(r.error().find("CRC-32"), std::string::npos) << r.error(); +} + +// ─── --merge ────────────────────────────────────────────────────────────── + +namespace { + +// A described package of one leg under `dir`. +fs::path leg_package(const fs::path& dir, std::string_view leg, std::string_view arch) { + const auto root = dir / "app-1.0"; + write(root / "lib" / arch / "app.oke", std::format("image of {}", arch)); + write(root / "README.md", "readme"); + d::Package p; + p.name = "app"; + p.version = "1.0"; + p.legs = { std::string(leg) }; + p.runner = { "okrun", "{}" }; + p.entry = { { std::string(arch), std::format("lib/{}/app.oke", arch) } }; + EXPECT_TRUE(d::describe(root, p, {})); + return root; +} + +} // namespace + +// A file changed after its package was written is refused, named with both +// digests. +TEST(PackMerge, AFileThatIsNotItsDigestIsRefused) { + Tmp t; + const auto a = leg_package(t.path / "a", "x86_64-openkal", "x86_64"); + const auto b = leg_package(t.path / "b", "aarch64-openkal", "aarch64"); + write(b / "lib/aarch64/app.oke", "changed"); + d::MergeRequest req; + req.inputs = { a, b }; + req.output = t.path / "out"; + auto r = d::merge(req); + ASSERT_FALSE(r); + EXPECT_NE(r.error().find("lib/aarch64/app.oke"), std::string::npos) << r.error(); + EXPECT_NE(r.error().find(d::sha256("changed")), std::string::npos) << r.error(); + EXPECT_NE(r.error().find(d::sha256("image of aarch64")), std::string::npos) << r.error(); + EXPECT_FALSE(fs::exists(t.path / "out")); +} + +// The legs merge into a new directory; an output that exists, or is an input, +// is refused, and an input written with a trailing separator is its directory. +TEST(PackMerge, TheOutputIsNewOrAPackage) { + Tmp t; + const auto a = leg_package(t.path / "a", "x86_64-openkal", "x86_64"); + const auto b = leg_package(t.path / "b", "aarch64-openkal", "aarch64"); + d::MergeRequest req; + req.inputs = { fs::path(a.string() + "/"), b }; + req.output = t.path / "out"; + auto r = d::merge(req); + ASSERT_TRUE(r) << r.error(); + auto merged = d::read(*r); + ASSERT_TRUE(merged) << merged.error(); + EXPECT_EQ(merged->legs, (std::vector{ "aarch64-openkal", "x86_64-openkal" })); + EXPECT_EQ(slurp(*r / "lib/aarch64/app.oke"), "image of aarch64"); + + // Again into the directory it wrote: a directory is never replaced. + auto again = d::merge(req); + ASSERT_FALSE(again); + EXPECT_NE(again.error().find("exists"), std::string::npos) << again.error(); + // Onto an input. + req.output = a; + ASSERT_FALSE(d::merge(req)); + EXPECT_TRUE(fs::exists(a / "mcpp-package.json")); + // Over a file that is not a package. + write(t.path / "notes.tar.gz", "my notes"); + req.output = t.path / "notes.tar.gz"; + auto over = d::merge(req); + ASSERT_FALSE(over); + EXPECT_NE(over.error().find("not a package"), std::string::npos) << over.error(); + EXPECT_EQ(slurp(t.path / "notes.tar.gz"), "my notes"); + // Over an archive that is one: replaced. + req.output = t.path / "app.tar.gz"; + ASSERT_TRUE(d::merge(req)); + auto replaced = d::merge(req); + ASSERT_TRUE(replaced) << replaced.error(); +} From 921b2994031d8aa394913d3e2df37c12a055b87c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 10:57:20 +0900 Subject: [PATCH 41/54] pack: a runner word a .cmd cannot hold is refused before staging; emit xpkg writes the metadata as Lua strings; the packaging spec states the in-process tar, the unpack refusals, the launcher's {} rule, merge's digests and destinations, the cache's ownership check Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/specs/packaging.md | 30 ++++++++++++++++------- src/pack/distribution.cppm | 35 +++++++++++++++++++-------- src/pack/pack.cppm | 3 +++ tests/unit/test_pack_distribution.cpp | 17 +++++++++++++ 4 files changed, 66 insertions(+), 19 deletions(-) diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index f5a457dff..0f724c6d7 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -162,8 +162,11 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 - 架构目录按 `uname -m`(POSIX)或 `%PROCESSOR_ARCHITEW6432%`、`%PROCESSOR_ARCHITECTURE%`(Windows)选择;映射为 `x86_64`、`amd64`、`AMD64` → `x86_64`,`aarch64`、`arm64`、`ARM64` → `aarch64`;没有对应目录时以状态 126 退出, 写出本机架构与包含的架构。 -- 运行 `runner.cmd`,`{}` 替换为所选架构目录中的入口映像,其余参数原样传递,随后是启动器自己的参数;运行器先在 - `libexec//-/` 中找,再找 `PATH`。 +- 运行 `runner.cmd`,与 `mcpp run` 的展开相同:含 `{}` 的词中 `{}` 替换为所选架构目录中的入口映像,没有词含 `{}` + 时映像追加在末尾;其余词原样传递,随后是启动器自己的参数;运行器先在 `libexec//-/` 中找, + 再找 `PATH`。 +- POSIX 启动器把每个词写成单引号字符串,沿符号链接找到自身的位置(不依赖 `readlink -f`);`.cmd` 启动器中 `%` + 写成 `%%`。`.cmd` 的词不能含双引号与换行:含有它们的运行器在打包时拒绝。 ### 4.4 `--systems` @@ -207,7 +210,7 @@ SDK 载荷时该次构建拒绝,写出该系统与载荷。 `role` 取 `program`、`image`、`library`、`loader`、`launcher`、`resource`、`metadata` 之一:通用包中 `lib//` 下的文件为该腿的 `image`,生成的启动器为 `launcher`;其余按位置与名字判定 —— `libexec/` 下为 `loader`; -`README*`、`LICENSE*`、`HOST-REQUIREMENTS`、`share/licenses/` 为 `metadata`;`.so*`、`.dylib`、`.dll`、`.okso` 为 +`README*`、`LICENSE*`、`HOST-REQUIREMENTS`、`share/licenses/` 为 `metadata`;`.so` 与 `.so.<版本>`、`.dylib`、`.dll`、`.okso` 为 `library`;`.oke` 为 `image`;`bin/` 下的可执行文件为 `program`;`.cmd`、`.sh` 与树根的可执行文件为 `launcher`; 其他为 `resource`。 @@ -215,7 +218,10 @@ SDK 载荷时该次构建拒绝,写出该系统与载荷。 `mcpp pack --merge --merge ...`(也可 `--merge ,`)读入至少两个本命令写出的包(目录,或 `.zip`、 `.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz` 归档),按 `mcpp-package.json` 合并为一棵树:名字、版本、平台、 -链接与模式**必须**相同;腿与系统取并集;同一路径的文件摘要**必须**相同,每个文件只取一份;合并后的树重新描述。 +链接与模式**必须**相同;腿与系统取并集;每个文件在复制前计算摘要并与记录的摘要比较,不符时拒绝并写出文件、输入 +与两个摘要;同一路径的文件摘要**必须**相同,每个文件只取一份;合并后的树重新描述。`mcpp-package.json` 中的每个 +路径**必须**是包内的规范相对路径。输出**必须**是新的路径,或一个已有的、不是输入之一的本命令写出的归档;目录输出 +只写新目录。 容器取 `--pkg-format`,否则取 `-o` 的扩展名,否则取第一个输入的容器;`-o` 缺省为当前目录下与树同名的文件。 不进行任何构建。 @@ -223,11 +229,16 @@ SDK 载荷时该次构建拒绝,写出该系统与载荷。 ### 5.1 压缩 -`--compress [:]` 与 `[pack] compress` 选择 `tar` 的压缩:`tar` 写出未压缩的归档,mcpp 以 -自带的编解码器(zlib、zstd、liblzma)压缩,产物扩展名为 `.tar.gz`(缺省)、`.tar.zst`、`.tar.xz` 或 `.tar`。级别 -范围:gzip 1–9,zstd 1–22,xz 0–9。`zip` 的条目在变小时以 DEFLATE 压缩(`gzip[:]` 设定级别),否则存储; +`--compress [:]` 与 `[pack] compress` 选择 `tar` 的压缩:mcpp 自己写出 ustar 归档(按 +路径排序;长名与链接目标用 pax 头;属主为 0,时间为 0,模式 0755、0644、0777),以自带的编解码器(zlib、zstd、 +liblzma)压缩,不调用 `tar` 等外部程序;产物扩展名为 `.tar.gz`(缺省)、`.tar.zst`、`.tar.xz` 或 `.tar`。级别 +范围:gzip 1–9,zstd 1–22,xz 0–9;写出的级别照用(含 `xz:0`、`zstd:1`),未写时取编解码器的缺省。`zip` 的条目在变小时以 DEFLATE 压缩(`gzip[:]` 设定级别),否则存储; `none` 时全部存储。单文件的载荷总是一个 zstd 帧(启动器只解 zstd),`none` 取级别 1。okb 中的映像不压缩(SPEC-013 平台插件的格式规定)。同一棵树的 -归档在每个宿主上字节相同:gzip 成员不带名字与时间,zip 条目使用固定日期。 +归档在每个宿主上字节相同:gzip 成员不带名字与时间、系统字节为 255,zip 条目使用固定日期。 + +**读取归档**(`--merge`、`mcpp emit xpkg --package`)在进程内完成,读取 ustar、pax 与 GNU 长名及首项为 `./` 的 +归档,zip 的每个条目校验 CRC-32。写出任何文件之前拒绝:绝对路径或含 `..` 的条目、位于链接条目之下的条目、重复的 +条目、指向树外的链接(链接目标逐段解析);链接在全部文件之后创建;目标目录必须不存在或为空。 ### 5.2 单文件 @@ -248,7 +259,8 @@ SDK 载荷时该次构建拒绝,写出该系统与载荷。 (Windows);其下为 `mcpp/single-file/<载荷的 sha256>/`,该摘要由 mcpp 打包时写入启动器。 4. 解压时**必须**校验帧的内容校验和,不符时不解开;解到同一目录下的临时名(`.<摘要>-<进程号>`),全部文件写完后 写入 `.mcpp-complete` 并以不替换的方式改名到位;**禁止**使用没有 `.mcpp-complete` 的目录;另一次运行先改名到位时 - 使用它的结果;缓存目录在 POSIX 上只对当前用户可写。 + 使用它的结果;缓存目录在 POSIX 上只对当前用户可写,`mcpp/`、`single-file/` 或摘要目录为符号链接或属于其他 + 用户时启动器拒绝运行。 5. 以原有的参数执行树中的程序(`bin/`;self-contained 的 Linux 树为根部的包装脚本;Windows 树为根部的 `.exe`),退出状态即程序的退出状态。 diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 44bf50b6d..53043c89a 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -1397,9 +1397,21 @@ std::string xlings_system(std::string_view triple) { return "linux"; } +// A text inside a Lua double-quoted literal: what the metadata states is +// written as a string, never as code. +std::string lua_text(std::string_view s) { + std::string out; + for (unsigned char c : s) { + if (c == '\\' || c == '"') { out += '\\'; out += static_cast(c); } + else if (c < 0x20 || c == 0x7f) out += std::format("\\{:03}", static_cast(c)); + else out += static_cast(c); + } + return out; +} + std::string lua_list(const std::vector& v) { std::string out = "{"; - for (std::size_t i = 0; i < v.size(); ++i) out += std::format("{}\"{}\"", i ? ", " : "", v[i]); + for (std::size_t i = 0; i < v.size(); ++i) out += std::format("{}\"{}\"", i ? ", " : "", lua_text(v[i])); return out + "}"; } @@ -1432,12 +1444,13 @@ std::expected xpkg_descriptor(const std::vector std::string deps; if (first.mode != "self-contained") deps = std::format(" deps = {{ \"{}\" }},\n", - std::filesystem::path(first.runner.front()).stem().string()); + lua_text(std::filesystem::path(first.runner.front()).stem().string())); for (auto system : { "linux", "macosx", "windows" }) blocks[system] = std::format( "{0} [\"latest\"] = {{ ref = \"{1}\" }},\n" " [\"{1}\"] = {{\n url = \"{2}/{3}\",\n sha256 = \"{4}\",\n }},\n", - deps, first.version, prefix, a.file.filename().string(), a.sha256); + deps, lua_text(first.version), lua_text(prefix), lua_text(a.file.filename().string()), + lua_text(a.sha256)); } else { // system -> (arch -> asset) std::map> bySystem; @@ -1463,12 +1476,13 @@ std::expected xpkg_descriptor(const std::vector return std::unexpected(std::format("the {} packages' names differ beyond the " "architecture ({} and {})", system, tmpl, name)); tmpl = name; - shas += std::format(" {} = \"{}\",\n", arch, a->sha256); + shas += std::format(" [\"{}\"] = \"{}\",\n", lua_text(arch), + lua_text(a->sha256)); } blocks[system] = std::format( " [\"latest\"] = {{ ref = \"{0}\" }},\n" " [\"{0}\"] = {{\n url = \"{1}/{2}\",\n sha256 = {{\n{3} }},\n }},\n", - first.version, prefix, tmpl, shas); + lua_text(first.version), lua_text(prefix), lua_text(tmpl), shas); } } std::ranges::sort(archs); @@ -1478,13 +1492,14 @@ std::expected xpkg_descriptor(const std::vector // Where the program a person runs lies in the installed tree. std::string bindir = "\"bin\"", alias; if (universal) { - alias = std::format("if os.host() == \"windows\" then opts.alias = \"{}.cmd\" end\n ", first.name); + alias = std::format("if os.host() == \"windows\" then opts.alias = \"{}.cmd\" end\n ", + lua_text(first.name)); } else { const auto program = std::filesystem::path(first.program); - bindir = program.has_parent_path() ? std::format("\"{}\"", program.parent_path().generic_string()) + bindir = program.has_parent_path() ? std::format("\"{}\"", lua_text(program.parent_path().generic_string())) : std::string("\".\""); } - const auto wrapper = std::format("{}-{}-", first.name, first.version); + const auto wrapper = lua_text(std::format("{}-{}-", first.name, first.version)); return std::format( "-- Generated by `mcpp emit xpkg --package` from the packages' mcpp-package.json\n" "-- (SPEC-014 §8).\n" @@ -1518,8 +1533,8 @@ std::expected xpkg_descriptor(const std::vector " xvm.remove(\"{0}\")\n" " return true\n" "end\n", - first.name, first.version, - universal ? std::format(", one package for every system ({})", first.platform) : std::string(), + lua_text(first.name), lua_text(first.version), + universal ? std::format(", one package for every system ({})", lua_text(first.platform)) : std::string(), lua_list(archs), xpm, wrapper, bindir, alias); } diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index abb3bed29..25f672ce7 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -2531,6 +2531,9 @@ run_loaded(const Plan& plan) mcpp::pack::distribution::Launch launch; launch.name = fs::path(plan.binaryName).stem().string(); launch.runner = runner.cmd; + // A word a `.cmd` cannot hold is refused now, before anything is staged. + if (auto ok = mcpp::pack::distribution::launchable(launch); !ok) + return std::unexpected(Error{ok.error()}); for (auto const& leg : legs) { const auto dir = plan.stagingRoot / "lib" / leg.arch; if (auto r = copy(leg.artifact, dir / leg.artifact.filename()); !r) return r; diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index b891aa0e7..3cdb94833 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -578,3 +578,20 @@ TEST(PackMerge, TheOutputIsNewOrAPackage) { auto replaced = d::merge(req); ASSERT_TRUE(replaced) << replaced.error(); } + +// What the metadata states reaches the generated Lua as strings: a quote or a +// line break in a name is escaped, never code. +TEST(PackDistribution, XpkgDescriptorWritesMetadataAsStrings) { + d::Asset a; + a.file = "evil-1.0-openkal.zip"; + a.sha256 = std::string(64, 'a'); + a.pkg.name = "ev\"il\nos.exit()"; + a.pkg.version = "1.0"; + a.pkg.runner = { "okrun", "{}" }; + a.pkg.entry = { { "x86_64", "lib/x86_64/evil.oke" } }; + auto text = d::xpkg_descriptor({ a }, "https://example.org/v1.0"); + ASSERT_TRUE(text) << text.error(); + EXPECT_NE(text->find("name = \"ev\\\"il\\010os.exit()\""), std::string::npos) << *text; + EXPECT_EQ(text->find("\nos.exit()"), std::string::npos) << *text; + EXPECT_NE(text->find("deps = { \"okrun\" }"), std::string::npos) << *text; +} From b48f61df71ae3d5e7e959e01e004e6659b9b37b6 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 11:38:51 +0900 Subject: [PATCH 42/54] review: config.toml's unknown platform keys are said; a tree's executables are what the files are on Windows; links are compared as a case-blind file system sees them and checked where they resolve; DEFLATE never expands past its bound; work directories are made exclusively; merge fits the compression to the container and the name; launchers follow links physically, keep `!` literal and refuse a word ending in a backslash; a program's artifact name holds no directory Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- modules/buildmcpp/src/directives.cppm | 3 + src/cli/cmd_publish.cppm | 46 ++++------- src/config.cppm | 16 +++- src/pack/compress_codecs.cpp | 12 ++- src/pack/distribution.cppm | 108 ++++++++++++++++++++++---- src/pack/pack.cppm | 4 +- tests/unit/test_build_directives.cpp | 1 + tests/unit/test_pack_distribution.cpp | 20 +++++ 8 files changed, 162 insertions(+), 48 deletions(-) diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index b4141a79a..ad4df83e7 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -1459,6 +1459,9 @@ std::string target_directive_error(const mcpp::manifest::Manifest& m, const Dire return std::format("build.mcpp emitted `mcpp:artifact-name={}`; the pattern names " "the artifact as {{name}} once{}", v, kind == "exe" ? ", at its start" : ""); + if (pattern.find_first_of("/\\") != std::string::npos) + return std::format("build.mcpp emitted `mcpp:artifact-name={}`; the pattern is a file " + "name, without a directory", v); } for (auto const& v : d.at(Slot::Seal)) { const auto colon = v.find(':'); diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index f97083520..81031d7c1 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -45,8 +45,9 @@ export int cmd_emit_xpkg(const mcpplibs::cmdline::ParsedArgs& parsed) { namespace d = mcpp::pack::distribution; std::vector assets; std::error_code ec; - const auto work = std::filesystem::temp_directory_path(ec) / std::format( - "mcpp-emit-{}", std::chrono::steady_clock::now().time_since_epoch().count()); + auto made = d::work_directory("emit"); + if (!made) { mcpp::ui::error(made.error()); return 1; } + const auto work = *made; for (auto const& p : parsed.option("package")->get().values) { const std::filesystem::path file{p}; // An entry names a published file, which a directory is not. @@ -606,26 +607,13 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, if (auto v = parsed.value("output")) req.output = *v; if (auto f = pack_format_option(parsed)) req.format = *f; else if (!packCfg.format.empty()) req.format = packCfg.format; - // The container: as stated, else as the output's name says. Only the - // flag's compression is the command's (a project's `[pack] compress` - // describes that project's packs, not a merge of packages). - std::string container = req.format; - const auto outName = req.output.filename().string(); - if (container.empty() && outName.ends_with(".zip")) container = "zip"; - else if (container.empty() && outName.find(".tar") != std::string::npos) container = "tar"; + // Only the flag's compression is the command's (a project's `[pack] + // compress` describes that project's packs, not a merge of packages); + // `merge` decides the container and refuses a compression that does + // not fit it. if (auto text = parsed.value("compress")) { auto c = mcpp::pack::compress::parse(*text); if (!c) { mcpp::ui::error(std::format("--compress: {}", c.error())); return 2; } - using mcpp::pack::compress::Algorithm; - if (container == "dir") { - mcpp::ui::error("--compress compresses a container, and the merge writes a directory"); - return 2; - } - if (container == "zip" && (c->algorithm == Algorithm::Zstd || c->algorithm == Algorithm::Xz)) { - mcpp::ui::error(std::format("a zip's members are deflated (gzip) or stored (none); " - "--compress {} is a tar's", mcpp::pack::compress::name(c->algorithm))); - return 2; - } req.compression = *c; } auto merged = mcpp::pack::distribution::merge(req); @@ -639,16 +627,16 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, report->format = std::filesystem::is_directory(*merged, mec) ? std::string("dir") : name.ends_with(".zip") ? std::string("zip") : std::string("tar"); // What the merged package says of itself. - const auto work = std::filesystem::temp_directory_path(mec) - / std::format("mcpp-merged-{}", std::chrono::steady_clock::now().time_since_epoch().count()); - if (auto root = mcpp::pack::distribution::unpack(*merged, work)) - if (auto pkg = mcpp::pack::distribution::read(*root)) { - report->targets = pkg->legs; - report->systems = pkg->systems; - report->linkage = pkg->linkage; - report->mode = pkg->mode; - } - std::filesystem::remove_all(work, mec); + if (auto work = mcpp::pack::distribution::work_directory("merged")) { + if (auto root = mcpp::pack::distribution::unpack(*merged, *work / "tree")) + if (auto pkg = mcpp::pack::distribution::read(*root)) { + report->targets = pkg->legs; + report->systems = pkg->systems; + report->linkage = pkg->linkage; + report->mode = pkg->mode; + } + std::filesystem::remove_all(*work, mec); + } } return 0; } diff --git a/src/config.cppm b/src/config.cppm index 3bbfcd6c2..1b4067740 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -732,7 +732,21 @@ parse_target_defaults(const mcpp::libs::toml::Document& doc, std::vector* platforms, mcpp::manifest::platform::Override* host) { namespace mp = mcpp::manifest::platform; - const mp::Reader reader{file, mp::Tier::Config, nullptr}; + // What the reader warns of (an unknown key, SPEC-013 §18.1), said once + // per process for each file: the configuration is read by every prepare. + struct Said { + const std::filesystem::path& file; + std::vector warnings; + ~Said() { + static std::mutex m; + static std::set said; + std::lock_guard lock(m); + for (auto const& w : warnings) + if (said.insert(file.string() + '\n' + w).second) + std::println(stderr, "warning: {}: {}", file.string(), w); + } + } said{ file, {} }; + const mp::Reader reader{file, mp::Tier::Config, &said.warnings}; // `[host]`: the platform of this machine's host graph (SPEC-013 §6.1). A // nearer file's choice stands. if (auto* ht = doc.get_table("host"); ht && host && !host->platformSet) { diff --git a/src/pack/compress_codecs.cpp b/src/pack/compress_codecs.cpp index 3e41cddae..6de8bac42 100644 --- a/src/pack/compress_codecs.cpp +++ b/src/pack/compress_codecs.cpp @@ -109,6 +109,12 @@ std::expected zlib_inflate(std::string_view in, int wi std::optional size) { if (in.size() > std::numeric_limits::max()) return std::unexpected(std::string("zlib: an input over 4 GiB")); + // DEFLATE expands at most 1032 times (a run of one byte): a stated size + // beyond that is a lie, refused before anything is allocated, and a stream + // that grows past it is refused as it grows. + const std::size_t most = in.size() * 1032 + 64; + if (size && *size > most) + return std::unexpected(std::format("zlib: {} bytes cannot hold the {} it states", in.size(), *size)); z_stream s{}; if (inflateInit2(&s, windowBits) != Z_OK) return std::unexpected(std::string("zlib: inflateInit2 failed")); @@ -126,7 +132,11 @@ std::expected zlib_inflate(std::string_view in, int wi return std::unexpected(std::format("zlib: the stream holds more than its {} bytes", *size)); } - out.resize(out.size() * 2); + if (out.size() > most) { + inflateEnd(&s); + return std::unexpected(std::string("zlib: the stream expands beyond what DEFLATE can")); + } + out.resize(std::min(out.size() * 2, most + 1)); } s.next_out = reinterpret_cast(out.data() + s.total_out); s.avail_out = static_cast(std::min(out.size() - s.total_out, diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 53043c89a..0b87e9de3 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -15,6 +15,7 @@ import std; import mcpp.libs.json; import mcpp.pack.compress; import mcpp.pack.zip; +import mcpplibs.os; export namespace mcpp::pack::distribution { @@ -170,6 +171,11 @@ struct MergeRequest { // that is a package and none of the inputs (`dir` writes a new directory only). std::expected merge(const MergeRequest& req); +// A new directory under the system's temporary directory, made by this call +// and no other (a name another made first is passed over) and open to this +// user alone: where packages are unpacked to be read. +std::expected work_directory(std::string_view purpose); + } // namespace mcpp::pack::distribution namespace mcpp::pack::distribution { @@ -255,11 +261,26 @@ bool spit(const fs::path& p, std::string_view bytes) { return bool(out); } +// Whether a file is executable. A POSIX host's mode says. Windows keeps no +// such bit (its libraries report every file executable), so there what the +// file is says: a program or library image (ELF, Mach-O, PE) or a script +// (`#!`), which are the files a POSIX host's linkers and launcher writer leave +// executable; the tree's bytes are then the same on every host. bool executable_file(const fs::path& p) { - std::error_code ec; - const auto perms = fs::status(p, ec).permissions(); - if (ec) return false; - return (perms & fs::perms::owner_exec) != fs::perms::none; + if constexpr (mcpplibs::os::is_windows) { + std::ifstream in(p, std::ios::binary); + char head[4]{}; + in.read(head, sizeof head); + const std::string_view h(head, static_cast(in.gcount())); + return h.starts_with("#!") || h.starts_with("MZ") || h == std::string_view("\x7f" "ELF", 4) + || h == std::string_view("\xcf\xfa\xed\xfe", 4) || h == std::string_view("\xce\xfa\xed\xfe", 4) + || h == std::string_view("\xca\xfe\xba\xbe", 4); + } else { + std::error_code ec; + const auto perms = fs::status(p, ec).permissions(); + if (ec) return false; + return (perms & fs::perms::owner_exec) != fs::perms::none; + } } // One entry of a staged tree: its path under the root (`/`-separated), what it @@ -501,6 +522,15 @@ std::optional member_path(std::string_view name) { // the extraction root: the target is relative and, walked from the link's // directory, never climbs above the root and goes on from no link member (one // may be where it ends: that link's own target is checked the same way). +// A member path as a file system that ignores case sees it (APFS, NTFS): two +// members, or a link and a path, that differ in case alone are one there. +std::string folded(std::string_view path) { + std::string out(path); + for (auto& c : out) c = static_cast(std::tolower(static_cast(c))); + return out; +} + +// `links` holds the link members' paths `folded`. bool link_inside(std::string_view path, std::string_view target, const std::set& links) { if (foreign(target)) return false; // Where the walk is, a path under the root (empty: the root itself); the @@ -509,7 +539,7 @@ bool link_inside(std::string_view path, std::string_view target, const std::set< std::string here(slash == std::string_view::npos ? std::string_view() : path.substr(0, slash)); for (auto part : parts(target)) { if (part.empty() || part == ".") continue; - if (!here.empty() && links.contains(here)) return false; + if (!here.empty() && links.contains(folded(here))) return false; if (part == "..") { if (here.empty()) return false; const auto up = here.rfind('/'); @@ -765,14 +795,16 @@ std::expected, std::string> tar_members(std::string_view b) // no link whose target leaves the tree. Then the directories and the files // are written, and the links made last, so no member is written through one. std::expected place(const std::vector& members, const fs::path& into) { + // Compared `folded`: on a file system that ignores case `A/B` is `a/b`. std::set paths, links; for (auto const& m : members) { - if (!paths.insert(m.path).second) return std::unexpected(std::format("'{}' is in the archive twice", m.path)); - if (m.kind == 'l') links.insert(m.path); + if (!paths.insert(folded(m.path)).second) + return std::unexpected(std::format("'{}' is in the archive twice", m.path)); + if (m.kind == 'l') links.insert(folded(m.path)); } for (auto const& m : members) { for (auto slash = m.path.find('/'); slash != std::string::npos; slash = m.path.find('/', slash + 1)) - if (links.contains(m.path.substr(0, slash))) + if (links.contains(folded(m.path.substr(0, slash)))) return std::unexpected(std::format("'{}' lies under the link '{}'", m.path, m.path.substr(0, slash))); if (m.kind == 'l' && !link_inside(m.path, m.target, links)) return std::unexpected(std::format("the link '{}' points to '{}', outside the package", m.path, @@ -804,6 +836,16 @@ std::expected place(const std::vector& members, const fs::create_symlink(fs::path(m.target), link, ec); if (ec) return std::unexpected(std::format("cannot link '{}': {}", link.string(), ec.message())); } + // And as the file system resolves them: every link ends under the root. + const auto root = fs::weakly_canonical(into, ec); + for (auto const& m : members) { + if (m.kind != 'l') continue; + const auto real = fs::weakly_canonical(into / fs::path(m.path), ec); + const auto [r, _] = std::mismatch(root.begin(), root.end(), real.begin(), real.end()); + if (ec || r != root.end()) + return std::unexpected(std::format("the link '{}' points to '{}', outside the package", m.path, + m.target)); + } return {}; } @@ -941,10 +983,12 @@ std::expected launchable(const Launch& l) { words.insert(words.end(), l.runner.begin(), l.runner.end()); for (auto const& [arch, file] : l.entry) words.insert(words.end(), {arch, file}); for (auto w : words) - if (w.find_first_of(std::string_view("\"\r\n\0", 4)) != std::string_view::npos) + // A word ending in `\` would make the quote that closes it a quote of + // the word for the program reading its arguments. + if (w.find_first_of(std::string_view("\"\r\n\0", 4)) != std::string_view::npos || w.ends_with('\\')) return std::unexpected(std::format( "the launcher of {} cannot state '{}': a word of a .cmd holds no double quote and no " - "line break", l.name, w)); + "line break, and does not end in a backslash", l.name, w)); return {}; } @@ -983,7 +1027,7 @@ std::string posix_launcher(const Launch& l) { " esac\n" " n=$((n + 1))\n" "done\n" - "top=$(CDPATH= cd -- \"$(dirname -- \"$self\")/..\" && pwd)\n" + "top=$(CDPATH= cd -P -- \"$(dirname -- \"$self\")/..\" && pwd -P)\n" "case \"$(uname -m)\" in\n" " x86_64|amd64|AMD64) arch=x86_64 ;;\n" " aarch64|arm64|ARM64) arch=aarch64 ;;\n" @@ -1028,7 +1072,7 @@ std::string windows_launcher(const Launch& l) { "@echo off\r\n" "rem Generated by mcpp pack (SPEC-014 §4.3): runs the package's program on this\r\n" "rem machine's architecture through the platform's runner.\r\n" - "setlocal\r\n" + "setlocal DisableDelayedExpansion\r\n" "for %%I in (\"%~dp0..\") do set \"top=%%~fI\"\r\n" "set \"machine=%PROCESSOR_ARCHITECTURE%\"\r\n" "if defined PROCESSOR_ARCHITEW6432 set \"machine=%PROCESSOR_ARCHITEW6432%\"\r\n" @@ -1553,13 +1597,30 @@ fs::path named(const fs::path& p) { } // namespace +std::expected work_directory(std::string_view purpose) { + std::error_code ec; + const auto base = fs::temp_directory_path(ec); + if (ec) return std::unexpected(std::format("no temporary directory: {}", ec.message())); + std::random_device rd; + for (int attempt = 0; attempt < 16; ++attempt) { + const auto p = base / std::format("mcpp-{}-{:08x}{:08x}", purpose, rd(), rd()); + if (!fs::create_directory(p, ec)) { + if (ec) return std::unexpected(std::format("cannot create '{}': {}", p.string(), ec.message())); + continue; // another's + } + fs::permissions(p, fs::perms::owner_all, fs::perm_options::replace, ec); + return p; + } + return std::unexpected(std::format("no new directory could be made under '{}'", base.string())); +} + std::expected merge(const MergeRequest& req) { if (req.inputs.size() < 2) return std::unexpected(std::string("--merge reads two packages or more")); std::error_code ec; - const auto work = fs::temp_directory_path(ec) - / std::format("mcpp-merge-{}", std::chrono::steady_clock::now().time_since_epoch().count()); - fs::create_directories(work, ec); + auto made = work_directory("merge"); + if (!made) return std::unexpected(made.error()); + const auto work = *made; struct Cleanup { fs::path p; ~Cleanup() { std::error_code e; fs::remove_all(p, e); } } cleanup{work}; std::vector inputs; @@ -1610,6 +1671,23 @@ std::expected merge(const MergeRequest& req) { : firstName.ends_with(".tar") ? Algorithm::None : Algorithm::Gzip; } + // The compression fits the container and the name the output is given. + { + using mcpp::pack::compress::Algorithm; + const auto a = compression.algorithm; + if (compression.stated && format == "dir") + return std::unexpected(std::string("--compress compresses a container, and the merge writes a " + "directory")); + if (compression.stated && format == "zip" && (a == Algorithm::Zstd || a == Algorithm::Xz)) + return std::unexpected(std::format("a zip's members are deflated (gzip) or stored (none); " + "--compress {} is a tar's", mcpp::pack::compress::name(a))); + const auto want = mcpp::pack::compress::tar_extension(a); + const bool namedTar = outName.find(".tar") != std::string::npos || outName.ends_with(".tgz"); + if (format == "tar" && namedTar && !outName.ends_with(want) + && !(a == Algorithm::Gzip && outName.ends_with(".tgz"))) + return std::unexpected(std::format("-o {} names another compression than the archive's ({})", + outName, want)); + } const auto ext = format == "zip" ? std::string(".zip") : format == "dir" ? std::string() : std::string(mcpp::pack::compress::tar_extension(compression.algorithm)); const fs::path dest = named(req.output.empty() ? fs::path(wrapper.string() + ext) : req.output); diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 25f672ce7..a085a0b1a 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -2531,13 +2531,14 @@ run_loaded(const Plan& plan) mcpp::pack::distribution::Launch launch; launch.name = fs::path(plan.binaryName).stem().string(); launch.runner = runner.cmd; + for (auto const& leg : legs) launch.entry[leg.arch] = leg.artifact.filename().string(); + if (plan.opts.mode == Mode::BundleAll) launch.loader = fs::path(runner.cmd.front()).stem().string(); // A word a `.cmd` cannot hold is refused now, before anything is staged. if (auto ok = mcpp::pack::distribution::launchable(launch); !ok) return std::unexpected(Error{ok.error()}); for (auto const& leg : legs) { const auto dir = plan.stagingRoot / "lib" / leg.arch; if (auto r = copy(leg.artifact, dir / leg.artifact.filename()); !r) return r; - launch.entry[leg.arch] = leg.artifact.filename().string(); if (bundles) { auto n = needed_images(leg.artifact, plan.binaryName); if (!n) return std::unexpected(n.error()); @@ -2547,7 +2548,6 @@ run_loaded(const Plan& plan) if (auto r = stage_runtime_files(plan, dir, leg.artifact.parent_path()); !r) return r; } if (plan.opts.mode == Mode::BundleAll) { - launch.loader = fs::path(runner.cmd.front()).stem().string(); std::vector systems = plan.opts.systems; if (systems.empty()) systems.emplace_back(host_system()); for (auto const& system : systems) diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index affa04ff6..df38f3fb2 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -1107,6 +1107,7 @@ TEST(BuildDirectives, PlatformFieldsAreProtocolSeventeenAndReachTheStatement) { EXPECT_EQ(s.link.driver.origin.tier, mcpp::manifest::platform::Tier::Program); for (auto bad : {"mcpp:link-driver=gold\n", "mcpp:link-input=from:x\n", "mcpp:artifact-name=exe:lib{name}\n", "mcpp:artifact-name=dll:{name}\n", + "mcpp:artifact-name=shared:lib/{name}.okso\n", "mcpp:seal=x:okld\n", "mcpp:run-host=far\n"}) EXPECT_NE(dirs::target_directive_error(m, parse(bad)), "") << bad; } diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index 3cdb94833..6580a4320 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -595,3 +595,23 @@ TEST(PackDistribution, XpkgDescriptorWritesMetadataAsStrings) { EXPECT_EQ(text->find("\nos.exit()"), std::string::npos) << *text; EXPECT_NE(text->find("deps = { \"okrun\" }"), std::string::npos) << *text; } + +// The compression fits the container the merge writes and the name it is +// given: zstd in a .tar.gz, or in a zip the first input decided, is refused. +TEST(PackMerge, TheCompressionFitsTheContainer) { + Tmp t; + const auto a = leg_package(t.path / "a", "x86_64-openkal", "x86_64"); + const auto b = leg_package(t.path / "b", "aarch64-openkal", "aarch64"); + d::MergeRequest req; + req.inputs = { a, b }; + req.compression = *c::parse("zstd"); + req.output = t.path / "out.tar.gz"; + auto named = d::merge(req); + ASSERT_FALSE(named); + EXPECT_NE(named.error().find(".tar.zst"), std::string::npos) << named.error(); + req.output = t.path / "out.zip"; + ASSERT_FALSE(d::merge(req)); + req.output = t.path / "out.tar.zst"; + auto fits = d::merge(req); + ASSERT_TRUE(fits) << fits.error(); +} From a40a8f6ac6d03ec59b6c95d7ea47f1dad36b95bf Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 11:49:50 +0900 Subject: [PATCH 43/54] pack: an image's single file for a system whose row this mcpp builds no program for (x86_64-macos, planned; aarch64-windows-gnu, unstated) is refused before the build, naming the row and the systems that are built; the -o refusal comes first; spec rows; e2e 956 E Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/specs/packaging.md | 2 ++ src/pack/pack.cppm | 11 +++++++++++ src/pack/pipeline.cppm | 19 +++++++++++++++++++ ...56_a_single_file_runs_without_unpacking.sh | 10 ++++++++-- 4 files changed, 40 insertions(+), 2 deletions(-) diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 0f724c6d7..835e38b82 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -85,6 +85,8 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` | `cxx_runtime = "toolchain-coupled"` 与 `--mode system` | 拒绝 | | 产物为可执行文件、非通用包,且多条 `--target` | 拒绝:一个可执行文件属于一个目标 | | `--systems` 而非 `--mode self-contained` 的通用包或映像的单文件 | 拒绝:只有携带加载器的包与映像的单文件按系统区分 | +| 映像的单文件,`--systems` 中某系统在映像架构上的行为本 mcpp 不构建的行(目标表标为 planned,如 `x86_64-macos`;或未列出,如 `aarch64-windows-gnu`) | 拒绝,写出该系统与行;出路为其余系统组成的 `--systems` | +| 映像的单文件,多个 `--systems` 与 `-o` | 拒绝:每个系统一个文件,`-o` 只能命名一个 | | 平台未声明 `runner` 时打通用包 | 拒绝:启动器经平台的运行器启动映像;出路 `--no-universal` | | `--merge` 的输入缺少 `mcpp-package.json`,或名字、版本、平台、链接、模式不同 | 拒绝,写出不一致的字段 | | `--merge` 的两个输入在同一路径上有摘要不同的文件 | 拒绝,写出该路径、两个摘要与两个输入 | diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index a085a0b1a..6be7bbc6e 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -539,6 +539,12 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg); // tree as a zstd frame. Returns the files written. std::expected, Error> single_file(const Plan& plan); +// The row a program of `system` (linux | macos | windows) and `arch` is built +// on -- an image's single file, a universal package's runner -- when this mcpp +// builds no program for it: a row its table states as planned (`x86_64-macos`) +// or does not state (`aarch64-windows-gnu`). Empty when it builds one. +std::string unbuilt_row(std::string_view system, std::string_view arch); + // The tree made a package: signed and described (`mcpp-package.json`). `run` // does it before writing a container; a dispatched format whose tree was // staged without its closure is finalized by the caller. @@ -3202,6 +3208,11 @@ std::expected finalize_staged(const Plan& plan) { return detail::finalize_tree(plan); } +std::string unbuilt_row(std::string_view system, std::string_view arch) { + const auto row = detail::program_row(system, arch, true); + return detail::row_is_built(row) ? std::string() : row.triple; +} + std::expected, Error> single_file(const Plan& plan) { namespace fs = std::filesystem; namespace d = mcpp::pack::distribution; diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index aa55872ca..459c125e6 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -820,6 +820,25 @@ std::optional check_format(PackRun& run, GroupJob& g) { "--output names one file, and --systems makes one single file per system ({}).\n" " use: leave --output out; each file is named ---", run.opts.systems.size())}; + // A single file of an image is made for each system on this mcpp's + // row of it; a row it builds no program for is refused before the build. + if (loaded && run.opts.format == mcpp::pack::Format::SingleFile) { + std::vector unbuilt, built; + for (auto const& system : run.opts.systems) { + auto row = mcpp::pack::unbuilt_row(system, t->arch); + (row.empty() ? built : unbuilt).push_back(row.empty() ? system : std::format("{} ({})", system, row)); + } + if (!unbuilt.empty()) { + std::string say, use; + for (auto const& u : unbuilt) say += (say.empty() ? "" : ", ") + u; + for (auto const& b : built) use += (use.empty() ? "" : ",") + b; + return Refusal{2, std::format( + "--systems: this mcpp builds no program for {}, so no single file of the {} " + "image is made there.\n use: {}", say, t->str(), + use.empty() ? std::string("--systems naming a system this mcpp builds for") + : "--systems " + use)}; + } + } } // ─── Is the requested format one anything provides? ────────────── // diff --git a/tests/e2e/956_a_single_file_runs_without_unpacking.sh b/tests/e2e/956_a_single_file_runs_without_unpacking.sh index 1dab46acc..d27706ad2 100755 --- a/tests/e2e/956_a_single_file_runs_without_unpacking.sh +++ b/tests/e2e/956_a_single_file_runs_without_unpacking.sh @@ -14,7 +14,8 @@ # D. (llvm, an ok-image row) An image's single file runs, and writes nothing # into the cache. # E. (an ok-image row) `-o` with several `--systems` is refused before a -# build: one file is made per system. +# build: one file is made per system; so is a system whose row this mcpp +# builds no program for (`x86_64-macos`, a planned row). set -e TMP=$(mktemp -d) @@ -111,7 +112,12 @@ if [ -n "$T" ] && command -v python3 > /dev/null && "$MCPP" toolchain list --for fi grep -q "one single file per system" e.log || fail "E: the refusal" e.log grep -q "Compiling" e.log && fail "E: refused after compiling" e.log - echo "ok: -o with several systems is refused before a build" + if "$MCPP" pack --target x86_64-openkal --pkg-format single-file --systems linux,macos > e2.log 2>&1; then + fail "E: a single file for x86_64-macos was accepted" e2.log + fi + grep -q "x86_64-macos" e2.log || fail "E: the refusal does not name the row" e2.log + grep -q -- "--systems linux" e2.log || fail "E: the refusal does not give the systems that are built" e2.log + echo "ok: -o with several systems, and a system of an unbuilt row, are refused before a build" fi echo "PASS: 956 a single file runs without unpacking" From 33c1a9c426bbe6f1a69198b655db55a8d485ae30 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 12:32:03 +0900 Subject: [PATCH 44/54] pack: every system's runner and carrier is built over the platform's hosted arrangement where it serves the row (a macOS or Windows runner made on any machine, no SDK, no VCRUNTIME); a flat program tree drops its own description; a directory named by -o keeps its dots; trim_paths makes clang's __FILE__ separator the target's (an image built on Windows is the bytes of one built elsewhere); a configuration keeps the lock entries a platform added for another (build and pack --target no longer rewrite each other's lock); e2e 951 C, 954 H, 955 E; specs Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/specs/packaging.md | 8 +++--- docs/specs/platforms.md | 3 ++- src/build/flags.cppm | 12 +++++++++ src/build/prepare/records.cpp | 12 +++++++++ src/pack/pack.cppm | 26 +++++++++++++++---- .../951_an_ok_image_needs_no_declaration.sh | 6 +++-- tests/e2e/954_pack_has_three_axes.sh | 3 +++ ...versal_package_holds_every_architecture.sh | 21 +++++++++++++++ 8 files changed, 80 insertions(+), 11 deletions(-) diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 835e38b82..93995281d 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -174,9 +174,11 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 `--systems [,...]`(`linux`、`macos`、`windows`;缺省为本机系统)对 `self-contained` 的通用包,为每个系统 × 包的每个架构构建平台的载体(不携带任何内容,即运行器形式,SPEC-013 §11)并放入 -`libexec//-/`;对映像的单文件,为每个系统写一个文件(§5.2)。Linux 的程序为静态(musl);其他 -系统的按其惯常行(`-macos`、`-windows-gnu`)构建,与本机系统和架构相同时取本机的行;缺少某个系统的 -SDK 载荷时该次构建拒绝,写出该系统与载荷。 +`libexec//-/`;对映像的单文件,为每个系统写一个文件(§5.2)。每个系统的程序在其行 +(`-linux-musl`、`-macos`、`-windows-gnu`)上构建;平台的托管安排(SPEC-013 §5.4)服务该行时以它 +构建,于是同一程序在任何打包的机器上相同,不需要该系统的 SDK 与 C 运行时;Linux 的程序为静态。托管安排不服务该行时 +按惯常行构建,与本机系统和架构相同时取本机的行。本 mcpp 不构建程序的行(目标表标为 planned 或未列出)不放入包,并 +写出该系统与行。 ### 4.5 `mcpp-package.json` diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 5f2ca1547..9ac67fc9b 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -366,7 +366,8 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc ### 15.1 锁 平台加入的包在 `mcpp.lock` 中带 `origin = "platform"`;无 `origin` 的条目视为工程声明。引擎升级不替换已锁定的 -版本。**已实现** +版本。平台加入的包属于该平台目标的配置:一次配置没有解析到的此类条目保留在锁中,解析到时替换,因此为本机的构建与 +为 `x86_64-openkal` 的打包不互相改写锁。**已实现** ### 15.2 身份 diff --git a/src/build/flags.cppm b/src/build/flags.cppm index a910e57b3..1dc88e531 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -1194,10 +1194,22 @@ CompileFlags compute_flags(const BuildPlan& plan) { // the flags every unit shares made a dependency's command line // depend on the members the plan builds, and switching members // recompiled it. A unit's map is its package's (`packageTrimMaps`). + // + // AND THE SEPARATOR IS THE TARGET'S (clang 17 and later, + // `-ffile-reproducible`). What follows a mapped prefix is the path + // the preprocessor joined, with the host's separator: an ok image + // built on Windows named `…/libcxxabi/src\demangle/ItaniumDemangle.h` + // and was not the bytes the same image built on Linux is. + int major = 0; + const auto& v = plan.toolchain.version; + std::from_chars(v.data(), v.data() + v.size(), major); + const char* reproducible = + mcpp::toolchain::is_clang(plan.toolchain) && major >= 17 ? " -ffile-reproducible" : ""; for (auto const& [package, map] : plan.packageTrimMaps) { std::string rendered; for (auto const& [from, to] : map) rendered += " -ffile-prefix-map=" + escape_path(from) + "=" + to; + rendered += reproducible; f.trimByPackage.emplace(package, std::move(rendered)); } } diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index ca1098ee5..2631c8676 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -278,6 +278,18 @@ std::expected step13_lockfile(PrepareState& state, BuildConte if (!indicesHere.contains(i.name)) lock.indices.push_back(i); } } + // A PACKAGE A PLATFORM ADDED BELONGS TO THE CONFIGURATIONS OF THAT + // PLATFORM'S TARGETS (SPEC-013 §15.1): a configuration that does not + // resolve it keeps the prior lock's entry, so `mcpp build` for this machine + // and `mcpp pack --target x86_64-openkal` do not rewrite each other's + // lock. One that resolves the package replaces the entry. + if (auto prior = mcpp::lockfile::load(state.workRoot / "mcpp.lock")) { + std::set resolvedHere; + for (auto const& p : lock.packages) resolvedHere.insert(lock_identity(p)); + for (auto const& p : prior->packages) + if (p.origin == "platform" && resolvedHere.insert(lock_identity(p)).second) + lock.packages.push_back(p); + } if (!lock.packages.empty() || !lock.indices.empty()) { auto lockPath = state.workRoot / "mcpp.lock"; // `--locked` ASSERTS THAT THIS RESOLUTION IS THE RECORDED ONE. diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 6be7bbc6e..9e1762444 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -635,8 +635,10 @@ std::string wrapper_dirname_from_archive(const std::filesystem::path& archive) { std::string_view{".tar"}, std::string_view{".zip"}}) { if (name.ends_with(suffix)) return name.substr(0, name.size() - suffix.size()); } - // No recognised compression suffix — fall back to the bare stem. - return archive.stem().string(); + // No archive's suffix: the name as written (`-o app-0.1.0-openkal` with + // `--pkg-format dir` is that directory; a stem would cut at the version's + // last dot and write `app-0.1`). + return name; } } // namespace detail @@ -2274,9 +2276,17 @@ ProgramRow program_row(std::string_view system, std::string_view arch, bool sing const auto triple = std::format("{}-linux-musl", arch); return { triple, "static", hosted_platform_for(P, triple) }; } + // Another system's program is built over the platform's hosted arrangement + // where it serves the row, as Linux's is: the same program whichever + // machine packs it, needing no SDK and no C run-time of that system. + // Measured without it: on Linux an aarch64-macos runner needed the macOS + // SDK no payload carries, and a runner packed on Windows took the MSVC + // row and VCRUNTIME140.dll with it. + const auto triple = std::format("{}-{}", arch, system == "macos" ? "macos" : "windows-gnu"); + const auto mode = system == "macos" || !single ? "vendored" : "system"; + if (auto hosted = hosted_platform_for(P, triple); !hosted.empty()) return { triple, mode, hosted }; const bool here = system == host_system() && arch == mcpp::toolchain::platform::host_arch; - if (system == "macos") return { here ? std::string() : std::format("{}-macos", arch), "vendored" }; - return { here ? std::string() : std::format("{}-windows-gnu", arch), single ? "system" : "vendored" }; + return { here ? std::string() : triple, mode }; } // The bytes of `carried` as a section of a program of `system`, with the @@ -2330,7 +2340,13 @@ pack_generated_program(const std::filesystem::path& dir, const ProgramRow& row) return std::unexpected(Error{std::format( "packing the program under {} failed:\n{}", dir.string(), built.output)}); if (fs::is_directory(out / "bin", ec)) return out / "bin"; - if (fs::is_directory(out, ec)) return out; + // A flat tree (PE) holds the generated program's own description beside + // it; what is placed from here is the program, and the package it is + // placed into describes itself. + if (fs::is_directory(out, ec)) { + fs::remove(out / "mcpp-package.json", ec); + return out; + } return std::unexpected(Error{std::format("the program's package under {} was not written", out.string())}); } diff --git a/tests/e2e/951_an_ok_image_needs_no_declaration.sh b/tests/e2e/951_an_ok_image_needs_no_declaration.sh index cba6ef2b0..40cbea34f 100755 --- a/tests/e2e/951_an_ok_image_needs_no_declaration.sh +++ b/tests/e2e/951_an_ok_image_needs_no_declaration.sh @@ -16,7 +16,7 @@ # B. The report names the platform and its source, marks the layers it added # `platform`, and the toolchain line says the toolchain is the platform's. # C. mcpp.lock records the packages the platform added with -# `origin = "platform"`. +# `origin = "platform"`, and a build for this machine keeps them. # D. `mcpp test` builds and runs a test image; `mcpp pack` stages the image # and its loader-run program (M3). # E. `mcpp self env --target … --format json` names the platform, its fields' @@ -77,7 +77,9 @@ echo "ok: the report names the platform and what it added" # C grep -A5 '"openkal-musl"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: openkal-musl is not locked as the platform's" mcpp.lock grep -A5 '"openkal-llvm-runtime"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: openkal-llvm-runtime is not locked as the platform's" mcpp.lock -echo "ok: the lock says which packages the platform added" +"$MCPP" build > c-host.log 2>&1 || fail "C: the build for this machine" c-host.log +grep -A5 '"openkal-musl"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: the build for this machine dropped the platform's entries" mcpp.lock +echo "ok: the lock says which packages the platform added, whichever configuration wrote it last" # D "$MCPP" test --target "$T" > test.log 2>&1 || fail "D: mcpp test" test.log diff --git a/tests/e2e/954_pack_has_three_axes.sh b/tests/e2e/954_pack_has_three_axes.sh index 466e752da..339246a41 100755 --- a/tests/e2e/954_pack_has_three_axes.sh +++ b/tests/e2e/954_pack_has_three_axes.sh @@ -161,5 +161,8 @@ for f in d["files"]: assert not open(os.path.join(root, f["path"]), "rb").read().endswith(b"SIGNED"), f["path"] PY echo "ok: [pack] sign signs each program, and the digests are the signed files'" +"$MCPP" pack --pkg-format dir -o axes-0.1.0-named > h2.log 2>&1 || fail "H: pack -o with dots" h2.log +[ -d target/dist/axes-0.1.0-named ] || fail "H: -o axes-0.1.0-named did not name the directory" h2.log +echo "ok: a directory is named as -o writes it" echo "PASS: 954 pack has three axes" diff --git a/tests/e2e/955_a_universal_package_holds_every_architecture.sh b/tests/e2e/955_a_universal_package_holds_every_architecture.sh index 9ee0554c2..d8cf44513 100755 --- a/tests/e2e/955_a_universal_package_holds_every_architecture.sh +++ b/tests/e2e/955_a_universal_package_holds_every_architecture.sh @@ -15,6 +15,10 @@ # each architecture under `libexec//-/`, and the launcher # runs with no runner on PATH. # D. `--no-universal` packs the selected target alone. +# E. (Linux) `--systems linux,macos,windows` carries the runner of each +# system this mcpp builds one for, made here over the platform's hosted +# arrangement: a Mach-O for macos-aarch64, a PE for windows-x86_64, and +# none of them with the runner's own package description beside it. set -e TMP=$(mktemp -d) @@ -96,4 +100,21 @@ echo "ok: a self-contained universal package carries its runner and needs nothin [ -d target/dist/one/lib/$OTHER ] && fail "D: --no-universal packed the other architecture" d.log echo "ok: --no-universal packs the selected target alone" +# E +if [ "$OS" = linux ]; then + "$MCPP" pack --target "$T" --mode self-contained --systems linux,macos,windows --pkg-format dir -o uni-0.1.0-all \ + > e.log 2>&1 || fail "E: --systems linux,macos,windows" e.log + python3 - target/dist/uni-0.1.0-all <<'PY' || fail "E: the runners" e.log +import os, sys +root = os.path.join(sys.argv[1], "libexec", "okrun") +def head(p): return open(p, "rb").read(4) +assert head(os.path.join(root, "macos-aarch64", "okrun")) == b"\xcf\xfa\xed\xfe", "macos-aarch64 is not a Mach-O" +assert head(os.path.join(root, "windows-x86_64", "okrun.exe"))[:2] == b"MZ", "windows-x86_64 is not a PE" +for d, _, files in os.walk(root): + assert "mcpp-package.json" not in files, d +PY + grep -q "no runner for macos-x86_64" e.log || fail "E: the planned row is not said" e.log + echo "ok: the runners of other systems are made here, over the hosted arrangement" +fi + echo "PASS: 955 a universal package holds every architecture" From ae24ce827ed2d76411c6a7735a29b78f38c99732 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 12:56:39 +0900 Subject: [PATCH 45/54] test: a launcher run through a link names the image by its physical path (macOS's /var is a link) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- tests/unit/test_pack_distribution.cpp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index 6580a4320..ebe6050b0 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -261,7 +261,10 @@ TEST(PackDistribution, APosixLauncherRunsThroughALink) { (t.path / "path").string(), (t.path / "elsewhere/hello").string(), out.string()).c_str()); EXPECT_EQ(rc, 0) << slurp(out); - EXPECT_EQ(slurp(out), std::format("--image={}|a b|c|", (pkg / "lib" / arch / "hello.oke").string())); + // The physical path: the launcher follows links to the directories too + // (macOS's temporary directory is under a link, /var -> /private/var). + EXPECT_EQ(slurp(out), std::format("--image={}|a b|c|", + (fs::canonical(pkg) / "lib" / arch / "hello.oke").string())); } // A shared object is `.so` or `.so.`; a name that merely holds `.so` From 95827f79551884271c4577fdda6f681aaa87648d Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 12:15:40 +0900 Subject: [PATCH 46/54] platform: the ok-image rows select openkal-platform 0.17.3 (openkal-musl 0.22.6 takes a program's arguments when it is entered; openkal-llvm-runtime 0.16.6); docs and e2e 953 follow Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/04-mcpp-toml.md | 2 +- docs/10-pack-and-release.md | 4 +-- docs/24-openkal-cross.md | 2 +- docs/35-platforms.md | 32 +++++++++---------- docs/50-machine-output.md | 2 +- docs/specs/packaging.md | 2 +- docs/specs/platforms.md | 4 +-- docs/zh/04-mcpp-toml.md | 2 +- docs/zh/10-pack-and-release.md | 4 +-- docs/zh/24-openkal-cross.md | 2 +- docs/zh/35-platforms.md | 32 +++++++++---------- docs/zh/50-machine-output.md | 2 +- modules/toolchain-model/src/triple.cppm | 4 +-- ...ost_graph_runs_build_programs_as_images.sh | 2 +- 14 files changed, 48 insertions(+), 48 deletions(-) diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 53113efe4..7fed0511b 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1496,7 +1496,7 @@ the platform's own key names ([35 — Platforms](35-platforms.md)): ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.17.2" # or { path = "…" }, or "none" +platform = "openkal-platform@0.17.3" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 38d88dc80..580311816 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -845,7 +845,7 @@ An ok image's file is `target/dist/---` (`.exe` on Windows), one per system of `--systems` (default: this machine's). The carrier carries the image alone when it names no other image, and otherwise a stored zip of all the images, each aligned to 64 KiB, which the carrier's entry reads -through openkal-run's bundle interface (openkal-platform 0.17.2). +through openkal-run's bundle interface (openkal-platform 0.17.2 and later). ### `mcpp-package.json` @@ -855,7 +855,7 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it ([SPEC-014](specs/packaging.md) §4.5): ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.2", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.3", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index a6462cb7b..4bc07a540 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -111,7 +111,7 @@ practice. ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.2" +platform = "openkal-platform@0.17.3" ``` One table (2026.10.27.1+). It selects the openkal platform for the host triples, diff --git a/docs/35-platforms.md b/docs/35-platforms.md index d18dffaa9..2393dad8b 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -40,12 +40,12 @@ mcpp run --target x86_64-openkal ``` Resolved llvm@23.1.3 → x86_64-openkal → …/xim-x-llvm/23.1.3/bin/clang++ - the platform's (openkal-platform 0.17.2), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` + the platform's (openkal-platform 0.17.3), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` Target x86_64-openkal → x86_64-unknown-none-elf - compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.5, platform) - c-abi musl (openkal-musl@0.22.5, platform) - c++-abi libc++ (openkal-llvm-runtime@0.16.5, platform) - Platform openkal-platform 0.17.2 (built-in) + compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.6, platform) + c-abi musl (openkal-musl@0.22.6, platform) + c++-abi libc++ (openkal-llvm-runtime@0.16.6, platform) + Platform openkal-platform 0.17.3 (built-in) tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) @@ -109,9 +109,9 @@ The first of these that answers selects the target's platform: | Source | Spelling | |---|---| -| the command line | `--platform openkal-platform@0.17.2`, `--platform path:../openkal-platform`, `--platform none` | +| the command line | `--platform openkal-platform@0.17.3`, `--platform path:../openkal-platform`, `--platform none` | | the environment | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| the project | `[target.] platform = "openkal-platform@0.17.2"`, or `{ path = "…" }` | +| the project | `[target.] platform = "openkal-platform@0.17.3"`, or `{ path = "…" }` | | this machine | the same key in `~/.mcpp/config.toml` | | the target table | the platform column (`mcpp toolchain list`) | @@ -125,11 +125,11 @@ its own conventions. `[host] platform` selects a platform for them instead: ```toml [host] -platform = "openkal-platform@0.17.2" +platform = "openkal-platform@0.17.3" ``` ``` - Host platform openkal-platform 0.17.2 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun + Host platform openkal-platform 0.17.3 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun ``` Every build program then compiles as an ok image of this machine's @@ -147,7 +147,7 @@ implementation: ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.2" +platform = "openkal-platform@0.17.3" ``` On Windows the arrangement is the GNU ABI's row (`--target x86_64-windows-gnu`); @@ -161,12 +161,12 @@ packages pin one another exactly. A graph that holds another version of one of them is refused before anything compiles: ``` -error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.2 is a version group that names openkal-musl 0.22.5. +error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.3 is a version group that names openkal-musl 0.22.6. requested by: own The family's packages pin one another exactly, so two versions of one of them cannot be linked into one program. Either select the platform release of that version ([target.] platform = "openkal-platform@"), or write - openkal-musl = "0.22.5" where it is requested. + openkal-musl = "0.22.6" where it is requested. ``` A package taken from a directory or a repository is not bound by the group. @@ -180,7 +180,7 @@ abridged: [package] namespace = "mcpplibs" name = "openkal-platform" -version = "0.17.2" +version = "0.17.3" mcpp = ">=2026.10.27.1" [platform] @@ -192,9 +192,9 @@ universal = true plugin = "build.mcpp" [platform.layers] -compiler-runtime = "openkal-llvm-runtime@0.16.5" -"c++-abi" = "openkal-llvm-runtime@0.16.5" -c-abi = { name = "openkal-musl", version = "0.22.5", default-features = false, features = ["posix"] } +compiler-runtime = "openkal-llvm-runtime@0.16.6" +"c++-abi" = "openkal-llvm-runtime@0.16.6" +c-abi = { name = "openkal-musl", version = "0.22.6", default-features = false, features = ["posix"] } [platform.tools] okld = "openkal-tools@0.1.3" diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 97c9ede5c..a71615e5f 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -486,7 +486,7 @@ installation, which is located on the machine rather than installed by mcpp. A target row is `{target, note, toolchain, pin, status, default}`, and `status` is one of `installed` / `available` / `via dependency graph` / `planned`. A target row also carries `platform` (2026.10.27.1+): the platform package the -target table selects for it (`mcpplibs.openkal-platform@0.17.2`), or `null`. +target table selects for it (`mcpplibs.openkal-platform@0.17.3`), or `null`. **`toolchain` and `pin` are not the same field twice.** `toolchain` is what the row is associated with — the installed payload on an installed row, the diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 93995281d..6348f7c16 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -190,7 +190,7 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 "schema": 1, "name": "hello", "version": "0.1.0", - "platform": "openkal-platform@0.17.2", + "platform": "openkal-platform@0.17.3", "linkage": "declared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 9ac67fc9b..3d334d681 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -379,7 +379,7 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc 目标报告中,平台加入的层的来源写作 `platform`,图已提供的写作 `graph`。层之后为平台块: ``` - Platform openkal-platform 0.17.2 (built-in) + Platform openkal-platform 0.17.3 (built-in) tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) @@ -396,7 +396,7 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc ```json {"name": "mcpplibs.openkal-platform", "version": "0.17.1", "source": "built-in", "hosted": false, "fields": {"toolchain": {"value": "llvm@23.1.3", "source": "platform"}, - "layers.c-abi": {"value": "openkal-musl@0.22.5", "source": "platform"}}} + "layers.c-abi": {"value": "openkal-musl@0.22.6", "source": "platform"}}} ``` `fields` 的键为 `toolchain`、`runner`、`runners.`、`layers.`、`tools.`(值为所用程序的路径)、 diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index f04a91df5..fc18d6cbb 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1380,7 +1380,7 @@ runner 与 `run_under`。 ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.17.2" # or { path = "…" }, or "none" +platform = "openkal-platform@0.17.3" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index b9a3a547a..c8fdcbb67 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -721,7 +721,7 @@ macOS 上为 `~/Library/Caches`，Windows 上为 `%LOCALAPPDATA%`：先解到旁 ok 映像的文件为 `target/dist/---`（Windows 上带 `.exe`），`--systems` 的每个系统 （缺省为本机系统）各一个。映像不引用其他映像时载体只携带它，否则携带全部映像的一个 stored zip（每个映像对齐到 -64 KiB），载体的入口经 openkal-run 的 bundle 接口读取（openkal-platform 0.17.2）。 +64 KiB），载体的入口经 openkal-run 的 bundle 接口读取（openkal-platform 0.17.2 起）。 ### `mcpp-package.json` @@ -729,7 +729,7 @@ ok 映像的文件为 `target/dist/---`（Windows 上 提供的格式、`--merge` 与 `mcpp emit xpkg` 读取它（[SPEC-014](../specs/packaging.md) §4.5）： ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.2", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.3", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index 59b352687..9f1f3de4e 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -93,7 +93,7 @@ SDK，或者区分 `linux`/`windows`/`macos`。那些是 C 环境层或平台层 ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.2" +platform = "openkal-platform@0.17.3" ``` 一张表（2026.10.27.1+）。它为宿主三元组选择 openkal 平台，平台的宿主部分命名编译器与目标侧三个层的包； diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index a72facffd..ffee112b1 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -32,12 +32,12 @@ mcpp run --target x86_64-openkal ``` Resolved llvm@23.1.3 → x86_64-openkal → …/xim-x-llvm/23.1.3/bin/clang++ - the platform's (openkal-platform 0.17.2), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` + the platform's (openkal-platform 0.17.3), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` Target x86_64-openkal → x86_64-unknown-none-elf - compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.5, platform) - c-abi musl (openkal-musl@0.22.5, platform) - c++-abi libc++ (openkal-llvm-runtime@0.16.5, platform) - Platform openkal-platform 0.17.2 (built-in) + compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.6, platform) + c-abi musl (openkal-musl@0.22.6, platform) + c++-abi libc++ (openkal-llvm-runtime@0.16.6, platform) + Platform openkal-platform 0.17.3 (built-in) tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) @@ -96,9 +96,9 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 | 来源 | 写法 | |---|---| -| 命令行 | `--platform openkal-platform@0.17.2`、`--platform path:../openkal-platform`、`--platform none` | +| 命令行 | `--platform openkal-platform@0.17.3`、`--platform path:../openkal-platform`、`--platform none` | | 环境变量 | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| 工程 | `[target.] platform = "openkal-platform@0.17.2"`,或 `{ path = "…" }` | +| 工程 | `[target.] platform = "openkal-platform@0.17.3"`,或 `{ path = "…" }` | | 本机 | `~/.mcpp/config.toml` 中的同一个键 | | 目标表 | 平台列(`mcpp toolchain list`) | @@ -110,11 +110,11 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 ```toml [host] -platform = "openkal-platform@0.17.2" +platform = "openkal-platform@0.17.3" ``` ``` - Host platform openkal-platform 0.17.2 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun + Host platform openkal-platform 0.17.3 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun ``` 此后每个构建程序都编译为本机架构的 ok 映像,在 Linux、macOS 与 Windows 上使用同一个工具链,经平台的运行器 @@ -128,7 +128,7 @@ platform = "openkal-platform@0.17.2" ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.2" +platform = "openkal-platform@0.17.3" ``` Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[platform.hosted] targets` 之外的 @@ -140,12 +140,12 @@ Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[pl 另一个版本时,在任何编译之前拒绝: ``` -error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.2 is a version group that names openkal-musl 0.22.5. +error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.3 is a version group that names openkal-musl 0.22.6. requested by: own The family's packages pin one another exactly, so two versions of one of them cannot be linked into one program. Either select the platform release of that version ([target.] platform = "openkal-platform@"), or write - openkal-musl = "0.22.5" where it is requested. + openkal-musl = "0.22.6" where it is requested. ``` 从目录或仓库取得的包不受版本组约束。 @@ -158,7 +158,7 @@ error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0 [package] namespace = "mcpplibs" name = "openkal-platform" -version = "0.17.2" +version = "0.17.3" mcpp = ">=2026.10.27.1" [platform] @@ -170,9 +170,9 @@ universal = true plugin = "build.mcpp" [platform.layers] -compiler-runtime = "openkal-llvm-runtime@0.16.5" -"c++-abi" = "openkal-llvm-runtime@0.16.5" -c-abi = { name = "openkal-musl", version = "0.22.5", default-features = false, features = ["posix"] } +compiler-runtime = "openkal-llvm-runtime@0.16.6" +"c++-abi" = "openkal-llvm-runtime@0.16.6" +c-abi = { name = "openkal-musl", version = "0.22.6", default-features = false, features = ["posix"] } [platform.tools] okld = "openkal-tools@0.1.3" diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 6160a7a5f..556db9c18 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -433,7 +433,7 @@ mcpp toolchain list --format json 的形状是 `{target, note, toolchain, pin, status, default}`，`status` 取 `installed` / `available` / `via dependency graph` / `planned` 之一。 目标行另有 `platform`（2026.10.27.1+）：目标表为该目标选择的平台包 -（`mcpplibs.openkal-platform@0.17.2`），或 `null`。 +（`mcpplibs.openkal-platform@0.17.3`），或 `null`。 **`toolchain` 与 `pin` 不是把同一个字段写了两遍。** `toolchain` 表示这一行 关联到什么 —— 已装的行是装好的载荷，词表行是它自己的约定。`pin` 只承载目标 diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 65fba50ba..8457fac74 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -906,9 +906,9 @@ inline constexpr TargetInfo kKnownTargets[] = { // `openkal-platform` (SPEC-013), pinned here. The two architectures the // loader relocates. { "x86_64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.17.2" }, + "mcpplibs.openkal-platform@0.17.3" }, { "aarch64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.17.2" }, + "mcpplibs.openkal-platform@0.17.3" }, }; inline std::span known_targets() { return kKnownTargets; } diff --git a/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh b/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh index 8ed6dc437..48e64ae2d 100755 --- a/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh +++ b/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh @@ -34,7 +34,7 @@ name = "hostgraph" version = "0.1.0" [host] -platform = "openkal-platform@0.17.2" +platform = "openkal-platform@0.17.3" [build-dependencies] helper = { path = "helper" } From 3293985002f239380d4f5f2766e7ebe10fe7b441 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 13:47:03 +0900 Subject: [PATCH 47/54] CHANGELOG: runners over the hosted arrangement; trim_paths and the target's separator Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 406aad5fc..db21e4ba7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,8 +66,10 @@ with `lib//` per leg and generated launchers (`bin/`, `bin/.cmd`) that pick the machine's architecture and start the platform's runner; `--mode self-contained` adds the runner of each system of - `--systems` and each architecture under `libexec/`; `--no-universal` packs the - selected target alone. e2e 955. + `--systems` and each architecture under `libexec/`, built over the platform's + hosted arrangement on any machine (a macOS or Windows runner is packed on + Linux with no SDK and no C run-time of that system); `--no-universal` packs + the selected target alone. e2e 955. - **Single files.** `--pkg-format single-file` writes one executable: for a native program the engine's launcher carrying the staged tree as a zstd frame, extracted once into the user's cache under the payload's sha256 and @@ -107,6 +109,10 @@ index, its revision and age, and whether it was refreshed now and why not; the provider's own report is printed under `-v`. - A package shared by several configurations of one command is announced once. +- **With `trim_paths`, clang names a source with the target's separator** + (`-ffile-reproducible`, clang 17 and later): an artifact built on Windows + named `…/src\demangle/…` in `__FILE__` and was not the bytes of the one + built on Linux or macOS. - **`--pkg-format tar` writes `.tar.gz` on every host, Windows included**, with mcpp's own zlib; no `tar` or `gzip` program is run. `--compress none` writes `.tar`. From 7c5ee0bbb29bfe453887223bd996ab804727c6da Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 14:10:54 +0900 Subject: [PATCH 48/54] lock: what only the platform's packages reach is the platform's (origin = "platform"), the tools' package included; entries in one order by identity, so a build for this machine and a pack for an ok image leave the file as the other wrote it; e2e 951 C compares the bytes Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/specs/platforms.md | 5 ++- src/build/prepare/platform.cpp | 13 ++++-- src/build/prepare/records.cpp | 42 +++++++++++++++++-- .../951_an_ok_image_needs_no_declaration.sh | 9 ++-- 4 files changed, 57 insertions(+), 12 deletions(-) diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 3d334d681..9e173ef91 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -366,8 +366,9 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc ### 15.1 锁 平台加入的包在 `mcpp.lock` 中带 `origin = "platform"`;无 `origin` 的条目视为工程声明。引擎升级不替换已锁定的 -版本。平台加入的包属于该平台目标的配置:一次配置没有解析到的此类条目保留在锁中,解析到时替换,因此为本机的构建与 -为 `x86_64-openkal` 的打包不互相改写锁。**已实现** +版本。平台加入的包,以及只经由它们到达的包(层的实现、工具与插件的库),属于该平台目标的配置,同样带 +`origin = "platform"`:一次配置没有解析到的此类条目保留在锁中,解析到时替换;条目按身份排序。因此为本机的构建与为 +`x86_64-openkal` 的打包不互相改写锁。**已实现** ### 15.2 身份 diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index f290411f5..ea080a5f6 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -97,6 +97,13 @@ std::string qualified(const mcpp::manifest::Manifest& m) { return mcpp::pm::compat::qualified_name(n.namespace_, n.shortName); } +// A package the platform added, as `platformInjected` records it: qualified, +// the default namespace spelled out (the lock asks in that spelling). +std::string injected_name(const mcpp::manifest::DependencySpec& spec) { + return mcpp::pm::compat::qualified_name( + spec.namespace_.empty() ? std::string(mcpp::pm::kDefaultNamespace) : spec.namespace_, spec.shortName); +} + // The key a worklist item for a reference is written under: its name, or // the short name its source gives. std::string item_key(const mp::PackageRef& r) { @@ -335,8 +342,7 @@ std::expected step4b_apply_platform(PrepareState& state) { state.platformLayerSpecs[key] = item.spec; state.worklist.push_back(std::move(item)); state.platformInjected.insert("layer:" + key); - state.platformInjected.insert(mcpp::pm::compat::qualified_name( - state.worklist.back().spec.namespace_, state.worklist.back().spec.shortName)); + state.platformInjected.insert(injected_name(state.worklist.back().spec)); pushed = true; } @@ -384,8 +390,7 @@ std::expected step4b_apply_platform(PrepareState& state) { // so the tool provisioning pass builds them for this machine. state.m->buildDependencies[key] = item.spec; for (auto const& n : tools) state.platformInjected.insert("tool:" + n); - state.platformInjected.insert(mcpp::pm::compat::qualified_name(item.spec.namespace_, - item.spec.shortName)); + state.platformInjected.insert(injected_name(item.spec)); state.worklist.push_back(std::move(item)); pushed = true; } diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 2631c8676..3e47e8ae8 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -207,6 +207,38 @@ std::expected step13_lockfile(PrepareState& state, BuildConte lock.packages.push_back(std::move(lp)); } + // WHAT THE PROJECT'S OWN GRAPH REACHES (SPEC-013 §15.1): from the + // root and the selected members along the edges, never entering a + // package the platform added. A package the platform added, and one + // only those reach (its layers' implementations, its tools' and its + // plugin's libraries), is the platform's. + auto qualified_of = [](const mcpp::manifest::Manifest& m) { + auto n = mcpp::pm::compat::resolve_package_name(m.package.name, m.package.namespace_); + return mcpp::pm::compat::qualified_name( + n.namespace_.empty() ? std::string(mcpp::pm::kDefaultNamespace) : n.namespace_, n.shortName); + }; + std::set projectReaches; + { + const auto n = state.packages.size(); + std::vector own(n, false); + std::vector stack; + for (std::size_t i = 0; i < n; ++i) + if (i == 0 || state.packages[i].selectedMember) { own[i] = true; stack.push_back(i); } + while (!stack.empty()) { + const auto p = stack.back(); + stack.pop_back(); + for (auto const& e : state.dependencyEdges) { + const auto d = e.dependencyPackageIndex; + if (e.consumerPackageIndex != p || d >= n || own[d]) continue; + if (state.platformInjected.contains(qualified_of(state.packages[d].manifest))) continue; + own[d] = true; + stack.push_back(d); + } + } + for (std::size_t i = 0; i < n; ++i) + if (own[i]) projectReaches.insert(qualified_of(state.packages[i].manifest)); + } + // Version deps: the whole resolved graph, at the versions actually // chosen. `resolved` is an ordered map, so the file is deterministic. std::set versionLocked; @@ -237,9 +269,11 @@ std::expected step13_lockfile(PrepareState& state, BuildConte // prefix claimed otherwise. `index_package_digest` is FNV-1a on // every host. lp.hash = mcpp::pm::index_package_digest(sourceIndex, lp.name, lp.version); - // The platform's packages say so (SPEC-013 §8.3). - if (state.platformInjected.contains(mcpp::pm::compat::qualified_name( - key.ns.empty() ? std::string(mcpp::pm::kDefaultNamespace) : key.ns, key.shortName))) + // The platform's packages say so (SPEC-013 §15.1). + const auto qualified = mcpp::pm::compat::qualified_name( + key.ns.empty() ? std::string(mcpp::pm::kDefaultNamespace) : key.ns, key.shortName); + if (state.platform.active + && (state.platformInjected.contains(qualified) || !projectReaches.contains(qualified))) lp.origin = "platform"; // One entry per identity: a dependency declared by the root and by // a member under different spellings is one package. @@ -290,6 +324,8 @@ std::expected step13_lockfile(PrepareState& state, BuildConte if (p.origin == "platform" && resolvedHere.insert(lock_identity(p)).second) lock.packages.push_back(p); } + // One order whichever configuration wrote the file last: by identity. + std::ranges::stable_sort(lock.packages, {}, [](auto const& p) { return lock_identity(p); }); if (!lock.packages.empty() || !lock.indices.empty()) { auto lockPath = state.workRoot / "mcpp.lock"; // `--locked` ASSERTS THAT THIS RESOLUTION IS THE RECORDED ONE. diff --git a/tests/e2e/951_an_ok_image_needs_no_declaration.sh b/tests/e2e/951_an_ok_image_needs_no_declaration.sh index 40cbea34f..29a8e2967 100755 --- a/tests/e2e/951_an_ok_image_needs_no_declaration.sh +++ b/tests/e2e/951_an_ok_image_needs_no_declaration.sh @@ -15,8 +15,9 @@ # image runs through okrun, with exceptions and the standard library. # B. The report names the platform and its source, marks the layers it added # `platform`, and the toolchain line says the toolchain is the platform's. -# C. mcpp.lock records the packages the platform added with -# `origin = "platform"`, and a build for this machine keeps them. +# C. mcpp.lock records the packages the platform added, and those only they +# reach, with `origin = "platform"`; a build for this machine leaves the +# file as it is. # D. `mcpp test` builds and runs a test image; `mcpp pack` stages the image # and its loader-run program (M3). # E. `mcpp self env --target … --format json` names the platform, its fields' @@ -77,8 +78,10 @@ echo "ok: the report names the platform and what it added" # C grep -A5 '"openkal-musl"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: openkal-musl is not locked as the platform's" mcpp.lock grep -A5 '"openkal-llvm-runtime"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: openkal-llvm-runtime is not locked as the platform's" mcpp.lock +grep -A5 '"openkal-tools"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: the platform's tools are not locked as the platform's" mcpp.lock +cp mcpp.lock lock.image "$MCPP" build > c-host.log 2>&1 || fail "C: the build for this machine" c-host.log -grep -A5 '"openkal-musl"' mcpp.lock | grep -q 'origin = "platform"' || fail "C: the build for this machine dropped the platform's entries" mcpp.lock +cmp -s lock.image mcpp.lock || { diff lock.image mcpp.lock > c-lock.diff; fail "C: the build for this machine rewrote the lock" c-lock.diff; } echo "ok: the lock says which packages the platform added, whichever configuration wrote it last" # D From 85060788565eb5def6b06b6cd32b5298d6d0e128 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 15:45:56 +0900 Subject: [PATCH 49/54] =?UTF-8?q?prepare:=20the=20Windows=20first=20run's?= =?UTF-8?q?=20closures=20are=20their=20own=20step=20and=20the=20graph's=20?= =?UTF-8?q?supply=20of=20a=20system=20a=20helper=20(both=20phase=20functio?= =?UTF-8?q?ns=20under=20the=20400-line=20gate);=20CHANGELOG=20and=20SPEC-0?= =?UTF-8?q?14=20=C2=A74.6=20state=20what=20-ffile-reproducible=20does=20an?= =?UTF-8?q?d=20does=20not=20(an=20image=20built=20on=20Windows=20keeps=20c?= =?UTF-8?q?lang's=20`\`=20in=20`=5F=5FFILE=5F=5F`,=20#850)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 10 +++-- docs/specs/packaging.md | 3 +- src/build/prepare/toolchain.cpp | 19 +++++---- src/build/prepare/toolchain_decision.cpp | 52 ++++++++++++++---------- 4 files changed, 50 insertions(+), 34 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index db21e4ba7..985d660ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -109,10 +109,12 @@ index, its revision and age, and whether it was refreshed now and why not; the provider's own report is printed under `-v`. - A package shared by several configurations of one command is announced once. -- **With `trim_paths`, clang names a source with the target's separator** - (`-ffile-reproducible`, clang 17 and later): an artifact built on Windows - named `…/src\demangle/…` in `__FILE__` and was not the bytes of the one - built on Linux or macOS. +- **With `trim_paths`, clang is given `-ffile-reproducible`** (clang 17 and + later): `__FILE__` takes the target's separator where clang applies it. An ok + image built on Linux and one built on macOS are the same bytes; one built on + a Windows host is not, because clang keeps the `\` it joined an included + path with for a POSIX target (#850, upstream), and `mcpp pack --merge` + refuses its images beside the others. - **`--pkg-format tar` writes `.tar.gz` on every host, Windows included**, with mcpp's own zlib; no `tar` or `gzip` program is run. `--compress none` writes `.tar`. diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 6348f7c16..d01cfd182 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -222,7 +222,8 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 `mcpp pack --merge --merge ...`(也可 `--merge ,`)读入至少两个本命令写出的包(目录,或 `.zip`、 `.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz` 归档),按 `mcpp-package.json` 合并为一棵树:名字、版本、平台、 -链接与模式**必须**相同;腿与系统取并集;每个文件在复制前计算摘要并与记录的摘要比较,不符时拒绝并写出文件、输入 +链接与模式**必须**相同;腿与系统取并集(同一映像在 Linux 与 macOS 上构建出相同字节;Windows 主机上 clang 在 +`__FILE__` 中保留它拼接包含路径时用的 `\`,其映像不同,合并时被拒绝,mcpp#850);每个文件在复制前计算摘要并与记录的摘要比较,不符时拒绝并写出文件、输入 与两个摘要;同一路径的文件摘要**必须**相同,每个文件只取一份;合并后的树重新描述。`mcpp-package.json` 中的每个 路径**必须**是包内的规范相对路径。输出**必须**是新的路径,或一个已有的、不是输入之一的本命令写出的归档;目录输出 只写新目录。 diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index 5dda17092..1ed8c41e7 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -426,6 +426,17 @@ static std::expected step1_define_early_toolchain_closures(Pr // every later reader parses: a managed spec, or `path:`. if (auto r = step1_local_toolchain(state); !r) return std::unexpected(r.error()); + // `[target.]`'s build-shaping keys, applied the same way whichever + // path found the row (#704). The section's toolchain is a statement about + // this row the author wrote down, so it replaces `[toolchain]` and the + // global default; `--toolchain` and a consumer's decision for a host tool + // are statements about THIS invocation and keep precedence over it. + return {}; +} + +// The closures that answer "is MSVC usable here" and "what would a native +// build pick", and the Windows first run that seeds the GNU target from them. +static void step1_define_host_default_closures(PrepareState& state) { // ─── Windows first run without Visual Studio ──────────────────────── // The host triple on Windows is MSVC-ABI, so the historical default // (llvm) resolves to clang targeting MSVC — which uses the MSVC STL and @@ -482,13 +493,6 @@ static std::expected step1_define_early_toolchain_closures(Pr } } } - - // `[target.]`'s build-shaping keys, applied the same way whichever - // path found the row (#704). The section's toolchain is a statement about - // this row the author wrote down, so it replaces `[toolchain]` and the - // global default; `--toolchain` and a consumer's decision for a host tool - // are statements about THIS invocation and keep precedence over it. - return {}; } // A phase-local struct passed by reference to the steps that resolve one @@ -1341,6 +1345,7 @@ std::expected phase1_toolchain_spec_and_axes(PrepareState& st !r) return std::unexpected(r.error()); if (auto r = step1_define_early_toolchain_closures(state); !r) return std::unexpected(r.error()); + step1_define_host_default_closures(state); if (auto r = step1_target_and_static_overrides(state); !r) return std::unexpected(r.error()); // The platform, once the target is known and before the conditional // tables merge: `cfg(platform = ...)` is one of their keys (SPEC-013 §3). diff --git a/src/build/prepare/toolchain_decision.cpp b/src/build/prepare/toolchain_decision.cpp index 7a3b435ec..9d399c73e 100644 --- a/src/build/prepare/toolchain_decision.cpp +++ b/src/build/prepare/toolchain_decision.cpp @@ -37,6 +37,35 @@ import mcpp.toolchain.post_install; namespace mcpp::build { +namespace { + +// WHAT "THE GRAPH SUPPLIES THE SYSTEM" MEANS FOR THIS TARGET. A C library of +// the graph does, on any target. A kernel-abi package does only where the +// target has no system of its own (a freestanding or a loader-supplied row): +// on a hosted target it sits over the system's C library, which the row's +// payload still supplies. Measured on linux-aarch64: openkal-tools reaches +// openkal-linux (kernel-abi over the system's C library), the musl row's +// convention was cancelled by it, and `mcpp pack --linkage static` compiled the +// musl target with the machine's llvm against glibc's headers. +bool graph_supplies_system(const PrepareState& state) { + const auto targetTriple = mcpp::toolchain::triple::parse( + state.overrides.target_triple.empty() ? mcpp::toolchain::triple::host_triple().str() + : state.overrides.target_triple); + const bool targetHasNoSystem = targetTriple + && (targetTriple->is_freestanding() || targetTriple->loader_supplies_os()); + auto supplies_system = [&](const std::string& entry) { + auto cap = mcpp::targetside::parse_capability(entry); + if (!cap || !*cap) return false; + return (*cap)->layer == mcpp::targetside::CapLayer::CAbi + || ((*cap)->layer == mcpp::targetside::CapLayer::KernelAbi && targetHasNoSystem); + }; + return std::ranges::any_of(state.packages, [&](auto const& pkg) { + return std::ranges::any_of(pkg.manifest.provides, supplies_system); + }); +} + +} // namespace + std::expected phase5_toolchain_after_graph(PrepareState& state) { // ─── The toolchain, resolved now that the graph exists ────────────────── @@ -161,29 +190,8 @@ std::expected phase5_toolchain_after_graph(PrepareState& stat reqCompilerBy, family, family, family)); }; - // WHAT "THE GRAPH SUPPLIES THE SYSTEM" MEANS FOR THIS TARGET. A C - // library of the graph does, on any target. A kernel-abi package does - // only where the target has no system of its own (a freestanding or a - // loader-supplied row): on a hosted target it sits over the system's C - // library, which the row's payload still supplies. Measured on - // linux-aarch64: openkal-tools reaches openkal-linux (kernel-abi over - // the system's C library), the musl row's convention was cancelled by - // it, and `mcpp pack --linkage static` compiled the musl target with - // the machine's llvm against glibc's headers. - const auto targetTriple = mcpp::toolchain::triple::parse( - state.overrides.target_triple.empty() ? mcpp::toolchain::triple::host_triple().str() - : state.overrides.target_triple); - const bool targetHasNoSystem = targetTriple - && (targetTriple->is_freestanding() || targetTriple->loader_supplies_os()); - auto supplies_system = [&](mcpp::targetside::CapLayer layer) { - return layer == mcpp::targetside::CapLayer::CAbi - || (layer == mcpp::targetside::CapLayer::KernelAbi && targetHasNoSystem); - }; + graphSuppliesSystem = graph_supplies_system(state); for (auto const& pkg : state.packages) { - for (auto const& entry : pkg.manifest.provides) { - auto cap = mcpp::targetside::parse_capability(entry); - if (cap && *cap && supplies_system((*cap)->layer)) graphSuppliesSystem = true; - } for (auto const& entry : pkg.manifest.requires_) { auto cap = mcpp::targetside::parse_capability(entry); if (!cap || !*cap) continue; From 75755996b66b90f63d173ab37f7539f3034a3059 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 16:11:52 +0900 Subject: [PATCH 50/54] =?UTF-8?q?lock:=20each=20entry=20names=20the=20conf?= =?UTF-8?q?igurations=20that=20resolve=20it=20(`targets`,=20absent=20for?= =?UTF-8?q?=20the=20native=20configuration=20alone);=20a=20configuration?= =?UTF-8?q?=20adds=20its=20target=20to=20what=20it=20resolved=20and=20take?= =?UTF-8?q?s=20it=20from=20what=20it=20did=20not,=20and=20drops=20what=20n?= =?UTF-8?q?one=20resolves;=20`origin=20=3D=20"platform"`=20only=20where=20?= =?UTF-8?q?every=20configuration=20says=20so;=20a=20native-only=20lock=20i?= =?UTF-8?q?s=20unchanged=20(e2e=20844=20holds).=20Spec=20=C2=A715.1,=20CHA?= =?UTF-8?q?NGELOG,=20unit=20test?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 10 ++++-- docs/specs/platforms.md | 7 ++-- src/build/prepare/records.cpp | 59 +++++++++++++++++++++++++++------- src/pm/lock_io.cppm | 12 +++++++ tests/unit/test_pm_lock_io.cpp | 31 ++++++++++++++++++ 5 files changed, 101 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 985d660ef..c87a4d798 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -91,9 +91,13 @@ - The `mcpp.pack` envelope gains `linkage`, `mode`, `pkgFormat`, `legs`, `systems` and each file artifact's `sha256`. A platform's plugin may provide package formats (`pack-format`, `artifact` actions) in the platform phase. -- `mcpp.lock` records the packages a platform added with - `origin = "platform"`; the `plan` event and `result.json` carry - `configuration.platform` and `hostPlatform`. +- `mcpp.lock` records the packages a platform added, and those only they + reach, with `origin = "platform"`, and names each entry's configurations in + `targets` (absent for an entry of this machine's native configuration alone, + so a lock of a natively built project is unchanged): a build for this machine + and a build or pack for an ok image leave each other's entries as they were. + The `plan` event and `result.json` carry `configuration.platform` and + `hostPlatform`. ### Changed diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 9e173ef91..dbb9cdd21 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -366,9 +366,10 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc ### 15.1 锁 平台加入的包在 `mcpp.lock` 中带 `origin = "platform"`;无 `origin` 的条目视为工程声明。引擎升级不替换已锁定的 -版本。平台加入的包,以及只经由它们到达的包(层的实现、工具与插件的库),属于该平台目标的配置,同样带 -`origin = "platform"`:一次配置没有解析到的此类条目保留在锁中,解析到时替换;条目按身份排序。因此为本机的构建与为 -`x86_64-openkal` 的打包不互相改写锁。**已实现** +版本。平台加入的包,以及只经由它们到达的包(层的实现、工具与插件的库),带 `origin = "platform"`(当解析它的每个配置 +都这样看时)。每个条目以 `targets` 写出解析它的配置(目标三元组,排序);只属于本机原生配置的条目不写 `targets`, +因此只在本机构建的工程的锁照旧。一次配置把自己的目标加入它解析到的条目、从没有解析到的条目中去掉;不再被任何配置 +解析的条目被删除;条目按身份排序。为本机的构建与为 `x86_64-openkal` 的构建、打包因此不互相改写锁。**已实现** ### 15.2 身份 diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 3e47e8ae8..9dfaaae81 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -294,6 +294,53 @@ std::expected step13_lockfile(PrepareState& state, BuildConte for (auto const& mp : state.overrides.workspace_request) planned.insert(mp); partialWorkspace = !std::ranges::includes(planned, all); } + // EACH ENTRY NAMES THE CONFIGURATIONS THAT RESOLVE IT (SPEC-013 + // §15.1): this one's target is added to what it resolved, and taken + // from what it did not; an entry no configuration resolves any more is + // dropped. A build for this machine and a pack for an ok image then + // leave each other's entries as they were. An entry of this machine's + // own configuration alone says nothing (`targets` absent), so the lock + // of a project built only natively reads as it always has. (A partial + // workspace plan keeps the entries it did not resolve whole, below.) + { + const auto native = mcpp::toolchain::triple::host_triple().str(); + const auto here = [&] { + auto t = mcpp::toolchain::triple::parse(state.overrides.target_triple); + return t ? t->str() : native; + }(); + auto targets_of = [&](const mcpp::lockfile::LockedPackage& p) { + return p.targets.empty() ? std::vector{ native } : p.targets; + }; + std::map was; + if (auto prior = mcpp::lockfile::load(state.workRoot / "mcpp.lock")) + for (auto const& p : prior->packages) was.emplace(lock_identity(p), p); + std::set listed; + for (auto& p : lock.packages) { + const auto id = lock_identity(p); + listed.insert(id); + auto it = was.find(id); + if (it == was.end() || it->second.version != p.version) { p.targets = { here }; continue; } + p.targets = targets_of(it->second); + // The platform's only where every configuration that resolves + // it says so: one whose project reaches it (`os` on Linux + // reaches openkal) keeps it the project's. + const bool others = std::ranges::any_of(p.targets, [&](auto const& t) { return t != here; }); + if (others && it->second.origin != "platform") p.origin.clear(); + if (std::ranges::find(p.targets, here) == p.targets.end()) p.targets.push_back(here); + } + for (auto& [id, p] : was) { + if (partialWorkspace || listed.contains(id)) continue; + auto kept = p; + kept.targets = targets_of(p); + std::erase(kept.targets, here); + if (!kept.targets.empty()) lock.packages.push_back(std::move(kept)); + } + for (auto& p : lock.packages) { + std::ranges::sort(p.targets); + if (p.targets == std::vector{ native }) p.targets.clear(); + } + + } if (partialWorkspace) { if (auto prior = mcpp::lockfile::load(state.workRoot / "mcpp.lock")) { // By identity, not by spelling: a prior entry that names a @@ -312,18 +359,6 @@ std::expected step13_lockfile(PrepareState& state, BuildConte if (!indicesHere.contains(i.name)) lock.indices.push_back(i); } } - // A PACKAGE A PLATFORM ADDED BELONGS TO THE CONFIGURATIONS OF THAT - // PLATFORM'S TARGETS (SPEC-013 §15.1): a configuration that does not - // resolve it keeps the prior lock's entry, so `mcpp build` for this machine - // and `mcpp pack --target x86_64-openkal` do not rewrite each other's - // lock. One that resolves the package replaces the entry. - if (auto prior = mcpp::lockfile::load(state.workRoot / "mcpp.lock")) { - std::set resolvedHere; - for (auto const& p : lock.packages) resolvedHere.insert(lock_identity(p)); - for (auto const& p : prior->packages) - if (p.origin == "platform" && resolvedHere.insert(lock_identity(p)).second) - lock.packages.push_back(p); - } // One order whichever configuration wrote the file last: by identity. std::ranges::stable_sort(lock.packages, {}, [](auto const& p) { return lock_identity(p); }); if (!lock.packages.empty() || !lock.indices.empty()) { diff --git a/src/pm/lock_io.cppm b/src/pm/lock_io.cppm index a1a8dab8c..99f43752f 100644 --- a/src/pm/lock_io.cppm +++ b/src/pm/lock_io.cppm @@ -38,6 +38,10 @@ struct LockedPackage { // for a package the project declared or one reached through it; a lock // written before the field reads as empty. std::string origin; + // The configurations that resolve it, by target triple (sorted): a build + // for one target rewrites its own and keeps the others' (SPEC-013 §15.1). + // Empty in a lock written before the field. + std::vector targets; }; // Parsed form of a git source string as written to mcpp.lock. @@ -144,6 +148,9 @@ std::expected load(const std::filesystem::path& path) { if (auto it = tt.find("source"); it != tt.end() && it->second.is_string()) lp.source = it->second.as_string(); if (auto it = tt.find("hash"); it != tt.end() && it->second.is_string()) lp.hash = it->second.as_string(); if (auto it = tt.find("origin"); it != tt.end() && it->second.is_string()) lp.origin = it->second.as_string(); + if (auto it = tt.find("targets"); it != tt.end() && it->second.is_array()) + for (auto const& e : it->second.as_array()) + if (e.is_string()) lp.targets.push_back(e.as_string()); // v1 → v2 migration: default namespace to "mcpplibs". if (fileVersion < 2 && lp.namespace_.empty()) { @@ -196,6 +203,11 @@ std::string serialize(const Lockfile& lock) { out += std::format("source = {}\n", t::escape_string(p.source)); out += std::format("hash = {}\n", t::escape_string(p.hash)); if (!p.origin.empty()) out += std::format("origin = {}\n", t::escape_string(p.origin)); + if (!p.targets.empty()) { + std::string list; + for (auto const& t : p.targets) list += (list.empty() ? "" : ", ") + t::escape_string(t); + out += std::format("targets = [{}]\n", list); + } out += "\n"; } return out; diff --git a/tests/unit/test_pm_lock_io.cpp b/tests/unit/test_pm_lock_io.cpp index b1ac74f58..d0d3c146d 100644 --- a/tests/unit/test_pm_lock_io.cpp +++ b/tests/unit/test_pm_lock_io.cpp @@ -213,3 +213,34 @@ TEST(PmLockIo, UpdatingALinkedLockKeepsTheLinkAndUpdatesItsTarget) { ASSERT_EQ(loaded->packages.size(), 1u); EXPECT_EQ(loaded->packages[0].version, "2.0.0"); } + +// An entry names the configurations that resolve it, and reads back as written +// (SPEC-013 §15.1); a lock written before the field reads with none. +TEST(PmLockIo, TargetsRoundTrip) { + const auto dir = std::filesystem::temp_directory_path() / std::format("mcpp-lock-{}", + std::chrono::steady_clock::now().time_since_epoch().count()); + std::filesystem::create_directories(dir); + mcpp::pm::Lockfile lock; + lock.schemaVersion = 2; + mcpp::pm::LockedPackage p; + p.name = "openkal-musl"; + p.namespace_ = "mcpplibs"; + p.version = "0.22.6"; + p.source = "index+mcpplibs@0.22.6"; + p.hash = "fnv1a:0"; + p.origin = "platform"; + p.targets = { "aarch64-openkal", "x86_64-openkal" }; + lock.packages.push_back(p); + ASSERT_TRUE(mcpp::pm::write(lock, dir / "mcpp.lock")); + auto back = mcpp::pm::load(dir / "mcpp.lock"); + ASSERT_TRUE(back); + ASSERT_EQ(back->packages.size(), 1u); + EXPECT_EQ(back->packages[0].targets, p.targets); + EXPECT_EQ(back->packages[0].origin, "platform"); + std::ofstream(dir / "old.lock") << "version = 2\n\n[package.\"x\"]\nnamespace = \"mcpplibs\"\nversion = \"1\"\n"; + auto old = mcpp::pm::load(dir / "old.lock"); + ASSERT_TRUE(old); + EXPECT_TRUE(old->packages.at(0).targets.empty()); + std::error_code ec; + std::filesystem::remove_all(dir, ec); +} From 8ae5699ab414ed7ca9c81584c70ee53d5189f54a Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 17:03:59 +0900 Subject: [PATCH 51/54] =?UTF-8?q?review=20(ecosystem):=20emit=20xpkg=20nam?= =?UTF-8?q?es=20the=20entry=20after=20the=20command=20it=20registers=20and?= =?UTF-8?q?=20moves=20the=20archive's=20own=20top=20directory=20(mcpp-lang?= =?UTF-8?q?uage-server=20runs=20as=20mcppls;=20okrun's=20archives=20are=20?= =?UTF-8?q?okrun---);=20the=20lock's=20native=20configuration?= =?UTF-8?q?=20is=20the=20default=20target;=20universal,=20leg=20and=20sing?= =?UTF-8?q?le-file=20combinations=20refused=20before=20the=20build;=20a=20?= =?UTF-8?q?library's=20package=20is=20written=20by=20mcpp=20(tar=20compres?= =?UTF-8?q?sions,=20zip=20as=20stated,=20no=20single=20file);=20the=20buil?= =?UTF-8?q?t-in=20formats=20are=20tar,=20zip,=20dir,=20single-file;=20docs?= =?UTF-8?q?:=20SPEC-013=20=C2=A79.2=20lld=20names=20and=20=C2=A715.1,=20SP?= =?UTF-8?q?EC-014=20=C2=A75.1=20default=20levels,=20docs/10=20runners=20an?= =?UTF-8?q?d=20refusal=20rows=20(en,=20zh),=20docs/50=20formats,=20docs/90?= =?UTF-8?q?=20counts,=20CHANGELOG=20Fixed=20heading=20and=20the=200.17.3?= =?UTF-8?q?=20pins;=20e2e=20242,=20638;=20unit?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 14 +++++- docs/10-pack-and-release.md | 16 ++++-- docs/35-platforms.md | 4 +- docs/50-machine-output.md | 6 +-- docs/90-build-from-source.md | 8 +-- docs/specs/packaging.md | 2 +- docs/specs/platforms.md | 17 ++++--- docs/zh/10-pack-and-release.md | 11 ++++- docs/zh/35-platforms.md | 3 +- docs/zh/50-machine-output.md | 6 +-- docs/zh/90-build-from-source.md | 8 +-- modules/manifest/src/platform_toml.cppm | 2 +- src/build/prepare/records.cpp | 18 +++++-- src/cli/cmd_publish.cppm | 2 +- src/pack/distribution.cppm | 32 +++++++++--- src/pack/library.cppm | 30 +++++------- src/pack/library_pipeline.cppm | 21 ++++++-- src/pack/pipeline.cppm | 49 +++++++++++++------ src/pack/stage_tree.cppm | 4 +- .../242_pack_library_interface_and_headers.sh | 14 ++++++ tests/e2e/638_pack_format_dispatch.sh | 2 +- tests/unit/test_pack_distribution.cpp | 40 ++++++++++++++- tests/unit/test_pack_stage_tree.cpp | 21 ++++---- 23 files changed, 234 insertions(+), 96 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c87a4d798..0f9d2df60 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -120,9 +120,19 @@ path with for a POSIX target (#850, upstream), and `mcpp pack --merge` refuses its images beside the others. - **`--pkg-format tar` writes `.tar.gz` on every host, Windows included**, with - mcpp's own zlib; no `tar` or `gzip` program is run. `--compress none` writes - `.tar`. + mcpp's own zlib, for a program's package and a library's alike; no `tar` or + `gzip` program is run. `--compress none` writes `.tar`. +- `x86_64-openkal` and `aarch64-openkal` select openkal-platform 0.17.3 + (openkal-musl 0.22.6, openkal-llvm-runtime 0.16.6): the C library takes a + program's arguments when it is entered, which a hosted program on macOS did + not receive. +- A platform's link with `ld.lld` itself finds the linker as `ld.lld`, `lld` + or `lld-link` beside the compiler (a Windows payload ships `lld-link`, run + with `-flavor gnu`); `mcpp explain ` explains each `platform-*` + reason. A build program's cached run is replayed only while what it named + below its output directory is there (e2e 958). +### Fixed - **A version the local index lacks is refreshed for when its install is refused.** The walk refreshes the index when a descriptor is missing, and not diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 580311816..99deef270 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -115,8 +115,12 @@ choices: | `--compress gzip` or `xz` with `--pkg-format single-file` | `--compress zstd[:]` or `none`: the launcher decompresses zstd | | several `--target` for an executable outside a universal package | one pack per target | | `--systems` on anything but a self-contained universal package or a single file of an image | `--mode self-contained` over a platform with `universal = true`, or `--pkg-format single-file` | +| an image's single file for a system whose row this mcpp builds no program for (`x86_64-macos`, `aarch64-windows-gnu`) | `--systems` naming the systems that are built | +| an image's single file for several `--systems` with `-o` | leave `-o` out: one file per system | | a universal package of a platform that states no `runner` | `--no-universal` | +| `--target` outside the platform's `targets` in a universal package | a target the platform serves | | a dispatched format for a `lib` or `shared` target | the library route, [12](12-binary-distribution.md) | +| `--pkg-format single-file` for a `lib` or `shared` target | `tar`, `zip` or `dir` | A `[pack] compress` that does not fit the container (a directory, `zstd` for a zip) is left aside for that pack; only the flag is refused. @@ -799,10 +803,14 @@ Under `--mode self-contained` the runner travels: for each system of `--systems linux,macos,windows` (default: this machine's) and each architecture of the package, mcpp builds the platform's carrier with nothing carried — a program that runs the image its first argument names, as `okrun` does — into -`libexec//-/`. A Linux runner is static (musl); another -system's is built for its conventional row (`-macos`, -`-windows-gnu`), and a system whose SDK payload this machine lacks is -refused by that build, naming it. +`libexec//-/`. Each runner is built on its row +(`-linux-musl`, `-macos`, `-windows-gnu`) over the platform's +hosted arrangement, so the same program results whichever machine packs, with +no SDK or C run-time of that system; a Linux runner is static. A row this mcpp +builds no program for (`x86_64-macos`, planned; `aarch64-windows-gnu`, not in +its table) is left out with a warning, and the launcher there looks for the +runner on `PATH`. An image built on a Windows host is not the bytes of one +built elsewhere (#850): pack universal packages on Linux or macOS. Packages built on several machines are joined with `mcpp pack --merge a.zip --merge b.zip` (also `--merge a.zip,b.zip`). The inputs diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 2393dad8b..40246b781 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -55,7 +55,9 @@ mcpp run --target x86_64-openkal Every value is reported with its source. `mcpp test`, `mcpp pack` and `--target aarch64-openkal` need nothing more. `mcpp.lock` records the packages -the platform added with `origin = "platform"`. +the platform added with `origin = "platform"`, and each entry's configurations +in `targets` (absent for this machine's native configuration alone), so a build +for this machine and one for an ok image leave each other's entries in place. ## What a platform states diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index a71615e5f..e988d6cc4 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -104,7 +104,7 @@ silent fallback. A command that builds asks for machine output with `--message-format json` instead, as `mcpp test` does: `build`, `test`, `run` and `pack`. The package -format of `mcpp pack` and `mcpp run` is `--pkg-format` (`tar`, `dir`, `msi`; +format of `mcpp pack` and `mcpp run` is `--pkg-format` (`tar`, `zip`, `dir`, `single-file`, `msi`; 2026.10.27.1+; `--format` there is the earlier spelling and warns), so the report of a pack is `mcpp pack --message-format json` (2026.9.16.1+). The shape follows the kind: `mcpp test` streams a record per test because tests finish over time, and @@ -349,7 +349,7 @@ select as they do for a build ([35](35-platforms.md)). "triple": "x86_64-openkal", "graph": "target", // or "host" "toolchain": {"compiler": "/…/xim-x-llvm/23.1.3/bin/clang++", "origin": "platform"}, - "platform": {"name": "mcpplibs.openkal-platform", "version": "0.17.1", "source": "built-in", + "platform": {"name": "mcpplibs.openkal-platform", "version": "0.17.3", "source": "built-in", "hosted": false, "fields": {"runner": {"value": ["okrun", "{}"], "source": "platform"}}}, "tools": {"okld": "/…/okld", "okrun": "/…/okrun"}, "runner": {"argv": ["okrun", "{}"], "host": "same-arch", "exclusive": false, @@ -1026,7 +1026,7 @@ project's `[indices]` entries, when run in a project). The packages are mcpp pack --list-formats ``` -`data.formats` lists the engine's own formats (`tar`, `dir`; `builtin: true`) +`data.formats` lists the engine's own formats (`tar`, `zip`, `dir`, `single-file`; `builtin: true`) and each format a package of the resolved graph provides (`builtin: false`, with `providers`). The graph is planned in a directory of the mcpp home, so nothing is written into the project. diff --git a/docs/90-build-from-source.md b/docs/90-build-from-source.md index c91a75101..4892ab520 100644 --- a/docs/90-build-from-source.md +++ b/docs/90-build-from-source.md @@ -81,19 +81,19 @@ src/ └── scaffold/ `mcpp new` and templates tests/ -├── unit/ 155 C++ tests, discovered by `mcpp test` -└── e2e/ 644 shell scripts against a real binary +├── unit/ 156 C++ tests, discovered by `mcpp test` +└── e2e/ 645 shell scripts against a real binary ``` ## Test Organization Two layers, and they answer different questions. -**Unit tests** (`tests/unit/`, 155 files) are C++ programs `mcpp test` +**Unit tests** (`tests/unit/`, 156 files) are C++ programs `mcpp test` discovers. They exercise a module's contract with no binary and no filesystem state. -**End-to-end tests** (`tests/e2e/NN_.sh`, 644 files) run a real `mcpp` +**End-to-end tests** (`tests/e2e/NN_.sh`, 645 files) run a real `mcpp` binary against a real project. `run_all.sh` is the CI entry point. `mcpp test` does **not** run them. diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index d01cfd182..d016911e8 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -237,7 +237,7 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 `--compress [:]` 与 `[pack] compress` 选择 `tar` 的压缩:mcpp 自己写出 ustar 归档(按 路径排序;长名与链接目标用 pax 头;属主为 0,时间为 0,模式 0755、0644、0777),以自带的编解码器(zlib、zstd、 liblzma)压缩,不调用 `tar` 等外部程序;产物扩展名为 `.tar.gz`(缺省)、`.tar.zst`、`.tar.xz` 或 `.tar`。级别 -范围:gzip 1–9,zstd 1–22,xz 0–9;写出的级别照用(含 `xz:0`、`zstd:1`),未写时取编解码器的缺省。`zip` 的条目在变小时以 DEFLATE 压缩(`gzip[:]` 设定级别),否则存储; +范围:gzip 1–9,zstd 1–22,xz 0–9;写出的级别照用(含 `xz:0`、`zstd:1`),未写时为 gzip 的缺省(6)、zstd 19、xz 6。`zip` 的条目在变小时以 DEFLATE 压缩(`gzip[:]` 设定级别),否则存储; `none` 时全部存储。单文件的载荷总是一个 zstd 帧(启动器只解 zstd),`none` 取级别 1。okb 中的映像不压缩(SPEC-013 平台插件的格式规定)。同一棵树的 归档在每个宿主上字节相同:gzip 成员不带名字与时间、系统字节为 255,zip 条目使用固定日期。 diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index dbb9cdd21..74a67fde0 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -274,8 +274,8 @@ plugin = "build.mcpp" | `shared` | 共享产物的名字模式,`{name}` 恰好一次,不含目录 | | `seal` | 封装步骤,每步一个 argv(§9.3) | -`driver = "lld"` 而工具链载荷中没有 `ld.lld` 时拒绝(`lld-required-absent`),消息写出平台的工具链。程序名为 -`lld` 时引擎加 `-flavor gnu`。链接或封装命名的程序既非图中为本机构建、也不在本机可找到时拒绝 +`driver = "lld"` 时引擎在编译器旁依次查找 `ld.lld`、`lld`、`lld-link`(Windows 载荷只带 `lld-link`);都没有时拒绝 +(`lld-required-absent`),消息写出平台的工具链。程序名不是 `ld.lld` 时引擎加 `-flavor gnu`。链接或封装命名的程序既非图中为本机构建、也不在本机可找到时拒绝 (`platform-tool-absent`),消息写出负责构建它的包,或写出 `[target.'cfg(platform = "")'.tools]` 的写法。 **已实现** @@ -365,11 +365,12 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc ### 15.1 锁 -平台加入的包在 `mcpp.lock` 中带 `origin = "platform"`;无 `origin` 的条目视为工程声明。引擎升级不替换已锁定的 -版本。平台加入的包,以及只经由它们到达的包(层的实现、工具与插件的库),带 `origin = "platform"`(当解析它的每个配置 -都这样看时)。每个条目以 `targets` 写出解析它的配置(目标三元组,排序);只属于本机原生配置的条目不写 `targets`, -因此只在本机构建的工程的锁照旧。一次配置把自己的目标加入它解析到的条目、从没有解析到的条目中去掉;不再被任何配置 -解析的条目被删除;条目按身份排序。为本机的构建与为 `x86_64-openkal` 的构建、打包因此不互相改写锁。**已实现** +平台加入的包,以及只经由它们到达的包(层的实现、工具与插件的库),在 `mcpp.lock` 中带 `origin = "platform"`(当解析 +它的每个配置都这样看时);无 `origin` 的条目属于工程。每个条目以 `targets` 写出解析它的配置(目标三元组,排序);原生 +配置——不带 `--target` 的构建解析到的目标,即 `[build] target`、本机的缺省目标或本机三元组——独有的条目不写 +`targets`,读取时无 `targets` 的条目属于写锁的机器的原生配置。一次配置把自己的目标加入它解析到的条目、从没有解析到的 +条目中去掉,不再被任何配置解析的条目被删除;一个条目的版本改变时其 `targets` 只剩这次的配置;条目按身份排序。为本机的 +构建与为 `x86_64-openkal` 的构建、打包因此不互相改写锁。**已实现** ### 15.2 身份 @@ -396,7 +397,7 @@ Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mc `null`;对象形式: ```json -{"name": "mcpplibs.openkal-platform", "version": "0.17.1", "source": "built-in", "hosted": false, +{"name": "mcpplibs.openkal-platform", "version": "0.17.3", "source": "built-in", "hosted": false, "fields": {"toolchain": {"value": "llvm@23.1.3", "source": "platform"}, "layers.c-abi": {"value": "openkal-musl@0.22.6", "source": "platform"}}} ``` diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index c8fdcbb67..088ad8c4d 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -103,8 +103,12 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 | `--compress gzip` 或 `xz` 与 `--pkg-format single-file` | `--compress zstd[:]` 或 `none`：启动器解压 zstd | | 通用包之外、对一个可执行文件给出多个 `--target` | 每个目标打一次 | | 自带加载器的通用包与映像的单文件之外使用 `--systems` | 在声明 `universal = true` 的平台上用 `--mode self-contained`，或 `--pkg-format single-file` | +| 映像的单文件，`--systems` 中某系统的行本 mcpp 不构建程序（`x86_64-macos`、`aarch64-windows-gnu`） | 只列出会构建的系统的 `--systems` | +| 映像的单文件，多个 `--systems` 与 `-o` 同用 | 去掉 `-o`：每个系统一个文件 | | 平台未声明 `runner` 时打通用包 | `--no-universal` | +| 通用包中给出平台 `targets` 之外的 `--target` | 平台服务的目标 | | 对 `lib` 或 `shared` 目标使用被分派的格式 | 库的路线，[12](12-binary-distribution.md) | +| 对 `lib` 或 `shared` 目标使用 `--pkg-format single-file` | `tar`、`zip` 或 `dir` | 与容器不相容的 `[pack] compress`（目录，或 zip 的 `zstd`）在这次打包中被搁置；只有选项被拒绝。 @@ -691,8 +695,11 @@ target/dist/hello-0.1.0-openkal/ `--mode self-contained` 时运行器随包携带：对 `--systems linux,macos,windows`（缺省为本机系统）的每个系统与包的 每个架构，mcpp 构建不携带任何内容的平台载体 —— 一个像 `okrun` 一样运行其第一个参数所指映像的程序 —— 放入 -`libexec//-/`。Linux 的运行器为静态（musl）；其他系统的按其惯常行（`-macos`、 -`-windows-gnu`）构建，本机缺少该系统的 SDK 载荷时该次构建拒绝并点名。 +`libexec//-/`。每个运行器在其行（`-linux-musl`、`-macos`、`-windows-gnu`）上 +以平台的托管安排构建，因此无论在哪台机器上打包结果都是同一个程序，不需要该系统的 SDK 与 C 运行时；Linux 的运行器 +为静态。本 mcpp 不构建程序的行（`x86_64-macos` 为 planned；`aarch64-windows-gnu` 不在目标表中）不放入包并给出警告， +启动器在那里从 `PATH` 查找运行器。在 Windows 主机上构建的映像与其他主机上的字节不同（#850）：通用包在 Linux 或 +macOS 上打包。 多台机器打出的包以 `mcpp pack --merge a.zip --merge b.zip`（也可 `--merge a.zip,b.zip`）合并。输入是本命令写出的 包目录或归档（`.zip`、`.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz`），不进行任何构建。腿与系统取并集，每个 diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index ffee112b1..2a0b2c5a4 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -46,7 +46,8 @@ mcpp run --target x86_64-openkal ``` 每个值都带来源报告。`mcpp test`、`mcpp pack` 与 `--target aarch64-openkal` 无需更多声明。`mcpp.lock` 以 -`origin = "platform"` 记录平台加入的包。 +`origin = "platform"` 记录平台加入的包,并以 `targets` 写出每个条目的配置(只属于本机原生配置的条目不写), +因此为本机的构建与为 ok 映像的构建不互相改写对方的条目。 ## 平台陈述的内容 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 556db9c18..c7c0acb35 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -93,7 +93,7 @@ mcpp --format json 构建类命令改用 `--message-format json` 来请求机器输出，`mcpp test` 就是这样做的： `build`、`test`、`run` 与 `pack`。`mcpp pack` 与 `mcpp run` 的包格式是 `--pkg-format` -（`tar`、`dir`、`msi`；2026.10.27.1+；在这两个命令上 `--format` 是旧写法，会给出警告），所以打包的 +（`tar`、`zip`、`dir`、`single-file`、`msi`；2026.10.27.1+；在这两个命令上 `--format` 是旧写法，会给出警告），所以打包的 报告方式是 `mcpp pack --message-format json`（2026.9.16.1+）。具体形状随 kind 而定： `mcpp test` 的测试是随时间陆续完成的，所以它按测试逐条输出流式记录；一次 `mcpp pack` 只有一个结果，所以它输出一个信封。`mcpp build --message-format json` @@ -307,7 +307,7 @@ mcpp self env --format json "triple": "x86_64-openkal", "graph": "target", // or "host" "toolchain": {"compiler": "/…/xim-x-llvm/23.1.3/bin/clang++", "origin": "platform"}, - "platform": {"name": "mcpplibs.openkal-platform", "version": "0.17.1", "source": "built-in", + "platform": {"name": "mcpplibs.openkal-platform", "version": "0.17.3", "source": "built-in", "hosted": false, "fields": {"runner": {"value": ["okrun", "{}"], "source": "platform"}}}, "tools": {"okld": "/…/okld", "okrun": "/…/okrun"}, "runner": {"argv": ["okrun", "{}"], "host": "same-arch", "exclusive": false, @@ -917,7 +917,7 @@ mcpp index list --format json mcpp pack --list-formats ``` -`data.formats` 列出引擎自己的格式（`tar`、`dir`；`builtin: true`）以及已解析依赖 +`data.formats` 列出引擎自己的格式（`tar`、`zip`、`dir`、`single-file`；`builtin: true`）以及已解析依赖 图中某个包提供的每种格式（`builtin: false`，带 `providers`）。依赖图在 mcpp home 的 一个目录中规划，因此不向工程写入任何东西。 diff --git a/docs/zh/90-build-from-source.md b/docs/zh/90-build-from-source.md index e01f3bc31..7d6c0097e 100644 --- a/docs/zh/90-build-from-source.md +++ b/docs/zh/90-build-from-source.md @@ -83,18 +83,18 @@ src/ └── scaffold/ `mcpp new` and templates tests/ -├── unit/ 155 C++ tests, discovered by `mcpp test` -└── e2e/ 644 shell scripts against a real binary +├── unit/ 156 C++ tests, discovered by `mcpp test` +└── e2e/ 645 shell scripts against a real binary ``` ## 测试组织 两层，回答的是不同的问题。 -**单元测试**（`tests/unit/`，155 个文件）是 `mcpp test` 发现的 C++ +**单元测试**（`tests/unit/`，156 个文件）是 `mcpp test` 发现的 C++ 程序。它们检验一个模块的契约，不需要二进制、也不需要文件系统状态。 -**端到端测试**（`tests/e2e/NN_.sh`，644 个文件）拿真实的 `mcpp` +**端到端测试**（`tests/e2e/NN_.sh`，645 个文件）拿真实的 `mcpp` 二进制跑真实工程。`run_all.sh` 是 CI 的入口。`mcpp test` **不**运行它们。 ```bash diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm index c9618e650..6e821e9c0 100644 --- a/modules/manifest/src/platform_toml.cppm +++ b/modules/manifest/src/platform_toml.cppm @@ -429,7 +429,7 @@ Reader::layers(const toml::Value& v, std::string_view where, if (!v.is_table()) return std::unexpected(std::format( "{} must be a table of = | false, for example " - "c-abi = \"openkal-musl@0.22.3\"", where)); + "c-abi = \"openkal-musl@0.22.6\"", where)); for (auto const& [k, val] : v.as_table()) { auto key = std::format("{}.{}", where, k); if (!is_layer(k)) diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 9dfaaae81..7c0e24c9b 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -303,11 +303,21 @@ std::expected step13_lockfile(PrepareState& state, BuildConte // of a project built only natively reads as it always has. (A partial // workspace plan keeps the entries it did not resolve whole, below.) { - const auto native = mcpp::toolchain::triple::host_triple().str(); - const auto here = [&] { - auto t = mcpp::toolchain::triple::parse(state.overrides.target_triple); - return t ? t->str() : native; + // The native configuration is the one a build without --target + // resolves to: the project's `[build] target`, this machine's + // default target, or this machine's triple. + const auto canonical = [](const std::string& text) -> std::optional { + auto t = mcpp::toolchain::triple::parse(text); + if (text.empty() || !t) return std::nullopt; + return t->str(); + }; + const auto native = [&] { + if (auto t = canonical(state.m->buildConfig.target)) return *t; + if (auto cfg = state.get_cfg(false); cfg) + if (auto t = canonical((*cfg)->defaultTarget)) return *t; + return mcpp::toolchain::triple::host_triple().str(); }(); + const auto here = canonical(state.overrides.target_triple).value_or(native); auto targets_of = [&](const mcpp::lockfile::LockedPackage& p) { return p.targets.empty() ? std::vector{ native } : p.targets; }; diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 81031d7c1..d57e15048 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -61,7 +61,7 @@ export int cmd_emit_xpkg(const mcpplibs::cmdline::ParsedArgs& parsed) { if (!root) { mcpp::ui::error(root.error()); std::filesystem::remove_all(work, ec); return 2; } auto pkg = d::read(*root); if (!pkg) { mcpp::ui::error(pkg.error()); std::filesystem::remove_all(work, ec); return 2; } - assets.push_back({ file, std::move(*pkg), d::sha256_file(file) }); + assets.push_back({ file, std::move(*pkg), d::sha256_file(file), root->filename().string() }); } std::filesystem::remove_all(work, ec); auto lua = d::xpkg_descriptor(assets, parsed.value("url").value_or("")); diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 0b87e9de3..4cb8979e7 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -145,6 +145,7 @@ struct Asset { std::filesystem::path file; Package pkg; std::string sha256; + std::string top; // the archive's one top-level directory }; // The xim-pkgindex entry (spec 2) of `assets`, fetched from `/ xpkg_descriptor(const std::vector std::string xpm; for (auto const& [system, text] : blocks) xpm += std::format(" {} = {{\n{} }},\n", system, text); - // Where the program a person runs lies in the installed tree. + // The command a person runs, and the entry's name: the launcher of a + // universal package (named after its images, `bin/`), else the + // program -- not the package, which may be named otherwise + // (`mcpp-language-server` runs as `mcppls`). + std::string command; + if (universal && !first.entry.empty()) + command = std::filesystem::path(first.entry.begin()->second).stem().string(); + else if (!first.program.empty()) + command = std::filesystem::path(first.program).stem().string(); + if (command.empty()) command = first.name; + // Where the program lies in the installed tree. std::string bindir = "\"bin\"", alias; if (universal) { alias = std::format("if os.host() == \"windows\" then opts.alias = \"{}.cmd\" end\n ", - lua_text(first.name)); + lua_text(command)); } else { const auto program = std::filesystem::path(first.program); bindir = program.has_parent_path() ? std::format("\"{}\"", lua_text(program.parent_path().generic_string())) : std::string("\".\""); } - const auto wrapper = lua_text(std::format("{}-{}-", first.name, first.version)); + // The archives' top-level directory: the one of a single asset, else what + // the assets' names share (they differ by system and architecture). + std::string top = assets.front().top; + for (auto const& a : assets) { + const auto [p, q] = std::ranges::mismatch(top, a.top); + top.erase(static_cast(p - top.begin())); + } + const auto wrapper = lua_text(assets.size() == 1 ? top : top + "*"); return std::format( "-- Generated by `mcpp emit xpkg --package` from the packages' mcpp-package.json\n" "-- (SPEC-014 §8).\n" "package = {{\n" " spec = \"2\",\n" " name = \"{0}\",\n" - " description = \"{0} {1}{2}\",\n" + " description = \"{8} {1}{2}\",\n" " type = \"package\",\n" " archs = {3},\n" " programs = {{\"{0}\"}},\n" @@ -1562,7 +1580,7 @@ std::expected xpkg_descriptor(const std::vector "function install()\n" " -- The archive's one top-level directory is the package.\n" " os.tryrm(pkginfo.install_dir())\n" - " for _, d in ipairs(os.dirs(\"{5}*\")) do\n" + " for _, d in ipairs(os.dirs(\"{5}\")) do\n" " os.mv(d, pkginfo.install_dir())\n" " return true\n" " end\n" @@ -1577,9 +1595,9 @@ std::expected xpkg_descriptor(const std::vector " xvm.remove(\"{0}\")\n" " return true\n" "end\n", - lua_text(first.name), lua_text(first.version), + lua_text(command), lua_text(first.version), universal ? std::format(", one package for every system ({})", lua_text(first.platform)) : std::string(), - lua_list(archs), xpm, wrapper, bindir, alias); + lua_list(archs), xpm, wrapper, bindir, alias, lua_text(first.name)); } // ─── --merge ────────────────────────────────────────────────────────────── diff --git a/src/pack/library.cppm b/src/pack/library.cppm index 5c56a9811..9d76773fa 100644 --- a/src/pack/library.cppm +++ b/src/pack/library.cppm @@ -42,6 +42,8 @@ import mcpp.pack.relocate; import mcpp.pack.strip; import mcpp.source_kind; // builtin_extension_table — what needs no declaring import mcpp.pack.zip; +import mcpp.pack.compress; +import mcpp.pack.distribution; import mcpplibs.os.platform; import mcpp.build.command_line; import mcpp.run; @@ -89,6 +91,8 @@ struct LibraryPackPlan { std::filesystem::path archivePath; // …/.tar.gz | .zip bool writeArchive = true; // false = `--format dir` bool zip = false; // PE targets get a .zip + mcpp::pack::compress::Choice compression; // the tar's, or the zip's members' + std::string namespace_, packageName, packageVersion, builtBy; std::string targetName; @@ -497,27 +501,15 @@ run_library_pack(const LibraryPackPlan& plan) .source = plan.stagingRoot / rel, }); } - if (auto r = zip::write(plan.archivePath, entries); !r) + zip::WriteOptions zo; + zo.deflate = plan.compression.algorithm != mcpp::pack::compress::Algorithm::None; + if (plan.compression.algorithm == mcpp::pack::compress::Algorithm::Gzip) zo.level = plan.compression.level; + if (auto r = zip::write(plan.archivePath, entries, zo); !r) return std::unexpected(LibraryPackError{ r.error() }); } else { - // `--force-local` on Windows, and it is not optional there: GNU tar - // reads `C:/path/x.tar.gz` as the rsh form `host:path`, tries to resolve - // a machine called `C`, and fails with - // tar (child): Cannot connect to C: resolve failed - // which names neither tar's argument nor the drive letter as the cause. - // - // Reached only by a package with more than one leg: a single PE leg is - // written as a zip, so a Windows host never ran this path until a fat - // package existed. Same fix, same reason, in pack.cppm's make_tarball. - std::vector argv{ "tar" }; - if constexpr (mcpplibs::os::is_windows) argv.push_back("--force-local"); - argv.insert(argv.end(), { "-czf", plan.archivePath.string(), - "-C", plan.stagingRoot.parent_path().string(), - plan.stagingRoot.filename().string() }); - auto r = mcpp::run::execute(argv); - if (r.exit_code != 0) - return std::unexpected(LibraryPackError{ std::format( - "tar failed (rc={}): {}", r.exit_code, r.output) }); + // Written by mcpp, as a program's tar is: the same bytes on every host. + if (auto r = mcpp::pack::distribution::write_tar(plan.stagingRoot, plan.archivePath, plan.compression); !r) + return std::unexpected(LibraryPackError{ r.error() }); } return plan.archivePath; } diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 27c3b0fc4..3a22d3469 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -38,6 +38,7 @@ import mcpp.pack; import mcpp.pack.abi_tag; import mcpp.pack.interface; import mcpp.pack.library; +import mcpp.pack.compress; import mcpp.pack.strip; import mcpp.toolchain.model; import mcpp.toolchain.dialect; @@ -137,10 +138,19 @@ export int build_and_pack_library(const std::string& targetName, mcpp::build::progress::defer_finished(); } + // A library's package is a tree, a tar or a zip; a single file runs a + // program, and a library has none. + if (opts.format == mcpp::pack::Format::SingleFile) { + mcpp::ui::error(std::format( + "--pkg-format single-file packs a program, and '{}' is a library.\n" + " use: --pkg-format tar | zip | dir", targetName)); + return 2; + } LibraryPackPlan plan; plan.builtBy = std::string(mcpp::MCPP_VERSION); plan.writeArchive = opts.format == mcpp::pack::Format::Tar || opts.format == mcpp::pack::Format::Zip; + plan.compression = opts.compression; InterfaceClosure closure; bool haveClosure = false; @@ -530,15 +540,20 @@ export int build_and_pack_library(const std::string& targetName, } // ── where it lands ──────────────────────────────────────────────── - const bool zip = plan.legs.size() == 1 - && plan.legs[0].triple.find("windows") != std::string::npos; + // The container: as stated, else a zip for a Windows leg and a tar + // elsewhere, as a program's is. + const bool zip = opts.format == mcpp::pack::Format::Zip + || (!opts.formatStated && plan.legs.size() == 1 + && plan.legs[0].triple.find("windows") != std::string::npos); auto dirName = plan.legs.size() == 1 ? std::format("{}-{}-{}", plan.packageName, plan.packageVersion, plan.legs[0].abiTag) : std::format("{}-{}", plan.packageName, plan.packageVersion); plan.zip = zip; plan.stagingRoot = plan.projectRoot / "target" / "dist" / dirName; plan.archivePath = plan.projectRoot / "target" / "dist" - / (dirName + (zip ? ".zip" : ".tar.gz")); + / (dirName + (zip ? std::string(".zip") + : std::string(mcpp::pack::compress::tar_extension( + plan.compression.algorithm)))); if (!opts.output.empty()) { auto o = std::filesystem::path(opts.output); plan.archivePath = o.has_parent_path() ? o diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 459c125e6..b5a1e03e8 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -415,6 +415,31 @@ bool packs_universal(const mcpp::build::BuildContext& ctx, const Options& opts) && opts.format != mcpp::pack::Format::SingleFile; } +// The combinations of a member's pack refused before anything is built +// (SPEC-014 §1.5): what its options and the plan's platform already decide. +std::optional refused_combination(const mcpp::build::BuildContext& ctx, const Options& opts) { + // A single file carries what it runs, whichever source stated the mode. + if (opts.format == mcpp::pack::Format::SingleFile && opts.mode == mcpp::pack::Mode::None) + return std::string("--pkg-format single-file carries what the program runs, and the mode is " + "system ([pack] default_mode).\n use: --mode vendored | self-contained"); + if (!opts.legTargets.empty() && !packs_universal(ctx, opts)) + return std::string("several --target name one package's legs only where the target's platform " + "packs them into one universal package (`[platform] universal = true`).\n" + " use: one pack per target"); + if (!packs_universal(ctx, opts)) return std::nullopt; + auto const& P = ctx.plan.platform; + if (!P.fields.runner.set || P.fields.runner.cmd.empty()) + return std::format("a universal package starts its images through the platform's runner, and " + "{} states none.\n use: --no-universal", P.display()); + auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + for (auto const& target : opts.legTargets) { + auto lt = mcpp::toolchain::triple::parse(target); + if (lt && t && lt->str() != t->str() && !mcpp::platform::serves(P.fields, P.package, *lt)) + return std::format("--target {} is not a target of {}", lt->str(), P.display()); + } + return std::nullopt; +} + std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { auto& ctx = *g.ctx; const bool sharedPlan = ctx.workspaceMembers.size() > 1; @@ -594,16 +619,7 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { const bool loaded = t && t->loader_supplies_os(); if (loaded && opts.mode == mcpp::pack::Mode::Static) opts.mode = mcpp::pack::Mode::BundleAll; auto const& P = ctx.plan.platform; - // A single file carries what it runs, whichever source stated the mode. - if (opts.format == mcpp::pack::Format::SingleFile && opts.mode == mcpp::pack::Mode::None) - return Refusal{2, about(run, m, std::string( - "--pkg-format single-file carries what the program runs, and the mode is system " - "([pack] default_mode).\n use: --mode vendored | self-contained"))}; - if (!opts.legTargets.empty() && !packs_universal(ctx, opts)) - return Refusal{2, about(run, m, std::string( - "several --target name one package's legs only where the target's platform packs " - "them into one universal package (`[platform] universal = true`).\n" - " use: one pack per target"))}; + if (auto why = refused_combination(ctx, opts)) return Refusal{2, about(run, m, *why)}; if (packs_universal(ctx, opts)) { opts.universalName = mcpp::manifest::platform::platform_name( P.package.substr(P.package.find('.') + 1)); @@ -613,9 +629,6 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { for (auto const& target : legs) { auto lt = mcpp::toolchain::triple::parse(target); if (!lt || lt->str() == t->str()) continue; - if (!mcpp::platform::serves(P.fields, P.package, *lt)) - return Refusal{2, about(run, m, std::format( - "--target {} is not a target of {}", lt->str(), P.display()))}; mcpp::build::BuildOverrides ov = g.ov; ov.target_triple = lt->str(); // The member's own plan, in a workspace; the project's otherwise. @@ -793,6 +806,14 @@ std::optional prepare_group(PackRun& run, GroupJob& g) { std::optional check_format(PackRun& run, GroupJob& g) { auto const& ctx = *g.ctx; const auto& formatName = run.opts.formatName; + // What each member's options and the platform decide. + for (auto const& m : g.members) { + std::optional why; + mcpp::build::with_member(*g.ctx, m.inWorkspace ? m.name : std::string_view{}, [&] { + why = refused_combination(ctx, m.request); + }); + if (why) return Refusal{2, about(run, m, *why)}; + } // ─── `--systems` names the loaders a package carries ───────────── // // A self-contained universal package carries a loader per system, and a @@ -863,7 +884,7 @@ std::optional check_format(PackRun& run, GroupJob& g) { return Refusal{2, std::format( "unknown --pkg-format '{}'.\n" " available in this build: {}\n" - " A format past `tar` and `dir` comes from a package in the " + " A format past the built-in ones comes from a package in the " "resolved graph, which declares\n" " it with `mcpp::provides_pack_format(\"\")` in its build " "program. Add the package\n" diff --git a/src/pack/stage_tree.cppm b/src/pack/stage_tree.cppm index c38ea83a8..f1e682fc3 100644 --- a/src/pack/stage_tree.cppm +++ b/src/pack/stage_tree.cppm @@ -209,13 +209,13 @@ bool write_stage_manifest(const std::filesystem::path& stagingRoot, return static_cast(os); } -// The two names `--format` answers for without consulting the graph. +// The names `--pkg-format` answers for without consulting the graph. // // Held here rather than in the CLI because two layers need the same list: the // parser decides whether a value is a built-in or a dispatch, and the // declaration check refuses a package that claims one of them. A package // claiming `tar` would be silently unreachable, since the built-in wins. -constexpr std::array kBuiltinPackFormats{"tar", "dir"}; +constexpr std::array kBuiltinPackFormats{"tar", "zip", "dir", "single-file"}; bool is_builtin_pack_format(std::string_view name) { return std::ranges::find(kBuiltinPackFormats, name) != kBuiltinPackFormats.end(); diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh index f7eb5c306..5997ef5bf 100755 --- a/tests/e2e/242_pack_library_interface_and_headers.sh +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -65,6 +65,20 @@ EOF cd mathkit "$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +# The container as stated, compressed by mcpp itself; a single file is a +# program's, not a library's. +"$MCPP" pack mathkit --compress zstd > pack-zst.log 2>&1 || { cat pack-zst.log; echo "pack --compress zstd failed"; exit 1; } +zst="$(find target/dist -maxdepth 1 -type f -name 'mathkit-0.1.0-*.tar.zst' | head -1)" +[[ -n "$zst" ]] || { cat pack-zst.log; ls target/dist; echo "no .tar.zst for --compress zstd"; exit 1; } +[[ "$(head -c 4 "$zst" | od -An -tx1 | tr -d ' \n')" == "28b52ffd" ]] || { echo "$zst is not a zstd frame"; exit 1; } +"$MCPP" pack mathkit --pkg-format zip > pack-zip.log 2>&1 || { cat pack-zip.log; echo "pack --pkg-format zip failed"; exit 1; } +ls target/dist/mathkit-0.1.0-*.zip > /dev/null 2>&1 || { cat pack-zip.log; ls target/dist; echo "no zip for --pkg-format zip"; exit 1; } +if "$MCPP" pack mathkit --pkg-format single-file > pack-sf.log 2>&1; then + cat pack-sf.log; echo "a library was packed as a single file"; exit 1 +fi +grep -q "is a library" pack-sf.log || { cat pack-sf.log; echo "the single-file refusal does not say why"; exit 1; } +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } + pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" # The manifest below is FILE CONTENT: on Git Bash a shell-spelled # /tmp/... path is read by a native mcpp.exe as "root of the current diff --git a/tests/e2e/638_pack_format_dispatch.sh b/tests/e2e/638_pack_format_dispatch.sh index 683aa9a60..cf8d338dc 100755 --- a/tests/e2e/638_pack_format_dispatch.sh +++ b/tests/e2e/638_pack_format_dispatch.sh @@ -133,7 +133,7 @@ grep -q "unknown --pkg-format 'bogus'" b2.log \ || { cat b2.log; echo "FAIL: the refusal does not name the value"; exit 1; } # THE LIST IS THE POINT. A refusal that named a fixed list would be the # coupling this mechanism removes; this one names the graph's own answer. -grep -q "available in this build: tar, dir, zap" b2.log \ +grep -q "available in this build: tar, zip, dir, single-file, zap" b2.log \ || { cat b2.log; echo "FAIL: the refusal does not name what is available"; exit 1; } # Nothing was compiled to find that out. grep -q "Compiling app" b2.log \ diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index ebe6050b0..175dca5bf 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -594,7 +594,7 @@ TEST(PackDistribution, XpkgDescriptorWritesMetadataAsStrings) { a.pkg.entry = { { "x86_64", "lib/x86_64/evil.oke" } }; auto text = d::xpkg_descriptor({ a }, "https://example.org/v1.0"); ASSERT_TRUE(text) << text.error(); - EXPECT_NE(text->find("name = \"ev\\\"il\\010os.exit()\""), std::string::npos) << *text; + EXPECT_NE(text->find("description = \"ev\\\"il\\010os.exit() 1.0"), std::string::npos) << *text; EXPECT_EQ(text->find("\nos.exit()"), std::string::npos) << *text; EXPECT_NE(text->find("deps = { \"okrun\" }"), std::string::npos) << *text; } @@ -618,3 +618,41 @@ TEST(PackMerge, TheCompressionFitsTheContainer) { auto fits = d::merge(req); ASSERT_TRUE(fits) << fits.error(); } + +// The entry is named after the command a person runs, not the package, and +// moves the archive's own top-level directory into place. +TEST(PackDistribution, XpkgDescriptorNamesTheCommandAndTheArchiveTop) { + d::Asset uni; + uni.file = "mcppls-0.0.18-openkal.zip"; + uni.sha256 = std::string(64, 'a'); + uni.top = "mcppls-0.0.18-openkal"; + uni.pkg.name = "mcpp-language-server"; + uni.pkg.version = "0.0.18"; + uni.pkg.runner = { "okrun", "{}" }; + uni.pkg.entry = { { "x86_64", "lib/x86_64/mcppls.oke" }, { "aarch64", "lib/aarch64/mcppls.oke" } }; + auto u = d::xpkg_descriptor({ uni }, "https://example.org/v0.0.18"); + ASSERT_TRUE(u) << u.error(); + EXPECT_NE(u->find("name = \"mcppls\""), std::string::npos) << *u; + EXPECT_NE(u->find("xvm.add(\"mcppls\", opts)"), std::string::npos) << *u; + EXPECT_NE(u->find("opts.alias = \"mcppls.cmd\""), std::string::npos) << *u; + EXPECT_NE(u->find("os.dirs(\"mcppls-0.0.18-openkal\")"), std::string::npos) << *u; + + std::vector per; + for (auto [triple, top] : { std::pair{ "x86_64-linux-musl", "okrun-0.1.3-linux-x86_64" }, + std::pair{ "aarch64-macos", "okrun-0.1.3-macos-aarch64" } }) { + d::Asset a; + a.file = std::string(top) + ".zip"; + a.sha256 = std::string(64, 'b'); + a.top = top; + a.pkg.name = "openkal-tools"; + a.pkg.version = "0.1.3"; + a.pkg.legs = { triple }; + a.pkg.program = "bin/okrun"; + per.push_back(a); + } + auto r = d::xpkg_descriptor(per, "https://example.org/0.1.3"); + ASSERT_TRUE(r) << r.error(); + EXPECT_NE(r->find("name = \"okrun\""), std::string::npos) << *r; + EXPECT_NE(r->find("os.dirs(\"okrun-0.1.3-*\")"), std::string::npos) << *r; + EXPECT_NE(r->find("description = \"openkal-tools 0.1.3"), std::string::npos) << *r; +} diff --git a/tests/unit/test_pack_stage_tree.cpp b/tests/unit/test_pack_stage_tree.cpp index f750d6c58..f8973118f 100644 --- a/tests/unit/test_pack_stage_tree.cpp +++ b/tests/unit/test_pack_stage_tree.cpp @@ -115,19 +115,20 @@ TEST(PackStageTree, TheOrderOfADirectoryWalkIsNotAPromise) { read_file(mcpp::pack::stage_manifest_path(b.path / "s"))); } -TEST(PackStageTree, TheEngineOwnsExactlyTwoFormatNames) { - // `tar` and `dir` are the archive shapes `mcpp pack` owns; every other - // value of `--format` is a name a package provides. The list lives beside - // the staged tree because two layers need the same answer -- the parser, - // which decides whether a value is a built-in or a dispatch, and the - // declaration check, which refuses a package that claims one of them and - // would therefore be silently unreachable. - EXPECT_TRUE(mcpp::pack::is_builtin_pack_format("tar")); - EXPECT_TRUE(mcpp::pack::is_builtin_pack_format("dir")); +TEST(PackStageTree, TheEngineOwnsFourFormatNames) { + // `tar`, `zip`, `dir` and `single-file` are the shapes `mcpp pack` owns + // (SPEC-014 §1.3); every other value of `--pkg-format` is a name a + // package provides. The list lives beside the staged tree because two + // layers need the same answer -- the listing and the refusal of an + // unknown name, and the declaration check, which warns of a package that + // claims one of them and would therefore be silently unreachable. + for (auto name : { "tar", "zip", "dir", "single-file" }) + EXPECT_TRUE(mcpp::pack::is_builtin_pack_format(name)) << name; EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("appimage")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("okb")); EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("msi")); EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("")); - EXPECT_EQ(mcpp::pack::kBuiltinPackFormats.size(), 2u); + EXPECT_EQ(mcpp::pack::kBuiltinPackFormats.size(), 4u); } // ── #630 §3: the `closure` field ───────────────────────────────────────── From a0dcd1c1fefd79cf7a219942b1be3ccf5e39fc2d Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 17:33:30 +0900 Subject: [PATCH 52/54] pack: a library's implicit zip refuses a stated zstd or xz and leaves the project's aside, as a program's does; e2e 242 states the tar Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/library_pipeline.cppm | 14 ++++++++++++++ .../e2e/242_pack_library_interface_and_headers.sh | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 3a22d3469..0fcb96b9c 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -549,6 +549,20 @@ export int build_and_pack_library(const std::string& targetName, ? std::format("{}-{}-{}", plan.packageName, plan.packageVersion, plan.legs[0].abiTag) : std::format("{}-{}", plan.packageName, plan.packageVersion); plan.zip = zip; + // A zip deflates or stores, as a program's does (SPEC-014 §5.1): a zstd or + // xz the command stated is refused, the project's default left aside. + if (zip && (plan.compression.algorithm == mcpp::pack::compress::Algorithm::Zstd + || plan.compression.algorithm == mcpp::pack::compress::Algorithm::Xz)) { + if (opts.compressionFromFlag) { + mcpp::ui::error(std::format( + "'{}' is packed as a zip, whose members are deflated or stored; --compress {} is a " + "tar's.\n use: --pkg-format tar --compress {}, or --compress gzip | none", targetName, + mcpp::pack::compress::name(plan.compression.algorithm), + mcpp::pack::compress::name(plan.compression.algorithm))); + return 2; + } + plan.compression = {}; + } plan.stagingRoot = plan.projectRoot / "target" / "dist" / dirName; plan.archivePath = plan.projectRoot / "target" / "dist" / (dirName + (zip ? std::string(".zip") diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh index 5997ef5bf..a9b7652a0 100755 --- a/tests/e2e/242_pack_library_interface_and_headers.sh +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -67,7 +67,7 @@ cd mathkit # The container as stated, compressed by mcpp itself; a single file is a # program's, not a library's. -"$MCPP" pack mathkit --compress zstd > pack-zst.log 2>&1 || { cat pack-zst.log; echo "pack --compress zstd failed"; exit 1; } +"$MCPP" pack mathkit --pkg-format tar --compress zstd > pack-zst.log 2>&1 || { cat pack-zst.log; echo "pack --compress zstd failed"; exit 1; } zst="$(find target/dist -maxdepth 1 -type f -name 'mathkit-0.1.0-*.tar.zst' | head -1)" [[ -n "$zst" ]] || { cat pack-zst.log; ls target/dist; echo "no .tar.zst for --compress zstd"; exit 1; } [[ "$(head -c 4 "$zst" | od -An -tx1 | tr -d ' \n')" == "28b52ffd" ]] || { echo "$zst is not a zstd frame"; exit 1; } From 63d2f41c2bbda4f1ce52d48d6e44968101293b01 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 17:34:37 +0900 Subject: [PATCH 53/54] =?UTF-8?q?design:=20=C2=A715.1=20what=20the=20imple?= =?UTF-8?q?mentation=20corrected=20(runners=20over=20the=20hosted=20arrang?= =?UTF-8?q?ement,=20#850,=20the=20macOS=20arguments,=20mcppls's=20per-syst?= =?UTF-8?q?em=20xlings=20package)=20and=20mcpp-ide's=20measured=20sizes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...part2-packaging-and-distribution-design.md | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md b/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md index 2a5e8f376..0e55428b5 100644 --- a/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md +++ b/.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md @@ -255,3 +255,30 @@ e2e：三维组合与拒绝；`--pkg-format` 与旧写法；通用包两架构 判据：两个 CI 各有一条"扩展包内只有一个 okb、每个系统 × 架构的 okrun 都能启动它"的检查；vsix 与 CLion 插件 zip 的体积在本方案 记录前后对比。交付给维护者上传的产物目录（part 1 §13.3）中的 vsix 与 zip 即为采用 okb 后的版本。 + +### 15.1 实施中的修正与实测 + +- **运行器在任何机器上以平台的托管安排构建。** 每个系统的运行器与载体在其行（`-linux-musl`、`-macos`、 + `-windows-gnu`）上以 openkal-platform 的托管安排构建，Linux 上一次打包即得四个 okrun（Linux 两个架构、macOS + arm64、Windows x64），Windows 的 okrun 不再带 VCRUNTIME140.dll。mcpp-ide 因此去掉了各系统分别打包与 `--merge` 的作业； + mcppls 保留 Linux 与 macOS 两份的合并，以继续交叉验证 `--merge`。 +- **Windows 主机上的映像字节不同（mcpp#850，上游）。** clang 在 Windows 主机上拼接包含路径时用 `\`，`-ffile-reproducible` + 对 POSIX 目标不改；libc++abi 的 `__FILE__` 因此不同，合并正确地拒绝。Linux 与 macOS 构建的映像逐字节相同，通用包在二者 + 之一上打包。 +- **macOS 托管安排的参数（openkal-musl 0.22.6）。** Mach-O 的构造函数顺序取决于链接载入对象的顺序；musl 的构造函数早于 + openkal-macos 记录参数时，`main` 得到 argc 0。mcppls 的 CI 测得；0.17.3 轮修复，平台的 hosted 检查断言参数。 +- **mcppls 的 xlings 包仍按系统。** 其载荷带每个系统的 clangd 与语义套件，载荷中的服务器为同一个 okb 与该系统的 okrun； + "一个资产对应三系统"适用于 mcpp-ide 的后端包与 okrun 之外的纯映像程序。 + +**mcpp-ide 实测（CI run 40640226458，openkal-platform 0.17.3）。** + +| 产物 | 采用 okb 之前 | 0.17.2 平台 | 0.17.3 平台（运行器以托管安排构建） | +|---|---|---|---| +| vsix | 4 686 866 B | 2 965 783 B | 3 217 042 B | +| CLion 插件 zip | 4 605 498 B | 2 924 975 B | 3 172 459 B | +| `mcpp-ide.okb`（两个架构） | — | 5 731 095 B | 5 731 231 B | +| okrun linux-x86_64 / linux-aarch64 | — | 682 944 / 655 096 B | 683 024 / 655 128 B | +| okrun macos-aarch64 | — | 1 398 448 B | 1 237 392 B | +| okrun windows-x86_64 | — | 462 336 B + VCRUNTIME140.dll | 828 416 B | + +扩展包由"每平台一个包"变为"一个包服务所有平台"；0.17.3 的增长来自 Windows 的 okrun 改为静态的托管安排程序。 From 8992d59e22b485045141e8c17875600c1889bf48 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 28 Oct 2026 18:27:05 +0900 Subject: [PATCH 54/54] e2e: Linux shard timings measured on #847's run, the tests since 900 among them (955's cross-built runners weighed at their cold cost), so no shard runs past its limit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- tests/e2e/timings/linux.tsv | 1096 +++++++++++++++++++---------------- 1 file changed, 585 insertions(+), 511 deletions(-) diff --git a/tests/e2e/timings/linux.tsv b/tests/e2e/timings/linux.tsv index 770bdb79a..fbbc90fea 100644 --- a/tests/e2e/timings/linux.tsv +++ b/tests/e2e/timings/linux.tsv @@ -1,515 +1,589 @@ -00_fixture_path_hygiene.sh 2543 -01_help_and_version.sh 12 -02_new_build_run.sh 5529 -03_multi_module.sh 5933 -04_incremental.sh 3342 -05_errors.sh 1223 -06_emit_xpkg.sh 15 -07_static_library.sh 133 -08_shared_library.sh 189 -09_path_dependency.sh 2437 -100_cppfly_reflection.sh 12254 -100_feature_sources_test_mode.sh 3887 -101_cppfly_llvm_soft.sh 3836 -103_target_vocabulary.sh 11073 -104_asm_sources_gas.sh 2741 -105_asm_sources_nasm.sh 8738 -106_feature_gated_sources_toml.sh 6781 -107_generated_files_toml.sh 1598 -108_cfg_conditional_sources.sh 1198 -109_per_glob_flags.sh 1218 -10_env_command.sh 5999 -110_build_mcpp_env_contract.sh 8839 -111_dep_build_mcpp.sh 2142 -112_build_mcpp_cross.sh 7194 -113_scanner_mcpp_dir_prune.sh 179 -114_run_scan_scope.sh 1563 -115_manifest_syntax_aot_brace.sh 1308 -116_include_family_and_define_spaces.sh 1322 -117_object_path_collision.sh 1311 -118_purview_include_rebuild.sh 8825 -119_dep_cfg_sources.sh 1351 -11_index_list.sh 7507 -120_ws_root_indices.sh 722 -121_default_ns_redirect.sh 491 -122_run_member.sh 2372 -123_same_named_main_across_dep.sh 1290 -124_gen_source_out_of_root_object_path.sh 1698 -125_build_mcpp_dep_dir.sh 1814 -126_default_features_opt_out.sh 391 -127_transitive_default_features_optout.sh 2577 -128_feature_forwarding.sh 279 -12_add_command.sh 450 -134_target_sysroot_and_queries.sh 4462 -135_diagnostic_promise_resolves.sh 7996 -136_aarch64_zero_libc_target.sh 577 -137_x86_64_zero_libc_target.sh 539 -138_feature_sources_gate_vs_provider.sh 601 -139_build_program_advisory.sh 637 -13_toolchain_pin.sh 8 -141_include_dirs_after.sh 1319 -142_generated_files_incremental.sh 2431 -143_build_mcpp_source_directive.sh 2716 -144_build_mcpp_include_dir.sh 3689 -145_root_build_mcpp_dep_dirs.sh 1621 -146_feature_flags.sh 8190 -147_per_os_features_xpkg.sh 504 -148_wide_include_list.sh 1408 -149_conditional_glob_flags.sh 1264 -14_toolchain_fallback.sh 7356 -150_clang_module_operator_template.sh 207 -151_index_url_artifact_migration.sh 5984 -152_test_subdir_names.sh 141 -153_test_isolation.sh 222 -154_test_filter.sh 150 -155_test_json.sh 188 -156_nested_ld_library_path.sh 124 -157_test_glob_flags.sh 140 -158_test_signal_and_duration.sh 308 -159_test_list.sh 24 -15_test_passing.sh 2176 -160_test_timeout.sh 4651 +00_fixture_path_hygiene.sh 2480 +01_help_and_version.sh 11 +02_new_build_run.sh 1310 +03_multi_module.sh 1960 +04_incremental.sh 3680 +05_errors.sh 1430 +06_emit_xpkg.sh 17 +07_static_library.sh 145 +08_shared_library.sh 288 +09_path_dependency.sh 2410 +100_cppfly_reflection.sh 8130 +100_feature_sources_test_mode.sh 3330 +101_cppfly_llvm_soft.sh 3210 +103_target_vocabulary.sh 5780 +104_asm_sources_gas.sh 2780 +105_asm_sources_nasm.sh 6700 +106_feature_gated_sources_toml.sh 7850 +107_generated_files_toml.sh 1610 +108_cfg_conditional_sources.sh 1110 +109_per_glob_flags.sh 1300 +10_env_command.sh 94 +110_build_mcpp_env_contract.sh 6020 +111_dep_build_mcpp.sh 2350 +112_build_mcpp_cross.sh 6820 +113_scanner_mcpp_dir_prune.sh 151 +114_run_scan_scope.sh 1550 +115_manifest_syntax_aot_brace.sh 1120 +116_include_family_and_define_spaces.sh 1480 +117_object_path_collision.sh 1290 +118_purview_include_rebuild.sh 7220 +119_dep_cfg_sources.sh 1230 +11_index_list.sh 9700 +120_ws_root_indices.sh 680 +121_default_ns_redirect.sh 619 +122_run_member.sh 1700 +123_same_named_main_across_dep.sh 939 +124_gen_source_out_of_root_object_path.sh 2420 +125_build_mcpp_dep_dir.sh 1340 +126_default_features_opt_out.sh 496 +127_transitive_default_features_optout.sh 2310 +128_feature_forwarding.sh 507 +12_add_command.sh 558 +134_target_sysroot_and_queries.sh 5130 +135_diagnostic_promise_resolves.sh 5640 +136_aarch64_zero_libc_target.sh 1150 +137_x86_64_zero_libc_target.sh 1390 +138_feature_sources_gate_vs_provider.sh 640 +139_build_program_advisory.sh 925 +13_toolchain_pin.sh 5 +141_include_dirs_after.sh 1030 +142_generated_files_incremental.sh 2470 +143_build_mcpp_source_directive.sh 2420 +144_build_mcpp_include_dir.sh 3840 +145_root_build_mcpp_dep_dirs.sh 1740 +146_feature_flags.sh 9390 +147_per_os_features_xpkg.sh 540 +148_wide_include_list.sh 1230 +149_conditional_glob_flags.sh 1370 +14_toolchain_fallback.sh 8130 +150_clang_module_operator_template.sh 390 +151_index_url_artifact_migration.sh 107 +152_test_subdir_names.sh 147 +153_test_isolation.sh 377 +154_test_filter.sh 173 +155_test_json.sh 202 +156_nested_ld_library_path.sh 112 +157_test_glob_flags.sh 136 +158_test_signal_and_duration.sh 277 +159_test_list.sh 17 +15_test_passing.sh 2160 +160_test_timeout.sh 4650 161_xpkg_name_form.sh 63 -162_bare_name_namespace_scope.sh 5879 -163_identity_first_resolution.sh 25198 -164_ninja_goal_command_length.sh 529 -165_bare_name_cross_namespace_wire_address.sh 6126 -166_run_env_no_private_glibc.sh 353 -167_build_defines_module_scan.sh 813 -168_build_mcpp_musl_host_static.sh 5623 -169_semver_project_index.sh 511 -16_test_failing.sh 2361 -170_bmi_staging_no_cascade.sh 1320 -171_bmi_staging_locked_dest.sh 2585 -172_build_cache_cross_project.sh 9478 -173_build_profile_isolation.sh 1114 -173_index_refresh_policy.sh 791 -174_cache_modes_and_commands.sh 17259 -175_cpp20_import_std.sh 7801 -176_cpp20_std_bmi_isolation.sh 8248 -178_test_observability.sh 10866 -179_spaced_paths.sh 1435 -17_test_no_tests.sh 16 -181_build_mcpp_import_std.sh 5332 -183_cross_no_relink.sh 1167 -183_cxx_runtime_contract.sh 4291 -184_build_cache_object_layout.sh 12353 -185_index_floor_degrades.sh 5647 -186_build_mcpp_protocol_and_bound.sh 7594 -187_dep_host_tool.sh 3472 -188_build_actions.sh 8591 -189_host_module_rules.sh 1893 -18_devdeps_isolation.sh 1181 -190_link_rspfile_newlines.sh 1278 -191_link_scale.sh 20371 -192_install_path_namespace.sh 3686 -193_provision_reexport.sh 8603 -194_build_mcpp_glob_input.sh 2471 -195_target_cfg_feature_deps.sh 410 -196_version_identity_and_lock.sh 15118 -198_windows_resources_cross.sh 3729 -199_resources_validation.sh 222 -19_bmi_cache_reuse.sh 6699 -200_subos_env_reaches_program.sh 1778 -201_gcc_no_specs_pollution.sh 8564 -202_machine_output_contract.sh 2576 -203_exact_selector_lock_migration.sh 358 -204_new_transactional_scaffold.sh 9698 -205_root_local_subos.sh 732 -206_runtime_binding_physics.sh 1177 -207_runtime_contract_provenance.sh 200 -208_private_libc_stays_in_the_binary.sh 314 -209_runtime_verdict_never_reruns_green.sh 723 -20_p1689_scanner.sh 4818 -210_local_index_addressing_on_every_host.sh 248 -211_configure_only_cdb.sh 486 -212_cached_dep_std_is_ordered.sh 5431 -213_build_after_test_is_not_the_test_graph.sh 396 -214_executable_carries_dt_rpath.sh 1506 -215_pack_has_no_build_machine_paths.sh 3483 -216_selfcontained_refuses_host_capability.sh 545 -217_module_extensions.sh 9988 -218_module_extensions_graph_shape.sh 5117 -219_runtime_search_farm_is_last.sh 345 -21_ninja_dyndep.sh 4670 -220_farm_only_library_runs.sh 880 -221_subos_without_info_still_builds.sh 157 -222_shared_library_cxx_runtime_contract.sh 2362 -22_doctor_cache_publish.sh 47292 -230_bench_harness.sh 55578 -231_jobs_option.sh 2197 -232_workflow_syntax.sh 325 -233_bench_matrix.sh 224 -234_bmi_schedule_on.sh 6695 -235_std_object_only_when_needed.sh 1071 -236_module_extensions_default_toolchain.sh 3494 -237_default_subos_without_info.sh 287 -238_c_only_unit_links_with_c_driver.sh 3775 -23_remove_update.sh 12842 -240_pack_pe_zip_cross.sh 414 -242_pack_library_interface_and_headers.sh 779 -243_pack_library_interface_closure.sh 681 -244_pack_library_gate.sh 536 -245_pack_library_fat_target_selection.sh 763 -246_explicit_empty_sources.sh 261 -247_bare_triple_conditional_native.sh 389 -248_pack_library_fat_pe_leg.sh 513 -249_pack_workspace_root_unchanged.sh 217 -24_git_dependency.sh 10859 -250_pack_names_the_target.sh 261 -251_pack_library_shared.sh 375 -252_pack_library_old_client.sh 308 -253_pack_library_undetermined_partition.sh 532 -254_pack_library_platform_coverage.sh 503 -25_convention_mode.sh 4566 -260_module_extension_is_configuration.sh 198 -261_pack_follows_the_module_extension.sh 465 -263_lib_root_follows_the_extension.sh 1252 -264_pack_library_is_relocatable.sh 424 -265_pack_strips_but_stays_usable.sh 916 -267_target_requested_is_empty_for_a_native_build.sh 744 -268_target_side_resolution.sh 242 -269_openkal_llvm_spelling_still_resolves.sh 485 -26_c_language_support.sh 1293 -26_toolchain_management.sh 40215 -27_namespace_dependencies.sh 8849 -27_self_contained_home.sh 16869 -280_target_side_layers.sh 2345 -281_target_side_rules.sh 223 -282_target_is_a_request.sh 67 -283_run_target_flag_owns_its_cache_slot.sh 323 -284_env_segment_is_optional_everywhere.sh 3132 -285_kernel_abi_from_graph_keeps_the_payload_c_library.sh 4622 -286_the_openkal_stack_still_builds.sh 40171 -287_the_openkal_stack_crosses_to_aarch64.sh 38940 -289_one_host_reaches_every_openkal_target.sh 77137 -28_target_static.sh 4 -290_the_declaration_puts_an_environment_in_front.sh 1998 -291_dynamic_linkage_is_refused_only_when_the_c_library_is_the_graphs.sh 30147 -292_a_package_that_names_a_layer_does_not_lose_the_targets_compiler.sh 355 -293_the_requested_target_and_the_resolved_one_name_one_os.sh 9515 -294_the_list_answers_what_can_be_built_not_what_has_a_payload.sh 23 -295_naming_the_host_target_changes_nothing.sh 1370 -296_what_the_report_names_is_what_the_link_line_uses.sh 26315 -297_a_capability_pin_is_not_a_preference.sh 251 -298_overriding_a_convention_requires_replacing_it.sh 28550 -299_a_request_that_named_no_c_library_resolves_to_a_row_that_exists.sh 9159 -29_toolchain_partial_versions.sh 5553 -300_a_registered_family_is_not_reported_unknown.sh 43 -301_the_graphs_compiler_is_taken_and_nothing_is_written.sh 298 -302_a_stated_compiler_outranks_the_graph_and_two_requirements_do_not_stack.sh 85 -303_the_query_gives_one_answer_for_the_c_library.sh 107 -304_the_target_side_reaches_every_unit.sh 188 -305_built_from_is_not_published.sh 228 -306_dependency_linkage_axis.sh 376 -307_symbol_provision.sh 551 -308_dependency_required_features.sh 213 -309_host_module_identity.sh 987 -30_dev_binary_home.sh 20 -30_pack_modes.sh 5699 -310_build_dependencies.sh 749 -311_rule_build_dependencies.sh 3019 -312_build_rules_example.sh 3353 -313_check_stamp_on_every_platform.sh 639 -314_dependency_action_generated_header.sh 2676 -315_blocking_check_gates_compilation.sh 4923 -316_link_unit_with_no_inputs_is_refused.sh 144 -317_project_build_hooks.sh 18666 -31_transitive_deps.sh 142 -321_workspace_inheritance.sh 687 -322_dialect_flag_refusal.sh 10582 -323_post_link_record_durability.sh 422 -324_fast_path_sees_path_deps.sh 3532 -325_system_toolchain_refused.sh 222 -326_dependency_standard_floor.sh 740 -327_declared_xlings_deps_report_their_own_failure.sh 7813 -328_layer_predicates_apply_and_unknown_ones_say_so.sh 808 -329_exit_codes_match_the_contract.sh 20 -32_semver_merge.sh 9628 -330_runner_hosted_targets.sh 1428 -331_xlings_workspace_one_table.sh 1855 -333_named_runners_locked_and_sbom.sh 7286 -334_dependency_declared_tool_is_found.sh 251 -335_xlings_tool_tiers.sh 37 -337_run_takes_features_and_profile.sh 961 -339_run_reports_the_program_status.sh 956 -33_multi_version_mangling.sh 15107 -340_no_orphan_survives_a_killed_mcpp.sh 11268 -35_workspace.sh 1807 -36_llvm_toolchain.sh 1246 -37_llvm_import_std.sh 3445 -38_llvm_modules.sh 4184 -38_self_config_mirror.sh 258 -39_llvm_incremental.sh 5127 -39_xlings_index_migration.sh 220 -40_llvm_bmi_cache.sh 5557 -41_llvm_std_compat.sh 2913 -42_custom_local_index.sh 8323 -43_indices_lockfile.sh 8452 -44_indices_e2e_integration.sh 7633 -45_install_platform_mapping.sh 12 -46_self_config_mirror_no_bootstrap.sh 6953 -47_cdb_prebuilt_module_path_abs.sh 1236 -47_llvm_atomic_link.sh 669 -48_build_error_output.sh 296 -49_bmi_cache_nested_custom_index.sh 6825 -50_package_owned_build_flags.sh 182 -51_package_generated_files.sh 351 -52_local_path_namespaced_index.sh 486 -53_namespaced_cache_label.sh 454 -54_package_owned_ldflags.sh 262 -55_dependency_shared_artifact.sh 206 -56_transitive_shared_artifact.sh 237 -57_static_dep_shared_artifact.sh 208 -58_preinstall_mcpp_deps_for_hooks.sh 295 -59_cpp_standard_config.sh 6668 -600_accel_variant_selection.sh 415 -601_exclusive_capability.sh 172 -603_version_floor.sh 303 -604_offline_skips_sandbox_bootstrap.sh 434 -605_probe_channel_and_accel_reach_the_build_program.sh 1349 -606_constrained_source_globs_narrow_to_the_build.sh 1145 -607_chained_actions_form_the_device_link.sh 1132 -608_object_actions_reach_a_static_library.sh 532 -609_clean_stale.sh 4532 -609_shader_sources_are_device_sources.sh 1679 -60_stale_xpkg_cache_reinstall.sh 330 -610_feature_controlled_rule_collection.sh 4584 -611_variant_switch_is_not_replayed.sh 601 -612_build_program_file_input_fast_path.sh 1232 -613_sycl_sources_are_device_sources.sh 1808 -614_feature_xlings_reaches_xpkg_dir.sh 1043 -615_feature_deps_shared_library_is_linked.sh 481 -616_examples_and_their_index_agree.sh 292 -617_build_program_knows_the_cxx_stdlib.sh 566 -618_dependency_feature_tools_reach_its_build_program.sh 45430 -619_a_tool_may_not_be_conditioned_on_a_layer.sh 29 -61_dependency_visibility.sh 476 -620_link_flag_reaches_the_link_line.sh 579 -621_exports_narrows_the_symbol_surface.sh 570 -622_rule_declared_payload_reaches_the_consumers_build_program.sh 50647 -623_device_sources_must_reach_an_action.sh 292 -624_a_missing_host_module_names_the_key.sh 304 +162_bare_name_namespace_scope.sh 769 +163_identity_first_resolution.sh 18500 +164_ninja_goal_command_length.sh 659 +165_bare_name_cross_namespace_wire_address.sh 5590 +166_run_env_no_private_glibc.sh 365 +167_build_defines_module_scan.sh 551 +168_build_mcpp_musl_host_static.sh 4970 +169_semver_project_index.sh 516 +16_test_failing.sh 2470 +170_bmi_staging_no_cascade.sh 1100 +171_bmi_staging_locked_dest.sh 2420 +172_build_cache_cross_project.sh 6820 +173_build_profile_isolation.sh 981 +173_index_refresh_policy.sh 920 +174_cache_modes_and_commands.sh 13830 +175_cpp20_import_std.sh 6700 +176_cpp20_std_bmi_isolation.sh 7490 +178_test_observability.sh 9480 +179_spaced_paths.sh 1390 +17_test_no_tests.sh 14 +181_build_mcpp_import_std.sh 5940 +183_cross_no_relink.sh 1230 +183_cxx_runtime_contract.sh 3570 +184_build_cache_object_layout.sh 11480 +185_index_floor_degrades.sh 5430 +186_build_mcpp_protocol_and_bound.sh 8029 +187_dep_host_tool.sh 4070 +188_build_actions.sh 8710 +189_host_module_rules.sh 2820 +18_devdeps_isolation.sh 1120 +190_link_rspfile_newlines.sh 686 +191_link_scale.sh 16149 +192_install_path_namespace.sh 3370 +193_provision_reexport.sh 9050 +194_build_mcpp_glob_input.sh 1780 +195_target_cfg_feature_deps.sh 437 +196_version_identity_and_lock.sh 13010 +198_windows_resources_cross.sh 3810 +199_resources_validation.sh 509 +19_bmi_cache_reuse.sh 6260 +200_subos_env_reaches_program.sh 1700 +201_gcc_no_specs_pollution.sh 10430 +202_machine_output_contract.sh 7620 +203_exact_selector_lock_migration.sh 482 +204_new_transactional_scaffold.sh 6840 +205_root_local_subos.sh 709 +206_runtime_binding_physics.sh 2850 +207_runtime_contract_provenance.sh 380 +208_private_libc_stays_in_the_binary.sh 246 +209_runtime_verdict_never_reruns_green.sh 4560 +20_p1689_scanner.sh 4420 +210_local_index_addressing_on_every_host.sh 262 +211_configure_only_cdb.sh 612 +212_cached_dep_std_is_ordered.sh 5090 +213_build_after_test_is_not_the_test_graph.sh 451 +214_executable_carries_dt_rpath.sh 1460 +215_pack_has_no_build_machine_paths.sh 2660 +216_selfcontained_refuses_host_capability.sh 521 +217_module_extensions.sh 5440 +218_module_extensions_graph_shape.sh 5000 +219_runtime_search_farm_is_last.sh 320 +21_ninja_dyndep.sh 5310 +220_farm_only_library_runs.sh 908 +221_subos_without_info_still_builds.sh 189 +222_shared_library_cxx_runtime_contract.sh 1810 +22_doctor_cache_publish.sh 4850 +230_bench_harness.sh 53150 +231_jobs_option.sh 2630 +232_workflow_syntax.sh 202 +233_bench_matrix.sh 161 +234_bmi_schedule_on.sh 4950 +235_std_object_only_when_needed.sh 968 +236_module_extensions_default_toolchain.sh 2520 +237_default_subos_without_info.sh 218 +238_c_only_unit_links_with_c_driver.sh 3660 +23_remove_update.sh 16010 +240_pack_pe_zip_cross.sh 379 +242_pack_library_interface_and_headers.sh 923 +243_pack_library_interface_closure.sh 808 +244_pack_library_gate.sh 555 +245_pack_library_fat_target_selection.sh 794 +246_explicit_empty_sources.sh 307 +247_bare_triple_conditional_native.sh 455 +248_pack_library_fat_pe_leg.sh 609 +249_pack_workspace_root_unchanged.sh 186 +24_git_dependency.sh 9150 +250_pack_names_the_target.sh 239 +251_pack_library_shared.sh 597 +252_pack_library_old_client.sh 346 +253_pack_library_undetermined_partition.sh 662 +254_pack_library_platform_coverage.sh 501 +25_convention_mode.sh 4770 +260_module_extension_is_configuration.sh 223 +261_pack_follows_the_module_extension.sh 445 +263_lib_root_follows_the_extension.sh 1470 +264_pack_library_is_relocatable.sh 494 +265_pack_strips_but_stays_usable.sh 832 +267_target_requested_is_empty_for_a_native_build.sh 554 +268_target_side_resolution.sh 312 +269_openkal_llvm_spelling_still_resolves.sh 1080 +26_c_language_support.sh 1290 +26_toolchain_management.sh 28430 +27_namespace_dependencies.sh 8430 +27_self_contained_home.sh 8290 +280_target_side_layers.sh 2750 +281_target_side_rules.sh 294 +282_target_is_a_request.sh 138 +283_run_target_flag_owns_its_cache_slot.sh 361 +284_env_segment_is_optional_everywhere.sh 2690 +285_kernel_abi_from_graph_keeps_the_payload_c_library.sh 4110 +286_the_openkal_stack_still_builds.sh 41110 +287_the_openkal_stack_crosses_to_aarch64.sh 26910 +289_one_host_reaches_every_openkal_target.sh 65430 +28_target_static.sh 3 +290_the_declaration_puts_an_environment_in_front.sh 2000 +291_dynamic_linkage_is_refused_only_when_the_c_library_is_the_graphs.sh 30790 +292_a_package_that_names_a_layer_does_not_lose_the_targets_compiler.sh 1080 +293_the_requested_target_and_the_resolved_one_name_one_os.sh 4430 +294_the_list_answers_what_can_be_built_not_what_has_a_payload.sh 53 +295_naming_the_host_target_changes_nothing.sh 1850 +296_what_the_report_names_is_what_the_link_line_uses.sh 21920 +297_a_capability_pin_is_not_a_preference.sh 414 +298_overriding_a_convention_requires_replacing_it.sh 20860 +299_a_request_that_named_no_c_library_resolves_to_a_row_that_exists.sh 295 +29_toolchain_partial_versions.sh 9020 +300_a_registered_family_is_not_reported_unknown.sh 56 +301_the_graphs_compiler_is_taken_and_nothing_is_written.sh 544 +302_a_stated_compiler_outranks_the_graph_and_two_requirements_do_not_stack.sh 131 +303_the_query_gives_one_answer_for_the_c_library.sh 351 +304_the_target_side_reaches_every_unit.sh 167 +305_built_from_is_not_published.sh 224 +306_dependency_linkage_axis.sh 498 +307_symbol_provision.sh 595 +308_dependency_required_features.sh 258 +309_host_module_identity.sh 1610 +30_dev_binary_home.sh 16 +30_pack_modes.sh 5360 +310_build_dependencies.sh 1050 +311_rule_build_dependencies.sh 2400 +312_build_rules_example.sh 2810 +313_check_stamp_on_every_platform.sh 807 +314_dependency_action_generated_header.sh 2710 +315_blocking_check_gates_compilation.sh 4860 +316_link_unit_with_no_inputs_is_refused.sh 225 +317_project_build_hooks.sh 26560 +31_transitive_deps.sh 118 +321_workspace_inheritance.sh 812 +322_dialect_flag_refusal.sh 10330 +323_post_link_record_durability.sh 684 +324_fast_path_sees_path_deps.sh 3510 +325_system_toolchain_refused.sh 193 +326_dependency_standard_floor.sh 782 +327_declared_xlings_deps_report_their_own_failure.sh 4620 +328_layer_predicates_apply_and_unknown_ones_say_so.sh 1190 +329_exit_codes_match_the_contract.sh 25 +32_semver_merge.sh 5420 +330_runner_hosted_targets.sh 1680 +331_xlings_workspace_one_table.sh 1380 +333_named_runners_locked_and_sbom.sh 4290 +334_dependency_declared_tool_is_found.sh 764 +335_xlings_tool_tiers.sh 59 +337_run_takes_features_and_profile.sh 1790 +339_run_reports_the_program_status.sh 1380 +33_multi_version_mangling.sh 10440 +340_no_orphan_survives_a_killed_mcpp.sh 11170 +35_workspace.sh 1930 +36_llvm_toolchain.sh 873 +37_llvm_import_std.sh 2650 +38_llvm_modules.sh 4010 +38_self_config_mirror.sh 20 +39_llvm_incremental.sh 4740 +39_xlings_index_migration.sh 169 +40_llvm_bmi_cache.sh 3790 +41_llvm_std_compat.sh 1980 +42_custom_local_index.sh 9900 +43_indices_lockfile.sh 9400 +44_indices_e2e_integration.sh 7680 +45_install_platform_mapping.sh 11 +46_self_config_mirror_no_bootstrap.sh 13 +47_cdb_prebuilt_module_path_abs.sh 1400 +47_llvm_atomic_link.sh 1000 +48_build_error_output.sh 971 +49_bmi_cache_nested_custom_index.sh 5970 +50_package_owned_build_flags.sh 575 +51_package_generated_files.sh 375 +52_local_path_namespaced_index.sh 468 +53_namespaced_cache_label.sh 713 +54_package_owned_ldflags.sh 907 +55_dependency_shared_artifact.sh 440 +56_transitive_shared_artifact.sh 416 +57_static_dep_shared_artifact.sh 153 +58_preinstall_mcpp_deps_for_hooks.sh 523 +59_cpp_standard_config.sh 6250 +600_accel_variant_selection.sh 392 +601_exclusive_capability.sh 175 +603_version_floor.sh 293 +604_offline_skips_sandbox_bootstrap.sh 131 +605_probe_channel_and_accel_reach_the_build_program.sh 1560 +606_constrained_source_globs_narrow_to_the_build.sh 805 +607_chained_actions_form_the_device_link.sh 917 +608_object_actions_reach_a_static_library.sh 547 +609_clean_stale.sh 4230 +609_shader_sources_are_device_sources.sh 1150 +60_stale_xpkg_cache_reinstall.sh 722 +610_feature_controlled_rule_collection.sh 3770 +611_variant_switch_is_not_replayed.sh 511 +612_build_program_file_input_fast_path.sh 704 +613_sycl_sources_are_device_sources.sh 1180 +614_feature_xlings_reaches_xpkg_dir.sh 705 +615_feature_deps_shared_library_is_linked.sh 1140 +616_examples_and_their_index_agree.sh 246 +617_build_program_knows_the_cxx_stdlib.sh 942 +618_dependency_feature_tools_reach_its_build_program.sh 26010 +619_a_tool_may_not_be_conditioned_on_a_layer.sh 44 +61_dependency_visibility.sh 600 +620_link_flag_reaches_the_link_line.sh 600 +621_exports_narrows_the_symbol_surface.sh 12360 +622_rule_declared_payload_reaches_the_consumers_build_program.sh 54630 +623_device_sources_must_reach_an_action.sh 651 +624_a_missing_host_module_names_the_key.sh 585 625_a_payload_can_be_gated_on_the_accelerator.sh 44636 -626_naming_a_subset_of_backends_is_not_a_mismatch.sh 434 -627_one_package_one_version.sh 46552 -628_a_pin_below_a_stated_floor_is_refused.sh 47993 -629_a_version_range_is_answered_by_xpkg_dir.sh 47222 -62_dotted_dependency_selector_priority.sh 420 -62_runtime_library_dirs.sh 414 -630_a_rule_brings_its_own_environment.sh 44172 -631_action_depfile_tracks_a_runtime_discovered_include.sh 944 -632_a_build_program_is_told_its_package_name.sh 550 -633_host_modules_are_ordered_by_what_they_import.sh 1288 -634_a_private_module_fragment_is_not_a_partition.sh 130 -635_a_byte_order_mark_does_not_hide_a_module.sh 305 -636_a_module_extension_file_need_not_provide_a_module.sh 165 -637_a_conditional_runtime_table_alone_is_applied.sh 519 -638_pack_format_dispatch.sh 5742 -639_the_scanner_does_not_read_inside_a_comment.sh 790 -63_bare_dependency_peer_root_priority.sh 378 -640_a_capability_pin_explains_its_own_row.sh 9920 -641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh 81543 -643_windows_subsystem_cross.sh 1455 -644_windows_keys_are_inert_elsewhere.sh 598 -645_the_fast_path_compares_the_toolchain_request.sh 684 -646_runtime_deploy_places_files_in_a_directory.sh 1881 -647_abi_threads_is_one_switch_for_the_graph.sh 642 -648_an_install_hook_sees_the_build_target.sh 868 -649_pack_carries_runtime_deploy_files.sh 3179 -64_shared_soname_runtime_alias.sh 404 -650_wasm_row_names_its_launcher.sh 21 -651_a_build_program_deploys_what_it_generated.sh 1872 -652_an_application_is_a_kind.sh 514 -653_exceptions_is_the_second_abi_member.sh 6 -654_a_requirement_on_the_target_axis.sh 715 -656_run_hands_the_distributable_to_the_runner.sh 2616 +626_naming_a_subset_of_backends_is_not_a_mismatch.sh 838 +627_one_package_one_version.sh 53560 +628_a_pin_below_a_stated_floor_is_refused.sh 26940 +629_a_version_range_is_answered_by_xpkg_dir.sh 27460 +62_dotted_dependency_selector_priority.sh 669 +62_runtime_library_dirs.sh 2160 +630_a_rule_brings_its_own_environment.sh 50180 +631_action_depfile_tracks_a_runtime_discovered_include.sh 2020 +632_a_build_program_is_told_its_package_name.sh 638 +633_host_modules_are_ordered_by_what_they_import.sh 1250 +634_a_private_module_fragment_is_not_a_partition.sh 167 +635_a_byte_order_mark_does_not_hide_a_module.sh 415 +636_a_module_extension_file_need_not_provide_a_module.sh 192 +637_a_conditional_runtime_table_alone_is_applied.sh 617 +638_pack_format_dispatch.sh 3900 +639_the_scanner_does_not_read_inside_a_comment.sh 1640 +63_bare_dependency_peer_root_priority.sh 822 +640_a_capability_pin_explains_its_own_row.sh 5820 +641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh 35480 +643_windows_subsystem_cross.sh 1270 +644_windows_keys_are_inert_elsewhere.sh 995 +645_the_fast_path_compares_the_toolchain_request.sh 498 +646_runtime_deploy_places_files_in_a_directory.sh 1680 +647_abi_threads_is_one_switch_for_the_graph.sh 620 +648_an_install_hook_sees_the_build_target.sh 547 +649_pack_carries_runtime_deploy_files.sh 2820 +64_shared_soname_runtime_alias.sh 792 +650_wasm_row_names_its_launcher.sh 25 +651_a_build_program_deploys_what_it_generated.sh 1370 +652_an_application_is_a_kind.sh 579 +653_exceptions_is_the_second_abi_member.sh 5 +654_a_requirement_on_the_target_axis.sh 875 +656_run_hands_the_distributable_to_the_runner.sh 1020 657_a_build_program_under_the_web_row_uses_the_host_compiler.sh 3 -659_an_action_declaration_has_no_size_limit.sh 1262 -65_toolchain_runtime_dirs_for_run.sh 661 -660_an_action_names_the_engine_through_mcpp_self.sh 1358 -661_one_identity_two_declarations.sh 754 -662_pack_stages_declared_files_before_the_closure.sh 7113 -663_a_graph_libcxx_over_the_payloads_c_library.sh 17317 -665_a_host_tool_follows_its_source.sh 9337 -669_an_elf_host_pack_states_its_closure.sh 1078 -66_runtime_provides.sh 1271 -670_an_rpath_loader_token_is_left_as_written.sh 189 -671_toolchain_is_an_option_of_run_test_and_pack.sh 4156 -672_a_runner_receives_the_files_the_artifact_carries.sh 9538 -673_test_discover_names_where_tests_are.sh 1177 -674_a_format_is_run_by_the_runner_named_after_it.sh 8008 -677_a_conditional_dependency_replaces_the_unconditional_one.sh 728 -678_a_row_states_a_library_form.sh 860 -679_a_path_dependency_takes_its_manifests_identity.sh 669 -67_features_strict.sh 7264 -681_index_repos_reach_an_existing_home.sh 9473 -682_why_deps_prints_the_resolved_graph.sh 213 -683_a_build_program_reads_the_payload_pkg_config_view.sh 514 -684_a_runner_supplied_twice_is_refused.sh 518 -685_a_nested_test_loads_a_graph_shared_library.sh 405 -686_an_activated_xlings_subos_does_not_redirect_the_registry.sh 7823 -687_the_vendored_xlings_probe_is_an_argument_vector.sh 629 -688_emit_build_database.sh 4886 -689_pack_takes_features.sh 2163 -68_profile_passthrough.sh 8118 -690_a_shared_library_over_a_graph_cxx_runtime.sh 29859 -692_a_package_states_its_default_link_form.sh 1247 -693_the_root_build_program_reads_dependency_link_forms.sh 4248 -696_a_cxx_layer_provider_keeps_its_own_standard.sh 33280 -698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 4322 -69_package_templates.sh 9948 -700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh 1123 -701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh 1437 -702_a_static_package_under_a_shared_library_is_linked_into_it.sh 747 -70_per_target_flags.sh 231 -710_a_forward_reaches_every_dependency_table.sh 454 -711_a_feature_deps_restatement_names_one_source.sh 5820 -712_a_tool_package_contributes_nothing_to_its_consumer.sh 6191 -713_a_git_dependency_selects_a_repository_member.sh 6173 -714_features_names_a_dependency_feature.sh 5951 -71_required_features_gate.sh 253 -720_the_root_build_program_reads_the_resolved_graph.sh 2778 -722_pack_strips_what_the_graph_built.sh 3741 -723_pack_reports_what_it_produced.sh 857 -72_test_profile_features.sh 337 -730_bare_compat_dependency_is_not_a_refresh_miss.sh 97 -731_planning_children_do_not_inherit_the_callers_pipe.sh 438 -732_xlings_refresh_is_bounded_and_owned.sh 4999 -733_envelope_reports_observed_network_access.sh 344 -734_auto_refresh_false_governs_first_index_sync.sh 159 -735_offline_missing_package_has_its_own_code.sh 118 -736_compile_flag_words_reach_the_compiler_and_the_databases.sh 414 -737_the_declared_runtime_payload_is_installed.sh 72189 -739_a_private_feature_dep_does_not_reach_the_consumer.sh 296 -73_issue131_per_target_cxxflag.sh 177 -740_gcc_over_a_graph_c_library_is_degraded_not_refused.sh 123 -741_c_abi_environment_declared_by_the_c_library.sh 177 -742_gcc_accepted_when_the_realisation_is_empty.sh 112 -743_kernel_abi_interfaces_are_resolved_not_preprocessed.sh 205 -744_a_declared_absence_explains_the_link_that_asked_for_it.sh 90 -745_no_run_builds_the_tests_and_says_so.sh 367 -746_macos_deployment_target_is_target_keyed_not_host_keyed.sh 233 -74_run_no_loader_env_leak.sh 1306 -75_index_status_offline.sh 8087 -765_a_consumer_include_directory_stays_in_the_consumer.sh 675 -76_compile_commands_generated.sh 1245 -76_qualified_custom_ns_noncanonical_filename.sh 458 -770_workspace_member_as_dependency.sh 1139 -772_a_published_member_is_self_contained.sh 5544 -773_commands_outside_the_build_read_the_effective_manifest.sh 349 -774_an_index_member_inherits_its_archive_workspace.sh 324 -775_a_host_tool_merges_its_conditional_sections_once.sh 5157 -776_a_path_with_no_utf8_spelling_is_named.sh 314 -777_c_standard_applies_to_the_package_that_declares_it.sh 187 -778_a_graph_link_searches_no_host_directory.sh 99076 -779_a_build_program_declares_a_runtime_search_dir.sh 1953 -77_cdb_preserves_test_entries.sh 4460 -780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 1689 -781_deleted_root_cdb_is_restored.sh 2308 -782_foreign_root_cdb_entries_are_replaced.sh 3587 -783_cdb_switches_whole_with_the_configuration.sh 3204 -784_cdb_replays_from_its_directory.sh 445 -785_cdb_interface_flag_module_extensions.sh 221 -786_std_unit_in_the_database_and_build_id.sh 1191 -787_emit_plans_every_member_on_its_own.sh 165 -788_emit_defers_an_unbuilt_host_tool.sh 1183 -789_emit_describes_a_member_past_a_failed_build_program.sh 327 -78_test_main_combinations.sh 11880 -790_a_prepare_action_populates_an_unknown_directory.sh 1823 -791_a_prepare_action_that_creates_nothing_fails.sh 955 -792_a_rerun_input_inside_a_prepare_directory_is_warned.sh 1743 -793_an_unknown_action_role_is_refused.sh 2595 -795_a_link_flag_reaches_the_linker_as_written.sh 960 -798_a_rule_applies_through_a_source_it_claims.sh 13286 -799_an_action_runs_with_its_env_and_cwd.sh 1024 -79_gtest_regular_dep_feature_main.sh 14959 -800_a_feature_provides_its_host_tools.sh 1031 -801_a_dependency_program_is_shipped_with_the_consumer.sh 1851 -802_a_host_build_applies_its_host_row.sh 1695 -803_a_recorded_payload_that_is_gone_is_refused_offline.sh 111 -804_a_path_host_tool_builds_with_its_chosen_toolchain.sh 1148 -805_rooted_workspace_reaches_its_own_path_dependency.sh 481 -806_dash_p_resolves_the_package_first.sh 755 -807_a_lib_root_imports_its_own_package.sh 508 -808_device_source_is_not_a_compile_unit.sh 15188 -809_failed_build_program_hides_no_device_source.sh 89 -80_feature_defines.sh 259 -810_two_packages_deploy_the_same_bytes_to_one_name.sh 1941 -812_an_index_floor_is_a_closing_tip.sh 252 -813_conditional_dialect_cxxflags_root_only.sh 23819 -815_the_database_describes_what_a_rule_generates.sh 970 -816_a_git_dependency_reports_its_fetch.sh 221 -817_emit_writes_no_xlings_json_under_private_work_dir.sh 283 -818_a_declared_deploy_outranks_a_search_dir_dll_cross.sh 949 -819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 377 -81_capability_binding.sh 573 -821_an_edge_states_its_advice_on_both_paths.sh 1614 -822_a_package_states_the_oldest_mcpp_it_supports.sh 288 -823_mcpp_core_and_mcpp_name_one_interface.sh 765 -824_a_build_program_reads_the_resolved_toolchain.sh 1076 -826_plugin_diagnostics_features_and_names.sh 726 -827_many_placements_run_as_one_process.sh 2327 -828_a_library_states_its_interface.sh 638 -829_pack_takes_a_member.sh 278 -82_feature_optional_deps.sh 264 -831_a_path_dependency_host_module_outside_src.sh 4402 -832_the_fast_path_resumes_after_an_edit.sh 2677 -833_a_workspace_is_one_graph_per_configuration.sh 1703 -834_a_workspace_plans_once_and_replays_in_one_check.sh 1180 -835_a_shared_library_is_placed_by_a_link.sh 319 -836_a_members_own_statements_reach_its_build.sh 1367 -837_a_members_resources_reach_its_image.sh 170 -838_a_program_shipped_through_artifacts_links_its_own_closure.sh 366 -839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 743 -83_feature_defines_propagate.sh 141 -840_a_workspace_database_describes_each_package_once.sh 602 -841_pack_summarizes_many_outputs.sh 3206 -842_the_build_reports_each_step_once.sh 7211 -843_a_terminal_names_the_action_that_runs.sh 4684 -844_a_planned_build_keeps_one_lock_entry_per_identity.sh 794 -845_play_game_restores_the_terminal.sh 12308 -846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 758 -847_a_module_name_is_unique_within_a_program.sh 1960 -849_a_staged_bmi_waits_for_a_module_compiled_here.sh 1607 -84_runtime_dll_deploy.sh 161 -850_a_selected_member_declares_as_the_root.sh 760 -851_a_shared_member_is_compiled_once_per_configuration.sh 536 -852_a_repeated_dash_p_selects_every_member_it_names.sh 273 -853_run_keeps_one_member_and_exclude_removes_members.sh 764 -854_a_test_over_several_members_plans_and_builds_once.sh 1333 -855_a_shared_test_build_is_reported_once_per_group.sh 346 -856_a_test_over_several_members_continues_past_a_failing_one.sh 1528 -857_a_host_module_several_members_import_is_compiled_once.sh 807 -858_the_engines_module_is_compiled_once_per_home.sh 4172 -859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 3220 -85_target_cfg_build_flags.sh 129 -860_a_workspaces_programs_compile_at_the_same_time.sh 16698 -861_concurrent_compiles_plan_what_serial_compiles_plan.sh 1228 -862_run_q_writes_exactly_the_programs_stdout.sh 199 -863_a_run_whose_build_failed_exits_101.sh 662 -864_a_builds_status_is_on_standard_error.sh 753 -865_mcpp_test_reports_its_steps_on_stderr_and_its_results_on_stdout.sh 575 -866_both_streams_on_one_pipe_keep_their_order_and_json_stays_clean.sh 3940 -867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh 10859 -868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh 5486 -869_a_pack_over_several_members_names_each_member_in_its_report.sh 1582 -86_llvm_hermetic_link.sh 3796 -86_target_cfg_dependencies.sh 156 -870_a_pack_over_several_members_resolves_and_fails_per_member.sh 6416 -871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh 11625 -872_a_member_compiles_once_whatever_the_selection.sh 1966 -873_payload_overrides.sh 1826 -874_payload_on_request.sh 2050 -875_toolchain_by_path.sh 665 -876_toolchain_phase.sh 1138 -877_why_sources.sh 83 -87_build_default_profile.sh 502 -88_xlings_environment.sh 9400 -89_build_mcpp.sh 3422 -90_workspace_test.sh 636 -91_target_bare_alias.sh 1021 -92_build_mcpp_import.sh 269 -93_xpkg_parse.sh 84 +659_an_action_declaration_has_no_size_limit.sh 1590 +65_toolchain_runtime_dirs_for_run.sh 922 +660_an_action_names_the_engine_through_mcpp_self.sh 1400 +661_one_identity_two_declarations.sh 707 +662_pack_stages_declared_files_before_the_closure.sh 5200 +663_a_graph_libcxx_over_the_payloads_c_library.sh 13860 +665_a_host_tool_follows_its_source.sh 8580 +669_an_elf_host_pack_states_its_closure.sh 1130 +66_runtime_provides.sh 1080 +670_an_rpath_loader_token_is_left_as_written.sh 181 +671_toolchain_is_an_option_of_run_test_and_pack.sh 3890 +672_a_runner_receives_the_files_the_artifact_carries.sh 8109 +673_test_discover_names_where_tests_are.sh 1440 +674_a_format_is_run_by_the_runner_named_after_it.sh 1600 +677_a_conditional_dependency_replaces_the_unconditional_one.sh 816 +678_a_row_states_a_library_form.sh 890 +679_a_path_dependency_takes_its_manifests_identity.sh 633 +67_features_strict.sh 7740 +681_index_repos_reach_an_existing_home.sh 283 +682_why_deps_prints_the_resolved_graph.sh 232 +683_a_build_program_reads_the_payload_pkg_config_view.sh 705 +684_a_runner_supplied_twice_is_refused.sh 838 +685_a_nested_test_loads_a_graph_shared_library.sh 438 +686_an_activated_xlings_subos_does_not_redirect_the_registry.sh 10470 +687_the_vendored_xlings_probe_is_an_argument_vector.sh 343 +688_emit_build_database.sh 5420 +689_pack_takes_features.sh 2370 +68_profile_passthrough.sh 7130 +690_a_shared_library_over_a_graph_cxx_runtime.sh 18760 +692_a_package_states_its_default_link_form.sh 1800 +693_the_root_build_program_reads_dependency_link_forms.sh 4470 +696_a_cxx_layer_provider_keeps_its_own_standard.sh 25180 +698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 4590 +69_package_templates.sh 11350 +700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh 928 +701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh 1360 +702_a_static_package_under_a_shared_library_is_linked_into_it.sh 752 +70_per_target_flags.sh 176 +710_a_forward_reaches_every_dependency_table.sh 580 +711_a_feature_deps_restatement_names_one_source.sh 4610 +712_a_tool_package_contributes_nothing_to_its_consumer.sh 6270 +713_a_git_dependency_selects_a_repository_member.sh 5270 +714_features_names_a_dependency_feature.sh 6070 +71_required_features_gate.sh 509 +720_the_root_build_program_reads_the_resolved_graph.sh 1670 +722_pack_strips_what_the_graph_built.sh 5890 +723_pack_reports_what_it_produced.sh 1110 +72_test_profile_features.sh 612 +730_bare_compat_dependency_is_not_a_refresh_miss.sh 105 +731_planning_children_do_not_inherit_the_callers_pipe.sh 583 +732_xlings_refresh_is_bounded_and_owned.sh 4180 +733_envelope_reports_observed_network_access.sh 312 +734_auto_refresh_false_governs_first_index_sync.sh 148 +735_offline_missing_package_has_its_own_code.sh 82 +736_compile_flag_words_reach_the_compiler_and_the_databases.sh 436 +737_the_declared_runtime_payload_is_installed.sh 58840 +739_a_private_feature_dep_does_not_reach_the_consumer.sh 328 +73_issue131_per_target_cxxflag.sh 252 +740_gcc_over_a_graph_c_library_is_degraded_not_refused.sh 155 +741_c_abi_environment_declared_by_the_c_library.sh 462 +742_gcc_accepted_when_the_realisation_is_empty.sh 178 +743_kernel_abi_interfaces_are_resolved_not_preprocessed.sh 285 +744_a_declared_absence_explains_the_link_that_asked_for_it.sh 176 +745_no_run_builds_the_tests_and_says_so.sh 536 +746_macos_deployment_target_is_target_keyed_not_host_keyed.sh 397 +74_run_no_loader_env_leak.sh 1140 +75_index_status_offline.sh 8119 +765_a_consumer_include_directory_stays_in_the_consumer.sh 694 +76_compile_commands_generated.sh 1310 +76_qualified_custom_ns_noncanonical_filename.sh 490 +770_workspace_member_as_dependency.sh 1040 +772_a_published_member_is_self_contained.sh 4520 +773_commands_outside_the_build_read_the_effective_manifest.sh 938 +774_an_index_member_inherits_its_archive_workspace.sh 685 +775_a_host_tool_merges_its_conditional_sections_once.sh 4260 +776_a_path_with_no_utf8_spelling_is_named.sh 389 +777_c_standard_applies_to_the_package_that_declares_it.sh 186 +778_a_graph_link_searches_no_host_directory.sh 31510 +779_a_build_program_declares_a_runtime_search_dir.sh 1840 +77_cdb_preserves_test_entries.sh 4330 +780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 1720 +781_deleted_root_cdb_is_restored.sh 2200 +782_foreign_root_cdb_entries_are_replaced.sh 3300 +783_cdb_switches_whole_with_the_configuration.sh 3270 +784_cdb_replays_from_its_directory.sh 443 +785_cdb_interface_flag_module_extensions.sh 368 +786_std_unit_in_the_database_and_build_id.sh 786 +787_emit_plans_every_member_on_its_own.sh 216 +788_emit_defers_an_unbuilt_host_tool.sh 1520 +789_emit_describes_a_member_past_a_failed_build_program.sh 655 +78_test_main_combinations.sh 10910 +790_a_prepare_action_populates_an_unknown_directory.sh 2080 +791_a_prepare_action_that_creates_nothing_fails.sh 1030 +792_a_rerun_input_inside_a_prepare_directory_is_warned.sh 1680 +793_an_unknown_action_role_is_refused.sh 2530 +795_a_link_flag_reaches_the_linker_as_written.sh 1050 +798_a_rule_applies_through_a_source_it_claims.sh 10570 +799_an_action_runs_with_its_env_and_cwd.sh 1090 +79_gtest_regular_dep_feature_main.sh 11370 +800_a_feature_provides_its_host_tools.sh 1050 +801_a_dependency_program_is_shipped_with_the_consumer.sh 1090 +802_a_host_build_applies_its_host_row.sh 1400 +803_a_recorded_payload_that_is_gone_is_refused_offline.sh 80 +804_a_path_host_tool_builds_with_its_chosen_toolchain.sh 2140 +805_rooted_workspace_reaches_its_own_path_dependency.sh 598 +806_dash_p_resolves_the_package_first.sh 861 +807_a_lib_root_imports_its_own_package.sh 777 +808_device_source_is_not_a_compile_unit.sh 13580 +809_failed_build_program_hides_no_device_source.sh 433 +80_feature_defines.sh 270 +810_two_packages_deploy_the_same_bytes_to_one_name.sh 1100 +812_an_index_floor_is_a_closing_tip.sh 214 +813_conditional_dialect_cxxflags_root_only.sh 22660 +815_the_database_describes_what_a_rule_generates.sh 1200 +816_a_git_dependency_reports_its_fetch.sh 197 +817_emit_writes_no_xlings_json_under_private_work_dir.sh 252 +818_a_declared_deploy_outranks_a_search_dir_dll_cross.sh 1570 +819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 303 +81_capability_binding.sh 593 +821_an_edge_states_its_advice_on_both_paths.sh 1620 +822_a_package_states_the_oldest_mcpp_it_supports.sh 266 +823_mcpp_core_and_mcpp_name_one_interface.sh 856 +824_a_build_program_reads_the_resolved_toolchain.sh 791 +826_plugin_diagnostics_features_and_names.sh 1020 +827_many_placements_run_as_one_process.sh 2650 +828_a_library_states_its_interface.sh 697 +829_pack_takes_a_member.sh 341 +82_feature_optional_deps.sh 316 +831_a_path_dependency_host_module_outside_src.sh 5000 +832_the_fast_path_resumes_after_an_edit.sh 2630 +833_a_workspace_is_one_graph_per_configuration.sh 1780 +834_a_workspace_plans_once_and_replays_in_one_check.sh 872 +835_a_shared_library_is_placed_by_a_link.sh 297 +836_a_members_own_statements_reach_its_build.sh 1440 +837_a_members_resources_reach_its_image.sh 155 +838_a_program_shipped_through_artifacts_links_its_own_closure.sh 425 +839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 1610 +83_feature_defines_propagate.sh 166 +840_a_workspace_database_describes_each_package_once.sh 850 +841_pack_summarizes_many_outputs.sh 1510 +842_the_build_reports_each_step_once.sh 7560 +843_a_terminal_names_the_action_that_runs.sh 4550 +844_a_planned_build_keeps_one_lock_entry_per_identity.sh 943 +845_play_game_restores_the_terminal.sh 11710 +846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 380 +847_a_module_name_is_unique_within_a_program.sh 1980 +849_a_staged_bmi_waits_for_a_module_compiled_here.sh 1870 +84_runtime_dll_deploy.sh 322 +850_a_selected_member_declares_as_the_root.sh 1000 +851_a_shared_member_is_compiled_once_per_configuration.sh 744 +852_a_repeated_dash_p_selects_every_member_it_names.sh 256 +853_run_keeps_one_member_and_exclude_removes_members.sh 1330 +854_a_test_over_several_members_plans_and_builds_once.sh 1760 +855_a_shared_test_build_is_reported_once_per_group.sh 329 +856_a_test_over_several_members_continues_past_a_failing_one.sh 1540 +857_a_host_module_several_members_import_is_compiled_once.sh 878 +858_the_engines_module_is_compiled_once_per_home.sh 4400 +859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 4110 +85_target_cfg_build_flags.sh 155 +860_a_workspaces_programs_compile_at_the_same_time.sh 13460 +861_concurrent_compiles_plan_what_serial_compiles_plan.sh 1390 +862_run_q_writes_exactly_the_programs_stdout.sh 399 +863_a_run_whose_build_failed_exits_101.sh 992 +864_a_builds_status_is_on_standard_error.sh 888 +865_mcpp_test_reports_its_steps_on_stderr_and_its_results_on_stdout.sh 742 +866_both_streams_on_one_pipe_keep_their_order_and_json_stays_clean.sh 4080 +867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh 3340 +868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh 4040 +869_a_pack_over_several_members_names_each_member_in_its_report.sh 1470 +86_llvm_hermetic_link.sh 2690 +86_target_cfg_dependencies.sh 285 +870_a_pack_over_several_members_resolves_and_fails_per_member.sh 2700 +871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh 10380 +872_a_member_compiles_once_whatever_the_selection.sh 2570 +873_payload_overrides.sh 1700 +874_payload_on_request.sh 2890 +875_toolchain_by_path.sh 634 +876_toolchain_phase.sh 1610 +877_why_sources.sh 133 +878_an_upgraded_engine_does_not_replay_the_old_graph.sh 2110 +879_a_providers_module_extension_reaches_the_fast_path.sh 4900 +87_build_default_profile.sh 631 +880_a_members_executable_keeps_its_own_implementation.sh 2210 +882_parent_directory_glob_inputs.sh 7430 +883_run_hands_the_terminal_to_the_program.sh 3380 +884_a_shared_library_links_with_its_own_closure.sh 1370 +885_workspace_profiles_and_the_root_package.sh 977 +886_std_compat_is_built_when_imported.sh 6050 +88_xlings_environment.sh 259 +890_a_toolchain_declared_for_an_unservable_target_is_still_installed.sh 22 +891_a_host_header_is_written_not_searched.sh 558 +893_the_workspace_prefix_carries_the_meaning.sh 819 +894_the_position_reading_is_kept_and_reported.sh 898 +895_conditional_configuration_rows_count_as_they_evaluate.sh 821 +896_an_action_output_has_one_writer.sh 5630 +897_each_configuration_has_its_own_generated_output.sh 4980 +898_module_units_in_cpp_files.sh 10390 +899_toolchain_flag_replacing_a_declared_toolchain_says_so.sh 894 +89_build_mcpp.sh 2680 +900_publish_writes_the_workspace_layer_into_the_member.sh 342 +901_the_toolchain_library_path_stays_with_the_tools.sh 3020 +902_a_build_dependency_library_is_compiled_as_its_package.sh 5490 +903_one_package_in_both_tables_has_two_instances.sh 9580 +904_a_build_dependency_feature_stays_out_of_the_target.sh 4850 +905_a_build_program_links_its_libraries_whole.sh 5230 +906_a_build_program_is_compiled_when_its_graph_changes.sh 8070 +907_the_library_interface_is_stated_once.sh 728 +908_what_a_build_dependency_asks_for.sh 5000 +909_deprecated_readings_are_registered_and_kept.sh 5660 +90_workspace_test.sh 789 +910_a_build_plugin_is_a_package_of_the_build_program.sh 8590 +911_a_workspace_states_each_table_once.sh 971 +912_two_configuration_groups_share_the_build_programs_graph.sh 5780 +913_the_build_database_describes_the_build_programs.sh 6370 +914_an_upstream_home_is_linked_never_written.sh 15410 +915_the_system_home_is_maintained_and_read.sh 169 +916_a_home_from_before_keeps_working.sh 228 +918_an_older_packaging_is_used_not_replaced.sh 12410 +919_a_home_selection_is_remembered_once.sh 144 +91_target_bare_alias.sh 365 +920_the_doctor_checks_resources_without_bootstrap.sh 2160 +921_emit_watches_inputs_not_host_outputs.sh 1020 +923_cross_dependency_images_keep_their_resources.sh 802 +924_visible_imports_resolve_transitive_modules.sh 517 +925_a_stated_sysroot_is_the_c_library.sh 21350 +926_the_hosts_own_root_is_a_base.sh 8260 +927_the_runner_contract.sh 677 +928_an_apple_sdk_is_located_on_any_host.sh 298 +929_release_trims_paths_and_defines_ndebug.sh 3230 +92_build_mcpp_import.sh 285 +930_a_toolchain_phase_states_the_base.sh 3900 +931_a_suggested_command_quotes_its_path.sh 17310 +932_a_fresh_home_with_a_space_and_non_ascii_builds.sh 40640 +933_one_cxx_runtime_per_process.sh 2460 +934_llvm_with_a_graph_supplied_libstdcxx.sh 3680 +935_two_builds_of_one_configuration.sh 31770 +936_the_control_contract.sh 17950 +937_the_root_compile_database_is_opt_in.sh 4400 +938_a_relink_never_exposes_a_partial_file.sh 2009 +939_a_lock_written_meanwhile_is_not_overwritten.sh 1270 +93_xpkg_parse.sh 66 +940_an_indirect_import_is_found_whatever_map_its_importer_has.sh 599 +941_a_closed_reader_ends_mcpp_and_no_child_inherits_the_ignore.sh 365 +942_an_ok_image_is_built_once_and_run_by_okrun.sh 61130 +943_a_transitive_git_branch_is_locked.sh 623 +944_a_root_checkout_answers_a_version_the_index_lacks.sh 432 +945_a_miss_anywhere_in_the_graph_asks_for_a_refresh.sh 390 +946_a_program_begins_in_a_dependency_entry.sh 7450 +947_an_image_is_entered_at_one_stated_symbol.sh 2230 +948_an_upgraded_dependency_is_restaged_at_the_same_size.sh 6990 +949_a_version_the_index_lacks_refreshes_at_install.sh 4220 +950_a_kernel_abi_package_over_the_system_c_library_is_in_the_shared_image.sh 397 +951_an_ok_image_needs_no_declaration.sh 157570 +952_a_platform_package_states_a_class_of_targets.sh 1840 +953_the_host_graph_runs_build_programs_as_images.sh 30120 +954_pack_has_three_axes.sh 910 +955_a_universal_package_holds_every_architecture.sh 360000 +956_a_single_file_runs_without_unpacking.sh 117250 +957_packages_merge_and_describe_themselves.sh 93760 +958_a_build_program_runs_again_when_its_output_is_gone.sh 1260 96_msvc_unavailable_nonwindows.sh 38 -98_reflection_import_std.sh 6868 +98_reflection_import_std.sh 7170