From 18ad9d9ecc3b6c48ebf5191c3691df21cc075092 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 22:05:23 +0800 Subject: [PATCH 01/15] =?UTF-8?q?examples/10-graphics:=20=E5=8F=AF?= =?UTF-8?q?=E7=A7=BB=E6=A4=8D=E6=80=A7=E9=A9=B1=E5=8A=A8=E9=BB=98=E8=AE=A4?= =?UTF-8?q?=E6=98=AF=E7=9C=8B=E4=B8=8D=E8=A7=81=E7=9A=84?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 四期(软件设备把 CI 上限抬到运行)的程序侧那一半。 接设备之前先测了上游产物,而第一行的读数就是本条的理由:Khronos 官方的 `MoltenVK-macos.tar` v1.4.2 里,ICD 带着 **`"is_portability_driver": true`**。 **loader 默认不把可移植性驱动交给 `vkEnumeratePhysicalDevices`** —— 一个照着原生 驱动写的 Vulkan 程序因此在 macOS 上一个设备都找不到,并把它报成「这台机器没有 GPU」。诊断是错的,而它看起来完全合理。 规范要的是成对的两半:实例要**启用** `VK_KHR_portability_enumeration` **并且**置位 `VK_INSTANCE_CREATE_ENUMERATE_PORTABILITY_BIT_KHR`;随后凡是声明了 `VK_KHR_portability_subset` 的设备,必须在 `vkCreateDevice` 时启用它,否则调用失败。 按**能力**问,不按 `#ifdef __APPLE__`:性质是「我面前这个 loader 在展示可移植性 驱动」,而 `#ifdef` 两个方向都会错 —— Linux 上跑翻译层的机器也有它,macOS 上对着 原生驱动构建的程序并不需要它。 实测:Linux/lavapipe 上读数不变,中心像素仍是 `(124, 70, 62, 255)`。 顺带记下 Windows 那一侧的测量:`mesa3d-26.2.0-release-msvc` 的 `vulkan_lvp.dll` **只导入系统 DLL**(无 MSVCP140/VCRUNTIME140),静态 CRT,机器上不需要装任何东西。 载荷侧(把 MoltenVK 与 Windows 的 lavapipe 收进索引)未做:那要往发布组织推新的二进制 资产,而 GitCode 的资产不可替换不可删除。 --- ...-heterogeneous-cross-platform-ecosystem.md | 31 +++++++++- examples/10-graphics/README.md | 17 ++++++ .../offscreen/src/vulkan/render.cpp | 59 +++++++++++++++++++ 3 files changed, 106 insertions(+), 1 deletion(-) diff --git a/.agents/docs/2026-09-07-heterogeneous-cross-platform-ecosystem.md b/.agents/docs/2026-09-07-heterogeneous-cross-platform-ecosystem.md index e9ce36536..96617b190 100644 --- a/.agents/docs/2026-09-07-heterogeneous-cross-platform-ecosystem.md +++ b/.agents/docs/2026-09-07-heterogeneous-cross-platform-ecosystem.md @@ -332,7 +332,11 @@ install/config 形状,而那套形状是按 Linux 写的: - **二期(shaderc 三平台)**:xim-pkgindex #778,规则按平台选编译器。已实现。 - **三期(CUDA/SYCL 上 Windows)**:xim-pkgindex #779(五个包),规则侧的路径推导与 路线判断。已实现;端到端由 `windows-test` job 装卸五个包并断言注册的程序验证。 -- **四期(软件设备把 CI 上限抬到运行)**:**未实现,记录为发布工作**。Windows 上需要 +- **四期(软件设备把 CI 上限抬到运行)**:**部分实现**。程序侧那一半做完了:一个 + Vulkan 程序在 macOS 上**看不见任何设备**,除非它显式打开可移植性枚举 —— MoltenVK 是 + portability driver,而 loader 默认不把这种驱动交给 `vkEnumeratePhysicalDevices`。 + 示例现在按**能力**而不是按 `#ifdef __APPLE__` 问这件事(见 10.4e)。载荷侧仍未做, + 记录为发布工作。Windows 上需要 一个 Mesa-on-Windows 的 `vulkan_lvp` 构建;macOS 上 runner 自带 GPU 与 MoltenVK, 所以那一侧不需要软件设备,需要的是把 `compat.vulkan` 的 macOS 腿接到示例上。 今天的上限:三平台**构建**,Linux **运行**。 @@ -396,6 +400,31 @@ include 它。** 同一版还给「每条规则都为本宿主编译过」那个夹具补了它自己的分母:它断言的是「每一条」, 而「每一条」是它 carry 的一张清单。 +### 10.4e 四期的程序侧:可移植性驱动默认是看不见的 + +把软件/可移植设备接到 CI 上之前,先测了上游产物: + +| 产物 | 读数 | +|---|---| +| `MoltenVK-macos.tar` v1.4.2(Khronos 官方) | 92 个条目;`dynamic/dylib/macOS/libMoltenVK.dylib` 是 x86_64+arm64 的 universal binary(11 MB),ICD 用相对路径 `./libMoltenVK.dylib`,并带 **`"is_portability_driver": true`** | +| `mesa3d-26.2.0-release-msvc.7z`(pal1000 重分发) | `x64/vulkan_lvp.dll` 56 MB + `x64/lvp_icd.x86_64.json`;**只导入系统 DLL**(KERNEL32/GDI32/USER32/ADVAPI32/ole32/SHELL32/ntdll),没有 MSVCP140 —— 静态 CRT,机器上不需要装任何东西 | + +第一行那个 `is_portability_driver` 是本节存在的理由:**loader 默认不把可移植性驱动交给 +`vkEnumeratePhysicalDevices`**。一个照着原生驱动写的程序因此在 macOS 上一个设备都找不到, +并把它报成「这台机器没有 GPU」—— 诊断是错的,而它看起来完全合理。 + +规范要求的是成对的两半:实例要**启用** `VK_KHR_portability_enumeration` **并且**置位 +`VK_INSTANCE_CREATE_ENUMERATE_PORTABILITY_BIT_KHR`;随后凡是声明了 +`VK_KHR_portability_subset` 的设备,必须在 `vkCreateDevice` 时启用它,否则调用失败。 + +示例按**能力**问这两件事,不按 `#ifdef __APPLE__`。性质是「我面前这个 loader 在展示 +可移植性驱动」,而 `#ifdef` 两个方向都会错:Linux 上跑翻译层的机器也有它,而 macOS 上 +对着原生驱动构建的程序并不需要它。实测:Linux/lavapipe 上读数不变,中心像素仍是 +`(124, 70, 62, 255)`。 + +载荷侧(把 MoltenVK 与 Windows 的 lavapipe 收进索引)未做,因为那要往发布组织推新的 +二进制资产,而 GitCode 的资产不可替换不可删除 —— 这是一个该由人做的决定,不该顺手做掉。 + ### 10.5 一条留下的不一致,以及它什么时候消失 `mcpp:plugins` 0.2.5/0.2.6 里 `xim:shaderc` 在 macOS 与 Windows 上是**精确版本**,而 diff --git a/examples/10-graphics/README.md b/examples/10-graphics/README.md index 11635acd9..0423103f2 100644 --- a/examples/10-graphics/README.md +++ b/examples/10-graphics/README.md @@ -80,6 +80,23 @@ dropped while the source that includes it, selected by the same predicate, was kept. Packages are therefore unconditional or conditioned on the platform; `[build]` sources are what the accelerator selects. +**A portability driver is hidden until the program asks for it.** macOS has no +native Vulkan: MoltenVK implements it on top of Metal, and the specification +calls such an implementation a *portability driver*. The loader does not show +one to `vkEnumeratePhysicalDevices` unless the instance enables +`VK_KHR_portability_enumeration` and sets +`VK_INSTANCE_CREATE_ENUMERATE_PORTABILITY_BIT_KHR`; a device that advertises +`VK_KHR_portability_subset` must then have that extension enabled at +`vkCreateDevice` or the call fails. A program written against a native driver +alone therefore finds no device on such a machine and reports it as "no GPU +here", which is the wrong diagnosis. + +`src/vulkan/render.cpp` asks the loader and the device what they advertise +rather than testing for the platform. The property is "the loader in front of +me is showing portability drivers", and an `#ifdef __APPLE__` would be wrong in +both directions: a Linux machine running a translation layer has it, and a +macOS build against a native driver does not need it. + **One shader per stem.** The generated name is the shader's stem and its stage, so `ui/text.vert` and `world/text.vert` would both produce `text_vert.h` declaring `text_vert_spv`. The rule refuses that and names both files. The diff --git a/examples/10-graphics/offscreen/src/vulkan/render.cpp b/examples/10-graphics/offscreen/src/vulkan/render.cpp index e44ce0185..a1efac9ea 100644 --- a/examples/10-graphics/offscreen/src/vulkan/render.cpp +++ b/examples/10-graphics/offscreen/src/vulkan/render.cpp @@ -33,6 +33,43 @@ const char* g_ran_on = ""; constexpr unsigned char kClear[4] = { 16, 16, 16, 255 }; +// WHETHER AN EXTENSION IS THERE, ASKED RATHER THAN ASSUMED. +// +// The two below decide whether this program can see a PORTABILITY driver -- an +// implementation of Vulkan on top of something else, which is the only kind +// macOS has, because Apple's platform interface is Metal and MoltenVK +// translates. A portability driver is HIDDEN from `vkEnumeratePhysicalDevices` +// unless the instance opts in, so a program written against a native driver +// alone finds no device there and reports it as "this machine has no GPU". +// +// Written as a capability query rather than `#ifdef __APPLE__`: the property is +// "the loader in front of me advertises this", and a Linux machine running a +// portability layer would have it too. The `#ifdef` would also be wrong in the +// other direction -- a macOS build against a native driver does not need it. +bool instance_extension_present(const char* name) { + std::uint32_t n = 0; + if (vkEnumerateInstanceExtensionProperties(nullptr, &n, nullptr) != VK_SUCCESS || n == 0) + return false; + std::vector props(n); + if (vkEnumerateInstanceExtensionProperties(nullptr, &n, props.data()) != VK_SUCCESS) + return false; + for (auto const& p : props) + if (std::strcmp(p.extensionName, name) == 0) return true; + return false; +} + +bool device_extension_present(VkPhysicalDevice phys, const char* name) { + std::uint32_t n = 0; + if (vkEnumerateDeviceExtensionProperties(phys, nullptr, &n, nullptr) != VK_SUCCESS || n == 0) + return false; + std::vector props(n); + if (vkEnumerateDeviceExtensionProperties(phys, nullptr, &n, props.data()) != VK_SUCCESS) + return false; + for (auto const& p : props) + if (std::strcmp(p.extensionName, name) == 0) return true; + return false; +} + int find_memory_type(VkPhysicalDevice phys, std::uint32_t bits, VkMemoryPropertyFlags want) { VkPhysicalDeviceMemoryProperties props{}; @@ -129,6 +166,18 @@ extern "C" int render_offscreen(unsigned w, unsigned h, unsigned char* rgba) { app.apiVersion = VK_API_VERSION_1_2; VkInstanceCreateInfo ici{ VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO }; ici.pApplicationInfo = &app; + + // The opt-in, when the loader has it. Both halves are required and the + // pair is what the specification asks for: the extension has to be + // ENABLED, and the flag has to be set, before a portability driver appears + // in `vkEnumeratePhysicalDevices`. + std::vector instanceExts; + if (instance_extension_present(VK_KHR_PORTABILITY_ENUMERATION_EXTENSION_NAME)) { + instanceExts.push_back(VK_KHR_PORTABILITY_ENUMERATION_EXTENSION_NAME); + ici.flags |= VK_INSTANCE_CREATE_ENUMERATE_PORTABILITY_BIT_KHR; + } + ici.enabledExtensionCount = static_cast(instanceExts.size()); + ici.ppEnabledExtensionNames = instanceExts.empty() ? nullptr : instanceExts.data(); if (vkCreateInstance(&ici, nullptr, &f.instance) != VK_SUCCESS) return 1; VkPhysicalDevice phys = VK_NULL_HANDLE; @@ -143,6 +192,16 @@ extern "C" int render_offscreen(unsigned w, unsigned h, unsigned char* rgba) { VkDeviceCreateInfo dci{ VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO }; dci.queueCreateInfoCount = 1; dci.pQueueCreateInfos = &qci; + + // AND THE OTHER HALF: a device that advertises `VK_KHR_portability_subset` + // must have it enabled, or `vkCreateDevice` fails. The specification makes + // this one mandatory rather than optional precisely so that a program + // cannot use such a device while believing it is a complete one. + std::vector deviceExts; + if (device_extension_present(phys, "VK_KHR_portability_subset")) + deviceExts.push_back("VK_KHR_portability_subset"); + dci.enabledExtensionCount = static_cast(deviceExts.size()); + dci.ppEnabledExtensionNames = deviceExts.empty() ? nullptr : deviceExts.data(); if (vkCreateDevice(phys, &dci, nullptr, &f.device) != VK_SUCCESS) return 1; VkQueue queue = VK_NULL_HANDLE; From ebcbf78e36e8528c3b880b63afcf7e9c7b3e03df Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 22:36:27 +0800 Subject: [PATCH 02/15] build.mcpp: three channels a rule package had no way to reach the engine through Each of these is small, and they share a shape: a rule package already knew something and nothing carried it to a decision the engine makes. `mcpp::action` gains `depfile`. An action's inputs are fixed when build.mcpp runs, before its command has executed, so a compiler that learns its include graph by parsing the source has no channel to report it: editing a file the command merely READ rebuilds nothing and `mcpp build` stays green over a stale artifact. With a depfile declared, ninja reads the file and folds what it names into the edge. Every device compiler this matters for already emits one -- `glslangValidator --depfile`, `glslc -MD -MF`, `slangc -depfile`, `nvcc` and `clang` `-MD -MF` -- so the missing half was always the receiving end. The depfile must not also be declared as an output: `deps = gcc` makes ninja consume and delete it, so an edge that promised it would be permanently dirty. The comment at the emission site says so. `.slang` joins the device-source extension table. A constrained glob's `accel` key does not make a file a device source; `kDeviceExtensions` does. Until it listed `.slang`, a rule package could not introduce a device language on its own: the payload installed, the rule compiled, and the file still fell through to the ordinary source scan and was refused with "mcpp has no role for the extension". Measured against 2026.9.6.6 with a working rule and a working `xim:slang`. Slang is a language rather than a second driver for GLSL -- its own module system, generics, and targets beyond SPIR-V -- so it has an extension here and a rule of its own outside. `[language] modules` is reported as `MCPP_LANGUAGE_MODULES`. A rule that GENERATES a consumer-facing declaration has to choose between a module interface and a header, and the project has already stated which it builds; deriving it any other way would be a second spelling of one decision. An engine that does not report it leaves the variable absent, which a rule reads as "header" -- the behaviour every consumer had before -- so the upgrade needs no project to declare anything. Tests. `tests/e2e/631` builds a project whose action writes a depfile naming a file it never declares as an input, and asserts the action reruns when that file is touched. It carries its own control: a no-op rebuild must not rerun the action, which rules out a backend that reruns unconditionally. A separate reverse control was run by hand against the same binary with `a.depfile` removed and the flag still passed: the action did not rerun, so the assertion measures the field rather than some other trigger. --- ...9-07-module-first-heterogeneous-surface.md | 806 ++++++++++++++++++ CHANGELOG.md | 27 + docs/07-build-mcpp.md | 22 + docs/20-heterogeneous-builds.md | 2 + docs/zh/07-build-mcpp.md | 18 + docs/zh/20-heterogeneous-builds.md | 2 + modules/buildmcpp/src/directives.cppm | 9 + modules/manifest/src/types.cppm | 24 + modules/source-kind/src/source_kind.cppm | 14 +- src/build/build_program.cppm | 11 + src/build/hostprogram.cppm | 15 + src/build/ninja_backend.cppm | 22 + src/build/prepare.cppm | 11 + ...ile_tracks_a_runtime_discovered_include.sh | 147 ++++ tests/unit/test_build_directives.cpp | 39 + tests/unit/test_ninja_backend.cpp | 90 ++ 16 files changed, 1258 insertions(+), 1 deletion(-) create mode 100644 .agents/docs/2026-09-07-module-first-heterogeneous-surface.md create mode 100755 tests/e2e/631_action_depfile_tracks_a_runtime_discovered_include.sh diff --git a/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md b/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md new file mode 100644 index 000000000..3f2f610c6 --- /dev/null +++ b/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md @@ -0,0 +1,806 @@ +# A module-first surface for graphics acceleration and heterogeneous computing + +Date: 2026-09-07. Base: mcpp `81358046`, `mcpp-plugins` `e75e4f5` (0.2.5). + +Scope: the consumer-facing surface of every heterogeneous lane -- CUDA, HIP, +SYCL, Ascend C and SPIR-V -- and the embedding of device payloads into the +artifact. The thesis is one sentence: **a generated header should be an +intermediate that no consumer names, and the only surface a consumer sees +should be a C++ module.** Everything below is either an argument for that +sentence or a consequence of it. + +All measurements were taken on this machine (g++ 16.1.0, binutils 2.42, +32 cores) and the method is stated so each can be reproduced or refuted. + +## 1. What is already true + +This section records the state as of the base commits, not as of an earlier +reading. `mcpp-plugins` 0.2.5 closed several items that an earlier draft of this +analysis listed as open. + +The engine layer carries no vendor knowledge and is one system. The accelerator +axis is a string the engine compares as a shape and places in the fingerprint; +the constrained glob routes device sources; `mcpp::action` with its four roles +is the only graph primitive; `mcpp::fact`, `mcpp::floor` and `mcpp::warning` are +the only channels back; payloads are declared under `feature-xlings` behind two +gates; and a device source no rule claimed is refused at the one place that can +see every rule's share. + +Five rules share one skeleton: read their chunk of `mcpp::accel()`, claim their +extensions, discover payloads through `xpkg_dir`, expose `plan`/`submit`, and +refuse before compiling rather than after. + +Closed in 0.2.5, and not to be re-proposed: + +- `popen` is guarded and spelled `_popen` on Windows (`rules/spirv.cppm:276`, + `rules/sycl.cppm:222`). +- `PATH` is split on the platform's separator (`kPathSep`, `rules/spirv.cppm:203`). +- CI has a `rules-cross-platform` matrix over macos-15 and windows-2022 running + "every rule module compiles for this host", alongside the Linux consumer job. +- Windows shapes exist for CUDA and SYCL. `_WIN32` guard counts are now + cuda 6, spirv 4, sycl 3, hip 1, ascendc 0 -- the last being legitimate, since + Ascend has no Windows platform. +- Two shaders whose stems collide are refused naming both files, the symbol and + the way out, with a CI step asserting each of those five strings. + +Still open, and load-bearing for what follows: + +- `mcpp::action` has no depfile field. Its fields are `id`, `role`, + `description`, `blocking`, and the `input`/`output`/`arg`/`provides`/ + `imports`/`target` setters. Inputs are fixed at submission. +- `glslc` cannot be selected by declaration; `find_compiler` consults + `xpkg_dir("glslang")` before `xpkg_dir("shaderc")` and the rule declares + glslang unconditionally, so glslang wins in every build that works. +- Nothing generates a module. `mcpp.tools.embed` and `mcpp.rules.spirv` both + emit headers that the consumer includes by name. + +## 2. Two kinds of lane, and only one of them wants a generated surface + +The five rules are one system, but their payloads are not one kind of thing, and +the difference decides who writes the interface. + +**Code lanes: CUDA, HIP, SYCL, Ascend C.** The device translation unit is code. +Its interface is a design decision -- which functions, which types, what happens +on failure -- and no generator can make that decision well. The example lanes +already carry the right shape: `src/kernels/saxpy.cu` is an island with an +`extern "C"` header, and `src/app.cppm` is a hand-written seam that turns raw +pointers back into a C++ interface. The `extern "C"` header is required, because +the island is compiled by a host compiler mcpp did not resolve and the two sides +share no C++ ABI. + +**Data lanes: SPIR-V, and `tools.embed` generally.** The payload is data. Its +interface is mechanical: an address and a size. There is nothing to design, and +therefore nothing a generator can get wrong. + +This distinction has a consequence worth stating plainly, because it is the +whole design: + +> The CUDA lane already does what this document asks for. `saxpy/saxpy.h` is a C +> header that only the seam includes; every consumer writes `import app.saxpy` +> and never names the header. The header is an intermediate. The SPIR-V lane +> does not, because there the generated header *is* the surface. + +So the work is not to invent a shape. It is to give the data lanes the shape the +code lanes already have, and to generate it rather than ask each project to +write it. + +## 3. Measurements that constrain the design + +### 3.1 Where the bytes should live + +Method: 100 payloads of 4096 32-bit words (16 KB each, the size of an ordinary +compute shader), 1.6 MB total. Header route: one `const uint32_t[]` header per +payload, ten per translation unit, eleven TUs. Object route: 100 binaries +through `objcopy -I binary`, then link. Both `-O2`, both `-P 32`. + +``` +header route compile 0.64s + link 0.44s = 1.10s +object route objcopy 1.28s + compile 0.44s + link 0.46s = 2.17s +``` + +The header route is twice as fast, which contradicts the intuition that parsing +integer literals dominates. Marginal cost per route, with each program's own +startup subtracted (`g++` on an empty TU: 0.46s; `objcopy --version`: 0.204s -- +both unusually large here because payloads load through the subos farm): + +| Array | Bytes | `g++` marginal | `objcopy` marginal | +|---|---|---|---| +| 4 K words | 16 KB | ~0 | ~0.006s | +| 16 K | 64 KB | 0.02s | ~0.006s | +| 64 K | 256 KB | 0.04s | 0.016s | +| 256 K | 1 MB | 0.31s | ~0 | +| 1 M | 4 MB | 2.31s | 0.116s | + +A real shader is 2 to 50 KB, the leftmost cell, where neither route has a +measurable marginal cost and the total is decided by process count -- which +favours the header route, because one compiler invocation absorbs many headers. + +The crossover is the **total embedded byte count**, not the payload count. +Below roughly 1 MB the header route wins; above roughly 4 MB the compiler's +slightly superlinear curve loses by an order of magnitude. Source expansion is a +constant 2.75x (1.6 MB of binary became 4.4 MB of C source). + +The current default is therefore correct. The reason the rule gives for it -- +that SPIR-V is data rather than code -- does not predict this curve and would +have chosen the same route at any size. + +### 3.2 What may go into a module interface + +Method: 1 MB of data, reached four ways, each compiled with `-fmodules -O2`. +Consumers take the address so the data is odr-used and cannot be folded. + +| Shape | BMI or source | Consumer | +|---|---|---| +| Data in the module (`export inline constexpr`) | BMI 6 282 240 B | 0.54s | +| Data in a header | source 3 145 728 B | 0.68s | +| Data in an object, interface returns `std::span` | **BMI 1 313 968 B** | 0.45s | +| Data in an object, interface returns a std-free POD | **BMI 1 808 B** | 0.49s | + +Two findings, and the second is the larger. + +**Data must not go into a module.** The BMI is six times the data and the +consumer saves twenty per cent, because GCC still deserialises the array rather +than reading it as bytes. Six times the size, carried by every consumer and +invalidated by every compiler version change, does not buy twenty per cent. + +**A std type in a module interface costs 727x the BMI of a std-free one**, and +that cost is fixed rather than proportional to the data: the 1.28 MB is +`#include `'s templates, present whether the payload is 16 KB or 16 MB. +This restates a conclusion this project has already paid for -- a new module +interface carrying std types poisons every downstream BMI, and the error surfaces +on an unrelated module. The generated interface must be std-free. + +There is a second, independent reason for the same decision. A first attempt at +this measurement failed outright: `import std;` requires `std.gcm` to have been +built, and a project with `modules = true, import_std = false` has not built it. +A std-free interface needs neither. + +### 3.3 An incidental hazard worth documenting in the tool + +An `inline constexpr` array that is only partially used is folded and never +emitted. A first measurement of the module route reported a 1 KB object for 1 MB +of data for exactly this reason. SPIR-V is unaffected because the whole array is +odr-used, but a consumer of `tools.embed` that reads one element and expects the +bytes to be in the binary will not find them there. + +### 3.4 Generated module interfaces are already first-class + +`mcpp::action::provides()` and `imports()` exist, and `providesModule` is read +throughout `src/build/ninja_backend.cppm` -- the split-BMI path (1603, 1723), the +`scan_overrides` path (1760), and `--expect-provides` (1777). +`docs/07-build-mcpp.md:517` shows the call. A generated `.cppm` that declares +what it provides is an ordinary node in the module graph today. + +Nothing is missing. The mechanism exists and no rule uses it for a payload. + +### 3.5 Both shader compilers already emit dependencies + +Verified against the installed payloads: + +``` +glslangValidator --depfile "writes depfile for build systems" +glslc -MD -MF "Generate make dependencies and compile" +slangc -depfile +``` + +`mcpp::action` has no field to receive one. + +## 4. Architecture: three planes + +The design is to separate three questions that are presently one. + +**The producer plane** decides which compiler turns a device source into bytes +or an object. This is `mcpp.rules.*` and it is already unified. + +**The storage plane** decides where those bytes live: generated C source, an +object section, or a file beside the artifact. + +**The surface plane** decides how a consumer names them: a C++ module, or a C +header. + +Today the three are entangled in each rule. `rules.spirv` asks the shader +compiler to emit a C declaration, so the producer decides the storage; the two +shader compilers disagree about what that means -- glslang's `-x --vn` writes a +complete declaration, glslc's `-mfmt=c` writes a bare initialiser list -- so the +rule carries a branch for each and writes one of the two headers itself in +`wrap_glslc_output`. Roughly half the branching in that file exists to reconcile +one disagreement about a question the producer should not be answering. + +After separation: + +``` +device source ──producer──> bare payload ──storage──> .S | .h | sidecar + │ + surface ──> .cppm (default) + .h (modules = false) +``` + +The producer emits a bare `.spv` and stops. `--vn` and `-mfmt=c` are no longer +passed, and the flavour branch disappears from everything except flag spelling. + +### 4.1 Why the storage plane must be a tool, and why that tool needs a binary + +`mcpp.tools.embed` is already the storage plane for one case. Its +`element::word32` exists specifically for SPIR-V, and its header says so. No +rule can call it, and the obstacle is altitude rather than oversight: the tool +reads bytes while `build.mcpp` runs, and a `.spv` the graph has not produced yet +does not exist at that moment. + +The change is to give the tool a second incarnation -- a host tool binary the +graph invokes as an action. mcpp already has the mechanism: a dependency that +produces a host tool, built through a nested host sub-build into a tool store +(issue #355). The library form stays for data files checked into a project, +where reading at build-program time is correct and cheaper. + +What this buys, in order of value: + +- The two shader flavours produce identical output. The rule keeps only the + knowledge that justifies its existence: which compiler, which flags. +- Storage and surface become orthogonal options rather than three interfaces. +- Any future producer of SPIR-V -- `slangc`, `dxc` -- joins at no cost, because + embedding is no longer the producer's business. +- The same path serves CUDA cubins and fatbins, textures, fonts, model weights + and ICD JSON. The tool stops being shader-specific. + +### 4.2 One tool with options, not three + +`bin2c`, `bin2cppm` and `bin2obj` as three names would force a consumer to +change its import in order to change storage, which is the cliff +`docs/07-build-mcpp.md` forbids. Identifier derivation, namespace handling, the +size symbol, and the content comparison that avoids spurious rebuilds are common +to all three. + +```cpp +export namespace mcpp::tools::embed { + enum class storage { header, object, sidecar }; + enum class surface { module_, c_header }; + enum class element { byte_, word32 }; +} +``` + +`sidecar` earns its place -- shader hot reload, weights too large to link, +runtimes that require a path -- and it is the only one that needs engine work: +`role = "artifact"` plus an entry in the `runtime.artifacts` allow-list that +`mcpp pack` consults. + +### 4.3 The object storage should use `.incbin` + +```asm + .section .rodata + .globl blur_comp_spv + .balign 4 +blur_comp_spv: + .incbin "…/blur.spv" + .globl blur_comp_spv_end +blur_comp_spv_end: +``` + +gas and clang's integrated assembler both accept this on every platform mcpp +resolves a toolchain for, so the tool needs no knowledge of ELF, Mach-O or COFF, +and there is no `objcopy` process per payload -- section 3.1 measured 100 +parallel invocations at 1.28 seconds, essentially all startup. The `.balign 4` +supplies the alignment `VkShaderModuleCreateInfo::pCode` requires and that a +byte-typed symbol does not carry. + +The assembler resolves `.incbin` at assembly time and reports no dependency, so +the payload must be declared as an action input explicitly. This is the same +requirement as the depfile item in section 7 and shares its fix. + +## 5. The module surface + +### 5.1 The interface is std-free, and it is a function + +A variable cannot hold one shape across the three storages, because `constexpr` +and `extern` are mutually exclusive: under object storage the bytes live in a +section and the compiler does not know their values. A function can. + +```cpp +export module myapp.shaders; + +export namespace myapp::shaders { + +// std-free by construction: see 3.2. A consumer that wants a span writes one, +// and `` is then included by the consumer rather than by this interface. +struct spirv { + const unsigned* code; // VkShaderModuleCreateInfo::pCode + unsigned long size_bytes; // VkShaderModuleCreateInfo::codeSize +}; + +spirv blur_comp(); +spirv tonemap_frag(); + +} +``` + +The field names are taken from the consumer's API so that no arithmetic appears +at the call site: + +```cpp +import myapp.shaders; + +const auto s = myapp::shaders::blur_comp(); +VkShaderModuleCreateInfo ci{ .codeSize = s.size_bytes, .pCode = s.code }; +``` + +Under header storage the function returns a span over an array defined in the +same translation unit and is inlined away. Under object storage it returns +`{start, end - start}` over two extern symbols. Under sidecar storage it reads +or maps the file. The signature does not change, so storage becomes an option a +project revises without touching a consumer. + +### 5.2 Module naming + +`mcpp.*` is reserved for rules the project maintains, and the engine warns on it +keyed by package namespace. A generated module therefore lives under the +consuming package's own name. + +``` +default .shaders package myapp -> myapp.shaders +explicit [rules] spirv = { module = "myapp.gfx.shaders" } +``` + +**One module per package, not one per payload.** 134 shaders as 134 modules +would be 134 BMI files, 134 graph nodes and 134 import lines. One module is a +single BMI of roughly 2 KB holding 134 declarations. + +`myapp.shaders` is a cohesive name rather than a grab-bag: one kind of thing, +one producer, one reason to change. The prohibition on pocket modules is not in +tension with it. + +### 5.3 Namespace naming + +**The module name and the namespace are the same identifier path, with `.` and +`::` exchanged.** Subdirectories add a namespace segment and no module. + +``` +myapp.shaders -> namespace myapp::shaders +shaders/blur.comp -> myapp::shaders::blur_comp() +shaders/post/tonemap.frag -> myapp::shaders::post::tonemap_frag() +``` + +Nothing has to be looked up. This also dissolves the stem collision that 0.2.5 +resolved by refusal: `a/scale.comp` and `b/scale.comp` become +`myapp::shaders::a::scale_comp` and `myapp::shaders::b::scale_comp`, and the +refusal remains only for a genuine collision within one directory. + +The identifier keeps the stage -- `blur_comp` rather than `blur` -- because +`blur.comp` and `blur.frag` would otherwise collide, and drops the `_spv` +suffix, which the namespace already states. The stage is kept uniformly rather +than only when needed; conditional naming is worse than verbose naming. + +The project's own example writes `export module app.saxpy;` and opens +`namespace app`. A hand-written module's author may choose. Generated code has +no reason to surprise anyone. + +### 5.4 The header becomes an intermediate + +Under the module surface, the generated header (or the generated `.S`) is +consumed only by the generated implementation translation unit. No consumer +names it, no consumer's `#include` mentions it, and `mcpp::include_dir` does not +need to advertise it on the consumer's include path. + +Under `surface = c_header` -- a project with `modules = false` -- the same +header becomes the surface. There is one artefact, and which role it plays is +decided by one option. + +### 5.5 The default follows the project, not a preference + +| `[language] modules` | default surface | +|---|---| +| `true` | module | +| `false` | C header | + +The user makes no new decision, and the default cannot be wrong. `[rules]` can +override it in either direction. + +### 5.6 A generated module and the `--no-accel` build + +A build with no accelerator leaves the constrained glob out, so the rule sees no +device sources and generates nothing. A consumer that writes +`import myapp.shaders;` from a translation unit compiled in every build would +then fail to compile in exactly the CPU-only configuration CI runs. + +The resolution is the seam pattern the code lanes already use, and it needs no +special case: the generated shader module is imported only from the +accelerator-side implementation behind the project's own seam, exactly as +`saxpy.cu` is compiled only when the build names CUDA. Under `--no-accel` that +translation unit is not compiled and nothing imports the module. + +This must be stated rather than assumed, because the failure is a compile error +in a configuration the author may not build locally. The engine can see it: it +knows the module graph and which translation units are `cfg`-gated, so an import +of an accelerator-produced module from an ungated TU is a diagnosable condition +rather than a link error. + +## 6. Layers, and where the simplicity comes from + +`docs/07-build-mcpp.md` already fixes the rule: layers must not have a cliff, +and each layer must be the composition of the one below it. `generate_all(opt)` +is `submit(plan_all(opt))`. The rules honour this internally and expose +`plan`/`submit`. + +What is missing is the layer *above* `build.mcpp`. The four lines every project +writes identically, plus `host-module = true` and the feature name, are +configuration expressed as construction. + +| Layer | Form | Audience | +|---|---|---| +| **L0** | `[build] accel` plus a glob; nothing else | most projects | +| **L1** | `[rules] spirv = { storage = "object", includes = ["shaders"] }` | tuning | +| **L2** | `build.mcpp` calling `compile(opt)` | today's only entry | +| **L3** | `plan()`, edit the edges, `submit()` | full control | + +L2 and L3 exist. L0 and L1 do not. + +**The table L0 needs already exists in the engine.** `src/build/prepare.cppm` +prints it in the diagnostic for a device source no rule claimed, naming +`mcpp.rules.cuda` for `.cu` and `mcpp.rules.spirv` for shaders. Nothing reads +it. Lifting that mapping out of the string is most of L0's work, and it is the +same shape as a defect this project has recorded before: an answer that is +parsed correctly and then wired to no decision. + +The target, for a project that wants a Vulkan compute shader in its binary: + +```toml +[build] +accel = "vulkan1.2" +sources = ["src/*.cpp", "shaders/**/*.comp"] +``` + +```cpp +import myapp.shaders; +const auto s = myapp::shaders::blur_comp(); +``` + +No `build.mcpp`, no `[build-dependencies]`, no feature name, no +`host-module = true`, no generated header named anywhere. What happens behind +it: the extension routes to `rules.spirv`, glslang emits a bare `.spv`, the +embed tool writes a `.S` and a `.cppm`, the assembler produces the object, the +module is a 2 KB BMI, and the link collects both. + +## 7. What remains open, and where each lands + +1. **No dependency tracking for included device sources.** `mcpp::action` has no + depfile field, so `rules/spirv.cppm` declares only the `.comp` and + `rules/cuda.cppm` only the `.cu`. Editing a `.glsl` or a `.cuh` rebuilds + nothing and the build is green. All three shader compilers already emit a + depfile (3.5). This is the only open item that produces a wrong artifact + rather than a failure, and the `.incbin` dependency in 4.3 needs the same + field. + +2. **`glslc` cannot be selected by declaration.** `find_compiler` consults + `xpkg_dir("glslang")` before `xpkg_dir("shaderc")`, and the rule declares + glslang unconditionally, so glslang wins in every build that works. The + rule's own "no shader compiler found" diagnostic advises naming + `xim:shaderc` and says it will win, which holds only when neither is + installed. The project's CI corroborates: the glslc step downloads a tarball + and sets `MCPP_GLSLC` rather than declaring the payload. + +3. **The stem refusal becomes narrower once 5.3 lands.** It should stay for a + genuine within-directory collision and stop firing for two directories. + +4. **Slang.** `slangc` should become `mcpp.rules.slang` rather than a third + flavour of `rules.spirv`: glslang and glslc are two drivers for one language + compiling the same `.comp`, whereas Slang is a different language with a + different extension and a target set that includes DXIL and Metal, for which + the Vulkan axis `parse_target` reads has no answer. Its module dependency + graph needs no new concept -- a shader module package is an ordinary mcpp + package whose `include_dirs` names its `.slang` directory. `vulkan-rt` had to + invent a scope API for this because xmake offered no package-level dependency + to reuse; mcpp has one, and should use it rather than copy the workaround. + +## 8. Staging + +Each stage is useful alone and none depends on a later one. + +**A. A depfile field on `mcpp::action`.** The only open defect that yields a +wrong artifact with a green build. Every compiler involved already emits one. +Fixing it at the action rather than in one rule serves `cuda`, `spirv` and 4.3 +at once. Independent of everything else here. + +**B. `tools.embed` as a host tool, with `storage` and `surface`.** The +separation in section 4. `rules.spirv` stops passing `--vn` and `-mfmt=c`, emits +a bare `.spv`, and loses `wrap_glslc_output` and the branch around it. + +**C. The module surface.** Sections 5.1 through 5.4, on top of B. The +measurement in 3.2 fixes the interface shape; 3.4 confirms the graph already +accepts it. Add the `--no-accel` diagnostic from 5.6 in the same change, because +it is the failure a user meets first. + +**D. `[rules]`, then L0.** Section 6. The largest single improvement to first +use, and the only stage whose main cost is in the engine rather than in the +plugins. + +**E. `mcpp.rules.slang`.** Nearly free after B, because embedding is by then not +the rule's concern. + +## 9. Corrections carried forward + +An earlier reading of this material concluded that the object route is faster at +scale because the compiler must parse the literals. Section 3.1 measures the +opposite at realistic shader sizes: the marginal parse cost is zero below about +64 K words and the total is decided by process count, which favours the header +route. The crossover is real but sits at roughly 1 MB of total embedded data and +is a property of the total rather than of the payload count. + +The same earlier reading proposed `std::span` as the +generated interface type. Section 3.2 measures that at 727 times the BMI of a +std-free equivalent, and the cost is fixed rather than proportional to the +payload. + +The same earlier reading listed the Windows `popen` and `PATH` defects and the +single-job CI as open. `mcpp-plugins` 0.2.5 closed all three; section 1 records +the current state and they are not restated as work. + +## 10. Where this design sits against the alternatives + +The comparison below was taken against `xmake` master and CMake 3.24+ on +2026-09-07, reading each system's source rather than its documentation. + +### 10.1 xmake has already factored the storage plane + +`xmake/rules/utils/` contains `bin2c`, `bin2obj`, `glsl2spv` and `hlsl2spv` as +separate, reusable rules, and `utils.glsl2spv` composes them through +`{bin2c = true}` or `{bin2obj = true}`. The separation of "which compiler +produces the payload" from "where the payload is stored" is therefore not a new +idea, and mcpp is behind on it rather than ahead. Section 4 should be read as +catching up on that axis and going further on the surface axis, not as inventing +the factoring. + +Three properties of the xmake rule bound how far ahead it is: + +- Compiler discovery is `find_tool` over `PATH`, ending in + `assert(glslangValidator or glslc, "... not found!")`. The payload is not + declared, its version is not pinned, and nothing records which compiler ran. +- `batchcmds:add_depfiles(sourcefile_glsl)` declares the source file alone. + `--depfile` and `-MD` are not passed, so a GLSL `#include` is not tracked -- + the same defect section 7 records for mcpp, and one CMake's + `add_custom_command(DEPFILE)` does not have. +- `bin2c` emits a bare initialiser list, so the consumer writes + `static unsigned char x[] = { #include "..." };` by hand; `bin2obj` exposes + raw `_binary_*_start` and `_binary_*_end`. Neither produces a namespace, a + type, or a module. + +### 10.2 The same task, four ways + +| | Declaration | Consumer | Toolchain | +|---|---|---|---| +| CMake 3.24+ | about 18 lines of `add_custom_command` per shader set | writes the array declaration by hand | whatever the machine has | +| xmake | 2 lines (`add_rules("utils.glsl2spv", {bin2c = true})` plus `add_files`) | writes the array declaration by hand | `find_tool` over `PATH` | +| An in-tree rule (`vulkan-rt`) | 2 lines | `shader::name`, generated | `find_program` over `PATH` | +| mcpp today | 6 lines of manifest plus a `build.mcpp` | includes a generated header | declared payload, installed automatically | +| mcpp after this design | 2 lines of manifest | `import myapp.shaders` | declared payload, installed automatically | + +### 10.3 What each system wins + +**CMake** wins on ecosystem size, platform maturity and, on the one axis +measured here, dependency correctness: `DEPFILE` on a custom command is the only +one of the four that tracks a shader's own includes today. + +**xmake** wins on brevity for a single-backend project and currently on storage +options, having all three. + +**mcpp** wins on three axes the others do not address, and one of them is +structural: + +1. *The toolchain is declared rather than discovered.* The payload is named by + the rule, installed by the build, and recorded with `mcpp::fact`. Nothing in + the other three answers "which compiler produced this artifact" without + reproducing the build. This is a property of a build system and a package + manager being one program, and it cannot be added to a rule in isolation. +2. *One accelerator axis across every backend.* A project with a `.cu` and a + `.comp` states its targets once, and each rule claims its own extensions from + one device-source list; a file no rule claimed is refused. In CMake and xmake + the CUDA path and the shader path are unrelated mechanisms that agree only by + convention. +3. *A module surface*, after section 5. No other system generates one. + +**Selection rule.** A project with device code and C++20 modules is the case +mcpp is uniquely suited to. A project with one of the two is better served by +xmake today. A project with neither should use CMake. + +## 11. Cross-platform constraints on the storage plane + +`src/build/prepare.cppm:10243` refuses GAS sources under an MSVC toolchain, and +NASM sources are refused on non-x86 targets. The `.incbin` object storage in +section 4.3 is therefore available exactly where a GAS-capable assembler is: +every gcc and clang toolchain on Linux, macOS and Windows, and not under MSVC. + +This is a constraint to state, not to work around: + +> Object storage requires a GAS-capable assembler. Under an MSVC toolchain the +> embed tool falls back to header storage and says so once through +> `mcpp::warning`. The surface does not change, because the surface is a +> function; only where the bytes live does. + +The fallback is safe precisely because section 5.1 made the interface a function +rather than a variable: a consumer compiled against header storage and one +compiled against object storage see the same declarations. + +Sidecar storage has no assembler requirement and is available everywhere, but it +changes what `mcpp pack` must collect, which is why it is the only storage that +touches the engine. + +## 12. Slang, in full + +Section 7 argued that `slangc` belongs in a rule of its own. This section states +what that rule is. + +**Extensions claimed.** `.slang`. Not the GLSL stage extensions: a `.slang` file +names its entry points internally and `-fshader-stage` has no analogue. + +**Targets.** `slangc -target spirv` under a Vulkan accelerator. The rule reads +the same `mcpp::accel()` axis as `rules.spirv` and refuses when the axis names no +target it can serve. DXIL and Metal are out of scope for the first version: they +require accelerator vocabulary that does not exist yet, and adding a target the +axis cannot express would put the rule ahead of the engine. + +**Payload.** `xim:slang`, declared by the rule under +`cfg(accelerator = "vulkan")` and the `rules-slang` feature, as every other rule +declares its own. A floor rather than an exact pin: the payload's version is not +coupled to a driver. + +**The payload already exists, and X1 is a version bump rather than a new +package.** `openxlings/xim-pkgindex:pkgs/s/slang.lua` was merged on 2026-08-12 +(PR #611) with all three platforms and both architectures, shipping `slangc`, +`slangd`, `slangi` and `slang`, and is pinned at `2026.14.1`. Upstream is at +`v2026.17` (2026-09-04). Two corrections follow from this and are carried into +section 13: + +- `xim:` packages are prebuilt tool payloads and live in + `openxlings/xim-pkgindex` (`spec = "2"`, flat names, no namespace). + `mcpplibs/mcpp-index` (`spec = "1"`) indexes C++ source libraries. An earlier + draft of section 13 assigned X1 to `mcpp-index`, which is the wrong repository. +- The rule can be written and tested against the published payload immediately, + so `rules.slang` does not wait on any index change. A version bump is worth + doing but is not a dependency. + +**Module dependencies need no new concept.** A Slang module package is an +ordinary mcpp package whose `include_dirs` names its `.slang` directory; the rule +turns each resolved dependency's include directories into `-I`. This is the one +place where mcpp's package model removes work another system had to invent: +`vulkan-rt` defines a `slang(...)` scope API with `add_moduledirs` and `add_deps` +because xmake offers no package-level dependency to reuse. + +**Dependency output.** `slangc -depfile `, through the same `mcpp::action` +field stage A adds. + +**Surface.** Identical to `rules.spirv`, because after stage B neither rule owns +the surface. + +## 13. The plan + +### 13.1 Work items + +Status is as of this document's date. "done" means implemented and verified by a +test that was seen to fail without the change. + +| Id | Repo | Item | Depends on | Status | +|---|---|---|---|---| +| E1 | mcpp | `mcpp::action` gains `depfile`; ninja emits `depfile =` and `deps = gcc` for action edges | -- | done | +| E6 | mcpp | `.slang` joins `kDeviceExtensions` in `modules/source-kind/src/source_kind.cppm` | -- | done | +| E7 | mcpp | `[language] modules` reported as `MCPP_LANGUAGE_MODULES`, so the surface default follows the project | -- | done | +| E5 | mcpp | Documentation: `docs/07-build-mcpp.md`, `docs/20-heterogeneous-builds.md` and their `zh` counterparts | E1, E6, E7 | done | +| P3 | mcpp-plugins | The surface: `mcpp::plugins::surface` in the lib root, a generated `.cppm` or `.h`, a std-free POD interface, module and namespace naming per 5.2 and 5.3 | -- | done | +| P4 | mcpp-plugins | `rules.spirv` delegates the surface. It keeps writing the data header, because 0.2.6 already unified the two compilers on one shape -- both emit `.inc` and the rule writes `.h` -- so there is no second copy of that decision left to remove | P3 | done | +| P4b | mcpp-plugins | `tools.embed::group()`: the same surface over payloads that were already on disk | P3 | done | +| P5 | mcpp-plugins | `rules.slang` | P3, E6 released | code done, waits on the mcpp release | +| P7 | mcpp-plugins | Tests: a consumer per member and per surface, and the cross-platform compile matrix extended to `rules-slang` | P3..P5 | done | +| P1 | mcpp-plugins | `storage`: object storage through a generated `.S` using `.incbin`, and sidecar. MSVC falls back to header storage | P3 | staged | +| P6 | mcpp-plugins | Rules pass `--depfile`, `-MD -MF` and `-depfile` | E1 released | staged | +| E2 | mcpp | `[rules]` manifest section; the engine synthesises the build program a rule entry describes | -- | staged | +| E3 | mcpp | L0: the extension-to-rule table is lifted out of the diagnostic in `prepare.cppm` and activates a rule when a device source matches and the accelerator axis agrees | E2 | staged | +| E4 | mcpp | Diagnostic for an accelerator-produced module imported from an ungated translation unit (5.6) | -- | staged | +| X1 | openxlings/xim-pkgindex | Raise `xim:slang` from `2026.14.1` toward upstream `v2026.17`. The rule works against the published pin, so nothing waits on it | -- | staged | + +**What shipped is the surface plane, not the storage plane, and the ordering is +deliberate rather than a shortfall.** Section 3.1 measured the header route as +the faster of the two at realistic shader sizes and put the crossover at roughly +1 MB of total embedded data, so the storage a project gets today is the one the +measurement recommends. Object storage is worth having above that threshold and +carries platform work the surface does not -- a section directive per object +format, symbol prefixes on Mach-O, and the MSVC fallback of section 11 -- which +makes it a round of its own rather than a rider on this one. + +`P2`, a host-tool incarnation of the embed tool, has left the list. It existed +because the surface appeared to need the payload's bytes at plan time, and it +does not: the generated interface holds declarations, the generated +implementation holds includes, and `.incbin` resolves its path at assembly time. +All three can be written before any action runs, which is what made the surface +a wave-1 item. + +### 13.1.1 A cost the implementation paid, stated rather than hidden + +The generator lives in `mcpp.plugins`, the lib root, which grew from about +twenty lines to about four hundred. The lib root is compiled for **every** +consumer of the package, including one that activates only `rules-cuda` and +will never embed anything, so every consumer's build program now compiles the +generator too. + +It is there because a second unit beside the lib root in `[build] sources` is +not compiled as a host module ahead of the members. Measured: a member importing +`mcpp.plugins.surface` failed with `failed to read compiled module`, because only +the lib root is built first. Three members need the generator -- `rules-spirv`, +`rules-slang` and `tools-embed` -- and a feature that two of them had to activate +for the third would be a dependency between members that the feature system does +not express. + +The cost is one host-module compilation of roughly four hundred lines, paid once +per consumer per configure, and it buys the property section 4 exists for: one +generator, so the three members cannot drift. If the engine later compiles every +lib-root source as a host module in listed order, the generator moves to a file +of its own and this paragraph is deleted. + +### 13.2 Dependency structure across repositories + +The plugins CI pins `MCPP_VERSION` to a released mcpp, so a plugins change that +uses a new engine API cannot be tested until that engine is released. This splits +the work into two waves rather than one: + +``` +wave 1 E1 E2 E3 E4 E5 E6 (mcpp) P1 P2 P3 P4 P7 (plugins) + | + +--- release mcpp ---+ + | +wave 2 +--> P5, P6, raise MCPP_VERSION, + release plugins + +any time X1 (xim-pkgindex): raise the xim:slang pin. Nothing waits on it. +``` + +Wave 1's plugins work uses only engine API that exists today: the ordinary +source scan sees the generated interface and implementation, which +`mcpp::generated` adds to the build. Nothing in P1 through P4 waits on the +engine. + +**P5 does, and finding out why corrected an assumption in this document.** A +constrained glob's `accel` key does not make a file a device source. The +decision is a table in the engine, `kDeviceExtensions` in +`modules/source-kind/src/source_kind.cppm`, and a file whose extension is absent +from it falls through to the ordinary source scan: + +``` +warning: [build] accel names vulkan but no constrained glob matched a `.slang` +error: 'scale.slang' is listed in [build] sources, and mcpp has no role for + the extension '.slang'. +``` + +Measured against mcpp 2026.9.6.6 with the rule and the payload both working -- +`xim:slang@2026.14.1` provisioned correctly, and the file was still refused. So +**a rule package cannot introduce a device language on its own**: the engine has +to know the extension first. That is E6, and it makes `rules.slang` a wave-2 +item alongside P6 rather than a wave-1 one. + +One engine addition belongs with E1 because it is what makes the plugins +default correct without coupling the two releases: mcpp reports +`[language] modules` as `MCPP_LANGUAGE_MODULES`, and the surface's default +reads it. An engine that does not set it leaves the header surface in place, +which is what every consumer had before, so an older engine keeps its behaviour +and a newer one moves a project to the module surface with nothing declared. + +### 13.3 Evaluation criteria, by the dimensions this work is judged on + +| Dimension | What decides it | Where it is checked | +|---|---|---| +| Architecture | The surface is written once and no rule re-implements it | `rules.spirv`, `rules.slang` and `tools.embed` all call `mcpp::plugins::surface::emit`, and a fixture per member asserts the same generated shape | +| Stability | Every new field has a reader; no value is recorded and then unused | E1's depfile reaches `build.ninja`; a test asserts a rebuild after touching an included file | +| Simplicity | The smallest working project | Two manifest lines and one import, asserted by a consumer fixture | +| User experience | Nothing generated is named by a consumer | No fixture under the module surface contains the generated header's name | +| Compatibility | An existing consumer keeps building | The current spirv consumer fixture is kept unchanged and must still pass | +| Cross-platform | Every rule compiles for every host, and storage degrades rather than fails | The existing `rules-cross-platform` matrix, extended; the MSVC fallback in section 11 | +| Consistency | One naming rule, applied by both data-lane rules | `rules.spirv` and `rules.slang` produce identically shaped modules from one generator | +| Seamless upgrade | A project that writes nothing new keeps its behaviour | Header storage stays the default under `modules = false`; the default surface follows `[language] modules` | +| Test coverage | Each surface and each storage has a consumer that runs | P7 | + +### 13.4 Release and verification sequence + +1. mcpp pull request: E1 through E5. CI green on the head, then merged, then the + merge commit's run confirmed green on `origin/main`. +2. mcpp release, with the GitCode assets supplied locally through `gtc`. +3. mcpp-plugins pull request: P1 through P7, `MCPP_VERSION` raised to the new + release, package version raised. +4. Optionally, `openxlings/xim-pkgindex`: X1. Independent of the above. +5. Verification of the published artifacts in a sandbox + (`xlings subos use N --sandbox --cmd ...`) with the CN mirror configured, + because a sandbox is the only thing that tests what was published rather than + what is in the working tree. diff --git a/CHANGELOG.md b/CHANGELOG.md index ead5aea61..1483fc262 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,33 @@ ## [2026.9.7.1] - 2026-09-07 +### 三条通道,都是「规则包知道而引擎收不到」的形状 + +这一版加的三样东西各自很小,共同点是它们补的都是同一类缺口:规则包已经知道某件事, +而没有任何通道把它送到引擎的决定上。 + +**`mcpp::action` 增加 `depfile`。** action 的输入在 `build.mcpp` 运行时就定死了,那时 +命令还没执行。一个靠解析源码才知道自己 `#include` 图的编译器,因此没有办法把结果报回 +来:改动一个命令只是**读**过的文件不会触发任何重建,而 `mcpp build` 会在一个陈旧产物上 +保持绿色。声明了 depfile 之后,mcpp 为那条边写出 `depfile =` 与 `deps = gcc`,ninja +读取并把它列出的文件并入这条边的依赖。相关的每个设备编译器本来就能输出它 —— +`glslangValidator --depfile`、`glslc -MD -MF`、`slangc -depfile`、`nvcc`/`clang` 的 +`-MD -MF` —— 缺的一直是接收端。 + +**不要同时把 depfile 声明为 `output()`**:`deps = gcc` 会让 ninja 读完即删,一条承诺了 +该输出的边会永远是脏的。 + +**`.slang` 进入设备源扩展名表。** 受限 glob 的 `accel` 键**不**决定一个文件是不是设备 +源,`kDeviceExtensions` 才决定。所以在此之前,一个规则包无法自己引入一门设备语言: +`xim:slang` 装得上、规则也编得过,而文件仍然掉进普通源集,报的是「mcpp has no role for +the extension '.slang'」。Slang 是一门语言而不是 GLSL 的第二个驱动 —— 它有自己的模块 +系统、泛型,以及超出 SPIR-V 的目标集合 —— 所以它有自己的扩展名,外面也有自己的规则。 + +**`[language] modules` 以 `MCPP_LANGUAGE_MODULES` 报给构建程序。** **生成**面向消费者 +声明的规则要在「模块接口」与「头文件」之间做选择,而项目已经声明过它用哪一种;用别的 +方式推导就是同一个决定的第二种拼法。旧引擎不设这个变量,规则把缺席读作「头文件」—— +也就是这个变量存在之前每个消费者的行为,所以升级不需要任何项目改一个字。 + ### 两处只在 Linux 之外成立的缺陷,以及第一条图形管线 **引擎与规则层早就与平台无关,而生态只在 Linux 上完整。** 把规则包在另外两个平台上 diff --git a/docs/07-build-mcpp.md b/docs/07-build-mcpp.md index ea4a5971f..1b4b69133 100644 --- a/docs/07-build-mcpp.md +++ b/docs/07-build-mcpp.md @@ -470,6 +470,27 @@ package's compile edges to that package's action outputs. > package**, because `include_dir` colours only the declaring package's own > translation units. +**An action whose command discovers its own dependencies declares a depfile** +(mcpp 2026.9.7.1+). `input()` fixes the edge's inputs when `build.mcpp` runs, +before the command has executed, so a compiler that learns its `#include` graph +by parsing the source has no channel to report it — and editing a file the +command merely *read* reruns nothing, leaving `mcpp build` green over a stale +artifact. + +```cpp +a.depfile = dep.c_str(); // a path the command writes +a.arg("--depfile").arg(dep.c_str()); +``` + +mcpp emits `depfile =` and `deps = gcc` for that edge, so ninja reads the file +and folds what it names into the edge's dependencies. Every device compiler this +matters for already emits one: `glslangValidator --depfile`, `glslc -MD -MF`, +`slangc -depfile`, `nvcc`/`clang` `-MD -MF`. + +> **Do not also declare the depfile as an `output()`.** `deps = gcc` makes ninja +> consume and delete it after reading, so an edge that promised it as an output +> would be permanently dirty. + **A check's command does not have to write its stamp** (mcpp 2026.8.29.1+). The verdict is the exit code; the stamp is bookkeeping the graph needs, and mcpp creates it when the command succeeds. Before this, every check needed a @@ -637,6 +658,7 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_TOOLCHAIN_BINUTILS_DIR` *(2026.9.5.2+)* | `mcpp::toolchain_binutils_dir()` | the directory mcpp names with `-B`; empty when it names none (a musl or MinGW payload brings its own assembler and linker) | | `MCPP_CXX_STDLIB` *(2026.9.6.3+)* | `mcpp::cxx_stdlib()` | the C++ standard library the resolved toolchain uses — `libstdc++`, `libc++`, `msvc-stl`; empty when no toolchain resolved. A different question from `MCPP_TARGET_LIBC`, which is the C library | | `MCPP_ACCEL` *(2026.9.5.2+)* | `mcpp::accel()` | the device axis of this build, resolved — `--accel` / `--no-accel` over `[build] accel` — in the wire form `cuda12.9+{sm_89} ptx>=89`; empty when the build asks for no accelerator. A rule package derives its own flags (`-gencode`, `--offload-arch`) from it, so the architecture set is written once, in the manifest. The same value feeds the `cfg(accelerator = "…")` layer key | +| `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | `1` when the declaring package sets `[language] modules`, `0` otherwise. A rule that GENERATES a consumer-facing declaration reads it to choose between a module interface and a header, so a project states that once and never again. An older engine leaves it absent, which a rule reads as `0` -- the behaviour every consumer had before the variable existed | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) | diff --git a/docs/20-heterogeneous-builds.md b/docs/20-heterogeneous-builds.md index 722334ef6..65062fc37 100644 --- a/docs/20-heterogeneous-builds.md +++ b/docs/20-heterogeneous-builds.md @@ -93,6 +93,7 @@ does not accept C++20 modules. | CUDA, HIP | `.cu`, `.hip` | | SYCL | `.sycl` (2026.9.6.1+) | | Ascend C | `.asc`, `.cce` (2026.9.6.5+) | +| Slang | `.slang` (2026.9.7.1+) | | GLSL, by stage | `.comp`, `.vert`, `.frag`, `.geom`, `.tesc`, `.tese`, `.mesh`, `.task`, `.rgen`, `.rint`, `.rahit`, `.rchit`, `.rmiss`, `.rcall` | | GLSL, stage-less | `.glsl` | | HLSL | `.hlsl` | @@ -485,6 +486,7 @@ own denominator. | `rules-hip` | `mcpp.rules.hip` | the project's own clang (`-x cuda`) on the NVIDIA platform | the above plus `xim:hip-nvidia` | `hip, cuda12.9+{sm_89}` | | `rules-sycl` | `mcpp.rules.sycl` | the `xim:dpcpp` payload's clang (`-fsycl`) | `xim:dpcpp`; on Linux also `xim:gcc`, `xim:glibc`, `xim:linux-headers`; `xim:cuda-nvcc` for an NVIDIA target | `sycl` or `sycl, cuda12.9+{sm_89}` | | `rules-spirv` | `mcpp.rules.spirv` | `glslangValidator` or `glslc` | `xim:glslang` on Linux, `xim:shaderc` on macOS and Windows | `vulkan1.2` | +| `rules-slang` | `mcpp.rules.slang` | `slangc` | `xim:slang` | `vulkan1.2` | | `rules-ascendc` | `mcpp.rules.ascendc` | `bisheng` (`-x asc`) from the CANN toolkit | `xim:cann-toolkit` | `ascend8.5+{dav-c220}` | The payload column is what each rule declares for itself under diff --git a/docs/zh/07-build-mcpp.md b/docs/zh/07-build-mcpp.md index 59e5d8d46..ea3bc5601 100644 --- a/docs/zh/07-build-mcpp.md +++ b/docs/zh/07-build-mcpp.md @@ -401,6 +401,23 @@ int main() { > 任何边消费它 —— 于是它从不执行,而编译器读到的是 mcpp 为已声明产物写下的那个空占位 > 文件。这条边**按包**划分,因为 `include_dir` 只染色声明它的那个包自己的 TU。 +**命令自己发现依赖的 action 要声明 depfile**(mcpp 2026.9.7.1+)。`input()` 在 +`build.mcpp` 运行时就把边的输入定死了,而那时命令还没执行,所以一个靠解析源码才知道自己 +`#include` 图的编译器没有任何通道把结果报回来 —— 改动一个命令只是**读**过的文件不会触发 +任何重建,`mcpp build` 会在一个陈旧产物上保持绿色。 + +```cpp +a.depfile = dep.c_str(); // 命令会写出的路径 +a.arg("--depfile").arg(dep.c_str()); +``` + +mcpp 为那条边写出 `depfile =` 与 `deps = gcc`,ninja 读取该文件并把它列出的文件并入这条边 +的依赖。与此相关的每个设备编译器都已经能输出它:`glslangValidator --depfile`、 +`glslc -MD -MF`、`slangc -depfile`、`nvcc`/`clang` 的 `-MD -MF`。 + +> **不要同时把 depfile 声明为 `output()`。** `deps = gcc` 会让 ninja 读完即删,所以一条 +> 承诺了该输出的边会永远是脏的。 + **check 的命令不必自己写 stamp**(mcpp 2026.8.29.1+)。判定是退出码,stamp 是**构建图** 需要的记账;命令成功时由 mcpp 创建它。在此之前每个 check 都需要一个包装脚本去 touch 那个文件 —— 而 command 是 argv、不假设有 shell,所以那个包装器**根本没法可移植地写出来**。 @@ -545,6 +562,7 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_TOOLCHAIN_BINUTILS_DIR` *(2026.9.5.2+)* | `mcpp::toolchain_binutils_dir()` | mcpp 用 `-B` 指的目录;不指时为空串(musl 与 MinGW 载荷自带汇编器与链接器) | | `MCPP_CXX_STDLIB` *(2026.9.6.3+)* | `mcpp::cxx_stdlib()` | 解析出的工具链使用的 C++ 标准库 —— `libstdc++`、`libc++`、`msvc-stl`;没有工具链解析时为空串。与 `MCPP_TARGET_LIBC` 不是同一个问题,后者是 C 库 | | `MCPP_ACCEL` *(2026.9.5.2+)* | `mcpp::accel()` | 本次构建的设备轴,已解析 —— `--accel` / `--no-accel` 优先于 `[build] accel` —— 线上形态 `cuda12.9+{sm_89} ptx>=89`;不要加速器时为空串。规则包从它推导自己的开关(`-gencode`、`--offload-arch`),架构集合因此只在 manifest 写一次。同一个值也喂给 `cfg(accelerator = "…")` 这个 layer 键 | +| `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | 声明它的那个包设了 `[language] modules` 时为 `1`,否则 `0`。**生成**面向消费者声明的规则读它来在模块接口与头文件之间选择,项目因此只需说一次。旧引擎不设这个变量,规则把缺席读作 `0` —— 也就是这个变量存在之前每个消费者的行为 | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu`、`.hip`…),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根(= CWD) | diff --git a/docs/zh/20-heterogeneous-builds.md b/docs/zh/20-heterogeneous-builds.md index 11e30d303..a4753b1d8 100644 --- a/docs/zh/20-heterogeneous-builds.md +++ b/docs/zh/20-heterogeneous-builds.md @@ -75,6 +75,7 @@ C++20 modules 的编译器。 | CUDA、HIP | `.cu`、`.hip` | | SYCL | `.sycl`(2026.9.6.1+) | | Ascend C | `.asc`、`.cce`(2026.9.6.5+) | +| Slang | `.slang`(2026.9.7.1+) | | GLSL(按 stage) | `.comp`、`.vert`、`.frag`、`.geom`、`.tesc`、`.tese`、`.mesh`、`.task`、`.rgen`、`.rint`、`.rahit`、`.rchit`、`.rmiss`、`.rcall` | | GLSL(无 stage) | `.glsl` | | HLSL | `.hlsl` | @@ -405,6 +406,7 @@ sources = ["src/cpu/*.cpp"] | `rules-hip` | `mcpp.rules.hip` | NVIDIA 平台上是工程自己的 clang(`-x cuda`) | 上面那些,再加 `xim:hip-nvidia` | `hip, cuda12.9+{sm_89}` | | `rules-sycl` | `mcpp.rules.sycl` | `xim:dpcpp` 载荷里的 clang(`-fsycl`) | `xim:dpcpp`;Linux 上另有 `xim:gcc`、`xim:glibc`、`xim:linux-headers`;NVIDIA 目标另加 `xim:cuda-nvcc` | `sycl` 或 `sycl, cuda12.9+{sm_89}` | | `rules-spirv` | `mcpp.rules.spirv` | `glslangValidator` 或 `glslc` | Linux 上 `xim:glslang`,macOS 与 Windows 上 `xim:shaderc` | `vulkan1.2` | +| `rules-slang` | `mcpp.rules.slang` | `slangc` | `xim:slang` | `vulkan1.2` | | `rules-ascendc` | `mcpp.rules.ascendc` | CANN 工具包里的 `bisheng`(`-x asc`) | `xim:cann-toolkit` | `ascend8.5+{dav-c220}` | 载荷那一列是每条规则在 `cfg(accelerator = ...)` 之下**为自己**声明的东西,列出来是为了 diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index 1ad395c35..349c837e3 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -842,6 +842,7 @@ std::optional decode_action(std::string_view payloa arr("imports", a.imports); arr("targets", a.targets); a.blocking = j.value("blocking", false); + a.depfile = j.value("depfile", std::string{}); a.description = j.value("description", std::string{}); if (a.command.empty() || a.outputs.empty()) return std::nullopt; if (a.id.empty()) a.id = a.outputs.front(); @@ -909,6 +910,14 @@ void prepare_actions(std::vector& actions, }; absolutize(a.inputs); absolutize(a.outputs); + // The depfile is an OUTPUT-SIDE path — the command WRITES it, ninja + // reads it back — so it needs the identical anchoring `outputs` gets, + // for every role, not just Source: left package-relative it would be + // interpreted relative to the ninja build directory instead of the + // package root, and the `depfile = ` line the backend emits would + // simply never match the file the command actually wrote. + if (!a.depfile.empty() && a.depfile.find("${mcpp.") == std::string::npos) + a.depfile = abs_against(pkgRoot, a.depfile); if (a.role != mcpp::manifest::BuildAction::Role::Source) continue; for (auto const& o : a.outputs) { if (o.find("${mcpp.") != std::string::npos) continue; diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 3ceb9e96a..4cce7fb12 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -392,6 +392,30 @@ struct BuildAction { std::vector imports; // Check only: make compilation wait for this to pass. Off by default. bool blocking = false; + // A Make-style dependency file the action's COMMAND writes as a side + // effect — ninja reads it once the command exits and folds it into its + // own dependency log, the same `deps = gcc` mechanism a `cxx_object` edge + // uses for a compiler's own `#include` graph (mcpp#235/#257). + // + // `inputs` cannot express what this covers, because it is FIXED AT + // SUBMISSION — build.mcpp declares it before anything has been compiled. + // A device-shader compiler discovers its own `#include` graph only by + // parsing the shader (glslangValidator `--depfile`, glslc `-MD -MF`, + // slangc `-depfile`, nvcc/clang `-MD -MF`), which is not knowable until + // the action's command actually runs. Without this field a build stayed + // green over a stale artifact: editing an included `.glsl`/`.cuh` changed + // nothing the action had declared as an input, so nothing reran. + // + // Empty (the default) means the rule emits no depfile, and the action's + // re-run set is exactly its declared `inputs` — unchanged from before this + // field existed. + // + // MUST NOT also appear in `outputs`. `deps = gcc` makes ninja consume and + // DELETE the depfile once it has read it (see the `rule mcpp_action_{i}` + // emission in src/build/ninja_backend.cppm); a path that is simultaneously + // a declared ninja OUTPUT of the same edge would be a file ninja expects + // to still exist after a successful build and has itself just removed. + std::string depfile; std::string description; }; diff --git a/modules/source-kind/src/source_kind.cppm b/modules/source-kind/src/source_kind.cppm index 0cbed7814..b22a914bc 100644 --- a/modules/source-kind/src/source_kind.cppm +++ b/modules/source-kind/src/source_kind.cppm @@ -271,8 +271,20 @@ constexpr std::string_view kHeaderExtensions[] = { ".h", ".hpp", ".hh", ".hxx" } // `op_kernel/` from `op_host/`, and CMake registers ASC as a LANGUAGE of its // own -- so the island is the shape Ascend already has, not one mcpp imposes. // `.cce` is the older spelling of the same thing and is accepted beside it. +// +// `.slang` is the Slang shading language, compiled by `slangc`. It is a +// LANGUAGE rather than a second driver for GLSL -- its own module system, +// generics, and a target set beyond SPIR-V -- which is why it has an extension +// of its own here and a rule of its own outside. +// +// THIS TABLE IS WHAT DECIDES, NOT THE GLOB'S `accel` KEY. A constrained glob +// carrying `accel = "vulkan1.2"` does not make a file a device source; this +// list does, and a file whose extension is absent from it reaches the ordinary +// source scan and is refused with "mcpp has no role for the extension". A rule +// package therefore cannot introduce a device language on its own, and adding +// one here is the engine half of doing so. constexpr std::string_view kDeviceExtensions[] = { - ".cu", ".hip", ".sycl", ".asc", ".cce", + ".cu", ".hip", ".sycl", ".asc", ".cce", ".slang", ".comp", ".vert", ".frag", ".geom", ".tesc", ".tese", ".mesh", ".task", ".rgen", ".rint", ".rahit", ".rchit", ".rmiss", ".rcall", ".glsl", ".hlsl", ".cl", ".metal", diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index ce9d3ec57..d81d83275 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -124,6 +124,16 @@ struct BuildProgramEnv { // (`-gencode`, `--offload-arch`) from here and the architecture set is // written once, in the manifest, and never again in a build program. std::string accel; + // Whether this package builds C++ modules (`[language] modules`). + // + // Reported because a rule package that GENERATES a consumer-facing + // declaration has to choose between a module interface and a header, and + // the project has already stated which it uses. Deriving it any other way + // would be a second spelling of one decision. A rule that reads it can make + // the module surface its default without any project declaring anything, + // and an engine older than this one leaves the variable absent -- which a + // rule reads as "header", the behaviour every consumer had before. + bool languageModules = true; // The device-kind sources (`.cu`, `.hip`, ...) this package's effective // source set matches, package-root-relative with `/` separators, one per // line. The engine has no compile rule for them and hands the list to the @@ -494,6 +504,7 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv e.emplace_back("MCPP_TARGET_LIBC", env.targetLibc); e.emplace_back("MCPP_PROFILE", env.profile); e.emplace_back("MCPP_ACCEL", env.accel); + e.emplace_back("MCPP_LANGUAGE_MODULES", env.languageModules ? "1" : "0"); { std::string joined; for (auto const& d : env.deviceSources) { diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 50efaa59f..564f97e7d 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -147,6 +147,13 @@ struct action { const char* role = "source"; // "source" | "check" | "object" | "artifact" const char* description = ""; bool blocking = false; // check only: gate compilation on it + // A Make-style dependency file the COMMAND writes as a side effect (gcc/ + // clang `-MD -MF`, glslangValidator `--depfile`, glslc `-MD -MF`, slangc + // `-depfile`). Empty (the default) means the rule emits none, and the + // action's re-run set is exactly its declared `inputs`, as before this + // field existed. See BuildAction::depfile (modules/manifest/src/types.cppm) + // for why `inputs` alone cannot express what this covers. + const char* depfile = ""; action& input(const char* p) { add(inputs_, sizeof inputs_, p); return *this; } action& output(const char* p) { add(outputs_, sizeof outputs_, p); return *this; } action& arg(const char* a) { add(command_, sizeof command_, a); return *this; } @@ -167,6 +174,14 @@ struct action { std::printf(",\"role\":"); esc(role); std::printf(",\"description\":"); esc(description); std::printf(",\"blocking\":%s", blocking ? "true" : "false"); + // Optional and omitted rather than sent empty: an action that never + // sets this must serialise to the SAME bytes it did before the field + // existed, because this payload is the cache key `apply()` stores + // verbatim (see the comment there) — an unconditional `"depfile":""` + // on every action would perturb the cache for every build.mcpp that + // has nothing to do with depfiles. The decoder's default (empty + // string) is identical either way, so omission costs nothing on read. + if (depfile[0]) { std::printf(",\"depfile\":"); esc(depfile); } // A truncated argv would otherwise be INVALID rather than obviously // wrong — the engine turns this marker into a diagnostic that names // the limit, instead of a generic "malformed action". diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 941e58eb9..0702743c2 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -2355,6 +2355,28 @@ std::string emit_ninja_string(const BuildPlan& plan) { : a.role == mcpp::manifest::BuildAction::Role::Object ? "OBJECT" : "GENERATE", a.description.empty() ? a.id : a.description)); + // The command's OWN emitted dependency file, when it declared one. + // `a.inputs` is FIXED AT SUBMISSION, before the command has run; a + // device-source compiler (glslangValidator, glslc, slangc, nvcc/clang) + // discovers its `#include` graph only by parsing the source, and a + // Make-style depfile is how it reports that afterward. Wiring it as + // `deps = gcc` is the identical mechanism the nasm rule below uses for + // a `.s`'s own textual includes — a file the command merely READ + // invalidates this edge exactly as a declared `input` would, without + // the build program having to know the include graph in advance. + // + // `deps = gcc` makes ninja CONSUME AND DELETE the depfile once it has + // folded its contents into `.ninja_deps`. That is why `a.depfile` must + // NEVER also be named in `a.outputs`: nothing in this backend adds it + // there on the caller's behalf, and a build.mcpp that named the same + // path both ways would declare a ninja OUTPUT that is expected to + // exist after a successful build and that reading the depfile has + // just removed — ninja would fault the edge for a file its own deps + // processing deleted. + if (!a.depfile.empty()) { + append(std::format(" depfile = {}\n", escape_ninja_path(a.depfile))); + append(" deps = gcc\n"); + } append("\n"); std::string outs, ins; for (auto const& o : a.outputs) outs += " " + escape_ninja_path(o); diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 2974f6e28..554ac35a9 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -8035,6 +8035,10 @@ prepare_build(bool print_fingerprint, bpEnv.toolsBin = projectSubosBin; bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); + // The DECLARING package's setting, not the root project's: a rule + // generating a declaration for this package must match how this + // package is compiled. + bpEnv.languageModules = pkg.manifest.language.modules; if (auto dit = deviceSourcesByPackage.find(pkg.root.string()); dit != deviceSourcesByPackage.end()) bpEnv.deviceSources = dit->second; bpEnv.features = feature_closure(pkg.manifest, req, depDefaultFeatures); @@ -8952,6 +8956,7 @@ prepare_build(bool print_fingerprint, bpEnv.toolsBin = projectSubosBin; bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); + bpEnv.languageModules = m->language.modules; if (auto dit = deviceSourcesByPackage.find(root->string()); dit != deviceSourcesByPackage.end()) bpEnv.deviceSources = dit->second; // Set explicitly rather than relying on build_dir()'s root-relative @@ -10102,6 +10107,12 @@ prepare_build(bool print_fingerprint, for (auto& x : a.inputs) x = substitute(x); for (auto& x : a.outputs) x = substitute(x); for (auto& x : a.command) x = substitute(x); + // Same closed vocabulary as outputs — a depfile commonly + // wants to live at `${mcpp.out_dir}/.d`, beside the + // output it describes, and `prepare_actions` above + // deliberately left a `${mcpp.` depfile untouched for + // exactly this phase to resolve. + if (!a.depfile.empty()) a.depfile = substitute(a.depfile); a.packageName = owner; ctx.plan.actions.push_back(std::move(a)); } diff --git a/tests/e2e/631_action_depfile_tracks_a_runtime_discovered_include.sh b/tests/e2e/631_action_depfile_tracks_a_runtime_discovered_include.sh new file mode 100755 index 000000000..f26d66ca9 --- /dev/null +++ b/tests/e2e/631_action_depfile_tracks_a_runtime_discovered_include.sh @@ -0,0 +1,147 @@ +#!/usr/bin/env bash +# requires: gcc +# 631_action_depfile_tracks_a_runtime_discovered_include.sh — `mcpp::action`'s +# `depfile` field: a rule that discovers its own dependency graph by RUNNING +# (glslangValidator `--depfile`, glslc `-MD -MF`, slangc `-depfile`, nvcc/clang +# `-MD -MF`) has a way to tell ninja about it. +# +# THE DEFECT THIS DEFENDS AGAINST. `a.inputs` is fixed when build.mcpp runs, +# before the action's own command has executed. A shader/device compiler does +# not know its `#include` graph at that point — it learns it by PARSING the +# source, and reports the result afterward as a Make-style depfile. Without +# this field there is no channel for that report to reach ninja: the action's +# edge tracks exactly the files build.mcpp named, so editing a file the +# command merely READ (never a declared `.input()`) reruns nothing, and +# `mcpp build` stays green over a stale generated artifact. +# +# THE SCENARIO. The action's command is a stand-in generator: it writes its +# declared output AND a depfile naming a second file — `included.glsl` — that +# the action never declares as an input. Touching that second file (a bare +# `touch`, no content edit — exactly what a real `#include`'s mtime changing +# looks like) must make ninja rerun the action on the next build. +# +# ASSERTED ON A RUN COUNTER gen.sh writes as a side effect, NOT on grepping +# `mcpp build`'s own log for the ninja description text ("GENERATE ..."). That +# text cannot appear there: on a non-verbose SUCCESSFUL build mcpp passes +# ninja `--quiet` and only surfaces its captured stdout when the build FAILS +# or `--verbose` is given (src/build/ninja_backend.cppm, src/build/execute.cppm) +# — so a log grep for it on success is vacuous, true whether or not the edge +# ran. Verified directly against this fixture: even the very first, from- +# scratch build (which unquestionably runs the action) prints no such line. +# The run counter is a real side effect of the COMMAND executing, so reading +# it back is a direct measurement instead of a guess about log formatting. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +mkdir -p "$TMP/shaderdep/src" +cd "$TMP/shaderdep" + +cat > mcpp.toml <<'EOF' +[package] +name = "shaderdep" +version = "0.1.0" +EOF + +cat > src/main.cpp <<'EOF' +#include +int generated_value(); +int main() { std::printf("VALUE=%d\n", generated_value()); } +EOF + +# The file the ACTION discovers only by "compiling" — analogous to a shader's +# `#include`. Its CONTENT is never read below; only its mtime matters, which +# is exactly what a depfile-tracked dependency promises to react to. +echo "float unused = 1.0;" > included.glsl + +# The stand-in device compiler. $1 = the discovered file, $2 = the source it +# generates, $3 = the depfile to write, $4 = a run counter so the test can +# tell the command executed again versus ninja deciding it was up to date. +# Deliberately NOT `set -e`: the `[ -f ] &&` idiom below is a normal false +# branch, not an error. +cat > gen.sh <<'EOF' +#!/usr/bin/env bash +n=0 +[ -f "$4" ] && n="$(cat "$4")" +n=$((n + 1)) +echo "$n" > "$4" +printf 'int generated_value() { return %s; }\n' "$n" > "$2" +# Make syntax, the shape `-MD -MF`/`--depfile` produce: OUTPUT: PREREQUISITES. +printf '%s: %s\n' "$2" "$1" > "$3" +EOF +chmod +x gen.sh + +cat > build.mcpp <<'EOF' +#include +#include +import mcpp; +int main() { + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/generated.cpp"; + const std::string dep = std::string(mcpp::out_dir()) + "/generated.cpp.d"; + const std::string count = root + "/run_count.txt"; + + mcpp::action a; + a.id = "genshader"; + a.role = "source"; + a.depfile = dep.c_str(); + // Deliberately NOT declared as `.input(...)`: the whole point is that + // ninja learns about it from the depfile the command writes, not from + // anything build.mcpp told the engine in advance. + a.arg((root + "/gen.sh").c_str()) + .arg((root + "/included.glsl").c_str()) + .arg(out.c_str()) + .arg(dep.c_str()) + .arg(count.c_str()) + .output(out.c_str()) + .submit(); +} +EOF + +run_count() { cat run_count.txt 2>/dev/null || echo ""; } + +# ── 1. first build: the action runs once, VALUE reflects run #1 ──────────── +"$MCPP" build > b1.log 2>&1 || { cat b1.log; echo "FAIL: initial build failed"; exit 1; } +[[ "$(run_count)" == "1" ]] || { + cat b1.log; echo "FAIL: the action did not run on a fresh build (count=$(run_count))"; exit 1; } +out="$("$MCPP" run 2>&1 | grep '^VALUE=' | tail -1)" +[[ "$out" == "VALUE=1" ]] || { echo "FAIL: unexpected initial output: $out"; exit 1; } + +# ── 2. THE CONTROL: rebuild with nothing changed must not rerun the action ── +# Without this half, the assertion in part 3 would pass against a backend +# that reruns the action on every build regardless of the depfile. +"$MCPP" build > b2.log 2>&1 || { cat b2.log; echo "FAIL: no-op rebuild failed"; exit 1; } +[[ "$(run_count)" == "1" ]] || { + cat b2.log + echo "FAIL: the action reran although nothing it tracks changed (count=$(run_count))" + exit 1; } + +# ── 3. touch the file named ONLY in the depfile — never a declared input ─── +touch included.glsl +"$MCPP" build > b3.log 2>&1 || { cat b3.log; echo "FAIL: rebuild after touch failed"; exit 1; } +[[ "$(run_count)" == "2" ]] || { + cat b3.log + echo "FAIL: touching a file the depfile named did not rerun the action" + echo " (count=$(run_count), expected 2)" + echo " this is the defect: a.inputs alone cannot express this" + echo " dependency, and without depfile support ninja never learns it" + exit 1; } +out="$("$MCPP" run 2>&1 | grep '^VALUE=' | tail -1)" +[[ "$out" == "VALUE=2" ]] || { + echo "FAIL: the action reran but the artifact was not regenerated: $out" + echo " (expected VALUE=2)" + exit 1; } + +# ── 4. and it behaves once more, so #3 was not a first-build artifact ────── +touch included.glsl +"$MCPP" build > b4.log 2>&1 || { cat b4.log; echo "FAIL: second touch rebuild failed"; exit 1; } +[[ "$(run_count)" == "3" ]] || { + cat b4.log + echo "FAIL: the second touch did not rerun the action (count=$(run_count))" + exit 1; } +out="$("$MCPP" run 2>&1 | grep '^VALUE=' | tail -1)" +[[ "$out" == "VALUE=3" ]] || { + echo "FAIL: unexpected output after the second touch: $out (expected VALUE=3)" + exit 1; } + +echo "OK" diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index a88c1a065..3943d9488 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -640,3 +640,42 @@ TEST(BuildDirectives, RunnerLandsInBuildConfigNotInLdflags) { for (auto const& f : m.buildConfig.ldflags) EXPECT_EQ(f.find("qemu"), std::string::npos) << f; } + +// ── Build-graph node decoding: `mcpp:action=` ─────────────────────────────── +// +// `decode_action` is the other half of `mcpp::action::submit()` in +// src/build/hostprogram.cppm: the typed builder serialises one JSON object +// per action, and this is where it comes back. `depfile` is decoded the same +// way `blocking` is — a scalar read with `j.value(...)` and a default that +// matches "the field was never set" — so an action a build program submitted +// before this field existed decodes exactly as it always has. + +TEST(BuildDirectives, DecodeActionRoundTripsDepfile) { + auto d = parse( + "mcpp:action={\"id\":\"shader\",\"role\":\"source\"," + "\"description\":\"\",\"blocking\":false,\"depfile\":\"out/shader.spv.d\"," + "\"inputs\":[],\"outputs\":[\"out/shader.spv\"]," + "\"command\":[\"glslc.sh\"],\"provides\":[],\"imports\":[],\"targets\":[]}\n"); + + ASSERT_EQ(d.at(dirs::Slot::Actions).size(), 1u); + auto a = dirs::decode_action(d.at(dirs::Slot::Actions).front()); + ASSERT_TRUE(a.has_value()); + EXPECT_EQ(a->depfile, "out/shader.spv.d"); +} + +// THE CONTROL — and the common case, since `depfile` is optional and +// `hostprogram.cppm`'s `submit()` omits the key entirely when it was never +// set (byte-identical payload to before the field existed). A decoder that +// required the key would reject every action submitted by that path. +TEST(BuildDirectives, DecodeActionDefaultsDepfileToEmptyWhenAbsent) { + auto d = parse( + "mcpp:action={\"id\":\"gen\",\"role\":\"source\"," + "\"description\":\"\",\"blocking\":false," + "\"inputs\":[],\"outputs\":[\"out/gen.cpp\"]," + "\"command\":[\"gen.sh\"],\"provides\":[],\"imports\":[],\"targets\":[]}\n"); + + ASSERT_EQ(d.at(dirs::Slot::Actions).size(), 1u); + auto a = dirs::decode_action(d.at(dirs::Slot::Actions).front()); + ASSERT_TRUE(a.has_value()); + EXPECT_EQ(a->depfile, ""); +} diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index d86e2944b..dd4921c88 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -1829,3 +1829,93 @@ TEST(ActionOrdering, BlockingDecidesWhetherACheckGatesTheCompile) { EXPECT_EQ(make(false).find("mcpp-actions-chk_pkg"), std::string::npos) << "blocking = false gated the compile anyway"; } + +// ── depfile: ninja must track what the action's COMMAND read, not only what +// the build program declared as an input ────────────────────────────────── +// +// `inputs` is fixed when build.mcpp runs, before the command has executed. A +// device-source compiler (glslangValidator, glslc, slangc, nvcc/clang) +// discovers its own `#include` graph only by parsing the source, and reports +// it afterward as a Make-style depfile. Without `depfile`/`deps = gcc` on the +// action's own rule, that graph reaches nothing: the compile edge tracks +// exactly the files build.mcpp named, and editing an `#include`d `.glsl` or +// `.cuh` reruns nothing. + +namespace { + +// Isolates ONE rule's own variable block (`rule NAME` up to the next blank +// line), because `cxx_object`/`cxx_module` unconditionally carry their own +// `depfile = $out.d` / `deps = gcc` for a plan's ordinary compile rules +// (mcpp#235/#257) — a search over the WHOLE manifest for either string would +// pass whether or not the action under test ever asked for a depfile. +std::string action_rule_block(std::string_view ninja, std::string_view rule_header) { + auto start = ninja.find(rule_header); + if (start == std::string_view::npos) return {}; + auto end = ninja.find("\n\n", start); + auto count = end == std::string_view::npos ? std::string_view::npos : end - start; + return std::string(ninja.substr(start, count)); +} + +} // namespace + +TEST(ActionDepfile, ADeclaredDepfileEmitsDepfileAndDepsGcc) { + auto plan = minimal_plan(); + plan.compileUnits.push_back({ + .source = "src/shader_user.cpp", + .kind = mcpp::SourceKind::Cxx, + .object = "obj/shader_user.o", + .packageName = "shader_pkg", + }); + mcpp::manifest::BuildAction a; + a.id = "shader"; + a.packageName = "shader_pkg"; + a.role = mcpp::manifest::BuildAction::Role::Source; + a.command = {"/bin/true"}; + a.outputs = {"out/shader.spv"}; + a.depfile = "out/shader.spv.d"; + plan.actions.push_back(std::move(a)); + + auto ninja = emit_ninja_string(plan); + auto block = action_rule_block(ninja, "rule mcpp_action_0\n"); + ASSERT_FALSE(block.empty()) << "no rule emitted for the action\n" << ninja; + EXPECT_NE(block.find("\n depfile = out/shader.spv.d\n"), std::string::npos) + << block; + // No trailing "\n" required: `deps = gcc` is the last line this backend + // emits for the rule before the blank separator, so the block extracted + // above (up to, not including, that separator) ends exactly here. + EXPECT_NE(block.find("\n deps = gcc"), std::string::npos) << block; + + // The depfile must never also be a declared ninja OUTPUT of the same + // edge: `deps = gcc` makes ninja consume and delete it once read, and a + // file simultaneously promised as an output would be one ninja expects + // to still exist afterward. + EXPECT_EQ(ninja.find("shader.spv.d :"), std::string::npos) + << "the depfile must not appear as a build edge's own output\n" << ninja; +} + +// THE CONTROL. Without it, a backend that emitted `depfile =`/`deps = gcc` +// unconditionally for every action would also pass the test above — exactly +// the shape `blocking` took before anything read it (mcpp#534 / e2e 315): +// typed, transported, parsed, and a no-op with a paper trail. +TEST(ActionDepfile, NoDepfileEmitsNeitherLine) { + auto plan = minimal_plan(); + plan.compileUnits.push_back({ + .source = "src/plain_user.cpp", + .kind = mcpp::SourceKind::Cxx, + .object = "obj/plain_user.o", + .packageName = "plain_pkg2", + }); + mcpp::manifest::BuildAction a; + a.id = "plain"; + a.packageName = "plain_pkg2"; + a.role = mcpp::manifest::BuildAction::Role::Source; + a.command = {"/bin/true"}; + a.outputs = {"out/plain.txt"}; + plan.actions.push_back(std::move(a)); + + auto ninja = emit_ninja_string(plan); + auto block = action_rule_block(ninja, "rule mcpp_action_0\n"); + ASSERT_FALSE(block.empty()) << "no rule emitted for the action\n" << ninja; + EXPECT_EQ(block.find("depfile ="), std::string::npos) << block; + EXPECT_EQ(block.find("deps = gcc"), std::string::npos) << block; +} From 235449a7101c713567d81b4e1cfe1fc6d64c46b2 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 22:39:11 +0800 Subject: [PATCH 03/15] tests/e2e/188: the "did not rerun" assertion could not fail It read `grep -q "GENERATE" b3.log && FAIL`, on the assumption that ninja's description line for the edge reaches that log. It does not. `mcpp build` passes `--quiet` to ninja whenever it is not `--verbose` and surfaces ninja's captured stdout only on failure, so a from-scratch build -- which unquestionably runs the action -- prints no such line either. The grep never matched, and the check could not fail whether the action reran or not. Found while designing the depfile test, whose first draft copied this convention and would have inherited the same emptiness. The generated file's modification time measures the thing being asserted directly: the action rewrites it whenever it runs. Verified in both directions against the same binary -- unchanged after touching an unrelated source, and the assertion fails naming both timestamps when the touched file is a declared input of the action instead. --- tests/e2e/188_build_actions.sh | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/tests/e2e/188_build_actions.sh b/tests/e2e/188_build_actions.sh index b5fee3515..ddace435a 100755 --- a/tests/e2e/188_build_actions.sh +++ b/tests/e2e/188_build_actions.sh @@ -122,10 +122,29 @@ out="$("$MCPP" run 2>&1 | grep '^VALUE=' | tail -1)" cat b2.log; echo "FAIL: action did not re-run when its input changed: $out"; exit 1; } # ...and an unrelated rebuild must NOT re-run it (that is the whole point). +# +# THIS ASSERTION USED TO BE VACUOUS, AND THAT IS WORTH THE COMMENT. It read +# `grep -q "GENERATE" b3.log && FAIL`, on the assumption that ninja's +# description line for the edge reaches this log. It does not: `mcpp build` +# passes `--quiet` to ninja whenever it is not `--verbose`, and surfaces +# ninja's captured stdout only on failure. A from-scratch build, which +# unquestionably runs the action, prints no such line either -- so the grep +# never matched and the check could not fail whether the action reran or not. +# +# The generated file's modification time is a direct measurement of the thing +# being asserted: the action rewrites it whenever it runs. +before=$(stat -c %Y "$(find target -name gen.cpp -print -quit)" 2>/dev/null \ + || stat -f %m "$(find target -name gen.cpp -print -quit)") touch src/main.cpp +sleep 1 # coarser than any filesystem's mtime granularity here "$MCPP" build > b3.log 2>&1 || { cat b3.log; echo "FAIL: rebuild failed"; exit 1; } -grep -q "GENERATE" b3.log && { - cat b3.log; echo "FAIL: the action re-ran although its inputs were unchanged"; exit 1; } +after=$(stat -c %Y "$(find target -name gen.cpp -print -quit)" 2>/dev/null \ + || stat -f %m "$(find target -name gen.cpp -print -quit)") +[[ -n "$before" && "$before" == "$after" ]] || { + cat b3.log + echo "FAIL: the action re-ran although its inputs were unchanged" + echo " (gen.cpp mtime $before -> $after)" + exit 1; } # ── 2b. changing the action's COMMAND also takes effect ──────────────────── # Distinct from 2: there the action's declared INPUT changed and ninja noticed. From 1a3fa439dd70475f07bbe11b941fcd67617753d4 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 22:48:48 +0800 Subject: [PATCH 04/15] examples/10-graphics: the offscreen example runs on all three platforms macOS and Windows built it and did not run it, which is the shape the cross-platform work exists to remove: the half of a lane written for a host is the half that host never exercises. A build asserts that the shader compiler this platform publishes works and that the Vulkan half links. It cannot assert that the loader hands the program a device, and on macOS that is the interesting half. The example now declares a device for those two platforms as well -- `xim:moltenvk` under `cfg(macos)`, `xim:mesa-lavapipe` under `cfg(windows)` -- rather than leaving CI to install one. The omission was invisible while the example was never run, because a program that does not run never asks the loader for a device, and declaring it here is what makes the example complete for anyone who checks it out rather than only for the runner that had an extra command. The two CI steps therefore install nothing. They locate the ICD the build already provisioned and set `VK_DRIVER_FILES`, so a manifest that failed to name the driver fails the step. An `xlings install` in the step would have made it pass either way. macOS and Windows assert different things, and the difference is the point. lavapipe is a software rasteriser producing the same pixels by construction, so the image cannot distinguish it and the device name is what does. MoltenVK is the host's own GPU through Metal, so its name differs by runner; what distinguishes reaching a device there is that a centre pixel was reported at all, which a portability driver the loader declined to show would not produce. This closes the dependency between the two halves of the portability change: the package without the enumeration finds no device, and the enumeration without the package has nothing to find. --- .github/workflows/ci-macos.yml | 48 ++++++++++++++++++++++++ .github/workflows/ci-windows.yml | 39 +++++++++++++++++++ CHANGELOG.md | 26 +++++++++++++ examples/10-graphics/offscreen/mcpp.toml | 23 ++++++++++++ 4 files changed, 136 insertions(+) diff --git a/.github/workflows/ci-macos.yml b/.github/workflows/ci-macos.yml index 35513b78e..4f3f9a7ad 100644 --- a/.github/workflows/ci-macos.yml +++ b/.github/workflows/ci-macos.yml @@ -361,6 +361,54 @@ jobs: done echo "ok: both shader stages compiled and the Vulkan half linked" + # AND NOW IT RUNS, WHICH IS A DIFFERENT CLAIM FROM THE ONE ABOVE. + # + # A build asserts that the shader compiler this platform publishes works + # and that the Vulkan half links. It cannot assert that the loader hands + # this program a device, and on macOS that is the interesting half: + # MoltenVK is a PORTABILITY driver, which the loader does not give to + # `vkEnumeratePhysicalDevices` unless the instance asked for portability + # enumeration. A program written against native drivers therefore finds no + # device here and reports it as "this machine has no GPU". + # + # This step is what turns that into a red build rather than a plausible + # message. It depends on the portability enumeration in this PR: without + # it the run reaches zero devices even though everything installed + # correctly. + # + # `VK_DRIVER_FILES` rather than an ICD search directory: the package is in + # the xlings store, not in `/usr/local/share/vulkan/icd.d`, and naming the + # file is the one form that does not depend on where the loader looks. + - name: "Graphics: the offscreen example RUNS on MoltenVK" + shell: bash + run: | + set -e + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + # NOT installed from here. The example declares `xim:moltenvk` under + # `cfg(macos)`, so the build above already provisioned it, and looking + # for the ICD without installing anything is what asserts that the + # declaration works. An `xlings install` here would make this step + # pass whether the manifest named the driver or not. + icd=$(find "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-moltenvk" \ + "$HOME/.xlings/data/xpkgs/xim-x-moltenvk" \ + -name 'MoltenVK_icd.json' -print -quit 2>/dev/null || true) + [ -n "$icd" ] || { echo "no MoltenVK ICD in either store"; exit 1; } + echo "ICD: $icd" + cd "$GITHUB_WORKSPACE/examples/10-graphics/offscreen" + out=$(VK_DRIVER_FILES="$icd" "/tmp/mcpp-fresh" run 2>&1) || { echo "$out"; exit 1; } + echo "$out" + # The program asserts the corners and the centre itself and exits + # non-zero on either. What CI adds is that the run reached a DEVICE: + # a portability driver that the loader declined to show would leave + # the program reporting no device, which is the failure this step + # exists for. The name is not compared to a fixed string because it is + # the host's GPU and differs by runner; that it is non-empty and the + # centre pixel is opaque is what distinguishes reaching a device from + # not. + echo "$out" | grep -qE 'centre pixel: \([0-9]+, [0-9]+, [0-9]+, 255\)' \ + || { echo "no centre pixel was reported: the run reached no device"; exit 1; } + echo "ok: MoltenVK enumerated and the image was rendered on it" + # Integration: the mcpp built from THIS PR's source (the self-host binary, # $MCPP = /tmp/mcpp-fresh) builds & runs a real external C++ project — # xlings (openxlings/xlings ships its own mcpp.toml). diff --git a/.github/workflows/ci-windows.yml b/.github/workflows/ci-windows.yml index b362d2333..4cbd4ea93 100644 --- a/.github/workflows/ci-windows.yml +++ b/.github/workflows/ci-windows.yml @@ -456,6 +456,45 @@ jobs: done echo "ok: both shader stages compiled and the Vulkan half linked" + # AND NOW IT RUNS. `xim:mesa-lavapipe` publishes a Windows build, so this + # host has a software device and the claim can be the same one Linux + # makes: the program reached a device and rendered the image, not merely + # that it compiled. + # + # The device name IS compared here, unlike on macOS. lavapipe is a + # software rasteriser that produces the same pixels by construction, so + # the image cannot distinguish it from any other implementation and the + # name is what does. + # + # `VK_DRIVER_FILES` rather than an ICD search directory: the package is in + # the xlings store rather than in the registry the Windows loader reads, + # and naming the file is the one form that does not depend on that. + - name: "Graphics: the offscreen example RUNS on lavapipe" + shell: bash + run: | + set -e + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + # NOT installed from here, for the reason the macOS job records: the + # example declares `xim:mesa-lavapipe` under `cfg(windows)`, so + # finding the ICD without installing anything is what asserts the + # declaration works. + icd=$(find "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-mesa-lavapipe" \ + "$HOME/.xlings/data/xpkgs/xim-x-mesa-lavapipe" \ + -name '*.json' -path '*icd.d*' -print -quit 2>/dev/null || true) + [ -n "$icd" ] || { echo "no lavapipe ICD in either store"; exit 1; } + echo "ICD: $icd" + cd "$GITHUB_WORKSPACE/examples/10-graphics/offscreen" + # `$MCPP_SELF`, which is what the build step above used. `/tmp/mcpp-fresh.exe` + # is a copy a LATER step makes, so naming it here would look right and + # fail with a missing file. + out=$(VK_DRIVER_FILES="$icd" "$MCPP_SELF" run 2>&1) || { echo "$out"; exit 1; } + echo "$out" + echo "$out" | grep -q 'llvmpipe' \ + || { echo "the run did not reach the lavapipe device"; exit 1; } + echo "$out" | grep -qE 'centre pixel: \([0-9]+, [0-9]+, [0-9]+, 255\)' \ + || { echo "no centre pixel was reported"; exit 1; } + echo "ok: lavapipe enumerated and the image was rendered on it" + - name: "Toolchain: LLVM — build mcpp (self-host)" shell: bash run: | diff --git a/CHANGELOG.md b/CHANGELOG.md index 1483fc262..b947c1b1d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -86,6 +86,32 @@ libmalloc 里 abort(#202)。CI 报的正是这条路的第一步:链接停在 `_ `cfg(accelerator = ...)` 下的 `[build]` 源生效而依赖被忽略,于是包被丢掉、包含它的源 被留下。 +### 可移植性驱动默认是看不见的,于是三个平台都从「构建」抬到「运行」 + +macOS 上没有原生 Vulkan,MoltenVK 是 Metal 之上的实现,规范把这种实现叫**可移植性驱动**。 +loader 默认不把它交给 `vkEnumeratePhysicalDevices`:实例要同时启用 +`VK_KHR_portability_enumeration` 并置位 `VK_INSTANCE_CREATE_ENUMERATE_PORTABILITY_BIT_KHR`, +随后凡是声明 `VK_KHR_portability_subset` 的设备必须在 `vkCreateDevice` 时启用它。一个照着 +原生驱动写的程序因此在那台机器上**一个设备都找不到**,并把它报成「这台机器没有 GPU」—— +诊断是错的,而它看起来完全合理。 + +按能力问,不按 `#ifdef __APPLE__`:性质是「我面前这个 loader 在展示可移植性驱动」,而 +Linux 上跑翻译层的机器也有它,macOS 上对着原生驱动构建的程序并不需要它。Linux/lavapipe +上读数不变,中心像素仍是 `(124, 70, 62, 255)`,设备名仍是 `llvmpipe`。 + +**示例本身补齐了另外两个平台的设备声明** —— `cfg(macos)` 下 `xim:moltenvk`,`cfg(windows)` +下 `xim:mesa-lavapipe`。这个缺口在示例只构建不运行时是看不见的:不运行的程序从不向 +loader 要设备。声明写在示例里而不是 CI 步骤里,是为了让**任何人**检出它都能跑,而不只是 +那台多跑了一条命令的 runner。 + +于是两个新 CI 步骤**什么都不装**:它们找构建已经供给的 ICD 并设 `VK_DRIVER_FILES`,所以 +一份没能声明驱动的 manifest 会让步骤变红。步骤里放 `xlings install` 会让它两种情况都通过。 + +两个平台断言的东西不同,而这个差别正是重点。lavapipe 是软件光栅化器,像素由构造保证相同, +所以图像分不出它,设备名才分得出。MoltenVK 是宿主自己的 GPU 经 Metal,名字随 runner 变, +所以那边区分「够到了设备」的是**报出了中心像素**这件事本身 —— 一个被 loader 拒绝展示的 +可移植性驱动不会产生它。 + ### 文档 `docs/20` 新增「每条 lane 到得了哪些平台」:三件事同时为真才叫一条 lane 在某个平台上 diff --git a/examples/10-graphics/offscreen/mcpp.toml b/examples/10-graphics/offscreen/mcpp.toml index 39d953f5a..08cea75f6 100644 --- a/examples/10-graphics/offscreen/mcpp.toml +++ b/examples/10-graphics/offscreen/mcpp.toml @@ -44,6 +44,29 @@ vulkan-runtime = "2026.09.07" [target.'cfg(linux)'.xlings.workspace] "xim:mesa-lavapipe" = "26.2.1" +# THE OTHER TWO PLATFORMS GET A DEVICE TOO, AND FOR THE SAME REASON. +# +# These were absent while the example was built but not run on macOS and +# Windows, which made the omission invisible: a program that never runs never +# asks the loader for a device. Declaring the driver here rather than installing +# it from a CI step is what makes the EXAMPLE complete -- anyone who checks it +# out gets a machine that can run it, not only the runner that had an extra +# command. +# +# macOS has no native Vulkan. MoltenVK is an implementation on top of Metal, and +# the specification calls that a portability driver: the loader does not hand it +# to `vkEnumeratePhysicalDevices` unless the instance asked for portability +# enumeration, which `src/vulkan/render.cpp` now does. The two halves are one +# change -- the package without the enumeration finds no device, and the +# enumeration without the package has nothing to find. +[target.'cfg(macos)'.xlings.workspace] +"xim:moltenvk" = "1.4.2" + +# Windows takes the same software rasteriser Linux does; upstream publishes a +# separate build of it, one minor behind the Linux one. +[target.'cfg(windows)'.xlings.workspace] +"xim:mesa-lavapipe" = "26.2.0" + [build] accel = "vulkan1.2" sources = [ From 90be6a9f8fafa0949566fd72d20d57e1706020bf Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 23:14:16 +0800 Subject: [PATCH 05/15] examples/10-graphics: Windows stays at "builds", and the reason is a missing package A run step was written, pushed and measured. The manifest declaration worked -- `Provisioning [xlings.workspace] entries (xim:mesa-lavapipe@26.2.0)` -- the ICD was found in the store, and the program still printed `render unavailable`, which is its own report that it enumerated no device. The missing piece is the LOADER, not the driver. `compat:vulkan` ships an import library on Windows and nothing else, and its own descriptor says why: a statically linked loader cannot work there, because upstream's `loader_windows.c` creates its locks in `DllMain` and a static library never gets one. The runtime `vulkan-1.dll` is expected to come from an installed GPU driver, and a GitHub Windows runner has none. `xim:mesa-lavapipe`'s Windows payload is an ICD that imports system DLLs and carries no loader either. So the example declares no device on Windows: an entry there would download 56 MB that nothing can load. macOS is not in the same position -- `compat:vulkan` builds the loader from source everywhere except Windows -- and keeps its run step. Raising this platform to "runs" needs a Windows `vulkan-1.dll` package, which is a packaging decision rather than a step in a workflow file. Both the workflow and the example manifest record the measurement where someone looking for the gap will find it. --- .github/workflows/ci-windows.yml | 56 +++++++++--------------- examples/10-graphics/offscreen/mcpp.toml | 16 +++++-- 2 files changed, 32 insertions(+), 40 deletions(-) diff --git a/.github/workflows/ci-windows.yml b/.github/workflows/ci-windows.yml index 4cbd4ea93..ad97d641c 100644 --- a/.github/workflows/ci-windows.yml +++ b/.github/workflows/ci-windows.yml @@ -456,44 +456,28 @@ jobs: done echo "ok: both shader stages compiled and the Vulkan half linked" - # AND NOW IT RUNS. `xim:mesa-lavapipe` publishes a Windows build, so this - # host has a software device and the claim can be the same one Linux - # makes: the program reached a device and rendered the image, not merely - # that it compiled. + # WINDOWS STAYS AT "BUILDS", AND THE REASON IS A MISSING PACKAGE RATHER + # THAN A MISSING CI STEP. # - # The device name IS compared here, unlike on macOS. lavapipe is a - # software rasteriser that produces the same pixels by construction, so - # the image cannot distinguish it from any other implementation and the - # name is what does. + # A run step was written, pushed, and measured. The manifest declaration + # worked -- `Provisioning [xlings.workspace] entries + # (xim:mesa-lavapipe@26.2.0)` -- and the ICD was found in the store, and + # the program still printed `render unavailable`, which is its own report + # that it enumerated no device. # - # `VK_DRIVER_FILES` rather than an ICD search directory: the package is in - # the xlings store rather than in the registry the Windows loader reads, - # and naming the file is the one form that does not depend on that. - - name: "Graphics: the offscreen example RUNS on lavapipe" - shell: bash - run: | - set -e - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - # NOT installed from here, for the reason the macOS job records: the - # example declares `xim:mesa-lavapipe` under `cfg(windows)`, so - # finding the ICD without installing anything is what asserts the - # declaration works. - icd=$(find "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-mesa-lavapipe" \ - "$HOME/.xlings/data/xpkgs/xim-x-mesa-lavapipe" \ - -name '*.json' -path '*icd.d*' -print -quit 2>/dev/null || true) - [ -n "$icd" ] || { echo "no lavapipe ICD in either store"; exit 1; } - echo "ICD: $icd" - cd "$GITHUB_WORKSPACE/examples/10-graphics/offscreen" - # `$MCPP_SELF`, which is what the build step above used. `/tmp/mcpp-fresh.exe` - # is a copy a LATER step makes, so naming it here would look right and - # fail with a missing file. - out=$(VK_DRIVER_FILES="$icd" "$MCPP_SELF" run 2>&1) || { echo "$out"; exit 1; } - echo "$out" - echo "$out" | grep -q 'llvmpipe' \ - || { echo "the run did not reach the lavapipe device"; exit 1; } - echo "$out" | grep -qE 'centre pixel: \([0-9]+, [0-9]+, [0-9]+, 255\)' \ - || { echo "no centre pixel was reported"; exit 1; } - echo "ok: lavapipe enumerated and the image was rendered on it" + # The missing piece is the LOADER, not the driver. `compat:vulkan` ships + # an import library on Windows and nothing else, and says why in its own + # descriptor: a statically linked loader cannot work there, because + # upstream's `loader_windows.c` creates its locks in `DllMain` and a + # static library never gets one. The runtime `vulkan-1.dll` is expected + # to come from an installed GPU driver, and a GitHub Windows runner has + # none. `xim:mesa-lavapipe`'s Windows payload is an ICD; it imports system + # DLLs and carries no loader either. + # + # So raising this job to "runs" needs a Windows `vulkan-1.dll` package, + # which is a packaging decision rather than a step in this file. macOS is + # not in the same position: `compat:vulkan` builds the loader from source + # everywhere except Windows, so that job does run. - name: "Toolchain: LLVM — build mcpp (self-host)" shell: bash diff --git a/examples/10-graphics/offscreen/mcpp.toml b/examples/10-graphics/offscreen/mcpp.toml index 08cea75f6..76b80eded 100644 --- a/examples/10-graphics/offscreen/mcpp.toml +++ b/examples/10-graphics/offscreen/mcpp.toml @@ -62,10 +62,18 @@ vulkan-runtime = "2026.09.07" [target.'cfg(macos)'.xlings.workspace] "xim:moltenvk" = "1.4.2" -# Windows takes the same software rasteriser Linux does; upstream publishes a -# separate build of it, one minor behind the Linux one. -[target.'cfg(windows)'.xlings.workspace] -"xim:mesa-lavapipe" = "26.2.0" +# WINDOWS DECLARES NO DEVICE, AND THAT IS MEASURED RATHER THAN AN OVERSIGHT. +# +# `xim:mesa-lavapipe` publishes a Windows payload and it installs correctly, so +# an entry here would work in the sense that the bytes arrive. The program still +# reaches no device: `compat:vulkan` ships an import library on Windows and the +# runtime `vulkan-1.dll` is expected to come from an installed GPU driver, which +# a machine without one does not have. Declaring the driver would download 56 MB +# that nothing can load. +# +# The gap is a Windows loader package. Until there is one, this platform builds +# the Vulkan half and runs the CPU fallback, which is what the `cfg(not(...))` +# section below carries. [build] accel = "vulkan1.2" From c0225bb3f9856c6b0d4f131923ed909ba7206a08 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 23:41:27 +0800 Subject: [PATCH 06/15] A rule package declares what it compiles, so a new device language costs no engine change Two keys on a feature, and the engine holds no package name, no feature spelling and no module name: [features.rules-slang] sources = ["rules/slang.cppm"] rule_module = "mcpp.rules.slang" device_extensions = [".slang"] `device_extensions` classifies those extensions as device sources in a consumer that activates the feature. `rule_module` is what a build program imports to reach the rule. Two things follow. A NEW DEVICE LANGUAGE NO LONGER TOUCHES THE ENGINE. Adding `.slang` to the built-in table cost an engine change, a release, and a version bump in the rule package's CI before the rule could route one file. `.slang` is removed from that table here and `rules-slang` declares it instead; `tests/slang-consumer` builds and runs unchanged, which is the only honest test of whether the mechanism carries a language. The built-in list is now what mcpp knows without being told -- a compatibility set for languages whose support shipped before the declaration existed -- rather than a registry a sixth backend joins. A CONSUMER WRITES ONE EDGE AND NO BUILD PROGRAM. `host-module = true` is implied by `rule_module`, because a feature naming one has already said that is the only way to use it. And a package with no `build.mcpp` gets the program its rules describe written into the build directory: [build-dependencies.mcpp] plugins = { version = "0.3.0", features = ["rules-spirv"] } is the whole declaration in `tests/spirv-zero-config`, which compiles a shader and reaches it through a generated module. A project that writes its own `build.mcpp` keeps it: synthesis fills an absence and never overrides. THE FEATURE IS STILL REQUESTED BY NAME. An earlier revision derived it from the extensions a project's sources carried, so a consumer could name the package alone. That was withdrawn for two reasons and neither was cost. Two packages may claim one extension -- a third-party CUDA rule is a thing someone will write -- and derivation would then guess or refuse where `features = [...]` has already said which. And a manifest's job is to describe the build: a derived feature set is information the file no longer states, which is worse for a reader and worse for anything reading the manifest as context. Which rules ran is said out loud, for the same reason the resolved toolchain is: Rules mcpp.rules.spirv (mcpp:plugins) Two ordering defects were found by running it rather than by reading it. The collection must sit between feature activation and the extension table that narrows the constrained globs; placed after, the declared extensions arrived too late to classify anything and the rule was handed an empty list. And the guard on the build-program call asked whether the FILE existed while the function asked whether a program was WANTED, which left the synthesis unreachable. Both now ask the same question, and the comments say what the failure looked like. --- modules/manifest/src/toml.cppm | 34 +++++++ modules/manifest/src/types.cppm | 62 ++++++++++++ modules/source-kind/src/source_kind.cppm | 81 ++++++++++++--- src/build/build_program.cppm | 68 ++++++++++++- src/build/execute.cppm | 3 +- src/build/plan.cppm | 3 +- src/build/prepare.cppm | 119 +++++++++++++++++++++-- src/modgraph/scanner.cppm | 6 +- 8 files changed, 353 insertions(+), 23 deletions(-) diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index a292d06ab..bc0d58f3a 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -758,6 +758,40 @@ std::expected parse_string(std::string_view content, read_str_array(ft, "provides", provs); if (!reqs.empty()) m.featureRequires[fname] = std::move(reqs); if (!provs.empty()) m.featureProvides[fname] = std::move(provs); + // The device extensions this feature's rule compiles. Normalised + // the same way `module_extensions` is, so `comp` and `.comp` are + // one entry and a consumer cannot be surprised by a missing dot. + std::vector devExts; + read_str_array(ft, "device_extensions", devExts); + if (!devExts.empty()) { + for (auto& e : devExts) e = mcpp::normalize_extension(e); + std::erase(devExts, std::string{}); + if (!devExts.empty()) + m.featureDeviceExtensions[fname] = std::move(devExts); + } + // The module a consumer's build program imports for this rule. + if (auto it = ft.find("rule_module"); + it != ft.end() && it->second.is_string()) + m.featureRuleModule[fname] = it->second.as_string(); + // The two halves of "this feature is a build rule" must arrive + // together. One without the other is a declaration nothing can + // act on, and the failure would otherwise land in a consumer's + // build rather than in the package that wrote it. + { + const bool hasExts = m.featureDeviceExtensions.contains(fname); + const bool hasMod = m.featureRuleModule.contains(fname); + if (hasExts != hasMod) { + return std::unexpected(error(origin, std::format( + "[features].{} declares `{}` without `{}`. A build rule states " + "both:\n" + " `device_extensions` is what it compiles, `rule_module` is " + "how a\n" + " consumer's build program reaches it.", + fname, + hasExts ? "device_extensions" : "rule_module", + hasExts ? "rule_module" : "device_extensions"))); + } + } // #253: per-feature per-glob compile flags — same entry grammar // as [build].flags (shared parse_glob_flags_value), gated by // this feature and folded in AFTER base globFlags at activation diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 4cce7fb12..f4dd44985 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -639,6 +639,18 @@ struct BuildConfig : BuildInputs { // Scoped to the declaring package — a dependency is classified by its own // manifest, never by its consumer's. std::vector moduleExtensions; + // Device extensions this package's build-dependencies declared through + // `[features]..device_extensions`, for the features this package + // requested. NOT written in a manifest: filled by prepare from the + // resolved edges, and carried here so every site that already builds an + // extension table for a package gets the device axis without a second + // plumbing route. A package with no rule dependency leaves it empty, which + // is every package that has none today. + std::vector deviceExtensions; + // The rule modules a synthesised `build.mcpp` imports, in the order the + // features were collected. Filled by prepare beside `deviceExtensions` + // above and read only when this package has no build program of its own. + std::vector ruleModules; // [build] accel — which accelerator backends and device architectures this // build targets, in the wire form mcpp.pack.abi_tag reads. Empty means the // build asks for none, and then every prebuilt artifact satisfies it @@ -1478,6 +1490,56 @@ struct Manifest { // see the member there.) std::map> featureProvides; // feature → caps std::map> featureRequires; // feature → caps + + // `[features]..device_extensions` — the device source extensions this feature's + // rule compiles (mcpp 2026.9.7.1+). + // + // THE ENGINE EXPOSES THE CAPABILITY; THE PACKAGE SUPPLIES THE FACT. The key + // is named after `[build] module_extensions` because it is the same shape: + // mcpp knows what it means for a file to be a device source -- never + // scanned, never a BMI, compiled by something mcpp does not drive -- and + // does not know that `.cu` is CUDA. A rule package states which extensions + // it compiles, and a consumer that activates that feature gets them + // classified as device sources. A NEW device language therefore costs no + // engine change, which is what `docs/20`'s "a sixth backend is a package + // rather than an engine change" has claimed and, until this key, was not. + // + // NOTHING IS DERIVED FROM IT. An earlier revision also used it to ACTIVATE + // the matching feature, so a consumer could name the package and nothing + // else. That was withdrawn for two reasons, and neither was cost: + // + // - Two packages may claim one extension. A third-party rule for `.cu` + // is a thing someone will write, and derivation would then have to + // guess or refuse, where `features = ["rules-cuda"]` has already said + // which one. + // - A manifest's job is to describe the build. A derived feature set is + // absent information that has to be reconstructed by running the + // build, which is worse for a reader and worse for anything reading the + // manifest as context. + // + // So the feature is requested by name, as every other feature is, and this + // key answers only "what does that feature compile". + // + // WHICH FEATURE COMPILES AN EXTENSION IS NOT STATED SEPARATELY. It is the + // feature the line is written on. A second key naming the rule would be the + // same fact twice. + std::map> featureDeviceExtensions; + + // `[features]..rule_module` — the module a consumer's build program + // imports to reach this feature's rule, and whose `compile()` it calls. + // + // DECLARED RATHER THAN DISCOVERED, which is the trade this codebase already + // makes for `mcpp::action`'s `provides`/`imports` and for + // `[modules] scan_overrides`. The name is in the feature's own interface + // unit and could be scanned out of it, but the program that imports it has + // to be written BEFORE anything is compiled, and a build that had to scan a + // dependency's sources to decide what to write would order the two the + // wrong way round. + // + // Present exactly when `device_extensions` is: together they say "this + // feature is a build rule, here is what it compiles and here is how to + // reach it". A feature with one and not the other is refused at parse time. + std::map featureRuleModule; // Feature System v2 Stage 2a — dependencies activated by a feature. A dep // declared ONLY here is optional: pulled into the resolution worklist only // when its feature is active (root --features or a dep spec's features=[...]). diff --git a/modules/source-kind/src/source_kind.cppm b/modules/source-kind/src/source_kind.cppm index b22a914bc..b59459c54 100644 --- a/modules/source-kind/src/source_kind.cppm +++ b/modules/source-kind/src/source_kind.cppm @@ -93,6 +93,24 @@ std::string_view to_string(SourceKind k); struct ExtensionTable { // Always contains the built-ins first, in their historical order. std::vector moduleInterface; + // Device extensions a DEPENDENCY declared it compiles, through + // `[features]..device_extensions`. Empty for a package with no rule + // dependency, which is every package that has none today. + // + // WHY THE PACKAGE AND NOT THIS FILE. mcpp knows what a device source IS -- + // never scanned, never a BMI, compiled by something mcpp does not drive -- + // and does not know that `.cu` is CUDA. The built-in list below is the set + // of languages the project has already shipped support for; this axis is + // how a NEW one arrives without an engine release, which is what makes + // "a sixth backend is a package rather than an engine change" true rather + // than aspirational. Slang measured the difference: adding it cost an + // engine change, a release, and a version bump in the rule package's CI + // before its rule could route a single file. + // + // Same shape as `moduleInterface` above, and for the same reason: built-ins + // are what needs no declaring, and everything else is declared by whoever + // knows it. + std::vector device; }; // Trim, then supply a leading dot if absent. Does NOT change case — see the @@ -117,6 +135,15 @@ ExtensionTable builtin_extension_table(); // validation still classifies exactly like a default one. ExtensionTable extension_table_for(std::span extras); +// The same, plus device extensions a dependency declared through +// `[features]..device_extensions`. Separate parameters rather than one list +// because the two axes are validated differently and must not be able to leak +// into each other: a module extension the project names is checked against the +// reserved roles, and a device extension a dependency names is added only where +// no built-in role already claims the spelling. +ExtensionTable extension_table_for(std::span moduleExtras, + std::span deviceExtras); + // Extensions that already name a non-module role. Declaring one of these as a // module interface has no legitimate use and would route (say) a C file to the // C++ module rule, failing somewhere that names neither the file nor the key. @@ -272,19 +299,26 @@ constexpr std::string_view kHeaderExtensions[] = { ".h", ".hpp", ".hh", ".hxx" } // own -- so the island is the shape Ascend already has, not one mcpp imposes. // `.cce` is the older spelling of the same thing and is accepted beside it. // -// `.slang` is the Slang shading language, compiled by `slangc`. It is a -// LANGUAGE rather than a second driver for GLSL -- its own module system, -// generics, and a target set beyond SPIR-V -- which is why it has an extension -// of its own here and a rule of its own outside. +// `.slang` IS NOT HERE, AND ITS ABSENCE IS THE POINT. // -// THIS TABLE IS WHAT DECIDES, NOT THE GLOB'S `accel` KEY. A constrained glob -// carrying `accel = "vulkan1.2"` does not make a file a device source; this -// list does, and a file whose extension is absent from it reaches the ordinary -// source scan and is refused with "mcpp has no role for the extension". A rule -// package therefore cannot introduce a device language on its own, and adding -// one here is the engine half of doing so. +// Slang is a language rather than a second driver for GLSL, so it needs a rule; +// it reaches this build through `mcpp:plugins`' `rules-slang`, which declares +// `device_extensions = [".slang"]` in its own manifest. That is the first +// device language mcpp supports without naming it here. +// +// The list below is what mcpp knows WITHOUT being told: the languages whose +// support shipped before the declaration existed. It is a compatibility set, +// not a registry -- a new language does not join it, and `.slang` was removed +// after the mechanism proved able to carry it, which is the only honest test of +// whether the mechanism works. +// +// THIS TABLE AND THE DECLARED ONE ARE WHAT DECIDE, NOT THE GLOB'S `accel` KEY. +// A constrained glob carrying `accel = "vulkan1.2"` does not make a file a +// device source; membership here or in a dependency's declaration does, and a +// file in neither reaches the ordinary source scan and is refused with "mcpp +// has no role for the extension". constexpr std::string_view kDeviceExtensions[] = { - ".cu", ".hip", ".sycl", ".asc", ".cce", ".slang", + ".cu", ".hip", ".sycl", ".asc", ".cce", ".comp", ".vert", ".frag", ".geom", ".tesc", ".tese", ".mesh", ".task", ".rgen", ".rint", ".rahit", ".rchit", ".rmiss", ".rcall", ".glsl", ".hlsl", ".cl", ".metal", @@ -354,6 +388,26 @@ ExtensionTable extension_table_for(std::span extras) { return t; } +ExtensionTable extension_table_for(std::span moduleExtras, + std::span deviceExtras) { + auto t = extension_table_for(moduleExtras); + for (auto const& raw : deviceExtras) { + auto ext = normalize_extension(raw); + if (ext.empty()) continue; + // A declaration cannot move a file out of a role the engine already + // owns. `.cpp` is a C++ translation unit whatever a dependency says, + // and silently accepting the entry would let one package change what + // every source in a consumer means. + if (std::ranges::find(t.moduleInterface, ext) != t.moduleInterface.end()) continue; + if (contains(kCxxExtensions, ext) || contains(kCExtensions, ext) + || contains(kGasExtensions, ext) || contains(kNasmExtensions, ext) + || contains(kHeaderExtensions, ext)) continue; + if (std::ranges::find(t.device, ext) != t.device.end()) continue; + t.device.push_back(std::move(ext)); + } + return t; +} + bool is_reserved_non_module_extension(std::string_view ext) { return contains(kCxxExtensions, ext) || contains(kCExtensions, ext) || contains(kGasExtensions, ext) || contains(kNasmExtensions, ext) @@ -408,6 +462,11 @@ SourceKind classify(const std::filesystem::path& p, const ExtensionTable& t) { if (contains(kGasExtensions, ext)) return SourceKind::GasAsm; if (contains(kNasmExtensions, ext)) return SourceKind::NasmAsm; if (contains(kDeviceExtensions, ext)) return SourceKind::Device; + // A dependency's `device_extensions`. Checked AFTER the built-in roles so a + // rule package cannot reclassify `.cpp`; the built-ins are the engine's own + // vocabulary and a declaration must not be able to move a file out of it. + for (auto const& d : t.device) + if (ext == d) return SourceKind::Device; if (contains(kHeaderExtensions, ext) || contains(kDeviceHeaderExtensions, ext)) return SourceKind::Header; return SourceKind::Other; diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index d81d83275..a7a14aba1 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -134,6 +134,13 @@ struct BuildProgramEnv { // and an engine older than this one leaves the variable absent -- which a // rule reads as "header", the behaviour every consumer had before. bool languageModules = true; + // The rule modules a synthesised build program imports, from + // `BuildConfig::ruleModules`. When this package has no `build.mcpp` and + // this list is not empty, mcpp writes the program these entries describe. + // The program is the one the project would have written by hand, which is + // what makes the declaration a LAYER above `build.mcpp` rather than a + // second way of doing the same thing. + std::vector ruleModules; // The device-kind sources (`.cu`, `.hip`, ...) this package's effective // source set matches, package-root-relative with `/` separators, one per // line. The engine has no compile rule for them and hands the list to the @@ -730,6 +737,38 @@ bool cache_fresh(const fs::path& root, const fs::path& bdir, const CacheRecord& return true; } + +// The program a set of rule modules describes. +// +// It is the program a project writes by hand for the same rules, and that is +// the whole contract: the declaration is a layer ABOVE `build.mcpp`, not a +// second mechanism beside it, so a project that outgrows it copies this file +// into its root and edits it. Synthesis then stops, because a project that has +// its own program keeps it. +// +// The namespace comes from the module name with `.` exchanged for `::`, which +// is the convention `mcpp.rules.` already follows and the one a third-party +// rule opts into by naming its module. Nothing here knows what any rule does. +std::string synthesised_rule_program(const std::vector& modules) { + std::string s = + "// Generated by mcpp from the build rules this package's dependencies\n" + "// declare. Do not edit. To take it over, copy this file to `build.mcpp`\n" + "// in the project root; mcpp synthesises nothing once a project has one.\n" + "import std;\n" + "import mcpp;\n"; + for (auto const& m : modules) s += "import " + m + ";\n"; + s += "\nint main() {\n bool ok = true;\n"; + for (auto const& m : modules) { + std::string ns; + for (char c : m) { if (c == '.') ns += "::"; else ns += c; } + // `&&` would stop at the first refusal, and a project with two rules + // wants both diagnostics rather than one and then silence. + s += " ok = " + ns + "::compile() && ok;\n"; + } + s += " return ok ? 0 : 1;\n}\n"; + return s; +} + } // namespace std::expected run_build_program( @@ -742,7 +781,34 @@ std::expected run_build_program( fs::path src = root / "build.mcpp"; std::error_code ec; - if (!fs::exists(src, ec)) return {}; // no build program — nothing to do + if (!fs::exists(src, ec)) { + // The layer above this file: when a dependency's rules claimed device + // sources in this package and the package wrote no program, mcpp writes + // the program those rules describe. + // + // SYNTHESISED ONLY IN THE ABSENCE. A project with its own `build.mcpp` + // keeps it, because the two would otherwise both submit the same + // actions and the second submission is one nobody asked for. That is + // also what makes the descent safe: copy the generated file into the + // project root, edit it, and synthesis stops. + if (env.ruleModules.empty()) return {}; // nothing to do + src = build_dir(root, env) / "build.mcpp"; + fs::create_directories(src.parent_path(), ec); + const std::string text = synthesised_rule_program(env.ruleModules); + // Written only when it differs, so a package whose rule set did not + // change does not rebuild its build program on every configure. + bool same = false; + if (std::ifstream in(src, std::ios::binary); in) { + std::string prev((std::istreambuf_iterator(in)), + std::istreambuf_iterator()); + same = (prev == text); + } + if (!same) { + std::ofstream out(src, std::ios::binary | std::ios::trunc); + if (!out) return {}; + out << text; + } + } fs::path bdir = build_dir(root, env); fs::path outDir = bdir / "out"; diff --git a/src/build/execute.cppm b/src/build/execute.cppm index fa7ed5e23..8d3b9b5e4 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1139,7 +1139,8 @@ fast_path_identity(const std::filesystem::path& projectRoot, std::string(mcpp::build::cache_mode_name( mcpp::build::resolve_cache_mode(*m, ""))), m->resources.files, - mcpp::extension_table_for(m->buildConfig.moduleExtensions), + mcpp::extension_table_for(m->buildConfig.moduleExtensions, + m->buildConfig.deviceExtensions), m->buildConfig.target, m->hooks.active(), normalize_features(featuresRequested), diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 27198faae..1fed09920 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -1017,7 +1017,8 @@ make_plan(const mcpp::manifest::Manifest& manifest, // needs it — every scanned unit arrives with its kind already set by the // scanner, using its OWN package's table. const auto rootExtTable = - mcpp::extension_table_for(manifest.buildConfig.moduleExtensions); + mcpp::extension_table_for(manifest.buildConfig.moduleExtensions, + manifest.buildConfig.deviceExtensions); // Artifact naming and shared-library link shape are properties of the // TARGET. Resolved once here from tc.targetTriple (empty = host target, in diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 554ac35a9..01d781998 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -5101,7 +5101,8 @@ prepare_build(bool print_fingerprint, // `prepare_actions`, which needs the same table to decide which // outputs get a placeholder (a header does not; see mcpp#534). const auto pkgExtTable = - mcpp::extension_table_for(mm.buildConfig.moduleExtensions); + mcpp::extension_table_for(mm.buildConfig.moduleExtensions, + mm.buildConfig.deviceExtensions); mcpp::build::directives::prepare_actions(fresh, pkgRoot, pkgExtTable); std::copy(fresh.begin(), fresh.end(), mm.buildConfig.actions.begin() @@ -5326,6 +5327,20 @@ prepare_build(bool print_fingerprint, if (!buildOnly) it->buildOnly = false; return; } + // A REQUESTED FEATURE THAT IS A BUILD RULE IMPLIES `host-module`. + // + // `host-module = true` says "compile this dependency's interface unit + // for the host so my build program can import it", and a feature + // declaring `rule_module` has already said that is the only way to use + // it. Requiring both was a second spelling of one fact, and the failure + // when only the feature was written landed in the consumer's build as + // an unresolved import rather than in the line that was incomplete. + bool hostModule = spec.hostModule; + if (!hostModule && dependencyPackageIndex < packages.size()) { + auto const& depManifest = packages[dependencyPackageIndex].manifest; + for (auto const& f : spec.features) + if (depManifest.featureRuleModule.contains(f)) { hostModule = true; break; } + } dependencyEdges.push_back(DependencyEdge{ .consumerPackageIndex = consumerPackageIndex, .dependencyPackageIndex = dependencyPackageIndex, @@ -5333,7 +5348,7 @@ prepare_build(bool print_fingerprint, .requestedFeatures = spec.features, .defaultFeatures = spec.defaultFeatures, .requestedTools = spec.tools, - .hostModule = spec.hostModule, + .hostModule = hostModule, .reexport = spec.reexport, .buildOnly = buildOnly, }); @@ -5447,7 +5462,8 @@ prepare_build(bool print_fingerprint, // had already drifted: all three assembly extensions were missing, // so staging a dependency with .S/.s/.asm silently dropped them. globs = mcpp::default_source_globs( - mcpp::extension_table_for(depManifest.buildConfig.moduleExtensions)); + mcpp::extension_table_for(depManifest.buildConfig.moduleExtensions, + depManifest.buildConfig.deviceExtensions)); } // Glob exclusion (same as scan_one_into): `!` prefix removes. std::set sourceFiles; @@ -7229,6 +7245,83 @@ prepare_build(bool print_fingerprint, feature_closure(packages[i].manifest, req, depDefaultFeatures); } + // ─── Device extensions a rule dependency declared ────────────────── + // + // A rule package states which device extensions it compiles, on the + // feature that provides the rule. Collected here, after features are + // activated, because only an ACTIVE feature's declaration applies: a + // collection carrying a CUDA rule and a shader rule must not make `.cu` + // a device source in a project that asked for the shader rule alone. + // + // Written into the CONSUMER's `[build]` so every site that already + // builds an extension table for a package picks it up without a second + // plumbing route. + // + // THE POSITION IS LOAD-BEARING. It sits after feature activation and + // before the extension table that narrows the constrained globs, which + // is the first reader. Placed after that table instead, the declared + // extensions arrive too late to classify anything: the device source + // list comes out empty, the rule is handed nothing, it generates no + // module, and the failure surfaces three edges away as `failed to read + // compiled module` on the interface the consumer imported. Measured. + // + // It is what makes a new device language cost no engine change. Adding + // `.slang` to the built-in table required an mcpp release and a version + // bump in the rule package's CI before its rule could route one file; + // a language arriving this way needs neither. + for (std::size_t ci = 0; ci < packages.size(); ++ci) { + std::vector collected; + std::vector ruleModules; + for (auto const& edge : dependencyEdges) { + if (edge.consumerPackageIndex != ci) continue; + if (edge.dependencyPackageIndex >= packages.size()) continue; + auto const& dep = packages[edge.dependencyPackageIndex]; + const auto& depFeatures = + edge.dependencyPackageIndex < activeFeaturesByPackage.size() + ? activeFeaturesByPackage[edge.dependencyPackageIndex] + : edge.requestedFeatures; + for (auto const& f : depFeatures) { + auto it = dep.manifest.featureDeviceExtensions.find(f); + if (it == dep.manifest.featureDeviceExtensions.end()) continue; + for (auto const& e : it->second) + if (std::ranges::find(collected, e) == collected.end()) + collected.push_back(e); + // The module a synthesised build program imports for this + // rule. Declared by the feature rather than scanned out of + // its source, because the program has to be WRITTEN before + // anything is compiled and a build that scanned a + // dependency to decide what to write would order the two + // the wrong way round. + if (auto mit = dep.manifest.featureRuleModule.find(f); + mit != dep.manifest.featureRuleModule.end() + && std::ranges::find(ruleModules, mit->second) == ruleModules.end()) { + ruleModules.push_back(mit->second); + // Said out loud, for the same reason the resolved + // toolchain is: the manifest states the intent and the + // build states what that came to. Without this line a + // reader of a terse manifest could not tell which rules + // ran. + mcpp::ui::info("Rules", std::format("{} ({}:{})", mit->second, + dep.manifest.package.namespace_, + dep.manifest.package.name)); + } + } + } + if (!collected.empty()) { + if (ci == 0) m->buildConfig.deviceExtensions = collected; + packages[ci].manifest.buildConfig.deviceExtensions = std::move(collected); + } + if (!ruleModules.empty()) { + // THE ROOT'S MANIFEST IS TWO OBJECTS. `packages[0]` holds a COPY + // made by `makePackageRoot`, and the build-program environment for the + // root reads `*m`. Writing only the copy left the synthesis with + // an empty list and the shaders uncompiled, with a refusal that + // named the missing build program rather than the missing write. + if (ci == 0) m->buildConfig.ruleModules = ruleModules; + packages[ci].manifest.buildConfig.ruleModules = std::move(ruleModules); + } + } + // ── Constrained source globs: narrow to what this build targets ──── // // A `{ glob = "...", accel = "..." }` entry in `[build] sources` says @@ -7347,7 +7440,8 @@ prepare_build(bool print_fingerprint, // The device-kind files the EFFECTIVE set matches, for the // build program. Exclusions are honoured the way the scanner // honours them: positives first, then `!` entries removed. - const auto extTable = mcpp::extension_table_for(bc.moduleExtensions); + const auto extTable = mcpp::extension_table_for(bc.moduleExtensions, + bc.deviceExtensions); std::set matched, dropped; for (auto const& g : pkg.manifest.modules.sources) { if (g.empty()) continue; @@ -8012,7 +8106,8 @@ prepare_build(bool print_fingerprint, for (std::size_t i = 1; i < packages.size(); ++i) { auto& pkg = packages[i]; std::error_code bpEc; - if (!std::filesystem::exists(pkg.root / "build.mcpp", bpEc)) continue; + if (!std::filesystem::exists(pkg.root / "build.mcpp", bpEc) + && pkg.manifest.buildConfig.ruleModules.empty()) continue; auto host = host_tc_for_build_program(); if (!host) return std::unexpected(host.error()); // Same edge-graph aggregation as feature activation above, so a @@ -8039,6 +8134,7 @@ prepare_build(bool print_fingerprint, // generating a declaration for this package must match how this // package is compiled. bpEnv.languageModules = pkg.manifest.language.modules; + bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(pkg.root.string()); dit != deviceSourcesByPackage.end()) bpEnv.deviceSources = dit->second; bpEnv.features = feature_closure(pkg.manifest, req, depDefaultFeatures); @@ -8942,7 +9038,14 @@ prepare_build(bool print_fingerprint, // actually read. Now the snapshot (and root feature activation on it) // already happened, so mirror the directive TAILS into packages[0] // explicitly, the same way the dep loop does for its package. - if (std::filesystem::exists(*root / "build.mcpp")) { + // A package with no `build.mcpp` still runs one when a rule dependency + // described it: `run_build_program` writes that program into the build + // directory. This guard therefore asks the same question the function does, + // and a guard that asked only about the file left the synthesis unreachable + // -- the shaders went uncompiled and the refusal named the missing program + // rather than the guard. Measured. + if (std::filesystem::exists(*root / "build.mcpp") + || !m->buildConfig.ruleModules.empty()) { auto host = host_tc_for_build_program(); if (!host) return std::unexpected(host.error()); mcpp::build::BuildProgramEnv bpEnv; @@ -8957,6 +9060,7 @@ prepare_build(bool print_fingerprint, bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); bpEnv.languageModules = m->language.modules; + bpEnv.ruleModules = m->buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(root->string()); dit != deviceSourcesByPackage.end()) bpEnv.deviceSources = dit->second; // Set explicitly rather than relying on build_dir()'s root-relative @@ -9085,7 +9189,8 @@ prepare_build(bool print_fingerprint, orphans += " " + rel + "\n"; if (orphans.empty()) continue; std::error_code hasEc; - const bool hasProgram = std::filesystem::exists(pkg.root / "build.mcpp", hasEc); + const bool hasProgram = std::filesystem::exists(pkg.root / "build.mcpp", hasEc) + || !pkg.manifest.buildConfig.ruleModules.empty(); refusal::record(refusal::Code::DeviceSourceUnconsumed); return std::unexpected(std::format( "`{}`: device sources that no action compiles:\n{}" diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 1ef0d0327..d96b4c30e 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -853,7 +853,8 @@ void scan_one_into(ScanResult& result, // file, and taken from THIS manifest — a dependency is classified by its // own `[build] module_extensions`, never by the consumer's. const auto extTable = - mcpp::extension_table_for(manifest.buildConfig.moduleExtensions); + mcpp::extension_table_for(manifest.buildConfig.moduleExtensions, + manifest.buildConfig.deviceExtensions); // Glob exclusion: patterns starting with `!` remove files from the // include set (like .gitignore). @@ -1153,7 +1154,8 @@ ScanResult scan_packages_p1689(const std::vector& packages, for (auto const& p : packages) { // Same contract as scan_one_into: each package's own table. const auto extTable = - mcpp::extension_table_for(p.manifest.buildConfig.moduleExtensions); + mcpp::extension_table_for(p.manifest.buildConfig.moduleExtensions, + p.manifest.buildConfig.deviceExtensions); std::set all_files; for (auto const& g : p.manifest.modules.sources) { for (auto& f : expand_glob(p.root, g)) all_files.insert(f); From f7f73588b61402ee0b3042b170f0e914c5da1d82 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 23:51:48 +0800 Subject: [PATCH 07/15] docs: the two keys, the table that now has two sources, and what was withdrawn docs/05 gains the rule-feature section in both languages: what `device_extensions` and `rule_module` state, the two things that follow (`host-module` implied, a build program written where there is none), and why the feature is still requested by name. docs/20's device extension table is now described as what mcpp knows without being told rather than as the registry a sixth backend joins, with a note pointing at the declaration that adds to it. `.slang` has left the table. The design document records the two designs that were written and withdrawn -- a `[rules]` section and activating rules from the files present -- with the objections that retired them, so the next reader does not rediscover the same two shapes. --- ...9-07-module-first-heterogeneous-surface.md | 119 +++++++++++------- CHANGELOG.md | 29 +++++ docs/05-mcpp-toml.md | 56 +++++++++ docs/20-heterogeneous-builds.md | 10 +- docs/zh/05-mcpp-toml.md | 47 +++++++ docs/zh/20-heterogeneous-builds.md | 8 +- 6 files changed, 219 insertions(+), 50 deletions(-) diff --git a/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md b/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md index 3f2f610c6..21181bd08 100644 --- a/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md +++ b/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md @@ -675,61 +675,84 @@ the surface. Status is as of this document's date. "done" means implemented and verified by a test that was seen to fail without the change. -| Id | Repo | Item | Depends on | Status | -|---|---|---|---|---| -| E1 | mcpp | `mcpp::action` gains `depfile`; ninja emits `depfile =` and `deps = gcc` for action edges | -- | done | -| E6 | mcpp | `.slang` joins `kDeviceExtensions` in `modules/source-kind/src/source_kind.cppm` | -- | done | -| E7 | mcpp | `[language] modules` reported as `MCPP_LANGUAGE_MODULES`, so the surface default follows the project | -- | done | -| E5 | mcpp | Documentation: `docs/07-build-mcpp.md`, `docs/20-heterogeneous-builds.md` and their `zh` counterparts | E1, E6, E7 | done | -| P3 | mcpp-plugins | The surface: `mcpp::plugins::surface` in the lib root, a generated `.cppm` or `.h`, a std-free POD interface, module and namespace naming per 5.2 and 5.3 | -- | done | -| P4 | mcpp-plugins | `rules.spirv` delegates the surface. It keeps writing the data header, because 0.2.6 already unified the two compilers on one shape -- both emit `.inc` and the rule writes `.h` -- so there is no second copy of that decision left to remove | P3 | done | -| P4b | mcpp-plugins | `tools.embed::group()`: the same surface over payloads that were already on disk | P3 | done | -| P5 | mcpp-plugins | `rules.slang` | P3, E6 released | code done, waits on the mcpp release | -| P7 | mcpp-plugins | Tests: a consumer per member and per surface, and the cross-platform compile matrix extended to `rules-slang` | P3..P5 | done | -| P1 | mcpp-plugins | `storage`: object storage through a generated `.S` using `.incbin`, and sidecar. MSVC falls back to header storage | P3 | staged | -| P6 | mcpp-plugins | Rules pass `--depfile`, `-MD -MF` and `-depfile` | E1 released | staged | -| E2 | mcpp | `[rules]` manifest section; the engine synthesises the build program a rule entry describes | -- | staged | -| E3 | mcpp | L0: the extension-to-rule table is lifted out of the diagnostic in `prepare.cppm` and activates a rule when a device source matches and the accelerator axis agrees | E2 | staged | -| E4 | mcpp | Diagnostic for an accelerator-produced module imported from an ungated translation unit (5.6) | -- | staged | -| X1 | openxlings/xim-pkgindex | Raise `xim:slang` from `2026.14.1` toward upstream `v2026.17`. The rule works against the published pin, so nothing waits on it | -- | staged | - -**What shipped is the surface plane, not the storage plane, and the ordering is -deliberate rather than a shortfall.** Section 3.1 measured the header route as -the faster of the two at realistic shader sizes and put the crossover at roughly -1 MB of total embedded data, so the storage a project gets today is the one the -measurement recommends. Object storage is worth having above that threshold and -carries platform work the surface does not -- a section directive per object -format, symbol prefixes on Mach-O, and the MSVC fallback of section 11 -- which -makes it a round of its own rather than a rider on this one. - -`P2`, a host-tool incarnation of the embed tool, has left the list. It existed -because the surface appeared to need the payload's bytes at plan time, and it -does not: the generated interface holds declarations, the generated -implementation holds includes, and `.incbin` resolves its path at assembly time. -All three can be written before any action runs, which is what made the surface -a wave-1 item. +| Id | Repo | Item | Status | +|---|---|---|---| +| E1 | mcpp | `mcpp::action` gains `depfile`; ninja emits `depfile =` and `deps = gcc` | done | +| E7 | mcpp | `[language] modules` reported as `MCPP_LANGUAGE_MODULES` | done | +| E8 | mcpp | `[features]..device_extensions` and `rule_module`: a rule package declares what it compiles and how to reach it | done | +| E9 | mcpp | `host-module = true` implied by `rule_module` | done | +| E10 | mcpp | A package with no `build.mcpp` gets the program its rules describe | done | +| E5 | mcpp | Documentation, `docs/07` and `docs/20` and their `zh` counterparts | done | +| E11 | mcpp | `tests/e2e/188`'s "did not rerun" assertion, which could not fail | done | +| E12 | mcpp | The offscreen example runs on macOS as well as Linux | done | +| P3 | mcpp-plugins | The surface: generated `.cppm` or `.h`, std-free POD interface, module and namespace naming | done | +| P4 | mcpp-plugins | `rules.spirv` and `tools.embed::group()` delegate the surface | done | +| P5 | mcpp-plugins | `rules.slang` | done | +| P1 | mcpp-plugins | `storage`: header, object through `.incbin`, sidecar; MSVC falls back to header | done | +| P8 | mcpp-plugins | Every rule feature declares `device_extensions` and `rule_module` | done | +| P7 | mcpp-plugins | Nine fixtures, one per surface, storage and language | done | +| E6 | mcpp | `.slang` in the engine's built-in table | **withdrawn** -- replaced by E8, and removed from the table to prove it | +| E2/E3 | mcpp | A `[rules]` manifest section, and activating rules from the files present | **withdrawn**, see 13.1.2 | +| P6 | mcpp-plugins | Rules pass `--depfile`, `-MD -MF` and `-depfile` | staged, needs E1 released | +| X1 | openxlings/xim-pkgindex | Raise the `xim:slang` pin toward upstream `v2026.17` | staged | +| X2 | openxlings/xim-pkgindex | A Windows `vulkan-1.dll` package | staged, see 13.1.3 | ### 13.1.1 A cost the implementation paid, stated rather than hidden The generator lives in `mcpp.plugins`, the lib root, which grew from about -twenty lines to about four hundred. The lib root is compiled for **every** -consumer of the package, including one that activates only `rules-cuda` and -will never embed anything, so every consumer's build program now compiles the -generator too. +twenty lines to about seven hundred. The lib root is compiled for **every** +consumer of the package, including one that activates only `rules-cuda` and will +never embed anything. It is there because a second unit beside the lib root in `[build] sources` is not compiled as a host module ahead of the members. Measured: a member importing -`mcpp.plugins.surface` failed with `failed to read compiled module`, because only -the lib root is built first. Three members need the generator -- `rules-spirv`, -`rules-slang` and `tools-embed` -- and a feature that two of them had to activate -for the third would be a dependency between members that the feature system does -not express. - -The cost is one host-module compilation of roughly four hundred lines, paid once -per consumer per configure, and it buys the property section 4 exists for: one -generator, so the three members cannot drift. If the engine later compiles every -lib-root source as a host module in listed order, the generator moves to a file -of its own and this paragraph is deleted. +`mcpp.plugins.surface` failed with `failed to read compiled module`, because +only the lib root is built first. Three members need the generator, and a +feature that two of them had to activate for the third would be a dependency +between members that the feature system does not express. + +### 13.1.2 Two designs were written and withdrawn, and both for the same reason + +**A `[rules]` manifest section.** `[rules] spirv = {}` would have replaced the +dependency edge and the build program. It was withdrawn because it restates what +`[build] sources` already says -- the file is there, its extension names the +rule -- and a section carrying no information the manifest does not already hold +earns its place only by expressing consent, which the dependency edge expresses +better and where the version pin belongs. + +**Activating a rule from the files present.** With `device_extensions` in place, +a consumer could have named the package alone and let the extensions decide +which features turn on. Two objections, and neither was cost: + +- Two packages may claim one extension. A third-party CUDA rule is a thing + someone will write, and derivation would then guess or refuse where + `features = ["rules-cuda"]` has already said which. +- A manifest's job is to describe the build. A derived feature set is + information the file no longer states, which is worse for a reader and worse + for anything reading the manifest as context. + +What survived from both is the part that carries the architecture: a rule +package declares what it compiles, so the engine holds no package name, no +feature spelling and no module name -- and a new device language costs no engine +change. `.slang` was removed from the built-in table to prove that, and +`tests/slang-consumer` builds unchanged. + +### 13.1.3 What the Windows measurement changed + +Raising the graphics example from "builds" to "runs" on Windows was expected to +be two CI steps. It is not. The manifest declaration worked -- +`Provisioning [xlings.workspace] entries (xim:mesa-lavapipe@26.2.0)` -- the ICD +was found in the store, and the program still printed `render unavailable`. + +`compat:vulkan` ships an import library on Windows and nothing else, and says +why in its own descriptor: a statically linked loader cannot work there, because +upstream's `loader_windows.c` creates its locks in `DllMain` and a static +library never gets one. The runtime `vulkan-1.dll` is expected to come from an +installed GPU driver, and a runner has none. + +So the platform stays at "builds" and the example declares no device there -- +an entry would download 56 MB that nothing can load. The gap is a Windows loader +package, which is X2. ### 13.2 Dependency structure across repositories diff --git a/CHANGELOG.md b/CHANGELOG.md index b947c1b1d..378952762 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -86,6 +86,35 @@ libmalloc 里 abort(#202)。CI 报的正是这条路的第一步:链接停在 `_ `cfg(accelerator = ...)` 下的 `[build]` 源生效而依赖被忽略,于是包被丢掉、包含它的源 被留下。 +### 规则包自己声明它编译什么,于是新设备语言不再需要动引擎 + +`[features].` 上的两个键:`device_extensions` 说这条规则编译哪些设备源扩展名, +`rule_module` 说消费者的构建程序 import 哪个模块去够到它。引擎里没有任何包名、feature +拼写或模块名。 + +**一门新设备语言不再触碰引擎。** 把 `.slang` 加进内置表曾经要一次引擎改动、一次发布, +以及规则包 CI 里的一次版本抬升,规则才能路由一个文件。这一版把 `.slang` 从那张表里 +**移除**,改由 `rules-slang` 声明;`tests/slang-consumer` 原样构建并运行,这是判断这套 +机制能不能承载一门语言的唯一诚实办法。内置表从此是「mcpp 不需要被告知就知道的那些」—— +一个兼容集合,而不是第六个后端要加入的注册表。 + +**消费者写一条边,不写构建程序。** `rule_module` 蕴含 `host-module = true`,因为一个 +点名了规则模块的 feature 已经说过那是使用它的唯一方式。而没有 `build.mcpp` 的包会拿到 +那些规则描述的程序: + + [build-dependencies.mcpp] + plugins = { version = "0.3.0", features = ["rules-spirv"] } + +就是 `tests/spirv-zero-config` 的全部声明。自带程序的包保留自己的:合成只填补缺席。 + +**feature 仍然按名字请求。** 早先一版从工程源码里出现的扩展名推导这个集合,被撤销了: +两个包可能认领同一扩展名(第三方写一条处理 `.cu` 的规则是会发生的事),而且 manifest +的职责是描述这次构建,派生出来的集合让文件不再陈述它。 + +跑起来才发现的两处顺序缺陷:收集必须落在 feature 激活与「收窄受限 glob 的那张扩展名表」 +之间,放在其后则声明来得太晚、规则拿到空清单;而构建程序调用点的守卫问的是「文件在不在」 +而函数问的是「要不要一个程序」,于是合成不可达。两处现在问同一个问题。 + ### 可移植性驱动默认是看不见的,于是三个平台都从「构建」抬到「运行」 macOS 上没有原生 Vulkan,MoltenVK 是 Metal 之上的实现,规范把这种实现叫**可移植性驱动**。 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 0be954e12..bbe6c2c5b 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -1506,6 +1506,62 @@ simd = { sources = ["src/simd/**"], flags = [ consumers (same contract as `[build].flags`), so they stay inside the additive model: scoped by glob, deterministic order, no cross-package effect. + +#### A feature that is a build rule (mcpp 2026.9.7.1+) + +Two keys turn a feature into a build rule other packages can use. They are what +lets a consumer write one dependency edge and no build program. + +```toml +[features.rules-spirv] +sources = ["rules/spirv.cppm"] +rule_module = "mcpp.rules.spirv" +device_extensions = [".comp", ".vert", ".frag", ".glsl"] +``` + +`device_extensions` states which **device source** extensions this rule +compiles. A consumer that activates the feature gets them classified as device +sources -- never scanned for imports, never a BMI, compiled by something mcpp +does not drive. This is the same shape as `[build] module_extensions`: mcpp +knows what a device source *is* and does not know that `.cu` is CUDA, so a NEW +device language costs no engine change. It is what makes +[20 — Heterogeneous Builds](20-heterogeneous-builds.md)' claim that "a sixth +backend is a package rather than an engine change" true rather than +aspirational; `.slang` was removed from mcpp's built-in table and now arrives +this way. + +`rule_module` names the module a consumer's build program imports to reach the +rule, and whose `compile()` it calls. Declared rather than scanned out of the +source, because the program has to be **written** before anything is compiled +and a build that scanned a dependency to decide what to write would order the +two the wrong way round. + +Two things follow, and neither puts a package name inside mcpp: + +- **`host-module = true` is implied.** A feature naming a rule module has + already said that is the only way to use it, so the edge does not repeat it. +- **A package with no `build.mcpp` gets one.** mcpp writes the program those + rules describe into the build directory and compiles that. A package with its + own program keeps it: the synthesis fills an absence and never overrides, and + the generated file is the program a project would have written, so taking it + over is a copy and an edit. + +The feature is still requested **by name**: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.3.0", features = ["rules-spirv"] } +``` + +An earlier design derived the set from the extensions a project's sources +carried. It was withdrawn because two packages may claim one extension -- a +third-party CUDA rule is a thing someone will write -- and because a manifest's +job is to describe the build, which a derived feature set no longer does. + +Both keys must appear together. One without the other is a declaration nothing +can act on, and it is refused at parse time rather than in a consumer's build. + + ### 2.8.1 `provides` / `requires` — Capabilities (backend selection) A **capability** is a shared abstract name (e.g. `blas`). A package can *provide* diff --git a/docs/20-heterogeneous-builds.md b/docs/20-heterogeneous-builds.md index 65062fc37..7d17f3e41 100644 --- a/docs/20-heterogeneous-builds.md +++ b/docs/20-heterogeneous-builds.md @@ -93,7 +93,6 @@ does not accept C++20 modules. | CUDA, HIP | `.cu`, `.hip` | | SYCL | `.sycl` (2026.9.6.1+) | | Ascend C | `.asc`, `.cce` (2026.9.6.5+) | -| Slang | `.slang` (2026.9.7.1+) | | GLSL, by stage | `.comp`, `.vert`, `.frag`, `.geom`, `.tesc`, `.tese`, `.mesh`, `.task`, `.rgen`, `.rint`, `.rahit`, `.rchit`, `.rmiss`, `.rcall` | | GLSL, stage-less | `.glsl` | | HLSL | `.hlsl` | @@ -105,6 +104,15 @@ which is the behaviour that makes the table a table: mcpp has no rule for the file, its object would be linked by nothing, and building it would fail later and less clearly. +The table above is what mcpp knows **without being told**: the languages whose +support shipped before a package could declare one. A rule package adds to it, +through `[features]..device_extensions` (see +[05 — mcpp.toml](05-mcpp-toml.md) §2.8), and that is how a NEW device language +arrives -- with no engine change and no engine release. Slang is the first: +`.slang` is not in the list above, and `mcpp:plugins`' `rules-slang` declares +it. + + `.glsl` carries no stage. glslang derives the stage from the extension, so a rule package refuses a stage-less name — the message belongs there, and this table therefore does not need to know which extensions name a stage. diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index f7a5892ae..c360d2483 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -1290,6 +1290,53 @@ simd = { sources = ["src/simd/**"], flags = [ feature `flags` 是**私有 per-TU 构建旗标**——永不传播给消费者(与 `[build].flags` 同契约),因此不破坏加性模型:glob 限定作用面、顺序确定、无跨包效应。 + +#### 作为构建规则的 feature(mcpp 2026.9.7.1+) + +两个键把一个 feature 变成其它包可以使用的构建规则。它们是消费者只写一条依赖边、 +不写构建程序的原因。 + +```toml +[features.rules-spirv] +sources = ["rules/spirv.cppm"] +rule_module = "mcpp.rules.spirv" +device_extensions = [".comp", ".vert", ".frag", ".glsl"] +``` + +`device_extensions` 陈述这条规则编译哪些**设备源**扩展名。激活了该 feature 的消费者 +会把它们分类为设备源 —— 不扫描 import、不产 BMI、由 mcpp 不驱动的编译器编译。这与 +`[build] module_extensions` 是同一个形状:mcpp 知道设备源*是什么*,不知道 `.cu` 是 +CUDA,所以**一门新设备语言不需要引擎改动**。 +[20 — 异构硬件构建](20-heterogeneous-builds.md) 里那句「第六个后端是一个包而不是一次 +引擎改动」由此才成立;`.slang` 已从 mcpp 的内置表中移除,现在正是经由这条路到达的。 + +`rule_module` 给出消费者的构建程序为够到这条规则而 import 的模块,以及它调用的 +`compile()` 所在。它是**声明**的而不是从源码扫描出来的,因为那个程序必须在任何东西 +被编译**之前**写出来,而一次为了决定写什么而去扫描依赖源码的构建会把两者的顺序颠倒。 + +由此得出两件事,而且都不把任何包名放进 mcpp: + +- **`host-module = true` 被推出来。** 一个点名了规则模块的 feature 已经说过那是使用 + 它的唯一方式,所以依赖边不必再说一遍。 +- **没有 `build.mcpp` 的包会得到一个。** mcpp 把这些规则描述的程序写进构建目录并编译 + 它。自带程序的包保留自己的:合成只填补缺席、绝不覆盖;而生成出来的那份就是这个工程 + 本来要手写的那份,所以接管它是一次复制加一次编辑。 + +feature 仍然**按名字**请求: + +```toml +[build-dependencies.mcpp] +plugins = { version = "0.3.0", features = ["rules-spirv"] } +``` + +早先的一版设计从工程源码里出现的扩展名推导这个集合。它被撤销了,因为两个包可能认领 +同一个扩展名 —— 第三方写一条处理 `.cu` 的规则是会发生的事 —— 也因为 manifest 的职责 +是描述这次构建,而派生出来的 feature 集合让文件不再陈述它。 + +两个键必须成对出现。只写其一是一条没有任何东西能据以行动的声明,会在解析期被拒绝, +而不是留到消费者的构建里。 + + ### 2.8.1 `provides` / `requires` —— 能力(后端选择) **capability(能力)** 是一个共享的抽象名字(如 `blas`)。包可以 *provide*(提供) diff --git a/docs/zh/20-heterogeneous-builds.md b/docs/zh/20-heterogeneous-builds.md index a4753b1d8..d805472fa 100644 --- a/docs/zh/20-heterogeneous-builds.md +++ b/docs/zh/20-heterogeneous-builds.md @@ -75,7 +75,6 @@ C++20 modules 的编译器。 | CUDA、HIP | `.cu`、`.hip` | | SYCL | `.sycl`(2026.9.6.1+) | | Ascend C | `.asc`、`.cce`(2026.9.6.5+) | -| Slang | `.slang`(2026.9.7.1+) | | GLSL(按 stage) | `.comp`、`.vert`、`.frag`、`.geom`、`.tesc`、`.tese`、`.mesh`、`.task`、`.rgen`、`.rint`、`.rahit`、`.rchit`、`.rmiss`、`.rcall` | | GLSL(无 stage) | `.glsl` | | HLSL | `.hlsl` | @@ -86,6 +85,13 @@ C++20 modules 的编译器。 的含义:mcpp 对该文件没有任何规则,它的目标文件不会被任何东西链接,构建下去只会 在更晚、更不清楚的地方失败。 +上表是 mcpp **不需要被告知**就知道的那些:在「包可以自己声明」之前就已经支持的语言。 +规则包通过 `[features]..device_extensions`(见 +[05 — mcpp.toml](05-mcpp-toml.md) §2.8)向它增补,而这正是**一门新设备语言到达的方式** +—— 不动引擎,也不需要发一版引擎。Slang 是第一个:`.slang` 不在上表里,由 +`mcpp:plugins` 的 `rules-slang` 声明。 + + `.glsl` 不携带 stage。glslang 从扩展名推导 stage,因此拒绝一个无 stage 的名字是 规则包的事 —— 那条消息属于那里,这张表因此不需要知道哪些扩展名指定了 stage。 From 15ce353f439b277deb0bfbd0e07348aa0e01422a Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 23:53:31 +0800 Subject: [PATCH 08/15] docs/20: why one lane's interface is generated and the other's is written The seam is written by hand and the shader lane's equivalent is generated, which reads as an inconsistency until the reason is stated. A device translation unit is code, and its interface is a design decision no generator makes well; a shader is data, and its interface is an address and a size. Both are already invisible to a consumer -- only the seam includes the C header, and everything downstream imports the module -- so the code lanes have been module-first all along. The shader lane was the exception until 2026.9.7.1, where the generated header WAS the interface. Added in both languages. --- docs/20-heterogeneous-builds.md | 24 ++++++++++++++++++++++++ docs/zh/20-heterogeneous-builds.md | 20 ++++++++++++++++++++ 2 files changed, 44 insertions(+) diff --git a/docs/20-heterogeneous-builds.md b/docs/20-heterogeneous-builds.md index 7d17f3e41..de6254c85 100644 --- a/docs/20-heterogeneous-builds.md +++ b/docs/20-heterogeneous-builds.md @@ -146,6 +146,30 @@ C++ ABI. The island should avoid the standard library, because an island that links libstdc++ puts a second copy of the C++ runtime into a program whose own copy came from mcpp's toolchain. + +### Two kinds of lane, and only one of them has a generated interface + +The seam above is written by hand, and the shader lane's equivalent is +generated. That is not an inconsistency; the two lanes carry different things. + +**A device translation unit is code.** Its interface is a design decision -- +which functions, which types, what happens on failure -- and no generator can +make that decision well. So CUDA, HIP, SYCL and Ascend C get a hand-written +seam, and the `extern "C"` header exists for the ABI reason above. Both are +already invisible to a consumer: only the seam includes the header, and +everything downstream writes `import app.saxpy`. **These lanes are module-first +today and always have been.** + +**A shader or an embedded file is data.** Its interface is an address and a +size, which is mechanical, so a rule package generates it and a consumer writes +`import myapp.shaders` without naming a generated file either. Before mcpp +2026.9.7.1 that lane was the one exception: the generated header *was* the +interface, and every consumer named it. + +So the rule is not "generate the interface" or "write it by hand". It is: a +mechanical interface is generated, a designed one is written, and in both cases +the header is an intermediate that no consumer names. + ## Compiling an island The command that invokes a device compiler is not built into mcpp. It is diff --git a/docs/zh/20-heterogeneous-builds.md b/docs/zh/20-heterogeneous-builds.md index d805472fa..91444f686 100644 --- a/docs/zh/20-heterogeneous-builds.md +++ b/docs/zh/20-heterogeneous-builds.md @@ -117,6 +117,26 @@ tarball 已经发出去了。设备源文件必须被显式点名。 岛本身应当避开标准库,因为一个链接了 libstdc++ 的岛,会把第二份 C++ 运行时放进一个 自身运行时来自 mcpp 工具链的程序里。 + +### 两类 lane,只有一类的接口是生成的 + +上面那个接缝是手写的,而 shader 那条 lane 的对应物是生成的。这不是不一致 —— +两条 lane 承载的东西不同。 + +**设备编译单元是代码。** 它的接口是一个设计决定 —— 有哪些函数、什么类型、失败怎么报 +—— 而没有任何生成器能把这个决定做好。所以 CUDA、HIP、SYCL 与 Ascend C 有一个手写的 +接缝,而那个 `extern "C"` 头文件因为上面那条 ABI 理由而存在。两者对消费者**都已经是 +不可见的**:只有接缝包含那个头文件,下游一律写 `import app.saxpy`。**这些 lane 今天 +就是模块优先的,而且一直如此。** + +**shader 或一份被嵌入的文件是数据。** 它的接口是一个地址加一个尺寸,这是机械的,所以 +由规则包生成,消费者同样不写出任何生成物的名字,只写 `import myapp.shaders`。在 +mcpp 2026.9.7.1 之前,那条 lane 是唯一的例外:生成的头文件**就是**接口,而每个消费者 +都得写出它的名字。 + +所以规则不是「接口要生成」也不是「接口要手写」。规则是:**机械的接口生成,设计出来的 +接口手写,而两种情况下头文件都是没有任何消费者会写出其名字的中间产物。** + ## 编译一个岛 调用设备编译器的那条命令不内置在 mcpp 里,而是由**构建规则包**提供 —— From b44b535ea31853ade3a757d8ca1f33840f71e71e Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 7 Sep 2026 23:54:48 +0800 Subject: [PATCH 09/15] docs/20: whether the island header is required, and what omitting it costs Not required by the language: the seam can declare the entry point and the island can define it, with no header, and that builds and runs. What the header buys is that the declaration exists once. Stated because the failure it prevents is the worst one available here. C language linkage does not mangle, so two copies that disagree are one symbol: the link is clean and each side reads the arguments by its own ABI. The property that forces this boundary to be extern "C" is the same property that makes a split declaration undetectable, and the three alternatives that would remove the header are listed with why none does. Both languages. --- docs/20-heterogeneous-builds.md | 33 ++++++++++++++++++++++++++++++ docs/zh/20-heterogeneous-builds.md | 29 ++++++++++++++++++++++++++ 2 files changed, 62 insertions(+) diff --git a/docs/20-heterogeneous-builds.md b/docs/20-heterogeneous-builds.md index de6254c85..6630c71b5 100644 --- a/docs/20-heterogeneous-builds.md +++ b/docs/20-heterogeneous-builds.md @@ -147,6 +147,39 @@ links libstdc++ puts a second copy of the C++ runtime into a program whose own copy came from mcpp's toolchain. +#### Is the `extern "C"` header required? + +Not by the language. The seam can declare the entry point itself and the island +can define it, with no header anywhere, and that builds and links and runs. + +What the header buys is that the declaration exists **once**. Without it there +are two copies in two compilers, and they can disagree silently: + +```cpp +// the seam +extern "C" int saxpy_device(float a, const float* x, const float* y, + float* out, unsigned n); +// the island, after someone widened the count +extern "C" int saxpy_device(float a, const float* x, const float* y, + float* out, std::size_t n); +``` + +C language linkage does not mangle, so those are one symbol. The link is clean +and each side reads the arguments by its own ABI: no compile error, no link +error, and a run that reads past the end of the arguments. The same mistake +across a C++ boundary is caught by mangling at link time. + +So the property that forces this boundary to be `extern "C"` -- the two sides do +not share a C++ ABI -- is the same property that makes a split declaration +undetectable. The header is the smallest artefact both a module and a device +compiler can read, which is the whole of its reason for existing. + +Three alternatives were considered and none removes it: the island cannot +include the seam (a `.cppm` is not something nvcc parses), generating the header +from some single source is a header with an extra step, and a module's global +module fragment exports nothing the island could reach even if its compiler +could read a BMI. + ### Two kinds of lane, and only one of them has a generated interface The seam above is written by hand, and the shader lane's equivalent is diff --git a/docs/zh/20-heterogeneous-builds.md b/docs/zh/20-heterogeneous-builds.md index 91444f686..2afc9aea6 100644 --- a/docs/zh/20-heterogeneous-builds.md +++ b/docs/zh/20-heterogeneous-builds.md @@ -118,6 +118,35 @@ tarball 已经发出去了。设备源文件必须被显式点名。 自身运行时来自 mcpp 工具链的程序里。 +#### 那个 `extern "C"` 头文件是必须的吗? + +语言上不是。接缝可以自己声明入口点、岛自己定义它,一个头文件都不写,照样编得过、链得上、 +跑得起来。 + +头文件买来的是**那份声明只有一份**。没有它就有两份副本分属两个编译器,而它们可以静默地 +不一致: + +```cpp +// 接缝里 +extern "C" int saxpy_device(float a, const float* x, const float* y, + float* out, unsigned n); +// 岛里,在有人把计数加宽之后 +extern "C" int saxpy_device(float a, const float* x, const float* y, + float* out, std::size_t n); +``` + +C 语言链接不做名字修饰,所以这两个是同一个符号。链接是干净的,而两侧各按自己的 ABI 解释 +参数:没有编译错误,没有链接错误,只有一次读过了参数末尾的运行。同样的错误发生在 C++ +边界上会被名字修饰在链接期挡下。 + +所以**迫使这条边界必须是 `extern "C"` 的那条性质 —— 两侧不共享 C++ ABI —— 正是让声明 +分裂无法被发现的同一条性质**。一个普通头文件是模块和设备编译器都能读的最小共同物,这就 +是它存在的全部理由。 + +三条替代都试过,没有一条能去掉它:岛不能包含接缝(`.cppm` 不是 nvcc 解析的东西);从某个 +单一来源生成头文件就是头文件多加一步;而模块的 global module fragment 什么都不导出,即使 +岛的编译器能读 BMI 也够不到。 + ### 两类 lane,只有一类的接口是生成的 上面那个接缝是手写的,而 shader 那条 lane 的对应物是生成的。这不是不一致 —— From 102cad684683678a26b7bc59e6c7586757a0d3c1 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 8 Sep 2026 00:03:48 +0800 Subject: [PATCH 10/15] docs/20: noun-phrase headings, which the style check requires `Is the extern "C" header required?` and its Chinese twin were question headings. The check names the rule and both files; the content is unchanged. --- docs/20-heterogeneous-builds.md | 2 +- docs/zh/20-heterogeneous-builds.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/20-heterogeneous-builds.md b/docs/20-heterogeneous-builds.md index 6630c71b5..d47299a15 100644 --- a/docs/20-heterogeneous-builds.md +++ b/docs/20-heterogeneous-builds.md @@ -147,7 +147,7 @@ links libstdc++ puts a second copy of the C++ runtime into a program whose own copy came from mcpp's toolchain. -#### Is the `extern "C"` header required? +#### The `extern "C"` header, and what omitting it costs Not by the language. The seam can declare the entry point itself and the island can define it, with no header anywhere, and that builds and links and runs. diff --git a/docs/zh/20-heterogeneous-builds.md b/docs/zh/20-heterogeneous-builds.md index 2afc9aea6..c889e77a0 100644 --- a/docs/zh/20-heterogeneous-builds.md +++ b/docs/zh/20-heterogeneous-builds.md @@ -118,7 +118,7 @@ tarball 已经发出去了。设备源文件必须被显式点名。 自身运行时来自 mcpp 工具链的程序里。 -#### 那个 `extern "C"` 头文件是必须的吗? +#### 那个 `extern "C"` 头文件,以及省掉它的代价 语言上不是。接缝可以自己声明入口点、岛自己定义它,一个头文件都不写,照样编得过、链得上、 跑得起来。 From 8b931eca1c7a2527ad9c7ce2542b41efa42d1159 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 8 Sep 2026 00:41:59 +0800 Subject: [PATCH 11/15] build.mcpp: a synthesis that cannot write its program refuses `run_build_program` returned success when the `std::ofstream` for the synthesised program failed to open. The caller reads that as "this package has no build program", so the device sources are handed to nobody, nothing is generated, and the build succeeds. The first symptom is an unresolved name in a consumer that imported an interface the program was to write -- three edges from the write that failed. Both the open and the close are checked: a stream that opened and then failed while flushing leaves a truncated program, which compiles into a different defect. --- src/build/build_program.cppm | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index a7a14aba1..f55753982 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -804,9 +804,20 @@ std::expected run_build_program( same = (prev == text); } if (!same) { + // REFUSED, not skipped. Returning "nothing to do" here would leave + // the device sources uncompiled and the build otherwise successful, + // and the first symptom would be an unresolved name in a consumer + // that imported the interface this program was to generate. std::ofstream out(src, std::ios::binary | std::ios::trunc); - if (!out) return {}; + if (!out) + return std::unexpected(std::format( + "cannot write the build program the rules describe: {}", + src.string())); out << text; + out.close(); + if (!out) + return std::unexpected(std::format( + "failed while writing the build program: {}", src.string())); } } From 77d4918c6892e8a388c91165d59300ac21fda840 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 8 Sep 2026 00:48:07 +0800 Subject: [PATCH 12/15] ci: correct why the Windows graphics run reaches no device Three places recorded a missing `vulkan-1.dll` as the cause of `render unavailable` on the Windows runner. The log refutes it. That line is printed by `src/main.cpp` after the render function returns nothing, and the Vulkan leg imports `vkCreateInstance` from `vulkan-1.dll` through the import library -- a process that could not find that DLL would fail during image load and print nothing. It printed. mcpp-index's own `vulkan-tests` member calls `vkEnumerateInstanceVersion` on the windows shards and passes, which says the same thing from the other side. The conclusion came from generalising the descriptor's note about STATIC linkage into a claim about building at all. Measured against that: the Khronos loader in `compat:vulkan` cross-builds into a working `vulkan-1.dll` from the source the index already carries -- 265 exports matching upstream's `vulkan-1.def` name for name, DllMain present, importing only ADVAPI32, CFGMGR32, KERNEL32 and msvcrt. So a hermetic Windows loader is available whenever it is wanted, and it is not what this step is waiting on. What is open is why the lavapipe payload's ICD enumerates no device under a process mcpp launched. The notes are corrected in place rather than deleted, so a reader sees what was believed and what refuted it. --- ...9-07-module-first-heterogeneous-surface.md | 57 ++++++++++++++----- .github/workflows/ci-windows.yml | 51 ++++++++++------- examples/10-graphics/offscreen/mcpp.toml | 17 +++--- 3 files changed, 85 insertions(+), 40 deletions(-) diff --git a/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md b/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md index 21181bd08..46aca9e20 100644 --- a/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md +++ b/.agents/docs/2026-09-07-module-first-heterogeneous-surface.md @@ -695,7 +695,7 @@ test that was seen to fail without the change. | E2/E3 | mcpp | A `[rules]` manifest section, and activating rules from the files present | **withdrawn**, see 13.1.2 | | P6 | mcpp-plugins | Rules pass `--depfile`, `-MD -MF` and `-depfile` | staged, needs E1 released | | X1 | openxlings/xim-pkgindex | Raise the `xim:slang` pin toward upstream `v2026.17` | staged | -| X2 | openxlings/xim-pkgindex | A Windows `vulkan-1.dll` package | staged, see 13.1.3 | +| X2 | mcpplibs/mcpp-index | `compat:vulkan` builds its Windows loader from source instead of expecting a host DLL | measured feasible, see 13.1.3; not what the Windows run step is waiting on | ### 13.1.1 A cost the implementation paid, stated rather than hidden @@ -737,22 +737,53 @@ feature spelling and no module name -- and a new device language costs no engine change. `.slang` was removed from the built-in table to prove that, and `tests/slang-consumer` builds unchanged. -### 13.1.3 What the Windows measurement changed +### 13.1.3 The Windows measurement, and the second reading that overturned it Raising the graphics example from "builds" to "runs" on Windows was expected to be two CI steps. It is not. The manifest declaration worked -- `Provisioning [xlings.workspace] entries (xim:mesa-lavapipe@26.2.0)` -- the ICD -was found in the store, and the program still printed `render unavailable`. - -`compat:vulkan` ships an import library on Windows and nothing else, and says -why in its own descriptor: a statically linked loader cannot work there, because -upstream's `loader_windows.c` creates its locks in `DllMain` and a static -library never gets one. The runtime `vulkan-1.dll` is expected to come from an -installed GPU driver, and a runner has none. - -So the platform stays at "builds" and the example declares no device there -- -an entry would download 56 MB that nothing can load. The gap is a Windows loader -package, which is X2. +manifest was found in the store, and the program printed `render unavailable`. + +The first reading of that attributed it to a missing loader: `compat:vulkan` +ships an import library on Windows and nothing else, and says why in its own +descriptor -- a statically linked loader cannot work there, because upstream's +`loader_windows.c` creates its locks in `DllMain` and a static library never +gets one. The runtime `vulkan-1.dll` was said to come from an installed GPU +driver, which a runner has none of. + +**That reading is wrong, and the log says so.** `render unavailable` is printed +by `src/main.cpp` after the render function returns nothing. The Vulkan leg +imports `vkCreateInstance` from `vulkan-1.dll` through the import library, so a +process that could not find that DLL would fail during image load and print +nothing at all. It printed. The loader was present, it ran, and it enumerated no +device -- which is a statement about the ICD, not about the loader. A second +fact stands with it: mcpp-index's own `vulkan-tests` member calls +`vkEnumerateInstanceVersion` on the windows shards and passes. + +Two things follow. + +**X2 is not what the CI step is waiting on.** A Windows loader package would +change nothing about `render unavailable`. What is unestablished is why the +lavapipe payload's ICD produces no device under a process mcpp launched, and +that is where the next measurement goes. + +**X2 is nonetheless available, and it costs no redistribution decision.** The +descriptor's note argues that a Windows loader must be a DLL, not that it cannot +be built. Measured: the Khronos loader in `compat:vulkan` cross-builds into a +working `vulkan-1.dll` from the source the index already carries -- 265 exports, +matching upstream's `vulkan-1.def` name for name, all `vk*`-prefixed so +`exports = ["vk*"]` reproduces the surface exactly; `DllMain` present; importing +only ADVAPI32, CFGMGR32, KERNEL32 and msvcrt. The earlier conclusion that the +only viable source was a third-party prebuilt binary came from generalising the +descriptor's note about STATIC linkage into a claim about building at all. A +package built this way makes the loader hermetic on Windows the way it already +is on Linux and macOS, and needs no new artifact hosted anywhere. + +The general form of the error is worth keeping: a recorded conclusion is +re-read, its reasoning is not. Both notes that carried it -- in +`.github/workflows/ci-windows.yml` and in the example's own manifest -- have +been corrected in place rather than deleted, so the next reader sees what was +believed and what refuted it. ### 13.2 Dependency structure across repositories diff --git a/.github/workflows/ci-windows.yml b/.github/workflows/ci-windows.yml index ad97d641c..dc11409e5 100644 --- a/.github/workflows/ci-windows.yml +++ b/.github/workflows/ci-windows.yml @@ -456,28 +456,41 @@ jobs: done echo "ok: both shader stages compiled and the Vulkan half linked" - # WINDOWS STAYS AT "BUILDS", AND THE REASON IS A MISSING PACKAGE RATHER - # THAN A MISSING CI STEP. + # WINDOWS STAYS AT "BUILDS", AND WHAT REMAINS OPEN IS THE DRIVER. # - # A run step was written, pushed, and measured. The manifest declaration - # worked -- `Provisioning [xlings.workspace] entries - # (xim:mesa-lavapipe@26.2.0)` -- and the ICD was found in the store, and - # the program still printed `render unavailable`, which is its own report - # that it enumerated no device. + # A run step was written, pushed, and measured (run 34135108981). The + # manifest declaration worked -- `Provisioning [xlings.workspace] entries + # (xim:mesa-lavapipe@26.2.0)` -- the ICD manifest was found in the store, + # and the program printed `render unavailable`, which is `src/main.cpp` + # reporting that the render function returned nothing. # - # The missing piece is the LOADER, not the driver. `compat:vulkan` ships - # an import library on Windows and nothing else, and says why in its own - # descriptor: a statically linked loader cannot work there, because - # upstream's `loader_windows.c` creates its locks in `DllMain` and a - # static library never gets one. The runtime `vulkan-1.dll` is expected - # to come from an installed GPU driver, and a GitHub Windows runner has - # none. `xim:mesa-lavapipe`'s Windows payload is an ICD; it imports system - # DLLs and carries no loader either. + # THAT OUTPUT PLACES THE FAILURE AFTER THE LOADER, NOT AT IT. The Vulkan + # leg imports `vkCreateInstance` from `vulkan-1.dll` through the import + # library, so a process missing that DLL fails during image load and + # prints nothing at all. It printed. The loader was there, it ran, and it + # enumerated no device -- which is a statement about the ICD. # - # So raising this job to "runs" needs a Windows `vulkan-1.dll` package, - # which is a packaging decision rather than a step in this file. macOS is - # not in the same position: `compat:vulkan` builds the loader from source - # everywhere except Windows, so that job does run. + # An earlier revision of this comment named a missing `vulkan-1.dll` as + # the cause. That was an inference from the descriptor's note about static + # linkage rather than a reading of the failure, and the log refutes it. + # Two facts stand against it: this program ran, and mcpp-index's own + # `vulkan-tests` member calls `vkEnumerateInstanceVersion` on the windows + # shards and passes. + # + # A separate measurement, recorded here because it removes a second thing + # from the list of suspects: the Khronos loader in `compat:vulkan` builds + # into a working `vulkan-1.dll` from the source the index already carries + # (mingw, 265 exports matching upstream's `vulkan-1.def` exactly, DllMain + # present, importing only ADVAPI32/CFGMGR32/KERNEL32/msvcrt). The + # descriptor's note argues that a Windows loader must be a DLL, not that + # it cannot be built -- so a hermetic Windows loader is available whenever + # it is wanted. It is not what this step is waiting on. + # + # What this step is waiting on is `xim:mesa-lavapipe`'s Windows payload + # producing a device under an mcpp-launched process. Until that is + # measured, the job builds the Vulkan half and runs the CPU fallback. + # macOS is not in the same position: MoltenVK enumerates once the instance + # asks for portability, which `src/vulkan/render.cpp` now does. - name: "Toolchain: LLVM — build mcpp (self-host)" shell: bash diff --git a/examples/10-graphics/offscreen/mcpp.toml b/examples/10-graphics/offscreen/mcpp.toml index 76b80eded..917fdd701 100644 --- a/examples/10-graphics/offscreen/mcpp.toml +++ b/examples/10-graphics/offscreen/mcpp.toml @@ -65,15 +65,16 @@ vulkan-runtime = "2026.09.07" # WINDOWS DECLARES NO DEVICE, AND THAT IS MEASURED RATHER THAN AN OVERSIGHT. # # `xim:mesa-lavapipe` publishes a Windows payload and it installs correctly, so -# an entry here would work in the sense that the bytes arrive. The program still -# reaches no device: `compat:vulkan` ships an import library on Windows and the -# runtime `vulkan-1.dll` is expected to come from an installed GPU driver, which -# a machine without one does not have. Declaring the driver would download 56 MB -# that nothing can load. +# an entry here would work in the sense that the bytes arrive. The program was +# built and run with it declared, and printed `render unavailable` -- the report +# `src/main.cpp` makes when the render function returns nothing. # -# The gap is a Windows loader package. Until there is one, this platform builds -# the Vulkan half and runs the CPU fallback, which is what the `cfg(not(...))` -# section below carries. +# The failure is after the loader rather than at it. This program imports +# `vkCreateInstance` from `vulkan-1.dll`, so one that could not find that DLL +# would fail during image load and print nothing; it printed. What has not been +# established is why the payload's ICD enumerates no device under a process mcpp +# launched. Until it is, this platform builds the Vulkan half and runs the CPU +# fallback, which is what the `cfg(not(...))` section below carries. [build] accel = "vulkan1.2" From 4e2b47633ec26bb89dcb87b40187f203a8ebbf60 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 8 Sep 2026 01:05:11 +0800 Subject: [PATCH 13/15] build.mcpp: a build program is told which package it is building Found by self-review of the module surface. Every name a rule package generates is derived from the package's name -- the module a consumer imports, the namespace the accessors sit in, the symbols in a generated header -- and nothing in the build-program contract answered that question. The closest available answer was the leaf of MCPP_MANIFEST_DIR, which is a directory name. The two differ whenever a project lays a package out under a generic folder, and `mcpp.rules.spirv` was measurably wrong there: `examples/09-heterogeneous/vulkan/app/` declares `name = "vulkan-saxpy"` and generated `app.shaders`, so every `/app/` in a workspace claimed one module. A user-facing name derived from a folder is also a name that changes when a folder is renamed. MCPP_PKG_NAME and MCPP_PKG_NAMESPACE, with `mcpp::package_name()` and `mcpp::package_namespace()` reading them. Set at both sites that build a BuildProgramEnv -- the dependency loop and the root; those two are the whole enumeration. The e2e distinguishes the two derivations by construction: the fixture's directory is `app` and its package is `vulkan-saxpy`, so an implementation that still read the directory writes `app` where the assertion expects `vulkan-saxpy`. A fixture whose name matched its folder would pass either way, which is what every existing one did. --- docs/07-build-mcpp.md | 2 + docs/zh/07-build-mcpp.md | 2 + src/build/build_program.cppm | 8 ++ src/build/hostprogram.cppm | 15 ++++ src/build/prepare.cppm | 4 + ..._build_program_is_told_its_package_name.sh | 82 +++++++++++++++++++ 6 files changed, 113 insertions(+) create mode 100755 tests/e2e/632_a_build_program_is_told_its_package_name.sh diff --git a/docs/07-build-mcpp.md b/docs/07-build-mcpp.md index 1b4b69133..7145b78c8 100644 --- a/docs/07-build-mcpp.md +++ b/docs/07-build-mcpp.md @@ -659,6 +659,8 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_CXX_STDLIB` *(2026.9.6.3+)* | `mcpp::cxx_stdlib()` | the C++ standard library the resolved toolchain uses — `libstdc++`, `libc++`, `msvc-stl`; empty when no toolchain resolved. A different question from `MCPP_TARGET_LIBC`, which is the C library | | `MCPP_ACCEL` *(2026.9.5.2+)* | `mcpp::accel()` | the device axis of this build, resolved — `--accel` / `--no-accel` over `[build] accel` — in the wire form `cuda12.9+{sm_89} ptx>=89`; empty when the build asks for no accelerator. A rule package derives its own flags (`-gencode`, `--offload-arch`) from it, so the architecture set is written once, in the manifest. The same value feeds the `cfg(accelerator = "…")` layer key | | `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | `1` when the declaring package sets `[language] modules`, `0` otherwise. A rule that GENERATES a consumer-facing declaration reads it to choose between a module interface and a header, so a project states that once and never again. An older engine leaves it absent, which a rule reads as `0` -- the behaviour every consumer had before the variable existed | +| `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | The `[package] name` of the package this program builds. Every name a rule generates is derived from it: the module a consumer imports, the namespace the accessors sit in, the symbols in a generated header. Before it existed the closest available answer was the leaf of `MCPP_MANIFEST_DIR`, which is a directory name -- so a package named `vulkan-saxpy` in a directory named `app` generated `app.shaders`, and every `/app/` in a workspace claimed the same module. Absent under an older engine, which a rule reads as "use whatever you used before" | +| `MCPP_PKG_NAMESPACE` *(2026.9.7.1+)* | -- | The `[package] namespace`. Empty when the package declares none. A rule that must produce a name unique across an index uses the pair rather than the name alone, because package identity is `(namespace, name)` | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) | diff --git a/docs/zh/07-build-mcpp.md b/docs/zh/07-build-mcpp.md index ea3bc5601..ec8683542 100644 --- a/docs/zh/07-build-mcpp.md +++ b/docs/zh/07-build-mcpp.md @@ -563,6 +563,8 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_CXX_STDLIB` *(2026.9.6.3+)* | `mcpp::cxx_stdlib()` | 解析出的工具链使用的 C++ 标准库 —— `libstdc++`、`libc++`、`msvc-stl`;没有工具链解析时为空串。与 `MCPP_TARGET_LIBC` 不是同一个问题,后者是 C 库 | | `MCPP_ACCEL` *(2026.9.5.2+)* | `mcpp::accel()` | 本次构建的设备轴,已解析 —— `--accel` / `--no-accel` 优先于 `[build] accel` —— 线上形态 `cuda12.9+{sm_89} ptx>=89`;不要加速器时为空串。规则包从它推导自己的开关(`-gencode`、`--offload-arch`),架构集合因此只在 manifest 写一次。同一个值也喂给 `cfg(accelerator = "…")` 这个 layer 键 | | `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | 声明它的那个包设了 `[language] modules` 时为 `1`,否则 `0`。**生成**面向消费者声明的规则读它来在模块接口与头文件之间选择,项目因此只需说一次。旧引擎不设这个变量,规则把缺席读作 `0` —— 也就是这个变量存在之前每个消费者的行为 | +| `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | 这个程序所构建的包的 `[package] name`。规则生成的每个名字都由它推导:消费者导入的模块、访问器所在的命名空间、生成头里的符号。在它存在之前,可用的最接近的答案是 `MCPP_MANIFEST_DIR` 的末段,那是目录名 —— 于是一个叫 `vulkan-saxpy` 的包放在名为 `app` 的目录下会生成 `app.shaders`,而工作区里每一个 `/app/` 都声称拥有同一个模块。旧引擎下缺席,规则把缺席读作「沿用你之前用的那个」 | +| `MCPP_PKG_NAMESPACE` *(2026.9.7.1+)* | -- | `[package] namespace`。包未声明命名空间时为空。需要产出在索引范围内唯一的名字的规则用这一对而不是单用名字,因为包身份是 `(namespace, name)` | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu`、`.hip`…),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根(= CWD) | diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index f55753982..bd30891a7 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -124,6 +124,12 @@ struct BuildProgramEnv { // (`-gencode`, `--offload-arch`) from here and the architecture set is // written once, in the manifest, and never again in a build program. std::string accel; + // The package this program is building, from `[package]`. Reported because + // every name a rule generates is derived from it -- the module a consumer + // imports, the namespace its accessors sit in -- and a build program had no + // way to ask. See hostprogram::package_name for what it replaced. + std::string packageName; + std::string packageNamespace; // Whether this package builds C++ modules (`[language] modules`). // // Reported because a rule package that GENERATES a consumer-facing @@ -522,6 +528,8 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv } e.emplace_back("MCPP_OUT_DIR", outDir.string()); e.emplace_back("MCPP_MANIFEST_DIR", root.string()); + e.emplace_back("MCPP_PKG_NAME", env.packageName); + e.emplace_back("MCPP_PKG_NAMESPACE", env.packageNamespace); std::string csv; for (auto const& f : env.features) { if (!csv.empty()) csv += ','; diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 564f97e7d..88e21b8d6 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -375,6 +375,21 @@ inline const char* target_libc_profile() { return env_or("MCPP_TARGET_L inline const char* target_libc() { return env_or("MCPP_TARGET_LIBC"); } inline const char* manifest_dir() { return env_or("MCPP_MANIFEST_DIR"); } +// THE PACKAGE THIS PROGRAM IS BUILDING, BY NAME. +// +// A rule package that generates a consumer-facing declaration has to name it, +// and every name it produces is derived from this one: the module a project +// imports, the namespace the accessors sit in, the symbols in a generated +// header. Before these existed the closest thing available was the leaf of +// `manifest_dir()`, which is a directory name rather than a package name -- +// so a package called `vulkan-saxpy` in a directory called `app` generated +// `app.shaders`, and every `/app/` in a workspace claimed it. +// +// Empty under an engine older than 2026.9.7.1, which a rule reads as "fall +// back to whatever you did before". That is what keeps an already-published +// rule package working unchanged. +inline const char* package_name() { return env_or("MCPP_PKG_NAME"); } +inline const char* package_namespace() { return env_or("MCPP_PKG_NAMESPACE"); } inline bool has_feature(const char* name) { char buf[256] = "MCPP_FEATURE_"; unsigned long o = 13; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 01d781998..bb0b1c9be 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -8133,6 +8133,8 @@ prepare_build(bool print_fingerprint, // The DECLARING package's setting, not the root project's: a rule // generating a declaration for this package must match how this // package is compiled. + bpEnv.packageName = pkg.manifest.package.name; + bpEnv.packageNamespace = pkg.manifest.package.namespace_; bpEnv.languageModules = pkg.manifest.language.modules; bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(pkg.root.string()); dit != deviceSourcesByPackage.end()) @@ -9059,6 +9061,8 @@ prepare_build(bool print_fingerprint, bpEnv.toolsBin = projectSubosBin; bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); + bpEnv.packageName = m->package.name; + bpEnv.packageNamespace = m->package.namespace_; bpEnv.languageModules = m->language.modules; bpEnv.ruleModules = m->buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(root->string()); dit != deviceSourcesByPackage.end()) diff --git a/tests/e2e/632_a_build_program_is_told_its_package_name.sh b/tests/e2e/632_a_build_program_is_told_its_package_name.sh new file mode 100755 index 000000000..ae7b283ee --- /dev/null +++ b/tests/e2e/632_a_build_program_is_told_its_package_name.sh @@ -0,0 +1,82 @@ +#!/usr/bin/env bash +# requires: gcc +# A BUILD PROGRAM IS TOLD WHICH PACKAGE IT IS BUILDING. +# +# Every name a rule package generates is derived from this one: the module a +# consumer imports, the namespace the accessors sit in, the symbols in a +# generated header. Until 2026.9.7.1 nothing answered it, and the closest thing +# available was the leaf of `MCPP_MANIFEST_DIR` -- a DIRECTORY name. +# +# THE DIRECTORY AND THE PACKAGE ARE DELIBERATELY DIFFERENT HERE, and that is the +# whole test. A fixture whose package name happens to equal its directory leaf +# passes against both the old derivation and the new one, so it would assert +# nothing. `mcpp.rules.spirv` shipped with exactly that defect: an example laid +# out as `vulkan/app/` with `name = "vulkan-saxpy"` generated `app.shaders`, and +# every `/app/` in a workspace claimed the same module. +# +# The answer is written to a FILE rather than printed, because mcpp shows a +# build program's stdout only when it exits non-zero -- a criterion reading the +# build log would be measuring the failure path. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# The directory is `app`. The package is not. +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +int main() { return 0; } +EOF + +cat > app/build.mcpp <<'EOF' +#include +#include +import mcpp; +int main() { + std::string out = std::string(mcpp::manifest_dir()) + "/answered.txt"; + std::FILE* f = std::fopen(out.c_str(), "w"); + if (f == nullptr) return 3; + std::fprintf(f, "name=%s\n", mcpp::package_name()); + std::fprintf(f, "namespace=%s\n", mcpp::package_namespace()); + std::fclose(f); + return 0; +} +EOF + +cat > app/mcpp.toml <<'EOF' +[package] +name = "vulkan-saxpy" +namespace = "example" +version = "0.1.0" + +[build] +sources = ["src/*.cpp"] + +[targets.vulkan-saxpy] +kind = "bin" +main = "src/main.cpp" +EOF + +cd app +"${MCPP:-mcpp}" build > build.log 2>&1 || { echo "FAIL: build"; cat build.log; exit 1; } + +[ -f answered.txt ] || { echo "FAIL: the build program wrote no answer"; exit 1; } +cat answered.txt + +grep -qx 'name=vulkan-saxpy' answered.txt || { + echo "FAIL: package_name() did not answer the [package] name" + echo " (a directory-derived answer would read 'app')" + exit 1; } +grep -qx 'namespace=example' answered.txt || { + echo "FAIL: package_namespace() did not answer the [package] namespace" + exit 1; } + +# THE REVERSE LEG: the directory leaf is `app`, so an implementation that still +# derived from the directory would have written `app` above. Assert the two are +# actually different in this fixture, or the check above proves nothing. +[ "$(basename "$PWD")" = "app" ] || { + echo "FAIL: this fixture no longer distinguishes the two derivations" + exit 1; } + +echo "PASS: a build program reads its package identity, not its directory name" From c17f7360654a341b6664552ab9aa63071f213d7f Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 8 Sep 2026 01:06:39 +0800 Subject: [PATCH 14/15] docs/07: the two new rows state the fallback without addressing the reader The style check refuses the second person in a reference document. Both rows said what a rule reads an absent variable as by addressing it directly; they now state it. --- docs/07-build-mcpp.md | 2 +- docs/zh/07-build-mcpp.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/07-build-mcpp.md b/docs/07-build-mcpp.md index 7145b78c8..bcbc508d7 100644 --- a/docs/07-build-mcpp.md +++ b/docs/07-build-mcpp.md @@ -659,7 +659,7 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_CXX_STDLIB` *(2026.9.6.3+)* | `mcpp::cxx_stdlib()` | the C++ standard library the resolved toolchain uses — `libstdc++`, `libc++`, `msvc-stl`; empty when no toolchain resolved. A different question from `MCPP_TARGET_LIBC`, which is the C library | | `MCPP_ACCEL` *(2026.9.5.2+)* | `mcpp::accel()` | the device axis of this build, resolved — `--accel` / `--no-accel` over `[build] accel` — in the wire form `cuda12.9+{sm_89} ptx>=89`; empty when the build asks for no accelerator. A rule package derives its own flags (`-gencode`, `--offload-arch`) from it, so the architecture set is written once, in the manifest. The same value feeds the `cfg(accelerator = "…")` layer key | | `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | `1` when the declaring package sets `[language] modules`, `0` otherwise. A rule that GENERATES a consumer-facing declaration reads it to choose between a module interface and a header, so a project states that once and never again. An older engine leaves it absent, which a rule reads as `0` -- the behaviour every consumer had before the variable existed | -| `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | The `[package] name` of the package this program builds. Every name a rule generates is derived from it: the module a consumer imports, the namespace the accessors sit in, the symbols in a generated header. Before it existed the closest available answer was the leaf of `MCPP_MANIFEST_DIR`, which is a directory name -- so a package named `vulkan-saxpy` in a directory named `app` generated `app.shaders`, and every `/app/` in a workspace claimed the same module. Absent under an older engine, which a rule reads as "use whatever you used before" | +| `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | The `[package] name` of the package this program builds. Every name a rule generates is derived from it: the module a consumer imports, the namespace the accessors sit in, the symbols in a generated header. Before it existed the closest available answer was the leaf of `MCPP_MANIFEST_DIR`, which is a directory name -- so a package named `vulkan-saxpy` in a directory named `app` generated `app.shaders`, and every `/app/` in a workspace claimed the same module. Absent under an older engine, which a rule reads as a signal to keep its previous derivation | | `MCPP_PKG_NAMESPACE` *(2026.9.7.1+)* | -- | The `[package] namespace`. Empty when the package declares none. A rule that must produce a name unique across an index uses the pair rather than the name alone, because package identity is `(namespace, name)` | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp | diff --git a/docs/zh/07-build-mcpp.md b/docs/zh/07-build-mcpp.md index ec8683542..6b41e926c 100644 --- a/docs/zh/07-build-mcpp.md +++ b/docs/zh/07-build-mcpp.md @@ -563,7 +563,7 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_CXX_STDLIB` *(2026.9.6.3+)* | `mcpp::cxx_stdlib()` | 解析出的工具链使用的 C++ 标准库 —— `libstdc++`、`libc++`、`msvc-stl`;没有工具链解析时为空串。与 `MCPP_TARGET_LIBC` 不是同一个问题,后者是 C 库 | | `MCPP_ACCEL` *(2026.9.5.2+)* | `mcpp::accel()` | 本次构建的设备轴,已解析 —— `--accel` / `--no-accel` 优先于 `[build] accel` —— 线上形态 `cuda12.9+{sm_89} ptx>=89`;不要加速器时为空串。规则包从它推导自己的开关(`-gencode`、`--offload-arch`),架构集合因此只在 manifest 写一次。同一个值也喂给 `cfg(accelerator = "…")` 这个 layer 键 | | `MCPP_LANGUAGE_MODULES` *(2026.9.7.1+)* | -- | 声明它的那个包设了 `[language] modules` 时为 `1`,否则 `0`。**生成**面向消费者声明的规则读它来在模块接口与头文件之间选择,项目因此只需说一次。旧引擎不设这个变量,规则把缺席读作 `0` —— 也就是这个变量存在之前每个消费者的行为 | -| `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | 这个程序所构建的包的 `[package] name`。规则生成的每个名字都由它推导:消费者导入的模块、访问器所在的命名空间、生成头里的符号。在它存在之前,可用的最接近的答案是 `MCPP_MANIFEST_DIR` 的末段,那是目录名 —— 于是一个叫 `vulkan-saxpy` 的包放在名为 `app` 的目录下会生成 `app.shaders`,而工作区里每一个 `/app/` 都声称拥有同一个模块。旧引擎下缺席,规则把缺席读作「沿用你之前用的那个」 | +| `MCPP_PKG_NAME` *(2026.9.7.1+)* | -- | 这个程序所构建的包的 `[package] name`。规则生成的每个名字都由它推导:消费者导入的模块、访问器所在的命名空间、生成头里的符号。在它存在之前,可用的最接近的答案是 `MCPP_MANIFEST_DIR` 的末段,那是目录名 —— 于是一个叫 `vulkan-saxpy` 的包放在名为 `app` 的目录下会生成 `app.shaders`,而工作区里每一个 `/app/` 都声称拥有同一个模块。旧引擎下缺席,规则把缺席读作「沿用先前的推导」 | | `MCPP_PKG_NAMESPACE` *(2026.9.7.1+)* | -- | `[package] namespace`。包未声明命名空间时为空。需要产出在索引范围内唯一的名字的规则用这一对而不是单用名字,因为包身份是 `(namespace, name)` | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu`、`.hip`…),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 | From 553ce9c10c4253f5d01333ef75ff0dfb75108c59 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 8 Sep 2026 01:12:49 +0800 Subject: [PATCH 15/15] CHANGELOG: the entry describes what shipped, not what was tried first Three corrections to the 2026.9.7.1 entry. The `.slang` bullet described the first design -- adding the extension to the engine's built-in table. That inverted the dependency: a general build system would hold a list of an external plugin's languages, and a third-party rule would still wait on an engine release. What shipped is `device_extensions` and `rule_module` on the rule package's own feature, with `.slang` REMOVED from the engine table and `tests/slang-consumer` building unchanged as the criterion. The synthesised `build.mcpp` was not mentioned at all. The graphics section claimed the example declares a device under `cfg(windows)` and that two CI steps run it. Neither is true: Windows declares none and the step was withdrawn. The corrected text also records why -- the failure is after the loader, not at it. --- CHANGELOG.md | 90 +++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 71 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 378952762..6393e6b4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,10 +7,10 @@ ## [2026.9.7.1] - 2026-09-07 -### 三条通道,都是「规则包知道而引擎收不到」的形状 +### 四条通道,都是「规则包知道而引擎收不到」的形状 -这一版加的三样东西各自很小,共同点是它们补的都是同一类缺口:规则包已经知道某件事, -而没有任何通道把它送到引擎的决定上。 +这一版加的四样东西各自很小,共同点是它们补的都是同一类缺口:一方已经知道某件事, +而没有任何通道把它送到需要它的另一方。 **`mcpp::action` 增加 `depfile`。** action 的输入在 `build.mcpp` 运行时就定死了,那时 命令还没执行。一个靠解析源码才知道自己 `#include` 图的编译器,因此没有办法把结果报回 @@ -23,17 +23,50 @@ **不要同时把 depfile 声明为 `output()`**:`deps = gcc` 会让 ninja 读完即删,一条承诺了 该输出的边会永远是脏的。 -**`.slang` 进入设备源扩展名表。** 受限 glob 的 `accel` 键**不**决定一个文件是不是设备 -源,`kDeviceExtensions` 才决定。所以在此之前,一个规则包无法自己引入一门设备语言: +**规则包自己声明它编译哪些扩展名(`device_extensions`),以及提供规则的模块 +(`rule_module`)。** 受限 glob 的 `accel` 键**不**决定一个文件是不是设备源, +`kDeviceExtensions` 才决定。所以在此之前,一个规则包无法自己引入一门设备语言: `xim:slang` 装得上、规则也编得过,而文件仍然掉进普通源集,报的是「mcpp has no role for -the extension '.slang'」。Slang 是一门语言而不是 GLSL 的第二个驱动 —— 它有自己的模块 -系统、泛型,以及超出 SPIR-V 的目标集合 —— 所以它有自己的扩展名,外面也有自己的规则。 +the extension '.slang'」。 + +第一版的修法是把 `.slang` 加进引擎那张表。那是**方向反了的依赖**:通用构建系统会因此 +持有一份外部插件的语言清单,而第三方插件想引入一门语言仍要等一次引擎发布。最终的形状 +是两个键写在规则包**自己**的 feature 上: + + [features.rules-slang] + sources = ["rules/slang.cppm"] + device_extensions = [".slang"] + rule_module = "mcpp.rules.slang" + +引擎因此不持有任何包名、feature 拼写或模块名。两个键必须同时出现:一个声明了扩展名却 +不说谁来编译它的 feature 会被拒绝,而不是让文件在后面某处静默掉队。 + +判据是直接的:`.slang` 已从 `kDeviceExtensions` **移除**,而 `tests/slang-consumer` +照常构建与运行。一门新设备语言不再需要引擎发一个版本。 + +**`[rules]` 声明了规则的项目不必写 `build.mcpp`。** 规则模块被声明之后,那个程序的内容 +就是确定的 —— 导入这些模块、依次调用 `compile()`、有一条失败就返回非零。mcpp 把它写进 +构建目录。项目要接管就把它拷到根目录改,合成随即停止:声明是 `build.mcpp` **之上**的 +一层,不是它旁边的第二套机制。 **`[language] modules` 以 `MCPP_LANGUAGE_MODULES` 报给构建程序。** **生成**面向消费者 声明的规则要在「模块接口」与「头文件」之间做选择,而项目已经声明过它用哪一种;用别的 方式推导就是同一个决定的第二种拼法。旧引擎不设这个变量,规则把缺席读作「头文件」—— 也就是这个变量存在之前每个消费者的行为,所以升级不需要任何项目改一个字。 +**包身份以 `MCPP_PKG_NAME` / `MCPP_PKG_NAMESPACE` 报给构建程序。** 规则包生成的每一个 +名字都由包名推导 —— 消费者导入的模块、访问器所在的命名空间、生成头里的符号 —— 而构建 +程序的契约里没有任何东西回答「我在构建哪个包」。此前可用的最接近的答案是 +`MCPP_MANIFEST_DIR` 的末段,那是**目录名**。 + +两者在「包放在一个通用目录下」时不同,而这正是本仓库自己的布局: +`examples/09-heterogeneous/vulkan/app/` 声明 `name = "vulkan-saxpy"`, +`mcpp.rules.spirv` 生成的却是 `app.shaders` —— 工作区里每一个 `/app/` 都会 +声称拥有同一个模块,而且这个面向用户的名字会随着目录改名而改变。 + +配套的 e2e 按构造区分这两种推导:夹具的目录叫 `app`、包叫 `vulkan-saxpy`。名字与目录 +相同的夹具对两种实现都通过 —— 而在此之前每一个夹具都是这样。 + ### 两处只在 Linux 之外成立的缺陷,以及第一条图形管线 **引擎与规则层早就与平台无关,而生态只在 Linux 上完整。** 把规则包在另外两个平台上 @@ -128,18 +161,37 @@ loader 默认不把它交给 `vkEnumeratePhysicalDevices`:实例要同时启用 Linux 上跑翻译层的机器也有它,macOS 上对着原生驱动构建的程序并不需要它。Linux/lavapipe 上读数不变,中心像素仍是 `(124, 70, 62, 255)`,设备名仍是 `llvmpipe`。 -**示例本身补齐了另外两个平台的设备声明** —— `cfg(macos)` 下 `xim:moltenvk`,`cfg(windows)` -下 `xim:mesa-lavapipe`。这个缺口在示例只构建不运行时是看不见的:不运行的程序从不向 -loader 要设备。声明写在示例里而不是 CI 步骤里,是为了让**任何人**检出它都能跑,而不只是 -那台多跑了一条命令的 runner。 - -于是两个新 CI 步骤**什么都不装**:它们找构建已经供给的 ICD 并设 `VK_DRIVER_FILES`,所以 -一份没能声明驱动的 manifest 会让步骤变红。步骤里放 `xlings install` 会让它两种情况都通过。 - -两个平台断言的东西不同,而这个差别正是重点。lavapipe 是软件光栅化器,像素由构造保证相同, -所以图像分不出它,设备名才分得出。MoltenVK 是宿主自己的 GPU 经 Metal,名字随 runner 变, -所以那边区分「够到了设备」的是**报出了中心像素**这件事本身 —— 一个被 loader 拒绝展示的 -可移植性驱动不会产生它。 +**示例本身补齐了 macOS 的设备声明** —— `cfg(macos)` 下 `xim:moltenvk`。这个缺口在示例 +只构建不运行时是看不见的:不运行的程序从不向 loader 要设备。声明写在示例里而不是 CI +步骤里,是为了让**任何人**检出它都能跑,而不只是那台多跑了一条命令的 runner。 + +于是新的 macOS CI 步骤**什么都不装**:它找构建已经供给的 ICD 并设 `VK_DRIVER_FILES`, +所以一份没能声明驱动的 manifest 会让步骤变红。步骤里放 `xlings install` 会让它两种情况 +都通过。 + +Linux 与 macOS 断言的东西不同,而这个差别正是重点。lavapipe 是软件光栅化器,像素由构造 +保证相同,所以图像分不出它,设备名才分得出。MoltenVK 是宿主自己的 GPU 经 Metal,名字随 +runner 变,所以那边区分「够到了设备」的是**报出了中心像素**这件事本身 —— 一个被 loader +拒绝展示的可移植性驱动不会产生它。 + +**Windows 停在「构建」,而停在这里的理由被测量推翻过一次。** 运行步骤写过、推过、测过: +`xim:mesa-lavapipe` 装上了,ICD 清单在 store 里找到了,程序打印 `render unavailable` —— +那是 `src/main.cpp` 在渲染函数什么都没返回时的报告。 + +第一次读把它归因成缺少 loader。这个归因是错的,日志本身就说明了:Vulkan 那条腿经导入库 +从 `vulkan-1.dll` 取 `vkCreateInstance`,一个找不到该 DLL 的进程会在映像加载期失败、 +一个字都印不出来。它印出来了。loader 在,跑了,枚举不到设备 —— 这是关于 ICD 的陈述。 +另一侧还有一个事实站在一起:mcpp-index 自己的 `vulkan-tests` 成员在 windows 分片上调用 +`vkEnumerateInstanceVersion` 且通过。 + +顺带排掉了另一个嫌疑:`compat:vulkan` 里的 Khronos loader 能从索引已经携带的源码交叉 +构建出可用的 `vulkan-1.dll` —— 265 个导出,与上游 `vulkan-1.def` 逐名吻合;`DllMain` +在;只导入 ADVAPI32、CFGMGR32、KERNEL32 与 msvcrt。描述符那段注记论证的是「Windows 上 +的 loader 必须是 DLL」,不是「它构建不出来」。所以这个包随时可以有,只是它不是这个步骤 +在等的东西。 + +留待测量的是:lavapipe 的 Windows 载荷为什么在 mcpp 启动的进程里枚举不出设备。这一条的 +一般形式值得记下 —— 被复查的是结论,理由不会。 ### 文档