build.mcpp: four channels a rule package had no way to reach the engine through - #587
Merged
Merged
Conversation
四期(软件设备把 CI 上限抬到运行)的程序侧那一半。 接设备之前先测了上游产物,而第一行的读数就是本条的理由:Khronos 官方的 `MoltenVK-macos.tar` v1.4.2 里,ICD 带着 **`"is_portability_driver": true`**。 **loader 默认不把可移植性驱动交给 `vkEnumeratePhysicalDevices`** —— 一个照着原生 驱动写的 Vulkan 程序因此在 macOS 上一个设备都找不到,并把它报成「这台机器没有 GPU」。诊断是错的,而它看起来完全合理。 规范要的是成对的两半:实例要**启用** `VK_KHR_portability_enumeration` **并且**置位 `VK_INSTANCE_CREATE_ENUMERATE_PORTABILITY_BIT_KHR`;随后凡是声明了 `VK_KHR_portability_subset` 的设备,必须在 `vkCreateDevice` 时启用它,否则调用失败。 按**能力**问,不按 `#ifdef __APPLE__`:性质是「我面前这个 loader 在展示可移植性 驱动」,而 `#ifdef` 两个方向都会错 —— Linux 上跑翻译层的机器也有它,macOS 上对着 原生驱动构建的程序并不需要它。 实测:Linux/lavapipe 上读数不变,中心像素仍是 `(124, 70, 62, 255)`。 顺带记下 Windows 那一侧的测量:`mesa3d-26.2.0-release-msvc` 的 `vulkan_lvp.dll` **只导入系统 DLL**(无 MSVCP140/VCRUNTIME140),静态 CRT,机器上不需要装任何东西。 载荷侧(把 MoltenVK 与 Windows 的 lavapipe 收进索引)未做:那要往发布组织推新的二进制 资产,而 GitCode 的资产不可替换不可删除。
…ine through Each of these is small, and they share a shape: a rule package already knew something and nothing carried it to a decision the engine makes. `mcpp::action` gains `depfile`. An action's inputs are fixed when build.mcpp runs, before its command has executed, so a compiler that learns its include graph by parsing the source has no channel to report it: editing a file the command merely READ rebuilds nothing and `mcpp build` stays green over a stale artifact. With a depfile declared, ninja reads the file and folds what it names into the edge. Every device compiler this matters for already emits one -- `glslangValidator --depfile`, `glslc -MD -MF`, `slangc -depfile`, `nvcc` and `clang` `-MD -MF` -- so the missing half was always the receiving end. The depfile must not also be declared as an output: `deps = gcc` makes ninja consume and delete it, so an edge that promised it would be permanently dirty. The comment at the emission site says so. `.slang` joins the device-source extension table. A constrained glob's `accel` key does not make a file a device source; `kDeviceExtensions` does. Until it listed `.slang`, a rule package could not introduce a device language on its own: the payload installed, the rule compiled, and the file still fell through to the ordinary source scan and was refused with "mcpp has no role for the extension". Measured against 2026.9.6.6 with a working rule and a working `xim:slang`. Slang is a language rather than a second driver for GLSL -- its own module system, generics, and targets beyond SPIR-V -- so it has an extension here and a rule of its own outside. `[language] modules` is reported as `MCPP_LANGUAGE_MODULES`. A rule that GENERATES a consumer-facing declaration has to choose between a module interface and a header, and the project has already stated which it builds; deriving it any other way would be a second spelling of one decision. An engine that does not report it leaves the variable absent, which a rule reads as "header" -- the behaviour every consumer had before -- so the upgrade needs no project to declare anything. Tests. `tests/e2e/631` builds a project whose action writes a depfile naming a file it never declares as an input, and asserts the action reruns when that file is touched. It carries its own control: a no-op rebuild must not rerun the action, which rules out a backend that reruns unconditionally. A separate reverse control was run by hand against the same binary with `a.depfile` removed and the flag still passed: the action did not rerun, so the assertion measures the field rather than some other trigger.
It read `grep -q "GENERATE" b3.log && FAIL`, on the assumption that ninja's description line for the edge reaches that log. It does not. `mcpp build` passes `--quiet` to ninja whenever it is not `--verbose` and surfaces ninja's captured stdout only on failure, so a from-scratch build -- which unquestionably runs the action -- prints no such line either. The grep never matched, and the check could not fail whether the action reran or not. Found while designing the depfile test, whose first draft copied this convention and would have inherited the same emptiness. The generated file's modification time measures the thing being asserted directly: the action rewrites it whenever it runs. Verified in both directions against the same binary -- unchanged after touching an unrelated source, and the assertion fails naming both timestamps when the touched file is a declared input of the action instead.
macOS and Windows built it and did not run it, which is the shape the cross-platform work exists to remove: the half of a lane written for a host is the half that host never exercises. A build asserts that the shader compiler this platform publishes works and that the Vulkan half links. It cannot assert that the loader hands the program a device, and on macOS that is the interesting half. The example now declares a device for those two platforms as well -- `xim:moltenvk` under `cfg(macos)`, `xim:mesa-lavapipe` under `cfg(windows)` -- rather than leaving CI to install one. The omission was invisible while the example was never run, because a program that does not run never asks the loader for a device, and declaring it here is what makes the example complete for anyone who checks it out rather than only for the runner that had an extra command. The two CI steps therefore install nothing. They locate the ICD the build already provisioned and set `VK_DRIVER_FILES`, so a manifest that failed to name the driver fails the step. An `xlings install` in the step would have made it pass either way. macOS and Windows assert different things, and the difference is the point. lavapipe is a software rasteriser producing the same pixels by construction, so the image cannot distinguish it and the device name is what does. MoltenVK is the host's own GPU through Metal, so its name differs by runner; what distinguishes reaching a device there is that a centre pixel was reported at all, which a portability driver the loader declined to show would not produce. This closes the dependency between the two halves of the portability change: the package without the enumeration finds no device, and the enumeration without the package has nothing to find.
Sunrisepeak
force-pushed
the
feat/vulkan-portability-enumeration
branch
from
September 7, 2026 14:50
0cbeee3 to
1a3fa43
Compare
…missing package A run step was written, pushed and measured. The manifest declaration worked -- `Provisioning [xlings.workspace] entries (xim:mesa-lavapipe@26.2.0)` -- the ICD was found in the store, and the program still printed `render unavailable`, which is its own report that it enumerated no device. The missing piece is the LOADER, not the driver. `compat:vulkan` ships an import library on Windows and nothing else, and its own descriptor says why: a statically linked loader cannot work there, because upstream's `loader_windows.c` creates its locks in `DllMain` and a static library never gets one. The runtime `vulkan-1.dll` is expected to come from an installed GPU driver, and a GitHub Windows runner has none. `xim:mesa-lavapipe`'s Windows payload is an ICD that imports system DLLs and carries no loader either. So the example declares no device on Windows: an entry there would download 56 MB that nothing can load. macOS is not in the same position -- `compat:vulkan` builds the loader from source everywhere except Windows -- and keeps its run step. Raising this platform to "runs" needs a Windows `vulkan-1.dll` package, which is a packaging decision rather than a step in a workflow file. Both the workflow and the example manifest record the measurement where someone looking for the gap will find it.
…sts no engine change
Two keys on a feature, and the engine holds no package name, no feature
spelling and no module name:
[features.rules-slang]
sources = ["rules/slang.cppm"]
rule_module = "mcpp.rules.slang"
device_extensions = [".slang"]
`device_extensions` classifies those extensions as device sources in a consumer
that activates the feature. `rule_module` is what a build program imports to
reach the rule. Two things follow.
A NEW DEVICE LANGUAGE NO LONGER TOUCHES THE ENGINE. Adding `.slang` to the
built-in table cost an engine change, a release, and a version bump in the rule
package's CI before the rule could route one file. `.slang` is removed from that
table here and `rules-slang` declares it instead; `tests/slang-consumer` builds
and runs unchanged, which is the only honest test of whether the mechanism
carries a language. The built-in list is now what mcpp knows without being told
-- a compatibility set for languages whose support shipped before the
declaration existed -- rather than a registry a sixth backend joins.
A CONSUMER WRITES ONE EDGE AND NO BUILD PROGRAM. `host-module = true` is implied
by `rule_module`, because a feature naming one has already said that is the only
way to use it. And a package with no `build.mcpp` gets the program its rules
describe written into the build directory:
[build-dependencies.mcpp]
plugins = { version = "0.3.0", features = ["rules-spirv"] }
is the whole declaration in `tests/spirv-zero-config`, which compiles a shader
and reaches it through a generated module. A project that writes its own
`build.mcpp` keeps it: synthesis fills an absence and never overrides.
THE FEATURE IS STILL REQUESTED BY NAME. An earlier revision derived it from the
extensions a project's sources carried, so a consumer could name the package
alone. That was withdrawn for two reasons and neither was cost. Two packages may
claim one extension -- a third-party CUDA rule is a thing someone will write --
and derivation would then guess or refuse where `features = [...]` has already
said which. And a manifest's job is to describe the build: a derived feature set
is information the file no longer states, which is worse for a reader and worse
for anything reading the manifest as context.
Which rules ran is said out loud, for the same reason the resolved toolchain is:
Rules mcpp.rules.spirv (mcpp:plugins)
Two ordering defects were found by running it rather than by reading it. The
collection must sit between feature activation and the extension table that
narrows the constrained globs; placed after, the declared extensions arrived too
late to classify anything and the rule was handed an empty list. And the guard
on the build-program call asked whether the FILE existed while the function
asked whether a program was WANTED, which left the synthesis unreachable. Both
now ask the same question, and the comments say what the failure looked like.
…withdrawn docs/05 gains the rule-feature section in both languages: what `device_extensions` and `rule_module` state, the two things that follow (`host-module` implied, a build program written where there is none), and why the feature is still requested by name. docs/20's device extension table is now described as what mcpp knows without being told rather than as the registry a sixth backend joins, with a note pointing at the declaration that adds to it. `.slang` has left the table. The design document records the two designs that were written and withdrawn -- a `[rules]` section and activating rules from the files present -- with the objections that retired them, so the next reader does not rediscover the same two shapes.
…tten The seam is written by hand and the shader lane's equivalent is generated, which reads as an inconsistency until the reason is stated. A device translation unit is code, and its interface is a design decision no generator makes well; a shader is data, and its interface is an address and a size. Both are already invisible to a consumer -- only the seam includes the C header, and everything downstream imports the module -- so the code lanes have been module-first all along. The shader lane was the exception until 2026.9.7.1, where the generated header WAS the interface. Added in both languages.
…costs Not required by the language: the seam can declare the entry point and the island can define it, with no header, and that builds and runs. What the header buys is that the declaration exists once. Stated because the failure it prevents is the worst one available here. C language linkage does not mangle, so two copies that disagree are one symbol: the link is clean and each side reads the arguments by its own ABI. The property that forces this boundary to be extern "C" is the same property that makes a split declaration undetectable, and the three alternatives that would remove the header are listed with why none does. Both languages.
`Is the extern "C" header required?` and its Chinese twin were question headings. The check names the rule and both files; the content is unchanged.
`run_build_program` returned success when the `std::ofstream` for the synthesised program failed to open. The caller reads that as "this package has no build program", so the device sources are handed to nobody, nothing is generated, and the build succeeds. The first symptom is an unresolved name in a consumer that imported an interface the program was to write -- three edges from the write that failed. Both the open and the close are checked: a stream that opened and then failed while flushing leaves a truncated program, which compiles into a different defect.
Three places recorded a missing `vulkan-1.dll` as the cause of `render unavailable` on the Windows runner. The log refutes it. That line is printed by `src/main.cpp` after the render function returns nothing, and the Vulkan leg imports `vkCreateInstance` from `vulkan-1.dll` through the import library -- a process that could not find that DLL would fail during image load and print nothing. It printed. mcpp-index's own `vulkan-tests` member calls `vkEnumerateInstanceVersion` on the windows shards and passes, which says the same thing from the other side. The conclusion came from generalising the descriptor's note about STATIC linkage into a claim about building at all. Measured against that: the Khronos loader in `compat:vulkan` cross-builds into a working `vulkan-1.dll` from the source the index already carries -- 265 exports matching upstream's `vulkan-1.def` name for name, DllMain present, importing only ADVAPI32, CFGMGR32, KERNEL32 and msvcrt. So a hermetic Windows loader is available whenever it is wanted, and it is not what this step is waiting on. What is open is why the lavapipe payload's ICD enumerates no device under a process mcpp launched. The notes are corrected in place rather than deleted, so a reader sees what was believed and what refuted it.
Found by self-review of the module surface. Every name a rule package generates is derived from the package's name -- the module a consumer imports, the namespace the accessors sit in, the symbols in a generated header -- and nothing in the build-program contract answered that question. The closest available answer was the leaf of MCPP_MANIFEST_DIR, which is a directory name. The two differ whenever a project lays a package out under a generic folder, and `mcpp.rules.spirv` was measurably wrong there: `examples/09-heterogeneous/vulkan/app/` declares `name = "vulkan-saxpy"` and generated `app.shaders`, so every `<something>/app/` in a workspace claimed one module. A user-facing name derived from a folder is also a name that changes when a folder is renamed. MCPP_PKG_NAME and MCPP_PKG_NAMESPACE, with `mcpp::package_name()` and `mcpp::package_namespace()` reading them. Set at both sites that build a BuildProgramEnv -- the dependency loop and the root; those two are the whole enumeration. The e2e distinguishes the two derivations by construction: the fixture's directory is `app` and its package is `vulkan-saxpy`, so an implementation that still read the directory writes `app` where the assertion expects `vulkan-saxpy`. A fixture whose name matched its folder would pass either way, which is what every existing one did.
…eader The style check refuses the second person in a reference document. Both rows said what a rule reads an absent variable as by addressing it directly; they now state it.
Three corrections to the 2026.9.7.1 entry. The `.slang` bullet described the first design -- adding the extension to the engine's built-in table. That inverted the dependency: a general build system would hold a list of an external plugin's languages, and a third-party rule would still wait on an engine release. What shipped is `device_extensions` and `rule_module` on the rule package's own feature, with `.slang` REMOVED from the engine table and `tests/slang-consumer` building unchanged as the criterion. The synthesised `build.mcpp` was not mentioned at all. The graphics section claimed the example declares a device under `cfg(windows)` and that two CI steps run it. Neither is true: Windows declares none and the step was withdrawn. The corrected text also records why -- the failure is after the loader, not at it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four channels a rule package had no way to reach the engine through, the layer
above
build.mcppthose channels make possible, and the graphics example's halfof raising CI from "builds" to "runs".
Each of the four is small. What they have in common is the shape of the gap: one
side already knew something, and nothing carried it to the side that had to
decide.
mcpp::actiongainsdepfileAn action's inputs are fixed when
build.mcppruns, before its command hasexecuted. A compiler that learns its include graph by parsing the source
therefore has no channel to report it: editing a file the command merely READ
rebuilds nothing, and
mcpp buildstays green over a stale artifact.a.depfile = dep.c_str(); a.arg("--depfile").arg(dep.c_str());mcpp emits
depfile =anddeps = gccfor that edge. Every device compilerthis matters for already emits one --
glslangValidator --depfile,glslc -MD -MF,slangc -depfile,nvccandclang-MD -MF-- so themissing half was always the receiving end.
The depfile must not also be a declared output:
deps = gccmakes ninja consumeand delete it, so an edge promising it would be permanently dirty. Documented at
the field, at the decoder and at the emission site.
A rule package declares the extensions it compiles, and the module that does it
A constrained glob's
accelkey does not make a file a device source;kDeviceExtensionsdoes. Until this change a rule package could not introducea device language on its own: measured against 2026.9.6.6 with a working rule
and a working
xim:slang, the payload installed, the rule compiled, and the filestill fell through to the ordinary source scan --
The first version of this branch fixed that by adding
.slangto the engine'stable. That inverts the dependency: a general build system would hold a list
of an external plugin's languages, and a third-party rule -- someone else's CUDA
rule, someone else's shader rule -- would still wait on an engine release. The
shape that shipped puts both facts on the rule package's own feature:
The engine holds no package name, no feature spelling and no module name. The
two keys are required together: a feature that claims an extension without
naming who compiles it is refused, rather than letting the file fall out
somewhere later.
The criterion is direct.
.slangwas removed fromkDeviceExtensions, andmcpp-plugins'tests/slang-consumerbuilds and runs unchanged. A new devicelanguage now costs no engine release.
Collected after feature activation and before the extension table that narrows
constrained globs, and that position is load-bearing: placed after it, the
declared extensions classify nothing, the device source list comes out empty,
the rule is handed no files, and the failure surfaces three edges away as
failed to read compiled module. Measured.A project that declares its rules need not write
build.mcppOnce the rule modules are declared, the program's content is determined: import
them, call
compile()on each, return non-zero if any refused. mcpp writes itinto the build directory.
A project that takes it over copies the file to its root and edits it, and
synthesis stops -- the declaration is a layer ABOVE
build.mcpp, not a secondmechanism beside it. A package that already has one keeps it, because two
programs submitting the same actions is a submission nobody asked for.
A synthesis that cannot write its program now refuses. Returning "nothing to do"
left the device sources handed to no one and the build otherwise successful,
with the first symptom an unresolved name in a consumer three edges away.
[language] modulesis reported asMCPP_LANGUAGE_MODULESA rule that GENERATES a consumer-facing declaration has to choose between a
module interface and a header, and the project has already said which it builds.
Deriving it any other way would be a second spelling of one decision.
An engine that does not report it leaves the variable absent, which a rule reads
as "header" -- the behaviour every consumer had before -- so the upgrade needs no
project to declare anything.
A build program is told which package it is building
MCPP_PKG_NAMEandMCPP_PKG_NAMESPACE, read throughmcpp::package_name()and
mcpp::package_namespace().Every name a rule generates is derived from the package's name -- the module a
consumer imports, the namespace the accessors sit in, the symbols in a generated
header -- and nothing answered that question. The closest available answer was
the leaf of
MCPP_MANIFEST_DIR, a DIRECTORY name, and the two differ whenever apackage sits under a generic folder. This repository is such a case:
examples/09-heterogeneous/vulkan/app/declaresname = "vulkan-saxpy"andmcpp.rules.spirvgeneratedapp.shaders, which every<something>/app/in aworkspace would also claim -- and which changes when a folder is renamed.
The e2e distinguishes the two derivations by construction: its fixture's
directory is
appand its package isvulkan-saxpy. A fixture whose namematched its folder passes against either implementation, which is what every
existing one did.
Portability drivers are invisible by default
The Vulkan loader does not hand portability drivers to
vkEnumeratePhysicalDevicesunless the instance both enablesVK_KHR_portability_enumerationand sets the matching create-info flag. Theoffscreen example now does, so the macOS job runs the render on MoltenVK rather
than only building it, and the example itself declares the driver under
cfg(macos)-- a checkout gets a machine that can run it, not only the runnerthat had an extra command.
Asked as a capability rather than as
#ifdef __APPLE__: the property is "thisloader is showing me a portability driver", which a Linux machine running a
translation layer also has and a macOS build against a native driver does not.
Linux/lavapipe readings are unchanged, centre pixel
(124, 70, 62, 255), devicellvmpipe.Windows stays at "builds", and the recorded reason was wrong
The run step was written, pushed and measured.
xim:mesa-lavapipeinstalled, theICD manifest was found in the store, and the program printed
render unavailable--src/main.cppreporting that the render function returnednothing.
An earlier revision of this branch recorded a missing
vulkan-1.dllas thecause. The log refutes it: the Vulkan leg imports
vkCreateInstancefrom thatDLL through the import library, so a process that could not find it would fail
during image load and print nothing at all. It printed. mcpp-index's own
vulkan-testsmember callsvkEnumerateInstanceVersionon the windows shardsand passes, which says the same thing from the other side.
A separate measurement removes a second suspect: the Khronos loader in
compat:vulkancross-builds into a workingvulkan-1.dllfrom the source theindex already carries -- 265 exports matching upstream's
vulkan-1.defname forname,
DllMainpresent, importing only ADVAPI32, CFGMGR32, KERNEL32 and msvcrt.The descriptor's note argues that a Windows loader must be a DLL, not that it
cannot be built. So a hermetic Windows loader is available whenever it is
wanted, and it is not what this step is waiting on.
What is open is why the lavapipe payload's ICD enumerates no device under a
process mcpp launched. Both notes that carried the wrong cause are corrected in
place rather than deleted, so a reader sees what was believed and what refuted
it.
What is not here
examples/09-heterogeneousmoving to the generated boundary and the modulepayload surface, which consumes
mcpp:plugins 0.3.0. That version needs areleased mcpp to build against, and mcpp's own CI resolves the plugin collection
from the index, so those changes cannot be green until this ships. They are
written, verified on this host against the plugins worktree -- cuda and sycl
build and run on an RTX 4080 with the generated boundary on the island's own
command line, vulkan reaches its SPIR-V as a module -- and parked on
feat/examples-module-surface.