feat(webcam): port segmentation to the Linux compositor back-end

[EtienneLescot]

The shader half already ran on all three back-ends — layer.wgsl declares @binding(4), the
layout requires it, and make_bind bound a dummy there. What Linux lacked was everything that
produces a mask to bind. It now has it, so the feature runs identically on wgpu, D3D11 and Metal,
and supportsWebcamSegmentation in RightPanes.tsx is gone: it existed so nobody was offered a
setting that does nothing, and with the last back-end landed it admitted every platform.

capture_webcam_rgb renders the camera into a 256x144 RGBA target through the existing layer
pipeline and reads it back. It passes the WHOLE valid frame, not the sub-rect being drawn — a
tight user crop would amputate the subject at the model's input, and the mask would be wrong
exactly where it matters. set_webcam_mask uploads R8 through Queue::write_texture, which
imposes no row alignment; linux_frames already writes NV12 planes through it with swscale's SIMD
strides. pump_segmentation is the Windows body verbatim — worker, inbox, rate limiter and lazy
start from scene.webcam_effect.model_path are platform-independent.

Three things are specific to this back-end

Row padding. copy_texture_to_buffer demands a bytes_per_row aligned to 256, where D3D11
reports a driver row pitch and Metal accepts the bare width. The capture pads its stride and
depads on read. At the shipped 256 px the padding is nil — 1024 is already aligned — which is
precisely why the depad branch is dead code at the size that actually runs, and why it gets a test
at a width that needs it (100 px: 400 bytes of payload in a 512-byte stride). A wrong depad does
not produce noise but a shear of 28 px per row.

The readback reuses ReadbackRing's lesson rather than its code. The ring exists to AVOID
waiting, and this capture must wait, since the frame it reads is the worker's input for this tick.
What it takes is WaitForSubmissionIndex and never Maintain::Wait, the regression the ring's
header records at 3.8–6.2 ms per frame.

pump_segmentation is called behind a scene check in compose_frame, not only inside itself.
It re-checks anyway, but on this back-end nv12_srvs ALLOCATES two TextureViews per call (there
is no cache — clear_srv_cache is a documented no-op), so merely reaching the function would cost
two allocations a frame on every project with no effect. The feature has to cost zero when off,
and here that is a call-site property.

fx and color are carried onto the webcam LayerCB, and effect_code stays 0 until a mask has
actually been uploaded — otherwise cutout renders an invisible webcam for the first frames, at
every editor launch. The PiP drop shadow is suppressed in cutout mode: a shadow cast by a box that
is no longer there reads as an artefact.

The shader contract (fx.z mode, fx.w intensity, fx.xy valid extent, mask sampled at
uv / fx.xy), the 25-tap background blur, and segmentation.rs are untouched.

Second commit: shipping the runtime on Linux

Rebased onto #530, which pinned every ONNX Runtime target but wired only build:mac,
build:win and build:win:store — for a stated reason: the Linux back-end carried the shader
and nothing that feeds it, so the library would have been 23 MB of installer for a code path that
could not run. "Landing the Linux capture half makes it a one-line change," said the script. This
is that line.

It also closes a gap the first commit opens. Removing supportsWebcamSegmentation makes the
control visible on Linux, and a control is only worth showing where a mask can actually reach the
shader; without the library staged, a packaged Linux build would offer the setting and do nothing
with it — precisely what the gate existed to prevent. The two changes have to travel together.

CI stages it for the Linux compositor job as #530 did for macOS, and for the same reason: with no
library on ORT_DYLIB_PATH every segmentation test returns early and the suite goes green having
exercised no inference at all. Verified rather than assumed — node scripts/fetch-onnxruntime.mjs --target=linux-x64 passes the SHA-256 check before unpacking and all three post-extraction checks
(MIT LICENSE, ELF magic, version string), and the whole suite runs green against that file
rather than a hand-installed one. No new pin and no new digest: the linux-x64 entry and the
api-27 cross-check already existed and are untouched.

Verification

cargo test -p openscreen-compositor --lib --tests — 190 tests, 0 failures, both in CI's exact
configuration (OPENSCREEN_REQUIRE_CPU_BACKEND=1, no ONNX Runtime) and with the runtime present.
The napi addon builds.

compositor_linux::tests mirrors compositor_macos::tests, all rendering real pixels, all
readable without ONNX Runtime because the mask is posted by hand. Two things differ, both forced by
the host: Gpu::create_auto rather than the hardware-strict Gpu::create, because the Linux CI
runner has no GPU and the strict constructor would make every one of these tests skip on the only
machine that runs them automatically; and the frames are built by hand, packing a
linux_frames::VkFrameTex carrier into AVFrame::data[0] exactly as CpuFrames::attach_carrier
does, so they still go through the real nv12_srvs.

Each wiring claim was checked by mutation, not just by a green run. Zeroing fx on the webcam
layer fails compose_frame_cuts_the_camera_out_once_a_mask_exists with 0 % of the camera cut;
dropping the shadow condition fails the_pip_shadow_is_suppressed_in_cutout_mode; rebinding the
dummy at binding 4 fails three tests including the_mask_actually_cuts_the_camera_out.

What I actually saw on screen

poc-d3d is cfg(windows), so the --cfg C8 --scene bench that proved the Windows path does not
exist here. seg_visual_renders_the_four_modes_from_a_real_photo is the substitute, opt-in behind
OPENSCREEN_SEG_VISUAL + OPENSCREEN_SEG_CAM. Run on a real GPU (Radeon 610M, RADV) with real
ONNX inference: cutout removes the PiP box and its shadow, the screen shows through;
custom background paints the colour where the mask says background, shadow kept; blur
blurs the background and leaves the subject sharp. The surviving fragments land at identical
coordinates in cutout and custom mode, which cross-checks the uv / fx.xy alignment.

Then on an actual photograph — a CC BY-SA selfie from Wikimedia Commons, cropped to 16:9 so the
model gets the aspect a webcam would give it. A deliberately hard case: locs against a dim indoor
wall, and dark clothing against a dark background.

What the port owes, it delivers. The kept region lands exactly on the subject — no mirror, no
offset, no shear — the custom colour replaces precisely the wall, and the mode behaviours are
right (cutout drops the box and its shadow, custom and blur keep the shadow, blur smears the wall
detail while the face stays sharp).

What the mask itself does is the model's business, and it is visibly imperfect. It is clean on
skin and along the jaw, but soft and lossy on the locs: outer strands are classified as background,
the replacement colour bleeds between strands rather than following them, and the subject is
trimmed at the frame edges (a rim of background colour along the left and top-left). This is the
MediaPipe 256x144 selfie model behaving as webcam-segmentation.md already describes — the same
model, resolution and shader contract as Windows and macOS, so it is not a property of this port.
Worth knowing before anyone reads a hair artefact on Linux as a Linux bug.

The photo and the renders are not attached or committed: third-party CC BY-SA material of an
identifiable person, used locally as a test input only.

Not measured

The capture adds a second synchronous readback to the preview path, which already pays one at
depth 1. On the export path, a GPU queue retires in order, so waiting on this submission also waits
on the previous frame's copy that the ring was keeping in flight. Bounded — 30 Hz, and zero with no
effect requested — but no §C.2 campaign has been run. Documented on the function and under Still
open in webcam-segmentation.md.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: de95c47d-99aa-4099-967a-9be813a4deb7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}