provider: verify CAPE contract against real CLI and fix run face + endpoint discovery - #157
Merged
Conversation
…dpoint discovery
The CAPE provider shipped with an assumed CLI contract. Every verb,
flag, and output shape has now been verified item by item against the
CAPE source, and the mismatches are fixed.
What matched: the run/status/logs/cancel verb set; --controller-url/
--token on every verb; the run flag table (--image, --gpus,
--cpu-cores, --memory-gb, --gpu-mode whole, --isolation-policy,
--runtime-adapter, --max-duration-seconds, --session-key, --bind,
--env, `-- <workload>`); "stdout is exactly one request id" without
--wait (real ids are req-%06d; the regex is tightened to req-\d{6,});
the status JSON on stdout with the terminal-state set {COMPLETED,
FAILED, CANCELLED, EXPIRED, LOST, INFEASIBLE}, a request_id echo, and
no stdout/stderr fields; cancel --reason with idempotent terminal
no-op; token passable only as --token argv (no env/file support in the
CLI); the serial session model.
What did not match (submit-fatal argv bugs): --task-id is required by
the real parser and was never passed (now = sandbox id); --pool/--user
are required but were optional (now checked at first use, fail-fast);
--workspace does not exist (argparse rc=2 "unrecognized arguments");
--cwd pointed at a directory no component creates, so the command
would die at process start.
The architectural finding: CAPE has no streaming logs. `cape logs`
succeeds for RUNNING requests but returns empty output until the
command exits (the node agent reports stdout/stderr once, after
reaping the process), and there is no native endpoint query. The old
stdout-marker + logs-polling discovery could therefore never see the
marker of a never-exiting runtime server: every create() would burn
the full 600s budget and fail. Discovery is redesigned around a
marker file through a rw bind: a new required meta_root config names a
submitter-and-node-visible directory (same shared-FS assumption as
bundle); each sandbox gets <meta_root>/<sandbox_id> bound at
/agentix-meta; the boot script now does `mkdir -p <ws> && cd <ws>`,
writes `AGENTIX_ENDPOINT <host> <port>` to /agentix-meta/endpoint, and
execs the bundle bootstrap; the provider polls the host side of the
marker file interleaved with `cape status` terminal-state checks, and
fetches logs only on terminal states (where they are populated) for
diagnostics. delete() removes the meta dir after a confirmed cancel,
and treats a controller 404 on cancel as already-gone so bookkeeping
cannot stick forever after a journal-less controller restart.
Docs now state the host-network constraint: the client-assigned-port
scheme needs apptainer/bubblewrap (host network stack); CAPE's podman
adapter has no host networking, so podman-only pools cannot work.
The fake cape in the tests now models the real contract (argparse-
mirrored run parser rejecting --workspace and requiring --pool/--user/
--task-id with rc=2; req-%06d ids; logs empty while RUNNING; cancel
404 stderr JSON) and really executes the boot script under sh -c with
a stubbed node environment, so discovery is tested against a marker
file the actual script wrote.
Validated against a real local CAPE deployment (controller + node
join, local-process adapter, auth enabled): create() submitted a
request the real parser accepted, discovery found the marker while
real `cape logs` returned empty, GET /health answered 200 on the
discovered URL, and delete() drove the request to CANCELLED, reaped
the workload process group, and removed the meta dir - 11/11 e2e
assertions passed. The local-process adapter ignores bind specs, so
that run exercises the boot script's documented host-path fallbacks;
container-adapter bind behavior still needs a Linux pool.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #155. The CAPE provider's assumed CLI contract has now been verified line-by-line against the real CAPE source, and the mismatches are fixed. Every item on the README's verification checklist has a source-level answer.
What matched (no change needed):
status/logs/cancelargv faces, single-line bare request-id stdout, terminal-state set,request_idecho cross-check,--token-argv-only auth,--bind/--envsyntax, the serial one-request-per-session model.What didn't — fixed here:
cape runrequires--task-id(now sent) and has no--workspaceflag (removed; the workspace is created by the boot script viamkdir -p && cd, and--cwdis dropped since no adapter creates missing dirs).pool/user/meta_rootnow fail fast when unset (the real parser exits 2 otherwise).cape logsreturns empty stdout until the workload exits, so stdout-marker + logs-polling could never converge for a long-lived runtime server. Discovery now uses a per-sandbox rw-bound marker file (meta_rootconfig, mounted at/agentix-meta; the boot script writeshost portthere before exec'ing the bundle bootstrap; the provider polls the host side, interleaved withcape statusterminal checks, and fetches logs only on terminal states for diagnostics).cancelthat reports an unknown request (404) is treated as already-gone so bookkeeping can't stick after a journal-less controller restart; request-id regex tightened toreq-\d{6,}; podman's lack of host networking (client-assigned-port scheme needs apptainer/bubblewrap) documented.Testing
caperewritten to model the real argparse table (rejects--workspace, requires--pool/--user/--task-id,req-%06dids, empty logs while RUNNING, 404 stderr JSON on unknown cancel) and actually executes the boot script undersh -c.uv run pytest tests/ plugins/ -q→ 718 passed;ruffclean;pyright0 errors;uv lock --checkcurrent.provider.create()→ marker-file discovery →GET /health200 →delete()→ request CANCELLED, meta dir removed, port released — 11/11 assertions PASS.🤖 Generated with Claude Code