MOCA — Micro Orchestrator for Cloud Agents

Run stateful AI coding agents on a dense, warm process-manager runtime — sessions resume exactly where they left off.

MOCA runs long-lived AI agents as warm, dense sessions on a process-manager runtime (P6). An agent process normally has to stay resident — holding its conversation, tool state, and working directory in memory — while it waits for the next turn or for a human to approve a step. MOCA decouples the agent's state (durable in Redis) and its tool execution (an isolated remote sandbox — a container, or a Firecracker microVM per Exec) from the agent process itself, so one supervisor hands connections to a fixed pool of warm workers: many sessions per worker, no cold-start on the next request.

Capacity scales the boring way: add process-manager pods, or size them up — on a VM, bare metal, or plain Kubernetes. The earlier scale-to-zero Knative + KEDA deployment is deprecated (ADR-0041); see Migrating off the Knative/KEDA deployment.

The result is a leaf-session backend: an external orchestrator dispatches isolated units of agent work ("leaves") over a simple HTTP + shared-volume contract — sync turns, detachable streaming turns, and human-gated runs, with async fan-out and scheduled dispatch owed P6 follow-ups (ADR-0034).

Maturity

Labels apply to each deployment path, not to the project as a whole (definitions in the documentation map).

Path	Label	Notes
Docker Compose	Preview	Five-minute trial, not production
Single VM	Preview	The production path for one host
P6 on Kubernetes / OpenShift	Preview	Same process model as ordinary cluster workloads; first target for Stable
P4 microVM sandboxes	Preview	Per-Exec Firecracker VMs on a KVM host; known limits listed on that page
Knative + KEDA	Legacy	Deprecated 2026-10-11; removal no earlier than end of October 2026

Criteria for Stable and the support policy are in Maturity and lifecycle. Still on Knative? Say so, or say you depend on async fan-out or the CronJob, in the tracking issue: the removal date waits for you.

Table of Contents

• Maturity
• Why
• Architecture
• Features
• Quick Start
• Legacy: Knative + KEDA deployment
• How It Works
• Dispatch Archetypes
• Repository Layout
• Evidence
• Roadmap
• Documentation
• Status & License

---

Why

Persistent agent	MOCA
Process stays resident between turns	Warm worker pool — many sessions per worker, no cold-start
State lives in process memory — lost on crash/evict	State lives in Redis — survives eviction, restart, and resume
Tools execute in the agent process	Tools execute in an isolated remote sandbox (brain/hands split)
One process per session	Many sessions per worker; scale by adding pods or sizing up
One invocation model	Sync, detachable streaming, human-gated

The economics measurement (experiment, run on the now-deprecated scale-to-zero path) showed the serverless shape consuming roughly a quarter of the pod-seconds of an always-on agent. P6 trades scale-to-zero for density — many sessions per worker with admission control — and keeps what that experiment bought on the latency side: no autoscaler cold-start between turns.

---

Architecture

flowchart LR
    O[External Orchestrator] -->|POST /turn| V[Supervisor<br/>L4 socket hand-off]
    V --> W1[worker<br/>session + model loop]
    V --> W2[worker<br/>session + model loop]
    W1 --> R[runLeaf / executeTurn]
    W2 --> R
    R -->|gRPC relay| S[remote-worker sandbox<br/>container or microVM per Exec]
    R <--> Q[(Redis<br/>session state + work queue)]

Loading

Component	Role
Supervisor	L4-only listener; hands each accepted socket to a warm worker and never touches a response byte (ADR-0034)
Worker pool	Fixed set of forked workers; many sessions per worker, bounded concurrent turns, admission control instead of autoscaling
Redis	Durable session state (resume by sessionId), work queue (Streams), gate state
remote-worker sandbox	Tool/code execution in a container — or a Firecracker microVM per Exec (ADR-0035) — dialed in over the gRPC relay (ADR-0024); needs no inbound route
Control plane	Multi-user identity and credentials (MU1); issues per-session tokens (ADR-0033)

Note: The supervisor and its workers are the same container image: the supervisor forks packages/knative-server/src/worker.ts (the package name is historical — the code is P6-load-bearing). The "brain" (model inference + session logic) runs in a worker; the "hands" (command/tool execution) always run in the sandbox.

---

Features

• Warm, dense sessions — a fixed worker pool serves many sessions each; no activator cold-start between turns, and capacity scales with pods, not autoscalers.
• Durable resume — sessions are append-only logs in Redis; a restarted worker recalls full conversation and state by sessionId, surviving process or pod death.
• Brain/hands isolation — the agent never executes tools in its own process; everything runs in a remote sandbox over the gRPC relay — container tier, or a microVM per Exec.
• Human-in-the-loop gates — a leaf can pause mid-run, report paused, and resume on an external approve/reject/abort verdict.
• Volume-envelope contract — orchestrators pass inputs and collect results as files, decoupling result size from HTTP limits.
• Hardened by default — non-root, read-only root filesystem, dropped capabilities where the substrate allows; isolation is enforced by the sandbox tier, not by trust.
• Built on Pi — wraps a pinned kagenti/pi coding agent through an injectable SessionStorageBackend seam; the agent itself is unmodified.
• Promote a local Claude Code workflow — mocactl promote DIR bundles your local skills and commands into a content-addressed bundle that a session or a leaf runs with, uploaded through the control plane (see the guide).

---

Quick Start

Try the P6 runtime in five minutes, no root, with Docker Compose: the supervisor, the relay, Redis and a sandbox on one machine.

Prerequisites: docker (or a compatible runtime) with Compose, curl, and an Anthropic-compatible model credential.

curl -fsSL https://raw.githubusercontent.com/rossoctl/moca/main/deploy/compose/install.sh \
  | ANTHROPIC_API_KEY=sk-ant-... sh

Verify it works, then stop it (Redis state goes with it):

curl -s -H 'Content-Type: application/json' -d '{"prompt":"Run uname -a and tell me the kernel."}' \
  http://127.0.0.1:8080/turn
cd ~/.serverless-harness && docker compose down

Details, configuration and the multi-user control plane are in the Compose README. To use the terminal client against a control plane, install mocactl (Node 22+):

curl -fsSL https://raw.githubusercontent.com/rossoctl/moca/main/scripts/install-mocactl.sh | sh

Where to go next:

• Choose an install path: docs/get-started/.
• Production, a dedicated VM or bare metal: deploy/vm/.
• Stay on Kubernetes or OpenShift: deploy/k8s/.
• Guided tour: docs/demos/vm-multi-user-demo.md: two users, one VM, two sandbox tiers (~40 min). All demos are indexed in docs/demos/.

---

Legacy: the Knative + KEDA deployment (deprecated)

The original scale-to-zero deployment under deploy/knative/ is deprecated as of 2026-10-11 (ADR-0041): security and breakage fixes only, removal no earlier than end of October 2026 and only once its remaining users have migrated. See Migrating off the Knative/KEDA deployment.

---

How It Works

1. An orchestrator POSTs a turn to /turn (or a leaf to /runs).
2. The supervisor accepts the connection and hands the socket to a warm worker — it never touches a response byte.
3. The worker runs the model loop; every tool call executes in a remote sandbox over the gRPC relay — container tier, or a microVM per Exec.
4. Session state streams to Redis as it goes, so the turn is resumable by sessionId even if the worker dies mid-run.
5. Capacity scales by adding process-manager pods or sizing them up; admission control (bounded concurrent turns per worker) replaces autoscaling.

---

Promoting a local Claude Code workflow

Iterate on skills and commands locally in Claude Code, then promote them with mocactl promote DIR (or /promote DIR inside the mocactl TUI). The bundle is built and credential-scanned on your machine, uploaded through the control plane with no cluster credentials, and fixed to the session you start with mocactl run --config DIGEST. Full guide: docs/guides/promote-claude-workflow.md.

---

Dispatch Archetypes

The same backend serves three orchestration patterns, all validated end-to-end on Kind:

Archetype	Pattern	Example use case
A — Async fan-out	{async:true} → Redis Streams → drained by KEDA ScaledJobs — deprecated path only	"Research 10 topics concurrently"
B — Human gate	Leaf pauses (paused) → external verdict → resume/terminate	"Draft a clause, pause for legal sign-off, finalize"
C — Scheduled	CronJob → cron-dispatch posts async leaves — deprecated path only	"Summarize yesterday's tickets at 02:00 daily"

B is served by the P6 runtime today. A and C have no P6 equivalent yet — the async drainer and the scheduled on-ramp are owed follow-ups flagged in ADR-0034 and ADR-0041; they work only on the deprecated Knative/KEDA path.

---

Repository Layout

moca/
├── harness/               # Pi SessionStorageBackend adapter (write-behind) + promote CLI
├── packages/
│   ├── supervisor/        # P6 process manager: L4 listener, socket hand-off, worker pool
│   ├── knative-server/    # HTTP server + P6 worker entry points (name is historical)
│   ├── k8s-sandbox/       # Sandbox transports (K8s pod-exec half deprecated, ADR-0041)
│   ├── session-backend/   # Generic append-only LogStore + Redis Streams impl
│   ├── work-queue/        # Redis Streams work queue (async dispatch)
│   ├── sandbox-relay/     # gRPC relay between control plane and remote-worker sandboxes
│   ├── control-plane/     # Multi-user control plane, credential issuance
│   ├── config-bundle/     # Promoted config bundle loading
│   ├── ibac-stub/         # IBAC stub
│   └── mocactl/           # MOCA terminal client for the control plane
├── remote-worker/         # Go: container + microVM sandbox workers
├── deploy/
│   ├── vm/                # P6 single-VM systemd deployment (production)
│   ├── k8s/               # P6 on Kubernetes/OpenShift (first-class)
│   ├── compose/           # Docker Compose trial of the P6 runtime
│   ├── microvm/           # P4 Firecracker microVM deployment
│   └── knative/           # Knative/KEDA deployment — DEPRECATED (ADR-0041)
├── pi-fork/               # Pinned Pi coding agent (submodule)
└── docs/                  # Docs map, guides, specs, ADRs, demos, migration guide

---

Evidence

Behaviour and economics are backed by reproducible experiments rather than claims:

• moca-experiments/knative/EXPERIMENTS.md — cluster experiments E1 (economics), E3 (mobility), E4 (recovery), run live on Kind. Moved to a separate repo 2026-09-25 (see docs/specs/2026-09-24-ra1-density-cutover-and-repo-rearchitecture-design.md).
• docs/archive/experiment-results.md: E2 (reconstruction cost) and E5 (budget enforcement), a dated write-up from June 2026. The code these ran from now lives in moca-experiments/experiments/.
• deploy/knative/SMOKE.md: the deprecated Knative path's smoke record, kept as history.

---

Roadmap

The milestone registry is the source of truth for the status of every milestone, including the process manager (P6), the microVM tier (P4), the multi-user control plane (MU), and the zero-trust credential plane (Z track), whose parts are at different stages. Open questions and planned documentation work are in docs/REORG-PLAN.md.

---

Documentation

Start at the documentation map.

I want to...	Go to
Pick an install path	Get started
Leave the Knative/KEDA deployment	Migration guide
Run it for several users	Multi-user guide
Attach a sandbox on another host	Remote sandbox guide
Promote a Claude Code workflow	Guide
Look up an environment variable	Configuration reference
Look up a mocactl command	mocactl reference
Know what changes need notice	Change policy
Understand the security model	Security model, hardening checklist
Operate it (Kubernetes)	Operations
Understand how it works	Concepts
Look up a term	Glossary
Call the HTTP API	HTTP API guide, leaf envelope, docs/api/openapi.yaml, docs/api/harness-openapi.yaml, Context service
Follow a guided walkthrough	Demos
See why a decision was made	ADRs, milestone registry
Build or contribute	CONTRIBUTING.md, harness/README.md, docs conventions

---

Status & License

This is explorative work and every deployment path except the deprecated Knative one is Preview (see Maturity). The warm-worker-pool runtime, durable resume, sandbox isolation (container and microVM tiers), and the multi-user control plane are built. Async fan-out and scheduled dispatch exist only on the deprecated path and have no P6 equivalent yet. Interfaces may change.

Licensed under the Apache License 2.0.