Skip to content

Add compile-time @Layout() graph auto-layout seam (v0.1.29) - #74

Merged
Volv-G merged 1 commit into
masterfrom
piforge/tangle-pipeline-crud/layout-annotation-intent-and-occ-772ea4c
Oct 3, 2026
Merged

Volv-G merged 1 commit into
masterfrom
piforge/tangle-pipeline-crud/layout-annotation-intent-and-occ-772ea4c

Conversation

@Volv-G

@Volv-G Volv-G commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator

(AI-assisted)

What

This PR adds compile-time graph auto-layout for Python-authored pipelines and bumps the version to 0.1.29.

@Layout("banded")                  # this graph and its undecorated subgraphs
@pipeline("Daily Pulse")
def daily_pulse(...): ...

@Layout(recursive=False)           # this graph only; algorithm=None = the transform's default
@pipeline("Judge")
def judge(...): ...

@Layout() only declares which graphs to lay out. A caller-supplied layout_transform does the actual layout, so the written root and subgraph YAML contain editor.position annotations:

compile_pipeline(script, out, layout_transform=my_engine)   # also PipelineCompiler.compile_file / pipelines.compile_pipeline_file

Tangle CLI ships no layout algorithm and has no algorithm-name allowlist. A downstream distribution (Discovery's tangle-deploy) installs its own engine as the transform.

New public API, exported from tangle_cli.python_pipeline and re-exported by tangle_cli.pipeline_compiler: Layout, GraphLayoutContext, GraphLayoutTransform, TaskInterface, InvalidLayoutError.

Behaviour

  • Coverage. An explicit declaration always applies to its own graph.
    • recursive=True (the default) passes it down to undecorated descendants. recursive=False covers only that graph.
    • A descendant's own @Layout(...) wins, and its own recursive flag governs further down.
    • @Layout() resets to the transform's default algorithm.
    • An undecorated root lays out only its decorated descendants.
  • Variants. A shared child is compiled once per (existing compile key, layout policy).
    • The same child reached under two different policies (A and B) becomes two correctly laid-out sidecars with distinct filenames. The same policy still dedups, including diamonds and repeated calls.
    • Uncovered children keep their existing sidecar filenames.
    • Cycle detection uses the layout-free key, so variants cannot mask a cycle.
  • Transform contract.
    • The transform is called once per covered artifact, post-order, after references are final and before validation and writing. It receives a private deep copy, and the result it returns is copied again.
    • GraphLayoutContext provides the governing Layout, the first-occurrence task-ID path (() for the root), the pipeline name, task_interfaces and artifact_dir.
    • A guard rejects any change other than string editor.position values on graph tasks and top-level inputs/outputs. Creating, replacing (which overwrites .with_position) and removing positions is allowed.
    • A CompileError from the transform propagates as-is; any other exception is wrapped. On any failure nothing is written.
  • Interfaces. TaskInterface(inputs, outputs, approximate) gives ordered port names.
    • @task and subpipeline tasks get their exact interface, unioned with observed names.
    • Opaque ref/@registered tasks get every supplied argument (edges and literals) plus the outputs other tasks consume, including through isEnabled, marked approximate=True.
    • Nothing is hydrated or fetched remotely, and interface data is never written back.
  • No transform, no change. Without a layout_transform (plain tangle sdk pipelines compile), or without @Layout(), compiled bytes, filenames and keys are identical to before. CompileResult.warnings notes each @Layout() that was not applied.
  • Validation without echoing values.
    • algorithm must be None or an exact non-empty str, positional or keyword. recursive must be an exact bool.
    • These raise InvalidLayoutError (a CompileError): a bare @Layout, a duplicate declaration, or a non-graph target (a @task, a subpipeline(...) handle, a class, a partial, a PipelineFn subclass, …).
    • Both stacking orders around @pipeline work and return the target unchanged. The intent lives on the PipelineFn, so imported and pre-imported/cached children keep it.

How

  • python_pipeline/layout.py (new): the immutable Layout, the context and protocol types, and TaskInterface.
  • pipeline.py adds PipelineFn.layout, which is excluded from equality and from compile keys. An inner @Layout() is snapshotted per wrapper.
  • pipeline_compiler.py:
    • _layout_policy_for resolves apply/carry per occurrence.
    • The registry key becomes (compile key, policy variant), with a variant-aware sidecar hash (legacy hash when uncovered).
    • _apply_layout_transform runs at the end of _compile_pipeline_fn, after children and before _validate_artifact.
    • _layout_task_interfaces builds the interfaces.
  • The README gains a "Compile-time auto-layout (@Layout())" section.

Failure modes

  • A transform that is not deterministic would give shared occurrences one arbitrary result. The contract documents that the transform must be deterministic.
  • Opaque-ref card geometry is approximate by design (approximate=True).
  • Uncovered children keep legacy filenames, but a parent's own bytes change when a descendant is covered, because it references the variant sidecar.

Testing

  • tests/test_layout_annotation.py: 22 consolidated, table-driven tests. They cover:
    • both decorator orders and the refusal boundaries;
    • seven coverage-policy scenarios with written positions and legacy filenames;
    • A/B variants and same-policy dedup (diamond, config variants, edge wrappers);
    • cycle detection across variants;
    • imported and cached children;
    • exact and approximate interfaces, including literals and isEnabled;
    • entry-point forwarding;
    • the guard table, including no-write-on-failure and the retained-reference regression;
    • byte identity with no transform or no layout.
  • Full suite: 2097 passed. Ruff (changed files), git diff --check, and uv lock --check against PyPI are clean. uv build passes, and the wheel smoke test reports tangle version = 0.1.29.
  • A plain-fixture base-vs-branch byte probe was identical.
  • mypy reports the same 7 errors as master (pre-existing, in _plan_task_sidecar).
  • Independent review by pi-165: several rounds, final verdict NO BLOCKING FINDINGS. It reproduced and verified fixes for a retained-alias guard bypass, a position-only annotation edge case and an isEnabled geometry gap. It independently killed in-memory mutants for each load-bearing rule (detach copy, recursive=False, inheritance, registry and filename policy, cycle detection, isEnabled, input union, guard, metadata preservation). Review pages:

Review focus

  • The policy and variant rules in _layout_policy_for and the registry/filename keying.
  • Guard completeness in _strip_positions / _apply_layout_transform.
  • That plain compiles are untouched.

`@Layout()` / `@Layout("name", recursive=...)` marks a @pipeline graph
definition for auto-layout. When the caller passes a `layout_transform`
to `compile_pipeline` (also forwarded by `PipelineCompiler.compile_file`
and `pipelines.compile_pipeline_file`), every covered graph is laid out
post-order before validation and writing, so the root and subgraph YAML
carry `editor.position` annotations. Tangle CLI ships no algorithm.

- Coverage: an explicit declaration wins; `recursive=True` (default)
  covers undecorated descendants, `recursive=False` only its own graph;
  `@Layout()` resets to the transform's default.
- Shared children compile per (compile key, layout policy): different
  inherited policies yield distinct sidecars, the same policy dedups.
  Uncovered children keep legacy filenames; cycle detection ignores
  layout.
- The transform receives a private copy plus a `GraphLayoutContext`
  (governing Layout, first-occurrence path, task interfaces, artifact
  dir). A guard allows only `editor.position` changes on graph tasks and
  top-level inputs/outputs; failures write nothing.
- `TaskInterface` is exact for @task/subpipeline tasks and approximate
  (supplied arguments + consumed outputs, incl. isEnabled) for opaque
  refs; nothing is fetched remotely.
- Without a transform, or without @layout(), output is byte-identical
  (a warning notes each unapplied @layout()).
@Volv-G
Volv-G requested a review from Ark-kun as a code owner October 3, 2026 01:02
@Volv-G
Volv-G merged commit 2597eab into master Oct 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant