Skip to content

fix(cli): preserve graph artifacts when community labeling fails - #2726

Open
DebadityaHait wants to merge 1 commit into
Graphify-Labs:v8from
DebadityaHait:fix/2573-label-write-guard
Open

fix(cli): preserve graph artifacts when community labeling fails#2726
DebadityaHait wants to merge 1 commit into
Graphify-Labs:v8from
DebadityaHait:fix/2573-label-write-guard

Conversation

@DebadityaHait

@DebadityaHait DebadityaHait commented Aug 13, 2026

Copy link
Copy Markdown

Summary

An explicit community-labeling run could fail every LLM batch, fall back to deterministic placeholder labels, and still rewrite graph artifacts while returning exit code 0. This change makes that path fail closed before any backup or artifact write.

Changes

  • Distinguish the no-backend placeholder fallback from an all-batches-failed result.
  • Exit nonzero with an actionable message when every requested labeling batch fails.
  • Preserve the existing graph, report, analysis, label sidecar/signature, HTML, and backup state on that failure path.
  • Keep no-backend fallback, --no-label, and partial-success behavior unchanged.
  • Add regression coverage for artifact preservation and empty successful responses.

Related to #2573.

Validation

  • uv run --frozen pytest tests/test_labeling.py tests/test_cli_export.py -q — 74 passed.
  • uv run --frozen ruff check graphify/llm.py graphify/cli.py tests/test_labeling.py — passed.
  • python -m py_compile graphify/llm.py graphify/cli.py tests/test_labeling.py — passed.
  • git diff --check — passed.
  • uv run --frozen graphify update . — completed successfully.
  • Skillgen check, coverage audit, schema singleton, and both round-trip checks — passed.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Formal verification. No changes could be formally verified in this run.


Graphify review — findings

This PR changes the failure semantics of community labeling in graphify. Previously, when the LLM backend failed or returned malformed data, generate_community_labels would silently degrade to "Community N" placeholder labels (source "placeholder"); now it distinguishes between "no backend configured" (still "placeholder") and "every requested batch failed" (new source value "failed"). The CLI dispatch_command is updated to detect the "failed" source, print an error to stderr, and exit with code 1 without modifying graph artifacts. label_communities also now raises when no labels are produced (including the empty-reply case), and the tests are updated to cover the new failure paths, the artifact-preservation behavior, and an adjusted prompt-parsing format in the token-usage fake. Surface area: the labeling logic in graphify/llm.py, the label command handling in graphify/cli.py, and tests/test_labeling.py.

Worth a look

  • label_communities raises ValueError when communities is empty (written==0, no errors)graphify/llm.py:3122 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 780 functions depend on the 262 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: generate_community_labels() — 7 callers, 3 callees

Verification — 780 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 678 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

Could not verify: Could not verify generate\_community\_labels.

The verifier did not have enough to check generate\_community\_labels, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 1 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

Could not verify: Could not verify label\_communities.

The verifier did not have enough to check label\_communities, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 47 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)

· 1 more finding(s) on lines outside this diff (see the check run).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant