fix(cli): preserve graph artifacts when community labeling fails - #2726
fix(cli): preserve graph artifacts when community labeling fails#2726DebadityaHait wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
This PR changes the failure semantics of community labeling in graphify. Previously, when the LLM backend failed or returned malformed data, generate_community_labels would silently degrade to "Community N" placeholder labels (source "placeholder"); now it distinguishes between "no backend configured" (still "placeholder") and "every requested batch failed" (new source value "failed"). The CLI dispatch_command is updated to detect the "failed" source, print an error to stderr, and exit with code 1 without modifying graph artifacts. label_communities also now raises when no labels are produced (including the empty-reply case), and the tests are updated to cover the new failure paths, the artifact-preservation behavior, and an adjusted prompt-parsing format in the token-usage fake. Surface area: the labeling logic in graphify/llm.py, the label command handling in graphify/cli.py, and tests/test_labeling.py.
Worth a look
- label_communities raises ValueError when communities is empty (written==0, no errors) —
graphify/llm.py:3122· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 780 functions depend on the 262 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
generate_community_labels()— 7 callers, 3 callees
Verification — 780 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 678 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify generate\_community\_labels.
The verifier did not have enough to check generate\_community\_labels, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: non-vacuity: domain too small (only 1 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous
Could not verify: Could not verify label\_communities.
The verifier did not have enough to check label\_communities, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 47 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
· 1 more finding(s) on lines outside this diff (see the check run).
Summary
An explicit community-labeling run could fail every LLM batch, fall back to deterministic placeholder labels, and still rewrite graph artifacts while returning exit code 0. This change makes that path fail closed before any backup or artifact write.
Changes
Related to #2573.
Validation