Report per-run cost metrics in output.yml

[calvinp0]

Stacked on #917 (feature_arc_result_export_contract), stack #946. This PR's diff shows only its own layer.

#945  Report per-run cost metrics in output.yml   ← you are here
 └── #917  Export tool-neutral ARC results        (base: main)

It depends on #917 rather than merely following it: cost_metrics is a section of the output.yml schema that #917 defines, and it extends write_output_yml()'s existing t0 parameter. It cannot land first.

What this adds

A cost_metrics section in output.yml, answering "what did this run actually cost?":

cost_metrics:
  wall_time_hrs: 26.4
  total_job_count: 118
  total_execution_time_hrs: 402.1
  total_core_hours: 3216.8
  jobs_missing_time: 0
  jobs_missing_cores: 2
  per_ess:
    gaussian: {job_count: 94, execution_time_hrs: 380.2, core_hours: 3041.6, jobs_missing_time: 0}
    molpro:   {job_count: 24, execution_time_hrs:  21.9, core_hours:  175.2, jobs_missing_time: 0}

Three layers, one per commit:

1. arc/scheduler.py — the Scheduler appends a lightweight record for every job reaching end_job (name, label, job type, adapter, server, cpu cores, run time, status). Pipe-mode tasks bypass the JobAdapter path and carry no per-task timing, so they are deliberately not recorded; only jobs ejected back to the Scheduler are.
2. arc/output.py — _compute_cost_metrics() aggregates them; _timedelta_to_seconds() normalizes the several shapes a run time arrives in (timedelta, numeric seconds, H:MM:SS, D days, H:MM:SS), returning None rather than raising on an unparseable value.
3. arc/main.py — wires the records through to write_output_yml(), and accepts them back on restart.

Incomplete accounting is counted, not hidden

Jobs with no recorded run time or core count land in jobs_missing_time / jobs_missing_cores instead of being silently dropped, so a consumer can tell how complete the numbers are. A run where a third of jobs lack timing produces a visibly partial total rather than a confidently wrong one.

The restart half is a real bug fix, not plumbing

Persisting the records is what makes them survive a restart — and that is exactly where this bit us in practice:

TypeError: ARC.__init__() got an unexpected keyword argument 'completed_job_records'

ARC.py restarts with ARC(**read_yaml_file('restart.yml')), so the restart file is ARC's constructor namespace — but it is ARC.as_dict() plus whatever Scheduler.save_restart_dict() adds to it. Writing the records there without a matching constructor parameter makes every restart of a project that has accumulated any fail instantly. as_dict() now emits the key when non-empty so the round trip is symmetric, and a restart file written before the key existed still constructs.

TestRestartRoundTrip asserts that every key written into a restart dictionary — by as_dict() and by the Scheduler — is a constructor parameter. That invariant had no coverage at all: ARC(**read_yaml_file(restart.yml)) was exercised by no test, which is why this class of breakage stays invisible until somebody actually restarts. Any future key added to Scheduler.save_restart_dict() without touching main.py now fails a test instead of a multi-day run.

Evidence

222 tests pass (output_test, main_test, scheduler_test), including 20 new ones: _timedelta_to_seconds across all input shapes, _compute_cost_metrics aggregation / missing-data counting / unknown-adapter bucketing, the cost_metrics section end-to-end through write_output_yml, and the four restart round-trip guards.

Notes for review

• Each commit touches a disjoint set of files (scheduler.py | output.py+tests+docs | main.py+tests), so the three are independently readable.
• write_output_yml()'s new completed_job_records defaults to None, alongside the existing t0; a caller supplying neither gets an empty cost_metrics block rather than a missing key.
• Scoped deliberately: this is the cost-metrics half only. The TS-guess provenance that shipped alongside it in our internal branch is not included — Export tool-neutral ARC results and parser evidence #917 already has its own ts_guesses export, and that is the version kept here.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (feature_arc_result_export_contract@fd0ea74). Learn more about missing BASE report.

Additional details and impacted files

@@                          Coverage Diff                          @@
##             feature_arc_result_export_contract     #945   +/-   ##
=====================================================================
  Coverage                                      ?   64.12%           
=====================================================================
  Files                                         ?      115           
  Lines                                         ?    39720           
  Branches                                      ?    10340           
=====================================================================
  Hits                                          ?    25469           
  Misses                                        ?    11235           
  Partials                                      ?     3016           

Flag	Coverage Δ
functionaltests	64.12% <ø> (?)
unittests	64.12% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.