Skip to content

Treat auto as a first-class Copilot model id#46751

Closed
pelikhan with Copilot wants to merge 5 commits into
mainfrom
copilot/update-model-aliases
Closed

Treat auto as a first-class Copilot model id#46751
pelikhan with Copilot wants to merge 5 commits into
mainfrom
copilot/update-model-aliases

Conversation

Copilot AI commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

This updates model: auto from a generic alias-only concept to a proper Copilot model id. When a workflow selects auto, gh-aw now preserves COPILOT_MODEL=auto through compilation and runtime while keeping pricing aligned with Sonnet 4.6.

  • Model identity

    • Add auto to the built-in model alias map as auto -> large so AWF alias metadata and fallback chains understand it.
    • Preserve literal auto during Copilot-side model resolution instead of eagerly collapsing it to a concrete model.
  • Pricing / accounting

    • Add auto to both embedded models.json catalogs with the same pricing as claude-sonnet-4.6.
    • This keeps AI credit / cost accounting consistent when runs report model: auto.
  • Workflow surface

    • Update smoke-copilot to use top-level model: auto.
    • Regenerate the compiled workflow output so the emitted env wiring now carries GH_AW_INFO_MODEL: "auto" and COPILOT_MODEL: auto.
  • Fixtures / expectations

    • Extend alias and pricing tests for auto.
    • Refresh generated wasm goldens to match the new built-in alias set and compiled Copilot model output.

Example:

name: Smoke Copilot
model: auto
engine:
  id: copilot

Compiled result now preserves the special Copilot mode instead of resolving it away:

GH_AW_INFO_MODEL: "auto"
COPILOT_MODEL: auto

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Add auto as a proper Copilot model alias id Treat auto as a first-class Copilot model id Jul 20, 2026
Copilot AI requested a review from pelikhan July 20, 2026 07:43
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot auto is a know model id for the copilot/github provider. Not a large alias.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

@copilot auto is a know model id for the copilot/github provider. Not a large alias.

Addressed in 177358d.

@pelikhan

Copy link
Copy Markdown
Collaborator

/smoke-copilot

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

📰 BREAKING: Smoke Copilot is now investigating this issue comment. Sources say the story is developing...

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot make sure the auto is in the model list and recompile

See error https://github.com/github/gh-aw/actions/runs/29727177706/job/88303175905#step:48:1

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

@copilot make sure the auto is in the model list and recompile

See error https://github.com/github/gh-aw/actions/runs/29727177706/job/88303175905#step:48:1

Addressed in 73e9001. Confirmed auto is present in the bundled model catalogs, re-ran make recompile (no additional generated changes), and added JS regression coverage for the action-side model catalog used by smoke-copilot.

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot investigate why awf refuses "auto" as a model id

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

@copilot investigate why awf refuses "auto" as a model id

Fixed in bfff4cb. The issue was that AWF’s compiled apiProxy.models map did not include auto, so AWF could reject model: auto even though the Copilot runtime preserved it. auto is now emitted as an exact Copilot model id (copilot/auto), and the affected compiled workflows and WASM goldens were refreshed.

@pelikhan

Copy link
Copy Markdown
Collaborator

/smoke-copilot

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

📰 BREAKING: Smoke Copilot is now investigating this issue comment. Sources say the story is developing...

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot set auto pricing to Sonnet 4.6 level in models.json

https://github.com/github/gh-aw/actions/runs/29734364547/job/88326241059#step:48:1

Copilot AI commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

@copilot set auto pricing to Sonnet 4.6 level in models.json

https://github.com/github/gh-aw/actions/runs/29734364547/job/88326241059#step:48:1

Handled in 8289bb1. auto already matches claude-sonnet-4.6 in both bundled models.json catalogs, and the targeted JS and Go pricing tests pass locally.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage

Field Value
Category feature
Risk 🟡 medium
Priority Score 55 / 100
Action defer

Score breakdown: impact 28 + urgency 15 + quality 12

Rationale: model: auto as first-class Copilot model ID. Draft PR, 2 CI failures. Large lock.yml churn (277 files, bulk recompile). Defer until undrafted and CI passes.

Run §29743972799

Generated by 🔧 PR Triage Agent · 47.4 AIC · ⌖ 5.32 AIC · ⊞ 5.6K ·

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage (updated)

Field Value
Category feature
Risk 🟡 medium
Priority Score 48/100
Recommended Action defer

Score breakdown: impact 22 + urgency 14 + quality 12

Treat auto as a first-class Copilot model id. Draft. CI shows agent job failures. Defer until CI is green and author undrafts.

Generated by 🔧 PR Triage Agent · 34.3 AIC · ⌖ 7.71 AIC · ⊞ 5.6K ·

@pelikhan

Copy link
Copy Markdown
Collaborator

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

@copilot set the default API pricing

See https://github.com/github/gh-aw/actions/runs/29742523537/job/88352626832#step:48:1

Addressed in 07aebdc. AWF config generation now emits apiProxy.defaultAiCreditsPricing for Copilot auto at Sonnet 4.6 pricing, and smoke-copilot.lock.yml was recompiled with the new fallback.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 PR Triage — Run §29791496271

Field Value
Category feature
Risk 🟡 medium
Score 42/100 (impact 22 + urgency 10 + quality 10)
Action defer (DRAFT)

Rationale: Valuable model-alias improvement but non-urgent. DRAFT with 279 changed files (mass lock regen). Undraft when ready for dedicated review cycle.

Generated by 🔧 PR Triage Agent · 40.1 AIC · ⌖ 6.65 AIC · ⊞ 5.6K ·

@pelikhan

Copy link
Copy Markdown
Collaborator

/smoke-copilot

@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

📰 BREAKING: Smoke Copilot is now investigating this issue comment. Sources say the story is developing...

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot review model info passed to awf and why it failed

https://github.com/github/gh-aw/actions/runs/29801597422/job/88543723289#step:48:1

Copilot AI commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@copilot review model info passed to awf and why it failed

https://github.com/github/gh-aw/actions/runs/29801597422/job/88543723289#step:48:1

I checked the failing run artifacts. gh-aw passed auto through to AWF as intended:

  • COPILOT_MODEL=auto
  • GH_AW_INFO_MODEL="auto"
  • apiProxy.defaultAiCreditsPricing was present in awf-config.json with the Sonnet 4.6 rates
  • apiProxy.models.auto=["copilot/auto"] was also present

The failure happened inside AWF/runtime anyway. The agent stderr was:
Model "auto" has no AI credits pricing and no default pricing is configured.

/reflect also showed that the Copilot endpoint was configured, but auto was not listed in the available models it reported. So the compile-side model info was wired through correctly; the breakage is that AWF 0.27.37 did not recognize/honor auto pricing at runtime.

@github-actions

Copy link
Copy Markdown
Contributor

PR Triage

Field Value
Category feature
Risk medium
Score 55/100
Breakdown impact:25 · urgency:15 · quality:15
Action defer

Carry-over draft. Treats auto as a first-class Copilot model id (30 files, +31/-31). Medium-risk model alias/pricing change. Defer until undrafted and CI passes.

Run §29809051050

Generated by 🔧 PR Triage Agent · 59.3 AIC · ⌖ 5.53 AIC · ⊞ 5.6K ·

@github-actions

Copy link
Copy Markdown
Contributor

Triage

Field Value
Category feature
Risk medium
Score 48/100 (impact:30 urgency:10 quality:8)
Action batch_review
Batch pr-batch:feature-updates

Rationale: Treats 'auto' as first-class Copilot model ID. CI has 1 failure. Age: ~1 day. Medium impact on model routing behavior; needs review to confirm regression risk.

Generated by 🔧 PR Triage Agent · sonnet46 45.9 AIC · ⌖ 5.48 AIC · ⊞ 5.6K ·

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage

Field Value
Category feature
Risk medium
Score 47 / 100
Action batch_review
Batch

Score breakdown: Impact 22 · Urgency 13 · Quality 12

Draft PR treating auto as first-class Copilot model id, touching 100+ lock files. CI passing. Medium risk due to scope. Still draft — needs promotion before review.

Generated by 🔧 PR Triage Agent · sonnet46 43.3 AIC · ⌖ 5.33 AIC · ⊞ 5.6K ·

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage (carry-over)

Field Value
Category feature
Risk 🔴 High
Score 48/100 (impact: 25, urgency: 10, quality: 13)
Action ⏸️ defer

100 files changed. Makes model: auto a first-class Copilot model id. Large scope, DRAFT, no CI data. Defer until scope can be scoped down or CI passes.

Generated by 🔧 PR Triage Agent · sonnet46 35.2 AIC · ⌖ 7.92 AIC · ⊞ 5.6K ·

@pelikhan pelikhan closed this Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants