Skip to content

[Nebius] Add opt-in capacity advisor - #4430

Merged
peterschmidt85 merged 6 commits into
masterfrom
pr_nebius_capacity
Nov 10, 2026
Merged

peterschmidt85 merged 6 commits into
masterfrom
pr_nebius_capacity

Conversation

@peterschmidt85

@peterschmidt85 peterschmidt85 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Nebius GPU offers currently report unknown availability, so users cannot see quota or capacity restrictions and dstack may attempt configurations that cannot be provisioned.

Add opt-in checks through capacity_advisor: true to show these statuses and guide provisioning and fabric selection. The option defaults to false because checking adds latency and requires caching quota and capacity data.

Match advice by region, platform, preset, spot mode and fabric. Catalog fabric compatibility and configured allowlists remain enforced in both modes. Missing, stale or failed advice stays unknown and eligible.

Notes

  • Fetching quota/capacity from Nebius adds roughly 1–3 seconds. Cache both for 10 minutes, matching AWS's quota-cache TTL. Pricing stays offline; the existing 3-minute offer cache can extend reuse of a fetched snapshot to ~13 minutes.
  • Preserve automatic reservation use. Reserved capacity keeps otherwise blocked on-demand offers eligible as unknown, since tenant-wide advice does not check project reservation limits. Spot offers use only preemptible advice.
  • Support an optional shared cache scoped by tenant and credential IDs. Factory wiring: https://github.com/dstackai/dstack-factory/pull/22. Project quota/balance checks remain separate.
  • Keep the existing 10-second timeout per page and reuse tenant discovery for project selection and advice.

Getting offers

Local backend-collection medians on November 5, with pricing already loaded; 3 cold/new-backend runs and 9 refreshes. Cold ranges are in parentheses. AWS: 11 regions / 2,049 offers. Nebius: 9 regions / 169 offers. HTTP overhead is excluded.

Scenario AWS Nebius, default Nebius, capacity_advisor: true
New backend, empty provider caches 1.77 s (1.72–1.92 s) 0.97 s (0.59–2.31 s) 1.50 s (1.35–1.59 s)
Offer refresh, provider data already cached 37.7 ms 5.1 ms 3.8 ms
New backend using shared provider caches 36 ms 757 ms 789 ms

The shared-cache row uses Factory's cache-injection pattern. Nebius still performs tenant/project discovery for each new backend. AWS checks quotas and zones, without querying live free capacity for ordinary offers.

@peterschmidt85
peterschmidt85 marked this pull request as ready for review October 29, 2026 08:16
@peterschmidt85
peterschmidt85 requested a review from jvstme October 29, 2026 08:17
@peterschmidt85 peterschmidt85 changed the title Use Nebius Capacity Advisor for GPU offers Add opt-in capacity advisor for Nebius Nov 5, 2026
@peterschmidt85

Copy link
Copy Markdown
Contributor Author

Made capacity advisor opt-in via capacity_advisor: true in the Nebius backend configuration. It defaults to false: existing configurations make no capacity-advice requests and keep native availability unknown.

Enabling it shows quota/availability statuses and applies advice to provisioning and fabric selection, with a 10-minute advice cache. The PR body now includes an AWS latency comparison with the option disabled and enabled.

@peterschmidt85 peterschmidt85 changed the title Add opt-in capacity advisor for Nebius [Nebius] Add opt-in capacity advisor Nov 5, 2026
@peterschmidt85
peterschmidt85 merged commit a4bdeb7 into master Nov 10, 2026
26 checks passed
@peterschmidt85
peterschmidt85 deleted the pr_nebius_capacity branch November 10, 2026 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant