Skip to content

[live-migration] disable unsupported features for migratable pods#2715

Open
rawahars wants to merge 1 commit intomicrosoft:mainfrom
rawahars:lm_disable_non_migratable
Open

[live-migration] disable unsupported features for migratable pods#2715
rawahars wants to merge 1 commit intomicrosoft:mainfrom
rawahars:lm_disable_non_migratable

Conversation

@rawahars
Copy link
Copy Markdown
Contributor

Summary

Pods marked live-migratable cannot use features whose runtime state would not survive a migration to a different host. Reject container creation in such pods when the spec requests any of:

  • device assignments
  • VHD mounts
  • directory mounts

Additionally, drop host<->guest plumbing that the target host of a migration is not guaranteed to provide:

  • Run /bin/gcs directly under init instead of wrapping it with /bin/vsockexec on LinuxLogVsockPort. With no host-side log listener bound, the vsockexec connect would otherwise stall init.
  • Skip starting the GCS log listener on the controller side.

Entropy injection and the remaining gcs flags (-loglevel, -scrub-logs, -core-dump-location, ...) are unaffected.


Also enforce a UVM-shape annotation policy on sandboxes that opt into live migration via the LiveMigrationAllowed sandbox option. The check runs at the top of BuildSandboxConfig so an unsupported request fails fast, before any boot/device/kernel-args parsing produces a non-migratable UVM:

  • Annotations outside the "io.microsoft.virtualmachine.*" scope are ignored.
  • Per-GUID HvSocket service-table entries (UVMHyperVSocketConfigPrefix.*) are always rejected, since each entry binds the UVM to a host-side service registration that cannot move with the VM.
  • A small allow-list of host-agnostic knobs may be set to any value: boot files root path, kernel direct boot, CPU/memory shaping (allow-overcommit, processor count/limit/weight, memory size and MMIO gap knobs), storage QoS rate limits, and in-guest policy-based routing.
  • A locked-value set must match exactly

Pods marked live-migratable cannot use features whose runtime state
would not survive a migration to a different host. Reject container
creation in such pods when the spec requests any of:

  - device assignments
  - VHD mounts
  - directory mounts

Additionally, drop host<->guest plumbing that the target host of a
migration is not guaranteed to provide:

  - Run /bin/gcs directly under init instead of wrapping it with
    /bin/vsockexec on LinuxLogVsockPort. With no host-side log
    listener bound, the vsockexec connect would otherwise stall init.
  - Skip starting the GCS log listener on the controller side.

Entropy injection and the remaining gcs flags (-loglevel, -scrub-logs,
-core-dump-location, ...) are unaffected.

Also enforce a UVM-shape annotation policy on sandboxes that opt into
live migration via the LiveMigrationAllowed sandbox option. The check
runs at the top of BuildSandboxConfig so an unsupported request fails
fast, before any boot/device/kernel-args parsing produces a
non-migratable UVM:

  - Annotations outside the "io.microsoft.virtualmachine.*" scope are
    ignored.
  - Per-GUID HvSocket service-table entries
    (UVMHyperVSocketConfigPrefix.*) are always rejected, since each
    entry binds the UVM to a host-side service registration that
    cannot move with the VM.
  - A small allow-list of host-agnostic knobs may be set to any value:
    boot files root path, kernel direct boot, CPU/memory shaping
    (allow-overcommit, processor count/limit/weight, memory size and
    MMIO gap knobs), storage QoS rate limits, and in-guest
    policy-based routing.
  - A locked-value set must match exactly: PreferredRootFSType=vhd,
    VPCIEnabled=false, VPMemCount=0, WritableOverlayDirs=false.
  - Every other "io.microsoft.virtualmachine.*" annotation is rejected
    (default-deny).

Signed-off-by: Harsh Rawat <harshrawat@microsoft.com>
@rawahars rawahars requested a review from a team as a code owner April 30, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant