[live-migration] disable unsupported features for migratable pods#2715
Open
rawahars wants to merge 1 commit intomicrosoft:mainfrom
Open
[live-migration] disable unsupported features for migratable pods#2715rawahars wants to merge 1 commit intomicrosoft:mainfrom
rawahars wants to merge 1 commit intomicrosoft:mainfrom
Conversation
Pods marked live-migratable cannot use features whose runtime state
would not survive a migration to a different host. Reject container
creation in such pods when the spec requests any of:
- device assignments
- VHD mounts
- directory mounts
Additionally, drop host<->guest plumbing that the target host of a
migration is not guaranteed to provide:
- Run /bin/gcs directly under init instead of wrapping it with
/bin/vsockexec on LinuxLogVsockPort. With no host-side log
listener bound, the vsockexec connect would otherwise stall init.
- Skip starting the GCS log listener on the controller side.
Entropy injection and the remaining gcs flags (-loglevel, -scrub-logs,
-core-dump-location, ...) are unaffected.
Also enforce a UVM-shape annotation policy on sandboxes that opt into
live migration via the LiveMigrationAllowed sandbox option. The check
runs at the top of BuildSandboxConfig so an unsupported request fails
fast, before any boot/device/kernel-args parsing produces a
non-migratable UVM:
- Annotations outside the "io.microsoft.virtualmachine.*" scope are
ignored.
- Per-GUID HvSocket service-table entries
(UVMHyperVSocketConfigPrefix.*) are always rejected, since each
entry binds the UVM to a host-side service registration that
cannot move with the VM.
- A small allow-list of host-agnostic knobs may be set to any value:
boot files root path, kernel direct boot, CPU/memory shaping
(allow-overcommit, processor count/limit/weight, memory size and
MMIO gap knobs), storage QoS rate limits, and in-guest
policy-based routing.
- A locked-value set must match exactly: PreferredRootFSType=vhd,
VPCIEnabled=false, VPMemCount=0, WritableOverlayDirs=false.
- Every other "io.microsoft.virtualmachine.*" annotation is rejected
(default-deny).
Signed-off-by: Harsh Rawat <harshrawat@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pods marked live-migratable cannot use features whose runtime state would not survive a migration to a different host. Reject container creation in such pods when the spec requests any of:
Additionally, drop host<->guest plumbing that the target host of a migration is not guaranteed to provide:
Entropy injection and the remaining gcs flags (-loglevel, -scrub-logs, -core-dump-location, ...) are unaffected.
Also enforce a UVM-shape annotation policy on sandboxes that opt into live migration via the LiveMigrationAllowed sandbox option. The check runs at the top of BuildSandboxConfig so an unsupported request fails fast, before any boot/device/kernel-args parsing produces a non-migratable UVM: