hyperv.txt (11596B)
1Hyper-V Enlightenments 2====================== 3 4 51. Description 6=============== 7In some cases when implementing a hardware interface in software is slow, KVM 8implements its own paravirtualized interfaces. This works well for Linux as 9guest support for such features is added simultaneously with the feature itself. 10It may, however, be hard-to-impossible to add support for these interfaces to 11proprietary OSes, namely, Microsoft Windows. 12 13KVM on x86 implements Hyper-V Enlightenments for Windows guests. These features 14make Windows and Hyper-V guests think they're running on top of a Hyper-V 15compatible hypervisor and use Hyper-V specific features. 16 17 182. Setup 19========= 20No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In 21QEMU, individual enlightenments can be enabled through CPU flags, e.g: 22 23 qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... 24 25Sometimes there are dependencies between enlightenments, QEMU is supposed to 26check that the supplied configuration is sane. 27 28When any set of the Hyper-V enlightenments is enabled, QEMU changes hypervisor 29identification (CPUID 0x40000000..0x4000000A) to Hyper-V. KVM identification 30and features are kept in leaves 0x40000100..0x40000101. 31 32 333. Existing enlightenments 34=========================== 35 363.1. hv-relaxed 37================ 38This feature tells guest OS to disable watchdog timeouts as it is running on a 39hypervisor. It is known that some Windows versions will do this even when they 40see 'hypervisor' CPU flag. 41 423.2. hv-vapic 43============== 44Provides so-called VP Assist page MSR to guest allowing it to work with APIC 45more efficiently. In particular, this enlightenment allows paravirtualized 46(exit-less) EOI processing. 47 483.3. hv-spinlocks=xxx 49====================== 50Enables paravirtualized spinlocks. The parameter indicates how many times 51spinlock acquisition should be attempted before indicating the situation to the 52hypervisor. A special value 0xffffffff indicates "never notify". 53 543.4. hv-vpindex 55================ 56Provides HV_X64_MSR_VP_INDEX (0x40000002) MSR to the guest which has Virtual 57processor index information. This enlightenment makes sense in conjunction with 58hv-synic, hv-stimer and other enlightenments which require the guest to know its 59Virtual Processor indices (e.g. when VP index needs to be passed in a 60hypercall). 61 623.5. hv-runtime 63================ 64Provides HV_X64_MSR_VP_RUNTIME (0x40000010) MSR to the guest. The MSR keeps the 65virtual processor run time in 100ns units. This gives guest operating system an 66idea of how much time was 'stolen' from it (when the virtual CPU was preempted 67to perform some other work). 68 693.6. hv-crash 70============== 71Provides HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 (0x40000100..0x40000105) and 72HV_X64_MSR_CRASH_CTL (0x40000105) MSRs to the guest. These MSRs are written to 73by the guest when it crashes, HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 MSRs 74contain additional crash information. This information is outputted in QEMU log 75and through QAPI. 76Note: unlike under genuine Hyper-V, write to HV_X64_MSR_CRASH_CTL causes guest 77to shutdown. This effectively blocks crash dump generation by Windows. 78 793.7. hv-time 80============= 81Enables two Hyper-V-specific clocksources available to the guest: MSR-based 82Hyper-V clocksource (HV_X64_MSR_TIME_REF_COUNT, 0x40000020) and Reference TSC 83page (enabled via MSR HV_X64_MSR_REFERENCE_TSC, 0x40000021). Both clocksources 84are per-guest, Reference TSC page clocksource allows for exit-less time stamp 85readings. Using this enlightenment leads to significant speedup of all timestamp 86related operations. 87 883.8. hv-synic 89============== 90Enables Hyper-V Synthetic interrupt controller - an extension of a local APIC. 91When enabled, this enlightenment provides additional communication facilities 92to the guest: SynIC messages and Events. This is a pre-requisite for 93implementing VMBus devices (not yet in QEMU). Additionally, this enlightenment 94is needed to enable Hyper-V synthetic timers. SynIC is controlled through MSRs 95HV_X64_MSR_SCONTROL..HV_X64_MSR_EOM (0x40000080..0x40000084) and 96HV_X64_MSR_SINT0..HV_X64_MSR_SINT15 (0x40000090..0x4000009F) 97 98Requires: hv-vpindex 99 1003.9. hv-stimer 101=============== 102Enables Hyper-V synthetic timers. There are four synthetic timers per virtual 103CPU controlled through HV_X64_MSR_STIMER0_CONFIG..HV_X64_MSR_STIMER3_COUNT 104(0x400000B0..0x400000B7) MSRs. These timers can work either in single-shot or 105periodic mode. It is known that certain Windows versions revert to using HPET 106(or even RTC when HPET is unavailable) extensively when this enlightenment is 107not provided; this can lead to significant CPU consumption, even when virtual 108CPU is idle. 109 110Requires: hv-vpindex, hv-synic, hv-time 111 1123.10. hv-tlbflush 113================== 114Enables paravirtualized TLB shoot-down mechanism. On x86 architecture, remote 115TLB flush procedure requires sending IPIs and waiting for other CPUs to perform 116local TLB flush. In virtualized environment some virtual CPUs may not even be 117scheduled at the time of the call and may not require flushing (or, flushing 118may be postponed until the virtual CPU is scheduled). hv-tlbflush enlightenment 119implements TLB shoot-down through hypervisor enabling the optimization. 120 121Requires: hv-vpindex 122 1233.11. hv-ipi 124============= 125Enables paravirtualized IPI send mechanism. HvCallSendSyntheticClusterIpi 126hypercall may target more than 64 virtual CPUs simultaneously, doing the same 127through APIC requires more than one access (and thus exit to the hypervisor). 128 129Requires: hv-vpindex 130 1313.12. hv-vendor-id=xxx 132======================= 133This changes Hyper-V identification in CPUID 0x40000000.EBX-EDX from the default 134"Microsoft Hv". The parameter should be no longer than 12 characters. According 135to the specification, guests shouldn't use this information and it is unknown 136if there is a Windows version which acts differently. 137Note: hv-vendor-id is not an enlightenment and thus doesn't enable Hyper-V 138identification when specified without some other enlightenment. 139 1403.13. hv-reset 141=============== 142Provides HV_X64_MSR_RESET (0x40000003) MSR to the guest allowing it to reset 143itself by writing to it. Even when this MSR is enabled, it is not a recommended 144way for Windows to perform system reboot and thus it may not be used. 145 1463.14. hv-frequencies 147============================================ 148Provides HV_X64_MSR_TSC_FREQUENCY (0x40000022) and HV_X64_MSR_APIC_FREQUENCY 149(0x40000023) allowing the guest to get its TSC/APIC frequencies without doing 150measurements. 151 1523.15 hv-reenlightenment 153======================== 154The enlightenment is nested specific, it targets Hyper-V on KVM guests. When 155enabled, it provides HV_X64_MSR_REENLIGHTENMENT_CONTROL (0x40000106), 156HV_X64_MSR_TSC_EMULATION_CONTROL (0x40000107)and HV_X64_MSR_TSC_EMULATION_STATUS 157(0x40000108) MSRs allowing the guest to get notified when TSC frequency changes 158(only happens on migration) and keep using old frequency (through emulation in 159the hypervisor) until it is ready to switch to the new one. This, in conjunction 160with hv-frequencies, allows Hyper-V on KVM to pass stable clocksource (Reference 161TSC page) to its own guests. 162 163Note, KVM doesn't fully support re-enlightenment notifications and doesn't 164emulate TSC accesses after migration so 'tsc-frequency=' CPU option also has to 165be specified to make migration succeed. The destination host has to either have 166the same TSC frequency or support TSC scaling CPU feature. 167 168Recommended: hv-frequencies 169 1703.16. hv-evmcs 171=============== 172The enlightenment is nested specific, it targets Hyper-V on KVM guests. When 173enabled, it provides Enlightened VMCS version 1 feature to the guest. The feature 174implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V) 175hypervisors making L2 exits to the hypervisor faster. The feature is Intel-only. 176Note: some virtualization features (e.g. Posted Interrupts) are disabled when 177hv-evmcs is enabled. It may make sense to measure your nested workload with and 178without the feature to find out if enabling it is beneficial. 179 180Requires: hv-vapic 181 1823.17. hv-stimer-direct 183======================= 184Hyper-V specification allows synthetic timer operation in two modes: "classic", 185when expiration event is delivered as SynIC message and "direct", when the event 186is delivered via normal interrupt. It is known that nested Hyper-V can only 187use synthetic timers in direct mode and thus 'hv-stimer-direct' needs to be 188enabled. 189 190Requires: hv-vpindex, hv-synic, hv-time, hv-stimer 191 1923.18. hv-avic (hv-apicv) 193======================= 194The enlightenment allows to use Hyper-V SynIC with hardware APICv/AVIC enabled. 195Normally, Hyper-V SynIC disables these hardware feature and suggests the guest 196to use paravirtualized AutoEOI feature. 197Note: enabling this feature on old hardware (without APICv/AVIC support) may 198have negative effect on guest's performace. 199 2003.19. hv-no-nonarch-coresharing=on/off/auto 201=========================================== 202This enlightenment tells guest OS that virtual processors will never share a 203physical core unless they are reported as sibling SMT threads. This information 204is required by Windows and Hyper-V guests to properly mitigate SMT related CPU 205vulnerabilities. 206When the option is set to 'auto' QEMU will enable the feature only when KVM 207reports that non-architectural coresharing is impossible, this means that 208hyper-threading is not supported or completely disabled on the host. This 209setting also prevents migration as SMT settings on the destination may differ. 210When the option is set to 'on' QEMU will always enable the feature, regardless 211of host setup. To keep guests secure, this can only be used in conjunction with 212exposing correct vCPU topology and vCPU pinning. 213 2143.20. hv-version-id-{build,major,minor,spack,sbranch,snumber} 215============================================================= 216This changes Hyper-V version identification in CPUID 0x40000002.EAX-EDX from the 217default (WS2016). 218- hv-version-id-build sets 'Build Number' (32 bits) 219- hv-version-id-major sets 'Major Version' (16 bits) 220- hv-version-id-minor sets 'Minor Version' (16 bits) 221- hv-version-id-spack sets 'Service Pack' (32 bits) 222- hv-version-id-sbranch sets 'Service Branch' (8 bits) 223- hv-version-id-snumber sets 'Service Number' (24 bits) 224 225Note: hv-version-id-* are not enlightenments and thus don't enable Hyper-V 226identification when specified without any other enlightenments. 227 2284. Supplementary features 229========================= 230 2314.1. hv-passthrough 232=================== 233In some cases (e.g. during development) it may make sense to use QEMU in 234'pass-through' mode and give Windows guests all enlightenments currently 235supported by KVM. This pass-through mode is enabled by "hv-passthrough" CPU 236flag. 237Note: "hv-passthrough" flag only enables enlightenments which are known to QEMU 238(have corresponding "hv-*" flag) and copies "hv-spinlocks="/"hv-vendor-id=" 239values from KVM to QEMU. "hv-passthrough" overrides all other "hv-*" settings on 240the command line. Also, enabling this flag effectively prevents migration as the 241list of enabled enlightenments may differ between target and destination hosts. 242 2434.2. hv-enforce-cpuid 244===================== 245By default, KVM allows the guest to use all currently supported Hyper-V 246enlightenments when Hyper-V CPUID interface was exposed, regardless of if 247some features were not announced in guest visible CPUIDs. 'hv-enforce-cpuid' 248feature alters this behavior and only allows the guest to use exposed Hyper-V 249enlightenments. 250 251 2525. Useful links 253================ 254Hyper-V Top Level Functional specification and other information: 255https://github.com/MicrosoftDocs/Virtualization-Documentation