</code></pre><p>Restart your machine and boot into BIOS. Enable a feature called <strong>IOMMU</strong>. You’ll also need to enable CPU virtualization. For Intel processors, look for something called <strong>VT-d</strong>. For AMD, look for something called <strong>AMD-Vi</strong>. My motherboard is unique so I had to enable a feature called <strong>SVM Mode</strong>. Save any changes and restart the machine.</p>
<p>Once you’ve booted into the host, make sure that IOMMU is enabled: <code>dmesg | grep IOMMU</code></p>
<p>Also check that CPU virtualization is enabled:</p>
<p>Now you’re going to need to pass the hardware-enabled IOMMU functionality into the kernel as a kernel parameter. For our purposes, it makes the most sense to enable this feature at boot-time. Depending on your boot-loader (i.e. grub, systemd, rEFInd), you’ll have to modify a specific configuration file. Since my machine uses systemd and these configuration files are often overwritten on updates, I will be using a tool called kernelstub:</p>
<p>Similarly, if your system is configured with GRUB2, you can achieve the same result by editing the /etc/default/grub file with sudo permissions and including the kernel parameter as follows:</p>
</code></pre><h1id="2-creating-a-stub-for-the-gpu">2. Creating a stub for the GPU</h1>
<p>Next, we need to update the kernel command line to configure <strong>vfio-pci</strong> to consume each passthrough device to prevent it from being used by the hypervisor. If using GRUB, add another argument to <strong>GRUB_CMDLINE_LINUX_DEFAULT</strong> in <strong>/etc/default/grub</strong> in the following format (where PLACEHOLDER should be replaced with a comma-separated list of hardware IDs for your passthrough devices):</p>
<pretabindex="0"><code>vfio_pci.ids=PLACEHOLDER
</code></pre><p>Once you modify this variable, it should look similar to the following example:</p>
</code></pre><p>Finally, run the following commands (in the listed order) and reboot the hypervisor yet again:</p>
<pretabindex="0"><code>update-initramfs -u
</code></pre><pretabindex="0"><code>update-grub
</code></pre><p>After the hypervisor reboots, check that each required device was consumed by the vfio-pci kernel module. You should see vfio-pci on the line starting with “Kernel driver in use:” for each passthrough device:</p>
</code></pre><h1id="3-imaging-the-gpu-rom">3. Imaging the GPU ROM</h1>
<p>Before you can image the GPU ROM, you need to make sure that you’ve successfully completed the prior steps to blacklist the GPU and that you’ve configured your motherboard to use another graphics device as the primary video output. This is important because the GPU needs to be uninitialized when it is imaged or the ROM file may be garbled from previous initializations.</p>
<p>For some graphics cards or other PCI-e devices, this step may be unnecessary. Some GPUs can operate just fine without mapping a static ROM file; the virtual machine can just directly access the device ROM. Your results may vary, though. The primary purpose of this step is to ensure that successive virtual machine reboots won’t require the hypervisor to be rebooted to reset the GPU’s ROM to an uninitialized state.</p>
<p>First, make sure that your hypervisor has been freshly rebooted so that your GPU’s ROM is uninitialized. Next, run the following commands (replacing the example BDF identifier with your device’s BDF identifier) to obtain an image of the GPU ROM:</p>
</code></pre><p>Note: The process of extracting your GPU’s ROM only needs to be done for the primary function of your device, i.e. the “graphics” portion of the device.</p>
<p>The GPU ROM is now available at /usr/share/qemu/gpu-YYYYMMDDTHHMMSS.rom for use by libvirt and QEMU. You can name this file whatever you want for your own convenience.</p>
<h1id="4-preparing-the-virtual-machine">4. Preparing the Virtual Machine</h1>
<p>Now that the hypervisor is prepared for GPU passthrough, we need to configure the Windows® 10 virtual machine so that the GPU’s driver is unable to detect the virtualization environment. For the purposes of this tutorial, we’ll be using virsh to manually edit the virtual machine configuration. All of these changes will remain intact, even if you use Virtual Machine Manager.</p>
<h2id="configuring-the-virtual-cpu">Configuring the Virtual CPU</h2>
<p>First, we must ensure that the virtual machine’s CPU model is set to <code>host-passthrough</code>. <strong>It is very important that you understand the difference between the “Copy host CPU configuration” checkbox and the</strong><code>host-passthrough</code> CPU model: The former picks a similar virtual CPU that simply covers the feature set of the host CPU, where the latter configures QEMU’s virtual CPU to directly masquerade as the host’s CPU model.</p>
<p>The below figure containing screenshots of Task Manager in the guest operating system demonstrates the difference between the two configurations:</p>
<p>Run virsh edit <code><machine></code> to edit the configuration of your virtual machine (where <code><machine></code> is the name of your virtual machine’s libvirt profile). Find the <code><cpu></code> node in the XML tree and edit the value of its <code>mode</code> attribute so that it looks similar to the following line.</p>
</code></pre><p><em>Note: You may save the configuration and reboot after every modification to observe how it modifies the virtualization environment from your guest operating system’s perspective.</em></p>
<h2id="hiding-the-virtualization-environment">Hiding the Virtualization Environment</h2>
<p>The first step to hide the virtualization environment is to disable the machine specific registers relating to KVM paravirtualization. Add the following code inside the <code><features></code> node of your virtual machine’s configuration using the <code>virsh edit <machine></code>command.</p>
<pretabindex="0"><code><kvm>
<hidden state='on'/>
</kvm>
</code></pre><p>Next, select one of the following methods to try to defeat virtualization detection from the guest operating system. There are two methods that can be used to hide the virtualization environment in Windows® 10:</p>
<ul>
<li>
<p>Method A is the preferred method since it makes the virtualization environment harder to detect in the guest operating system and provides great performance if your CPU supports <code>constant_tsc</code>.</p>
</li>
<li>
<p>Method B is an alternative that uses Hyper-V enlightenments for improved performance on systems that don’t support <code>constant_tsc</code>, but it may be easier to defeat via driver updates.</p>
</li>
</ul>
<p><em>Note: Each method is mutually exclusive since Method B requires the hypervisor CPUID bit to be set so that Windows® will recognize and use the Hyper-V clock.</em></p>
<p>Open your virtual machine’s configuration for editing again by running virsh edit <!-- raw HTML omitted --> and pick one of the following methods to follow. It is recommended to reboot and perform benchmarks after each modification (where permitted by applicable end-user license agreements) so that you may determine which is the best performing configuration for your system.</p>
<p>Open your virtual machine’s configuration for editing again by running <code>virsh edit <machine></code> and pick one of the following methods to follow. It is recommended to reboot and perform benchmarks after each modification (where permitted by applicable end-user license agreements) so that you may determine which is the best performing configuration for your system.</p>
<h2id="method-a-disabling-the-hypervisor-cpuid-bit">Method A: Disabling the Hypervisor CPUID Bit</h2>
<p>Inside the <code><cpu></code> block of your virtual machine’s configuration, add the following line to disable the hypervisor CPUID bit.</p>
</code></pre><p>This line should completely hide the virtualization environment from the perspective of the guest operating system, thus causing any virtualization check to pass.</p>
<h2id="method-b-adjusting-the-hyper-v-enlightenments">Method B: Adjusting the Hyper-V Enlightenments</h2>
<p>Inside the <code><features></code> node of your virtual machine’s configuration, make sure that you have a <code><hyperv></code> node that looks similar to the below example. You may use this example verbatim in your configuration.</p>
</code></pre><p>The <code><hyperv></code> node’s contents will ensure that Hyper-V enlightenments are available to the guest operating system for higher performance. The <code>vendor_id</code> tag overrides the default Hyper-V vendor ID to something unexpected by the graphics drivers, causing them to successfully pass the virtualization check.</p>
<p>Next, inside the <code><clock></code> node of your virtual machine’s configuration, add the following line to enable the Hyper-V clock.</p>
<h2id="passing-keyboardmouse-via-evdev">Passing keyboard/mouse via Evdev</h2>
<p>If you do not have a spare mouse or keyboard to dedicate to your guest, and you do not want to suffer from the video overhead of Spice, you can setup evdev to share them between your Linux host and your virtual machine.</p>
<p><em>Note: By default, press both left and right Ctrl keys at the same time to swap control between the host and the guest.
You can change this hotkeys. You need to set grabToggle variable to one of available combination Ctrl+Ctrl, Alt+Alt, Shift+Shift, Meta+Meta, ScrollLock or Ctrl+ScrollLock for your keyboard. More information: <ahref="https://github.com/libvirt/libvirt/blob/master/docs/formatdomain.rst#input-devices"target="_blank">https://github.com/libvirt/libvirt/blob/master/docs/formatdomain.rst#input-devices</a></em></p>
<p>First, find your keyboard and mouse devices in /dev/input/by-id/. Only devices with event in their name are valid. You may find multiple devices associated to your mouse or keyboard, so try cat /dev/input/by-id/device_id and either hit some keys on the keyboard or wiggle your mouse to see if input comes through, if so you have got the right device. Now add those devices to your configuration:</p>
</code></pre><p>Replace MOUSE_NAME and KEYBOARD_NAME with your device path. Now you can startup the guest OS and test swapping control of your mouse and keyboard between the host and guest by pressing both the left and right control keys at the same time.</p>
<p>You may also consider switching from PS/2 to Virtio inputs in your configurations. Add these two devices:</p>
</code></pre><p>The virtio input devices will not actually be used until the guest drivers are installed. QEMU will continue to send key events to the PS2 devices until it detects the virtio input driver initialization. Note that the PS2 devices cannot be removed as they are an internal function of the emulated Q35/440FX chipsets.</p>
