HOW TO CONFIGURE WINDOWS 11 FOR OPENSTACK

To quote a one of my fellow researchers “Not a single person on this planet understands fully how Openstack works”. Throw in Windows 11 and you’ve got yourself a recipe for carnage. Here is how to configure a virtual machine in VirtualBox using Windows 11, and upload the image with proper settings (or magic) to work with Openstack.

PREP

Download Windows 11 & Virtualbox

The first thing you need to do is get VirtualBox downloaded and installed onto your machine. For the purposes of this tutorial I will be using a Windows 11 machine with WSL2(Ubuntu 22.04) for the command line interface.

Windows 11: https://www.microsoft.com/en-us/windows
VirtualBox: https://www.virtualbox.org/

VBOX CONFIG

Open VBOX and create new .vdi

Once VirtualBox has been installed, open the application and launch a new virtual machine by clicking on the NEW tab.

NOTE: You may need to enable EXPERT MODE located in the bottom of the new tab for some following options to appear.

Configure basic settings

Configure the following for your new VM:

  1. Name your VM: For my puproses I called it win11_virt_demo

  2. Load your win11 ISO: mine was located: C:\home\Downloads

  3. Set your version of win11: I set mine to Education

NOTE: You may want to uncheck unattended installation, as this is not entirely necessary but gives you manual control over the user you set up during windows 11’s initial boot phase (which will also prevent your VM from booting automatically which is needed because their are additional configuration steps before you can boot your VM). I found unchecking unattended install resulted in my installations being less buggy, but am not sure if this was directly the cause or something else.

Configure HARDWARE settings

Configure your hardware to the following (may be default) settings:

  1. Set RAM to >= 4096MB

  2. Set VCPU’s to 2

  3. Enable EFI (for special os’s only (and boy-howdy is win11 special))

NOTE: MAKE SURE YOU CHECK ENABLE EFI, if you don’t you will not be able to boot the OS properly. Or maybe you will, to be honest only the Great Old Ones know how this works.

Configure HARDDISK settings

Configure your disk drive to the following settings:

  1. Check create a virtual hard disk now

  2. Set size >=40GB: I set mine to 80GB for safeties sake

  3. Leave hard disk type as a .vdi: you can try something else, but don’t say I didn’t warn you

Click Finish to close out this panel.

NOTE: DO NOT BOOT YOUR VM, and if for some reason it does boot, make sure to shut it down as soon as possible. Booting your VM now will very likely break your system as you need to install the VIRTIO drivers before Win11 installs, with instructions on how to do this in the following steps of this guide.

V irtio CONFIG

OPen settings

Open the settings tab in VirtualBox. Make sure that the state underneath your virtual machine says powered off.

add virtio driver hard drive

In settings, click Storage and then the Controller:SATA.

We need to add the VIRTIO drivers as another hard drive to boot during windows installation. Click on the floppy disk with a green plus sign.

Attach a new optical disk containing the VIRTIO drivers by:

  1. Select new optical disk from the aformentioned floppy disk icon

  2. Add the VIRTIO iso drivers from the directory in which they were downloaded: mine was again in C:\home\Downloads\ Select the virtio iso that appears in the attached volumes

  3. Click choose

You will know you have done this step correctly when a new optical drive appears attached in the storage devices section.

NOTE: Make sure the drive is second or third in the boot order, just as long as it is not first you should be fine.

Configure NETwork adapter settings

Configure Adapter 1 to the following settings:

  1. Set the first network adapter to a bridged adapter in the attached to section.

  2. Set the first network adapter to virtio-net in the advanced settings.

Configure Serial port settings

Enable the first port for logging with cloudinit in the serial ports section.

Click OK to close out this panel.

BOOT win11

Start the VM

Click the start button in the VirtualBox control panel. Make sure to hit enter when the VM boots to boot from drive. If you miss this command, you may have to restart your VM.

INSTALL WIN11 w/custom settings

Click install Windows. Wait until installation progresses to the point where you can select which type of installation you want, and select custom: install windows only.

INSTALL VIRTIO DRIVERS

Once the custom installation screen boots, select load driver from the bottom of the page.

Browse to the CD drive of the Virtio drivers:

  1. Browse to AMD64/WIN11:

  2. Uncheck the hide drivers that aren't compatible with this computer's hardware box.

NOTE: If the VM still fails to boot, you can try other drivers located at VIRTIOSTOR/AMD64/W11.

Select the Red Hat VirtIO SCSI controller and install by clicking next.

Repeat the above steps for the network driver contained in the Virtio drive folder: NetKVM/W11/amd64 entitled Red Hat VirtIO Ethernet Adapter.

BOOT windows from drive 0

Select Drive 0 Unallocated Space and click next to begin installing Windows normally.

INSTALL:
CLOUDINIT,
RDP,
ETC…

ENAble Remote Desktop Protocol

Once you are logged in to the home screen, bring up powershell as administrator and run the following command:

Enable-NetFirewallRule -name RemoteDesktop-UserMode-In-TCP

Then, make sure to open up Remote Desktop Settings, and enable remote desktop.

INSTALL CLOUDINIT

In powershell as administrator, run the following commands:

Set-ExecutionPolicy Unrestricted

In powershell as administrator, run the following commands:

Invoke-WebRequest -UseBasicParsing https://cloudbase.it/downloads/CloudbaseInitSetup_Stable_x64.msi -OutFile cloudbaseinit.msi ./cloudbaseinit.msi

Once the installer opens, click next until you reach the configuration options and enable COM1 as the serial port for logging.

Once the installation finished, make sure to check the run sysprep and shutdown boxes to prepare your VM for upload into openstack. Click finish and let your VM shutdown fully.

Upload vm to openstack

CONVERT IMAGE TO QCOW2

Congrats! If you’ve made it this far that means their is still plenty of time for something to go horribly wrong. Luckily, you won’t know until the very end of this process, so its time to find out if the Openstack gods have blessed you with salvation.

First, you need to convert your .vdi file to a .qcow2 before uploading into openstack using the below command:

qemu-img convert -f vdi -O qcow2 win11_virtio_demo.vdi win11_virtio_demo.qcow2

NOTE: My VM would sometimes fail when doing this convert on my WSL2 system. If you have a native Linux system, it may be worth doing the qemu conversion there.

Upload qcow2 to openstack

Upload your image to openstack using the following command:

openstack image create "win11_virtio_demo" --disk-format qcow2 --min-ram 4096 --min-disk 80 --file win11_virtio_demo.qcow2

Once the image has been created successfully, enable UEFI boot on your openstack image with the following command:

openstack image set --property hw_firmware_type=uefi ID-OF-YOUR-VM-HERE

NOTE: You can get the ID of your VM imager using:

openstack image list

Launch instance

Using your image, launch an instance on openstack. Some people prefer the webgui, and some prefere the cmdline interface. If you are unaware on how to do this, here:

https://docs.openstack.org/horizon/latest/user/launch-instances.html

COMMON ERRORS & HOw to fix

IMAGE failed to boot

If you receive the failed to boot or no bootable device found error, but Openstack boots the VM up until this point, likely UEFI boot is not enabled properly.

Make sure to check that you enabled UEFI boot in both setting up the .vdi in VirtualBox and on Openstack in the image settings.

IMAGE FAILED to load at all

If your image fails to load at all, it could be something went wrong during the conversion process from the .vdi to .qcow2.

Make sure to perform the conversion on a native linux machine

OR

Openstack may accept .vdi, and while it will take a lot longer, you can upload the .vdi into Openstack and let it handle the automatic conversion to a .qcow2. Note that the time I tried this it took over an hour to complete.

IMAGE BOOTs, but no RDP

If you managed to get the VM to boot, but are only able to see it via the console view and cannot remote desktop in, try the following steps:

  1. Ping the IP address:

    If it is pingable, but isin’t RDP accessible, then likely an installation step was messed up from above. Make sure to not only install and enable RDP but open the firewall using the command in the steps above.

    If the IP address is NOT pingable, then something is likely wrong in Openstack. Make sure you enabled the ssh port in your instance, and make sure that connections are allowed through on your network.

  2. Be Patient and Wait:

    I noticed that most Windows VM’s took a long time to properly boot, and sometimes even longer before RDP was up. I would say 1hr is a safe bet for most systems to boot properly, but especially if you uploaded a .vdi may need to wait an entire day to be certain.

  3. Retry Installation:

    The nuclear option. It took me 18 reinstalls to finally get the right combination of magic to make this work. Look over all of your settings and make sure everything is exactly as pictured above. If you are still having trouble, their are more guides at the beginning of this one, make sure to read those over in case their is something that I missed there.