Intro

In this chapter, I intended to explain the management of Linux Containers (LXC) via virt-manager and the direct kernel boot option. Sadly, I could only make LXC containers work, not LXC VM. Also, no luck on direct kernel boot yet. As a result, this chapter just contains a few notes for reference only.

Notes

Knowledge Base

Simply an improvement of Adding an LXC container with/to virt-manager as I use more up-to-date Incus and virt-manager, these are linked in the article (which contains unnecessary setup you don’t have to do nowadays with LXD and Incus):

• Unprivileged LXC container with libvirt - 2021
• Linux Containers via LXC and Libvirt - 2013

Also a few debug info for my own reference:

• “QEMU failed to run feature checks” – how to find out what checks?: missing OVMF firmware
• Related to Void incus package: incus/issue#475, incus/issue#606 and void-packages/PR#50251
• TL;DR: to use incus VM on Void, install edk2-ovmf. And it’s currently impossible to use incus VM on arm platforms as Void does not offer OVMF for those architectures.
• Use incus with Debian/Devuan if you wanna a good experience on rpi and incus dev also packages patched LXD Web UI (developed by Canonical) which comes in handy if you prefer GUI. Sadly I couldn’t find a good TUI like lazydocker for LXC/LXD as lazy-lxd is already archived.

Some preliminary knowledge you should know when running LXC on libvirt:

• Incus uses KVM/QEMU under the hood for VM implementation, thus having the capability to manage it under libvirt
• Libvirt uses XML under the hood for VM configuration, enable XML editing in virt-manager and apply the template below wisely
• Container rootfs usually does not have the required files for a VM usage, and NOT ALL LXC images can be used as Incus VM
• You can find the path of incus storage pool via incus storage list, or simply create a new pool if you somehow have permission issue
• While /sbin/init probably is a symlink to the init system of desired distro, you should at least know what’s actually used (SystemD, SysVinit, OpenRC, runit, s6 et cetera) in case of boot failure

Example

I’ll use Debian Trixie LXC image as an example (as my favorite Devuan cannot be run in VM mode):

# pull the image and launcher a VM named "debian-vm"
incus launch images:debian/trixie debian-vm --vm
# stop it
incus stop debian-vm

My updated incus VM template:

• <name>debian</name>: container name
• <source dir='/var/lib/incus/storage-pools/default/virtual-machines/debian/rootfs'/>: rootfs path, change virtual-machines to containers if you ignore my warning and insist the usage of container.
• <init>/sbin/init</init>: init script, in case it fails, change to the real path of symlink. In case you really want to use container, change to /sbin/init to /bin/bash.

<domain type="lxc">
  <name>debian</name>
  <description>Debian Incus VM</description>
  <memory unit="KiB">2097152</memory>
  <currentMemory unit="KiB">2097152</currentMemory>
  <vcpu placement="static">2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch="x86_64">exe</type>
    <init>/sbin/init</init>
  </os>
  <clock offset="utc"/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/incus/storage-pools/default/virtual-machines/debian/rootfs"/>
      <target dir="/"/>
    </filesystem>
    <interface type="network">
      <source network="default"/>
    </interface>
    <console type="pty">
      <target type="lxc" port="0"/>
    </console>
  </devices>
</domain>

Once you get it running, it should be safe to add new components via GUI, no more XML editing.

One more side note on my keeping warning about container usage: for many images it would NOT work due to missing essential packages. And if you ask about the direct kernel boot option in virt-manager, no, in LXC domain this simply won’t show up in boot option (I did not try XML editing but chances are it still would not work). You can’t just boot the container with host system kernel with libvirt (I get the idea of shared kernel, just like WSL2-Linux-Kernel, but WSL implementation is quite different). However, it’s possible to do this in QEMU/KVM, use that if you wanna such feature.

Foreword

As my attempt to install Windows Server 2025 for ARM Preview just failed a few weeks ago (due to it introduced ARMv8.1 CPU requirements in Windows 11 25163+, which is unavailable for my rpi4), I’m afraid later chapters would not get updated any time soon. And for listed Windows Server images on UUPdump, it’s either not providing ARM64 images (like Server 2022), or UUPdump won’t be able to generate an ISO (due to either cumulative update or inaccessible architecture packages).

Moreover, Windows on Raspberry (WoR) project is pretty much abandoned nowadays, and rpi5 heads to a different route than many SBC hobbyists expected. Maybe I’d buy a toy RISC-V SBC for like $15 and see what it can do (MS-DOS 6.22/Windows 95 via QEMU as an emulator or LXC + Wine, and I’m pretty sure that latter is feasible), but I would not expect that happening quite soon.