siil.arti.ee has support for libvirt based VM-s
Notes for when doing a clean install on a new box
Create a empty zfs filesystem for libvirt virtual machines
sudo zfs create rpool/var/lib/libvirt
Install libvirt and ovmf for uefi vm support
sudo apt install --no-install-recommends libvirt-daemon-system libvirt-clients qemu-kvm qemu-utils ovmf
Add your self to libvirt group
gpasswd --add <user> libvirt
Configure default networking
sudo -e /etc/libvirt/qemu/networks/default.xml
And change it to use existing bridge
<network>
<name>default</name>
<uuid>e9e68fd5-0c59-4bc0-a025-def4c7840cb2</uuid>
<forward mode='bridge'/>
<bridge name='br-int'/>
</network>
Allow users in libvirt group to add iso images into /var/lib/libvirt/boot folder
chown :libvirt /var/lib/libvirt/boot
chmod g+rws /var/lib/libvirt/boot
This way you can easily wget iso files into the boot folder and then boot them up in libvirt without having to become root
This should create a VM image quite fast.
qemu-img create -f qcow2 -o cluster_size=128k -o preallocation=metadata -o lazy_refcounts=on /var/lib/libvirt/images/name.qcow2 20G
virsh --connect qemu:///system domifaddr k3s
example output
Name MAC address Protocol Address
-------------------------------------------------------------------------------
vnet1 52:54:00:0e:6e:f4 ipv4 198.51.100.118/24
Send shutdown signal
virsh --connect qemu:///system shutdown k3s
Force power-off
virsh --connect qemu:///system destroy <machine>
virsh --connect qemu:///system undefine <machine> --remove-all-storage --nvram
Linux kernel has to booted with console=ttyS0 for this to work. After that systemd will start a getty instance on the serial line.
https://wiki.archlinux.org/index.php/Working_with_the_serial_console
From commandline you can connect to the serial like so:
virsh --connect=qemu:///system console <machine>
or
virsh --connect=qemu:///system console --devname serial0 <machine>
Default should be writeback where writes go through linux page cache unless guest does a fsync.
none cache mode bypasses page cache. unsafe is like writeback but ignores fsync so its really fast for OS installs.
https://documentation.suse.com/sles/11-SP4/html/SLES-kvm4zseries/cha-qemu-cachemodes.html
libvirt uses the NoCloud Config drive method of providing cloud-init config for the VM.
Get a generic or a bit smaller libvirt only genericcloud image from https://cloud.debian.org/images/cloud/
wget https://cloud.debian.org/images/cloud/trixie/latest/debian-13-genericcloud-amd64.qcow2 -O /var/lib/libvirt/images/debian-13-genericcloud-amd64.qcow2
Create a sparse VM image and resize it
qemu-img create \
-b /var/lib/libvirt/images/debian-13-genericcloud-amd64.qcow2 \
-f qcow2 \
-F qcow2 \
/var/lib/libvirt/images/<machine>.qcow2 \
10G
For cloud-init you need few files that configure the VM
meta-data.yaml should contain the Instance Metadata. Mainly just the requried libvirt machine name in instance-id field. This is used by cloud-init to detect if its a first boot or not.
instance-id: <machine>
local-hostname: <machine>.example.com
We also need network-config.yaml. Coud-init should default to DHCP but for some reason it didn't work for me. Here we are using networking config version 2
network:
version: 2
ethernets:
all-ethernet:
match:
name: "en*"
dhcp4: true
dhcp6: true
And the last you need a normal standard user-data.yaml that setups rest of the system. Here we are using YAML format but it can also be a shell script. What is chosen depends on the first line in the file. Cloud-init supports several user-data formats.
#cloud-config
# Install all updates on first boot
package_update: true
package_upgrade: true
# As we are defining our own user, the default user is not created
users:
- name: arti
ssh_authorized_keys:
- ssh-ed25519 AAAA...
sudo: ["ALL=(ALL) NOPASSWD:ALL"]
groups: sudo
shell: /bin/bash
# After all is done, reboot the machine
# Reboot also signals libvirt-install that install is finished.
power_state:
delay: now
mode: reboot
message: Reboot
timeout: 2
condition: true
After cloud-init files are created, we can use virt-install to create the VM.
virt-install \
--connect qemu:///system \
--import \
--autostart \
--name <machine> \
--os-variant debian13 \
--memory 4096 \
--vcpus 2 \
--disk /var/lib/libvirt/images/<machine>.qcow2,bus=virtio \
--network default \
--boot uefi \
--cloud-init disable=on,user-data=user-data.yaml,meta-data=meta-data.yaml,network-config=network-config.yaml
Configure serial console boot log
https://www.freebsd.org/doc/handbook/serialconsole-setup.html
sysrc -f /boot/loader.conf console=comconsole