This short document will show how to turn systemd-nspawn into a usable containeration system.
Those instructions here should work under Arch Linux and Debian 12 (Bookworm)
We need to create a network bridge device where systemd-nspawn can connect all the containers.
Using systemd-networkd you can create those two files under /etc/systemd/network/.
/etc/systemd/network/br0.netdev
# Tell systemd-networkd to create a bridge device
[NetDev]
Name=br0
Kind=bridge
/etc/systemd/network/br0.network
# Configure ip address for the bridge
[Match]
Name=br0
[Network]
Address=172.23.0.1/24
LinkLocalAddressing=yes
IPMasquerade=yes
LLDP=yes
EmitLLDP=customer-bridge
After creating those configuration files and running command below you should be good.
systemctl enable --now systemd-networkd
If you need to make any changes to the network configuration then you can just restart systemd-networkd to reload the config.
systemctl restart systemd-networkd
So that all of our containers would always get a static ip address we are gona use dnsmasq.
Next we are going to tell dnsmasq to do static leases based on container hostname. In this config file im using 172.23.0.1/24 as the container network.
# /etc/dnsmasq.conf
domain=.local
#no-poll # don't constanly poll /etc/resolv.conf
#resolv-file=/etc/resolv.conf
no-resolv
server=8.8.8.8
server=8.8.4.4
domain-needed
bogus-priv
listen-address=127.0.0.1,172.23.0.1
# Force bind only on interfaces that are listed in listen-address, allows systemd-resolve to work at the same time
bind-dynamic
dhcp-range=172.23.0.100,172.23.0.200,255.255.255.0,12h
# When systemd-networkd or some other dhcp client requests for a ip address
# they will be given a address based on their hostname.
# This section can be moved to seperate file using dhcp-hostsfile option
dhcp-host=http,172.23.0.2
dhcp-host=mail,172.23.0.3
dhcp-host=worpress,172.23.0.4
dhcp-host=sql,172.23.0.5
systemd-nspawn@.service must be told to use the br0 bridge
For that we have to edit ExecStart line in the service file and add --network-bridge=br0 to the end of it.
First get the currently used ExecStart value by running
systemctl cat systemd-nspawn@.service
After that we need to create a override.conf file that changes that file.
You can do that either by using command systemctl edit systemd-nspawn@.service or by manualy creating a file called /etc/systemd/system/systemd-nspawn@.service.d/override.conf
Contents of the override file should be something like this:
[Service]
ExecStart=
ExecStart=<current execstart line> --network-bridge=br0
For Arch Linux the contents would be
[Service]
ExecStart=
ExecStart=/usr/bin/systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --network-veth -U --settings=override --machine=%i --network-bridge=br0
NB: nspawn service by default adds -U argument that turns on private users support and shifts all UID/GID-s up some random amount. If you plan on sharing files between containers then this will mess up yout file owners.
You can enable mymachines nsswitch module that will do user and group id translation between host and container private users. https://www.freedesktop.org/software/systemd/man/nss-mymachines.html
That part is quite simple, just create a folder /var/lib/machines/<name> where name is your container hostname
and deboostrap into it. machinectl start <name> should boot your container up and after staring systemd-networkd you should have a working network.
I have created some scripts automate container creation part
*create_arch_container.sh - Creates a Arch Linux container
*create_deb_container.sh - Creates a Ubuntu 16.04 container, but you can change it quite easily by changing SUITE argument from xenial to something else
*create_container.sh - Creates a Debian 10 container on a ZFS pool, also configures locale and timezone inside the container.
NB: debootstrap copies your host /etc/hostname file into the container, remove it or replace it with your container hostname
*machinectl start <name> - Boot container named name
*machinectl poweroff <name> - Poweroff the container
*machinectl shell <name> [/bin/sh] - Run a command in the container, by default the command is /bin/sh. For Debian/Ubuntu containers you probably want to tell it to use /bin/bash
*machinectl restart <name> - this will probably break nspawn because on startup it will try to register
*Port forwarding
If there is a file /etc/systemd/nspawn/<name>.nspawn then systemd-nspawn will load additional configuration options from it.
To do port forwarding you can do something like this:
# /etc/systemd/nspawn/http.nspawn
[Network]
Port=tcp:80:80
Port=tcp:443:443
You can add many more options from systemd.nspawn(5) man page
/etc/systemd/system/systemd-nspawn@paper.c7.ee.service.d/override.conf
[Service]
LimitNOFILE=65535
ExecStart=
ExecStart=/usr/bin/systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --network-veth --settings=override --machine=%i --network-bridge=br-zoo --bind=/fryer/jtagcat --system-call-filter="add_key keyctl bpf" --bind=/dev/fuse
systemctl set-property systemd-nspawn@paper.c7.ee DeviceAllow='/dev/fuse rwm' # sets a bunch of allows in /etc/system/system.control
systemctl daemon-reload
systemctl enable --now systemd-nspawn@paper.c7.ee
/etc/systemd/network/50-host.network
[Match]
Name=host0
[Link]
RequiredForOnline=yes
[Network]
DHCP=yes
IPv6Token=::c7:ee
IPv6AcceptRA=yes
Address=193.40.103.7/24
Address=62.65.250.7/26
Gateway=193.40.103.1
Address=2001:bb8:4008:20::c7:ee/64
Gateway=2001:bb8:4008:20::1
LLMNR=yes
MulticastDNS=yes
IPForward=yes
[DHCP]
UseDomains=yes
Debian 10 (Buster) defaults to modern nftables but systemd still uses legacy iptables framework for firewall configuration. Don't mix those two.