nixfiles/docs/sites/home/palace.md

palace

The physical VM host for the home site. Runs the river, cellar and sfh VMs and feeds them SR-IOV VFs, PCI NVMe drives and LVM disks.

• Source: nixos/boxes/home/palace/default.nix (VM definitions in palace/vms/default.nix)
• Host: physical
• nixpkgs: mine-stable

Role

• Home hypervisor: VMs are declared in my.vms.instances (palace/vms/default.nix); disks are LVs in the main thin pool (services.lvm.boot.thin.enable).
• AMD box (kvm-amd, amd_iommu=on, microcode updates); the kernel is built with ACPI_APEI_PCIEAER/PCIEAER for the PCIe passthrough work below.

Network assignments

See the consolidated network assignments table (this box: palace).

Hardware

Component	Inventory
Platform	Gigabyte X399 DESIGNARE EX
CPU	AMD Ryzen Threadripper 1950X (16 cores / 32 threads)
Memory	128 GiB
Host storage	500 GB Samsung SSD 860 EVO containing the EFI partition and the main LVM thin pool
Bulk storage	Three 8 TB Seagate IronWolf disks in the hdds VG, providing the RAID-backed hdd-storage and frigate LVs
NVMe storage	Three 2 TB Samsung NVMe devices passed through to cellar; SPDK combines them as the NVMeRaid RAID 0 device
Network / graphics	Mellanox ConnectX-4 100G adapter with four SR-IOV VFs, two Intel I211 Gigabit Ethernet controllers, and an AMD Radeon RX 550/560-family GPU

Networking

100G et100g (mlx5, MTU 9000) uplinks to the dave switch and carries lan-hi (VLAN 100, the hi assignment). A udev rule creates four SR-IOV VFs on the PF:

VF	Consumer	VLAN handling
0	cellar	hi
1	river	untagged VF; river tags its LAN and WAN VLANs
2	sfh	hi
3	sfh container MACVLAN parent	hi

• lan-core is a bridge with the core assignment (no gateway); the 1G lan-core-phy and the lan-lo-phy VLAN ride on it. lan-lo is a second, L3-less bridge used for VM netboot and lo clients.
• The 1G et1g0 (igb) is exported to river as a passthru-mode macvtap (vm-et1g0) — river sees it as wan-old.

VMs

VM	vCPUs	RAM	Passthrough	Notes
cellar	8c × 2t	16 GiB	VF 0 (44:00.1); NVMe 41:00.0–43:00.0	pinned to NUMA node 1; split IRQ chip + vIOMMU
river	3c × 2t	4 GiB	VF 1 (44:00.2); macvtap vm-et1g0	only an ESP local disk (plus an installer ISO) — root is NVMe-oF from cellar
sfh	8c × 2t	32 GiB	VF 2 (44:00.3), VF 3 (44:00.4); two USB host ports	no boot disk — netboots; gets the hdds/frigate LV

Boot ordering is enforced with systemd dependencies: vm@river waits for cellar's SSH port (and for the vm-et1g0 device), and vm@sfh waits for river — storage first, then the router, then everything that boots off both.

Notable config files

• nixos/boxes/home/palace/default.nix — host hardware, networkd (links/bridges/SR-IOV), LVM.
• nixos/boxes/home/palace/vms/default.nix — VM instances and boot ordering.