dev/nixfiles
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
master
nixfiles/docs/sites/home/palace.md
T
jackos1998 08605ab422 docs/boxes: Prioritize hardware details 
Place physical hardware inventories and VPS resource allocations near
the top of each box page, while leaving broader platform details in
their topical sections.
2026-08-02 00:23:30 +01:00

3.3 KiB
Raw Permalink Blame History

palace

The physical VM host for the home site. Runs the river, cellar and sfh VMs and feeds them SR-IOV VFs, PCI NVMe drives and LVM disks.

Hardware

Component Inventory
Platform Gigabyte X399 DESIGNARE EX
CPU AMD Ryzen Threadripper 1950X (16 cores / 32 threads)
Memory 128 GiB
Host storage 500 GB Samsung SSD 860 EVO containing the EFI partition and the main LVM thin pool
Bulk storage Three 8 TB Seagate IronWolf disks in the hdds VG, providing the RAID-backed hdd-storage and frigate LVs
NVMe storage Three 2 TB Samsung NVMe devices passed through to cellar; SPDK combines them as the NVMeRaid RAID 0 device
Network / graphics Mellanox ConnectX-4 100G adapter with four SR-IOV VFs, two Intel I211 Gigabit Ethernet controllers, and an AMD Radeon RX 550/560-family GPU

Role

  • Home hypervisor: VMs are declared in my.vms.instances (palace/vms/default.nix); disks are LVs in the main thin pool (services.lvm.boot.thin.enable).
  • AMD box (kvm-amd, amd_iommu=on, microcode updates); the kernel is built with ACPI_APEI_PCIEAER/PCIEAER for the PCIe passthrough work below.

Network assignments

See the consolidated network assignments table (this box: palace).

Networking

100G et100g (mlx5, MTU 9000) uplinks to the dave switch and carries lan-hi (VLAN 100, the hi assignment). A udev rule creates four SR-IOV VFs on the PF:

VF Consumer VLAN handling
0 cellar hi
1 river untagged VF; river tags its LAN and WAN VLANs
2 sfh hi
3 sfh container MACVLAN parent hi
  • lan-core is a bridge with the core assignment (no gateway); the 1G lan-core-phy and the lan-lo-phy VLAN ride on it. lan-lo is a second, L3-less bridge used for VM netboot and lo clients.
  • The 1G et1g0 (igb) is exported to river as a passthru-mode macvtap (vm-et1g0) — river sees it as wan-old.

VMs

VM vCPUs RAM Passthrough Notes
cellar 8c × 2t 16 GiB VF 0 (44:00.1); NVMe 41:00.0–43:00.0 pinned to NUMA node 1; split IRQ chip + vIOMMU
river 3c × 2t 4 GiB VF 1 (44:00.2); macvtap vm-et1g0 only an ESP local disk (plus an installer ISO) — root is NVMe-oF from cellar
sfh 8c × 2t 32 GiB VF 2 (44:00.3), VF 3 (44:00.4); two USB host ports no boot disk — netboots; gets the hdds/frigate LV

Boot ordering is enforced with systemd dependencies: vm@river waits for cellar's SSH port (and for the vm-et1g0 device), and vm@sfh waits for river — storage first, then the router, then everything that boots off both.

Notable config files

Reference in New Issue View Git Blame Copy Permalink