# palace

The physical VM host for the home site. Runs the `river`, `cellar` and `sfh` VMs and feeds them
SR-IOV VFs, PCI NVMe drives and LVM disks.

- **Source:** [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix)
  (VM definitions in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
- **Host:** physical
- **nixpkgs:** `mine-stable`

## Hardware

| Component | Inventory |
|---|---|
| Platform | Gigabyte X399 DESIGNARE EX |
| CPU | AMD Ryzen Threadripper 1950X (16 cores / 32 threads) |
| Memory | 128 GiB |
| Host storage | 500 GB Samsung SSD 860 EVO containing the EFI partition and the `main` LVM thin pool |
| Bulk storage | Three 8 TB Seagate IronWolf disks in the `hdds` VG, providing the RAID-backed `hdd-storage` and `frigate` LVs |
| NVMe storage | Three 2 TB Samsung NVMe devices passed through to `cellar`; SPDK combines them as the `NVMeRaid` RAID 0 device |
| Network / graphics | Mellanox ConnectX-4 100G adapter with four SR-IOV VFs, two Intel I211 Gigabit Ethernet controllers, and an AMD Radeon RX 550/560-family GPU |

## Role

- Home hypervisor: VMs are declared in `my.vms.instances`
  ([`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix)); disks are LVs in
  the `main` thin pool (`services.lvm.boot.thin.enable`).
- AMD box (`kvm-amd`, `amd_iommu=on`, microcode updates); the kernel is built with
  `ACPI_APEI_PCIEAER`/`PCIEAER` for the PCIe passthrough work below.

## Network assignments

See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `palace`).

## Networking

100G `et100g` (mlx5, MTU 9000) uplinks to the `dave` switch and carries `lan-hi` (VLAN 100, the
`hi` assignment). A udev rule creates four SR-IOV VFs on the PF:

| VF | Consumer | VLAN handling |
|---|---|---|
| 0 | `cellar` | `hi` |
| 1 | `river` | untagged VF; `river` tags its LAN and WAN VLANs |
| 2 | `sfh` | `hi` |
| 3 | `sfh` container MACVLAN parent | `hi` |

- `lan-core` is a bridge with the `core` assignment (no gateway); the 1G
  `lan-core-phy` and the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used
  for VM netboot and `lo` clients.
- The 1G `et1g0` (igb) is exported to `river` as a passthru-mode macvtap (`vm-et1g0`) — river sees
  it as `wan-old`.

## VMs

| VM | vCPUs | RAM | Passthrough | Notes |
|---|---|---|---|---|
| `cellar` | 8c × 2t | 16 GiB | VF 0 (`44:00.1`); NVMe `41:00.0`–`43:00.0` | pinned to NUMA node 1; split IRQ chip + vIOMMU |
| `river` | 3c × 2t | 4 GiB | VF 1 (`44:00.2`); macvtap `vm-et1g0` | only an ESP local disk (plus an installer ISO) — root is NVMe-oF from `cellar` |
| `sfh` | 8c × 2t | 32 GiB | VF 2 (`44:00.3`), VF 3 (`44:00.4`); two USB host ports | no boot disk — netboots; gets the `hdds/frigate` LV |

Boot ordering is enforced with systemd dependencies: `vm@river` waits for `cellar`'s SSH port (and
for the `vm-et1g0` device), and `vm@sfh` waits for `river` — storage first, then the router, then
everything that boots off both.

## Notable config files

- [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix) — host
  hardware, networkd (links/bridges/SR-IOV), LVM.
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
  VM instances and boot ordering.