nixos/home: Separate router storage and uplinks

Give `river` a dedicated RDMA VF and two RSTP trunk attachments. Keep client services on the router VIPs and carry Digiweb VLANs over both switch paths. Set VF trust, multicast reception, bridge MAC and ARP behavior for local switching. Wait for Kea interface readiness and apply RSTP costs through `mstpd` despite port-registration and rename races. Load `palace`'s RAID module in the initrd. Record RouterOS and RouterBOOT upgrades and validate port, carrier and switch outages. Document the deployed architecture, interface handoffs and recovery procedures. Add physical socket maps, NIC capacities and cable connections alongside the logical topology and failover checks.
2026-11-15 16:23:06 +00:00
parent b02948d826
commit a04d11423a
26 changed files with 2037 additions and 243 deletions
@@ -31,16 +31,21 @@ See the consolidated [network assignments](../../../networking.md#box-assignment

 ## Networking

-Four NICs, all MTU 9000 where jumbo-capable:
+| Interface | `palace` attachment | Purpose |
+|---|---|---|
+| `lan-hi` | `sfh-hi` SR-IOV VF 2 | Box's `hi` assignment and NVMe/RDMA root path; MTU 9000 |
+| `ctrs-hi` | `sfh-ctrs` SR-IOV VF 3 | L3-less MACVLAN parent for each container's `host0`; MTU 9000 |
+| `ctrs-core` | Virtio via `sfh-core-tap`, `lan-core` and `host-jim` | L3-less MACVLAN parent for UniFi's `lan-core` management attachment |
+| `ctrs-lo` | Virtio via `sfh-lo-tap`, `lan-lo` and `host-jim` | L3-less MACVLAN parent for `hass`'s `lan-lo` and the VM's PXE path |

- `lan-hi` — SR-IOV VF 2, the box's own `hi` assignment.
- `lan-hi-ctrs` — SR-IOV VF 3, no L3: the MACVLAN parent for the containers' `hi` interfaces
-  (`host0` inside each container).
- `lan-core-ctrs` / `lan-lo-ctrs` — virtio NICs (bridged to palace's `lan-core` / `lan-lo`), no
-  L3: MACVLAN parents for containers that need a `core` or `lo` interface.
+The two VFs separate the box's RDMA root traffic from container `hi` traffic; they do not provide
+redundant switch paths. The Virtio attachments reach the `jim`-facing `palace` host bridges. Container
+`host0` names are intentionally unchanged.

 The per-container MACVLAN wiring lives in `systemd.nspawn.*.networkConfig` in
-[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix).
+[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix). The
+[`palace` interface topology](../palace.md#end-to-end-interface-topology) traces these interfaces
+back through the VM handoffs to the physical switch connections.

 ## Containers

@@ -47,8 +47,8 @@ Passed through from `sfh` (USB host ports on `palace`):

 ## Networking

-MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment,
-alt name `frigate`, default gateway via the VIP) and `lan-lo` on `lan-lo-ctrs` (the `lo`
+MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `ctrs-hi` (the `hi` assignment,
+alt name `frigate`, default gateway via the VIP) and `lan-lo` on `ctrs-lo` (the `lo`
 assignment, no gateway) — the `lo` interface reaches the IoT devices (the Reolink camera lives there).

 ## Notable config files
@@ -29,8 +29,8 @@ management network. It is imported, listed in `sfh`'s `my.containers.instances`,

 ## Networking

-Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` carries the `hi`
-assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `lan-core-ctrs` carries
+Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `ctrs-hi` carries the `hi`
+assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `ctrs-core` carries
 the gatewayless `core` assignment (`unifi-ctr-core`). The `core` interface is how the controller
 talks to `brian` and the other switches on their management network.