# cellar

The home storage target. A VM on `palace` that drives three passed-through NVMe disks with SPDK
and exports them over NVMe-oF/RDMA — `river`, `sfh` and `castle` all run their root storage off
it.

- **Source:** [`nixos/boxes/home/palace/vms/cellar/`](../../../nixos/boxes/home/palace/vms/cellar)
  (`default.nix`, `spdk.nix`)
- **Host:** VM on `palace`

## Role

- Runs an **SPDK userspace target** (`my.spdk`, [`spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix)):
  the kernel `nvme` driver is blacklisted so SPDK can claim the three NVMe controllers directly
  (host BDFs `41:00.0`–`43:00.0`, attached in the guest as `02:00.0`–`04:00.0`).
- Builds a **RAID-0** (`NVMeRaid`, 64 KiB strip) across the three drives and exports one
  partition per consumer as an **NVMe-oF subsystem over RDMA** (port 4420) on the `hi` network:

  | Bdev | NQN | Consumer |
  |---|---|---|
  | `NVMeRaidp1` | `nqn.2016-06.io.spdk:river` | [`river`](river.md) |
  | `NVMeRaidp2` | `nqn.2016-06.io.spdk:castle` | [`castle`](castle.md) |
  | `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh.md) |

  Each subsystem is pinned to its consumer's `hostnqn` (the `my.nvme.uuid` on the client side).
- `spdk-tgt` is ordered after `lan-hi` is online; the RDMA listener binds the `hi` address
  (`192.168.68.80:4420`). The VM itself is pinned to NUMA node 1 on `palace` and gets SR-IOV VF 0.
- `netdata` (port 19999 allowed in the firewall) and `fstrim`.
- Deploy address `192.168.68.80` (`my.deploy.node.hostname`).

## Network assignments

<!-- assignments: cellar -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| cellar | hi | `192.168.68.80/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:1/64` | h.nul.ie |  |
<!-- assignments-end -->

## Notes

- The `ublk_*` calls in `my.spdk.debugCommands` are only a debugging aid — they create a local
  ublk device so the RAID can be mounted and inspected on `cellar` itself. Client exports are the
  `nvmf` subsystems above.

## Notable config files

- [`nixos/boxes/home/palace/vms/cellar/default.nix`](../../../nixos/boxes/home/palace/vms/cellar/default.nix) —
  box config (assignment, networking, netdata).
- [`nixos/boxes/home/palace/vms/cellar/spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix) —
  SPDK target: RAID-0, NVMe-oF/RDMA subsystems.
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
  the VM definition on `palace` (VF 0, NVMe passthrough, NUMA pinning).