dev/nixfiles
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs: Document the deployment

Browse Source 
Add a top-level README mapping the boxes and a full docs/ tree: topic
pages (architecture, networking, deployment), per-site box pages for
colony and home with containers nested under their hosts, remote and
mobile boxes, the installer, and the home switch fabric reference
(folded in from home-switches.md, with AGENTS.md and code comments
retargeted to its new home). Box pages carry marked assignment tables
that CI regenerates from nixos.allAssignments.

AGENTS.md points at the new docs and keeps its terse agent version of
the mechanics, referring to the topic pages for depth.
This commit is contained in:
Jack O'Sullivan
2026-07-24 00:24:31 +01:00
parent 9b582bec7b
commit d51f2d62b6
45 changed files with 3072 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/remote/README.md
+14
View File
@@ -0,0 +1,14 @@
# Remote boxes
The "remote" group covers the boxes that live outside the `colony` and `home` sites: the two
edge VPSes (`britway` in London, `britnet` in Birmingham) and the `kelder` site — a secondary
server at a remote location, linked back to colony over WireGuard and acting as a NixOS
container host.
| Box | What it is | Docs |
| --- | --- | --- |
| `britway` | Vultr VPS (London, `lon1`): Headscale control plane, Tailscale exit node, BGP edge, nginx | [britway.md](britway.md) |
| `britnet` | VPS (Birmingham, `bhx1`): Tailscale exit node / WireGuard hub | [britnet.md](britnet.md) |
| `kelder` | Secondary home server (`hentai.engineer`): container host, Samba, DDNS | [kelder.md](kelder.md) |
| `kelder-acquisition` | Media stack container on `kelder` (Transmission over VPN, *arr, Jellyfin) | [kelder-acquisition.md](kelder-acquisition.md) |
| `kelder-spoder` | Web container on `kelder` (Nextcloud + nginx reverse proxy) | [kelder-spoder.md](kelder-spoder.md) |
docs/remote/britnet.md
+39
View File
@@ -0,0 +1,39 @@
# britnet
A small VPS in Birmingham (`bhx1`) acting as a second Tailscale/WireGuard egress point — a
narrower gateway role than [`britway`](britway.md) (no control plane, no BGP).
- **Source:** [`nixos/boxes/britnet.nix`](../../nixos/boxes/britnet.nix)
- **Host:** VPS (Birmingham, `bhx1`; provider uplink assignment `allhost`)
## Role
- **Tailscale exit node** — logs into the headscale on [`britway`](britway.md)
(`--login-server=https://hs.nul.ie`) with `--advertise-exit-node`.
- **WireGuard hub** — `wg0` listens on UDP 51820 on the `vpn` network
(`10.200.0.0/24` / `fdfb:5ebf:6e84::/64`), with a single peer at `10.200.0.10` /
`fdfb:5ebf:6e84::10`.
- **NAT gateway** — traffic arriving on `tailscale0`/`wg0` is forwarded out `veth0` and SNATed
to the `allhost` v4/v6 addresses.
## Network assignments
<!-- assignments: britnet -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| britnet | allhost | `77.74.199.67/24 gw 77.74.199.1` | `2a12:ab46:5344:99::a/64 gw 2a12:ab46:5344::1` | bhx1.int.nul.ie | |
| britnet | vpn | `10.200.0.1/24` | `fdfb:5ebf:6e84::1/64` | — | |
<!-- assignments-end -->
## Networking
- The provider interface is renamed to `veth0` by MAC. The v6 default gateway
(`2a12:ab46:5344::1`) sits off-subnet, so a link-scope route is added to reach it.
- `wg0` is a networkd WireGuard netdev keyed from `britnet/wg.key`; RA is disabled on it.
- Upstream DNS is hardcoded to Cloudflare (`1.1.1.1` / `1.0.0.1`).
- `iperf3` runs with an open port for bandwidth testing.
## Notable config files
- [`nixos/boxes/britnet.nix`](../../nixos/boxes/britnet.nix) — the whole box (single file).
docs/remote/britway.md
+51
View File
@@ -0,0 +1,51 @@
# britway
A Vultr VPS in London (`lon1`) acting as the network edge node: the self-hosted Tailscale
control plane, a tailnet exit node, and the BGP speaker for AS211024.
- **Source:** [`nixos/boxes/britway/`](../../nixos/boxes/britway)
- **Host:** VPS at Vultr (London, `lon1`)
## Role
- **Headscale** — the self-hosted Tailscale control plane at `hs.nul.ie`; every other box's
`tailscaled` logs in here (`--login-server=https://hs.nul.ie`). Google OIDC for auth,
SQLite state, MagicDNS under `ts.nul.ie`, and split DNS pointing the colony/home domains
at their internal resolvers.
- **Tailscale exit node** — advertises `--advertise-exit-node` plus routes to the home v4/v6
prefixes; tailnet traffic is SNATed out `veth0` (v4 to the Vultr public IP, v6 to the
`as211024` mesh address).
- **BGP edge** — `bird2` speaks BGP as AS211024 to Vultr transit (AS64515, separate v4/v6
sessions authenticated with a password from `britway/bgp-password-vultr.conf`) and exports
everything to a `bgp.tools` monitoring session. It originates the internal, colony and home
v6 prefixes (`2a0e:97c0:4df::/48`, `2a0e:97c0:4d2::/48`, `2a0e:97c0:4d0::/48`).
- **nginx** — reverse proxy fronting headscale (`hs.nul.ie` → `localhost` headscale port),
with a wildcard ACME cert for `nul.ie` issued via Cloudflare DNS.
## Network assignments
<!-- assignments: britway -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| britway | as211024 | `10.100.50.5/24` | `2a0e:97c0:4df:0:2::1/64` | — | |
| britway | vultr | `45.76.141.188/23 gw 45.76.140.1` | `2001:19f0:7402:128b::1/64` | lon1.int.nul.ie | |
<!-- assignments-end -->
## Networking
- Two assignments: `vultr` on the provider interface `veth0` (renamed by MAC), and `as211024`
on the l2mesh VXLAN interface (`my.vpns.l2`, a mesh shared with `estuary` and the home
routers `river`/`stream`).
- Static routes steer colony/home v4 traffic over the `as211024` mesh. A separate `ts-extra`
routing table (1337) with a policy rule on `tailscale0` ingress sends Tailscale-sourced v6
traffic for colony via `estuary`, while the box's own v6 uses WAN.
- The firewall trusts the `as211024` prefixes (`lib.my.c.as211024.nftTrust`) and
`tailscale0`; `iperf3` runs with an open port for bandwidth testing.
## Notable config files
- [`nixos/boxes/britway/default.nix`](../../nixos/boxes/britway/default.nix) — system, assignments, networkd, firewall/SNAT.
- [`nixos/boxes/britway/bgp.nix`](../../nixos/boxes/britway/bgp.nix) — `bird2` config (Vultr transit, `bgp.tools`).
- [`nixos/boxes/britway/nginx.nix`](../../nixos/boxes/britway/nginx.nix) — nginx vhosts + ACME.
- [`nixos/boxes/britway/tailscale.nix`](../../nixos/boxes/britway/tailscale.nix) — headscale + the tailnet node itself.
docs/remote/kelder-acquisition.md
+48
View File
@@ -0,0 +1,48 @@
# kelder-acquisition
The media acquisition stack for the kelder site — Transmission (forced over VPN), the *arrs
and Jellyfin in one NixOS container.
- **Source:** [`nixos/boxes/kelder/containers/acquisition/`](../../nixos/boxes/kelder/containers/acquisition)
- **Host:** NixOS container on [`kelder`](kelder.md)
## Role
- **Transmission** (`transmission_4`) — BitTorrent client bound to the VPN
(`bindsTo systemd-networkd-wait-online@vpn.service`); peer port 26180 (forwarded in the
AirVPN config), 20 MiB/s down / 1 MiB/s up limits, ratio limit 2.0. Downloads land in
`/mnt/media/downloads/torrents` (`/mnt/media` is bind-mounted from the host's
`/mnt/storage/media`).
- **Jackett, Radarr, Sonarr** — indexer + media managers, in the shared `media` group with
`UMask 0002`.
- **Jellyfin** — streaming with Intel hardware transcoding: `/dev/dri` is bind-mounted from
the host, `intel-vaapi-driver`/`intel-ocl` are installed and the `jellyfin` user is in the
`render` group.
## Network assignments
<!-- assignments: kelder-acquisition -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| acquisition-ctr | internal | `172.16.64.2/24 gw 172.16.64.1` | — | hentai.engineer | |
<!-- assignments-end -->
## Networking
- `internal` assignment (name `acquisition-ctr`) on the host's `ctrs` bridge, MTU 1460 to
match the site WAN.
- All non-site traffic goes over an AirVPN WireGuard tunnel (`vpn` netdev, MTU 1320, AirVPN
IE endpoint) using fwmark 42 / route table 51820: policy rules keep traffic to and from the
kelder prefixes on the main table and push everything else via the VPN.
- An nftables input chain drops new TCP connections from the VPN interface except the
Transmission peer port; the web UI ports (9091 Transmission, 9117 Jackett, 7878 Radarr,
8989 Sonarr, 8096 Jellyfin) are accepted from the site. When built as a dev VM, those ports
are forwarded to the host.
- Sonarr still needs the EOL .NET 6 runtime, allowed via
`nixpkgs.config.permittedInsecurePackages`.
## Notable config files
- [`nixos/boxes/kelder/containers/acquisition/default.nix`](../../nixos/boxes/kelder/containers/acquisition/default.nix) — services and users.
- [`nixos/boxes/kelder/containers/acquisition/networking.nix`](../../nixos/boxes/kelder/containers/acquisition/networking.nix) — AirVPN tunnel + firewall.
docs/remote/kelder-spoder.md
+54
View File
@@ -0,0 +1,54 @@
# kelder-spoder
The web container for the kelder site: Nextcloud plus an nginx (OpenResty) reverse proxy for
the site's services.
- **Source:** [`nixos/boxes/kelder/containers/spoder/`](../../nixos/boxes/kelder/containers/spoder)
- **Host:** NixOS container on [`kelder`](kelder.md)
## Role
- **Nextcloud** (`nextcloud32`) at `cloud.hentai.engineer` (trusted alias
`cloud-local.hentai.engineer`), SQLite backend, data in `/mnt/storage/nextcloud`
(`/mnt/storage` is bind-mounted from the host).
- **nginx reverse proxy** (`openresty`) terminating TLS for the site's public vhosts, with a
wildcard ACME cert for `hentai.engineer` via Cloudflare DNS. The kelder host forwards
`http`/`https` to this container; uploads are unlimited (`clientMaxBodySize = 0`) for
Nextcloud's sake.
## Network assignments
<!-- assignments: kelder-spoder -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| spoder-ctr | internal | `172.16.64.3/24 gw 172.16.64.1` | — | hentai.engineer | |
<!-- assignments-end -->
## Proxy vhosts
All under `hentai.engineer`, each with a `*-local` alias:
| vhost | Target | Auth |
| --- | --- | --- |
| `monitor` | `netdata` on the kelder host (:19999) | `htpasswd` |
| `kontent` | Jellyfin on `kelder-acquisition` (:8096, incl. websocket) | — |
| `torrents` | Transmission on `kelder-acquisition` (:9091) | `htpasswd` |
| `jackett` | Jackett on `kelder-acquisition` (:9117) | `htpasswd` |
| `radarr` | Radarr on `kelder-acquisition` (:7878) | `htpasswd` |
| `sonarr` | Sonarr on `kelder-acquisition` (:8989) | `htpasswd` |
| `cloud` | Nextcloud (local) | — |
An `init_worker_by_lua` timer polls `v4.ident.me` every 5 minutes to track the site's public
IP; it feeds `localRedirect` rewrites (bounce public-IP clients to the `*-local` name) that
are currently **disabled** (commented out — Virgin Media filters DNS answers containing local
IPs, so the split doesn't work as intended).
## Networking
- `internal` assignment (name `spoder-ctr`) on the host's `ctrs` bridge, MTU 1420.
## Notable config files
- [`nixos/boxes/kelder/containers/spoder/default.nix`](../../nixos/boxes/kelder/containers/spoder/default.nix) — Nextcloud + ACME.
- [`nixos/boxes/kelder/containers/spoder/nginx.nix`](../../nixos/boxes/kelder/containers/spoder/nginx.nix) — reverse proxy vhosts.
docs/remote/kelder.md
+67
View File
@@ -0,0 +1,67 @@
# kelder
Secondary home server at a remote site, domain `hentai.engineer`. Linked back to colony over
WireGuard and acting as a NixOS container host (like `shill`/`sfh`).
- **Source:** [`nixos/boxes/kelder/`](../../nixos/boxes/kelder)
- **Host:** physical (Intel; LTS kernel, `kvm-intel`, IOMMU on)
## Role
- **Container host** — runs two NixOS containers on the `ctrs` bridge
(`my.containers.instances`): `kelder-acquisition` and `kelder-spoder` (below).
- **Public services via colony** — a WireGuard tunnel (`estuary` netdev) connects to colony's
`estuary` box, which DNATs public traffic to kelder's `/32`; policy routing (table 51820 +
connmark) sends replies back through the tunnel while ordinary traffic uses the LAN.
kelder's own NAT forwards `http`/`https` on to `kelder-spoder`.
- **Nextcloud host** — served from the `kelder-spoder` container.
- **Samba** — the `storage` share backed by `/mnt/storage`, with `nmbd` and `samba-wsdd` for
Windows discovery.
- **DDNS** — a `ddns-update` timer (every 10 min) runs `dns_update.py` to sync the
`hentai.engineer` and `kelder-local.hentai.engineer` Cloudflare records with the address on
`et1g0`.
## Network assignments
<!-- assignments: kelder -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| kelder-ctrs | ctrs | `172.16.64.1/24` | — | hentai.engineer | |
| kelder | estuary | `94.142.242.254/32` | — | — | |
<!-- assignments-end -->
## Containers
| Container | Role | Docs |
| --- | --- | --- |
| `kelder-acquisition` | Media stack (Transmission over AirVPN, Jackett/Radarr/Sonarr, Jellyfin) | [kelder-acquisition.md](kelder-acquisition.md) |
| `kelder-spoder` | Nextcloud + nginx reverse proxy | [kelder-spoder.md](kelder-spoder.md) |
The containers are not deploy targets (`my.deploy.enable = false`); they're managed through
the host.
## Networking
- LAN on `et1g0` (renamed by MAC) with DHCP and MTU 1460 (`lib.my.c.kelder.ipv4MTU`); the
kelder v4 prefixes are masqueraded out of it.
- The `estuary` WireGuard peer (`AllowedIPs = 0.0.0.0/0`, keepalive 25s) is combined with
rules that keep LAN traffic on the main table and only route tunnel-marked/owned traffic
via table 51820.
## Services
- `netdata` (proxied as `monitor.hentai.engineer` by `kelder-spoder`), `smartd`, `fstrim`,
LVM thin provisioning.
- `minecraft-server` is present but **disabled** (`enable = false`); the firewall still opens
25565 tcp/udp.
- Primary user `kontent` (in the `storage`/`media` groups).
- Sets `system.nixos.distroName = "KelderOS"`, a custom Plymouth theme and an `amogus-beep`
boot jingle ([`boot.nix`](../../nixos/boxes/kelder/boot.nix)).
## Notable config files
- [`nixos/boxes/kelder/default.nix`](../../nixos/boxes/kelder/default.nix) — system, assignments, tunnel, NAT, containers.
- [`nixos/boxes/kelder/boot.nix`](../../nixos/boxes/kelder/boot.nix) — Plymouth theme + boot beep.
- [`nixos/boxes/kelder/containers/`](../../nixos/boxes/kelder/containers) — the two container definitions.
- [`nixos/boxes/kelder/dns_update.py`](../../nixos/boxes/kelder/dns_update.py) — Cloudflare DDNS script.