Compare commits
9 Commits
docs
..
300103f2ba
| Author | SHA1 | Date | |
|---|---|---|---|
 jackos1998
|
300103f2ba | ||
|
jackos1998
|
d33f32ce5b | ||
|
jackos1998
|
14d9bba4eb | ||
|
jackos1998
|
e63cee7b09 | ||
|
jackos1998
|
a8318d3de2 | ||
|
jackos1998
|
aab4a193ae | ||
|
jackos1998
|
541102f683 | ||
|
jackos1998
|
805590a705 | ||
|
jackos1998
|
f8dbd99a7b |
@@ -5,8 +5,8 @@ This file provides guidance to coding agents when working with code in this repo
|
||||
## Overview
|
||||
|
||||
Personal Nix flake managing NixOS systems and home-manager configurations for a set of
|
||||
machines ("boxes": servers, home machines, routers). It is built around a **custom module system** layered
|
||||
on top of NixOS/home-manager, not the stock flake `nixosConfigurations` pattern.
|
||||
machines — always called **"boxes"**, never "fleet". It is built around a **custom module
|
||||
system** layered on top of NixOS/home-manager, not the stock flake `nixosConfigurations` pattern.
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -31,7 +31,8 @@ Common ones:
|
||||
VMs and containers).
|
||||
- `ssh-machine <name> [cmd]` — SSH to a NixOS system or home-manager config by name. Resolves the
|
||||
target and ssh options (identity, port) from its deploy-rs node, so it needs `my.deploy.enable`
|
||||
(same gate as `deploy`).
|
||||
(same gate as `deploy`). Boxes default to the `fish` login shell, so pipe multi-statement remote
|
||||
scripts through `bash` (e.g. `ssh-machine <name> bash -s < script.sh`) rather than `&&`/`for`.
|
||||
- `ragenix` — edit age secrets using `.keys/dev.key` as identity (see Secrets).
|
||||
- `repl` — `nix repl .#`.
|
||||
- `update-nixpkgs` / `update-home-manager` — bump pinned inputs.
|
||||
@@ -91,10 +92,28 @@ Per-host configs live under `nixos/boxes/<host>` (some are single `.nix` files,
|
||||
with nested VMs/containers under e.g. `colony/vms`). Many "systems" are VMs or containers managed
|
||||
via the `vms` / `containers` modules and the `l2mesh` VXLAN module.
|
||||
|
||||
For a human-readable map of what is actually deployed (per-machine roles, services and networking),
|
||||
see `README.md` and the per-machine docs under `docs/boxes/` (grouped `docs/boxes/colony/`,
|
||||
`docs/boxes/home/`, `docs/boxes/misc/`). Keep these in sync when adding, removing or repurposing
|
||||
a machine or service.
|
||||
### Home routers (`nixos/boxes/home/routing-common`)
|
||||
The two home routers, `river` and `stream`, share `routing-common`, which is a **function of an
|
||||
`index`** (`import ../../routing-common 0` for river, `1` for stream). The index derives per-box
|
||||
addresses, keepalived VRRP priorities/state, DNS `ns` numbering, etc., so the two boxes are an
|
||||
active/backup HA pair from one definition. They differ where hardware/uplink differ: `stream` has a
|
||||
DHCP WAN, `river` runs PPPoE (`services.pppd`, Digiweb) — box-specific bits live in the respective
|
||||
box file, not `routing-common`.
|
||||
|
||||
- **HA is VRRP (`keepalived`).** Per-VLAN floating **VIPs** (`lib.my.c.home.vips`) are what clients
|
||||
use as both gateway *and* DNS server. `kea` (DHCP) and `radvd` (RAs; started only on the master)
|
||||
hand out the VIP, and `pdns-recursor` binds the VIPs (with `net.ipv*.ip_nonlocal_bind` so the
|
||||
backup can pre-bind). Point client-facing services at the VIP, not a box's real address, so
|
||||
failover follows the master instead of relying on client resolver timeouts.
|
||||
- **`wan-online.target`** is a shared abstract target meaning "the public WAN/IPv4 route is up".
|
||||
`routing-common` only declares it; each box wires *how it is reached* (`stream`: a oneshot that
|
||||
waits for the DHCP default route; `river`: the pppd `ip-up`/`ip-down` hooks). Services that need
|
||||
the WAN attach **to** it via `wantedBy` + `partOf` + `after` (not `requires`/`wants`), so an empty
|
||||
target is never pulled in and prematurely activated, and they re-load on WAN flap.
|
||||
- networkd helpers used heavily here: `lib.my.networkdAssignment` and `lib.my.mkVLAN` live under
|
||||
**`lib.my`**, while networkd snippet constants like `networkd.noL3` live under **`lib.my.c`** —
|
||||
easy to mix up. Set an interface MTU via the `.network`'s `linkConfig.MTUBytes` (`[Link]`), not
|
||||
`netdevConfig` (`[NetDev]` rejects `MTUBytes`).
|
||||
|
||||
## Secrets
|
||||
|
||||
@@ -116,3 +135,6 @@ private keys) is required for editing secrets, deploying, and running dev VMs.
|
||||
as `overlays.default`.
|
||||
- In prose and commit messages, quote code-like identifiers (commands, options, paths, package and
|
||||
attribute names) in backticks.
|
||||
- Call the machines **"boxes"**, never "fleet".
|
||||
- Commit subjects follow `area/scope: Capitalized summary` (e.g. `nixos/home: ...`); keep logically
|
||||
distinct changes in separate commits.
|
||||
|
||||
@@ -1,80 +0,0 @@
|
||||
# nixfiles
|
||||
|
||||
Personal Nix flake managing every machine I run: hosted servers, home
|
||||
infrastructure, routers, a remote site, VPSes and personal workstations. It is
|
||||
built around a **custom module system** layered on top of NixOS and
|
||||
home-manager rather than the stock per-host `nixosConfigurations` pattern.
|
||||
|
||||
For day-to-day commands and a deeper explanation of the module system,
|
||||
conventions and secrets, see [`AGENTS.md`](AGENTS.md). This README is the map of
|
||||
**what is actually deployed**; the per-machine details live under [`docs/boxes/`](docs/boxes).
|
||||
|
||||
> **Note:** This documentation (the README and everything under `docs/`) is a
|
||||
> work in progress and was **agent-generated** from the repository. It may be
|
||||
> incomplete or out of date — treat the Nix configuration as the source of truth.
|
||||
|
||||
## The boxes at a glance
|
||||
|
||||
Machines are grouped by deployment/location. Each group has its own directory
|
||||
under `docs/boxes/` with a `README.md` overview and one file per machine.
|
||||
|
||||
| Group | What it is | Docs |
|
||||
| --- | --- | --- |
|
||||
| **colony** | Hosted dedicated server in Amsterdam (`ams1`). A VM host running the public-facing infrastructure: routing, web, git, media, object storage, chat, game servers. | [`docs/boxes/colony/`](docs/boxes/colony) |
|
||||
| **home** | Home network: a VM host (`palace`), the home routers, storage, Home Assistant, and personal desktops. | [`docs/boxes/home/`](docs/boxes/home) |
|
||||
| **misc** | Everything else: edge VPSes (`britway`, `britnet`), the remote `kelder` site, the `tower` workstation, and the installer image. | [`docs/boxes/misc/`](docs/boxes/misc) |
|
||||
|
||||
## The "big machine" pattern
|
||||
|
||||
The larger sites (`colony`, `home`) all follow the same shape:
|
||||
|
||||
```
|
||||
physical host (VM host)
|
||||
├── VM ── thing impractical to containerise (router, storage, podman host, …)
|
||||
├── VM ── container host ──┬── NixOS container ── application
|
||||
│ ├── NixOS container ──┬── application
|
||||
│ │ ├── application
|
||||
│ │ └── application
|
||||
│ └── …
|
||||
└── VM ── …
|
||||
```
|
||||
|
||||
- A **physical host** (`colony`, `palace`) does little itself beyond running
|
||||
**VMs** via the custom `my.vms` module (QEMU + systemd units, LVM-backed
|
||||
disks).
|
||||
- **VMs** exist for things that are impractical to put in a container — kernel
|
||||
features, separate networking, podman/OCI workloads, foreign OSes.
|
||||
- One VM is usually a **container host** (`shill` on colony, `sfh` on home; and
|
||||
`kelder` directly). It runs **NixOS containers** via the custom `my.containers`
|
||||
module (systemd-nspawn based, each with its own address on a bridge).
|
||||
- **Most applications live in those NixOS containers.** A container isn't limited
|
||||
to a single application — it commonly hosts a **group of related applications**
|
||||
that belong together (e.g. `jackflix` runs Jellyfin, the *arr stack,
|
||||
Transmission, PhotoPrism and copyparty; `object` runs MinIO, Harmonia, HedgeDoc
|
||||
and wastebin). Each container is a first-class entry in `docs/boxes/`.
|
||||
|
||||
Networking between everything is largely defined by per-system `assignments`
|
||||
(IPs/prefixes) plus an L2 VXLAN mesh (`my.vpns.l2`, AS211024) that ties the edge
|
||||
routers together. See [`AGENTS.md`](AGENTS.md) for the mechanics.
|
||||
|
||||
## Repo layout
|
||||
|
||||
```
|
||||
README.md <- you are here
|
||||
nixos/
|
||||
boxes/ per-machine configuration ("boxes")
|
||||
colony/ colony host + its VMs (vms/) + shill's containers
|
||||
home/ palace host + its VMs, plus stream, castle
|
||||
britway/ britnet.nix, kelder/, tower/, installer.nix …
|
||||
modules/ shared NixOS modules (my.* options); registered in _list.nix
|
||||
home-manager/ home-manager modules + configs
|
||||
lib/ lib.my helpers, constants (lib.my.c), net/dns helpers
|
||||
pkgs/ custom packages (overlays.default)
|
||||
secrets/ age-encrypted secrets (ragenix)
|
||||
devshell/ devshell commands (build/deploy/check/ssh helpers)
|
||||
docs/boxes/ per-machine documentation (colony/, home/, misc/)
|
||||
```
|
||||
|
||||
A machine is wired into the flake by adding its box file to the `configs` list
|
||||
in `flake.nix`. See [`AGENTS.md`](AGENTS.md#architecture) for how `evalModules`
|
||||
turns these into `nixosConfigurations`, `homeConfigurations` and `deploy` nodes.
|
||||
@@ -1,68 +0,0 @@
|
||||
# colony
|
||||
|
||||
The hosted dedicated server in Amsterdam (`ams1`). This is the public-facing
|
||||
half of the boxes: almost everything reachable from the internet lives here.
|
||||
|
||||
- **Internal domain:** `ams1.int.nul.ie`
|
||||
- **Public domain:** `nul.ie` (public services are published as `*.nul.ie`)
|
||||
- **Source:** [`nixos/boxes/colony/`](../../../nixos/boxes/colony)
|
||||
|
||||
## Shape
|
||||
|
||||
`colony` is the physical VM host. It runs the VMs below; one of them (`shill`)
|
||||
is itself a NixOS container host where most applications run.
|
||||
|
||||
```
|
||||
colony (physical VM host)
|
||||
├── estuary ── edge router / firewall / DNS / BGP
|
||||
├── shill ──── NixOS container host ──┬── middleman (reverse proxy, ACME, SSO)
|
||||
│ ├── colony-psql (shared PostgreSQL)
|
||||
│ ├── vaultwarden (password manager)
|
||||
│ ├── chatterbox (Matrix + bridges)
|
||||
│ ├── toot (Mastodon)
|
||||
│ ├── jackflix (media stack)
|
||||
│ ├── object (MinIO, Nix cache, …)
|
||||
│ ├── waffletail (Tailscale subnet router)
|
||||
│ ├── qclk (clock service)
|
||||
│ └── gam (game servers)
|
||||
├── whale2 ─── podman/OCI host (game servers)
|
||||
├── git ────── Gitea + Actions runner
|
||||
├── mail ───── Debian VM running Mailcow (not NixOS — configured out of repo)
|
||||
└── darts ──── third-party/customer VM (opaque to this repo)
|
||||
```
|
||||
|
||||
## Machines
|
||||
|
||||
| Machine | Role | Docs |
|
||||
| --- | --- | --- |
|
||||
| `colony` | Physical VM host (AMD, LVM-thin, borgthin backups → rsync.net) | [colony.md](colony.md) |
|
||||
| `estuary` | Edge router: WAN, firewall/NAT, DNS, BGP, IXP peering, WireGuard | [estuary.md](estuary.md) |
|
||||
| `shill` | NixOS container host (see containers below) | [shill.md](shill.md) |
|
||||
| `whale2` | podman/OCI host for game servers | [whale2.md](whale2.md) |
|
||||
| `git` | Gitea + Gitea Actions runner | [git.md](git.md) |
|
||||
|
||||
### shill containers
|
||||
|
||||
| Container | Role | Docs |
|
||||
| --- | --- | --- |
|
||||
| `middleman` | Front-end nginx reverse proxy, ACME certs, nginx-sso, librespeed | [middleman.md](middleman.md) |
|
||||
| `colony-psql` | Shared PostgreSQL (14) for colony services | [colony-psql.md](colony-psql.md) |
|
||||
| `vaultwarden` | Vaultwarden (Bitwarden-compatible password manager) | [vaultwarden.md](vaultwarden.md) |
|
||||
| `chatterbox` | Matrix homeserver + bridges (heisenbridge, mautrix-*) | [chatterbox.md](chatterbox.md) |
|
||||
| `toot` | Bluesky PDS (Mastodon disabled) | [toot.md](toot.md) |
|
||||
| `jackflix` | Media: Jellyfin, *arr stack, Transmission, PhotoPrism, copyparty | [jackflix.md](jackflix.md) |
|
||||
| `object` | MinIO (S3), Harmonia (Nix cache), HedgeDoc, wastebin | [object.md](object.md) |
|
||||
| `waffletail` | Tailscale subnet router (advertises colony prefixes into the tailnet) | [waffletail.md](waffletail.md) |
|
||||
| `qclk` | `qclk` clock service (reachable over WireGuard) | [qclk.md](qclk.md) |
|
||||
| `gam` | Game servers (Terraria, …) | [gam.md](gam.md) |
|
||||
|
||||
## Non-NixOS VMs
|
||||
|
||||
These run on `colony` but are **not** managed by this repo (no NixOS config). The
|
||||
QEMU instances are still declared in `colony`'s `my.vms.instances`, and colony's
|
||||
networking routes/firewalls traffic to them:
|
||||
|
||||
- **`mail`** — a Debian VM running [Mailcow](https://mailcow.email/) (`mail.nul.ie`).
|
||||
ACME certs are pushed to it from `middleman` (see [middleman.md](middleman.md)).
|
||||
- **`darts`** — a third-party/customer VM; opaque to this repo, given a routed
|
||||
prefix and otherwise left alone.
|
||||
@@ -1,19 +0,0 @@
|
||||
# chatterbox
|
||||
|
||||
The Matrix homeserver (`nul.ie`) and its chat-network bridges.
|
||||
|
||||
- **Source:** [`shill/containers/chatterbox.nix`](../../../nixos/boxes/colony/vms/shill/containers/chatterbox.nix)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- **Matrix homeserver** for `server_name = "nul.ie"`.
|
||||
- **Bridges** to other chat networks:
|
||||
- `heisenbridge` (IRC),
|
||||
- `mautrix-whatsapp` (WhatsApp),
|
||||
- `mautrix-meta` / `mautrix-messenger` (Facebook Messenger / Instagram).
|
||||
- Fronted by `middleman` (federation on `:8448`).
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `chatterbox-ctr`).
|
||||
@@ -1,18 +0,0 @@
|
||||
# colony-psql
|
||||
|
||||
The shared PostgreSQL instance for colony. Several other containers
|
||||
(`middleman`, `chatterbox`, `toot`, `git`, …) connect here rather than each
|
||||
running their own database.
|
||||
|
||||
- **Source:** [`shill/containers/colony-psql.nix`](../../../nixos/boxes/colony/vms/shill/containers/colony-psql.nix)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- **PostgreSQL 14** serving the other colony services over the `ctrs` network.
|
||||
Consumers wait for it to be ready via the `systemdAwaitPostgres` helper.
|
||||
- netdata for monitoring.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `colony-psql-ctr`).
|
||||
@@ -1,39 +0,0 @@
|
||||
# colony (host)
|
||||
|
||||
The physical dedicated server in Amsterdam and the VM host for everything in
|
||||
this group.
|
||||
|
||||
- **Source:** [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
|
||||
(VM instances in [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix))
|
||||
- **nixpkgs:** `mine-stable`
|
||||
|
||||
## Role
|
||||
|
||||
Bare-metal AMD host. It does little application work itself — its job is to run
|
||||
the VMs and provide them with storage, networking and backups.
|
||||
|
||||
- **Virtualisation:** QEMU/KVM (`kvm-amd`, IOMMU on) via the `my.vms` module. VM
|
||||
disks are LVM logical volumes (`vm-<name>-<disk>`) in the `main` volume group;
|
||||
`estuary` additionally gets a WAN NIC by PCI passthrough.
|
||||
- **Storage:** LVM-thin (`services.lvm.boot.thin`), `/persist` for state,
|
||||
`/mnt/backup` for the local borg repo. `smartd` + `rasdaemon` for health.
|
||||
- **Backups:** `my.borgthin` snapshots the persist/data LVs of the host and its
|
||||
VMs into `/mnt/backup/main`, which is then `rsync`'d (along with LVM metadata)
|
||||
to rsync.net (`zh2855.rsync.net`).
|
||||
- **Monitoring:** netdata (with freeipmi), smartd.
|
||||
|
||||
## Networking
|
||||
|
||||
- Two bridges: `base` (the colony "base" network, shared with `estuary`) and
|
||||
`vms` (the VM network). Dummy interfaces keep the bridges up so dependent VMs
|
||||
can start.
|
||||
- Default gateway / edge is `estuary`; `colony` itself holds the `routing` and
|
||||
`internal` (a.k.a. `vm`) assignments and routes container/OCI/Tailscale
|
||||
prefixes to `shill` and `whale2`.
|
||||
- `my.firewall` trusts the `vms` interface and forwards customer prefixes
|
||||
(`vm-mail`, `vm-darts`) through.
|
||||
|
||||
## VMs hosted here
|
||||
|
||||
`estuary`, `shill`, `whale2`, `git` (all NixOS, documented in this directory),
|
||||
plus the non-NixOS `mail` and `darts` (see [README](README.md#non-nixos-vms)).
|
||||
@@ -1,41 +0,0 @@
|
||||
# estuary
|
||||
|
||||
The colony edge router and firewall — the machine that holds colony's public
|
||||
IPs and connects everything else to the internet.
|
||||
|
||||
- **Source:** [`nixos/boxes/colony/vms/estuary/`](../../../nixos/boxes/colony/vms/estuary)
|
||||
(`default.nix`, `bgp.nix`, `dns.nix`, `bandwidth.nix`)
|
||||
- **nixpkgs:** `mine`
|
||||
- **Host:** VM on `colony` (gets the WAN NIC by PCI passthrough)
|
||||
|
||||
## Role
|
||||
|
||||
- **Edge routing / firewall / NAT:** owns the colony public IPv4/IPv6
|
||||
(`94.142.241.x` / `2a02:898:0:20::`), does NAT and port-forwarding for the
|
||||
internal services (`my.firewall.nat.forwardPorts` driven by
|
||||
`firewallForwards`). Forwards HTTP/S to `middleman`, git to `git`, game ports
|
||||
to the OCI game servers on `whale2`, etc.
|
||||
- **BGP:** runs BIRD2 ([`bgp.nix`](../../../nixos/boxes/colony/vms/estuary/bgp.nix))
|
||||
announcing AS211024, over VLANs on the WAN link:
|
||||
- peers at the IXPs **Frys-IX**, **NL-ix** and **FogIXP**;
|
||||
- plus **iFog transit** (`ifog-transit`) — an upstream transit provider from
|
||||
iFog, **not** an IXP.
|
||||
- **DNS:** authoritative/recursive DNS ([`dns.nix`](../../../nixos/boxes/colony/vms/estuary/dns.nix)),
|
||||
redirected to port 5353 locally.
|
||||
- **VPNs:**
|
||||
- Part of the AS211024 **L2 VXLAN mesh** (`my.vpns.l2`) with `river`, `stream`
|
||||
and `britway`.
|
||||
- WireGuard endpoints for the remote `kelder` site, `hillcrest`, and
|
||||
`john-valorant`.
|
||||
- **Misc:** iperf3 server. (A bandwidth-accounting script,
|
||||
[`bandwidth.py`](../../../nixos/boxes/colony/vms/estuary/bandwidth.py), exists but
|
||||
is **legacy and not currently used**.)
|
||||
|
||||
## Networking
|
||||
|
||||
- `wan` — the passed-through igb NIC (9000 MTU), carrying the upstream uplink and
|
||||
tagged IXP VLANs (`ifog` 409 → `frys-ix`/`nl-ix`/`fogixp`/`ifog-transit`).
|
||||
- `base` — colony base network; sends RAs and provides DNS to the base prefix,
|
||||
routes the VM/container/OCI/Tailscale prefixes back to `colony`.
|
||||
- `as211024` — the L2 mesh interface.
|
||||
- Assignments: `internal` (public, alt name `fw`), `base`, `as211024`.
|
||||
@@ -1,17 +0,0 @@
|
||||
# gam
|
||||
|
||||
A game-server container (the lightweight counterpart to the OCI game servers on
|
||||
`whale2`).
|
||||
|
||||
- **Source:** [`shill/containers/gam.nix`](../../../nixos/boxes/colony/vms/shill/containers/gam.nix)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- Hosts game servers run directly as NixOS services — currently **Terraria**
|
||||
(config/world from secrets). Exposed to the internet via `estuary`'s port
|
||||
forwards (`:7777`).
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `gam-ctr`).
|
||||
@@ -1,38 +0,0 @@
|
||||
# git
|
||||
|
||||
The Gitea VM — source hosting and CI for the boxes (`git.nul.ie`).
|
||||
|
||||
- **Source:** [`nixos/boxes/colony/vms/git/`](../../../nixos/boxes/colony/vms/git)
|
||||
(`default.nix`, `gitea.nix`, `gitea-actions.nix`)
|
||||
- **nixpkgs:** `mine`
|
||||
- **Host:** VM on `colony`
|
||||
|
||||
## Role
|
||||
|
||||
- **Gitea** ([`gitea.nix`](../../../nixos/boxes/colony/vms/git/gitea.nix)) — the Git
|
||||
forge (`git.nul.ie`). PostgreSQL-backed (the shared `colony-psql`), LFS
|
||||
enabled, with object storage backed by MinIO on `object` (a MinIO secret is
|
||||
spliced into `app.ini` at startup).
|
||||
- **Gitea Actions runner**
|
||||
([`gitea-actions.nix`](../../../nixos/boxes/colony/vms/git/gitea-actions.nix)) — a
|
||||
Docker-mode runner (`main-docker`) using podman. Labels provide Debian/node-24
|
||||
(Trixie) and Ubuntu 26.04 images; runner config comes from the upstream
|
||||
module's `settings` option. The Actions cache lives on a dedicated disk
|
||||
(`/var/cache/gitea-runner`). Runs as a fixed `gitea-runner` user (not
|
||||
`DynamicUser`) so it can read its token.
|
||||
- **nginx** — terminates TLS for `git.nul.ie` and proxies to Gitea on `:3000`.
|
||||
ACME certs for `nul.ie` / `*.nul.ie` via the Cloudflare DNS challenge.
|
||||
- **podman** — also hosts the OCI registry/build images; `/var/lib/containers`
|
||||
is an XFS data disk.
|
||||
|
||||
## Networking
|
||||
|
||||
- `vms` interface with `routing` / `internal` assignments.
|
||||
- HTTP/HTTPS forwarded in from `estuary`; podman default subnet `10.88.0.0/16` is
|
||||
allowed to forward.
|
||||
|
||||
## CI
|
||||
|
||||
This runner is what executes the repo's own `.gitea/workflows/ci.yaml`, building
|
||||
each `.#ci.x86_64-linux` attribute and pushing to the Harmonia binary cache. See
|
||||
[`AGENTS.md`](../../../AGENTS.md#commands).
|
||||
@@ -1,25 +0,0 @@
|
||||
# jackflix
|
||||
|
||||
The media stack — acquisition, library and streaming.
|
||||
|
||||
- **Source:** [`shill/containers/jackflix/`](../../../nixos/boxes/colony/vms/shill/containers/jackflix)
|
||||
(`default.nix`, `networking.nix`)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- **Streaming:** Jellyfin.
|
||||
- **Acquisition (*arr stack):** Transmission, Jackett, FlareSolverr, Radarr,
|
||||
Sonarr, and Jellyseerr (`seerr`) for requests.
|
||||
- **Photos:** PhotoPrism (`photos.nul.ie`).
|
||||
- **File sharing:** copyparty (`:3923`) serving public + private media volumes.
|
||||
- Media lives on the shared `/mnt/media` volume (bind-mounted read-write from
|
||||
`shill`). Downloaders bind to a VPN interface
|
||||
([`networking.nix`](../../../nixos/boxes/colony/vms/shill/containers/jackflix/networking.nix)),
|
||||
so torrent traffic only flows while `systemd-networkd-wait-online@vpn` is up.
|
||||
- A shared `media` group (gid 2000) gives the apps coordinated access.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `jackflix-ctr`), plus its
|
||||
own VPN interface for the download clients.
|
||||
@@ -1,63 +0,0 @@
|
||||
# middleman
|
||||
|
||||
The front-end reverse proxy for all public colony web services — the single
|
||||
ingress that `estuary` forwards HTTP/HTTPS to.
|
||||
|
||||
- **Source:** [`shill/containers/middleman/`](../../../nixos/boxes/colony/vms/shill/containers/middleman)
|
||||
(`default.nix`, `vhosts.nix`)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- **nginx** reverse proxy ([`vhosts.nix`](../../../nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix)
|
||||
holds the per-service vhosts) with VTS stats, fancyindex, brotli, caching, and
|
||||
a dynamic resolver pointed at `estuary` so upstreams can be re-resolved at
|
||||
runtime. It is the single public ingress for almost every web service — colony,
|
||||
home, and beyond.
|
||||
- **ACME** — issues the wildcard certificates that **its own** vhosts are served
|
||||
with (it is not a shared CA for the other boxes; `git`, `britway`, `kelder-spoder`, etc. each
|
||||
run their own ACME):
|
||||
- `nul.ie` / `*.nul.ie` (+ `*.s3.nul.ie`) via the Cloudflare DNS challenge,
|
||||
- the internal `ams1.int.nul.ie` / `*` via an `exec` challenge that calls
|
||||
`estuary`'s pdns over SSH.
|
||||
- As a one-off consumer, it then pushes the public cert to the `mail` (Mailcow)
|
||||
VM via `scp` + a remote `mailcow-ssl-reload`.
|
||||
- **nginx-sso** — single-sign-on (`sso.nul.ie`) with Google OAuth and a simple
|
||||
username/password provider; protects the SSO-gated vhosts below.
|
||||
- **librespeed** — speed-test frontend + backend (`librespeed.${domain}` /
|
||||
`speed.nul.ie`).
|
||||
|
||||
## Published vhosts
|
||||
|
||||
All under `*.nul.ie` with the wildcard cert unless noted. Upstreams are addressed
|
||||
by their internal container/VM hostnames. "SSO" = gated behind nginx-sso.
|
||||
|
||||
| Host(s) | Upstream | Notes |
|
||||
| --- | --- | --- |
|
||||
| `nul.ie` (default `_`) | static | landing page (CV, SSH pubkey) + Matrix/atproto `.well-known` |
|
||||
| `sso.nul.ie` | nginx-sso | SSO endpoint |
|
||||
| `pass.nul.ie` | `vaultwarden` | password manager |
|
||||
| `matrix.nul.ie` (+`:8448`) | `chatterbox` | Matrix client + federation |
|
||||
| `element.nul.ie` | element-web | Matrix web client |
|
||||
| `toot.nul.ie` | `toot` :80 | Mastodon (currently disabled — see [toot.md](toot.md)) |
|
||||
| `pds.nul.ie` | `toot` :3000 | Bluesky PDS |
|
||||
| `jackflix.nul.ie` | `jackflix` Jellyfin | streaming |
|
||||
| `torrents` / `jackett` / `radarr` / `sonarr` `.nul.ie` | `jackflix` | *arr stack (**SSO**) |
|
||||
| `gib.nul.ie` | `jackflix` Jellyseerr | requests |
|
||||
| `photos.nul.ie` | `jackflix` PhotoPrism | |
|
||||
| `stuff` / `public` / `p.nul.ie` | `jackflix` copyparty + `/mnt/media` | file sharing / index |
|
||||
| `share.nul.ie` | `object` :9090 | |
|
||||
| `minio` / `s3` / `*.s3.nul.ie` | `object` MinIO | S3 + console (Docker manifest MIME hack) |
|
||||
| `nix-cache.nul.ie` | `object` Harmonia | Nix binary cache (immutable cache headers) |
|
||||
| `md.nul.ie` / `pb.nul.ie` | `object` | HedgeDoc / wastebin |
|
||||
| `mc-map` / `mc-rail` / `mc-map-kink` `.nul.ie` | `whale2` OCI | Minecraft maps |
|
||||
| `netdata-colony.nul.ie` | many hosts :19999 | netdata fan-out (**SSO**) |
|
||||
| `pront.nul.ie` | `stream-hi` (home) | print/webcam (**SSO**) |
|
||||
| `hass.nul.ie` | `hass` (home) | Home Assistant |
|
||||
| `hass-john.nul.ie` | `john-valorant-tun` | remote HASS over WireGuard tunnel |
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network; bind-mounts `/mnt/media` for
|
||||
serving static/media content.
|
||||
- nginx waits for `colony-psql` before starting (DNS bootstrap hack).
|
||||
@@ -1,24 +0,0 @@
|
||||
# object
|
||||
|
||||
Object storage and Nix binary cache, plus a couple of small self-hosted web
|
||||
apps.
|
||||
|
||||
- **Source:** [`shill/containers/object.nix`](../../../nixos/boxes/colony/vms/shill/containers/object.nix)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- **MinIO** — S3-compatible object storage (`s3.nul.ie` / `*.s3.nul.ie`). Backs
|
||||
several other services (Gitea LFS/artifacts, social-media uploads, …). Stored on the
|
||||
`/mnt/minio` volume (XFS, bind-mounted from `shill`).
|
||||
- **Harmonia** — serves the Nix binary cache for all the boxes (`nix-cache.nul.ie`), backed
|
||||
by the `/mnt/nix-cache` volume.
|
||||
- **atticd** — an alternative Nix cache (stores into MinIO/S3). **Currently
|
||||
disabled** — present in the config but not running.
|
||||
- **HedgeDoc** — collaborative markdown notes.
|
||||
- **wastebin** — pastebin.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `object-ctr`).
|
||||
- `/mnt/minio` and `/mnt/nix-cache` bind-mounted read-write from `shill`.
|
||||
@@ -1,22 +0,0 @@
|
||||
# qclk
|
||||
|
||||
The `qclk` service container.
|
||||
|
||||
- **Source:** [`shill/containers/qclk/`](../../../nixos/boxes/colony/vms/shill/containers/qclk)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- Runs the custom `qclk` service, exposing an API that is reached over a
|
||||
dedicated WireGuard **`management`** network. Managed devices are configured as
|
||||
WireGuard peers (each gets an address in the `qclk` prefix), and AS211024
|
||||
trusted hosts are allowed to reach the API.
|
||||
- `shill` routes the `qclk` prefix to this container.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `qclk-ctr`), plus the
|
||||
`management` WireGuard interface carrying the `qclk` prefix.
|
||||
|
||||
> Check `qclk/default.nix` for the current peer list and exactly what the service
|
||||
> does — this entry intentionally stays high-level.
|
||||
@@ -1,48 +0,0 @@
|
||||
# shill
|
||||
|
||||
The colony **NixOS container host**. Most colony applications run as
|
||||
systemd-nspawn containers on `shill`.
|
||||
|
||||
- **Source:** [`nixos/boxes/colony/vms/shill/`](../../../nixos/boxes/colony/vms/shill)
|
||||
(`default.nix`, `containers-ext.nix`, `hercules.nix`, `containers/`)
|
||||
- **nixpkgs:** `mine`
|
||||
- **Host:** VM on `colony` (large: 12 cores, 40 GiB RAM)
|
||||
|
||||
## Role
|
||||
|
||||
- Runs the colony NixOS containers via `my.containers.instances`, each attached
|
||||
to the `ctrs` bridge with its own address.
|
||||
- Provides shared data volumes to those containers via bind mounts from
|
||||
LVM-backed disks: `/mnt/media` (→ `middleman`, `jackflix`), `/mnt/minio` and
|
||||
`/mnt/nix-cache` (→ `object`).
|
||||
- Acts as the router between the `vms` network and the `ctrs` container network
|
||||
(sends RAs on `ctrs`, routes Tailscale prefixes via `waffletail` and the
|
||||
`qclk` prefix via `qclk`). Includes an nftables `ct mark` hack to make
|
||||
internal DNAT return paths work.
|
||||
- Tuned sysctls for high connection counts / torrent traffic; netdata.
|
||||
|
||||
## Containers
|
||||
|
||||
Defined in [`shill/containers/`](../../../nixos/boxes/colony/vms/shill/containers)
|
||||
and wired up in `shill`'s `my.containers.instances`:
|
||||
|
||||
| Container | Role |
|
||||
| --- | --- |
|
||||
| [`middleman`](middleman.md) | Front-end nginx reverse proxy, ACME, nginx-sso, librespeed |
|
||||
| [`colony-psql`](colony-psql.md) | Shared PostgreSQL |
|
||||
| [`vaultwarden`](vaultwarden.md) | Password manager |
|
||||
| [`chatterbox`](chatterbox.md) | Matrix homeserver + bridges |
|
||||
| [`toot`](toot.md) | Bluesky PDS (Mastodon disabled) |
|
||||
| [`jackflix`](jackflix.md) | Media stack |
|
||||
| [`object`](object.md) | MinIO / Harmonia / HedgeDoc / wastebin |
|
||||
| [`waffletail`](waffletail.md) | Tailscale subnet router |
|
||||
| [`qclk`](qclk.md) | Clock service |
|
||||
| [`gam`](gam.md) | Game servers |
|
||||
|
||||
## Notes
|
||||
|
||||
- Container systems set `my.deploy.enable = false` (they are deployed as part of
|
||||
`shill`'s container profiles, not as standalone deploy nodes) and render via
|
||||
`my.asContainer`.
|
||||
- `hercules.nix` configures Hercules CI agent bits;
|
||||
`containers-ext.nix` holds extra per-container host wiring.
|
||||
@@ -1,19 +0,0 @@
|
||||
# toot
|
||||
|
||||
A federated-social container. Despite the name (Mastodon = "toots"), it currently
|
||||
hosts a **Bluesky PDS**; the Mastodon instance is disabled.
|
||||
|
||||
- **Source:** [`shill/containers/toot.nix`](../../../nixos/boxes/colony/vms/shill/containers/toot.nix)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- **Bluesky PDS** (Personal Data Server) — the active service, published as
|
||||
`pds.nul.ie` (proxied by `middleman` to `:3000`).
|
||||
- **Mastodon** — **currently disabled**. The config is still present and
|
||||
`toot.nul.ie` still maps to `:80`, but the instance is not running. (It was
|
||||
backed by the shared `colony-psql` and MinIO/S3 on `object`.)
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `toot-ctr`).
|
||||
@@ -1,15 +0,0 @@
|
||||
# vaultwarden
|
||||
|
||||
[Vaultwarden](https://github.com/dani-garcia/vaultwarden), a Bitwarden-compatible
|
||||
password manager.
|
||||
|
||||
- **Source:** [`shill/containers/vaultwarden.nix`](../../../nixos/boxes/colony/vms/shill/containers/vaultwarden.nix)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- Runs Vaultwarden, fronted by `middleman` and published under `nul.ie`.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `vaultwarden-ctr`).
|
||||
@@ -1,19 +0,0 @@
|
||||
# waffletail
|
||||
|
||||
The colony Tailscale node / subnet router.
|
||||
|
||||
- **Source:** [`shill/containers/waffletail.nix`](../../../nixos/boxes/colony/vms/shill/containers/waffletail.nix)
|
||||
- **Host:** NixOS container on `shill`
|
||||
|
||||
## Role
|
||||
|
||||
- Joins the Tailscale tailnet (auth key from secrets) and **advertises the colony
|
||||
prefixes** into it, acting as the subnet router so tailnet clients can reach
|
||||
colony services and vice-versa.
|
||||
- nftables rules SNAT/forward between `host0` and `tailscale0` for the colony
|
||||
v4/v6 ranges. `shill` routes the Tailscale prefixes here.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment on the `ctrs` network (alt name `waffletail-ctr`); owns
|
||||
the `tailscale0` interface.
|
||||
@@ -1,27 +0,0 @@
|
||||
# whale2
|
||||
|
||||
A podman/OCI host on colony dedicated to game servers (kept off `shill` so the
|
||||
container churn and resource use stay isolated).
|
||||
|
||||
- **Source:** [`nixos/boxes/colony/vms/whale2/`](../../../nixos/boxes/colony/vms/whale2)
|
||||
(`default.nix`, `valheim.nix`, `minecraft/`, `enshrouded.nix`)
|
||||
- **nixpkgs:** `mine`
|
||||
- **Host:** VM on `colony`
|
||||
|
||||
## Role
|
||||
|
||||
- Runs OCI containers via **podman** (`virtualisation.oci-containers`, netavark
|
||||
backend) on a dedicated `colony` bridge network (`oci`) with both IPv4 and
|
||||
IPv6, so each game server gets its own routable address.
|
||||
- Game servers configured in-repo: **Valheim**, **Minecraft** (several worlds —
|
||||
see `extraAssignments`: `simpcraft`, `simpcraft-staging`, `kevcraft`,
|
||||
`kinkcraft`, `graeme`), and **Enshrouded** (currently commented out).
|
||||
- `/var/lib/containers` is an XFS data disk (project quotas).
|
||||
|
||||
## Networking
|
||||
|
||||
- `vms` interface with `routing` / `internal` (alt name `oci`) assignments.
|
||||
- An `oci` bridge carrying the `prefixes.oci` v4/v6 ranges; per-game addresses
|
||||
are handed out via `extraAssignments` (`valheim-oci`, `simpcraft-oci`, …) and
|
||||
exposed to the internet through `estuary`'s port forwards.
|
||||
- Firewall trusts the `oci` interface and forwards `vms → oci`.
|
||||
@@ -1,44 +0,0 @@
|
||||
# home
|
||||
|
||||
The home network. A VM host (`palace`), a redundant pair of routers, a storage
|
||||
server, Home Assistant, and personal desktops.
|
||||
|
||||
- **Domain:** `h.nul.ie`
|
||||
- **Source:** [`nixos/boxes/home/`](../../../nixos/boxes/home)
|
||||
|
||||
## Shape
|
||||
|
||||
```
|
||||
palace (physical VM host)
|
||||
├── river ──── home router (HA pair with stream)
|
||||
├── cellar ── NVMe-oF storage server (SPDK)
|
||||
└── sfh ───── NixOS container host ──┬── hass (Home Assistant)
|
||||
└── unifi (UniFi controller — defined, currently disabled)
|
||||
|
||||
stream ── standalone home router (HA pair with river)
|
||||
castle ── desktop workstation (boots its disks over NVMe-oF from cellar)
|
||||
```
|
||||
|
||||
The two routers `river` (a VM on `palace`) and `stream` (standalone hardware)
|
||||
share the [`routing-common`](../../../nixos/boxes/home/routing-common) config and
|
||||
form a **keepalived/VRRP high-availability pair**: DHCP (kea), router
|
||||
advertisements (radvd), DNS with blocklists, NAT, and the AS211024 L2 mesh link
|
||||
back to colony.
|
||||
|
||||
## Machines
|
||||
|
||||
| Machine | Role | Docs |
|
||||
| --- | --- | --- |
|
||||
| `palace` | Physical VM host | [palace.md](palace.md) |
|
||||
| `river` | Home router (VM; VRRP pair with `stream`) | [river.md](river.md) |
|
||||
| `cellar` | NVMe-oF storage server (SPDK) | [cellar.md](cellar.md) |
|
||||
| `sfh` | NixOS container host | [sfh.md](sfh.md) |
|
||||
| `stream` | Home router (standalone hardware; VRRP pair with `river`) | [stream.md](stream.md) |
|
||||
| `castle` | Desktop workstation | [castle.md](castle.md) |
|
||||
|
||||
### sfh containers
|
||||
|
||||
| Container | Role | Docs |
|
||||
| --- | --- | --- |
|
||||
| `hass` | Home Assistant | [hass.md](hass.md) |
|
||||
| `unifi` | UniFi network controller (defined, **currently not imported**) | [unifi.md](unifi.md) |
|
||||
@@ -1,19 +0,0 @@
|
||||
# castle
|
||||
|
||||
A desktop workstation.
|
||||
|
||||
- **Source:** [`nixos/boxes/home/castle/default.nix`](../../../nixos/boxes/home/castle/default.nix)
|
||||
|
||||
## Role
|
||||
|
||||
- AMD desktop running the GUI environment (`my.gui.enable`, Sway / Wayland via
|
||||
home-manager).
|
||||
- **Diskless-style boot:** its `/nix`, `/persist` and `/home` are NVMe-oF volumes
|
||||
served by [`cellar`](cellar.md) (`/dev/nvmeof/{nix,persist,home}`). The
|
||||
networking is careful not to drop the IP used for the NVMe-oF connection.
|
||||
- Uses the `my.nvme` module for the NVMe-oF client setup.
|
||||
|
||||
## Networking
|
||||
|
||||
- Sits on the home network; depends on the high-speed link to `cellar` for its
|
||||
root storage.
|
||||
@@ -1,39 +0,0 @@
|
||||
# cellar
|
||||
|
||||
The home storage server. Exports fast NVMe storage over the network so other
|
||||
machines (notably `castle`) can boot and run from it.
|
||||
|
||||
- **Source:** [`nixos/boxes/home/palace/vms/cellar/`](../../../nixos/boxes/home/palace/vms/cellar)
|
||||
(`default.nix`, `spdk.nix`)
|
||||
- **Host:** VM on `palace` (NVMe drives passed through)
|
||||
- **Deploy address:** `192.168.68.80`
|
||||
|
||||
## Role
|
||||
|
||||
- Runs **SPDK** ([`spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix)) as
|
||||
a userspace storage target. The kernel `nvme` driver is blacklisted so SPDK can
|
||||
drive the NVMe devices directly (attached by PCI BDF).
|
||||
- Builds a **RAID0** (`NVMeRaid`) across three NVMe drives, partitioned into three
|
||||
namespaces, and exports each as an **NVMe-oF target over RDMA** (port 4420) on
|
||||
the high-speed home network — one namespace per consumer:
|
||||
|
||||
| Namespace | NQN | Consumer |
|
||||
| --- | --- | --- |
|
||||
| `NVMeRaidp1` | `nqn.2016-06.io.spdk:river` | [`river`](river.md) |
|
||||
| `NVMeRaidp2` | `nqn.2016-06.io.spdk:castle` | [`castle`](castle.md) |
|
||||
| `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh.md) |
|
||||
|
||||
Each client is pinned by `hostnqn`, so `river`, `castle` and `sfh` all run their
|
||||
storage off `cellar` over the network.
|
||||
|
||||
## Networking
|
||||
|
||||
- Exports on the high-speed home network (`lan-hi` / the `hi` assignment) over
|
||||
RDMA; the SPDK target waits for that link to be online before starting.
|
||||
|
||||
## Notes
|
||||
|
||||
- The `ublk_*` calls in `my.spdk.debugCommands` are **only a debugging script** —
|
||||
they let you create a local ublk block device to mount and inspect the RAID on
|
||||
`cellar` itself. They are **not** how storage is exported to clients; that is the
|
||||
`nvmf` config above.
|
||||
@@ -1,18 +0,0 @@
|
||||
# hass
|
||||
|
||||
[Home Assistant](https://www.home-assistant.io/) — home automation.
|
||||
|
||||
- **Source:** [`sfh/containers/hass.nix`](../../../nixos/boxes/home/palace/vms/sfh/containers/hass.nix)
|
||||
- **Host:** NixOS container on `sfh`
|
||||
|
||||
## Role
|
||||
|
||||
- Runs Home Assistant plus supporting services in the container. The `hass-cli`
|
||||
is wired up against the local server for convenience.
|
||||
- Integrations/automations are configured here (see commit history for things
|
||||
like the "West Wood" integration).
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment (alt name `hass-ctr`), plus a loopback assignment
|
||||
(`hass-ctr-lo`) used internally.
|
||||
@@ -1,21 +0,0 @@
|
||||
# palace (host)
|
||||
|
||||
The physical VM host for the home network — the home equivalent of `colony`.
|
||||
|
||||
- **Source:** [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix)
|
||||
(VM instances in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
|
||||
|
||||
## Role
|
||||
|
||||
- Bare-metal host whose job is to run the home VMs via the `my.vms` module:
|
||||
`river` (router), `cellar` (storage), `sfh` (container host).
|
||||
- Provides the bridged networking those VMs sit on and passes through hardware
|
||||
where needed (e.g. NVMe drives to `cellar`, NICs to `river`).
|
||||
|
||||
## VMs hosted here
|
||||
|
||||
| VM | Role | Docs |
|
||||
| --- | --- | --- |
|
||||
| `river` | Home router (VRRP pair with `stream`) | [river.md](river.md) |
|
||||
| `cellar` | NVMe-oF storage server | [cellar.md](cellar.md) |
|
||||
| `sfh` | NixOS container host (Home Assistant, …) | [sfh.md](sfh.md) |
|
||||
@@ -1,25 +0,0 @@
|
||||
# river
|
||||
|
||||
One of the two home routers. `river` is a VM on `palace`; it forms a
|
||||
high-availability pair with the standalone `stream`.
|
||||
|
||||
- **Source:** [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix),
|
||||
built from [`routing-common`](../../../nixos/boxes/home/routing-common) (instance `0`)
|
||||
- **Host:** VM on `palace`
|
||||
- **Deploy address:** `192.168.68.1`
|
||||
|
||||
## Role
|
||||
|
||||
Everything in [`routing-common`](../../../nixos/boxes/home/routing-common):
|
||||
|
||||
- **VRRP/keepalived** failover with `stream` (`keepalived.nix`) — one router is
|
||||
master at a time, sharing virtual IPs.
|
||||
- **DHCP** via kea (`kea.nix`), **router advertisements** via radvd
|
||||
(`radvd.nix`).
|
||||
- **DNS** (`dns.nix`) — local resolver with a blocklist
|
||||
(`dns-blocklist.txt`) and a periodic update script.
|
||||
- **NAT / firewall** for the home LAN, with policy routing.
|
||||
- **AS211024 L2 mesh** link back to colony/`estuary` (and the other edge
|
||||
routers), so home and colony networks interconnect.
|
||||
|
||||
See [stream.md](stream.md) for the other half of the pair.
|
||||
@@ -1,24 +0,0 @@
|
||||
# sfh
|
||||
|
||||
The home **NixOS container host** ("smart from home" / home services).
|
||||
|
||||
- **Source:** [`nixos/boxes/home/palace/vms/sfh/`](../../../nixos/boxes/home/palace/vms/sfh)
|
||||
(`default.nix`, `containers/`)
|
||||
- **Host:** VM on `palace`
|
||||
|
||||
## Role
|
||||
|
||||
- Runs the home NixOS containers via `my.containers.instances`, in the same way
|
||||
`shill` does on colony.
|
||||
- Sits on the home network and connects to NVMe-oF storage (`cellar`) where
|
||||
needed.
|
||||
|
||||
## Containers
|
||||
|
||||
Defined in [`sfh/containers/`](../../../nixos/boxes/home/palace/vms/sfh/containers)
|
||||
and imported from its `containers/default.nix`:
|
||||
|
||||
| Container | Role | Docs |
|
||||
| --- | --- | --- |
|
||||
| `hass` | Home Assistant | [hass.md](hass.md) |
|
||||
| `unifi` | UniFi controller — **defined but currently commented out** of `containers/default.nix` | [unifi.md](unifi.md) |
|
||||
@@ -1,19 +0,0 @@
|
||||
# stream
|
||||
|
||||
One of the two home routers. `stream` is standalone hardware; it forms a
|
||||
high-availability pair with `river` (a VM on `palace`).
|
||||
|
||||
- **Source:** [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix),
|
||||
built from [`routing-common`](../../../nixos/boxes/home/routing-common) (instance `1`)
|
||||
- **Deploy address:** `192.168.68.2`
|
||||
|
||||
## Role
|
||||
|
||||
- Same [`routing-common`](../../../nixos/boxes/home/routing-common) role as
|
||||
[`river`](river.md): keepalived/VRRP, kea DHCP, radvd, DNS + blocklist, NAT and
|
||||
the AS211024 L2 mesh link to colony.
|
||||
- Additionally pulls in `mstpd` (`routing-common/mstpd.nix`) for spanning-tree on
|
||||
its bridged ports — `stream` is the one wired into the physical switching, so
|
||||
it manages the L2 topology.
|
||||
|
||||
See [river.md](river.md) for the other half of the pair.
|
||||
@@ -1,23 +0,0 @@
|
||||
# unifi
|
||||
|
||||
The UniFi network controller.
|
||||
|
||||
- **Source:** [`sfh/containers/unifi.nix`](../../../nixos/boxes/home/palace/vms/sfh/containers/unifi.nix)
|
||||
- **Host:** NixOS container on `sfh`
|
||||
|
||||
## Status
|
||||
|
||||
> **Currently disabled.** The system is still defined (`nixos.systems.unifi`),
|
||||
> but its import is commented out in
|
||||
> [`sfh/containers/default.nix`](../../../nixos/boxes/home/palace/vms/sfh/containers/default.nix),
|
||||
> so it is not deployed as a container right now. Re-enable by uncommenting
|
||||
> `./unifi.nix` there.
|
||||
|
||||
## Role
|
||||
|
||||
- Runs the UniFi controller (`services.unifi`) to manage the home UniFi network
|
||||
gear.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment (alt name `unifi-ctr`).
|
||||
@@ -1,21 +0,0 @@
|
||||
# misc
|
||||
|
||||
Everything that isn't part of the `colony` or `home` sites: the edge VPSes, the
|
||||
remote `kelder` site, a workstation, and the installer image.
|
||||
|
||||
| Machine | What it is | Docs |
|
||||
| --- | --- | --- |
|
||||
| `britway` | Vultr VPS (London, `lon1`): Headscale, Tailscale exit node, BGP edge, nginx | [britway.md](britway.md) |
|
||||
| `britnet` | VPS (Birmingham, `bhx1`): Tailscale/WireGuard gateway | [britnet.md](britnet.md) |
|
||||
| `kelder` | Remote site host (`hentai.engineer`): NixOS container host | [kelder.md](kelder.md) |
|
||||
| `tower` | Framework Laptop 13 (12th-gen Intel) workstation | [tower.md](tower.md) |
|
||||
| `installer` | Custom NixOS installer image | [installer.md](installer.md) |
|
||||
|
||||
## kelder containers
|
||||
|
||||
`kelder` is itself a container host (like `shill`/`sfh`):
|
||||
|
||||
| Container | Role | Docs |
|
||||
| --- | --- | --- |
|
||||
| `kelder-acquisition` | Media stack (Jellyfin + *arr + Transmission) | [kelder-acquisition.md](kelder-acquisition.md) |
|
||||
| `kelder-spoder` | nginx web host | [kelder-spoder.md](kelder-spoder.md) |
|
||||
@@ -1,21 +0,0 @@
|
||||
# britnet
|
||||
|
||||
A VPS in Birmingham (`bhx1`) acting as a Tailscale/WireGuard gateway node.
|
||||
|
||||
- **Source:** [`nixos/boxes/britnet.nix`](../../../nixos/boxes/britnet.nix)
|
||||
- **Internal domain:** `bhx1.int.nul.ie`
|
||||
|
||||
## Role
|
||||
|
||||
- **Tailscale node** + **WireGuard** (`wg0`) gateway: provides a second egress /
|
||||
entry point into the boxes' overlay networks.
|
||||
- nftables SNATs traffic arriving on `tailscale0` / `wg0` out of the provider
|
||||
interface (`veth0`), using the `allhost` assignment addresses.
|
||||
|
||||
## Networking
|
||||
|
||||
- Provider uplink with gateways `77.74.199.1` (v4) / `2a12:ab46:5344::1` (v6).
|
||||
- `tailscale0` and `wg0` overlay interfaces; `allhost` assignment for SNAT.
|
||||
|
||||
> `britnet` is a separate machine from [`britway`](britway.md) — different
|
||||
> provider/site, narrower role (gateway rather than control plane + BGP edge).
|
||||
@@ -1,27 +0,0 @@
|
||||
# britway
|
||||
|
||||
A Vultr VPS in London (`lon1`) acting as a network edge node: the Tailscale
|
||||
control plane, an exit node, and a BGP speaker in the AS211024 mesh.
|
||||
|
||||
- **Source:** [`nixos/boxes/britway/`](../../../nixos/boxes/britway)
|
||||
(`default.nix`, `bgp.nix`, `nginx.nix`, `tailscale.nix`)
|
||||
- **Internal domain:** `lon1.int.nul.ie`
|
||||
|
||||
## Role
|
||||
|
||||
- **Headscale** ([`tailscale.nix`](../../../nixos/boxes/britway/tailscale.nix)) — the
|
||||
self-hosted Tailscale control server (`hs.nul.ie`) the rest of the boxes log
|
||||
into.
|
||||
- **Tailscale node** — advertises itself as an **exit node** and advertises the
|
||||
tailnet routes, so tailnet clients can egress / reach internal prefixes via
|
||||
britway.
|
||||
- **BGP** ([`bgp.nix`](../../../nixos/boxes/britway/bgp.nix)) — part of the AS211024
|
||||
L2 VXLAN mesh (`my.vpns.l2`) alongside `estuary`, `river` and `stream`.
|
||||
- **nginx** ([`nginx.nix`](../../../nixos/boxes/britway/nginx.nix)) — reverse proxy /
|
||||
web front-end with ACME certs.
|
||||
|
||||
## Networking
|
||||
|
||||
- `vultr` assignment on the provider interface; `as211024` on the mesh.
|
||||
- A `veth0`/`tailscale0` setup with SNAT so tailnet traffic egresses via the VPS
|
||||
public IP.
|
||||
@@ -1,19 +0,0 @@
|
||||
# installer
|
||||
|
||||
The custom NixOS installer image used to bootstrap new boxes.
|
||||
|
||||
- **Source:** [`nixos/installer.nix`](../../../nixos/installer.nix)
|
||||
|
||||
## Role
|
||||
|
||||
- Defines `nixos.systems.installer`, a minimal system whose `my.buildAs.*`
|
||||
outputs produce installable artifacts — primarily a bootable **ISO**
|
||||
(`isoImage`), and the same base is reused for kexec/netboot trees.
|
||||
- Build it with the devshell commands (see [`AGENTS.md`](../../../AGENTS.md#commands)):
|
||||
- `build-iso installer`
|
||||
- `build-kexec installer` / `build-netboot installer`
|
||||
- A released ISO is what `colony`'s VM definitions reference as install media; the
|
||||
`update-installer` devshell command tags a release to trigger a rebuild.
|
||||
|
||||
This is a build target rather than a deployed machine — there is no running
|
||||
`installer` host.
|
||||
@@ -1,20 +0,0 @@
|
||||
# kelder-acquisition
|
||||
|
||||
The media stack for the `kelder` site — a slimmer cousin of colony's
|
||||
[`jackflix`](../colony/jackflix.md).
|
||||
|
||||
- **Source:** [`kelder/containers/acquisition/`](../../../nixos/boxes/kelder/containers/acquisition)
|
||||
(`default.nix`, `networking.nix`)
|
||||
- **Host:** NixOS container on `kelder`
|
||||
|
||||
## Role
|
||||
|
||||
- **Jellyfin** for streaming (with hardware transcoding — the `jellyfin` user is
|
||||
in the `render` group, `jellyfin-ffmpeg`).
|
||||
- **Acquisition:** Transmission, Jackett, Radarr, Sonarr.
|
||||
- Runs under the site's shared `kontent` user.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment (alt name `acquisition-ctr`) on the kelder container
|
||||
network; download client networking in `networking.nix`.
|
||||
@@ -1,17 +0,0 @@
|
||||
# kelder-spoder
|
||||
|
||||
An nginx web host on the `kelder` site.
|
||||
|
||||
- **Source:** [`kelder/containers/spoder/`](../../../nixos/boxes/kelder/containers/spoder)
|
||||
(`default.nix`, `nginx.nix`)
|
||||
- **Host:** NixOS container on `kelder`
|
||||
|
||||
## Role
|
||||
|
||||
- Serves web content via **nginx** ([`nginx.nix`](../../../nixos/boxes/kelder/containers/spoder/nginx.nix)),
|
||||
with ACME-managed certificates (nginx in the `acme` group, reloads on renewal).
|
||||
- Runs under the site's shared `kontent` user.
|
||||
|
||||
## Networking
|
||||
|
||||
- `internal` assignment (alt name `spoder-ctr`) on the kelder container network.
|
||||
@@ -1,24 +0,0 @@
|
||||
# kelder
|
||||
|
||||
A host at a remote site ("kelder" = cellar/basement), linked back to the rest of
|
||||
the other boxes over WireGuard. It is itself a **NixOS container host**.
|
||||
|
||||
- **Source:** [`nixos/boxes/kelder/`](../../../nixos/boxes/kelder)
|
||||
(`default.nix`, `boot.nix`, `containers/`, `plymouth/`)
|
||||
- **Domain:** `hentai.engineer`
|
||||
|
||||
## Role
|
||||
|
||||
- **Site uplink:** connects to colony's `estuary` over **WireGuard**
|
||||
(`kelder` peer; see [estuary.md](../colony/estuary.md)), so the remote site is
|
||||
reachable through the colony edge. A periodic `dns_update.py` keeps DNS current.
|
||||
- **Container host:** runs NixOS containers via `my.containers.instances`
|
||||
(`acquisition`, `spoder`).
|
||||
- Custom boot/splash (`boot.nix`, Plymouth theme in `plymouth/`).
|
||||
|
||||
## Containers
|
||||
|
||||
| Container | Role | Docs |
|
||||
| --- | --- | --- |
|
||||
| `kelder-acquisition` | Media stack (Jellyfin + *arr + Transmission) | [kelder-acquisition.md](kelder-acquisition.md) |
|
||||
| `kelder-spoder` | nginx web host | [kelder-spoder.md](kelder-spoder.md) |
|
||||
@@ -1,16 +0,0 @@
|
||||
# tower
|
||||
|
||||
A laptop workstation — a Framework Laptop 13 (12th-gen Intel).
|
||||
|
||||
- **Source:** [`nixos/boxes/tower/default.nix`](../../../nixos/boxes/tower/default.nix)
|
||||
|
||||
## Role
|
||||
|
||||
- Framework Laptop 13 (12th-gen Intel) running the GUI environment
|
||||
(`my.gui.enable`) with home-manager on the `mine` channel.
|
||||
- Joins the tailnet via the self-hosted Headscale on [`britway`](britway.md)
|
||||
(`tailscale up --login-server=https://hs.nul.ie --accept-routes`).
|
||||
- Local virtualisation enabled (`kvm-intel`, IOMMU on).
|
||||
|
||||
> Unlike [`castle`](../home/castle.md), `tower` lives outside the `home/` box
|
||||
> tree and boots from local disks rather than NVMe-oF.
|
||||
@@ -312,6 +312,9 @@ rec {
|
||||
lo = 110;
|
||||
untrusted = 120;
|
||||
wan = 130;
|
||||
wan-pon = 131;
|
||||
|
||||
pon-isp = 10;
|
||||
};
|
||||
hiMTU = 9000;
|
||||
routers = [
|
||||
|
||||
+4
-3
@@ -175,11 +175,11 @@ rec {
|
||||
};
|
||||
|
||||
vm = rec {
|
||||
lvmDisk' = name: lv: {
|
||||
lvmDisk'' = name: vg: lv: {
|
||||
inherit name;
|
||||
backend = {
|
||||
driver = "host_device";
|
||||
filename = "/dev/main/${lv}";
|
||||
filename = "/dev/${vg}/${lv}";
|
||||
# It appears this needs to be set on the backend _and_ the format
|
||||
discard = "unmap";
|
||||
};
|
||||
@@ -189,7 +189,8 @@ rec {
|
||||
};
|
||||
frontend = "virtio-blk";
|
||||
};
|
||||
lvmDisk = lv: lvmDisk' lv lv;
|
||||
lvmDisk' = vg: lv: lvmDisk'' lv vg lv;
|
||||
lvmDisk = lvmDisk' "main";
|
||||
disk = vm: lv: lvmDisk' lv "vm-${vm}-${lv}";
|
||||
};
|
||||
|
||||
|
||||
@@ -95,6 +95,7 @@ in
|
||||
shell = pkgs.bashInteractive;
|
||||
openssh.authorizedKeys.keyFiles = [
|
||||
lib.my.c.sshKeyFiles.harmonia
|
||||
lib.my.c.sshKeyFiles.me
|
||||
];
|
||||
};
|
||||
};
|
||||
|
||||
@@ -53,7 +53,7 @@ in
|
||||
};
|
||||
};
|
||||
});
|
||||
kernelModules = [ "kvm-amd" ];
|
||||
kernelModules = [ "dm-raid" "kvm-amd" ];
|
||||
kernelParams = [ "amd_iommu=on" ];
|
||||
initrd = {
|
||||
availableKernelModules = [ "xhci_pci" "ahci" "usb_storage" "usbhid" "sd_mod" "sr_mod" ];
|
||||
@@ -126,9 +126,9 @@ in
|
||||
};
|
||||
linkConfig.Name = "et1g0";
|
||||
};
|
||||
"10-lan-core" = {
|
||||
"10-lan-core-phy" = {
|
||||
matchConfig.PermanentMACAddress = "e0:d5:5e:68:0c:70";
|
||||
linkConfig.Name = "lan-core";
|
||||
linkConfig.Name = "lan-core-phy";
|
||||
};
|
||||
"10-et100g" = {
|
||||
matchConfig = {
|
||||
@@ -145,6 +145,12 @@ in
|
||||
netdevs = mkMerge [
|
||||
(mkVLAN "lan-hi" vlans.hi)
|
||||
(mkVLAN "lan-lo-phy" vlans.lo)
|
||||
{
|
||||
"25-lan-core".netdevConfig = {
|
||||
Name = "lan-core";
|
||||
Kind = "bridge";
|
||||
};
|
||||
}
|
||||
{
|
||||
"25-lan-lo".netdevConfig = {
|
||||
Name = "lan-lo";
|
||||
@@ -199,6 +205,12 @@ in
|
||||
};
|
||||
"60-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
|
||||
|
||||
"50-lan-core-phy" = {
|
||||
matchConfig.Name = "lan-core-phy";
|
||||
networkConfig = {
|
||||
Bridge = "lan-core";
|
||||
} // networkd.noL3;
|
||||
};
|
||||
"50-lan-lo-phy" = {
|
||||
matchConfig.Name = "lan-lo-phy";
|
||||
networkConfig = {
|
||||
|
||||
@@ -172,12 +172,23 @@
|
||||
};
|
||||
memory = 32768;
|
||||
cleanShutdown.timeout = 120;
|
||||
networks.netboot = {
|
||||
bridge = "lan-lo";
|
||||
waitOnline = "carrier";
|
||||
mac = "52:54:00:a5:7e:93";
|
||||
extraOptions.bootindex = 1;
|
||||
networks = {
|
||||
netboot = {
|
||||
bridge = "lan-lo";
|
||||
waitOnline = "carrier";
|
||||
mac = "52:54:00:a5:7e:93";
|
||||
extraOptions.bootindex = 1;
|
||||
};
|
||||
core = {
|
||||
bridge = "lan-core";
|
||||
ifname = "vm-sfh-core";
|
||||
waitOnline = "carrier";
|
||||
mac = "52:54:00:72:67:51";
|
||||
};
|
||||
};
|
||||
drives = [
|
||||
(vm.lvmDisk' "hdds" "frigate")
|
||||
];
|
||||
hostDevices = {
|
||||
et100g0vf2 = {
|
||||
index = 0;
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
|
||||
configuration = { lib, modulesPath, pkgs, config, assignments, allAssignments, ... }:
|
||||
let
|
||||
inherit (lib) mkForce mkMerge;
|
||||
inherit (lib.my) networkdAssignment mkVLAN;
|
||||
inherit (lib.my.c) networkd;
|
||||
inherit (lib.my.c.home) vlans domain prefixes roceBootModules;
|
||||
@@ -71,11 +72,72 @@
|
||||
dmeventd.enable = true;
|
||||
};
|
||||
fstrim.enable = true;
|
||||
|
||||
# TODO: re-enable once scheduling is tested
|
||||
networkd-dispatcher.enable = mkForce false;
|
||||
|
||||
pppd = {
|
||||
enable = true;
|
||||
peers.digiweb = {
|
||||
autostart = true;
|
||||
enable = true;
|
||||
# Password is shared across all Digiweb customers, so no need for a secret
|
||||
config = ''
|
||||
plugin pppoe.so wan-vlan-inner
|
||||
name "digiweb@nga.digiweb.ie"
|
||||
password "digiweb"
|
||||
noipdefault
|
||||
# no usepeerdns: we ignore Digiweb's resolvers and use the local recursive resolver
|
||||
lcp-echo-interval 1
|
||||
lcp-echo-failure 4
|
||||
noauth
|
||||
persist
|
||||
maxfail 0
|
||||
holdoff 5
|
||||
mtu 1500
|
||||
mru 1500
|
||||
noaccomp
|
||||
default-asyncmap
|
||||
ifname wan
|
||||
'';
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
# PPPoE WAN (Digiweb): pppd owns the `wan` interface on top of VLAN 10, and its
|
||||
# ip-up/ip-down hooks toggle the shared wan-online.target. Nothing else Wants the
|
||||
# target, so it stays inactive until the link is actually up.
|
||||
systemd.targets.wan-online.unitConfig.DefaultDependencies = false;
|
||||
|
||||
environment.etc = {
|
||||
ppp-up = {
|
||||
target = "ppp/ip-up";
|
||||
mode = "0755";
|
||||
text = ''
|
||||
#!${pkgs.runtimeShell}
|
||||
${pkgs.iproute2}/bin/ip route add default dev wan scope link metric 100
|
||||
${config.systemd.package}/bin/systemctl --no-block start wan-online.target
|
||||
'';
|
||||
};
|
||||
ppp-down = {
|
||||
target = "ppp/ip-down";
|
||||
mode = "0755";
|
||||
text = ''
|
||||
#!${pkgs.runtimeShell}
|
||||
${config.systemd.package}/bin/systemctl --no-block stop wan-online.target
|
||||
${pkgs.iproute2}/bin/ip route del default dev wan scope link metric 100
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
systemd.network = {
|
||||
netdevs = mkMerge [
|
||||
(mkVLAN "wan-vlan-outer" vlans.wan-pon)
|
||||
(mkVLAN "wan-vlan-inner" vlans.pon-isp)
|
||||
];
|
||||
|
||||
links = {
|
||||
"10-wan" = {
|
||||
"10-wan-old" = {
|
||||
matchConfig = {
|
||||
# Matching against MAC address seems to break VLAN interfaces
|
||||
# (since they share the same MAC address)
|
||||
@@ -83,7 +145,7 @@
|
||||
PermanentMACAddress = "e0:d5:5e:68:0c:6e";
|
||||
};
|
||||
linkConfig = {
|
||||
Name = "wan";
|
||||
Name = "wan-old";
|
||||
RxBufferSize = 4096;
|
||||
TxBufferSize = 4096;
|
||||
};
|
||||
@@ -101,8 +163,32 @@
|
||||
};
|
||||
};
|
||||
|
||||
# So we don't drop the IP we use to connect to NVMe-oF!
|
||||
networks."60-lan-hi".networkConfig.KeepConfiguration = "static";
|
||||
networks = {
|
||||
"55-lan" = {
|
||||
vlan = [ "wan-vlan-outer" ];
|
||||
};
|
||||
# So we don't drop the IP we use to connect to NVMe-oF!
|
||||
"60-lan-hi".networkConfig.KeepConfiguration = "static";
|
||||
|
||||
"70-wan-vlan-outer" = {
|
||||
matchConfig.Name = "wan-vlan-outer";
|
||||
vlan = [ "wan-vlan-inner" ];
|
||||
networkConfig = networkd.noL3;
|
||||
# baby jumbo: carries the inner VLAN's frames, whose 4B tag counts as payload
|
||||
# at this layer, so it needs 1512 (inner's 1508B payload + the inner 802.1Q tag)
|
||||
linkConfig.MTUBytes = "1512";
|
||||
};
|
||||
# pppd attaches PPPoE to this; just needs to be up with no L3
|
||||
"71-wan-vlan-inner" = {
|
||||
matchConfig.Name = "wan-vlan-inner";
|
||||
linkConfig = {
|
||||
RequiredForOnline = "no";
|
||||
# baby jumbo: PPPoE's 8B overhead leaves a clean 1500 on ppp
|
||||
MTUBytes = "1508";
|
||||
};
|
||||
networkConfig = networkd.noL3;
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
my = {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
imports = [
|
||||
# ./unifi.nix
|
||||
./unifi.nix
|
||||
./hass.nix
|
||||
];
|
||||
}
|
||||
|
||||
@@ -24,6 +24,15 @@ in
|
||||
address = net.cidr.host (65536*5+1) prefixes.hi.v6;
|
||||
};
|
||||
};
|
||||
core = {
|
||||
inherit domain;
|
||||
name = "unifi-ctr-core";
|
||||
mtu = 1500;
|
||||
ipv4 = {
|
||||
address = net.cidr.host 21 prefixes.core.v4;
|
||||
gateway = null;
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
configuration = { lib, config, pkgs, assignments, ... }:
|
||||
@@ -48,7 +57,10 @@ in
|
||||
};
|
||||
|
||||
systemd = {
|
||||
network.networks."80-container-host0" = networkdAssignment "host0" assignments.hi;
|
||||
network.networks = {
|
||||
"80-container-host0" = networkdAssignment "host0" assignments.hi;
|
||||
"80-lan-core" = networkdAssignment "lan-core" assignments.core;
|
||||
};
|
||||
};
|
||||
|
||||
services = {
|
||||
|
||||
@@ -72,6 +72,10 @@ in
|
||||
fsType = "ext4";
|
||||
neededForBoot = true;
|
||||
};
|
||||
"/mnt/frigate" = {
|
||||
device = "/dev/disk/by-label/frigate";
|
||||
fsType = "ext4";
|
||||
};
|
||||
};
|
||||
|
||||
networking = { inherit domain; };
|
||||
@@ -111,6 +115,13 @@ in
|
||||
MTUBytes = toString lib.my.c.home.hiMTU;
|
||||
};
|
||||
};
|
||||
"10-lan-core-ctrs" = {
|
||||
matchConfig = {
|
||||
Driver = "virtio_net";
|
||||
PermanentMACAddress = "52:54:00:72:67:51";
|
||||
};
|
||||
linkConfig.Name = "lan-core-ctrs";
|
||||
};
|
||||
"10-lan-lo-ctrs" = {
|
||||
matchConfig = {
|
||||
Driver = "virtio_net";
|
||||
@@ -131,6 +142,11 @@ in
|
||||
linkConfig.RequiredForOnline = "no";
|
||||
networkConfig = networkd.noL3;
|
||||
};
|
||||
"30-lan-core-ctrs" = {
|
||||
matchConfig.Name = "lan-core-ctrs";
|
||||
linkConfig.RequiredForOnline = "no";
|
||||
networkConfig = networkd.noL3;
|
||||
};
|
||||
"30-lan-lo-ctrs" = {
|
||||
matchConfig.Name = "lan-lo-ctrs";
|
||||
linkConfig.RequiredForOnline = "no";
|
||||
@@ -145,6 +161,11 @@ in
|
||||
MACVLAN = mkForce "lan-hi-ctrs:host0 lan-lo-ctrs:lan-lo";
|
||||
};
|
||||
};
|
||||
unifi = {
|
||||
networkConfig = {
|
||||
MACVLAN = mkForce "lan-hi-ctrs:host0 lan-core-ctrs:lan-core";
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
systemd.services = {
|
||||
@@ -174,9 +195,13 @@ in
|
||||
containers.instances =
|
||||
let
|
||||
instances = {
|
||||
# unifi = {};
|
||||
unifi = {};
|
||||
hass = {
|
||||
bindMounts = {
|
||||
"/mnt/frigate" = {
|
||||
mountPoint = "/var/lib/frigate";
|
||||
readOnly = false;
|
||||
};
|
||||
"/dev/bus/usb/001/002".readOnly = false;
|
||||
"/dev/video0".readOnly = false;
|
||||
"/dev/serial/by-id/usb-Nabu_Casa_Home_Assistant_Connect_ZBT-1_ce549704fe38ef11a2c2e5d154516304-if00-port0" = {
|
||||
|
||||
@@ -152,26 +152,34 @@ in
|
||||
|
||||
networking = { inherit domain; };
|
||||
|
||||
systemd.services =
|
||||
let
|
||||
waitOnline = "systemd-networkd-wait-online@wan.service";
|
||||
in
|
||||
{
|
||||
# Uniform "WAN is up" gate. Consumers attach to this target (via wantedBy +
|
||||
# partOf) rather than depending on it, so it is never pulled in / prematurely
|
||||
# activated. Each box wires up how the target actually gets reached: stream
|
||||
# gates it on networkd's wait-online, river drives it from the pppd hooks.
|
||||
systemd.targets.wan-online.description = "WAN is online";
|
||||
|
||||
systemd.services = {
|
||||
ipsec = {
|
||||
after = [ waitOnline ];
|
||||
requires = [ waitOnline ];
|
||||
after = [ "wan-online.target" ];
|
||||
# strongswan/libreswan force wantedBy=multi-user.target; drop it so the
|
||||
# target is a true gate rather than mere ordering. This matters most on
|
||||
# river, where the target is hook-driven and not in the boot transaction,
|
||||
# so plain ordering wouldn't hold ipsec back at all. partOf re-loads ipsec
|
||||
# (re-orienting its connections) whenever the WAN drops and returns.
|
||||
wantedBy = mkForce [ "wan-online.target" ];
|
||||
partOf = [ "wan-online.target" ];
|
||||
};
|
||||
|
||||
ipv6-clear-default-route = {
|
||||
description = "Clear IPv6 RA default route";
|
||||
after = [ waitOnline ];
|
||||
requires = [ waitOnline ];
|
||||
after = [ "wan-online.target" ];
|
||||
wantedBy = [ "wan-online.target" ];
|
||||
partOf = [ "wan-online.target" ];
|
||||
script = ''
|
||||
# Seems like we can sometimes pick up a default route somehow...
|
||||
${pkgs.iproute2}/bin/ip -6 route del default via fe80::1 || true
|
||||
'';
|
||||
serviceConfig.Type = "oneshot";
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
};
|
||||
};
|
||||
|
||||
@@ -220,41 +228,6 @@ in
|
||||
in
|
||||
mkMerge [
|
||||
{
|
||||
"50-wan-ifb" = {
|
||||
matchConfig.Name = "wan-ifb";
|
||||
networkConfig = networkd.noL3;
|
||||
extraConfig = ''
|
||||
[CAKE]
|
||||
Bandwidth=490M
|
||||
RTTSec=50ms
|
||||
PriorityQueueingPreset=besteffort
|
||||
# DOCSIS preset
|
||||
OverheadBytes=18
|
||||
MPUBytes=64
|
||||
CompensationMode=none
|
||||
'';
|
||||
};
|
||||
"50-wan" = mkMerge [
|
||||
(networkdAssignment "wan" assignments.modem)
|
||||
{
|
||||
matchConfig.Name = "wan";
|
||||
DHCP = "ipv4";
|
||||
dns = [ "127.0.0.1" "::1" ];
|
||||
dhcpV4Config.UseDNS = false;
|
||||
|
||||
qdiscConfig = {
|
||||
Parent = "ingress";
|
||||
Handle = "0xffff";
|
||||
};
|
||||
extraConfig = ''
|
||||
[CAKE]
|
||||
Parent=root
|
||||
Bandwidth=48M
|
||||
RTTSec=50ms
|
||||
'';
|
||||
}
|
||||
];
|
||||
|
||||
"55-lan" = {
|
||||
matchConfig.Name = "lan";
|
||||
vlan = [ "lan-hi" "lan-lo" "lan-untrusted" "wan-tunnel" ];
|
||||
|
||||
@@ -13,6 +13,13 @@ let
|
||||
in
|
||||
{
|
||||
config = {
|
||||
# Let pdns-recursor bind the VRRP VIPs even on the backup, where the addresses
|
||||
# aren't present locally
|
||||
boot.kernel.sysctl = {
|
||||
"net.ipv4.ip_nonlocal_bind" = 1;
|
||||
"net.ipv6.ip_nonlocal_bind" = 1;
|
||||
};
|
||||
|
||||
my = {
|
||||
secrets.files = {
|
||||
"home/pdns/auth.conf" = {
|
||||
@@ -40,6 +47,10 @@ in
|
||||
"127.0.0.1" "::1"
|
||||
assignments.hi.ipv4.address assignments.hi.ipv6.address
|
||||
assignments.lo.ipv4.address assignments.lo.ipv6.address
|
||||
# VRRP VIPs: DNS follows the master, so clients only ever have one
|
||||
# (always-live) resolver address and never hang on a dead router
|
||||
vips.hi.v4 vips.hi.v6
|
||||
vips.lo.v4 vips.lo.v6
|
||||
];
|
||||
allow_from = [
|
||||
"127.0.0.0/8" "::1/128"
|
||||
@@ -223,6 +234,7 @@ in
|
||||
dave-lo IN AAAA ${net.cidr.host (65536+2) prefixes.lo.v6}
|
||||
|
||||
shytzel IN A ${net.cidr.host 12 prefixes.core.v4}
|
||||
brian IN A ${net.cidr.host 13 prefixes.core.v4}
|
||||
|
||||
wave IN A ${net.cidr.host 12 prefixes.hi.v4}
|
||||
; wave IN AAAA ${net.cidr.host (65536+3) prefixes.hi.v6}
|
||||
|
||||
@@ -33,7 +33,7 @@ def main():
|
||||
|
||||
print(f'Updating {args.record} -> {address}')
|
||||
cf.dns.records.edit(
|
||||
zone_id=zone.id, dns_record_id=record.id, name=args.record,
|
||||
zone_id=zone.id, dns_record_id=record.id, name=args.record, ttl=60,
|
||||
type='A', content=address)
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
||||
@@ -83,7 +83,8 @@ in
|
||||
}
|
||||
{
|
||||
name = "domain-name-servers";
|
||||
data = "${net.cidr.host 1 prefixes.hi.v4}, ${net.cidr.host 2 prefixes.hi.v4}";
|
||||
# VRRP VIP so DNS follows the master and clients never hit a dead router
|
||||
data = vips.hi.v4;
|
||||
}
|
||||
{
|
||||
name = "interface-mtu";
|
||||
@@ -116,7 +117,8 @@ in
|
||||
}
|
||||
{
|
||||
name = "domain-name-servers";
|
||||
data = "${net.cidr.host 1 prefixes.lo.v4}, ${net.cidr.host 2 prefixes.lo.v4}";
|
||||
# VRRP VIP so DNS follows the master and clients never hit a dead router
|
||||
data = vips.lo.v4;
|
||||
}
|
||||
];
|
||||
pools = [
|
||||
|
||||
@@ -3,11 +3,14 @@ let
|
||||
# TODO: Move into nixpkgs
|
||||
mstpd = pkgs.mstpd.overrideAttrs {
|
||||
patches = [ ./mstpd.patch ];
|
||||
# Delete postInstall since it nukes the bridge-stp script we need
|
||||
postInstall = "";
|
||||
};
|
||||
in
|
||||
{
|
||||
environment = {
|
||||
systemPackages = [
|
||||
# For kernel to call bridge-stp (see ./pkgs/os-specific/linux/kernel/bridge-stp-helper.patch)
|
||||
mstpd
|
||||
];
|
||||
etc = {
|
||||
@@ -39,8 +42,8 @@ in
|
||||
before = [ "network-pre.target" ];
|
||||
serviceConfig = {
|
||||
Type = "forking";
|
||||
ExecStart = "${mstpd}/sbin/bridge-stp restart";
|
||||
ExecReload = "${mstpd}/sbin/bridge-stp restart_config";
|
||||
ExecStart = "${mstpd}/bin/bridge-stp restart";
|
||||
ExecReload = "${mstpd}/bin/bridge-stp restart_config";
|
||||
PIDFile = "/run/mstpd.pid";
|
||||
Restart = "always";
|
||||
PrivateTmp = true;
|
||||
|
||||
@@ -2,7 +2,14 @@ index: { lib, pkgs, ... }:
|
||||
let
|
||||
inherit (lib) mkForce concatMapStringsSep;
|
||||
inherit (lib.my) net;
|
||||
inherit (lib.my.c.home) domain prefixes;
|
||||
inherit (lib.my.c.home) domain prefixes vips;
|
||||
|
||||
# untrusted uses external (Cloudflare) resolvers, matching the v4 kea config;
|
||||
# trusted VLANs use the internal recursor via its floating VRRP VIP
|
||||
rdnss = name:
|
||||
if name == "untrusted"
|
||||
then "2606:4700:4700::1111 2606:4700:4700::1001"
|
||||
else vips."${name}".v6;
|
||||
|
||||
mkInterface = name: ''
|
||||
interface lan-${name} {
|
||||
@@ -10,7 +17,7 @@ let
|
||||
AdvRASrcAddress { fe80::1; };
|
||||
AdvLinkMTU ${toString prefixes."${name}".mtu};
|
||||
prefix ${prefixes."${name}".v6} {};
|
||||
RDNSS ${net.cidr.host 1 prefixes."${name}".v6} ${net.cidr.host 2 prefixes."${name}".v6} {};
|
||||
RDNSS ${rdnss name} {};
|
||||
DNSSL ${domain} dyn.${domain} ${lib.my.c.colony.domain} ${lib.my.c.britway.domain} {};
|
||||
};
|
||||
'';
|
||||
|
||||
@@ -6,9 +6,11 @@
|
||||
nixpkgs = "mine";
|
||||
home-manager = "mine";
|
||||
|
||||
configuration = { lib, pkgs, config, ... }:
|
||||
configuration = { lib, pkgs, config, assignments, ... }:
|
||||
let
|
||||
inherit (lib);
|
||||
inherit (lib) mkMerge;
|
||||
inherit (lib.my) networkdAssignment;
|
||||
inherit (lib.my.c) networkd;
|
||||
in
|
||||
{
|
||||
imports = [ ./routing-common/mstpd.nix ];
|
||||
@@ -75,6 +77,32 @@
|
||||
};
|
||||
};
|
||||
|
||||
# wan carries a permanent static modem-management address (assignments.modem)
|
||||
# alongside the DHCP public IP, so wait-online@wan reports "online" as soon as
|
||||
# the static address is up - before the DHCP lease arrives. ipsec's left= is the
|
||||
# public IP, so gating on wait-online lets it start unoriented and never connect.
|
||||
# Gate instead on the DHCP default route, which only exists once the public lease
|
||||
# is up (the static modem address has no gateway).
|
||||
systemd.services.wan-wait-online = {
|
||||
description = "Wait for the wan default route (public DHCP lease)";
|
||||
after = [ "systemd-networkd.service" ];
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
TimeoutStartSec = "300";
|
||||
};
|
||||
script = ''
|
||||
until [ -n "$(${pkgs.iproute2}/bin/ip -4 route show default dev wan)" ]; do
|
||||
sleep 1
|
||||
done
|
||||
'';
|
||||
};
|
||||
systemd.targets.wan-online = {
|
||||
requires = [ "wan-wait-online.service" ];
|
||||
after = [ "wan-wait-online.service" ];
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
};
|
||||
|
||||
systemd.network = {
|
||||
netdevs = {
|
||||
"25-lan" = {
|
||||
@@ -147,6 +175,41 @@
|
||||
matchConfig.Name = "lan-dave";
|
||||
networkConfig.Bridge = "lan";
|
||||
};
|
||||
|
||||
"50-wan-ifb" = {
|
||||
matchConfig.Name = "wan-ifb";
|
||||
networkConfig = networkd.noL3;
|
||||
extraConfig = ''
|
||||
[CAKE]
|
||||
Bandwidth=490M
|
||||
RTTSec=50ms
|
||||
PriorityQueueingPreset=besteffort
|
||||
# DOCSIS preset
|
||||
OverheadBytes=18
|
||||
MPUBytes=64
|
||||
CompensationMode=none
|
||||
'';
|
||||
};
|
||||
"50-wan" = mkMerge [
|
||||
(networkdAssignment "wan" assignments.modem)
|
||||
{
|
||||
matchConfig.Name = "wan";
|
||||
DHCP = "ipv4";
|
||||
dns = [ "127.0.0.1" "::1" ];
|
||||
dhcpV4Config.UseDNS = false;
|
||||
|
||||
qdiscConfig = {
|
||||
Parent = "ingress";
|
||||
Handle = "0xffff";
|
||||
};
|
||||
extraConfig = ''
|
||||
[CAKE]
|
||||
Parent=root
|
||||
Bandwidth=48M
|
||||
RTTSec=50ms
|
||||
'';
|
||||
}
|
||||
];
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ let
|
||||
tftpRoot = pkgs.linkFarm "tftp-root" [
|
||||
{
|
||||
name = "ipxe-x86_64.efi";
|
||||
path = "${pkgs.ipxe}/ipxe.efi";
|
||||
path = "${pkgs.ipxe}/snp.efi";
|
||||
}
|
||||
];
|
||||
menuFile = pkgs.runCommand "menu.ipxe" {
|
||||
|
||||
Reference in New Issue
Block a user