35 Commits

Author SHA1 Message Date
jackos1998 88d0d19239 docs/dns: Add live record reference
CI / Check, build and cache nixfiles (push) Successful in 46m57s
Update docs / update (push) Failing after 1m6s
Generate forward and reverse record tables from authoritative AXFRs
while preserving handwritten Markdown outside per-zone markers. Run the
generator in CI and link the reference from the relevant docs.
2026-08-02 01:07:27 +01:00
jackos1998 8f9ca5e1c4 nixos/dns: Allow internal zone transfers
Permit AXFR from shared internal networks and cross-site egress
addresses while retaining loopback and the existing secondary.
2026-08-02 01:07:03 +01:00
jackos1998 08605ab422 docs/boxes: Prioritize hardware details
Place physical hardware inventories and VPS resource allocations near
the top of each box page, while leaving broader platform details in
their topical sections.
2026-08-02 00:23:30 +01:00
jackos1998 c354998e11 ci/docs: Regenerate docs on master
Update docs / update (push) Successful in 1m4s
CI / Check, build and cache nixfiles (push) Successful in 51m19s
Run both documentation generators after master updates and commit their
outputs only when the generated files change.
2026-08-02 00:12:57 +01:00
jackos1998 b44095f1ed docs/options: Add option reference generator
Evaluate custom module options through a synthetic system, stabilize
configuration-dependent defaults for rendering, and commit the generated
reference beside its updater.
2026-08-02 00:12:57 +01:00
jackos1998 4084dfe64e docs/assignments: Add table generator
Render site tables from the aggregated assignments, group rows by
assignment name, preserve handwritten notes, and expose the updater as a
flake package.
2026-08-02 00:12:57 +01:00
jackos1998 f3d921bcdc AGENTS.md: Add documentation guidance
Keep agent context lean and define documentation conventions for
terminology, page structure, ownership, generated content, and commit
messages.
2026-08-02 00:12:57 +01:00
jackos1998 a2f3410e42 docs/boxes: Document deployed boxes
Add per-site and per-box inventories, consolidate shared network design,
and relocate the switch and access-point references under the home site.
2026-08-02 00:12:57 +01:00
jackos1998 43cf35d54e docs: Add infrastructure overview
Document the module architecture, deployment workflows, installer, and
high-level box map.
2026-08-02 00:12:57 +01:00
jackos1998 390818ffdb home-manager/gui: Disable pi-coding-agent (breaks CI)
`pi-coding-agent-bun` breaks `nix flake check` in CI: its bun2nix
`fetchBunDeps` calls `builtins.filterSource` on subpaths of the pi.nix
flake source, which requires that source derivation to be realised in the
local store. A fresh CI runner doesn't have it, so eval aborts with
`path '...-source.drv' is not valid`. It only works locally because the
source is already realised there.

Not using pi right now, so comment out the input, overlays, home module
import and the package (leaving a root-cause note in `flake.nix`) rather
than working around the upstream issue.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-02 00:12:51 +01:00
jackos1998 43b5bedb48 nixos/vms: Fix LVM disk helper regression
Adding volume-group support changed the LVM disk helper's argument
meaning, so standard VM disks and the legacy darts calls resolved disk
names as volume groups.

Restore the main volume group for standard disks and map the differently
named darts volumes explicitly.
2026-08-02 00:12:36 +01:00
jackos1998 e47e30df19 .claude: Add shared permissions allow list, ignore local settings
CI / Check, build and cache nixfiles (push) Failing after 2m30s
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-28 20:13:03 +01:00
jackos1998 e077c6e30c home-manager/common: Add treemd 2026-07-28 12:06:24 +01:00
jackos1998 6f8d499ec2 docs: Add home APs and switches references
Document the home network's out-of-band devices that aren't managed by
this flake:

- `home-aps.md` — the two dumb APs (`vibe` MikroTik cAP ax, `wave` Cudy
  AX3000 on OpenWrt): shared VLAN-trunk design (`hi`/`lo` are both
  trusted client VLANs differing only by MTU), SSIDs, per-AP management
  addressing, roaming (802.11k/v), and the OpenWrt flash + on-device
  config for `wave` (incl. the `mtk_eth_soc` eth0 2026-byte MTU cap that
  keeps it off the jumbo `hi` VLAN, wpad-mbedtls swap, HE160).
- `home-switches.md` — the hand-configured switches (`jim`/`dave`
  MikroTik, `brian` UniFi) topology, VLAN map, and ONT/WAN path.

Add the `home-aps.md` pointer to AGENTS.md alongside the existing
switches reference.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-28 12:06:24 +01:00
jackos1998 d0859d4c5c nixos/home: Add wave AP DNS records, renumber vibe to .15
New Cudy AX3000 AP `wave` takes host `.14` on core (backup) and lo
(primary); replaces the decommissioned AP of the same name. It's managed
on `lo`, not `hi`, because its 2.5GbE (`mtk_eth_soc` eth0) caps at MTU
2026 and so can't join `hi`'s jumbo (9000) fabric — and `hi`/`lo` are
both trusted client VLANs differing only by MTU.

Renumber the MikroTik AP `vibe` from `.13` to `.15` and give it the same
core/hi/lo records as `jim`/`dave` (was `hi`-only), matching its
on-device reconfiguration. `vibe` stays on `hi` since it can do jumbo.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 09:46:13 +01:00
jackos1998 9b582bec7b home-manager/gui: Add pi-coding-agent
CI / Check, build and cache nixfiles (push) Failing after 2m29s
2026-07-26 18:55:20 +01:00
jackos1998 8c4223af18 AGENTS.md: Document switches, deploy --boot, and memory policy
CI / Check, build and cache nixfiles (push) Successful in 45m13s
- Note `CLAUDE.md` is a symlink to `AGENTS.md` (edit the real file).
- Policy: prefer recording durable guidance here over agent memory.
- Add the "Home switches" section (`jim`/`dave`/`brian`, access,
  confirm-before-apply) pointing at `home-switches.md`.
- Document `deploy --boot` for staging a config without live-switching.
- Note the `ssh-agent` hang workaround for outbound SSH (`ssh-machine`/`deploy`).
- Add a comment convention: comment where useful, not for trivial code, and
  describe general purpose rather than the one-off reason for a change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 21:35:50 +01:00
jackos1998 b95992735e nixos/home: Trunk Digiweb VLAN 10 to river instead of translating
With a single ONT the ISP's VLAN 10 is unique on the fabric, so there's no
need to swap it to a private fabric VLAN. Carry pon-isp (10) trunked straight
through the switches to river and run PPPoE directly on it, dropping the old
wan-pon-isp (141) translation. The ONT's untagged management is PVID'd onto
wan-pon-ont (140) at its edge switch (brian).

wan-pon-isp (141) is kept as a constant, reserved for the future multi-ONT
case where per-port VLAN translation disambiguates two VLAN-10 ONTs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 21:29:16 +01:00
jackos1998 0d1562d150 home-manager/gui: Add more LSPs
CI / Check, build and cache nixfiles (push) Successful in 44m33s
2026-07-23 01:37:33 +01:00
jackos1998 49de78d47f nixos/home: De-share Virgin Media modem specifics from routing-common
The modem management subnet is stream-specific (it shares stream's `wan`
interface), so it no longer belongs in the shared routing-common. Drop the
`modem` network assignment and configure the address directly on stream's
`wan` (modemV4). The firewall reject and DNS wan-A-record skipBroadcasts that
existed only because of the modem move to per-box options
`my.homeRouter.{firewall.untrustedRejectV4,dns.wanSkipBroadcasts}`, set solely
by stream. river needs neither: its ONT sits on a separate interface already
blocked by the interface-based forward rules.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 01:35:08 +01:00
jackos1998 832d0b5542 nixos/home: Rework river WAN to single-tag VLAN 140/141
Digiweb's ONT delivers its untagged management traffic (192.168.100.0/24)
alongside the tagged ISP VLAN 10. The switch feeding river now tags the ONT's
untagged port as wan-pon-ont (140) and swaps the ISP VLAN 10 to wan-pon-isp
(141) on ingress, so both reach river single-tagged: river takes an address in
the ONT's /24 on 140 to reach its web UI, and PPPoE runs directly on 141.

Replaces the old QinQ-into-131 (wanStacked) transport; the wan-pon (131) VLAN
constant is dropped and prefixes.ont added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 01:35:08 +01:00
jackos1998 4d4c05ea70 Symlink CLAUDE.md to AGENTS.md
CI / Check, build and cache nixfiles (push) Successful in 49m29s
2026-07-22 18:44:03 +01:00
jackos1998 0bd08c4a91 nixos/home: Recurse over IPv4 only on the home routers
CI / Check, build and cache nixfiles (push) Successful in 46m1s
The routers' IPv6 default route runs over the as211024 mesh as a
proto-static route that blackholes rather than withdrawing when the mesh
flaps (e.g. during ipsec churn on a deploy). With "::" in the recursor's
outgoing.source_address, pdns-recursor kept selecting IPv6 to reach
authoritative servers, stalled on timeouts, and took recursion down with
it whenever the mesh bounced. Drop "::" so upstream recursion goes out
the WAN over IPv4 and stays up; AAAA records are still served fine.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 19:56:43 +01:00
jackos1998 a86888a2c7 nixos/home: Use Digiweb static IP for river WAN
Digiweb assigned river a static IPv4 (84.203.124.128). Point river's
public address at it and have pppd actively request it as the IPCP local
address (local:remote form) rather than passively accepting whatever the
peer offers via noipdefault. The address stays sourced from the
routersPubV4 constant, so the ipsec l2 mesh peer, the nft DNS redirect,
and the cross-router route all follow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 18:06:21 +01:00
jackos1998 dc1ec3bf5a docs: Ask before running fmt
CI / Check, build and cache nixfiles (push) Successful in 45m46s
fmt reindents non-canonically-formatted files wholesale, burying logical
changes in whitespace churn. Note to match surrounding style by hand.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 16:11:25 +01:00
jackos1998 823ed83252 nixos/home: Unstack river WAN to single-tag VLAN 10
Digiweb's BRAS answers PADI only on a single-tagged ISP VLAN (pon-isp,
10); the double-stacked wan-pon (131) transport never got a PADO because
the UniFi access switch can't push the outer tag onto already-tagged
ingress frames (no QinQ/tag-stacking). Attach VLAN 10 directly to the
physical `lan` link for now.

Kept reversible via a `wanStacked` flag: flip it back to true to restore
the 131->10 stack once a QinQ-capable device sits in front of the ONT.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 16:11:25 +01:00
jackos1998 300103f2ba nixos/home: Correct river wan-vlan-outer MTU to 1512
CI / Check, build and cache nixfiles (push) Successful in 45m9s
The outer WAN VLAN carries the inner VLAN's frames, whose 4B 802.1Q tag
counts as payload at this layer. To fit the inner's 1508B baby-jumbo
payload plus that tag it needs 1512, not 1508. Harmless before only
because the physical `lan` (9000) absorbed the extra bytes; this makes
the outer's MTU literally reflect what it transports.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 10:57:13 +01:00
jackos1998 d33f32ce5b docs: Document home-router HA pattern and boxes terminology
CI / Check, build and cache nixfiles (push) Successful in 45m40s
Capture what this session surfaced: the routing-common indexed-module
active/backup pattern, VRRP VIPs for gateway+DNS, the wan-online.target
convention, the networkd helper locations, and the "boxes not fleet"
terminology. Also note the fish login shell caveat for ssh scripting.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 22:44:22 +01:00
jackos1998 14d9bba4eb nixos/home: Serve DNS on VRRP VIP and fix ipsec WAN ordering
Clients were handed both routers' real addresses as resolvers, so a downed
router meant per-query resolver timeouts. Serve pdns-recursor on the VRRP
VIPs (with non-local bind so the backup can pre-bind them) and advertise the
VIP via kea and radvd, so DNS follows the master. untrusted advertises
Cloudflare over v6 to match its v4 config.

ipsec started before the WAN's public IP was up: stream's wan carries a
static modem address that satisfies wait-online before the DHCP lease, so
libreswan loaded its mesh conns (left=<public IP>) unoriented and never
initiated. Gate stream's wan-online.target on the DHCP default route
instead, and mkForce ipsec onto wan-online.target only (dropping the
strongswan/libreswan multi-user.target pull-in) so the gate actually holds
on both boxes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 22:38:02 +01:00
jackos1998 e63cee7b09 nixos/home: Migrate river WAN to PPPoE over Digiweb
river's WAN moves to a PPPoE session (Digiweb, over VLAN 10 with baby-jumbo
1508 MTU) driven by pppd, with its ip-up/ip-down hooks toggling a new
wan-online.target. stream keeps its DHCP WAN, so the existing 50-wan /
50-wan-ifb networkd config moves out of routing-common into stream, and the
shared consumers (ipsec, ipv6-clear-default-route) now attach to
wan-online.target instead of depending on systemd-networkd-wait-online@wan
directly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 22:37:05 +01:00
jackos1998 a8318d3de2 nixos/home: Separate HDD LV for frigate
CI / Check, build and cache nixfiles (push) Successful in 51m19s
2026-07-19 11:47:37 +01:00
jackos1998 aab4a193ae nixos/home: Add brian
CI / Check, build and cache nixfiles (push) Failing after 3h2m50s
2026-07-15 02:22:15 +01:00
jackos1998 541102f683 nixos/netboot: Switch to iPXE snp.efi
CI / Check, build and cache nixfiles (push) Successful in 45m47s
2026-07-14 22:42:04 +01:00
jackos1998 805590a705 nixos/routing-common: Fix mstpd missing bridge-stp script
CI / Check, build and cache nixfiles (push) Successful in 46m37s
2026-07-12 18:40:38 +01:00
jackos1998 f8dbd99a7b nixos/routing-common: Add missing ttl param to DDNS update 2026-07-12 17:15:57 +01:00
113 changed files with 5356 additions and 1062 deletions
+18
View File
@@ -0,0 +1,18 @@
{
"permissions": {
"allow": [
"Bash(nix eval:*)",
"Bash(nix flake check:*)",
"Bash(nix build:*)",
"Bash(check-system:*)",
"Bash(build-system:*)",
"Bash(build-home:*)",
"Bash(git status:*)",
"Bash(git diff:*)",
"Bash(git log:*)",
"Bash(git show:*)",
"Bash(drill:*)",
"Bash(ping:*)"
]
}
}
+51
View File
@@ -0,0 +1,51 @@
name: Update docs
on:
push:
branches: [master]
jobs:
update:
if: "!contains(github.event.head_commit.message, 'docs: Update generated references')"
runs-on: ubuntu-26.04
permissions:
contents: write
steps:
- uses: actions/checkout@v6
- uses: cachix/install-nix-action@v31
with:
github_access_token: ${{ secrets.GH_PULL_TOKEN }}
extra_nix_config: |
extra-substituters = https://nix-cache.nul.ie
extra-trusted-public-keys = nix-cache.nul.ie-1:BzH5yMfF4HbzY1C977XzOxoPhEc9Zbu39ftPkUbH+m4=
- name: Update assignment tables
run: nix run .#update-docs-assignments
- name: Update option reference
run: nix run .#update-docs-options
- name: Update DNS reference
run: >
nix run .#update-docs-dns --
ams1.int.nul.ie
100.10.in-addr.arpa
2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa
h.nul.ie
168.192.in-addr.arpa
0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa
- name: Commit and push if changed
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
REPO_URL: ${{ gitea.repositoryUrl }}
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git remote set-url origin "${REPO_URL/https:\/\//https:\/\/oauth2:${GITEA_TOKEN}@}"
git add docs/
if ! git diff --cached --quiet; then
git commit -m "docs: Update generated references"
git push
fi
+1
View File
@@ -4,3 +4,4 @@ result*
!/.vms/.gitkeep
/.keys/*.key
*.swp
/.claude/settings.local.json
+155 -11
View File
@@ -2,11 +2,29 @@
This file provides guidance to coding agents when working with code in this repository.
`CLAUDE.md` at the repo root is a symlink to this file — edit `AGENTS.md`, not the symlink (some
tools refuse to write through a symlink and will error on `CLAUDE.md`).
**Prefer this file over agent memory.** When you learn something durable about this repo — a
convention, a workflow gotcha, a design rationale — record it here (or in a repo doc this file
points to, e.g. `docs/sites/home/switches.md`), not in agent memory. AGENTS.md is versioned and
shared; memory is not.
Keep this file lean: it should contain only rules, workflows, safety constraints and sharp gotchas
an agent needs before starting work. Detailed inventories, topology, service operation and design
rationale belong in the canonical document under `docs/`; summarize only the essential constraint
here and link to that document instead of maintaining a second copy.
Claude Code permissions live in two files: `.claude/settings.json` (versioned, shared — the
committed allow list of safe-to-auto-approve commands) and `.claude/settings.local.json` (personal,
gitignored — where interactive "always allow" grants accumulate). Put durable, generally-safe
commands in the shared file; leave one-off or workstation-specific grants in the local one.
## Overview
Personal Nix flake managing NixOS systems and home-manager configurations for a set of
machines ("boxes": servers, home machines, routers). It is built around a **custom module system** layered
on top of NixOS/home-manager, not the stock flake `nixosConfigurations` pattern.
**boxes**, never a "fleet". It is built around a **custom module
system** layered on top of NixOS/home-manager, not the stock flake `nixosConfigurations` pattern.
## Commands
@@ -28,20 +46,46 @@ Common ones:
Pass the flake-qualified node, e.g. `deploy .#git`. The deploy node name is **always** the system
name (`deploy-rs.nix` keys nodes directly off `nixos.systems` / `home-manager.homes`); a system is
only a deploy target when `config.my.deploy.enable` is true (defaults true; auto-disabled for dev
VMs and containers).
VMs and containers). Pass `--boot` to stage a config as the boot default **without** live-switching
(`deploy --boot .#<host>`) — the box keeps running its current generation until it reboots. Use this
when a live `switch` would break connectivity mid-change (e.g. a router's WAN VLAN rework), then
reboot to cut over.
- `ssh-machine <name> [cmd]` — SSH to a NixOS system or home-manager config by name. Resolves the
target and ssh options (identity, port) from its deploy-rs node, so it needs `my.deploy.enable`
(same gate as `deploy`).
(same gate as `deploy`). Boxes default to the `fish` login shell, so pipe multi-statement remote
scripts through `bash` (e.g. `ssh-machine <name> bash -s < script.sh`) rather than `&&`/`for`.
If outbound SSH hangs at the publickey step (flaky `ssh-agent`), disable the agent for the call:
`SSH_AUTH_SOCK= ssh-machine …` (or add `-o IdentityAgent=none` to a raw `ssh`).
- `ragenix` — edit age secrets using `.keys/dev.key` as identity (see Secrets).
- `repl``nix repl .#`.
- `update-nixpkgs` / `update-home-manager` — bump pinned inputs.
Check everything (what CI runs): `nix flake check --no-build`.
Use the narrowest relevant evaluation while iterating: `check-system <host>` for a box config,
`nix eval .#nixfiles.config.nixos.allAssignments --json` for assignment generation, or
`nix build --no-link .#nixfiles.config.nixos.optionsDoc` for the option reference. Reserve
`nix flake check --no-build` for final broad validation or reproducing CI.
CI builds each attr of `.#ci.x86_64-linux` (systems, homes, packages, shell) and pushes to the
Harmonia binary cache; see `.gitea/workflows/ci.yaml` and `ci/push-to-cache.sh`.
Harmonia binary cache; see `.gitea/workflows/ci.yaml` and `ci/push-to-cache.sh`. A separate
workflow (`.gitea/workflows/update-docs.yaml`) regenerates the network assignments, NixOS option
reference and live DNS reference under `docs/`.
For DNS lookups use **`drill`** (ldns) — `dig` isn't installed in this environment (it fails with
exit 127, which is easy to miss if stderr is redirected). E.g. `drill -Q @<resolver> <name> A`.
For privilege escalation use **`doas`**, not `sudo` — the boxes don't install `sudo` (it fails with
`command not found`). E.g. `doas ip link set <if> up`.
For ad-hoc packages prefer **`pkgs#<name>`** over `nixpkgs#<name>` (e.g. `nix run pkgs#python3`):
`pkgs` is a system registry alias for the same pinned nixpkgs the boxes are built from, so it
resolves from the local store instead of fetching a different channel.
## Architecture
The mechanics in this section have expanded human-readable write-ups under `docs/`:
`docs/architecture.md` (module system), `docs/networking.md` (assignments, topology, meshes) and
`docs/deployment.md` (deploy-rs, devshell, secrets, CI). This section stays the terse agent
version; consult those for depth.
### The custom module system
`flake.nix` does **not** call `nixosSystem` per host directly. Instead it `evalModules` over
`./nixos`, `./home-manager`, `./deploy-rs.nix`, and the per-host files listed in the `configs`
@@ -91,10 +135,33 @@ Per-host configs live under `nixos/boxes/<host>` (some are single `.nix` files,
with nested VMs/containers under e.g. `colony/vms`). Many "systems" are VMs or containers managed
via the `vms` / `containers` modules and the `l2mesh` VXLAN module.
For a human-readable map of what is actually deployed (per-machine roles, services and networking),
see `README.md` and the per-machine docs under `docs/boxes/` (grouped `docs/boxes/colony/`,
`docs/boxes/home/`, `docs/boxes/misc/`). Keep these in sync when adding, removing or repurposing
a machine or service.
For a human-readable map of what is actually deployed (per-box roles, services and networking),
see `README.md` and `docs/` (index at `docs/README.md`; box pages under `docs/sites/`,
`docs/remote/`, `docs/mobile/`). Keep these in sync when adding, removing or repurposing a box or
service. The network-assignment tables are consolidated in `docs/networking.md` (one per site,
CI-generated from `allAssignments` between `<!-- assignments: <site> -->` markers); box pages
link to that section. Write the prose and let the updater refresh the tables.
### Home routers (`nixos/boxes/home/routing-common`)
`river` and `stream` share `routing-common`, a function of an `index`; keep common HA behavior there
and box-specific WAN behavior in the respective box file. Client-facing gateway and DNS services
must use the floating VIPs. `routing-common` only declares the inert `wan-online.target`; each box
wires its own reachability mechanism. See `docs/networking.md#router-ha` and the router box pages
for the design and operational detail. Keep pair-wide behavior in `docs/networking.md`; router
pages should document only their WAN, platform and other box-specific responsibilities.
The easy-to-mix-up implementation details are that `lib.my.networkdAssignment` and
`lib.my.mkVLAN` live under `lib.my`, networkd snippet constants such as `networkd.noL3` live under
`lib.my.c`, and interface MTU belongs in the `.network`'s `linkConfig.MTUBytes`.
### Home switches (`jim` / `dave` / `brian`)
These switches are not managed by the flake, and changing them is out-of-band and hard to revert.
Read `docs/sites/home/switches.md` before WAN or VLAN work, always confirm before applying a switch
change, and keep the documented layout and `lib.my.c.home.vlans` in sync.
### Home wireless APs (`vibe` / `wave`)
These APs are not managed by the flake; only their DNS records are. Read and update
`docs/sites/home/aps.md` when changing an AP or its VLAN layout.
## Secrets
@@ -107,7 +174,14 @@ private keys) is required for editing secrets, deploying, and running dev VMs.
## Conventions
- Format with `nixpkgs-fmt` (`fmt`). 2-space indent, `inherit (...)` blocks at the top of `let`.
- Format with `nixpkgs-fmt` (`fmt`). 2-space indent, `inherit (...)` blocks at the top of `let`
prefer `inherit (lib) mkOption ...;` (and bare use) over qualifying inline as `lib.mkOption`.
**Ask before running `fmt`** — some files aren't canonically formatted, so `fmt` can reindent a
whole file and bury a logical change in whitespace churn. Match the surrounding style by hand and
leave formatting to the user unless they ask.
- Comment where it genuinely aids understanding, but not for trivial/obvious code — match the file's
existing (fairly sparse) comment density. When adding something general, comment its general
purpose, not the specific change or one-off reason it was introduced for.
- Prefer `lib.my` helpers (`mkOpt'`, `mkBoolOpt'`, `mkDefault'`) and `lib.my.c` constants over
reimplementing.
- New shared functionality → a module in `*/modules/` + entry in `_list.nix`, options under `my.*`.
@@ -116,3 +190,73 @@ private keys) is required for editing secrets, deploying, and running dev VMs.
as `overlays.default`.
- In prose and commit messages, quote code-like identifiers (commands, options, paths, package and
attribute names) in backticks.
- Keep docs and prose plain — avoid vague AI-tell filler. In particular use "layout"/"structure"
rather than "shape", and "General" rather than "cross-cutting topics"; name things directly
instead of reaching for umbrella words.
- For network connectivity, use the concrete term — "interface", "connection", "network
attachment" or the network's name — rather than calling it a "leg".
- Call the systems **"boxes"**, never "machines" or "fleet", except where "machine" is part of a
command, option or upstream technical term such as QEMU's machine type.
- Commit subjects follow `area/scope: Capitalized summary` (e.g. `nixos/home: ...`); keep logically
distinct changes in separate commits. Aim for 50-character subjects and do not exceed 72
characters. Wrap commit bodies at 72 columns. A concise body describing the change and its
rationale is welcome when the subject alone does not provide enough context.
## Documentation
Human-readable docs live under `docs/` (index: `docs/README.md`). Box pages are under
`docs/sites/<site>/`, `docs/remote/`, `docs/mobile/`. A box that itself hosts sub-systems
(containers/nested VMs) gets a **directory named for it** with a `README.md` and the child pages
beneath it (e.g. `shill/README.md` + `shill/containers/*.md`, `sfh/README.md`). When a site and its
physical box share a name, the site keeps the `README.md` and the box page stays alongside it (for
example `sites/colony/README.md` and `sites/colony/colony.md`).
**Box page layout** (match the existing pages): H1 + a one-line intro; a short bullet list of
`Source` / `Host` / `nixpkgs`; an optional hardware inventory or VPS resource-allocation section;
`## Role`; `## Network assignments` that **links** to
[`networking.md#box-assignments`](docs/networking.md#box-assignments) (never inline the table); one
`##` section per topic; `## Notable config files` last. Keep non-hardware platform details in their
topical sections rather than moving them with the inventory. A box without static assignments still
gets the section with a short explanation instead of a generated-table link.
**Structure and layout:**
- Use **tables** for lists of structured items (BGP peers, forwarded ports, vhosts, containers,
VMs, VLANs). Make the item **name the link**; don't add a separate `Page`/`Docs` column.
- Break up **long prose**: any bullet running past ~4 lines over several distinct facts becomes a
`###` subheading (lead sentence + nested bullets). Don't leave walls of long bullets.
- Prefer durable concepts and named configuration identifiers over copying exact numbers that are
tunable, generated or likely to drift. Exact values are appropriate when they are useful parts of
the inventory or interface: VM resource allocations, hardware-fixed properties, network and
protocol identifiers, exposed service ports, and safety or recovery values. Describe mutable
implementation tunables by purpose and link to their source instead of duplicating the current
value.
- **Per-site index pages** (`docs/sites/*/README.md`, `docs/remote/README.md`, and
`docs/mobile/README.md`) use one table listing **boxes only**; don't repeat that inventory in a
diagram. A box's containers / nested VMs are tabulated on that box's own page. The global
`docs/README.md` is the exception: it may show them in high-level site diagrams, but should not
add a second detailed inventory outside those diagrams.
**De-dup by ownership** — each fact has one home:
- Shared cross-box **fabric** (the AS211024 mesh, Tailscale/headscale topology, the BGP overview,
the WireGuard-tunnel summary) is documented once in `docs/networking.md`; box pages give a
one-line summary and link to it.
- Site-wide network definitions (prefixes, VLANs, router VIPs and router HA) also live in
`docs/networking.md`; site pages summarize and link to the canonical section.
- **Box-specific** detail (a box's own BGP peers, the WireGuard tunnels it terminates, its
services) lives on the box page; `networking.md` carries only a per-box summary that links out.
- Addresses represented by generated `assignments` or `extraAssignments` live only in the
assignment tables; handwritten box and workload inventories link there instead of copying them.
Addresses outside that data model (such as external peers or service endpoints) stay with the
topic that owns them.
**Generated content:** the network-assignment tables in `networking.md`, the option reference
(`docs/reference/nixos-options.md`) and the live DNS tables (`docs/reference/dns.md`) are
CI-generated by the corresponding `update-docs-*` packages; the workflow supplies the DNS zones.
Don't hand-edit content between `<!-- ... -->` markers; write the surrounding prose or source
configuration and let the updater refresh the tables. The option-reference file is generated in
full.
**Keep docs current:** when you add, remove or repurpose a box or service, update its box page, the
relevant site-index `README.md`, and any affected prose in `networking.md` (the assignment/option
tables refresh via CI). This is the same "prefer docs over agent memory" rule from the top of this
file.
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+34 -49
View File
@@ -1,13 +1,14 @@
# nixfiles
Personal Nix flake managing every machine I run: hosted servers, home
Personal Nix flake managing every box I run: hosted servers, home
infrastructure, routers, a remote site, VPSes and personal workstations. It is
built around a **custom module system** layered on top of NixOS and
home-manager rather than the stock per-host `nixosConfigurations` pattern.
For day-to-day commands and a deeper explanation of the module system,
conventions and secrets, see [`AGENTS.md`](AGENTS.md). This README is the map of
**what is actually deployed**; the per-machine details live under [`docs/boxes/`](docs/boxes).
For the module-system internals, see [`docs/architecture.md`](docs/architecture.md); for day-to-day
commands, deployment and secrets, see [`docs/deployment.md`](docs/deployment.md). This README is the
map of **what is actually deployed**; the per-box details live under [`docs/`](docs) (start at
[`docs/README.md`](docs/README.md)).
> **Note:** This documentation (the README and everything under `docs/`) is a
> work in progress and was **agent-generated** from the repository. It may be
@@ -15,66 +16,50 @@ conventions and secrets, see [`AGENTS.md`](AGENTS.md). This README is the map of
## The boxes at a glance
Machines are grouped by deployment/location. Each group has its own directory
under `docs/boxes/` with a `README.md` overview and one file per machine.
Boxes are grouped by deployment/location. Each group has its own directory
under `docs/` with a `README.md` overview and one page per box.
| Group | What it is | Docs |
| --- | --- | --- |
| **colony** | Hosted dedicated server in Amsterdam (`ams1`). A VM host running the public-facing infrastructure: routing, web, git, media, object storage, chat, game servers. | [`docs/boxes/colony/`](docs/boxes/colony) |
| **home** | Home network: a VM host (`palace`), the home routers, storage, Home Assistant, and personal desktops. | [`docs/boxes/home/`](docs/boxes/home) |
| **misc** | Everything else: edge VPSes (`britway`, `britnet`), the remote `kelder` site, the `tower` workstation, and the installer image. | [`docs/boxes/misc/`](docs/boxes/misc) |
| Group | What it is |
| --- | --- |
| [**colony**](docs/sites/colony) | Hosted dedicated server in Amsterdam (`ams1`). A VM host running the public-facing infrastructure: routing, web, git, media, object storage, chat, game servers. |
| [**home**](docs/sites/home) | Home network: a VM host (`palace`), the home routers, storage, Home Assistant, and a workstation — plus the hand-configured switches and wireless APs tying it together. |
| [**remote**](docs/remote) | Edge VPSes (`britway`, `britnet`) and the remote `kelder` site. |
| [**mobile**](docs/mobile) | The `tower` laptop. |
## The "big machine" pattern
The custom installer image is documented at [`docs/misc/installer.md`](docs/misc/installer.md).
The general topics — the module system, network and deployment — live next to the index:
[`docs/architecture.md`](docs/architecture.md), [`docs/networking.md`](docs/networking.md) and
[`docs/deployment.md`](docs/deployment.md). The generated custom-module option reference is at
[`docs/reference/nixos-options.md`](docs/reference/nixos-options.md).
The larger sites (`colony`, `home`) all follow the same shape:
```
physical host (VM host)
├── VM ── thing impractical to containerise (router, storage, podman host, …)
├── VM ── container host ──┬── NixOS container ── application
│ ├── NixOS container ──┬── application
│ │ ├── application
│ │ └── application
│ └── …
└── VM ── …
```
- A **physical host** (`colony`, `palace`) does little itself beyond running
**VMs** via the custom `my.vms` module (QEMU + systemd units, LVM-backed
disks).
- **VMs** exist for things that are impractical to put in a container — kernel
features, separate networking, podman/OCI workloads, foreign OSes.
- One VM is usually a **container host** (`shill` on colony, `sfh` on home; and
`kelder` directly). It runs **NixOS containers** via the custom `my.containers`
module (systemd-nspawn based, each with its own address on a bridge).
- **Most applications live in those NixOS containers.** A container isn't limited
to a single application — it commonly hosts a **group of related applications**
that belong together (e.g. `jackflix` runs Jellyfin, the *arr stack,
Transmission, PhotoPrism and copyparty; `object` runs MinIO, Harmonia, HedgeDoc
and wastebin). Each container is a first-class entry in `docs/boxes/`.
Networking between everything is largely defined by per-system `assignments`
(IPs/prefixes) plus an L2 VXLAN mesh (`my.vpns.l2`, AS211024) that ties the edge
routers together. See [`AGENTS.md`](AGENTS.md) for the mechanics.
The high-level site diagrams and box hierarchy live in [`docs/README.md`](docs/README.md).
Networking is largely defined by per-box `assignments` plus the AS211024 L2 VXLAN mesh; see
[`docs/networking.md`](docs/networking.md) for the full picture and
[`docs/architecture.md`](docs/architecture.md) for the implementation mechanics.
## Repo layout
```
README.md <- you are here
nixos/
boxes/ per-machine configuration ("boxes")
boxes/ per-box configuration
colony/ colony host + its VMs (vms/) + shill's containers
home/ palace host + its VMs, plus stream, castle
britway/ britnet.nix, kelder/, tower/, installer.nix …
home/ palace host + its VMs, routing-common, plus stream, castle
britway/ London VPS
kelder/ remote box + its containers
tower/ laptop
britnet.nix Birmingham VPS
installer.nix installer-image configuration
modules/ shared NixOS modules (my.* options); registered in _list.nix
home-manager/ home-manager modules + configs
lib/ lib.my helpers, constants (lib.my.c), net/dns helpers
pkgs/ custom packages (overlays.default)
secrets/ age-encrypted secrets (ragenix)
devshell/ devshell commands (build/deploy/check/ssh helpers)
docs/boxes/ per-machine documentation (colony/, home/, misc/)
ci/ CI helpers (binary-cache push, docs generators)
docs/ deployment documentation (index at docs/README.md)
```
A machine is wired into the flake by adding its box file to the `configs` list
in `flake.nix`. See [`AGENTS.md`](AGENTS.md#architecture) for how `evalModules`
turns these into `nixosConfigurations`, `homeConfigurations` and `deploy` nodes.
A box is wired into the flake by adding its config file to the `configs` list
in `flake.nix`. See [the top-level evaluation](docs/architecture.md#the-top-level-evaluation) for
how `evalModules` turns these into `nixosConfigurations`, `homeConfigurations` and `deploy` nodes.
+194
View File
@@ -0,0 +1,194 @@
#!/usr/bin/env python3
"""Update the consolidated network-assignment tables in docs/networking.md.
Reads nixos.allAssignments from the flake and renders assignment-key subgroups for each site
(colony / home / remote / other) between per-site `<!-- assignments: <site> -->` markers. The
hand-written Notes column is preserved, keyed by (box, assignment).
"""
import json
import re
import subprocess
import sys
from pathlib import Path
DOCS = Path("docs")
TARGET = DOCS / "networking.md"
HEADER = "| Box | IPv4 | IPv6 | Domain | Notes |"
SEP = "|---|---|---|---|---|"
# Table order; also the set of valid `<!-- assignments: <site> -->` tokens.
SITE_ORDER = ["colony", "home", "remote", "other"]
# Per-site internal domains of the remote boxes (britway=lon1, britnet=bhx1, kelder=hentai).
REMOTE_DOMAINS = ("lon1.int.nul.ie", "bhx1.int.nul.ie", "hentai.engineer")
def site_of(assignments: dict) -> str:
"""Classify a box into a site table by its assignment domain.
Prefer the `internal` assignment's domain, else the first assignment that has one.
"""
dom = (assignments.get("internal") or {}).get("domain")
if not dom:
for a in assignments.values():
if a.get("domain"):
dom = a["domain"]
break
if not dom:
return "other"
if dom.endswith("ams1.int.nul.ie"):
return "colony"
if dom == "h.nul.ie" or dom.endswith(".h.nul.ie"):
return "home"
if any(dom.endswith(s) for s in REMOTE_DOMAINS):
return "remote"
return "other"
def fmt_ip(ip: dict) -> str:
addr = ip.get("address")
if addr is None:
return ""
parts = [f"{addr}/{ip.get('mask')}"]
if ip.get("gateway") is not None:
parts.append(f"gw {ip['gateway']}")
return f"`{' '.join(parts)}`"
def find_page(box: str) -> str | None:
"""Doc page for a box (relative to docs/), if one exists."""
for pattern in (f"{box}.md", f"{box}/README.md"):
for p in sorted(DOCS.rglob(pattern)):
return p.relative_to(DOCS).as_posix()
return None
def box_cell(box: str) -> str:
page = find_page(box)
return f"[`{box}`]({page})" if page else f"`{box}`"
def box_name_from_cell(cell: str) -> str:
m = re.match(r"\[`?([^`\]]+)`?\]", cell)
return m.group(1) if m else cell.strip("`")
def parse_notes(lines: list[str]) -> dict[tuple[str, str], str]:
"""Existing Notes indexed by (box, assignment), from either table layout."""
notes: dict[tuple[str, str], str] = {}
assignment = None
for line in lines:
stripped = line.strip()
heading = re.match(r"^####\s+`([^`]+)`$", stripped)
if heading:
assignment = heading.group(1)
continue
if not stripped.startswith("|"):
continue
cells = [c.strip() for c in stripped.split("|")]
# "| a | b | ... |" splits to ['', 'a', 'b', ..., '']
if len(cells) >= 7 and assignment is not None:
# Current layout: one table under each assignment heading.
box, row_assignment, note = cells[1], assignment, cells[5]
elif len(cells) >= 8:
# Previous layout: one site table with an Assignment column.
box, row_assignment, note = cells[1], cells[2], cells[6]
else:
continue
# Skip header and separator rows.
if box == "Box" or set(box) <= {"-"}:
continue
notes[(box_name_from_cell(box), row_assignment)] = note
return notes
def render_site(boxes: list[str], all_assignments: dict, notes: dict) -> list[str]:
groups: dict[str, list[tuple[str, dict]]] = {}
for box in sorted(boxes):
for key, a in all_assignments[box].items():
groups.setdefault(key, []).append((box, a))
lines: list[str] = []
group_order = sorted(groups, key=lambda key: (key != "internal", key))
for key in group_order:
if lines:
lines.append("")
lines.extend([f"#### `{key}`", "", HEADER, SEP])
for box, a in groups[key]:
lines.append(
"| "
+ " | ".join(
[
box_cell(box),
fmt_ip(a.get("ipv4", {})),
fmt_ip(a.get("ipv6", {})),
a.get("domain") or "",
notes.get((box, key), ""),
]
)
+ " |"
)
return lines
def update_target(all_assignments: dict) -> bool:
by_site: dict[str, list[str]] = {s: [] for s in SITE_ORDER}
for box, assignments in all_assignments.items():
if not assignments:
continue
by_site[site_of(assignments)].append(box)
text = TARGET.read_text()
lines = text.splitlines()
marker_re = re.compile(r"^<!--\s*assignments:\s*(\S+)\s*-->$")
# Walk the file line by line; at each `<!-- assignments: <site> -->` marker, replace
# everything up to the matching `<!-- assignments-end -->` with a freshly rendered table
# (carrying the hand-written Notes over), leaving all other lines untouched.
out: list[str] = []
i = 0
while i < len(lines):
m = marker_re.match(lines[i].strip())
if not m:
out.append(lines[i])
i += 1
continue
site = m.group(1)
end = i + 1
while end < len(lines) and lines[end].strip() != "<!-- assignments-end -->":
end += 1
if end >= len(lines):
out.append(lines[i])
i += 1
continue
notes = parse_notes(lines[i + 1 : end])
out.append(lines[i])
out.append("<!-- assignments-start -->")
out.extend(render_site(by_site.get(site, []), all_assignments, notes))
out.append("<!-- assignments-end -->")
i = end + 1
new_text = "\n".join(out) + "\n"
if new_text != text:
TARGET.write_text(new_text)
return True
return False
def main() -> int:
result = subprocess.run(
["nix", "eval", ".#nixfiles.config.nixos.allAssignments", "--json"],
capture_output=True,
text=True,
check=True,
)
all_assignments = json.loads(result.stdout)
if update_target(all_assignments):
print(f"updated {TARGET}")
return 0
if __name__ == "__main__":
sys.exit(main())
+347
View File
@@ -0,0 +1,347 @@
#!/usr/bin/env python3
"""Render docs/reference/dns.md from live authoritative DNS zone transfers.
The authoritative servers are queried directly over AXFR. Records owned by Kea are
identified by DHCID records and omitted together with their forward and reverse data.
"""
import argparse
import ipaddress
import re
import socket
import sys
from dataclasses import dataclass
from pathlib import Path
import dns.exception
import dns.name
import dns.query
import dns.resolver
import dns.rdatatype
OUT = Path("docs/reference/dns.md")
DEFAULT_PORT = 53
IGNORED_TYPES = {"DHCID", "SOA"}
POWERDNS_ALIAS = 65401
POWERDNS_LUA = 65402
@dataclass(frozen=True)
class Record:
zone: str
owner: str
type: str
value: str
def normalize_name(name: str) -> str:
return name.rstrip(".").lower()
def record_data(rdtype: int, rdata) -> tuple[str, str]:
if rdtype == POWERDNS_ALIAS:
target, _used = dns.name.from_wire(rdata.data, 0)
return "ALIAS", target.to_text()
if rdtype == POWERDNS_LUA:
logical_type = int.from_bytes(rdata.data[:2], byteorder="big")
return "LUA", dns.rdatatype.to_text(logical_type)
return dns.rdatatype.to_text(rdtype), rdata.to_text()
def server_addresses(server: str, port: int) -> list[str]:
addresses = []
try:
for result in socket.getaddrinfo(server, port, type=socket.SOCK_STREAM):
address = result[4][0]
if address not in addresses:
addresses.append(address)
except socket.gaierror as error:
raise RuntimeError(f"cannot resolve nameserver {server}: {error}") from error
return addresses
def transfer(server: str, port: int, zone: str) -> list[Record]:
addresses = server_addresses(server, port)
errors = []
for address in addresses:
try:
records = []
for message in dns.query.xfr(
address, zone, port=port, lifetime=60, relativize=False
):
for rrset in message.answer:
for rdata in rrset:
record_type, value = record_data(rrset.rdtype, rdata)
records.append(
Record(
zone=normalize_name(zone),
owner=normalize_name(rrset.name.to_text()),
type=record_type,
value=value,
)
)
if not any(record.type == "SOA" for record in records):
raise RuntimeError("transfer returned no SOA record")
return records
except (dns.exception.DNSException, OSError, RuntimeError) as error:
errors.append(f"{address}: {error}")
raise RuntimeError(f"AXFR of {zone} from {server} failed ({'; '.join(errors)})")
def system_nameservers(domain: str) -> list[str]:
try:
answer = dns.resolver.resolve(domain, "NS", lifetime=30)
except dns.exception.DNSException as error:
message = f"cannot discover authoritative servers for {domain}: {error}"
raise RuntimeError(message) from error
return [rdata.target.to_text() for rdata in answer]
def nameservers_via(server: str, port: int, domain: str) -> list[str]:
errors = []
for address in server_addresses(server, port):
resolver = dns.resolver.Resolver(configure=False)
resolver.nameservers = [address]
resolver.port = port
try:
answer = resolver.resolve(domain, "NS", lifetime=15, search=False)
return [rdata.target.to_text() for rdata in answer]
except dns.exception.DNSException as error:
errors.append(f"{address}: {error}")
raise RuntimeError(f"NS query for {domain} via {server} failed ({'; '.join(errors)})")
def discover_nameservers(domains: list[str], port: int) -> dict[str, list[str]]:
discovered = {}
unresolved = {}
candidates = []
for domain in domains:
try:
servers = system_nameservers(domain)
discovered[domain] = servers
for server in servers:
if server not in candidates:
candidates.append(server)
except RuntimeError as error:
unresolved[domain] = [str(error)]
for domain, errors in list(unresolved.items()):
for server in candidates:
try:
discovered[domain] = nameservers_via(server, port, domain)
del unresolved[domain]
break
except RuntimeError as error:
errors.append(str(error))
if unresolved:
details = "; ".join(
f"{domain}: {'; '.join(errors)}" for domain, errors in unresolved.items()
)
raise RuntimeError(details)
return discovered
def transfer_domain(port: int, domain: str, servers: list[str]) -> list[Record]:
errors = []
for server in servers:
try:
return transfer(server, port, domain)
except RuntimeError as error:
errors.append(str(error))
raise RuntimeError(f"no authoritative server allowed AXFR for {domain} ({'; '.join(errors)})")
def dynamic_names(records: list[Record]) -> set[str]:
return {record.owner for record in records if record.type == "DHCID"}
def dynamic_addresses(records: list[Record], names: set[str]) -> set[str]:
return {
record.value.rstrip(".").lower()
for record in records
if record.owner in names and record.type in {"A", "AAAA"}
}
def reverse_address(owner: str) -> str | None:
if owner.endswith(".in-addr.arpa"):
labels = owner.removesuffix(".in-addr.arpa").split(".")
if len(labels) != 4:
return None
try:
return str(ipaddress.IPv4Address(".".join(reversed(labels))))
except ValueError:
return None
if owner.endswith(".ip6.arpa"):
labels = owner.removesuffix(".ip6.arpa").split(".")
if len(labels) != 32:
return None
try:
value = int("".join(reversed(labels)), 16)
return str(ipaddress.IPv6Address(value))
except ValueError:
return None
return None
def static_records(records: list[Record]) -> list[Record]:
names = dynamic_names(records)
addresses = dynamic_addresses(records, names)
static = []
for record in records:
if record.type in IGNORED_TYPES or record.owner in names:
continue
if record.type == "PTR":
target = normalize_name(record.value.split()[0])
address = reverse_address(record.owner)
if target in names or address in addresses:
continue
static.append(record)
return static
def relative_name(owner: str, zone: str) -> str:
if owner == zone:
return "@"
suffix = f".{zone}"
return owner[: -len(suffix)] if owner.endswith(suffix) else owner
def markdown_code(value: str) -> str:
escaped = value.replace("|", "\\|")
return f"`{escaped}`"
def display_record(record: Record) -> tuple[str, str]:
if record.type != "LUA":
return record.type, record.value
return f"{record.value} (LUA)", "generated at query time"
def render_forward(domain: str, records: list[Record]) -> list[str]:
domain = normalize_name(domain)
rows = []
for record in records:
if record.zone != domain or record.type == "PTR":
continue
record_type, value = display_record(record)
rows.append((relative_name(record.owner, record.zone), record_type, value))
rows.sort(key=lambda row: (row[0] != "@", row[0], row[1], row[2]))
lines = ["| Name | Type | Value |", "|---|---|---|"]
lines.extend(
f"| {markdown_code(name)} | {markdown_code(record_type)} | {markdown_code(value)} |"
for name, record_type, value in rows
)
return lines
def render_reverse(domain: str, records: list[Record]) -> list[str]:
domain = normalize_name(domain)
rows = []
for record in records:
if record.zone != domain or record.type != "PTR":
continue
rows.append((reverse_address(record.owner) or record.owner, record.value))
rows.sort(key=lambda row: ipaddress.ip_address(row[0]))
lines = ["| Address | Name |", "|---|---|"]
lines.extend(
f"| {markdown_code(address)} | {markdown_code(name)} |" for address, name in rows
)
return lines
def is_reverse(domain: str) -> bool:
domain = normalize_name(domain)
return domain.endswith(".in-addr.arpa") or domain.endswith(".ip6.arpa")
def rendered_zones(transferred: list[tuple[str, list[Record]]]) -> dict[str, list[str]]:
records = static_records([record for _domain, zone in transferred for record in zone])
rendered = {}
for domain, _raw_records in transferred:
if is_reverse(domain):
rendered[normalize_name(domain)] = render_reverse(domain, records)
else:
rendered[normalize_name(domain)] = render_forward(domain, records)
return rendered
def update_target(transferred: list[tuple[str, list[Record]]], target: Path) -> bool:
rendered = rendered_zones(transferred)
text = target.read_text()
lines = text.splitlines()
marker_re = re.compile(r"^<!--\s*dns:\s*(\S+)\s*-->$")
found = set()
output = []
i = 0
while i < len(lines):
match = marker_re.match(lines[i].strip())
if not match or normalize_name(match.group(1)) not in rendered:
output.append(lines[i])
i += 1
continue
domain = normalize_name(match.group(1))
end = i + 1
while end < len(lines) and lines[end].strip() != "<!-- dns-end -->":
end += 1
if end >= len(lines):
raise RuntimeError(f"missing <!-- dns-end --> for {domain}")
output.extend(
[
lines[i],
"<!-- dns-start -->",
*rendered[domain],
"<!-- dns-end -->",
]
)
found.add(domain)
i = end + 1
missing = rendered.keys() - found
if missing:
raise RuntimeError(f"missing DNS markers for: {', '.join(sorted(missing))}")
new = "\n".join(output) + "\n"
if new == text:
return False
target.write_text(new)
return True
def main() -> int:
parser = argparse.ArgumentParser(prog="update-docs-dns", description=__doc__)
parser.add_argument("domain", nargs="+", help="DNS zone to transfer")
parser.add_argument("--port", type=int, default=DEFAULT_PORT)
parser.add_argument("--output", type=Path, default=OUT)
args = parser.parse_args()
try:
nameservers = discover_nameservers(args.domain, args.port)
transferred = [
(domain, transfer_domain(args.port, domain, nameservers[domain]))
for domain in args.domain
]
except RuntimeError as error:
print(f"update-docs-dns: {error}", file=sys.stderr)
return 1
try:
changed = update_target(transferred, args.output)
except (OSError, RuntimeError) as error:
print(f"update-docs-dns: {error}", file=sys.stderr)
return 1
if changed:
print(f"updated {args.output}")
return 0
if __name__ == "__main__":
sys.exit(main())
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""Render docs/reference/nixos-options.md from the `my.*` module options.
Builds the `optionsDoc` flake output (a `nixosOptionsDoc` JSON dump of the custom
`my.*` options) and renders it to a per-module table. The prose lives in the option
descriptions in `nixos/modules/`; this file is generated.
"""
import json
import re
import subprocess
import sys
from pathlib import Path
OPTIONS_DOC_ATTR = ".#nixfiles.config.nixos.optionsDoc"
OUT = Path("docs/reference/nixos-options.md")
HEADER = "| Option | Type | Default | Description |"
SEP = "|---|---|---|---|"
def collapse(text: str) -> str:
# Strip the flake's own /nix/store/<hash>-source/ prefix from rendered values so
# defaults like `./.keys/deploy.pub` are stable (the hash changes every time this
# generated file — part of the source tree — is rewritten).
text = re.sub(r"/nix/store/[a-z0-9]{32}-source/", "", text)
text = re.sub(
r'<link\s+xlink:href="([^"]+)">([\s\S]+?)</link>',
r"[\2](\1)",
text,
)
return re.sub(r"\s+", " ", text).strip().replace("|", "\\|")
def render_default(entry: dict) -> str:
if "default" not in entry:
return ""
d = entry["default"]
# A dict default is a `defaultText` / `literalExpression` (pre-rendered Nix in `text`);
# anything else is a literal value we JSON-encode.
if isinstance(d, dict):
text = d.get("text")
if text is None:
text = json.dumps(d.get("value"))
else:
text = json.dumps(d)
text = collapse(text)
return f"`{text}`" if text else ""
def build_options_json() -> dict:
# `optionsDoc` builds to a derivation containing share/doc/nixos/options.json; the last
# printed out-path is the derivation root.
out = subprocess.run(
["nix", "build", "--no-link", "--print-out-paths", OPTIONS_DOC_ATTR],
capture_output=True,
text=True,
check=True,
).stdout.strip().splitlines()[-1]
return json.loads((Path(out) / "share/doc/nixos/options.json").read_text())
def main() -> int:
opts = build_options_json()
by_module: dict[str, list[tuple[str, dict]]] = {}
for key, entry in opts.items():
decls = entry.get("declarations") or []
# Only document options declared in this repo's modules. Some `my.*` options
# (e.g. `my.buildAs`, which aliases `options.system.build`) pull in sub-options
# declared in nixpkgs; their declaration paths don't exist here — skip them.
if not decls or not Path(decls[0]).exists():
continue
by_module.setdefault(decls[0], []).append((key, entry))
lines = [
"# NixOS module options (`my.*`)",
"",
"> **Generated** from the module option descriptions by `nix run .#update-docs-options`;",
"> CI keeps it current. The source of truth is the `mkOpt'` / `mkBoolOpt'` declarations in",
"> `nixos/modules/` — edit those, not this file. For what each module is *for*, see the",
"> [shared-modules overview](../architecture.md#shared-modules).",
"",
]
for decl in sorted(by_module):
mod = Path(decl).stem
if decl != "(unknown)":
lines.append(f"## `{mod}` — [`{decl}`](../../{decl})")
else:
lines.append(f"## `{mod}`")
lines += ["", HEADER, SEP]
for key, entry in sorted(by_module[decl], key=lambda kv: kv[0]):
lines.append(
f"| `{key}` | {collapse(entry.get('type', ''))} | "
f"{render_default(entry)} | {collapse(entry.get('description', ''))} |"
)
lines.append("")
OUT.parent.mkdir(parents=True, exist_ok=True)
new = "\n".join(lines).rstrip() + "\n"
old = OUT.read_text() if OUT.exists() else ""
if new != old:
OUT.write_text(new)
print(f"updated {OUT}")
return 0
if __name__ == "__main__":
sys.exit(main())
+92
View File
@@ -0,0 +1,92 @@
# Deployment documentation
> **Note:** these pages are a work in progress and were **agent-generated** from the repository.
> They may be incomplete or out of date — treat the Nix configuration as the source of truth.
This directory documents the boxes managed by this flake: their roles, network assignments,
hierarchy, and the services they run. For the mechanics of the repo itself (conventions, module
system internals for contributors, agent guidance), see [`AGENTS.md`](../AGENTS.md).
The two big sites follow the pattern:
```
physical host (VM host)
└── VM (for things impractical to containerise)
└── container host VM
└── NixOS containers (one per application group)
```
Not every box fits this pattern, but **colony** and **home** are organised this way.
## General
- [`architecture.md`](architecture.md) — the custom module system, `my.*` namespace, multiple
nixpkgs channels, shared module inventory.
- [`networking.md`](networking.md) — network assignments, domains, site topologies, router HA,
the AS211024 L2 mesh, BGP, WireGuard, Tailscale.
- [`deployment.md`](deployment.md) — deploy-rs, devshell commands, secrets workflow, CI.
- [`reference/dns.md`](reference/dns.md) — generated forward and reverse DNS record reference.
- [`reference/nixos-options.md`](reference/nixos-options.md) — generated per-option reference for
the custom `my.*` NixOS modules.
## Site: colony (Amsterdam)
Physical host and public-infrastructure hub — see [`sites/colony/README.md`](sites/colony/README.md).
```
colony (physical VM host, ams1)
├── estuary ── edge router: WAN, firewall/NAT, DNS, BGP (AS211024), WireGuard
├── shill ──── NixOS container host ──┬── middleman (reverse proxy, ACME, nginx-sso, librespeed)
│ ├── vaultwarden (password manager)
│ ├── colony-psql (shared PostgreSQL)
│ ├── chatterbox (Matrix Synapse + bridges)
│ ├── jackflix (media stack)
│ ├── object (MinIO, Harmonia Nix cache, Sharry, HedgeDoc, wastebin)
│ ├── toot (Bluesky PDS; Mastodon disabled)
│ ├── waffletail (Tailscale subnet router / exit node)
│ ├── qclk (WireGuard management appliance)
│ ├── gam (Terraria server)
│ └── jam (raw nspawn customer container)
├── whale2 ─── podman/OCI host for game servers
├── git ────── Gitea + Gitea Actions runner
├── mail ───── Debian VM running mailcow (not NixOS)
└── darts ──── third-party/customer VM (opaque, not NixOS)
```
## Site: home
Redundant routers, VM host, storage, IoT containers and the workstation — see
[`sites/home/README.md`](sites/home/README.md). The hand-configured switch fabric (jim/dave/brian)
and the Digiweb WAN path are documented in [`sites/home/switches.md`](sites/home/switches.md).
```
h.nul.ie
├── palace (physical VM host — AMD, 100G, SR-IOV)
│ ├── river ── primary router VM (PPPoE / Digiweb WAN)
│ ├── cellar ─ NVMe-oF / SPDK storage target VM
│ └── sfh ──── container host VM ("services for home")
│ ├── hass ── Home Assistant + Frigate + MQTT (container)
│ └── unifi ─ UniFi controller (container)
├── stream (physical secondary router — Virgin Media WAN)
└── castle (workstation / gaming desktop — netboot, NVMe-oF root)
```
## Remote boxes
The edge VPSes and remote `kelder` site are indexed in [`remote/README.md`](remote/README.md).
## Mobile boxes
The laptop is indexed in [`mobile/README.md`](mobile/README.md).
## Misc
- [`misc/installer.md`](misc/installer.md) — the custom NixOS installer image.
## A note on the assignment tables
The consolidated [`Box assignments`](networking.md#box-assignments) tables in
[`networking.md`](networking.md) (one per site, between `<!-- assignments: <site> -->` markers)
are **generated from the flake** (`nixos.allAssignments`) by `nix run .#update-docs-assignments`
CI refreshes them on push. Individual box pages link to that section rather than carrying their
own table. Only the Notes column is hand-written; don't hand-edit the other cells.
+266
View File
@@ -0,0 +1,266 @@
# Architecture
This flake does **not** use the stock pattern of calling `nixosSystem` once per host in
[`flake.nix`](../flake.nix). Instead it runs a single `lib.evalModules` evaluation over its own
module tree, producing one big top-level config (`self.nixfiles`) from which the real flake
outputs (`nixosConfigurations`, `homeConfigurations`, `nixosModules`, `deploy`, …) are derived.
Per-host files declare *options* (`nixos.systems.<name>`); the machinery in
[`nixos/default.nix`](../nixos/default.nix) and
[`home-manager/default.nix`](../home-manager/default.nix) turns those into evaluated NixOS /
home-manager configurations.
## The top-level evaluation
`flake.nix`'s `outputs` builds a `nixfiles` attrset via `evalModules` over:
- An inline module that seeds `_module.args` (`lib`, `pkgsFlakes`, `hmFlakes`, `self`, `inputs`,
`pkgs'`), sets `nixos.secretsPath = ./secrets`, and sets the global deploy-rs SSH option
`deploy-rs.deploy.sshOpts = [ "-i" ".keys/deploy.key" ]`.
- `nixos/modules/misc/assertions.nix` from the unstable nixpkgs (so the top-level evaluation has
the standard `assertions` / `warnings` options).
- [`nixos/`](../nixos/default.nix) — defines `nixos.*` options (`systems`, `modules`,
`allAssignments`, `vpns`, `secretsPath`) and `mkSystem`.
- [`home-manager/`](../home-manager/default.nix) — defines `home-manager.*` options (`homes`,
`modules`) and `mkHome`.
- [`deploy-rs.nix`](../deploy-rs.nix) — defines `deploy-rs.*` and renders the deploy config.
- Every file in the `configs` list — the boxes themselves
([`nixos/boxes/`](../nixos/boxes) plus [`nixos/installer.nix`](../nixos/installer.nix); the
home-manager entries are currently commented out, see [Home-manager](#home-manager)).
The resulting `nixfiles.config` is mapped onto flake outputs:
| Top-level option (`nixfiles.config`) | Flake output |
|---|---|
| `nixos.systems.<name>.rendered` | `nixosConfigurations.<name>` |
| `home-manager.homes.<name>.configuration` | `homeConfigurations.<name>` |
| `nixos.modules` | `nixosModules` |
| `home-manager.modules` | `homeModules` |
| `deploy-rs.rendered` | `deploy` |
`nixfiles` itself is also a flake output, so anything in the top-level config can be addressed
directly — e.g. `build-iso` builds
`.#nixfiles.config.nixos.systems."<host>".configuration.config.my.buildAs.iso`. The flake also
exposes `lib` (the extended unstable nixpkgs lib, including `lib.my`), `inputs`, `nixpkgs`
(the `pkgs'` channel sets), and `overlays.default` (the custom packages from
[`pkgs/`](../pkgs/default.nix)).
Per platform (`eachDefaultSystem`), the flake produces:
- `packages` — everything from `pkgs/`, flattened.
- `checks` — every `homeConfigurations.*.activationPackage` plus deploy-rs's own `deployChecks`.
- `devShells.default` — the `numtide/devshell` from [`devshell/`](../devshell) (see
[deployment.md](deployment.md)).
- `ci.<system>` — one attr per buildable thing (`system-<name>`, `home-<name>` with `@` mangled
to `-at-`, `package-<name>`, plus `shell`), consumed by CI; `ciDrv` is a `linkFarm` of all of
them.
## Systems: `systemOpts` and `mkSystem`
Each entry of `nixos.systems` is a submodule defined by `systemOpts` in
[`nixos/default.nix`](../nixos/default.nix):
| Option | Type / default | Meaning |
|---|---|---|
| `system` | enum of `defaultSystems` | Nix platform string, e.g. `"x86_64-linux"`. |
| `nixpkgs` | one of `unstable`/`stable`/`mine`/`mine-stable`, default `"unstable"` | nixpkgs channel for the system. |
| `home-manager` | same enum, defaults to `nixpkgs` | home-manager channel. |
| `hmNixpkgs` | same enum, defaults to `nixpkgs` | nixpkgs channel used for home-manager's own pkgs when it doesn't share the system's. |
| `docCustom` | bool, default `false` | Include nixfiles' custom modules in the generated NixOS manual (slow). |
| `assignments` | attrsOf `assignmentOpts` | The box's network assignments (see [networking.md](networking.md)). |
| `extraAssignments` | attrsOf attrsOf `assignmentOpts` | Extra assignments for things not on the box itself (e.g. the routers' floating VIP entries). |
| `configuration` | custom merged type | The actual NixOS configuration module(s); merging runs `mkSystem`. |
| `rendered` | unspecified, default `configuration` | What ends up in `nixosConfigurations.<name>` — overridden for boxes built as something other than a plain system (e.g. `installer` renders `config.my.asISO`, containers render `config.my.asContainer`). |
`mkSystem` does the real work:
1. It selects the channel's nixpkgs flake (`pkgsFlakes.${config'.nixpkgs}`).
- It imports `nixos/lib/eval-config.nix` **by hand** because the flake force-sets `lib`;
otherwise `eval-config.nix` would import its own unextended copy.
- The supplied library is the channel's `pkgs.lib` extended with `lib.my` and a
`versionOverlay` that stamps `system.nixos` with the flake revision and channel.
2. `specialArgs` gives every module access to `self`, `inputs`, `pkgsFlakes`, the box's own
`pkgsFlake`, `allAssignments` (every box's assignments), and `systems` (all of
`nixos.systems`). Passing these via `specialArgs` (rather than module `imports`) avoids
infinite recursion.
3. The module list is: the home-manager channel's `nixosModules.default`, every module in
`nixos.modules` (the shared modules, see below), an inline module, and the box's own
`configuration` definitions (wrapped with `inlineModule'` so error messages keep file
provenance).
4. The inline module wires the plumbing:
- `_module.args`: `secretsPath`, `vpns` (the flake-level `nixos.vpns`), the box's own
`assignments`, and `pkgs'` — an attrset of **all four nixpkgs channels** for this box's
platform, so a module can grab a package from another channel (e.g. `pkgs'.stable.foo`).
- `system.name` is the attribute name of the box.
- `networking.hostName` / `networking.domain` default from the `internal` assignment (see
[networking.md](networking.md)).
- `nixpkgs.system` is set and the overlays computed at flake level (lib overlay + custom
packages) are passed through, so `pkgs` is imported modularly with config and overlays
applied.
- `home-manager.useGlobalPkgs` defaults to true when the system and home-manager channels
match; a warning is emitted when they deliberately differ.
- `home-manager.sharedModules` includes every module in `home-manager.modules` plus an inline
module. The inline module passes `pkgsPath` / `pkgs'`, disables the release check, and pins
`home.stateVersion` through `homeStateVersion` (`22.11` for stable-flavoured channels,
`23.05` otherwise).
5. Finally `applyAssertions` throws with all failed assertion messages (and shows warnings)
when the merged configuration is forced.
## The four nixpkgs channels
`flake.nix` builds `pkgsFlakes` (nixpkgs) and `hmFlakes` (home-manager):
| Channel | nixpkgs input | home-manager input |
|---|---|---|
| `unstable` | `nixpkgs/nixos-unstable` | `home-manager` (master) |
| `stable` | `nixpkgs/nixos-26.05` | `home-manager/release-26.05` |
| `mine` | `github:devplayer0/nixpkgs/devplayer0` (personal fork) | alias of `unstable` (no fork exists) |
| `mine-stable` | `github:devplayer0/nixpkgs/devplayer0-stable` | alias of `stable` |
Each channel's `lib` is extended with the `libOverlay` (`lib.my` + flake-utils). Two package
sets are built per channel:
- `pkgs'` — with the devshell/ragenix/deploy-rs/home-manager overlays; used for the dev shell
and `packages`, and exposed as the flake's `nixpkgs` output.
- `configPkgs'` — with just the lib + custom-packages overlays and an `allowUnfreePredicate`
(Widevine / Chromium); this is the `pkgs'` threaded into the top-level evaluation, from
which each box's `pkgs` and per-channel `pkgs'` module args (and home-manager's base
`pkgs`) are taken.
A box picks its channel with `nixpkgs = "mine"` (most boxes), `"mine-stable"` (e.g. `colony`),
etc. Inside a module, `pkgs` is the selected channel and `pkgs'` is the attrset of all four
(`pkgs'.<channel>.<attr>`).
## `lib.my` and the `my.*` namespace
[`lib/default.nix`](../lib/default.nix) extends nixpkgs `lib` with a `my` attrset. The
flake-level `lib` (`pkgsFlakes.unstable.lib` so extended) is for platform-independent flake
use only; each box gets its own channel's lib extended the same way. Contents:
- **Option helpers** — `mkOpt'`, `mkBoolOpt'`, `nullOrOpt'`, `mkDefault'` (slightly stronger than
`mkDefault`), `mkVMOverride'`, `inlineModule'` / `inlineModule`
(attach `_file` provenance), `commonOpts` (the shared `system`/`nixpkgs`/`home-manager`
options and the `moduleType` used for exported modules), `applyAssertions`, `duplicates`.
- **`lib.my.net`** — CIDR/IP math (`net.cidr.host`, `net.cidr.subnet`, `net.types.ipv4`, …)
from the `libnetRepo` input (`oddlama/nixos-extra-modules`' `netu.nix`). Used for nearly all
address arithmetic; addresses are almost never written literally.
- **`lib.my.c`** — shared constants from [`lib/constants.nix`](../lib/constants.nix): static
UID/GID assignments, kernel package selection (`kernel.lts`/`kernel.latest`), nginx config
snippets, networkd snippets (`networkd.noL3`), the binary-cache settings (`nix.cache`),
and the per-site domains/prefixes/VIPs described in [networking.md](networking.md).
- **`lib.my.dns`** — zone-generation helpers from [`lib/dns.nix`](../lib/dns.nix): `fwdRecords`
/ `ptrRecords` / `ptr6Records` rendered from `allAssignments`, plus the LUA-record helpers
(`ifaceA`, `lookupIP`) used by the home routers.
- **networkd helpers** — `networkdAssignment` (assignment → `systemd.network` network, see
[networking.md](networking.md)), `mkVLAN`, `dockerNetAssignment`.
### Misc
The remaining helpers are `mkDefaultSystemsPkgs`, `flakePackageOverlay` (wrap another flake's
package as an overlay), `isIPv6` / `parseIPPort` / `netBroadcast`, `nft` chain-name helpers,
`systemdAwaitPostgres`, `vm.*` LVM disk descriptors for the `vms` module, the typed `deploy-rs`
option definitions (`lib.my.deploy-rs`), `netbootKeaClientClasses` (iPXE/EFI DHCP client classes),
and `homeStateVersion`.
Custom NixOS / home-manager modules declare their options under the `my.*` namespace (the root
`options.my` is declared in `nixos/modules/common.nix` / `home-manager/modules/common.nix`) —
e.g. `my.secrets`, `my.build`, `my.tmproot`, `my.firewall`, `my.server`, `my.deploy`,
`my.vms`, `my.containers`.
## Shared modules
Registered in [`nixos/modules/_list.nix`](../nixos/modules/_list.nix) and applied to **every**
system (box files opt in per-feature via `my.*` options). The table below is the overview — what
each module is *for*; for the exhaustive, CI-generated per-option reference (types, defaults,
descriptions) see [`reference/nixos-options.md`](reference/nixos-options.md).
| Module | Provides |
|---|---|
| `common` | Baseline for all boxes: imports the impermanence, ragenix (age), sharry, copyparty and harmonia NixOS modules; pins `system.stateVersion`; `doas` instead of `sudo`; immutable users; nix settings (flakes, `ca-derivations`, the `nix-cache.nul.ie` substituter); declares the `my` option root. |
| `user` | `my.user` — the primary user: `users.users` + matching `home-manager.users` entry, wheel/doas, SSH authorized key from `.keys/me.pub`, shell taken from the home config, home persistence under tmproot. |
| `build` | `my.build` — alternate build targets via `extendModules`: `my.buildAs.devVM` (QEMU dev VM), `iso`, `container`, `kexecTree`, `netbootTree`/`netbootArchive`; `my.build.isDevVM` marker; `allHardware` profile toggle. |
| `dynamic-motd` | `my.dynamic-motd` — runs a script via `pam_exec` to generate the MOTD on login/ssh. |
| `tmproot` | `my.tmproot` — tmpfs `/` plus impermanence persistence (`persistence.dir`), with a `tmproot-unsaved` helper that walks the root tmpfs and lists files not covered by persistence. |
| `firewall` | `my.firewall` — nftables firewall: `tcp.allowed`/`udp.allowed` port lists, `trustedInterfaces`, NAT with `forwardPorts`, `extraRules` escape hatch; sane ICMP/ICMPv6/PMTUD rules baked in. |
| `server` | `my.server.enable` — server commonalities: getty autologin, LLMNR off, tightened fstrim timers, GUI and NixOS documentation off. |
| `deploy-rs` | `my.deploy` — per-box deploy-rs node/profile generation and the `deploy` user; see [deployment.md](deployment.md). |
| `secrets` | `my.secrets` — ragenix/agenix wiring: `files` to decrypt from `secrets/`, host `key` to encrypt for, identity paths derived from the OpenSSH host keys (persistence-aware), dev-key identity inside dev VMs. |
| `containers` | `my.containers.instances``systemd-nspawn` NixOS containers whose systems come from `nixos.systems` rendered `asContainer`, with bridge/macvlan networking, bind mounts, `hotReload` (reload instead of reboot) and a placeholder "dummy" init before first deploy. |
| `vms` | `my.vms.instances` — QEMU/KVM VMs as systemd units: TAP/bridge networking, LVM-backed disks, VFIO host-device passthrough (with udev tagging), UEFI, SPICE/TTY/QMP unix sockets under `/run/vms/`, clean shutdown via QMP `system_powerdown`. |
| `network` | Baseline networking: networkd only (`useDHCP = false`), IPv6 on, resolved domain/negative-cache settings; dev VMs get DHCP on `eth0` and an SSH port forward. |
| `pdns` | `my.pdns` — PowerDNS: authoritative server driven by BIND-style zone files (with templating and serial management) and recursor extra-settings wiring. |
| `nginx-sso` | `my.nginx-sso` — runs `nginx-sso` (instead of the stock NixOS module) and generates per-instance nginx `auth_request` include files. |
| `gui` | `my.gui.enable` (default on) — desktop baseline: graphics, polkit, swaylock PAM entry, Android udev rules, screenshot tmpdir. |
| `l2mesh` | Consumes `nixos.vpns.l2` — builds the VXLAN + IPsec layer-2 meshes between edge routers; see [networking.md](networking.md). |
| `borgthin` | `my.borgthin.jobs` — borg backups of thin-LVM snapshots to local or SSH repos on a systemd timer, with pruning. |
| `nvme` | `my.nvme` — sets the NVMe host NQN/hostid and, optionally, NVMe-oF-over-RDMA boot in the initrd. |
| `spdk` | `my.spdk` — SPDK target configuration (JSON RPC-driven) plus `spdk-rpc`/`spdk-setup`/`spdk-debug` helper tools. |
| `librespeed` | `my.librespeed` — a LibreSpeed speedtest: generated frontend (from the `librespeed-go` package in `pkgs/`) plus backend settings. |
| `netboot` | `my.netboot` — iPXE netboot server (TFTP/HTTP, menu) and the client-side loader that installs boot entries. |
Home-manager modules, registered in
[`home-manager/modules/_list.nix`](../home-manager/modules/_list.nix) and applied to every
home (including the per-user homes attached to systems via `my.user.homeConfig`):
| Module | Provides |
|---|---|
| `common` | Home baseline: `my.shell`, `my.ssh.authKeys`, `my.isStandalone` (home-manager running standalone vs inside NixOS), fish setup and completions generation, common programs. |
| `gui` | `my.gui` — the sway-based desktop: waybar, notifications, lock/saver plumbing (including the "doomsaver"), terminal and fonts. |
| `deploy-rs` | `my.deploy` for standalone homes — generates a `home` profile (home-manager activation) and deploy node; auto-disabled for NixOS-attached homes. |
| `swaync` | `my.swaync` — typed configuration module for Sway Notification Center. |
## Adding a box
1. Create a file (or directory with a `default.nix`) under [`nixos/boxes/`](../nixos/boxes)
that sets `nixos.systems.<name> = { system = "x86_64-linux"; nixpkgs = "mine"; assignments =
{ … }; configuration = { … }: { … }; };`. Nested boxes (VMs, containers) live under their
host's directory (e.g. `nixos/boxes/colony/vms/estuary/`).
2. Add the path to the `configs` list in [`flake.nix`](../flake.nix).
3. Give the box an `internal` assignment (name/domain) if it gets one — `hostName`/`domain`
default from it — and declare `my.secrets.key` if it has secrets.
4. Evaluate it with `check-system <name>` (cheap) before `build-system <name>`.
## Adding a shared module
1. Drop the file in [`nixos/modules/`](../nixos/modules) (or
[`home-manager/modules/`](../home-manager/modules)) with options under `my.*`, declared via
the `lib.my` helpers (`mkOpt'`, `mkBoolOpt'`).
2. Register it in the corresponding `_list.nix` (name → path). It is then applied to every
box and exported as `nixosModules.<name>` / `homeModules.<name>`.
## Home-manager
Home-manager follows the same option-definition and evaluation split as NixOS.
### `homeOpts`
[`home-manager/default.nix`](../home-manager/default.nix) defines each entry with these options:
| Option | Meaning |
|---|---|
| `system` | Target platform. |
| `nixpkgs` | nixpkgs channel used for the home packages. |
| `home-manager` | home-manager channel. |
| `homeDirectory` / `username` | User identity and home path. |
| `configuration` | The home-manager module definitions evaluated by `mkHome`. |
### `mkHome`
1. Calls the selected channel's `lib.homeManagerConfiguration`.
2. Supplies that channel's `pkgs'` with `config` emptied, allowing home-manager to apply package
configuration and overlays through its module system.
3. Passes `inputs`, `pkgsFlakes` and the selected `pkgsFlake` through `extraSpecialArgs`.
4. Loads every module in `home-manager.modules` plus an inline module that sets
`home.homeDirectory` / `home.username` and exposes every channel through the `pkgs'` module
argument.
5. Pins `home.stateVersion` through `homeStateVersion`.
### Standalone homes
Standalone homes live in [`home-manager/configs/`](../home-manager/configs) and are named
`<user>@<host>` (deploy-rs mangles the `@` to `-at-`). `macsimum` is a home-manager-only
config — an `x86_64-darwin` macOS box with no NixOS side.
No standalone homes are currently wired into the flake: `home-manager/configs/macsimum.nix` is
commented out of `configs`, while `home-manager/configs/castle.nix` is not listed. Consequently,
`nixfiles.config.home-manager.homes` evaluates to `{}`. Most user configuration instead accompanies
NixOS boxes through `my.user.homeConfig`.
-68
View File
@@ -1,68 +0,0 @@
# colony
The hosted dedicated server in Amsterdam (`ams1`). This is the public-facing
half of the boxes: almost everything reachable from the internet lives here.
- **Internal domain:** `ams1.int.nul.ie`
- **Public domain:** `nul.ie` (public services are published as `*.nul.ie`)
- **Source:** [`nixos/boxes/colony/`](../../../nixos/boxes/colony)
## Shape
`colony` is the physical VM host. It runs the VMs below; one of them (`shill`)
is itself a NixOS container host where most applications run.
```
colony (physical VM host)
├── estuary ── edge router / firewall / DNS / BGP
├── shill ──── NixOS container host ──┬── middleman (reverse proxy, ACME, SSO)
│ ├── colony-psql (shared PostgreSQL)
│ ├── vaultwarden (password manager)
│ ├── chatterbox (Matrix + bridges)
│ ├── toot (Mastodon)
│ ├── jackflix (media stack)
│ ├── object (MinIO, Nix cache, …)
│ ├── waffletail (Tailscale subnet router)
│ ├── qclk (clock service)
│ └── gam (game servers)
├── whale2 ─── podman/OCI host (game servers)
├── git ────── Gitea + Actions runner
├── mail ───── Debian VM running Mailcow (not NixOS — configured out of repo)
└── darts ──── third-party/customer VM (opaque to this repo)
```
## Machines
| Machine | Role | Docs |
| --- | --- | --- |
| `colony` | Physical VM host (AMD, LVM-thin, borgthin backups → rsync.net) | [colony.md](colony.md) |
| `estuary` | Edge router: WAN, firewall/NAT, DNS, BGP, IXP peering, WireGuard | [estuary.md](estuary.md) |
| `shill` | NixOS container host (see containers below) | [shill.md](shill.md) |
| `whale2` | podman/OCI host for game servers | [whale2.md](whale2.md) |
| `git` | Gitea + Gitea Actions runner | [git.md](git.md) |
### shill containers
| Container | Role | Docs |
| --- | --- | --- |
| `middleman` | Front-end nginx reverse proxy, ACME certs, nginx-sso, librespeed | [middleman.md](middleman.md) |
| `colony-psql` | Shared PostgreSQL (14) for colony services | [colony-psql.md](colony-psql.md) |
| `vaultwarden` | Vaultwarden (Bitwarden-compatible password manager) | [vaultwarden.md](vaultwarden.md) |
| `chatterbox` | Matrix homeserver + bridges (heisenbridge, mautrix-*) | [chatterbox.md](chatterbox.md) |
| `toot` | Bluesky PDS (Mastodon disabled) | [toot.md](toot.md) |
| `jackflix` | Media: Jellyfin, *arr stack, Transmission, PhotoPrism, copyparty | [jackflix.md](jackflix.md) |
| `object` | MinIO (S3), Harmonia (Nix cache), HedgeDoc, wastebin | [object.md](object.md) |
| `waffletail` | Tailscale subnet router (advertises colony prefixes into the tailnet) | [waffletail.md](waffletail.md) |
| `qclk` | `qclk` clock service (reachable over WireGuard) | [qclk.md](qclk.md) |
| `gam` | Game servers (Terraria, …) | [gam.md](gam.md) |
## Non-NixOS VMs
These run on `colony` but are **not** managed by this repo (no NixOS config). The
QEMU instances are still declared in `colony`'s `my.vms.instances`, and colony's
networking routes/firewalls traffic to them:
- **`mail`** — a Debian VM running [Mailcow](https://mailcow.email/) (`mail.nul.ie`).
ACME certs are pushed to it from `middleman` (see [middleman.md](middleman.md)).
- **`darts`** — a third-party/customer VM; opaque to this repo, given a routed
prefix and otherwise left alone.
-19
View File
@@ -1,19 +0,0 @@
# chatterbox
The Matrix homeserver (`nul.ie`) and its chat-network bridges.
- **Source:** [`shill/containers/chatterbox.nix`](../../../nixos/boxes/colony/vms/shill/containers/chatterbox.nix)
- **Host:** NixOS container on `shill`
## Role
- **Matrix homeserver** for `server_name = "nul.ie"`.
- **Bridges** to other chat networks:
- `heisenbridge` (IRC),
- `mautrix-whatsapp` (WhatsApp),
- `mautrix-meta` / `mautrix-messenger` (Facebook Messenger / Instagram).
- Fronted by `middleman` (federation on `:8448`).
## Networking
- `internal` assignment on the `ctrs` network (alt name `chatterbox-ctr`).
-18
View File
@@ -1,18 +0,0 @@
# colony-psql
The shared PostgreSQL instance for colony. Several other containers
(`middleman`, `chatterbox`, `toot`, `git`, …) connect here rather than each
running their own database.
- **Source:** [`shill/containers/colony-psql.nix`](../../../nixos/boxes/colony/vms/shill/containers/colony-psql.nix)
- **Host:** NixOS container on `shill`
## Role
- **PostgreSQL 14** serving the other colony services over the `ctrs` network.
Consumers wait for it to be ready via the `systemdAwaitPostgres` helper.
- netdata for monitoring.
## Networking
- `internal` assignment on the `ctrs` network (alt name `colony-psql-ctr`).
-39
View File
@@ -1,39 +0,0 @@
# colony (host)
The physical dedicated server in Amsterdam and the VM host for everything in
this group.
- **Source:** [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
(VM instances in [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix))
- **nixpkgs:** `mine-stable`
## Role
Bare-metal AMD host. It does little application work itself — its job is to run
the VMs and provide them with storage, networking and backups.
- **Virtualisation:** QEMU/KVM (`kvm-amd`, IOMMU on) via the `my.vms` module. VM
disks are LVM logical volumes (`vm-<name>-<disk>`) in the `main` volume group;
`estuary` additionally gets a WAN NIC by PCI passthrough.
- **Storage:** LVM-thin (`services.lvm.boot.thin`), `/persist` for state,
`/mnt/backup` for the local borg repo. `smartd` + `rasdaemon` for health.
- **Backups:** `my.borgthin` snapshots the persist/data LVs of the host and its
VMs into `/mnt/backup/main`, which is then `rsync`'d (along with LVM metadata)
to rsync.net (`zh2855.rsync.net`).
- **Monitoring:** netdata (with freeipmi), smartd.
## Networking
- Two bridges: `base` (the colony "base" network, shared with `estuary`) and
`vms` (the VM network). Dummy interfaces keep the bridges up so dependent VMs
can start.
- Default gateway / edge is `estuary`; `colony` itself holds the `routing` and
`internal` (a.k.a. `vm`) assignments and routes container/OCI/Tailscale
prefixes to `shill` and `whale2`.
- `my.firewall` trusts the `vms` interface and forwards customer prefixes
(`vm-mail`, `vm-darts`) through.
## VMs hosted here
`estuary`, `shill`, `whale2`, `git` (all NixOS, documented in this directory),
plus the non-NixOS `mail` and `darts` (see [README](README.md#non-nixos-vms)).
-41
View File
@@ -1,41 +0,0 @@
# estuary
The colony edge router and firewall — the machine that holds colony's public
IPs and connects everything else to the internet.
- **Source:** [`nixos/boxes/colony/vms/estuary/`](../../../nixos/boxes/colony/vms/estuary)
(`default.nix`, `bgp.nix`, `dns.nix`, `bandwidth.nix`)
- **nixpkgs:** `mine`
- **Host:** VM on `colony` (gets the WAN NIC by PCI passthrough)
## Role
- **Edge routing / firewall / NAT:** owns the colony public IPv4/IPv6
(`94.142.241.x` / `2a02:898:0:20::`), does NAT and port-forwarding for the
internal services (`my.firewall.nat.forwardPorts` driven by
`firewallForwards`). Forwards HTTP/S to `middleman`, git to `git`, game ports
to the OCI game servers on `whale2`, etc.
- **BGP:** runs BIRD2 ([`bgp.nix`](../../../nixos/boxes/colony/vms/estuary/bgp.nix))
announcing AS211024, over VLANs on the WAN link:
- peers at the IXPs **Frys-IX**, **NL-ix** and **FogIXP**;
- plus **iFog transit** (`ifog-transit`) — an upstream transit provider from
iFog, **not** an IXP.
- **DNS:** authoritative/recursive DNS ([`dns.nix`](../../../nixos/boxes/colony/vms/estuary/dns.nix)),
redirected to port 5353 locally.
- **VPNs:**
- Part of the AS211024 **L2 VXLAN mesh** (`my.vpns.l2`) with `river`, `stream`
and `britway`.
- WireGuard endpoints for the remote `kelder` site, `hillcrest`, and
`john-valorant`.
- **Misc:** iperf3 server. (A bandwidth-accounting script,
[`bandwidth.py`](../../../nixos/boxes/colony/vms/estuary/bandwidth.py), exists but
is **legacy and not currently used**.)
## Networking
- `wan` — the passed-through igb NIC (9000 MTU), carrying the upstream uplink and
tagged IXP VLANs (`ifog` 409 → `frys-ix`/`nl-ix`/`fogixp`/`ifog-transit`).
- `base` — colony base network; sends RAs and provides DNS to the base prefix,
routes the VM/container/OCI/Tailscale prefixes back to `colony`.
- `as211024` — the L2 mesh interface.
- Assignments: `internal` (public, alt name `fw`), `base`, `as211024`.
-17
View File
@@ -1,17 +0,0 @@
# gam
A game-server container (the lightweight counterpart to the OCI game servers on
`whale2`).
- **Source:** [`shill/containers/gam.nix`](../../../nixos/boxes/colony/vms/shill/containers/gam.nix)
- **Host:** NixOS container on `shill`
## Role
- Hosts game servers run directly as NixOS services — currently **Terraria**
(config/world from secrets). Exposed to the internet via `estuary`'s port
forwards (`:7777`).
## Networking
- `internal` assignment on the `ctrs` network (alt name `gam-ctr`).
-38
View File
@@ -1,38 +0,0 @@
# git
The Gitea VM — source hosting and CI for the boxes (`git.nul.ie`).
- **Source:** [`nixos/boxes/colony/vms/git/`](../../../nixos/boxes/colony/vms/git)
(`default.nix`, `gitea.nix`, `gitea-actions.nix`)
- **nixpkgs:** `mine`
- **Host:** VM on `colony`
## Role
- **Gitea** ([`gitea.nix`](../../../nixos/boxes/colony/vms/git/gitea.nix)) — the Git
forge (`git.nul.ie`). PostgreSQL-backed (the shared `colony-psql`), LFS
enabled, with object storage backed by MinIO on `object` (a MinIO secret is
spliced into `app.ini` at startup).
- **Gitea Actions runner**
([`gitea-actions.nix`](../../../nixos/boxes/colony/vms/git/gitea-actions.nix)) — a
Docker-mode runner (`main-docker`) using podman. Labels provide Debian/node-24
(Trixie) and Ubuntu 26.04 images; runner config comes from the upstream
module's `settings` option. The Actions cache lives on a dedicated disk
(`/var/cache/gitea-runner`). Runs as a fixed `gitea-runner` user (not
`DynamicUser`) so it can read its token.
- **nginx** — terminates TLS for `git.nul.ie` and proxies to Gitea on `:3000`.
ACME certs for `nul.ie` / `*.nul.ie` via the Cloudflare DNS challenge.
- **podman** — also hosts the OCI registry/build images; `/var/lib/containers`
is an XFS data disk.
## Networking
- `vms` interface with `routing` / `internal` assignments.
- HTTP/HTTPS forwarded in from `estuary`; podman default subnet `10.88.0.0/16` is
allowed to forward.
## CI
This runner is what executes the repo's own `.gitea/workflows/ci.yaml`, building
each `.#ci.x86_64-linux` attribute and pushing to the Harmonia binary cache. See
[`AGENTS.md`](../../../AGENTS.md#commands).
-25
View File
@@ -1,25 +0,0 @@
# jackflix
The media stack — acquisition, library and streaming.
- **Source:** [`shill/containers/jackflix/`](../../../nixos/boxes/colony/vms/shill/containers/jackflix)
(`default.nix`, `networking.nix`)
- **Host:** NixOS container on `shill`
## Role
- **Streaming:** Jellyfin.
- **Acquisition (*arr stack):** Transmission, Jackett, FlareSolverr, Radarr,
Sonarr, and Jellyseerr (`seerr`) for requests.
- **Photos:** PhotoPrism (`photos.nul.ie`).
- **File sharing:** copyparty (`:3923`) serving public + private media volumes.
- Media lives on the shared `/mnt/media` volume (bind-mounted read-write from
`shill`). Downloaders bind to a VPN interface
([`networking.nix`](../../../nixos/boxes/colony/vms/shill/containers/jackflix/networking.nix)),
so torrent traffic only flows while `systemd-networkd-wait-online@vpn` is up.
- A shared `media` group (gid 2000) gives the apps coordinated access.
## Networking
- `internal` assignment on the `ctrs` network (alt name `jackflix-ctr`), plus its
own VPN interface for the download clients.
-63
View File
@@ -1,63 +0,0 @@
# middleman
The front-end reverse proxy for all public colony web services — the single
ingress that `estuary` forwards HTTP/HTTPS to.
- **Source:** [`shill/containers/middleman/`](../../../nixos/boxes/colony/vms/shill/containers/middleman)
(`default.nix`, `vhosts.nix`)
- **Host:** NixOS container on `shill`
## Role
- **nginx** reverse proxy ([`vhosts.nix`](../../../nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix)
holds the per-service vhosts) with VTS stats, fancyindex, brotli, caching, and
a dynamic resolver pointed at `estuary` so upstreams can be re-resolved at
runtime. It is the single public ingress for almost every web service — colony,
home, and beyond.
- **ACME** — issues the wildcard certificates that **its own** vhosts are served
with (it is not a shared CA for the other boxes; `git`, `britway`, `kelder-spoder`, etc. each
run their own ACME):
- `nul.ie` / `*.nul.ie` (+ `*.s3.nul.ie`) via the Cloudflare DNS challenge,
- the internal `ams1.int.nul.ie` / `*` via an `exec` challenge that calls
`estuary`'s pdns over SSH.
- As a one-off consumer, it then pushes the public cert to the `mail` (Mailcow)
VM via `scp` + a remote `mailcow-ssl-reload`.
- **nginx-sso** — single-sign-on (`sso.nul.ie`) with Google OAuth and a simple
username/password provider; protects the SSO-gated vhosts below.
- **librespeed** — speed-test frontend + backend (`librespeed.${domain}` /
`speed.nul.ie`).
## Published vhosts
All under `*.nul.ie` with the wildcard cert unless noted. Upstreams are addressed
by their internal container/VM hostnames. "SSO" = gated behind nginx-sso.
| Host(s) | Upstream | Notes |
| --- | --- | --- |
| `nul.ie` (default `_`) | static | landing page (CV, SSH pubkey) + Matrix/atproto `.well-known` |
| `sso.nul.ie` | nginx-sso | SSO endpoint |
| `pass.nul.ie` | `vaultwarden` | password manager |
| `matrix.nul.ie` (+`:8448`) | `chatterbox` | Matrix client + federation |
| `element.nul.ie` | element-web | Matrix web client |
| `toot.nul.ie` | `toot` :80 | Mastodon (currently disabled — see [toot.md](toot.md)) |
| `pds.nul.ie` | `toot` :3000 | Bluesky PDS |
| `jackflix.nul.ie` | `jackflix` Jellyfin | streaming |
| `torrents` / `jackett` / `radarr` / `sonarr` `.nul.ie` | `jackflix` | *arr stack (**SSO**) |
| `gib.nul.ie` | `jackflix` Jellyseerr | requests |
| `photos.nul.ie` | `jackflix` PhotoPrism | |
| `stuff` / `public` / `p.nul.ie` | `jackflix` copyparty + `/mnt/media` | file sharing / index |
| `share.nul.ie` | `object` :9090 | |
| `minio` / `s3` / `*.s3.nul.ie` | `object` MinIO | S3 + console (Docker manifest MIME hack) |
| `nix-cache.nul.ie` | `object` Harmonia | Nix binary cache (immutable cache headers) |
| `md.nul.ie` / `pb.nul.ie` | `object` | HedgeDoc / wastebin |
| `mc-map` / `mc-rail` / `mc-map-kink` `.nul.ie` | `whale2` OCI | Minecraft maps |
| `netdata-colony.nul.ie` | many hosts :19999 | netdata fan-out (**SSO**) |
| `pront.nul.ie` | `stream-hi` (home) | print/webcam (**SSO**) |
| `hass.nul.ie` | `hass` (home) | Home Assistant |
| `hass-john.nul.ie` | `john-valorant-tun` | remote HASS over WireGuard tunnel |
## Networking
- `internal` assignment on the `ctrs` network; bind-mounts `/mnt/media` for
serving static/media content.
- nginx waits for `colony-psql` before starting (DNS bootstrap hack).
-24
View File
@@ -1,24 +0,0 @@
# object
Object storage and Nix binary cache, plus a couple of small self-hosted web
apps.
- **Source:** [`shill/containers/object.nix`](../../../nixos/boxes/colony/vms/shill/containers/object.nix)
- **Host:** NixOS container on `shill`
## Role
- **MinIO** — S3-compatible object storage (`s3.nul.ie` / `*.s3.nul.ie`). Backs
several other services (Gitea LFS/artifacts, social-media uploads, …). Stored on the
`/mnt/minio` volume (XFS, bind-mounted from `shill`).
- **Harmonia** — serves the Nix binary cache for all the boxes (`nix-cache.nul.ie`), backed
by the `/mnt/nix-cache` volume.
- **atticd** — an alternative Nix cache (stores into MinIO/S3). **Currently
disabled** — present in the config but not running.
- **HedgeDoc** — collaborative markdown notes.
- **wastebin** — pastebin.
## Networking
- `internal` assignment on the `ctrs` network (alt name `object-ctr`).
- `/mnt/minio` and `/mnt/nix-cache` bind-mounted read-write from `shill`.
-22
View File
@@ -1,22 +0,0 @@
# qclk
The `qclk` service container.
- **Source:** [`shill/containers/qclk/`](../../../nixos/boxes/colony/vms/shill/containers/qclk)
- **Host:** NixOS container on `shill`
## Role
- Runs the custom `qclk` service, exposing an API that is reached over a
dedicated WireGuard **`management`** network. Managed devices are configured as
WireGuard peers (each gets an address in the `qclk` prefix), and AS211024
trusted hosts are allowed to reach the API.
- `shill` routes the `qclk` prefix to this container.
## Networking
- `internal` assignment on the `ctrs` network (alt name `qclk-ctr`), plus the
`management` WireGuard interface carrying the `qclk` prefix.
> Check `qclk/default.nix` for the current peer list and exactly what the service
> does — this entry intentionally stays high-level.
-48
View File
@@ -1,48 +0,0 @@
# shill
The colony **NixOS container host**. Most colony applications run as
systemd-nspawn containers on `shill`.
- **Source:** [`nixos/boxes/colony/vms/shill/`](../../../nixos/boxes/colony/vms/shill)
(`default.nix`, `containers-ext.nix`, `hercules.nix`, `containers/`)
- **nixpkgs:** `mine`
- **Host:** VM on `colony` (large: 12 cores, 40 GiB RAM)
## Role
- Runs the colony NixOS containers via `my.containers.instances`, each attached
to the `ctrs` bridge with its own address.
- Provides shared data volumes to those containers via bind mounts from
LVM-backed disks: `/mnt/media` (→ `middleman`, `jackflix`), `/mnt/minio` and
`/mnt/nix-cache` (→ `object`).
- Acts as the router between the `vms` network and the `ctrs` container network
(sends RAs on `ctrs`, routes Tailscale prefixes via `waffletail` and the
`qclk` prefix via `qclk`). Includes an nftables `ct mark` hack to make
internal DNAT return paths work.
- Tuned sysctls for high connection counts / torrent traffic; netdata.
## Containers
Defined in [`shill/containers/`](../../../nixos/boxes/colony/vms/shill/containers)
and wired up in `shill`'s `my.containers.instances`:
| Container | Role |
| --- | --- |
| [`middleman`](middleman.md) | Front-end nginx reverse proxy, ACME, nginx-sso, librespeed |
| [`colony-psql`](colony-psql.md) | Shared PostgreSQL |
| [`vaultwarden`](vaultwarden.md) | Password manager |
| [`chatterbox`](chatterbox.md) | Matrix homeserver + bridges |
| [`toot`](toot.md) | Bluesky PDS (Mastodon disabled) |
| [`jackflix`](jackflix.md) | Media stack |
| [`object`](object.md) | MinIO / Harmonia / HedgeDoc / wastebin |
| [`waffletail`](waffletail.md) | Tailscale subnet router |
| [`qclk`](qclk.md) | Clock service |
| [`gam`](gam.md) | Game servers |
## Notes
- Container systems set `my.deploy.enable = false` (they are deployed as part of
`shill`'s container profiles, not as standalone deploy nodes) and render via
`my.asContainer`.
- `hercules.nix` configures Hercules CI agent bits;
`containers-ext.nix` holds extra per-container host wiring.
-19
View File
@@ -1,19 +0,0 @@
# toot
A federated-social container. Despite the name (Mastodon = "toots"), it currently
hosts a **Bluesky PDS**; the Mastodon instance is disabled.
- **Source:** [`shill/containers/toot.nix`](../../../nixos/boxes/colony/vms/shill/containers/toot.nix)
- **Host:** NixOS container on `shill`
## Role
- **Bluesky PDS** (Personal Data Server) — the active service, published as
`pds.nul.ie` (proxied by `middleman` to `:3000`).
- **Mastodon** — **currently disabled**. The config is still present and
`toot.nul.ie` still maps to `:80`, but the instance is not running. (It was
backed by the shared `colony-psql` and MinIO/S3 on `object`.)
## Networking
- `internal` assignment on the `ctrs` network (alt name `toot-ctr`).
-15
View File
@@ -1,15 +0,0 @@
# vaultwarden
[Vaultwarden](https://github.com/dani-garcia/vaultwarden), a Bitwarden-compatible
password manager.
- **Source:** [`shill/containers/vaultwarden.nix`](../../../nixos/boxes/colony/vms/shill/containers/vaultwarden.nix)
- **Host:** NixOS container on `shill`
## Role
- Runs Vaultwarden, fronted by `middleman` and published under `nul.ie`.
## Networking
- `internal` assignment on the `ctrs` network (alt name `vaultwarden-ctr`).
-19
View File
@@ -1,19 +0,0 @@
# waffletail
The colony Tailscale node / subnet router.
- **Source:** [`shill/containers/waffletail.nix`](../../../nixos/boxes/colony/vms/shill/containers/waffletail.nix)
- **Host:** NixOS container on `shill`
## Role
- Joins the Tailscale tailnet (auth key from secrets) and **advertises the colony
prefixes** into it, acting as the subnet router so tailnet clients can reach
colony services and vice-versa.
- nftables rules SNAT/forward between `host0` and `tailscale0` for the colony
v4/v6 ranges. `shill` routes the Tailscale prefixes here.
## Networking
- `internal` assignment on the `ctrs` network (alt name `waffletail-ctr`); owns
the `tailscale0` interface.
-27
View File
@@ -1,27 +0,0 @@
# whale2
A podman/OCI host on colony dedicated to game servers (kept off `shill` so the
container churn and resource use stay isolated).
- **Source:** [`nixos/boxes/colony/vms/whale2/`](../../../nixos/boxes/colony/vms/whale2)
(`default.nix`, `valheim.nix`, `minecraft/`, `enshrouded.nix`)
- **nixpkgs:** `mine`
- **Host:** VM on `colony`
## Role
- Runs OCI containers via **podman** (`virtualisation.oci-containers`, netavark
backend) on a dedicated `colony` bridge network (`oci`) with both IPv4 and
IPv6, so each game server gets its own routable address.
- Game servers configured in-repo: **Valheim**, **Minecraft** (several worlds —
see `extraAssignments`: `simpcraft`, `simpcraft-staging`, `kevcraft`,
`kinkcraft`, `graeme`), and **Enshrouded** (currently commented out).
- `/var/lib/containers` is an XFS data disk (project quotas).
## Networking
- `vms` interface with `routing` / `internal` (alt name `oci`) assignments.
- An `oci` bridge carrying the `prefixes.oci` v4/v6 ranges; per-game addresses
are handed out via `extraAssignments` (`valheim-oci`, `simpcraft-oci`, …) and
exposed to the internet through `estuary`'s port forwards.
- Firewall trusts the `oci` interface and forwards `vms → oci`.
-44
View File
@@ -1,44 +0,0 @@
# home
The home network. A VM host (`palace`), a redundant pair of routers, a storage
server, Home Assistant, and personal desktops.
- **Domain:** `h.nul.ie`
- **Source:** [`nixos/boxes/home/`](../../../nixos/boxes/home)
## Shape
```
palace (physical VM host)
├── river ──── home router (HA pair with stream)
├── cellar ── NVMe-oF storage server (SPDK)
└── sfh ───── NixOS container host ──┬── hass (Home Assistant)
└── unifi (UniFi controller — defined, currently disabled)
stream ── standalone home router (HA pair with river)
castle ── desktop workstation (boots its disks over NVMe-oF from cellar)
```
The two routers `river` (a VM on `palace`) and `stream` (standalone hardware)
share the [`routing-common`](../../../nixos/boxes/home/routing-common) config and
form a **keepalived/VRRP high-availability pair**: DHCP (kea), router
advertisements (radvd), DNS with blocklists, NAT, and the AS211024 L2 mesh link
back to colony.
## Machines
| Machine | Role | Docs |
| --- | --- | --- |
| `palace` | Physical VM host | [palace.md](palace.md) |
| `river` | Home router (VM; VRRP pair with `stream`) | [river.md](river.md) |
| `cellar` | NVMe-oF storage server (SPDK) | [cellar.md](cellar.md) |
| `sfh` | NixOS container host | [sfh.md](sfh.md) |
| `stream` | Home router (standalone hardware; VRRP pair with `river`) | [stream.md](stream.md) |
| `castle` | Desktop workstation | [castle.md](castle.md) |
### sfh containers
| Container | Role | Docs |
| --- | --- | --- |
| `hass` | Home Assistant | [hass.md](hass.md) |
| `unifi` | UniFi network controller (defined, **currently not imported**) | [unifi.md](unifi.md) |
-19
View File
@@ -1,19 +0,0 @@
# castle
A desktop workstation.
- **Source:** [`nixos/boxes/home/castle/default.nix`](../../../nixos/boxes/home/castle/default.nix)
## Role
- AMD desktop running the GUI environment (`my.gui.enable`, Sway / Wayland via
home-manager).
- **Diskless-style boot:** its `/nix`, `/persist` and `/home` are NVMe-oF volumes
served by [`cellar`](cellar.md) (`/dev/nvmeof/{nix,persist,home}`). The
networking is careful not to drop the IP used for the NVMe-oF connection.
- Uses the `my.nvme` module for the NVMe-oF client setup.
## Networking
- Sits on the home network; depends on the high-speed link to `cellar` for its
root storage.
-39
View File
@@ -1,39 +0,0 @@
# cellar
The home storage server. Exports fast NVMe storage over the network so other
machines (notably `castle`) can boot and run from it.
- **Source:** [`nixos/boxes/home/palace/vms/cellar/`](../../../nixos/boxes/home/palace/vms/cellar)
(`default.nix`, `spdk.nix`)
- **Host:** VM on `palace` (NVMe drives passed through)
- **Deploy address:** `192.168.68.80`
## Role
- Runs **SPDK** ([`spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix)) as
a userspace storage target. The kernel `nvme` driver is blacklisted so SPDK can
drive the NVMe devices directly (attached by PCI BDF).
- Builds a **RAID0** (`NVMeRaid`) across three NVMe drives, partitioned into three
namespaces, and exports each as an **NVMe-oF target over RDMA** (port 4420) on
the high-speed home network — one namespace per consumer:
| Namespace | NQN | Consumer |
| --- | --- | --- |
| `NVMeRaidp1` | `nqn.2016-06.io.spdk:river` | [`river`](river.md) |
| `NVMeRaidp2` | `nqn.2016-06.io.spdk:castle` | [`castle`](castle.md) |
| `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh.md) |
Each client is pinned by `hostnqn`, so `river`, `castle` and `sfh` all run their
storage off `cellar` over the network.
## Networking
- Exports on the high-speed home network (`lan-hi` / the `hi` assignment) over
RDMA; the SPDK target waits for that link to be online before starting.
## Notes
- The `ublk_*` calls in `my.spdk.debugCommands` are **only a debugging script**
they let you create a local ublk block device to mount and inspect the RAID on
`cellar` itself. They are **not** how storage is exported to clients; that is the
`nvmf` config above.
-18
View File
@@ -1,18 +0,0 @@
# hass
[Home Assistant](https://www.home-assistant.io/) — home automation.
- **Source:** [`sfh/containers/hass.nix`](../../../nixos/boxes/home/palace/vms/sfh/containers/hass.nix)
- **Host:** NixOS container on `sfh`
## Role
- Runs Home Assistant plus supporting services in the container. The `hass-cli`
is wired up against the local server for convenience.
- Integrations/automations are configured here (see commit history for things
like the "West Wood" integration).
## Networking
- `internal` assignment (alt name `hass-ctr`), plus a loopback assignment
(`hass-ctr-lo`) used internally.
-21
View File
@@ -1,21 +0,0 @@
# palace (host)
The physical VM host for the home network — the home equivalent of `colony`.
- **Source:** [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix)
(VM instances in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
## Role
- Bare-metal host whose job is to run the home VMs via the `my.vms` module:
`river` (router), `cellar` (storage), `sfh` (container host).
- Provides the bridged networking those VMs sit on and passes through hardware
where needed (e.g. NVMe drives to `cellar`, NICs to `river`).
## VMs hosted here
| VM | Role | Docs |
| --- | --- | --- |
| `river` | Home router (VRRP pair with `stream`) | [river.md](river.md) |
| `cellar` | NVMe-oF storage server | [cellar.md](cellar.md) |
| `sfh` | NixOS container host (Home Assistant, …) | [sfh.md](sfh.md) |
-25
View File
@@ -1,25 +0,0 @@
# river
One of the two home routers. `river` is a VM on `palace`; it forms a
high-availability pair with the standalone `stream`.
- **Source:** [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix),
built from [`routing-common`](../../../nixos/boxes/home/routing-common) (instance `0`)
- **Host:** VM on `palace`
- **Deploy address:** `192.168.68.1`
## Role
Everything in [`routing-common`](../../../nixos/boxes/home/routing-common):
- **VRRP/keepalived** failover with `stream` (`keepalived.nix`) — one router is
master at a time, sharing virtual IPs.
- **DHCP** via kea (`kea.nix`), **router advertisements** via radvd
(`radvd.nix`).
- **DNS** (`dns.nix`) — local resolver with a blocklist
(`dns-blocklist.txt`) and a periodic update script.
- **NAT / firewall** for the home LAN, with policy routing.
- **AS211024 L2 mesh** link back to colony/`estuary` (and the other edge
routers), so home and colony networks interconnect.
See [stream.md](stream.md) for the other half of the pair.
-24
View File
@@ -1,24 +0,0 @@
# sfh
The home **NixOS container host** ("smart from home" / home services).
- **Source:** [`nixos/boxes/home/palace/vms/sfh/`](../../../nixos/boxes/home/palace/vms/sfh)
(`default.nix`, `containers/`)
- **Host:** VM on `palace`
## Role
- Runs the home NixOS containers via `my.containers.instances`, in the same way
`shill` does on colony.
- Sits on the home network and connects to NVMe-oF storage (`cellar`) where
needed.
## Containers
Defined in [`sfh/containers/`](../../../nixos/boxes/home/palace/vms/sfh/containers)
and imported from its `containers/default.nix`:
| Container | Role | Docs |
| --- | --- | --- |
| `hass` | Home Assistant | [hass.md](hass.md) |
| `unifi` | UniFi controller — **defined but currently commented out** of `containers/default.nix` | [unifi.md](unifi.md) |
-19
View File
@@ -1,19 +0,0 @@
# stream
One of the two home routers. `stream` is standalone hardware; it forms a
high-availability pair with `river` (a VM on `palace`).
- **Source:** [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix),
built from [`routing-common`](../../../nixos/boxes/home/routing-common) (instance `1`)
- **Deploy address:** `192.168.68.2`
## Role
- Same [`routing-common`](../../../nixos/boxes/home/routing-common) role as
[`river`](river.md): keepalived/VRRP, kea DHCP, radvd, DNS + blocklist, NAT and
the AS211024 L2 mesh link to colony.
- Additionally pulls in `mstpd` (`routing-common/mstpd.nix`) for spanning-tree on
its bridged ports — `stream` is the one wired into the physical switching, so
it manages the L2 topology.
See [river.md](river.md) for the other half of the pair.
-23
View File
@@ -1,23 +0,0 @@
# unifi
The UniFi network controller.
- **Source:** [`sfh/containers/unifi.nix`](../../../nixos/boxes/home/palace/vms/sfh/containers/unifi.nix)
- **Host:** NixOS container on `sfh`
## Status
> **Currently disabled.** The system is still defined (`nixos.systems.unifi`),
> but its import is commented out in
> [`sfh/containers/default.nix`](../../../nixos/boxes/home/palace/vms/sfh/containers/default.nix),
> so it is not deployed as a container right now. Re-enable by uncommenting
> `./unifi.nix` there.
## Role
- Runs the UniFi controller (`services.unifi`) to manage the home UniFi network
gear.
## Networking
- `internal` assignment (alt name `unifi-ctr`).
-21
View File
@@ -1,21 +0,0 @@
# misc
Everything that isn't part of the `colony` or `home` sites: the edge VPSes, the
remote `kelder` site, a workstation, and the installer image.
| Machine | What it is | Docs |
| --- | --- | --- |
| `britway` | Vultr VPS (London, `lon1`): Headscale, Tailscale exit node, BGP edge, nginx | [britway.md](britway.md) |
| `britnet` | VPS (Birmingham, `bhx1`): Tailscale/WireGuard gateway | [britnet.md](britnet.md) |
| `kelder` | Remote site host (`hentai.engineer`): NixOS container host | [kelder.md](kelder.md) |
| `tower` | Framework Laptop 13 (12th-gen Intel) workstation | [tower.md](tower.md) |
| `installer` | Custom NixOS installer image | [installer.md](installer.md) |
## kelder containers
`kelder` is itself a container host (like `shill`/`sfh`):
| Container | Role | Docs |
| --- | --- | --- |
| `kelder-acquisition` | Media stack (Jellyfin + *arr + Transmission) | [kelder-acquisition.md](kelder-acquisition.md) |
| `kelder-spoder` | nginx web host | [kelder-spoder.md](kelder-spoder.md) |
-21
View File
@@ -1,21 +0,0 @@
# britnet
A VPS in Birmingham (`bhx1`) acting as a Tailscale/WireGuard gateway node.
- **Source:** [`nixos/boxes/britnet.nix`](../../../nixos/boxes/britnet.nix)
- **Internal domain:** `bhx1.int.nul.ie`
## Role
- **Tailscale node** + **WireGuard** (`wg0`) gateway: provides a second egress /
entry point into the boxes' overlay networks.
- nftables SNATs traffic arriving on `tailscale0` / `wg0` out of the provider
interface (`veth0`), using the `allhost` assignment addresses.
## Networking
- Provider uplink with gateways `77.74.199.1` (v4) / `2a12:ab46:5344::1` (v6).
- `tailscale0` and `wg0` overlay interfaces; `allhost` assignment for SNAT.
> `britnet` is a separate machine from [`britway`](britway.md) — different
> provider/site, narrower role (gateway rather than control plane + BGP edge).
-27
View File
@@ -1,27 +0,0 @@
# britway
A Vultr VPS in London (`lon1`) acting as a network edge node: the Tailscale
control plane, an exit node, and a BGP speaker in the AS211024 mesh.
- **Source:** [`nixos/boxes/britway/`](../../../nixos/boxes/britway)
(`default.nix`, `bgp.nix`, `nginx.nix`, `tailscale.nix`)
- **Internal domain:** `lon1.int.nul.ie`
## Role
- **Headscale** ([`tailscale.nix`](../../../nixos/boxes/britway/tailscale.nix)) — the
self-hosted Tailscale control server (`hs.nul.ie`) the rest of the boxes log
into.
- **Tailscale node** — advertises itself as an **exit node** and advertises the
tailnet routes, so tailnet clients can egress / reach internal prefixes via
britway.
- **BGP** ([`bgp.nix`](../../../nixos/boxes/britway/bgp.nix)) — part of the AS211024
L2 VXLAN mesh (`my.vpns.l2`) alongside `estuary`, `river` and `stream`.
- **nginx** ([`nginx.nix`](../../../nixos/boxes/britway/nginx.nix)) — reverse proxy /
web front-end with ACME certs.
## Networking
- `vultr` assignment on the provider interface; `as211024` on the mesh.
- A `veth0`/`tailscale0` setup with SNAT so tailnet traffic egresses via the VPS
public IP.
-19
View File
@@ -1,19 +0,0 @@
# installer
The custom NixOS installer image used to bootstrap new boxes.
- **Source:** [`nixos/installer.nix`](../../../nixos/installer.nix)
## Role
- Defines `nixos.systems.installer`, a minimal system whose `my.buildAs.*`
outputs produce installable artifacts — primarily a bootable **ISO**
(`isoImage`), and the same base is reused for kexec/netboot trees.
- Build it with the devshell commands (see [`AGENTS.md`](../../../AGENTS.md#commands)):
- `build-iso installer`
- `build-kexec installer` / `build-netboot installer`
- A released ISO is what `colony`'s VM definitions reference as install media; the
`update-installer` devshell command tags a release to trigger a rebuild.
This is a build target rather than a deployed machine — there is no running
`installer` host.
-20
View File
@@ -1,20 +0,0 @@
# kelder-acquisition
The media stack for the `kelder` site — a slimmer cousin of colony's
[`jackflix`](../colony/jackflix.md).
- **Source:** [`kelder/containers/acquisition/`](../../../nixos/boxes/kelder/containers/acquisition)
(`default.nix`, `networking.nix`)
- **Host:** NixOS container on `kelder`
## Role
- **Jellyfin** for streaming (with hardware transcoding — the `jellyfin` user is
in the `render` group, `jellyfin-ffmpeg`).
- **Acquisition:** Transmission, Jackett, Radarr, Sonarr.
- Runs under the site's shared `kontent` user.
## Networking
- `internal` assignment (alt name `acquisition-ctr`) on the kelder container
network; download client networking in `networking.nix`.
-17
View File
@@ -1,17 +0,0 @@
# kelder-spoder
An nginx web host on the `kelder` site.
- **Source:** [`kelder/containers/spoder/`](../../../nixos/boxes/kelder/containers/spoder)
(`default.nix`, `nginx.nix`)
- **Host:** NixOS container on `kelder`
## Role
- Serves web content via **nginx** ([`nginx.nix`](../../../nixos/boxes/kelder/containers/spoder/nginx.nix)),
with ACME-managed certificates (nginx in the `acme` group, reloads on renewal).
- Runs under the site's shared `kontent` user.
## Networking
- `internal` assignment (alt name `spoder-ctr`) on the kelder container network.
-24
View File
@@ -1,24 +0,0 @@
# kelder
A host at a remote site ("kelder" = cellar/basement), linked back to the rest of
the other boxes over WireGuard. It is itself a **NixOS container host**.
- **Source:** [`nixos/boxes/kelder/`](../../../nixos/boxes/kelder)
(`default.nix`, `boot.nix`, `containers/`, `plymouth/`)
- **Domain:** `hentai.engineer`
## Role
- **Site uplink:** connects to colony's `estuary` over **WireGuard**
(`kelder` peer; see [estuary.md](../colony/estuary.md)), so the remote site is
reachable through the colony edge. A periodic `dns_update.py` keeps DNS current.
- **Container host:** runs NixOS containers via `my.containers.instances`
(`acquisition`, `spoder`).
- Custom boot/splash (`boot.nix`, Plymouth theme in `plymouth/`).
## Containers
| Container | Role | Docs |
| --- | --- | --- |
| `kelder-acquisition` | Media stack (Jellyfin + *arr + Transmission) | [kelder-acquisition.md](kelder-acquisition.md) |
| `kelder-spoder` | nginx web host | [kelder-spoder.md](kelder-spoder.md) |
-16
View File
@@ -1,16 +0,0 @@
# tower
A laptop workstation — a Framework Laptop 13 (12th-gen Intel).
- **Source:** [`nixos/boxes/tower/default.nix`](../../../nixos/boxes/tower/default.nix)
## Role
- Framework Laptop 13 (12th-gen Intel) running the GUI environment
(`my.gui.enable`) with home-manager on the `mine` channel.
- Joins the tailnet via the self-hosted Headscale on [`britway`](britway.md)
(`tailscale up --login-server=https://hs.nul.ie --accept-routes`).
- Local virtualisation enabled (`kvm-intel`, IOMMU on).
> Unlike [`castle`](../home/castle.md), `tower` lives outside the `home/` box
> tree and boots from local disks rather than NVMe-oF.
+206
View File
@@ -0,0 +1,206 @@
# Deployment and workflows
How boxes get deployed, the devshell commands that drive everyday work, secrets, dev VMs, and
CI.
## deploy-rs
### Rendering
The top-level [`deploy-rs.nix`](../deploy-rs.nix) renders the flake's `deploy` output:
- It collects `nixos.systems` and `home-manager.homes` (mangling `@` in home names to
`-at-`, since deploy-rs node names can't contain `@`). An assertion rejects name collisions
between systems and homes.
- A system/home becomes a deploy node only when `configuration.config.my.deploy.enable` is
true. The node config is the *definitions* of the box's `my.deploy.node` option, re-imported
as a module so deploy-rs's submodule merging still applies.
- `autoRollback` and `magicRollback` are off globally; global `sshOpts` are
`[ "-i" ".keys/deploy.key" ]` (set in [`flake.nix`](../flake.nix)).
- The result is passed through `lib.my.deploy-rs.filterOpts`, which strips nulls so unset
options don't override deploy-rs defaults, and exposed as `deploy = deploy-rs.rendered`.
### Per-box nodes
The shared [`nixos/modules/deploy-rs.nix`](../nixos/modules/deploy-rs.nix) module provides
`my.deploy`:
- `my.deploy.enable` defaults to true, but is **automatically forced off for dev VMs and
NixOS containers** (`my.build.isDevVM` / `boot.isContainer` — containers are deployed
through their host instead, see below). The installer also disables it explicitly.
- Node defaults: `hostname` = the box's FQDN, `sshUser = "deploy"`, `user = "root"`,
`sudo = "doas -u"` (or `sudo -u`), `sshOpts` = the box's first OpenSSH port. The module
creates the `deploy` system user (bash shell, wheel, keys from `my.deploy.authorizedKeys`,
which defaults to `.keys/deploy.pub`).
Generated profiles deploy in this order:
1. **`system`** activates `config.system.build.toplevel` with `switch-to-configuration switch`,
applies the `/tmp` cwd and systemd-boot `loader.conf` workarounds, then prunes old generations.
`keepGenerations` controls how much history is retained; zero disables pruning.
2. **`container-<name>`** activates each `my.containers.instances` entry into
`/nix/var/nix/profiles/per-container/<name>/system`. With `hotReload` (the default), it reloads
`systemd-nspawn@<name>` and restarts only a stopped container or one still running the dummy
init; otherwise it restarts the unit. The same generation cleanup applies.
### Usage
- `deploy .#<host>` — the devshell's `deploy` is a wrapper that adds `--skip-checks`. Node
names are the system names (`deploy .#git`).
- `deploy-multi <hosts...>` — loops `deploy` over several nodes (extra args via `$O`).
- `deploy --boot .#<host>` — stages the config as the boot default **without** live-switching
(deploy-rs's `--boot` maps to the custom activation's `boot` phase, which also does the
generation cleanup). Use it when a live `switch` would cut the box off mid-change (e.g. a
router WAN rework), then reboot to cut over.
- `nix flake check` includes deploy-rs's own `deployChecks` for the whole `deploy` attrset.
#### `ssh-machine`
`ssh-machine <name> [cmd]` resolves `user@host` and the merged global/node `sshOpts` from
`.#deploy`, using the same `my.deploy.enable` gate and `@``-at-` mangling as `deploy`, then runs
`ssh`. Boxes default to `fish`, so pipe multi-statement remote scripts through `bash`, for example
`ssh-machine <name> bash -s < script.sh`. If a flaky agent stalls public-key authentication, use
`SSH_AUTH_SOCK= ssh-machine …`.
## Devshell commands
The repo ships a `numtide/devshell` ([`devshell/`](../devshell), entered via `direnv`). Run a
command with no arguments for its help. From
[`devshell/commands.nix`](../devshell/commands.nix):
| Command | What it does |
|---|---|
| `check-system <host> [nix args]` | Evaluates `.#nixosConfigurations."<host>".config.system.build.toplevel.drvPath` — catches module/option errors without building. Prefer this to validate a config change. |
| `build-system <host> [nix args]` | Builds the system's `toplevel` (extra args pass through to `nix build`). |
| `build-n-switch <args>` | `doas nixos-rebuild --flake .` (adds the repo as a git `safe.directory` for root first). |
| `build-home <name> [nix args]` | Builds `.#homeConfigurations."<name>".activationPackage`. |
| `home-switch [args]` | `home-manager switch --flake .`. |
| `deploy [args]` | `deploy-rs --skip-checks` (wrapper package in `devshell/default.nix`). |
| `deploy-multi <nodes...>` | Deploys several nodes in sequence. |
| `ssh-machine <name> [cmd]` | SSH to a system or home by name, resolving target/options from its deploy-rs node (see above). |
| `run-vm <host>` | Boots a system as a dev VM: installs `.keys/dev.key` into a temp `xchg/`, then `nix run`s `config.my.buildAs.devVM`. |
| `build-iso <host>` | Builds `config.my.buildAs.iso`. |
| `build-kexec <host>` | Builds `config.my.buildAs.kexecTree`. |
| `build-netboot <host>` | Builds `config.my.buildAs.netbootTree`. |
| `ragenix [args]` | `ragenix --identity .keys/dev.key` (see [Secrets](#secrets)). |
| `repl` | `nix repl .#`. |
| `fmt [args]` | `nixpkgs-fmt` (the canonical formatter). |
| `update-nixpkgs` | `nix flake update nixpkgs-{unstable,stable,mine,mine-stable}`. |
| `update-home-manager` | `nix flake update home-manager-{unstable,stable}`. |
| `update-installer` | Force-pushes the `installer` tag to trigger the installer release workflow. |
| `home-link` / `home-unlink` | Symlink (or remove) this `flake.nix` at `~/.config/home-manager/flake.nix` for standalone `home-manager` use. |
| `qemu-genmac` | Prints a random QEMU-suitable MAC (`52:54:00:xx:xx:xx`). |
| `ssh-get-ed25519 <host>` | Prints a host's ed25519 pubkey via `ssh-keyscan`. |
| `json2nix` | Converts JSON on stdin to formatted Nix. |
From [`devshell/install.nix`](../devshell/install.nix) (driven by `$INSTALLER`, the address of
a running custom installer, SSHing as root with `.keys/deploy.key`):
| Command | What it does |
|---|---|
| `installer-shell [cmd]` | Runs a command (default: a shell) inside the installer. |
| `do-install [--no-bootloader] [--no-substitute] <system>` | Builds the system's `toplevel`, `nix copy`s it into the installer's target store, sets the system profile, and activates it with `switch-to-configuration boot` (with `NIXOS_INSTALL_BOOTLOADER=1` unless `--no-bootloader`). |
From [`devshell/vm-tasks.nix`](../devshell/vm-tasks.nix) (remote VM consoles; they forward the
VM's unix sockets from `/run/vms/<vm>/` on `<host>` over SSH):
| Command | What it does |
|---|---|
| `vm-tty <host> <vm>` | Serial TTY of a VM in `minicom`. |
| `vm-monitor <host> <vm>` | QEMU monitor socket in `minicom`. |
| `vm-viewer <host> <vm>` | SPICE display in `virt-viewer` (not on Darwin). |
## Secrets
Secrets are age-encrypted files in [`secrets/`](../secrets), managed with **ragenix** (a fork
with a rekey flag, from the flake inputs).
### Per-box declarations
Each box declares `my.secrets.key`, the host public key its secrets encrypt to, and
`my.secrets.files.<name>`, whose values merge settings such as `owner` and `mode` into the agenix
secret. At runtime, identity paths come from the box's OpenSSH host keys. On tmproot boxes they are
read from the persistence directory because agenix runs before the persisted keys would otherwise
be available.
### Recipient rules
[`secrets.nix`](../secrets.nix) is generated from every system's `my.secrets.files` and `key`.
Each recipient list also includes `.keys/dev.pub`, so the development key can open every secret.
Run `ragenix -r` after adding a box or secret to re-key the files.
### Local keys and editing
The `ragenix` devshell command wraps `ragenix --identity .keys/dev.key`. The `.keys/` directory
contains that development key, the deploy key authorized for every box's `deploy` user, and other
keys referenced by `lib.my.c.sshKeyFiles`; it is required for editing secrets, deploying and running
development VMs.
## Dev VMs
Any system can be built as a throwaway QEMU VM via `config.my.buildAs.devVM` (the `build`
module extends the config with `qemu-vm.nix` and sets `my.build.isDevVM`). `run-vm <host>`
creates a temp dir, installs `.keys/dev.key` as `xchg/dev.key`, and runs the VM; inside, the
`secrets` module switches `age.identityPaths` to that dev key (`my.secrets.vmKeyPath`,
default `/tmp/xchg/dev.key`), so dev VMs can decrypt the boxes' secrets without the real host
keys. Dev VMs also get DHCP on `eth0`, an SSH port forward (host 2222 → guest 22), and are
automatically excluded from deploy targets.
## CI
GitHub/Gitea Actions workflows live in [`.gitea/workflows/`](../.gitea/workflows).
### `ci.yaml`
On pushes to `master`, this runs `nix flake check --no-build`, then builds every attribute of
`.#ci.x86_64-linux`: systems as `system-<name>`, homes as `home-<name>` (with `@` changed to
`-at-`), packages as `package-<name>`, and the development `shell`. Each result is pushed to the
Harmonia cache with [`ci/push-to-cache.sh`](../ci/push-to-cache.sh).
It then builds `.#ciDrv.x86_64-linux`, a `linkFarm` of all CI attributes, and pushes it with
`UPDATE_PROFILE=1`. That updates the `nixfiles` profile on the cache box and collects old paths
according to the workflow's retention setting. The SSH store uses `/var/lib/harmonia`,
`HARMONIA_SSH_KEY`, and pinned `ci/known_hosts`; clients use `https://nix-cache.nul.ie` through
`lib.my.c.nix.cache`.
### `installer.yaml`
Pushing the `installer` tag (refreshed by `update-installer`) builds `my.buildAs.iso` and
`my.buildAs.netbootArchive`, then attaches both to a release.
### `update-docs.yaml`
On pushes to `master`, excluding its own commits, this runs the assignment, option and DNS
reference generators and commits changed outputs as `docs: Update generated references`.
### The docs generators
The generators are registered in [`pkgs/default.nix`](../pkgs/default.nix) as wrappers around
Python scripts under [`ci/`](../ci). They leave the worktree unchanged when their output is current;
the workflow stages `docs/` and uses `git diff --cached --quiet` to decide whether to commit.
`update-docs-assignments` ([`ci/update-docs-assignments.py`](../ci/update-docs-assignments.py))
evaluates `.#nixfiles.config.nixos.allAssignments` to JSON and rewrites the consolidated
[`Box assignments`](networking.md#box-assignments) tables in `docs/networking.md` — one table
per site (`colony` / `home` / `remote`), each between a `<!-- assignments: <site> -->` marker
and a closing `<!-- assignments-end -->` line. Boxes are grouped by site from their assignment
domain, and each `Box` cell links to that box's page when one exists. Hand-written text in the
**Notes** column is preserved across runs (keyed by box + assignment), so notes survive
regeneration. Individual box pages don't carry tables; they link to the consolidated section.
`update-docs-options` ([`ci/update-docs-options.py`](../ci/update-docs-options.py)) builds the
`nixos.optionsDoc` output (declared in [`nixos/default.nix`](../nixos/default.nix)) — a
`nixosOptionsDoc` JSON dump of the custom `my.*` options (declared with `mkOpt'` / `mkBoolOpt'` in
`nixos/modules/`). It's evaluated against a **minimal synthetic system**, since the shared modules
apply to every box, so defaults don't pick up a real host's values. The renderer writes
[`docs/reference/nixos-options.md`](reference/nixos-options.md), one table per module file. The
whole file is generated; edit the option descriptions in the modules, not the reference. The
internal `asX` build-target options are marked `internal = true` so they're excluded.
`update-docs-dns` ([`ci/update-docs-dns.py`](../ci/update-docs-dns.py)) accepts forward and reverse
zone names, discovers their authoritative nameservers through NS queries, and transfers each zone
over AXFR. If a private reverse zone is not visible through the configured recursive resolver, it
asks the authoritative servers discovered for the other requested zones. It updates only the
matching `<!-- dns: <zone> -->` blocks in the [`DNS records`](reference/dns.md) reference; the page's
headings and prose remain handwritten. Kea-managed owners are identified by `DHCID` records and
removed together with their A, AAAA and PTR records; SOA records and TTLs are also omitted.
+41
View File
@@ -0,0 +1,41 @@
# installer
The custom NixOS installer image used to bootstrap new boxes.
- **Source:** [`nixos/installer.nix`](../../nixos/installer.nix)
- **Host:** — (a build target, not a deployed box)
## Role
- Defines `nixos.systems.installer`, a minimal server system rendered as a bootable ISO via
`config.my.buildAs.iso` (`my.asISO`); the same base can also be built as a kexec or netboot
tree.
- Build it with the devshell commands: `build-iso installer` (or `build-kexec installer` /
`build-netboot installer`). The `update-installer` command force-tags `installer` to
trigger a release rebuild in CI.
## Image contents
- Broad hardware support (`my.build.allHardware` pulls in the nixpkgs all-hardware profile);
EFI- and USB-bootable, zstd-compressed squashfs. Volume ID
`jackos-<release>-<arch>`, menu label "/dev/player0 Installer", image base name
`jackos-installer`.
- Root SSH with the deploy key authorized (`PermitRootLogin prohibit-password`); a random
`installer-<hex>` hostname is set at boot.
- `INSTALL_ROOT=/mnt` in the session environment, plus a `show-hw-config` alias wrapping
`nixos-generate-config --show-hardware-config --root $INSTALL_ROOT`.
- NixOS documentation enabled, `wpa_supplicant` available but not started, GC and memory-overcommit
tuning for low-memory targets, LVM thin and NFS support.
- No regular user (`my.user.enable = false`), no tmpfs-root management, no NAT, and not a
deploy target (`my.deploy.enable = false`).
## Installing a box
The devshell's installer commands ([`devshell/install.nix`](../../devshell/install.nix)) drive
an install over SSH against a booted installer reachable at `$INSTALLER`:
- `installer-shell` — get a shell on the installer.
- `do-install <system>` — builds the system's toplevel, `nix copy`s the closure to the
installer's `$INSTALL_ROOT` remote store, sets the system profile, touches `/etc/NIXOS`,
and runs `switch-to-configuration boot` with `NIXOS_INSTALL_BOOTLOADER=1` (skip the
bootloader with `--no-bootloader`, skip substitution with `--no-substitute`).
+9
View File
@@ -0,0 +1,9 @@
# Mobile boxes
Portable workstations that move between networks — currently just `tower`. Mobile boxes have
no static network assignments; they use DHCP/NetworkManager and reach other boxes over the
tailnet.
| Box | What it is |
| --- | --- |
| [`tower`](tower.md) | Framework Laptop 13 workstation |
+50
View File
@@ -0,0 +1,50 @@
# tower
Portable workstation — a Framework Laptop 13 (Intel), running the full GUI environment.
- **Source:** [`nixos/boxes/tower/default.nix`](../../nixos/boxes/tower/default.nix)
- **Host:** physical (laptop)
- **nixpkgs:** `mine`
## Hardware / platform
| Component | Inventory |
|---|---|
| Platform | Framework Laptop 13 (12th Gen Intel Core) |
| CPU | Intel Core i5-1240P (12 cores / 16 threads) |
| Memory | 32 GiB |
| Storage | 2 TB WD Black SN850 NVMe SSD |
| Graphics | Integrated Intel Iris Xe |
| Connectivity | Intel AX210 Wi-Fi 6E, Bluetooth and Thunderbolt 4 |
The configuration enables Intel microcode updates, `kvm-intel`, `intel_iommu=on`,
`intel-media-driver` and the latest kernel (`lib.my.c.kernel.latest`). Thunderbolt security
(`bolt`), the fingerprint reader (`fprintd`) and `tlp` power management are also enabled.
## Role
- Personal portable workstation: `my.gui.enable`, with Sway managed by home-manager.
- Joins the tailnet through the headscale on [`britway`](../remote/britway.md) (fish abbr
`tsup` = `doas tailscale up --login-server=https://hs.nul.ie --accept-routes`).
## Network assignments
`tower` has no static assignment; it uses DHCP through NetworkManager and reaches the other boxes
over Tailscale.
## Storage
- Two LUKS-encrypted partitions, `persist` and `home` (both `allowDiscards`); `/nix` is a
separate ext4 filesystem, `/boot` is vfat.
- Persistent `/home` (`my.user.tmphome = false`) with a size-limited tmpfs root (`my.tmproot.size`).
## Networking / services
- NetworkManager (`wpa_supplicant` backend) with `systemd-resolved`; networkd `wait-online`
is disabled. The Wi-Fi interface is renamed to `wifi` by MAC.
- Steam and Wireshark enabled; `fstrim`, LVM thin provisioning.
- `nix.gc.automatic = false` — GC is run manually on the laptop.
## Notable config files
- [`nixos/boxes/tower/default.nix`](../../nixos/boxes/tower/default.nix) — the whole box (single file).
+434
View File
@@ -0,0 +1,434 @@
# Networking
This page describes how addressing works across the boxes: the assignment mechanism, the
per-site domains and prefixes, the home router HA pair, and the overlays/tunnels that tie the
sites together. Switch-level home topology (jim/dave/brian, the ONT path) lives in
[sites/home/switches.md](sites/home/switches.md).
## Assignments
Every box declares `nixos.systems.<name>.assignments`, an attrset of *assignments* (one per
network the box is attached to). The option definition (`assignmentOpts` in
[`nixos/default.nix`](../nixos/default.nix)):
- `name` (defaults to the attribute name) and `altNames` — DNS names for the assignment.
- `visible` (default `true`) — whether DNS helpers include it.
- `domain` — DNS suffix for this assignment.
- `mtu` — interface MTU (applied via the network's `linkConfig.MTUBytes`).
- `ipv4.address` / `ipv4.mask` (default 24) / `ipv4.gateway` (defaults to host 1 of the
prefix; set explicitly to `null` when there is no gateway) / `ipv4.genPTR`.
- `ipv6.address` (nullable — an assignment can be v4-only) / `ipv6.mask` (default 64) /
`ipv6.iid` (SLAAC static token instead of a full address) / `ipv6.gateway` / `ipv6.genPTR`.
`extraAssignments` is a second, nested level for addresses that belong *to* a network but not
to any single box — the home routers use it for their floating VIP entries (`router-hi`,
`router-lo`, `router-ut`).
All assignments are aggregated into `nixos.allAssignments` — every system's `assignments`
merged with every system's `extraAssignments` — and passed to every module as the
`allAssignments` argument, so any box can route to any other box's addresses without
hardcoding. A flake-wide assertion fails evaluation if any IPv4 or IPv6 address appears in
more than one assignment. Each box also receives its own assignments as the `assignments`
module argument.
Two pieces of machinery consume assignments:
- `lib.my.networkdAssignment` ([`lib/default.nix`](../lib/default.nix)) renders an assignment
as a `systemd.network` network: static `address`/`gateway`, MTU, LLDP, and IPv6 RA handling
(`IPv6AcceptRA` when there's no static gateway or a static `iid` is set, with
`Token = static:<iid>`).
- `mkSystem` defaults `networking.hostName` to `assignments.internal.name` (falling back to
the system name) and `networking.domain` to `assignments.internal.domain`. The shared
`network` module sets a fallback domain of `int.nul.ie` for boxes without one.
## Box assignments
Every box's `assignments` (plus the routers' floating VIP `extraAssignments`), grouped first by site
and then by assignment key. Generated from `nixos.allAssignments` by
`nix run .#update-docs-assignments`; CI keeps it current. Only the **Notes** column is hand-written —
edit prose there, never the other generated cells.
### colony
<!-- assignments: colony -->
<!-- assignments-start -->
#### `internal`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`chatterbox`](sites/colony/shill/containers/chatterbox.md) | `10.100.2.5/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::5/64` | ams1.int.nul.ie | |
| [`colony`](sites/colony/colony.md) | `94.142.241.224/32` | `2a0e:97c0:4d2:10::2/64` | ams1.int.nul.ie | |
| [`colony-psql`](sites/colony/shill/containers/colony-psql.md) | `10.100.2.4/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::4/64` | ams1.int.nul.ie | |
| `enshrouded-oci` | `10.100.3.5/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::5/64` | ams1.int.nul.ie | Enshrouded OCI container on [`whale2`](sites/colony/whale2.md#game-servers); disabled |
| [`estuary`](sites/colony/estuary.md) | `94.142.240.44/24 gw 94.142.240.254` | `2a02:898:0:20::329:1/64 gw 2a02:898:0:20::1` | ams1.int.nul.ie | |
| [`gam`](sites/colony/shill/containers/gam.md) | `10.100.2.11/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::b/64` | ams1.int.nul.ie | |
| [`git`](sites/colony/git.md) | `94.142.241.117/32` | `2a0e:97c0:4d2:11::4/64` | ams1.int.nul.ie | |
| `graeme-oci` | `10.100.3.8/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::8/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| [`jackflix`](sites/colony/shill/containers/jackflix.md) | `10.100.2.6/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::6/64` | ams1.int.nul.ie | |
| `kevcraft-oci` | `10.100.3.6/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::6/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| `kinkcraft-oci` | `10.100.3.7/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::7/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| [`middleman`](sites/colony/shill/containers/middleman.md) | `10.100.2.2/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::2/64` | ams1.int.nul.ie | |
| [`object`](sites/colony/shill/containers/object.md) | `10.100.2.7/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::7/64` | ams1.int.nul.ie | |
| [`qclk`](sites/colony/shill/containers/qclk.md) | `10.100.2.10/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::a/64` | ams1.int.nul.ie | |
| [`shill`](sites/colony/shill/README.md) | `94.142.241.225/32` | `2a0e:97c0:4d2:11::2/64` | ams1.int.nul.ie | |
| `simpcraft-oci` | `10.100.3.3/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::3/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| `simpcraft-staging-oci` | `10.100.3.4/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::4/64` | ams1.int.nul.ie | Minecraft staging OCI container on [`whale2`](sites/colony/whale2.md#game-servers); disabled |
| [`toot`](sites/colony/shill/containers/toot.md) | `10.100.2.8/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::8/64` | ams1.int.nul.ie | |
| `valheim-oci` | `10.100.3.2/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::2/64` | ams1.int.nul.ie | Valheim OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| [`vaultwarden`](sites/colony/shill/containers/vaultwarden.md) | `10.100.2.3/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::3/64` | ams1.int.nul.ie | |
| [`waffletail`](sites/colony/shill/containers/waffletail.md) | `10.100.2.9/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::9/64` | ams1.int.nul.ie | |
| [`whale2`](sites/colony/whale2.md) | `94.142.241.226/32` | `2a0e:97c0:4d2:11::3/64` | ams1.int.nul.ie | |
#### `as211024`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`estuary`](sites/colony/estuary.md) | `10.100.50.1/24` | `2a0e:97c0:4df::1/64` | — | |
#### `base`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`estuary`](sites/colony/estuary.md) | `10.100.0.1/24` | `2a0e:97c0:4d2:10::1/64` | ams1.int.nul.ie | |
#### `ctrs`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`shill`](sites/colony/shill/README.md) | `10.100.2.1/24` | `2a0e:97c0:4d2:12::1/64` | ams1.int.nul.ie | |
#### `oci`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`whale2`](sites/colony/whale2.md) | `10.100.3.1/24` | `2a0e:97c0:4d2:13::1/64` | ams1.int.nul.ie | |
#### `qclk`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`qclk`](sites/colony/shill/containers/qclk.md) | `10.100.4.1/24` | — | — | |
#### `routing`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`colony`](sites/colony/colony.md) | `10.100.0.2/24 gw 10.100.0.1` | — | ams1.int.nul.ie | |
| [`git`](sites/colony/git.md) | `10.100.1.4/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
| [`shill`](sites/colony/shill/README.md) | `10.100.1.2/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
| [`whale2`](sites/colony/whale2.md) | `10.100.1.3/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
#### `tailscale`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`waffletail`](sites/colony/shill/containers/waffletail.md) | `100.64.0.5/32` | `fd7a:115c:a1e0::5/128` | — | |
#### `vms`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`colony`](sites/colony/colony.md) | `10.100.1.1/24` | `2a0e:97c0:4d2:11::1/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
### home
<!-- assignments: home -->
<!-- assignments-start -->
#### `as211024`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`river`](sites/home/river.md) | `10.100.50.2/24` | `2a0e:97c0:4df:0:1::1/64 gw 2a0e:97c0:4df:0:2::1` | — | |
| [`stream`](sites/home/stream.md) | `10.100.50.3/24` | `2a0e:97c0:4df:0:1::2/64 gw 2a0e:97c0:4df:0:2::1` | — | |
#### `core`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`palace`](sites/home/palace.md) | `192.168.64.20/24` | — | h.nul.ie | |
| [`river`](sites/home/river.md) | `192.168.64.1/24` | — | h.nul.ie | |
| [`stream`](sites/home/stream.md) | `192.168.64.2/24` | — | h.nul.ie | |
| [`unifi`](sites/home/sfh/containers/unifi.md) | `192.168.64.21/24` | — | h.nul.ie | |
#### `hi`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`castle`](sites/home/castle.md) | `192.168.68.40/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::3:1/64` | h.nul.ie | |
| [`cellar`](sites/home/cellar.md) | `192.168.68.80/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:1/64` | h.nul.ie | |
| [`hass`](sites/home/sfh/containers/hass.md) | `192.168.68.103/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:3/64` | h.nul.ie | |
| [`palace`](sites/home/palace.md) | `192.168.68.22/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::2:1/64` | h.nul.ie | |
| [`river`](sites/home/river.md) | `192.168.68.1/22` | `2a0e:97c0:4d0:1::1/64` | h.nul.ie | |
| `router-hi` | `192.168.71.254/22 gw 192.168.68.1` | `2a0e:97c0:4d0:1::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
| [`sfh`](sites/home/sfh/README.md) | `192.168.68.81/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:2/64` | h.nul.ie | |
| [`stream`](sites/home/stream.md) | `192.168.68.2/22` | `2a0e:97c0:4d0:1::2/64` | h.nul.ie | |
| [`unifi`](sites/home/sfh/containers/unifi.md) | `192.168.68.100/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:1/64` | h.nul.ie | |
#### `lo`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`hass`](sites/home/sfh/containers/hass.md) | `192.168.72.103/21` | `2a0e:97c0:4d0:2::5:3/64` | h.nul.ie | |
| [`river`](sites/home/river.md) | `192.168.72.1/21` | `2a0e:97c0:4d0:2::1/64` | h.nul.ie | |
| `router-lo` | `192.168.79.254/21 gw 192.168.72.1` | `2a0e:97c0:4d0:2::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
| [`stream`](sites/home/stream.md) | `192.168.72.2/21` | `2a0e:97c0:4d0:2::2/64` | h.nul.ie | |
#### `untrusted`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`river`](sites/home/river.md) | `192.168.80.1/24` | `2a0e:97c0:4d0:3::1/64` | h.nul.ie | |
| `router-ut` | `192.168.80.254/24 gw 192.168.80.1` | `2a0e:97c0:4d0:3::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
| [`stream`](sites/home/stream.md) | `192.168.80.2/24` | `2a0e:97c0:4d0:3::2/64` | h.nul.ie | |
<!-- assignments-end -->
### remote
<!-- assignments: remote -->
<!-- assignments-start -->
#### `internal`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`kelder-acquisition`](remote/kelder/containers/kelder-acquisition.md) | `172.16.64.2/24 gw 172.16.64.1` | — | hentai.engineer | |
| [`kelder-spoder`](remote/kelder/containers/kelder-spoder.md) | `172.16.64.3/24 gw 172.16.64.1` | — | hentai.engineer | |
#### `allhost`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`britnet`](remote/britnet.md) | `77.74.199.67/24 gw 77.74.199.1` | `2a12:ab46:5344:99::a/64 gw 2a12:ab46:5344::1` | bhx1.int.nul.ie | |
#### `as211024`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`britway`](remote/britway.md) | `10.100.50.5/24` | `2a0e:97c0:4df:0:2::1/64` | — | |
#### `ctrs`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`kelder`](remote/kelder/README.md) | `172.16.64.1/24` | — | hentai.engineer | |
#### `estuary`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`kelder`](remote/kelder/README.md) | `94.142.242.254/32` | — | — | |
#### `vpn`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`britnet`](remote/britnet.md) | `10.200.0.1/24` | `fdfb:5ebf:6e84::1/64` | — | |
#### `vultr`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`britway`](remote/britway.md) | `45.76.141.188/23 gw 45.76.140.1` | `2001:19f0:7402:128b::1/64` | lon1.int.nul.ie | |
<!-- assignments-end -->
## Domains
The public domain is `nul.ie` (`lib.my.c.pubDomain`). Each site has its own internal domain
(constants in [`lib/constants.nix`](../lib/constants.nix)):
| Site | Domain |
|---|---|
| colony | `ams1.int.nul.ie` |
| home | `h.nul.ie` |
| britway | `lon1.int.nul.ie` |
| britnet | `bhx1.int.nul.ie` |
| kelder | `hentai.engineer` |
## colony
The colony box is a hosted server in Amsterdam (`ams1`); its public edge is the `estuary` VM
(`94.142.240.44`, `2a02:898:0:20::329:1`), which NATs and filters for everything behind it.
The internal prefixes (`lib.my.c.colony.prefixes`) are carved from `10.100.0.0/16` and
`2a0e:97c0:4d2:10::/60`:
| Network | IPv4 | IPv6 | Purpose |
|---|---|---|---|
| `base` | `10.100.0.0/24` | `2a0e:97c0:4d2:10::/64` | Base/management LAN (bridge on the host; estuary is `.1`) |
| `vms` | `10.100.1.0/24` | `2a0e:97c0:4d2:11::/64` | VM network (host is `.1`, hands out RAs) |
| `ctrs` | `10.100.2.0/24` | `2a0e:97c0:4d2:12::/64` | `systemd-nspawn` containers on the `shill` VM |
| `oci` | `10.100.3.0/24` | `2a0e:97c0:4d2:13::/64` | Podman/OCI workloads on the `whale2` VM |
| `qclk` | `10.100.4.0/24` | — | WireGuard endpoint instances in the `qclk` container |
On top of that: `p2pTunnels` (`10.100.5.0/24`) holds point-to-point tunnel /30s (see
[WireGuard tunnels](#wireguard-point-to-point-tunnels)); the `as211024` mesh gets
`10.100.50.0/24` + `2a0e:97c0:4df::/64` (see [the L2 mesh](#the-as211024-l2-mesh)); and the
`cust` block (`10.100.100.0/24`, `2a0e:97c0:4d2:2000::/56`) plus the `vip1`/`vip2`/`vip3`
public blocks and the per-customer `mail` / `darts` / `jam` prefixes carry customer-facing
services with their own public addresses (announced by BGP, routed via the host).
## home
The home site prefixes (`lib.my.c.home.prefixes`) come from `192.168.64.0/18` and
`2a0e:97c0:4d0::/60`, with VLAN IDs from `lib.my.c.home.vlans`:
| Network | VLAN | IPv4 | IPv6 | MTU | Purpose |
|---|---|---|---|---|---|
| `core` | — (macvlan) | `192.168.64.0/24` | — | 1500 | Router-to-router/core link |
| `hi` | 100 | `192.168.68.0/22` | `2a0e:97c0:4d0:1::/64` | 9000 | High-speed LAN (jumbo frames) |
| `lo` | 110 | `192.168.72.0/21` | `2a0e:97c0:4d0:2::/64` | 1500 | General LAN |
| `untrusted` | 120 | `192.168.80.0/24` | `2a0e:97c0:4d0:3::/64` | 1500 | Untrusted / IoT |
| `modem` | 130 (`wan`) | `192.168.0.0/24` | — | — | Virgin Media modem management (stream) |
| `ont` | 140 (`wan-pon-ont`) | `192.168.100.0/24` | — | — | Digiweb ONT management (river) |
Two more WAN-side VLANs exist: `pon-isp` (10), the ISP VLAN Digiweb delivers at the ONT and
which is trunked untranslated to river, and `wan-pon-isp` (141), reserved for a future
multi-ONT translation scheme — see
[sites/home/switches.md](sites/home/switches.md) for the fabric side.
The routers themselves (`river` = host 1, `stream` = host 2 in each prefix) are built from one
definition, [`nixos/boxes/home/routing-common`](../nixos/boxes/home/routing-common/default.nix),
parameterised by an index (`0` = river, `1` = stream) that derives per-box addresses, DHCP
pool splits, VRRP state/priority and DNS `ns` numbering.
### Router VIPs
Clients never use a router's real address: each client VLAN has a floating VIP
(`lib.my.c.home.vips`) that follows the VRRP master. The VIPs are also declared as
`extraAssignments` (`router-hi`/`router-lo`/`router-ut`) so they appear in `allAssignments`
and DNS:
| Assignment | IPv4 | IPv6 |
|---|---|---|
| `router-hi` | `192.168.71.254/22` | `2a0e:97c0:4d0:1::ffff/64` |
| `router-lo` | `192.168.79.254/21` | `2a0e:97c0:4d0:2::ffff/64` |
| `router-ut` | `192.168.80.254/24` | `2a0e:97c0:4d0:3::ffff/64` |
There is also a mesh-side VIP (`as211024`): `10.100.50.4` and `2a0e:97c0:4df:0:1::ffff`,
which the other sites use as their next-hop into the home prefixes.
### Router HA
#### VRRP
[`routing-common/keepalived.nix`](../nixos/boxes/home/routing-common/keepalived.nix) defines separate
v4/v6 instances (router IDs 51/52) on `lan-core`. Index 0 (`river`) starts as `MASTER`, priorities
are `255 - index`, and track scripts demote a router whose WAN checks fail. All VIPs of an address
family move together.
#### Client gateway and DNS
`kea` hands out `vips.<vlan>.v4` as both `routers` and `domain-name-servers`, with the two routers
serving disjoint pool halves. `radvd` advertises the v6 VIP as RDNSS (`untrusted` gets Cloudflare)
and keepalived's `notify_master`/`notify_backup` hooks ensure that only the master sends RAs.
#### DNS binding
`pdns-recursor` binds the VIPs directly; see
[`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix). The
`net.ipv4.ip_nonlocal_bind` / `net.ipv6.ip_nonlocal_bind` settings let the backup listen before it
owns the addresses, so failover does not depend on client resolver timeouts. The recursor forwards
the site's zones to authoritative PowerDNS on `127.0.0.1:5353`. The generated
[DNS reference](reference/dns.md) lists the live forward and reverse records; the authoritative
servers allow its AXFRs from the shared internal prefixes and the colony site's egress address.
#### `wan-online.target`
This shared, initially inert systemd target means "the public IPv4 WAN route is up".
`routing-common` only declares it: `stream` gates it on a oneshot that waits for the DHCP default
route on `wan`, while `river`'s `pppd` hooks start and stop it. Consumers such as `ipsec` attach with
`wantedBy` + `partOf` + `after`, never `requires`/`wants`, so they cannot pull the target in early
and they reload on WAN flap.
### WAN paths (summary)
#### `river`
The VM on `palace` runs Digiweb PPPoE directly on VLAN 10 (`vlans.pon-isp`), trunked untranslated
through the switches. The carrying interface is named `wan-pon-isp`; VLAN ID 141 with that name is
reserved for a future translation scheme. MTU 1508 preserves a 1500-byte PPP session, IPCP requests
the static address, and the `pppd` hooks own `wan-online.target`. `wan-pon-ont` (VLAN 140) reaches
the ONT management subnet at `192.168.100.0/24`, where `river` takes `.100`.
#### `stream`
The bare-metal backup uses DHCP from the Virgin Media modem on `wan` (VLAN 130), keeps a static
`192.168.0.100/24` management address beside the public lease, and applies CAKE shaping through
`wan-ifb`.
The full fabric story — which switch port carries what, why VLAN 10 is trunked untranslated,
and the multi-ONT plan — is in [sites/home/switches.md](sites/home/switches.md); the
`my.homeRouter.*` options (`dns.wanSkipBroadcasts`, `firewall.untrustedRejectV4`) let each box
tell `routing-common` about subnets sharing its WAN interface.
## The AS211024 L2 mesh
The edge routers are joined by a layer-2 mesh, defined once as `nixos.vpns.l2.as211024` in
[`nixos/boxes/colony/vms/estuary/default.nix`](../nixos/boxes/colony/vms/estuary/default.nix)
and realised on each member by the [`l2mesh` module](../nixos/modules/l2mesh.nix):
### Members
`estuary`, `river`, `stream` and `britway` peer on their public addresses.
### Transport
VXLAN (VNI 211024, UDP port 4789) uses static per-peer FDB entries and UDP-encapsulated IPsec in
Libreswan transport mode. It authenticates without encryption by default; `security.encrypt`
switches ESP from `null-sha256` to AES-GCM. The shared `l2mesh/as211024.key` PSK is expanded into
`/run/l2mesh.secrets` when `ipsec` starts.
### Overlay addressing
The overlay uses `10.100.50.0/24` / `2a0e:97c0:4df::/64`. Each router holds `10.100.50.<n>`:
`estuary` `.1`, `river` `.2`, `stream` `.3`, and `britway` `.5`. Interface MTU is calculated from
the physical MTU minus VXLAN/UDP/IPsec overhead.
### Routing
The home routers route colony prefixes via `estuary` and Tailscale prefixes via `britway`;
`estuary` and `britway` route home prefixes through the `10.100.50.4` VIP. The home IPv6 default
route also crosses the mesh through `britway`, which is why the recursor pins its upstreams to IPv4
as noted in [`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix). The `nftTrust`
snippet in `lib.my.c.as211024` admits trusted colony, home, mesh and Tailscale prefixes at the edge
firewalls.
## BGP
Both edge routers run `bird2` as AS211024.
### `estuary`
The colony edge takes a full table from ColoClue, IPv6 transit from iFog and Hurricane Electric,
and peers through the Frys-IX, NL-ix and FogIXP route servers. It also has direct and monitoring
sessions; the complete peer table and originated routes live in [estuary.md](sites/colony/estuary.md#bgp).
### `britway`
The London edge uses secret-backed MD5 authentication for Vultr transit (AS64515), connects to
`bgp.tools`, and originates the internal, colony and home IPv6 prefixes. See
[britway.md](remote/britway.md) for its box-specific routing detail.
## WireGuard point-to-point tunnels
Separate from the mesh, a few boxes run their own WireGuard (private keys in per-box secrets):
- **estuary** terminates point-to-point tunnels to `kelder`, `hillcrest` and `john-valorant`,
addressed out of `p2pTunnels` — see
[estuary.md](sites/colony/estuary.md#wireguard-tunnels) for the per-tunnel ports and prefixes.
- **`qclk`** (container on `shill`) runs its own WireGuard on port 51821 out of `10.100.4.0/24`
— see [qclk.md](sites/colony/shill/containers/qclk.md).
- **`britnet`** hosts a road-warrior WireGuard VPN on port 51820 serving `10.200.0.0/24` /
`fdfb:5ebf:6e84::/64` — see [britnet.md](remote/britnet.md).
## Tailscale / headscale
Tailscale runs against a self-hosted **headscale** control plane on britway at `https://hs.nul.ie`
(OIDC, MagicDNS, split DNS — see [britway.md](remote/britway.md)). The tailnet prefixes are
`100.64.0.0/10` / `fd7a:115c:a1e0::/48`.
Notable nodes:
- **waffletail** (container on `shill`) — the colony subnet router: advertises the colony
prefixes, acts as an exit node, and SNATs tailnet traffic into the colony networks.
- **britway** — advertises the home prefixes (routed via the mesh) and is also an exit node.
- Other boxes join with the shared `tailscale-auth.key` auth-key secret.
+319
View File
@@ -0,0 +1,319 @@
# DNS records
The tables on this page are generated from live authoritative zone transfers by
`nix run .#update-docs-dns -- <zones...>`; CI keeps them current. The Nix DNS configuration is
the source of truth. Edit the prose and headings here, but not content between the DNS markers.
DHCP-managed records are excluded. The generator identifies them by `DHCID` and removes the
corresponding forward and reverse records. SOA records and TTLs are also omitted because they are
operational metadata rather than useful inventory.
## Colony
These zones are served by [`estuary`](../sites/colony/estuary.md); their source configuration is
[`estuary/dns.nix`](../../nixos/boxes/colony/vms/estuary/dns.nix).
### Forward zone: `ams1.int.nul.ie`
<!-- dns: ams1.int.nul.ie -->
<!-- dns-start -->
| Name | Type | Value |
|---|---|---|
| `@` | `ALIAS` | `estuary-vm.ams1.int.nul.ie.` |
| `@` | `NS` | `ns.ams1.int.nul.ie.` |
| `_acme-challenge` | `TXT (LUA)` | `generated at query time` |
| `andrey-cust` | `A` | `94.142.242.254` |
| `chatterbox-ctr` | `A` | `10.100.2.5` |
| `chatterbox-ctr` | `AAAA` | `2a0e:97c0:4d2:12::5` |
| `colony` | `A` | `94.142.241.224` |
| `colony` | `AAAA` | `2a0e:97c0:4d2:10::2` |
| `colony-psql` | `CNAME` | `colony-psql-ctr.ams1.int.nul.ie.` |
| `colony-psql-ctr` | `A` | `10.100.2.4` |
| `colony-psql-ctr` | `AAAA` | `2a0e:97c0:4d2:12::4` |
| `colony-routing` | `A` | `10.100.0.2` |
| `colony-vms` | `A` | `10.100.1.1` |
| `colony-vms` | `AAAA` | `2a0e:97c0:4d2:11::1` |
| `ctr` | `CNAME` | `shill-vm.ams1.int.nul.ie.` |
| `darts-cust` | `A` | `94.142.242.255` |
| `darts-cust` | `AAAA` | `2a0e:97c0:4d2:2001::1` |
| `enshrouded` | `A` | `94.142.240.44` |
| `enshrouded-oci` | `A` | `10.100.3.5` |
| `enshrouded-oci` | `AAAA` | `2a0e:97c0:4d2:13::5` |
| `estuary-vm` | `A` | `94.142.240.44` |
| `estuary-vm` | `AAAA` | `2a02:898:0:20::329:1` |
| `estuary-vm-base` | `A` | `10.100.0.1` |
| `estuary-vm-base` | `AAAA` | `2a0e:97c0:4d2:10::1` |
| `fw` | `CNAME` | `estuary-vm.ams1.int.nul.ie.` |
| `gam-ctr` | `A` | `10.100.2.11` |
| `gam-ctr` | `AAAA` | `2a0e:97c0:4d2:12::b` |
| `git-vm` | `A` | `94.142.241.117` |
| `git-vm` | `AAAA` | `2a0e:97c0:4d2:11::4` |
| `git-vm-routing` | `A` | `10.100.1.4` |
| `graeme` | `A` | `94.142.240.44` |
| `graeme` | `AAAA` | `2a0e:97c0:4d2:13::8` |
| `graeme-oci` | `A` | `10.100.3.8` |
| `graeme-oci` | `AAAA` | `2a0e:97c0:4d2:13::8` |
| `hillcrest-tun` | `A` | `10.100.5.2` |
| `http` | `A` | `94.142.240.44` |
| `http` | `AAAA` | `2a0e:97c0:4d2:12::2` |
| `jackflix-ctr` | `A` | `10.100.2.6` |
| `jackflix-ctr` | `AAAA` | `2a0e:97c0:4d2:12::6` |
| `jam-cust` | `A` | `10.100.100.4` |
| `jam-cust` | `AAAA` | `2a0e:97c0:4d2:2002::1` |
| `jam-fwd` | `A` | `94.142.241.225` |
| `john-valorant-tun` | `A` | `10.100.5.6` |
| `kevcraft` | `A` | `94.142.240.44` |
| `kevcraft` | `AAAA` | `2a0e:97c0:4d2:13::6` |
| `kevcraft-oci` | `A` | `10.100.3.6` |
| `kevcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::6` |
| `kinkcraft` | `A` | `94.142.240.44` |
| `kinkcraft` | `AAAA` | `2a0e:97c0:4d2:13::7` |
| `kinkcraft-oci` | `A` | `10.100.3.7` |
| `kinkcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::7` |
| `librespeed` | `CNAME` | `http.ams1.int.nul.ie.` |
| `mail-vm` | `A` | `94.142.241.227` |
| `mail-vm` | `AAAA` | `2a0e:97c0:4d2:2000::1` |
| `middleman-ctr` | `A` | `10.100.2.2` |
| `middleman-ctr` | `AAAA` | `2a0e:97c0:4d2:12::2` |
| `ns` | `ALIAS` | `estuary-vm.ams1.int.nul.ie.` |
| `object-ctr` | `A` | `10.100.2.7` |
| `object-ctr` | `AAAA` | `2a0e:97c0:4d2:12::7` |
| `oci` | `CNAME` | `whale-vm.ams1.int.nul.ie.` |
| `qclk-ctr` | `A` | `10.100.2.10` |
| `qclk-ctr` | `AAAA` | `2a0e:97c0:4d2:12::a` |
| `shill-vm` | `A` | `94.142.241.225` |
| `shill-vm` | `AAAA` | `2a0e:97c0:4d2:11::2` |
| `shill-vm-ctrs` | `A` | `10.100.2.1` |
| `shill-vm-ctrs` | `AAAA` | `2a0e:97c0:4d2:12::1` |
| `shill-vm-routing` | `A` | `10.100.1.2` |
| `simpcraft` | `A` | `94.142.240.44` |
| `simpcraft` | `AAAA` | `2a0e:97c0:4d2:13::3` |
| `simpcraft-oci` | `A` | `10.100.3.3` |
| `simpcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::3` |
| `simpcraft-staging` | `A` | `94.142.240.44` |
| `simpcraft-staging` | `AAAA` | `2a0e:97c0:4d2:13::4` |
| `simpcraft-staging-oci` | `A` | `10.100.3.4` |
| `simpcraft-staging-oci` | `AAAA` | `2a0e:97c0:4d2:13::4` |
| `terraria` | `A` | `94.142.240.44` |
| `terraria` | `AAAA` | `2a0e:97c0:4d2:12::b` |
| `toot-ctr` | `A` | `10.100.2.8` |
| `toot-ctr` | `AAAA` | `2a0e:97c0:4d2:12::8` |
| `valheim` | `A` | `94.142.240.44` |
| `valheim` | `AAAA` | `2a0e:97c0:4d2:13::2` |
| `valheim-oci` | `A` | `10.100.3.2` |
| `valheim-oci` | `AAAA` | `2a0e:97c0:4d2:13::2` |
| `vaultwarden-ctr` | `A` | `10.100.2.3` |
| `vaultwarden-ctr` | `AAAA` | `2a0e:97c0:4d2:12::3` |
| `vm` | `CNAME` | `colony.ams1.int.nul.ie.` |
| `waffletail-ctr` | `A` | `10.100.2.9` |
| `waffletail-ctr` | `AAAA` | `2a0e:97c0:4d2:12::9` |
| `whale-vm` | `A` | `94.142.241.226` |
| `whale-vm` | `AAAA` | `2a0e:97c0:4d2:11::3` |
| `whale-vm-routing` | `A` | `10.100.1.3` |
<!-- dns-end -->
### IPv4 reverse zone: `100.10.in-addr.arpa`
<!-- dns: 100.10.in-addr.arpa -->
<!-- dns-start -->
| Address | Name |
|---|---|
| `10.100.0.1` | `estuary-vm-base.ams1.int.nul.ie.` |
| `10.100.0.2` | `colony-routing.ams1.int.nul.ie.` |
| `10.100.1.1` | `colony-vms.ams1.int.nul.ie.` |
| `10.100.1.2` | `shill-vm-routing.ams1.int.nul.ie.` |
| `10.100.1.3` | `whale-vm-routing.ams1.int.nul.ie.` |
| `10.100.1.4` | `git-vm-routing.ams1.int.nul.ie.` |
| `10.100.2.1` | `shill-vm-ctrs.ams1.int.nul.ie.` |
| `10.100.2.2` | `middleman-ctr.ams1.int.nul.ie.` |
| `10.100.2.3` | `vaultwarden-ctr.ams1.int.nul.ie.` |
| `10.100.2.4` | `colony-psql-ctr.ams1.int.nul.ie.` |
| `10.100.2.5` | `chatterbox-ctr.ams1.int.nul.ie.` |
| `10.100.2.6` | `jackflix-ctr.ams1.int.nul.ie.` |
| `10.100.2.7` | `object-ctr.ams1.int.nul.ie.` |
| `10.100.2.8` | `toot-ctr.ams1.int.nul.ie.` |
| `10.100.2.9` | `waffletail-ctr.ams1.int.nul.ie.` |
| `10.100.2.10` | `qclk-ctr.ams1.int.nul.ie.` |
| `10.100.2.11` | `gam-ctr.ams1.int.nul.ie.` |
| `10.100.3.2` | `valheim-oci.ams1.int.nul.ie.` |
| `10.100.3.3` | `simpcraft-oci.ams1.int.nul.ie.` |
| `10.100.3.4` | `simpcraft-staging-oci.ams1.int.nul.ie.` |
| `10.100.3.5` | `enshrouded-oci.ams1.int.nul.ie.` |
| `10.100.3.6` | `kevcraft-oci.ams1.int.nul.ie.` |
| `10.100.3.7` | `kinkcraft-oci.ams1.int.nul.ie.` |
| `10.100.3.8` | `graeme-oci.ams1.int.nul.ie.` |
<!-- dns-end -->
### IPv6 reverse zone: `2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa`
<!-- dns: 2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa -->
<!-- dns-start -->
| Address | Name |
|---|---|
| `2a0e:97c0:4d2:10::1` | `estuary-vm-base.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:10::2` | `colony.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:11::1` | `colony-vms.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:11::2` | `shill-vm.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:11::3` | `whale-vm.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:11::4` | `git-vm.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::1` | `shill-vm-ctrs.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::2` | `middleman-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::3` | `vaultwarden-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::4` | `colony-psql-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::5` | `chatterbox-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::6` | `jackflix-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::7` | `object-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::8` | `toot-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::9` | `waffletail-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::a` | `qclk-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::b` | `gam-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::2` | `valheim-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::3` | `simpcraft-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::4` | `simpcraft-staging-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::5` | `enshrouded-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::6` | `kevcraft-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::7` | `kinkcraft-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::8` | `graeme-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:2000::1` | `mail.nul.ie.` |
| `2a0e:97c0:4d2:2001::1` | `darts-cust.ams1.int.nul.ie.` |
<!-- dns-end -->
## Home
These zones are served by [`river`](../sites/home/river.md) and
[`stream`](../sites/home/stream.md); their shared source configuration is
[`routing-common/dns.nix`](../../nixos/boxes/home/routing-common/dns.nix).
### Forward zone: `h.nul.ie`
<!-- dns: h.nul.ie -->
<!-- dns-start -->
| Name | Type | Value |
|---|---|---|
| `@` | `NS` | `ns1.h.nul.ie.` |
| `@` | `NS` | `ns2.h.nul.ie.` |
| `boot` | `CNAME` | `river-hi.h.nul.ie.` |
| `brian` | `A` | `192.168.64.13` |
| `castle` | `A` | `192.168.68.40` |
| `castle` | `AAAA` | `2a0e:97c0:4d0:1::3:1` |
| `cellar` | `A` | `192.168.68.80` |
| `cellar` | `AAAA` | `2a0e:97c0:4d0:1::4:1` |
| `dave` | `A` | `192.168.68.11` |
| `dave` | `AAAA` | `2a0e:97c0:4d0:1::1:2` |
| `dave-core` | `A` | `192.168.64.11` |
| `dave-lo` | `A` | `192.168.72.11` |
| `dave-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:2` |
| `dyn` | `NS` | `ns1.dyn.h.nul.ie.` |
| `dyn` | `NS` | `ns2.dyn.h.nul.ie.` |
| `frigate` | `CNAME` | `hass-ctr.h.nul.ie.` |
| `hass-ctr` | `A` | `192.168.68.103` |
| `hass-ctr` | `AAAA` | `2a0e:97c0:4d0:1::5:3` |
| `hass-ctr-lo` | `A` | `192.168.72.103` |
| `hass-ctr-lo` | `AAAA` | `2a0e:97c0:4d0:2::5:3` |
| `jim` | `A` | `192.168.68.10` |
| `jim` | `AAAA` | `2a0e:97c0:4d0:1::1:1` |
| `jim-core` | `A` | `192.168.64.10` |
| `jim-lo` | `A` | `192.168.72.10` |
| `jim-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:1` |
| `nixlight` | `A` | `192.168.72.46` |
| `ns1` | `ALIAS` | `river.h.nul.ie.` |
| `ns1.dyn` | `ALIAS` | `river.h.nul.ie.` |
| `ns2` | `ALIAS` | `stream.h.nul.ie.` |
| `ns2.dyn` | `ALIAS` | `stream.h.nul.ie.` |
| `palace` | `A` | `192.168.68.22` |
| `palace` | `AAAA` | `2a0e:97c0:4d0:1::2:1` |
| `palace-core` | `A` | `192.168.64.20` |
| `palace-kvm` | `A` | `192.168.72.21` |
| `reolink-living-room` | `A` | `192.168.72.45` |
| `river` | `A (LUA)` | `generated at query time` |
| `river` | `AAAA` | `2a0e:97c0:4df:0:1::1` |
| `river-core` | `A` | `192.168.64.1` |
| `river-hi` | `A` | `192.168.68.1` |
| `river-hi` | `AAAA` | `2a0e:97c0:4d0:1::1` |
| `river-lo` | `A` | `192.168.72.1` |
| `river-lo` | `AAAA` | `2a0e:97c0:4d0:2::1` |
| `river-ut` | `A` | `192.168.80.1` |
| `river-ut` | `AAAA` | `2a0e:97c0:4d0:3::1` |
| `router-hi` | `A` | `192.168.71.254` |
| `router-hi` | `AAAA` | `2a0e:97c0:4d0:1::ffff` |
| `router-lo` | `A` | `192.168.79.254` |
| `router-lo` | `AAAA` | `2a0e:97c0:4d0:2::ffff` |
| `router-ut` | `A` | `192.168.80.254` |
| `router-ut` | `AAAA` | `2a0e:97c0:4d0:3::ffff` |
| `sfh` | `A` | `192.168.68.81` |
| `sfh` | `AAAA` | `2a0e:97c0:4d0:1::4:2` |
| `shytzel` | `A` | `192.168.64.12` |
| `stream` | `A (LUA)` | `generated at query time` |
| `stream` | `AAAA` | `2a0e:97c0:4df:0:1::2` |
| `stream-core` | `A` | `192.168.64.2` |
| `stream-hi` | `A` | `192.168.68.2` |
| `stream-hi` | `AAAA` | `2a0e:97c0:4d0:1::2` |
| `stream-lo` | `A` | `192.168.72.2` |
| `stream-lo` | `AAAA` | `2a0e:97c0:4d0:2::2` |
| `stream-ut` | `A` | `192.168.80.2` |
| `stream-ut` | `AAAA` | `2a0e:97c0:4d0:3::2` |
| `unifi-ctr` | `A` | `192.168.68.100` |
| `unifi-ctr` | `AAAA` | `2a0e:97c0:4d0:1::5:1` |
| `unifi-ctr-core` | `A` | `192.168.64.21` |
| `ups` | `A` | `192.168.72.20` |
| `vibe` | `A` | `192.168.68.15` |
| `vibe` | `AAAA` | `2a0e:97c0:4d0:1::1:6` |
| `vibe-core` | `A` | `192.168.64.15` |
| `vibe-lo` | `A` | `192.168.72.15` |
| `vibe-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:6` |
| `wave` | `A` | `192.168.72.14` |
| `wave` | `AAAA` | `2a0e:97c0:4d0:2::1:5` |
| `wave-core` | `A` | `192.168.64.14` |
<!-- dns-end -->
### IPv4 reverse zone: `168.192.in-addr.arpa`
<!-- dns: 168.192.in-addr.arpa -->
<!-- dns-start -->
| Address | Name |
|---|---|
| `192.168.64.1` | `river-core.h.nul.ie.` |
| `192.168.64.2` | `stream-core.h.nul.ie.` |
| `192.168.64.20` | `palace-core.h.nul.ie.` |
| `192.168.64.21` | `unifi-ctr-core.h.nul.ie.` |
| `192.168.68.1` | `river-hi.h.nul.ie.` |
| `192.168.68.2` | `stream-hi.h.nul.ie.` |
| `192.168.68.22` | `palace.h.nul.ie.` |
| `192.168.68.40` | `castle.h.nul.ie.` |
| `192.168.68.80` | `cellar.h.nul.ie.` |
| `192.168.68.81` | `sfh.h.nul.ie.` |
| `192.168.68.100` | `unifi-ctr.h.nul.ie.` |
| `192.168.68.103` | `hass-ctr.h.nul.ie.` |
| `192.168.71.254` | `router-hi.h.nul.ie.` |
| `192.168.72.1` | `river-lo.h.nul.ie.` |
| `192.168.72.2` | `stream-lo.h.nul.ie.` |
| `192.168.72.103` | `hass-ctr-lo.h.nul.ie.` |
| `192.168.79.254` | `router-lo.h.nul.ie.` |
| `192.168.80.1` | `river-ut.h.nul.ie.` |
| `192.168.80.2` | `stream-ut.h.nul.ie.` |
| `192.168.80.254` | `router-ut.h.nul.ie.` |
<!-- dns-end -->
### IPv6 reverse zone: `0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa`
<!-- dns: 0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa -->
<!-- dns-start -->
| Address | Name |
|---|---|
| `2a0e:97c0:4d0:1::1` | `river-hi.h.nul.ie.` |
| `2a0e:97c0:4d0:1::2` | `stream-hi.h.nul.ie.` |
| `2a0e:97c0:4d0:1::ffff` | `router-hi.h.nul.ie.` |
| `2a0e:97c0:4d0:1::2:1` | `palace.h.nul.ie.` |
| `2a0e:97c0:4d0:1::3:1` | `castle.h.nul.ie.` |
| `2a0e:97c0:4d0:1::4:1` | `cellar.h.nul.ie.` |
| `2a0e:97c0:4d0:1::4:2` | `sfh.h.nul.ie.` |
| `2a0e:97c0:4d0:1::5:1` | `unifi-ctr.h.nul.ie.` |
| `2a0e:97c0:4d0:1::5:3` | `hass-ctr.h.nul.ie.` |
| `2a0e:97c0:4d0:2::1` | `river-lo.h.nul.ie.` |
| `2a0e:97c0:4d0:2::2` | `stream-lo.h.nul.ie.` |
| `2a0e:97c0:4d0:2::ffff` | `router-lo.h.nul.ie.` |
| `2a0e:97c0:4d0:2::5:3` | `hass-ctr-lo.h.nul.ie.` |
| `2a0e:97c0:4d0:3::1` | `river-ut.h.nul.ie.` |
| `2a0e:97c0:4d0:3::2` | `stream-ut.h.nul.ie.` |
| `2a0e:97c0:4d0:3::ffff` | `router-ut.h.nul.ie.` |
<!-- dns-end -->
+286
View File
@@ -0,0 +1,286 @@
# NixOS module options (`my.*`)
> **Generated** from the module option descriptions by `nix run .#update-docs-options`;
> CI keeps it current. The source of truth is the `mkOpt'` / `mkBoolOpt'` declarations in
> `nixos/modules/` — edit those, not this file. For what each module is *for*, see the
> [shared-modules overview](../architecture.md#shared-modules).
## `borgthin` — [`nixos/modules/borgthin.nix`](../../nixos/modules/borgthin.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.borgthin.enable` | boolean | `false` | Whether to enable borgthin jobs |
| `my.borgthin.jobs` | attribute set of (submodule) | `{ }` | borgthin jobs |
| `my.borgthin.jobs.<name>.archivePrefix` | string | `"${config.networking.hostName}-${name}-"` | Prefix to start new archives with |
| `my.borgthin.jobs.<name>.compression` | string | `"zstd,3"` | Compression options |
| `my.borgthin.jobs.<name>.dateFormat` | string | `"+%Y-%m-%dT%H:%M:%S"` | Format passed to the date command |
| `my.borgthin.jobs.<name>.environment` | attribute set of string | `{ }` | Extra environment variables to pass to borg |
| `my.borgthin.jobs.<name>.extraArgs` | list of string | `[ "--iec" ]` | Extra args to pass to all borg commands |
| `my.borgthin.jobs.<name>.extraCreateArgs` | list of string | `[ ]` | Extra args to pass to tcreate command |
| `my.borgthin.jobs.<name>.lvs` | list of string | `null` | Thin LVs to backup (vg/lv format) |
| `my.borgthin.jobs.<name>.passFile` | null or string | `null` | Path to file containing passphrase |
| `my.borgthin.jobs.<name>.prune.keep` | attribute set of (signed integer or string) | `{ }` | Borg pruning params |
| `my.borgthin.jobs.<name>.prune.pattern` | string | `"sh:${config.archivePrefix}*"` | Borg pattern to select archives for pruning |
| `my.borgthin.jobs.<name>.repo` | string | `null` | borg repository URL |
| `my.borgthin.jobs.<name>.timer.at` | string or list of string | `"5:00"` | systemd calendar time(s) to run backup at |
| `my.borgthin.jobs.<name>.timer.persistent` | boolean | `false` | Persistent systemd timer |
| `my.borgthin.lvmPackage` | package | `<derivation lvm2-2.03.39>` | Packge containing LVM tools |
| `my.borgthin.package` | package | `inputs.borgthin.packages.${system}.borgthin` | borgthin package |
| `my.borgthin.thinToolsPackage` | package | `<derivation thin-provisioning-tools-1.3.2>` | Package containing thin-provisioning-tools |
## `build` — [`nixos/modules/build.nix`](../../nixos/modules/build.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.build.allHardware` | boolean | `false` | Whether to enable a lot of firmware and kernel modules for a wide range of hardware.Only applies to some build targets. |
| `my.build.isDevVM` | boolean | `false` | Whether the system is a development VM. |
| `my.buildAs` | open submodule of lazy attribute set of unspecified value | `{ }` | Attribute set of derivations used to set up the system. |
## `common` — [`nixos/modules/common.nix`](../../nixos/modules/common.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.tmproot.persistence.config.users` | attribute set of (submodule) | `{ }` | A set of user submodules listing the files and directories to link to their respective user's home directories. Each attribute name should be the name of the user. For detailed usage, check the [documentation](https://github.com/nix-community/impermanence). |
## `containers` — [`nixos/modules/containers.nix`](../../nixos/modules/containers.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.containers.instances` | attribute set of (submodule) | `{ }` | Individual containers. |
| `my.containers.instances.<name>.autoStart` | boolean | `true` | Whether to start the container automatically at boot. |
| `my.containers.instances.<name>.bindMounts` | attribute set of (submodule) | `{ }` | An extra list of directories that is bound to the container. |
| `my.containers.instances.<name>.bindMounts.<name>.hostPath` | null or string | `"name"` | Location of the host path to be mounted. |
| `my.containers.instances.<name>.bindMounts.<name>.mountPoint` | string | `"name"` | Mount point on the container file system. |
| `my.containers.instances.<name>.bindMounts.<name>.readOnly` | boolean | `true` | Determine whether the mounted path will be accessed in read-only mode. |
| `my.containers.instances.<name>.containerSystem` | absolute path | `"/nix/var/nix/profiles/system"` | Path to NixOS system configuration from within container. |
| `my.containers.instances.<name>.hotReload` | boolean | `true` | Whether to apply new configuration by running `switch-to-configuration` instead of rebooting the container. |
| `my.containers.instances.<name>.networking.bridge` | null or string | `null` | Network bridge to connect to. |
| `my.containers.instances.<name>.networking.macVLAN` | null or string | `null` | Network interface to make MACVLAN interface from. |
| `my.containers.instances.<name>.system` | absolute path | `"/nix/var/nix/profiles/per-container/name/system"` | Path to NixOS system configuration. |
| `my.containers.persistDir` | string | `"/persist/containers"` | Where to store container persistence data. |
## `deploy-rs` — [`nixos/modules/deploy-rs.nix`](../../nixos/modules/deploy-rs.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.deploy.authorizedKeys.keyFiles` | list of absolute path | `[ .keys/deploy.pub ]` | SSH public key files to add to the default deployment user. |
| `my.deploy.authorizedKeys.keys` | list of (optionally newline-terminated) single-line string | `[ ]` | SSH public keys to add to the default deployment user. |
| `my.deploy.enable` | boolean | `true` | Whether to expose deploy-rs configuration for this system. |
| `my.deploy.generate.containers.enable` | boolean | `true` | Whether to generate deploy-rs profiles for this system's containers. |
| `my.deploy.generate.containers.keepGenerations` | unsigned integer, meaning >=0 | `10` | Number of generations to keep when cleaning up old deployments (0 to disable deletion on deployment). |
| `my.deploy.generate.system.enable` | boolean | `true` | Whether to generate a deploy-rs profile for this system's config. |
| `my.deploy.generate.system.keepGenerations` | unsigned integer, meaning >=0 | `10` | Number of generations to keep when cleaning up old deployments (0 to disable deletion on deployment). |
| `my.deploy.generate.system.mode` | string | `"switch"` | switch-to-configuration mode. |
| `my.deploy.node` | submodule | `{ }` | deploy-rs node configuration. |
| `my.deploy.node.autoRollback` | null or boolean | `null` | Whether to roll back the profile if activation fails. |
| `my.deploy.node.confirmTimeout` | null or 16 bit unsigned integer; between 0 and 65535 (both inclusive) | `null` | Timeout for confirming activation succeeded. |
| `my.deploy.node.fastConnection` | null or boolean | `null` | Whether to copy the whole closure instead of using substitution. |
| `my.deploy.node.hostname` | string | `""` | Hostname deploy-rs will connect to. |
| `my.deploy.node.magicRollback` | null or boolean | `null` | Whether to roll back the profile if connectivity to the deployer is lost. |
| `my.deploy.node.profiles` | attribute set of (submodule) | `{ }` | Profiles to deploy. |
| `my.deploy.node.profiles.<name>.autoRollback` | null or boolean | `null` | Whether to roll back the profile if activation fails. |
| `my.deploy.node.profiles.<name>.confirmTimeout` | null or 16 bit unsigned integer; between 0 and 65535 (both inclusive) | `null` | Timeout for confirming activation succeeded. |
| `my.deploy.node.profiles.<name>.fastConnection` | null or boolean | `null` | Whether to copy the whole closure instead of using substitution. |
| `my.deploy.node.profiles.<name>.magicRollback` | null or boolean | `null` | Whether to roll back the profile if connectivity to the deployer is lost. |
| `my.deploy.node.profiles.<name>.path` | package | `""` | Derivation to build (should include activation script). |
| `my.deploy.node.profiles.<name>.profilePath` | null or string | `null` | Path to profile location |
| `my.deploy.node.profiles.<name>.sshOpts` | list of string | `[ ]` | Options deploy-rs will pass to ssh. Note: overriding at a lower level _merges_ options. |
| `my.deploy.node.profiles.<name>.sshUser` | null or string | `null` | Username deploy-rs will deploy with. |
| `my.deploy.node.profiles.<name>.sudo` | null or string | `null` | Command to elevate privileges with (used if the deployment user != profile user). |
| `my.deploy.node.profiles.<name>.tempPath` | null or string | `null` | Path that deploy-rs will use for temporary files. |
| `my.deploy.node.profiles.<name>.user` | null or string | `null` | Username deploy-rs will deploy with. |
| `my.deploy.node.profilesOrder` | null or (list of string) | `null` | Order to deploy profiles in (remainder will be deployed in arbitrary order). |
| `my.deploy.node.sshOpts` | list of string | `[ ]` | Options deploy-rs will pass to ssh. Note: overriding at a lower level _merges_ options. |
| `my.deploy.node.sshUser` | null or string | `null` | Username deploy-rs will deploy with. |
| `my.deploy.node.sudo` | null or string | `null` | Command to elevate privileges with (used if the deployment user != profile user). |
| `my.deploy.node.tempPath` | null or string | `null` | Path that deploy-rs will use for temporary files. |
| `my.deploy.node.user` | null or string | `null` | Username deploy-rs will deploy with. |
## `dynamic-motd` — [`nixos/modules/dynamic-motd.nix`](../../nixos/modules/dynamic-motd.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.dynamic-motd.enable` | boolean | `true` | Whether to enable the dynamic message of the day PAM module. |
| `my.dynamic-motd.script` | null or strings concatenated with "\n" | `null` | Script that generates message of the day. |
| `my.dynamic-motd.services` | list of string | `[ "login" "sshd" ]` | PAM services to enable the dynamic message of the day module for. |
## `firewall` — [`nixos/modules/firewall.nix`](../../nixos/modules/firewall.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.firewall.enable` | boolean | `true` | Whether to enable the nftables-based firewall. |
| `my.firewall.extraRules` | strings concatenated with "\n" | `""` | Arbitrary additional nftables rules. |
| `my.firewall.nat.enable` | boolean | `true` | Whether to enable IP forwarding and NAT. |
| `my.firewall.nat.externalInterface` | null or string | `null` | The name of the external network interface. |
| `my.firewall.nat.forwardPorts` | (list of (submodule)) or attribute set of list of (submodule) | `[ ]` | IPv4 port forwards |
| `my.firewall.tcp.allowed` | list of (16 bit unsigned integer; between 0 and 65535 (both inclusive) or string) | `[ ]` | TCP ports to open. |
| `my.firewall.trustedInterfaces` | list of string | `[ ]` | Traffic coming in from these interfaces will be accepted unconditionally. Traffic from the loopback (lo) interface will always be accepted. |
| `my.firewall.udp.allowTraceroute` | boolean | `true` | Whethor or not to add a rule to accept UDP traceroute packets. |
| `my.firewall.udp.allowed` | list of (16 bit unsigned integer; between 0 and 65535 (both inclusive) or string) | `[ ]` | UDP ports to open. |
## `gui` — [`nixos/modules/gui`](../../nixos/modules/gui)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.gui.enable` | boolean | `true` | Whether to enable GUI system options. |
## `l2mesh` — [`nixos/modules/l2mesh.nix`](../../nixos/modules/l2mesh.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.vpns.l2.pskFiles` | attribute set of string | `{ }` | PSK files for secured meshes. |
## `librespeed` — [`nixos/modules/librespeed`](../../nixos/modules/librespeed)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.librespeed.backend.enable` | boolean | `false` | Whether to enable librespeed backend. |
| `my.librespeed.backend.extraSettingsFile` | null or string | `null` | Extra settings file. |
| `my.librespeed.backend.settings` | attribute set of unspecified value | `{ }` | Backend settings. |
| `my.librespeed.frontend.servers` | list of attribute set of unspecified value | `{ }` | Server configs. |
| `my.librespeed.frontend.webroot` | package | — | Frontend webroot. |
## `netboot` — [`nixos/modules/netboot`](../../nixos/modules/netboot)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.netboot.client.configurationLimit` | unsigned integer, meaning >=0 | `10` | Max generations to show in boot menu. |
| `my.netboot.client.enable` | boolean | `false` | Whether network booting should be enabled. |
| `my.netboot.server.allowedPrefixes` | list of string | `null` | Prefixes clients should be allowed to connect from (NFS). |
| `my.netboot.server.enable` | boolean | `false` | Whether a netboot server should be enabled. |
| `my.netboot.server.host` | string | `config.networking.fqdn` | Hostname clients should connect to over HTTP / NFS. |
| `my.netboot.server.installer.storeSize` | string | `"16GiB"` | Total allowed writable size of store. |
| `my.netboot.server.instances` | list of string | `[ ]` | Systems to hold boot files for. |
| `my.netboot.server.ip` | string | `null` | IP clients should connect to via TFTP. |
## `nginx-sso` — [`nixos/modules/nginx-sso.nix`](../../nixos/modules/nginx-sso.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.nginx-sso.configuration` | attribute set of unspecified value | `{ }` | nginx-sso configuration. |
| `my.nginx-sso.enable` | boolean | `false` | Whether to enable custom nginx-sso. |
| `my.nginx-sso.extraConfigFile` | null or string | `null` | Path to configuration (e.g. for secrets). |
| `my.nginx-sso.includes.baseURL` | string | `null` | Base URL for redirects. |
| `my.nginx-sso.includes.endpoint` | string | `"http://localhost:8082"` | Upstream for proxied auth requests. |
| `my.nginx-sso.includes.instances` | attribute set of (submodule) | `{ }` | nginx includes instances. |
| `my.nginx-sso.includes.instances.<name>.auth.path` | string | `"/sso-auth"` | HTTP path for SSO auth. |
| `my.nginx-sso.includes.instances.<name>.auth.redirect` | string | `"$scheme://$http_host$request_uri"` | URL to redirect to upon successful login. |
| `my.nginx-sso.includes.instances.<name>.logout.path` | string | `"/sso-logout"` | HTTP path for SSO logout. |
| `my.nginx-sso.includes.instances.<name>.logout.redirect` | string | `"$scheme://$http_host/"` | URL to redirect to upon successful logout. |
| `my.nginx-sso.package` | package | `<derivation nginx-sso-0.27.7>` | nginx-sso package to use. |
## `nvme` — [`nixos/modules/nvme`](../../nixos/modules/nvme)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.nvme.boot.address` | string | `null` | Address of NVMe-oF target. |
| `my.nvme.boot.nqn` | null or string | `null` | NQN to connect to on boot |
| `my.nvme.uuid` | null or string | `null` | NVMe host ID |
## `pdns` — [`nixos/modules/pdns.nix`](../../nixos/modules/pdns.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.pdns.auth.bind.file-records.sshKey` | null or string | `null` | SSH public key for file record update user. |
| `my.pdns.auth.bind.options.also-notify` | list of string | `[ ]` | List of additional address to send DNS NOTIFY messages to. |
| `my.pdns.auth.bind.zones` | attribute set of (submodule) | `{ }` | BIND-style zones definitions. |
| `my.pdns.auth.bind.zones.<name>.also-notify` | list of string | `[ ]` | List of additional address to send DNS NOTIFY messages to. |
| `my.pdns.auth.bind.zones.<name>.masters` | list of string | `[ ]` | List of masters to retrieve data from (as slave). |
| `my.pdns.auth.bind.zones.<name>.path` | absolute path | `null` | Path to zone file. |
| `my.pdns.auth.bind.zones.<name>.template` | boolean | `true` | Whether to run the zone contents through a template for post-processing. |
| `my.pdns.auth.bind.zones.<name>.text` | null or strings concatenated with "\n" | `null` | Inline content of the zone file. |
| `my.pdns.auth.bind.zones.<name>.type` | one of "master", "slave", "native" | `"native"` | Zone type. |
| `my.pdns.auth.enable` | boolean | `false` | Whether to enable PowerDNS authoritative nameserver. |
| `my.pdns.auth.extraSettingsFile` | null or string | `null` | Path to extra settings (e.g. for secrets). |
| `my.pdns.auth.settings` | attribute set of (null or signed integer or string or boolean or absolute path or list of (signed integer or string or boolean or absolute path)) | `{ }` | Authoritative server settings. |
| `my.pdns.recursor.enable` | boolean | `false` | Whether to enable PowerDNS recursive nameserver. |
| `my.pdns.recursor.extraSettingsFile` | null or string | `null` | Path to extra settings (e.g. for secrets). |
## `secrets` — [`nixos/modules/secrets.nix`](../../nixos/modules/secrets.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.secrets.files` | attribute set of unspecified value | `{ }` | Secrets to decrypt with agenix. |
| `my.secrets.key` | null or string | `null` | Public key that secrets for this system should be encrypted for. |
| `my.secrets.vmKeyPath` | string | `"/tmp/xchg/dev.key"` | Path to dev key when in a dev VM. |
## `server` — [`nixos/modules/server.nix`](../../nixos/modules/server.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.server.enable` | boolean | `false` | Whether to enable common configuration for servers. |
## `spdk` — [`nixos/modules/spdk.nix`](../../nixos/modules/spdk.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.spdk.config.subsystems` | attribute set of list of (submodule) | `{ }` | Subsystem config / RPCs. |
| `my.spdk.config.subsystems.<name>.*.method` | string | `null` | RPC method name. |
| `my.spdk.config.subsystems.<name>.*.params` | attribute set of unspecified value | `{ }` | RPC params |
| `my.spdk.debugCommands` | strings concatenated with "\n" | `""` | Commands to run with the spdk-debug script. |
| `my.spdk.enable` | boolean | `false` | Whether to enable SPDK target. |
| `my.spdk.extraArgs` | string | `""` | Extra arguments to pass to spdk_tgt. |
## `tmproot` — [`nixos/modules/tmproot.nix`](../../nixos/modules/tmproot.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.tmproot.enable` | boolean | `true` | Whether to enable tmproot. |
| `my.tmproot.persistence.config` | submodule | `{ }` | Persistence configuration |
| `my.tmproot.persistence.dir` | null or string | `"/persist"` | Path where persisted files are stored. |
| `my.tmproot.size` | string | `"2G"` | Size of tmpfs root |
| `my.tmproot.unsaved.ignore` | list of string | `[ ]` | Path prefixes to ignore if unsaved. |
| `my.tmproot.unsaved.showMotd` | boolean | `true` | Whether to show unsaved files with `dynamic-motd`. |
## `user` — [`nixos/modules/user.nix`](../../nixos/modules/user.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.user.config` | submodule | `{ }` | User definition (as `users.users.*`). |
| `my.user.enable` | boolean | `true` | Whether to create a primary user. |
| `my.user.homeConfig` | Home Manager module | `{ }` | Home configuration (as `home-manager.users.*`) |
| `my.user.passwordSecret` | null or string | `"user-passwd.txt"` | Name of user password secret. |
| `my.user.tmphome` | boolean | `true` | Whether to persist home directory files under tmproot |
## `vms` — [`nixos/modules/vms.nix`](../../nixos/modules/vms.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.vms.instances` | attribute set of (submodule) | `{ }` | VM instances. |
| `my.vms.instances.<name>.autoStart` | boolean | `true` | Whether to start the VM automatically at boot. |
| `my.vms.instances.<name>.boot` | string or (attribute set of unspecified value) convertible to it | `{ menu = "on"; splash-time = 5000; }` | Boot options. |
| `my.vms.instances.<name>.cleanShutdown.enabled` | boolean | `true` | Whether to attempt to cleanly shut down the guest. |
| `my.vms.instances.<name>.cleanShutdown.timeout` | unsigned integer, meaning >=0 | `30` | Clean shutdown timeout (in seconds). |
| `my.vms.instances.<name>.cpu` | string | `"host"` | QEMU CPU model. |
| `my.vms.instances.<name>.drives` | list of (submodule) | `[ ]` | Drives to attach to VM. |
| `my.vms.instances.<name>.drives.*.backend` | string or (attribute set of unspecified value) convertible to it | `{ }` | Backend blockdev options. |
| `my.vms.instances.<name>.drives.*.format` | string or (attribute set of unspecified value) convertible to it | `{ }` | Format blockdev options. |
| `my.vms.instances.<name>.drives.*.formatBackendProp` | string | `"file"` | Property that references the backend blockdev. |
| `my.vms.instances.<name>.drives.*.frontend` | string | `"virtio-blk"` | Frontend device driver. |
| `my.vms.instances.<name>.drives.*.frontendOpts` | string or (attribute set of unspecified value) convertible to it | `{ }` | Frontend device options. |
| `my.vms.instances.<name>.drives.*.name` | string | `null` | Drive name. |
| `my.vms.instances.<name>.enableKVM` | boolean | `true` | Whether to enable KVM. |
| `my.vms.instances.<name>.enableUEFI` | boolean | `true` | Whether to enable UEFI. |
| `my.vms.instances.<name>.hostDevices` | attribute set of (submodule) | `{ }` | Host PCI devices to pass to the VM. |
| `my.vms.instances.<name>.hostDevices.<name>.bindVFIO` | boolean | `true` | Whether to automatically bind the device to vfio-pci. |
| `my.vms.instances.<name>.hostDevices.<name>.extraOptions` | string or (attribute set of unspecified value) convertible to it | `{ }` | Extra QEMU options for the vfio-pci QEMU device. |
| `my.vms.instances.<name>.hostDevices.<name>.hostBDF` | string | `null` | PCI BDF of host device. |
| `my.vms.instances.<name>.hostDevices.<name>.index` | unsigned integer, meaning >=0 | `null` | Index of device in guest (for root port chassis and slot). |
| `my.vms.instances.<name>.machine` | string | `"q35"` | QEMU machine type. |
| `my.vms.instances.<name>.memory` | unsigned integer, meaning >=0 | `1024` | Amount of RAM (mebibytes). |
| `my.vms.instances.<name>.networks` | attribute set of (submodule) | `{ }` | Networks to attach VM to. |
| `my.vms.instances.<name>.networks.<name>.bridge` | null or string | `"name"` | Network bridge to connect to (null to not attach to bridge). |
| `my.vms.instances.<name>.networks.<name>.extraOptions` | string or (attribute set of unspecified value) convertible to it | `{ }` | Extra QEMU options to set for the NIC. |
| `my.vms.instances.<name>.networks.<name>.ifname` | string | `"vm-name"` | TAP device to create / use. |
| `my.vms.instances.<name>.networks.<name>.mac` | string | `null` | Guest MAC address. |
| `my.vms.instances.<name>.networks.<name>.model` | string | `"virtio-net"` | Device type for network interface. |
| `my.vms.instances.<name>.networks.<name>.tapFD` | null or (unsigned integer, meaning >=0) | `null` | FD to use to pass existing TAP device. |
| `my.vms.instances.<name>.networks.<name>.waitOnline` | boolean or string | `true` | Whether to wait for networkd to consider the bridge / existing TAP device online. Pass a string to set the OPERSTATE will wait for. |
| `my.vms.instances.<name>.qemuBin` | absolute path | `"/nix/store/w4yhckm5wyvml3pqw8ai5fl174j14nrb-qemu-host-cpu-only-11.0.0/bin/qemu-kvm"` | Path to QEMU executable. |
| `my.vms.instances.<name>.qemuFlags` | list of string | `[ ]` | Additional flags to pass to QEMU. |
| `my.vms.instances.<name>.smp.cpus` | unsigned integer, meaning >=0 | `1` | Number of CPU cores. |
| `my.vms.instances.<name>.smp.threads` | unsigned integer, meaning >=0 | `1` | Number of threads per core. |
| `my.vms.instances.<name>.spice.enable` | boolean | `true` | Whether to enable SPICE. |
| `my.vms.instances.<name>.uuid` | string | `null` | QEMU machine UUID. |
| `my.vms.instances.<name>.vga` | string | `"virtio"` | VGA card type. |
| `my.vms.ovmfPackage` | package | `<derivation OVMF-202602>` | OVMF package. |
+12
View File
@@ -0,0 +1,12 @@
# Remote boxes
The "remote" group covers the boxes that live outside the `colony` and `home` sites: the two
edge VPSes (`britway` in London, `britnet` in Birmingham) and the `kelder` site — a secondary
server at a remote location, linked back to colony over WireGuard and acting as a NixOS
container host.
| Box | What it is |
| --- | --- |
| [`britway`](britway.md) | Vultr VPS (London, `lon1`): Headscale control plane, Tailscale exit node, BGP edge, nginx |
| [`britnet`](britnet.md) | VPS (Birmingham, `bhx1`): Tailscale exit node / WireGuard hub |
| [`kelder`](kelder/README.md) | Secondary home server (`hentai.engineer`): container host, Samba, DDNS (containers on its page) |
+41
View File
@@ -0,0 +1,41 @@
# britnet
A small VPS in Birmingham (`bhx1`) acting as a second Tailscale/WireGuard egress point — a
narrower gateway role than [`britway`](britway.md) (no control plane, no BGP).
- **Source:** [`nixos/boxes/britnet.nix`](../../nixos/boxes/britnet.nix)
- **Host:** VPS (Birmingham, `bhx1`; provider uplink assignment `allhost`)
- **nixpkgs:** `mine`
## Platform
| Component | Allocation |
|---|---|
| Virtualisation | KVM/QEMU guest |
| Compute | 2 vCPUs and 2 GiB RAM |
| Storage | 32 GiB virtio disk with separate ext4 filesystems for `/boot`, `/nix` and `/persist`; root is tmpfs |
## Role
- **Tailscale exit node** — logs into the headscale on [`britway`](britway.md)
(`--login-server=https://hs.nul.ie`) with `--advertise-exit-node`.
- **WireGuard hub** — `wg0` listens on UDP 51820 on the `vpn` assignment, with a single static
peer.
- **NAT gateway** — traffic arriving on `tailscale0`/`wg0` is forwarded out `veth0` and SNATed
to the `allhost` v4/v6 addresses.
## Network assignments
See the consolidated [network assignments](../networking.md#box-assignments) table (this box: `britnet`).
## Networking
- The provider interface is renamed to `veth0` by MAC. Its IPv6 default gateway sits off-subnet, so
a link-scope route is added to reach it.
- `wg0` is a networkd WireGuard netdev keyed from `britnet/wg.key`; RA is disabled on it.
- Upstream DNS is hardcoded to Cloudflare (`1.1.1.1` / `1.0.0.1`).
- `iperf3` runs with an open port for bandwidth testing.
## Notable config files
- [`nixos/boxes/britnet.nix`](../../nixos/boxes/britnet.nix) — the whole box (single file).
+54
View File
@@ -0,0 +1,54 @@
# britway
A Vultr VPS in London (`lon1`) acting as the network edge node: the self-hosted Tailscale
control plane, a tailnet exit node, and the BGP speaker for AS211024.
- **Source:** [`nixos/boxes/britway/`](../../nixos/boxes/britway)
- **Host:** VPS at Vultr (London, `lon1`)
- **nixpkgs:** `mine`
## Platform
| Component | Allocation |
|---|---|
| Virtualisation | Vultr VC2 virtual guest on a QEMU-compatible platform |
| Compute | 2 vCPUs and 2 GiB RAM |
| Storage | 65 GiB virtio disk with separate ext4 filesystems for `/boot`, `/nix` and `/persist`; root is tmpfs |
## Role
- **Headscale** — the self-hosted Tailscale control plane at `hs.nul.ie`; every other box's
`tailscaled` logs in here (`--login-server=https://hs.nul.ie`). Google OIDC for auth,
SQLite state, MagicDNS under `ts.nul.ie`, and split DNS pointing the colony/home domains
at their internal resolvers.
- **Tailscale exit node** — advertises `--advertise-exit-node` plus routes to the home v4/v6
prefixes; tailnet traffic is SNATed out `veth0` (v4 to the Vultr public IP, v6 to the
`as211024` mesh address).
- **BGP edge** — `bird2` speaks BGP as AS211024 to Vultr transit (AS64515, separate v4/v6
sessions authenticated with a password from `britway/bgp-password-vultr.conf`) and exports
everything to a `bgp.tools` monitoring session. It originates the internal, colony and home
IPv6 prefixes documented in [networking](../networking.md#domains).
- **nginx** — reverse proxy fronting headscale (`hs.nul.ie``localhost` headscale port),
with a wildcard ACME cert for `nul.ie` issued via Cloudflare DNS.
## Network assignments
See the consolidated [network assignments](../networking.md#box-assignments) table (this box: `britway`).
## Networking
- Two assignments: `vultr` on the provider interface `veth0` (renamed by MAC), and `as211024`
on the `l2mesh` VXLAN interface (`my.vpns.l2`) — member of the shared mesh (see
[The AS211024 L2 mesh](../networking.md#the-as211024-l2-mesh)).
- Static routes steer colony/home v4 traffic over the `as211024` mesh. A separate `ts-extra`
routing table with a policy rule on `tailscale0` ingress sends Tailscale-sourced v6
traffic for colony via `estuary`, while the box's own v6 uses WAN.
- The firewall trusts the `as211024` prefixes (`lib.my.c.as211024.nftTrust`) and
`tailscale0`; `iperf3` runs with an open port for bandwidth testing.
## Notable config files
- [`nixos/boxes/britway/default.nix`](../../nixos/boxes/britway/default.nix) — system, assignments, networkd, firewall/SNAT.
- [`nixos/boxes/britway/bgp.nix`](../../nixos/boxes/britway/bgp.nix) — `bird2` config (Vultr transit, `bgp.tools`).
- [`nixos/boxes/britway/nginx.nix`](../../nixos/boxes/britway/nginx.nix) — nginx vhosts + ACME.
- [`nixos/boxes/britway/tailscale.nix`](../../nixos/boxes/britway/tailscale.nix) — headscale + the tailnet node itself.
+61
View File
@@ -0,0 +1,61 @@
# kelder
Secondary home server at a remote site, domain `hentai.engineer`. Linked back to colony over
WireGuard and acting as a NixOS container host (like `shill`/`sfh`).
- **Source:** [`nixos/boxes/kelder/`](../../../nixos/boxes/kelder)
- **Host:** physical (Intel; LTS kernel, `kvm-intel`, IOMMU on)
- **nixpkgs:** `mine`
## Role
- **Container host** — runs two NixOS containers on the `ctrs` bridge
(`my.containers.instances`): `kelder-acquisition` and `kelder-spoder` (below).
- **Public services via colony** — a WireGuard tunnel (`estuary` netdev) connects to colony's
`estuary` box, which DNATs public traffic to kelder's tunneled assignment; connection-mark-based
policy routing sends replies back through the tunnel while ordinary traffic uses the LAN.
kelder's own NAT forwards `http`/`https` on to `kelder-spoder`.
- **Nextcloud host** — served from the `kelder-spoder` container.
- **Samba** — the `storage` share backed by `/mnt/storage`, with `nmbd` and `samba-wsdd` for
Windows discovery.
- **DDNS** — a `ddns-update` timer runs `dns_update.py` periodically to sync the
`hentai.engineer` and `kelder-local.hentai.engineer` Cloudflare records with the address on
`et1g0`.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `kelder`).
## Containers
| Container | Role |
| --- | --- |
| [`kelder-acquisition`](containers/kelder-acquisition.md) | Media stack (Transmission over AirVPN, Jackett/Radarr/Sonarr, Jellyfin) |
| [`kelder-spoder`](containers/kelder-spoder.md) | Nextcloud + nginx reverse proxy |
The containers are not deploy targets (`my.deploy.enable = false`); they're managed through
the host.
## Networking
- LAN on `et1g0` (renamed by MAC) with DHCP and MTU 1460 (`lib.my.c.kelder.ipv4MTU`); the
kelder v4 prefixes are masqueraded out of it.
- The `estuary` WireGuard peer is combined with rules that keep LAN traffic on the main table and
only route tunnel-marked or owned traffic through the tunnel's dedicated table.
## Services
- `netdata` (proxied as `monitor.hentai.engineer` by `kelder-spoder`), `smartd`, `fstrim`,
LVM thin provisioning.
- `minecraft-server` is present but **disabled** (`enable = false`); the firewall still opens
25565 tcp/udp.
- Primary user `kontent` (in the `storage`/`media` groups).
- Sets `system.nixos.distroName = "KelderOS"`, a custom Plymouth theme and an `amogus-beep`
boot jingle ([`boot.nix`](../../../nixos/boxes/kelder/boot.nix)).
## Notable config files
- [`nixos/boxes/kelder/default.nix`](../../../nixos/boxes/kelder/default.nix) — system, assignments, tunnel, NAT, containers.
- [`nixos/boxes/kelder/boot.nix`](../../../nixos/boxes/kelder/boot.nix) — Plymouth theme + boot beep.
- [`nixos/boxes/kelder/containers/`](../../../nixos/boxes/kelder/containers) — the two container definitions.
- [`nixos/boxes/kelder/dns_update.py`](../../../nixos/boxes/kelder/dns_update.py) — Cloudflare DDNS script.
@@ -0,0 +1,45 @@
# kelder-acquisition
The media acquisition stack for the kelder site — Transmission (forced over VPN), the *arrs
and Jellyfin in one NixOS container.
- **Source:** [`nixos/boxes/kelder/containers/acquisition/`](../../../../nixos/boxes/kelder/containers/acquisition)
- **Host:** NixOS container on [`kelder`](../README.md)
- **nixpkgs:** `mine`
## Role
### Transmission
`transmission_4` is bound to the VPN wait-online unit and uses AirVPN's forwarded peer port. Upload,
download and seed-ratio limits are configured in the service source. Downloads use
`/mnt/media/downloads/torrents`, backed by the host's `/mnt/storage/media`.
### Media services
Jackett, Radarr and Sonarr share the `media` group with a group-writable umask. Jellyfin uses the
host's bind-mounted `/dev/dri` with `intel-vaapi-driver` / `intel-ocl`; its user belongs to
`render`.
## Network assignments
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `kelder-acquisition`).
## Networking
- `internal` assignment (name `acquisition-ctr`) on the host's `ctrs` bridge, MTU 1460 to
match the site WAN.
- All non-site traffic goes over an AirVPN WireGuard tunnel (`vpn` netdev, AirVPN IE endpoint).
Policy rules keep traffic to and from the kelder prefixes on the main table and push everything
else through the VPN's dedicated table.
- An nftables input chain drops new TCP connections from the VPN interface except the
Transmission peer port; the web UI ports (9091 Transmission, 9117 Jackett, 7878 Radarr,
8989 Sonarr, 8096 Jellyfin) are accepted from the site. When built as a dev VM, those ports
are forwarded to the host.
- Sonarr still needs the EOL .NET 6 runtime, allowed via
`nixpkgs.config.permittedInsecurePackages`.
## Notable config files
- [`nixos/boxes/kelder/containers/acquisition/default.nix`](../../../../nixos/boxes/kelder/containers/acquisition/default.nix) — services and users.
- [`nixos/boxes/kelder/containers/acquisition/networking.nix`](../../../../nixos/boxes/kelder/containers/acquisition/networking.nix) — AirVPN tunnel + firewall.
@@ -0,0 +1,50 @@
# kelder-spoder
The web container for the kelder site: Nextcloud plus an nginx (OpenResty) reverse proxy for
the site's services.
- **Source:** [`nixos/boxes/kelder/containers/spoder/`](../../../../nixos/boxes/kelder/containers/spoder)
- **Host:** NixOS container on [`kelder`](../README.md)
- **nixpkgs:** `mine`
## Role
- **Nextcloud** (`nextcloud32`) at `cloud.hentai.engineer` (trusted alias
`cloud-local.hentai.engineer`), SQLite backend, data in `/mnt/storage/nextcloud`
(`/mnt/storage` is bind-mounted from the host).
- **nginx reverse proxy** (`openresty`) terminating TLS for the site's public vhosts, with a
wildcard ACME cert for `hentai.engineer` via Cloudflare DNS. The kelder host forwards
`http`/`https` to this container; uploads are unlimited (`clientMaxBodySize = 0`) for
Nextcloud's sake.
## Network assignments
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `kelder-spoder`).
## Proxy vhosts
All under `hentai.engineer`, each with a `*-local` alias:
| vhost | Target | Auth |
| --- | --- | --- |
| `monitor` | `netdata` on the kelder host (:19999) | `htpasswd` |
| `kontent` | Jellyfin on `kelder-acquisition` (:8096, incl. websocket) | — |
| `torrents` | Transmission on `kelder-acquisition` (:9091) | `htpasswd` |
| `jackett` | Jackett on `kelder-acquisition` (:9117) | `htpasswd` |
| `radarr` | Radarr on `kelder-acquisition` (:7878) | `htpasswd` |
| `sonarr` | Sonarr on `kelder-acquisition` (:8989) | `htpasswd` |
| `cloud` | Nextcloud (local) | — |
An `init_worker_by_lua` timer polls `v4.ident.me` periodically to track the site's public IP; it
feeds `localRedirect` rewrites (bounce public-IP clients to the `*-local` name) that
are currently **disabled** (commented out — Virgin Media filters DNS answers containing local
IPs, so the split doesn't work as intended).
## Networking
- `internal` assignment (name `spoder-ctr`) on the host's `ctrs` bridge, MTU 1420.
## Notable config files
- [`nixos/boxes/kelder/containers/spoder/default.nix`](../../../../nixos/boxes/kelder/containers/spoder/default.nix) — Nextcloud + ACME.
- [`nixos/boxes/kelder/containers/spoder/nginx.nix`](../../../../nixos/boxes/kelder/containers/spoder/nginx.nix) — reverse proxy vhosts.
+32
View File
@@ -0,0 +1,32 @@
# colony
The hosted dedicated server in Amsterdam (`ams1`) and the public-facing half of
the boxes: almost everything reachable from the internet lives here.
- **Internal domain:** `ams1.int.nul.ie` (`lib.my.c.colony.domain`)
- **Public domain:** `nul.ie` — public services are published as `*.nul.ie`
- **Source:** [`nixos/boxes/colony/`](../../../nixos/boxes/colony)
## Networking
`colony` separates the host, VMs, `shill` containers and `whale2` OCI workloads onto dedicated
networks behind [`estuary`](estuary.md), which terminates the public addressing. The canonical
prefixes and routing overview are in the [`colony` section of networking.md](../../networking.md#colony).
## Boxes
| Box | Role |
|---|---|
| [`colony`](colony.md) | Physical VM host (AMD, KVM, LVM-thin, `borgthin` backups → rsync.net) |
| [`estuary`](estuary.md) | Edge router: WAN, firewall/NAT, DNS, BGP (AS211024), WireGuard |
| [`shill`](shill/README.md) | NixOS container host (most applications; per-container pages under `shill/`) |
| [`whale2`](whale2.md) | podman/OCI game-server host |
| [`git`](git.md) | Gitea + Gitea Actions runner |
| [`mail`](mail.md) | Debian VM running mailcow (not NixOS) |
| [`darts`](darts.md) | Third-party/customer VM (not NixOS) |
The applications running on `shill` are listed on its own page — see
[shill/README.md](shill/README.md#containers).
`mail` and `darts` are host-defined VMs whose guest operating systems are managed out of band; their
pages document only what this repository controls.
+110
View File
@@ -0,0 +1,110 @@
# colony
The physical dedicated server in Amsterdam (`ams1`) and the VM host for
everything at the colony site.
- **Source:** [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
(VM instances in [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix))
- **Host:** bare metal (this *is* the physical box)
- **nixpkgs:** `mine-stable`
## Hardware
| Component | Inventory |
|---|---|
| Platform | ASRock Rack X570D4U server board |
| CPU | AMD Ryzen 9 5950X (16 cores / 32 threads) |
| Memory | 128 GiB |
| NVMe storage | Three 2 TB Samsung SSD 980 PRO devices providing the NVMe-backed LVM thin pool and data LVs |
| Bulk storage | Three 12 TB WD120EDBZ disks and one 18 TB WD180EDGZ disk for the bulk LVM volumes |
| Boot | SanDisk USB device holding the EFI system partition |
| Network / management | Two Intel I210 Gigabit Ethernet controllers, one passed through to `estuary`; ASPEED BMC graphics and console |
## Role
Bare-metal AMD host. It does little application work itself — its job is to run
the VMs and provide them with storage, networking and backups.
### Virtualisation
The `my.vms` module drives QEMU/KVM with `kvm-amd` and IOMMU enabled. Each `my.vms.instances` entry
becomes a `vm@<name>` systemd service with UEFI, QMP/monitor sockets under `/run/vms/<name>/`, TAP
networking and optional `hostDevices` passthrough through `vfio-pci`. `estuary` receives the WAN NIC
this way.
### Storage
All three NVMe SSDs and all four SATA HDDs are physical volumes in one `main` VG. Separate pools
and standalone RAID LVs keep workloads on the appropriate media:
| Layer | Physical layout | Main consumers |
|---|---|---|
| `nvme-tpool` | Thin-pool data is RAID 0 across the three NVMe SSDs; thin metadata is RAID 1 on two of them. The data itself has no redundancy. | Host `/nix` and `/persist`; VM system and persistence disks; fast data LVs such as `minio`, `oci`, `git`, `gitea-actions-cache`, `nix-cache` and `jam` |
| `hdds-tpool` | Thin-pool data is RAID 5 across the three 12 TB HDDs; thin metadata is RAID 1 on two NVMe SSDs. | `media`, passed to `shill`, and `backup`, mounted by the host at `/mnt/backup` |
| `darts-media` | Standalone RAID 5 LV spanning all four HDDs. | Bulk storage passed to `darts` |
| `darts-ext` | Standalone linear LV using the remaining capacity of the 18 TB HDD. | Expansion storage passed to `darts` |
`media` and `backup` are thin-provisioned, so their virtual capacities are not additive physical
capacity and overcommit `hdds-tpool`. VM disks built with `lib.my.vm.disk` are named
`vm-<name>-<disk>` in `main`; `lib.my.vm.lvmDisk` attaches the named data LVs.
The initrd activates only `colony-nix` and `colony-persist`. This avoids checking and activating all
of the storage before switching root; `lvm-activate-main.service` activates the remaining LVs before
local filesystems and VMs need them.
### Backups
`my.borgthin` job `main` snapshots host and VM persistence/data LVs into `/mnt/backup/main`.
`borgthin-rsync.service` copies the repository to rsync.net and `rsync-lvm-meta.service` sends the
LVM metadata; both run at idle priority after the Borg job.
### Monitoring
Netdata uses FreeIPMI while ignoring the VCCM sensor. The box also runs `smartd` with logs under
`/var/log/smartd`, `rasdaemon`, and `fstrim` before the backup job.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `colony`).
## Networking
- Two bridges: `base` (the colony base network, shared with `estuary`) and
`vms` (the VM network). Dummy interfaces (`base0`, `vms0`) keep the bridges
up in networkd's eyes so dependent VMs can start.
- `colony` sends RAs on `vms` (DNS = `estuary`'s base address) and carries
static routes for the downstream prefixes: `ctrs` via `shill`, `oci` via
`whale2`, plus the Tailscale, `qclk` and `jam` prefixes via `shill`.
- `estuary` is the default gateway (via the `base` bridge); `colony`'s own public-facing address is
its `internal` assignment (alt name `vm`).
- The customer VMs attach to dedicated TAP devices (`vm-mail`, `vm-darts`)
which are **not** bridged: networkd puts the point-to-point /32
(`lib.my.c.colony.custRouting`) and the customer's IPv6 /64 on each, sends
RAs, and link-routes the customer's public /32 down the tap.
- `my.firewall` trusts the `vms` bridge, DNATs the shared
`lib.my.c.colony.firewallForwards` list for traffic addressed to `estuary`'s
public IP (so the port forwards also work from inside), and forwards the
customer prefixes through with minimal filtering ("trust for now").
## VMs
Declared in `my.vms.instances` (`cpus`/`threads` are QEMU `smp` values):
| VM | Cores | Threads | Memory | Network | Disks |
|---|---|---|---|---|---|
| `estuary` | 2 | 2 | 3 GiB | `base` | `esp` / `nix` / `persist` LVs + WAN NIC passthrough |
| `shill` | 12 | 2 | 40 GiB | `vms` | `esp` / `nix` / `persist` + `media` / `minio` / `nix-cache` / `jam` LVs |
| `whale2` | 8 | 2 | 16 GiB | `vms` | `esp` / `nix` / `persist` + `oci` LV |
| `git` | 12 | 2 | 40 GiB | `vms` | `esp` / `nix` / `persist` / `oci` + `git` / `gitea-actions-cache` LVs |
| `mail` | 3 | 2 | 6 GiB | `vm-mail` tap | `root` / `data` LVs |
| `darts` | 4 | 2 | 16 GiB | `vm-darts` tap | `root` LV + `darts-media` / `darts-ext` LVs |
`estuary`, `shill`, `whale2` and `git` are NixOS systems with their own pages
(see the [README](README.md#boxes)); [`mail`](mail.md) and [`darts`](darts.md) have host-side
definitions here, but their guest operating systems are managed out of band.
## Notable config files
- [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix) — host hardware, networkd, firewall, backups.
- [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix) — `my.vms.instances` for all six VMs.
- [`nixos/modules/vms.nix`](../../../nixos/modules/vms.nix) — the `my.vms` module itself.
+37
View File
@@ -0,0 +1,37 @@
# darts
An opaque third-party/customer VM. Declared in `colony`'s `my.vms.instances`
but **not a NixOS system**: this repo knows nothing about what runs inside it
and doesn't manage it.
- **Source (host-side only):** the `darts` instance in
[`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix)
and the `90-vm-darts` network in
[`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
- **Host:** VM on `colony`
- **nixpkgs:** not applicable (unmanaged guest; host-side definition uses `colony`'s `mine-stable`)
## Role
- Customer/dedicated VM, left alone beyond hosting and connectivity.
- A thin-provisioned `root` LV on the NVMe pool, plus `darts-media` (RAID 5 across the four HDDs)
and `darts-ext` (linear storage on the 18 TB HDD) in the `main` VG. See
[`colony`'s storage layout](colony.md#storage).
## Network assignments
This guest is not a NixOS system, so its host-routed addresses are not rows in the generated
[network assignments](../../networking.md#box-assignments) table.
- Same customer-VM pattern as [`mail`](mail.md): dedicated unbridged TAP
(`vm-darts`), point-to-point address
(`custRouting.darts-vm`) on the host side, link-routed public /32
`94.142.242.255`, and the IPv6 /64 `2a0e:97c0:4d2:2001::/64` with RAs.
- DNS: `darts-cust.ams1.int.nul.ie`. Like the other customer prefixes, its
inbound traffic is accepted by `estuary` without per-port filtering and
forwarded on by `colony`.
## Notable config files
- [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix) — VM definition.
- [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix) — host-side network and routing.
+166
View File
@@ -0,0 +1,166 @@
# estuary
The colony edge router and firewall — the box that holds colony's public IPs
and connects everything else at the site to the internet.
- **Source:** [`nixos/boxes/colony/vms/estuary/`](../../../nixos/boxes/colony/vms/estuary)
(`default.nix`, `bgp.nix`, `dns.nix`, `bandwidth.nix`)
- **Host:** VM on `colony` (gets the WAN NIC by PCI passthrough)
- **nixpkgs:** `mine`
## Role
- **Edge routing / firewall / NAT** — owns the colony public IPv4/IPv6 assignments, NATs outbound
traffic, and port-forwards inbound services (see [Firewall and NAT](#firewall-and-nat)).
- **DNS** — PowerDNS authoritative server *and* recursor (see [DNS](#dns)).
- **BGP** — BIRD2 speaking AS211024 with upstreams, IXP route servers and
direct peers (see [BGP](#bgp)).
- **VPNs** — the `as211024` L2 VXLAN mesh plus three point-to-point WireGuard
tunnels (see [VPNs](#vpns)).
- **Misc** — `iperf3` server, netdata.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `estuary`).
## WAN and IXP VLANs
- `wan` — the passed-through `igb` NIC (9000 MTU, enlarged rings). It carries
the plain upstream uplink (static v4/v6 with gateways from the `internal`
assignment) plus the tagged `ifog` VLAN.
- `ifog` (VLAN 409) is an iFog QinQ transport that carries the IXP VLANs as
nested tags:
| Interface | VLAN | IPv4 | IPv6 | Purpose |
|---|---|---|---|---|
| `frys-ix` | 701 | `185.1.160.196/23` | `2001:7f8:10f::3:3850:196/64` | Frys-IX peering LAN |
| `nl-ix` | 1845 | `193.239.116.145/22` | `2001:7f8:13::a521:1024:1/64` | NL-ix peering LAN |
| `fogixp` | 1147 | `185.1.147.159/24` | `2001:7f8:ca:1::159/64` | FogIXP peering LAN |
| `ifog-transit` | 702 | — | `2a0c:9a40:100f:370::2/64` | iFog IPv6 transit |
The IXP interfaces run at 1500 MTU with DHCP/RA/LLDP off; an nftables `ixp`
chain rejects non-IP/ARP ethertypes in both directions.
- `base` — colony base network; sends RAs and serves DNS to the site, and
routes the `vms`/`ctrs`/`oci`, Tailscale, `qclk`, `vip*` and customer
prefixes back via `colony`.
- `as211024` — the L2 mesh interface (see VPNs).
## Firewall and NAT
`my.firewall` (nftables). Inbound port forwards (`my.firewall.nat.forwardPorts`,
driven by the shared `lib.my.c.colony.firewallForwards` list):
| Service | Forwarded to |
|---|---|
| HTTP/S, Matrix federation | `middleman` |
| Git | `git` |
| Game servers | OCI servers on `whale2`, `gam` |
| Tailscale | `waffletail` |
| `qclk` WireGuard | `qclk` |
Besides the forwards, `extraRules` defines:
- `routing-tcp` / `routing-udp` chains — the inbound allow-list for new
connections from `wan`/`as211024`/IXPs towards internal services (SSH
anywhere, otherwise per-service v4/v6 rules mirroring `firewallForwards`).
- `filter-routing` — applied to `wan`/`as211024`/IXPs → `base` forwards;
customer prefixes (`mail`/`darts` v4, `cust.v6`) are accepted wholesale, the
rest goes through the `routing-*` chains.
- SNAT: everything from `prefixes.all.v4` leaving non-`as211024` interfaces is
NATed to the public IP; the WireGuard tunnel prefixes get their own SNAT
addresses.
- DNS redirect: DNS traffic arriving at estuary's own public addresses is
redirected to port 5353 (the authoritative server) — see below.
## DNS
Both halves are PowerDNS ([`dns.nix`](../../../nixos/boxes/colony/vms/estuary/dns.nix)). The live
forward and reverse records are listed in the generated [DNS reference](../../reference/dns.md).
### Authoritative
`my.pdns.auth`, listening on `0.0.0.0:5353` / `[::]:5353`. Primary for
`ams1.int.nul.ie`, `100.10.in-addr.arpa` and the `2a0e:97c0:4d2::/48` reverse
zone.
- Zone contents are largely generated from `allAssignments`
(`lib.my.dns.fwdRecords` / `ptrRecords` / `ptr6Records`); `ALIAS` records
(with `expand-alias`) point the zone apex at estuary itself.
- AXFR is allowed to HE.net's secondary and the trusted internal/site-egress sources used by the
generated DNS reference.
- `_acme-challenge` is a LUA `TXT` record answered from a file (DNS-01 issuance).
- Reached publicly via the NAT redirect of port 53 → 5353; the `base` side also
accepts DNS directly.
### Recursor
`my.pdns.recursor` (`pdns-recursor`), listening on localhost and the `base`
addresses, serving `prefixes.all` and the Tailscale prefixes.
- Authoritative zones are forwarded back to `127.0.0.1:5353` (with NOTIFY
support, so changes show up immediately).
- A small Lua `preresolve` hook rewrites `nix-cache.nul.ie`
`http.ams1.int.nul.ie` so cache traffic stays on-site.
## BGP
BIRD2 ([`bgp.nix`](../../../nixos/boxes/colony/vms/estuary/bgp.nix)) speaking **AS211024**:
| Peer | ASN | Role | Where / notes |
|---|---|---|---|
| ColoClue | AS8283 | Upstream | `euNetworks` 2/3, v4+v6 |
| iFog | AS34927 | Upstream | IPv6 transit |
| Hurricane Electric | AS6939 | Upstream | IPv6 over Frys-IX |
| Frys-IX | AS56393 | IXP route server | |
| NL-ix | AS34307 | IXP route server | lower preference |
| FogIXP | AS47498 | IXP route server | |
| LUJE.net | AS212855 | Direct peer | ColoClue/Frys-IX/FogIXP + multihop labs |
| jurrian | AS212635 | Direct peer | |
| Meta | AS32934 | Direct peer | Frys-IX/NL-ix |
| Cloudflare | AS13335 | Direct peer | Frys-IX |
| Apple | AS714 | Direct peer | NL-ix |
| HE | AS6939 | Direct peer | Frys-IX (v4) |
| bgp.tools | AS212232 | Monitoring | multihop collector, `add paths` |
Statics originate the site prefixes: the `vip*` ranges via `base`, the
ams1/internal v6 prefixes via `base`/`as211024`, and the home /48 towards the
home routers' `as211024` VIP.
Currently disabled (commented out): efero transit over FogIXP ("not working so
well lately") and the NL-ix Cloudflare sessions.
## VPNs
### `as211024` L2 mesh
Member alongside `river`/`stream`/`britway` (`my.vpns.l2`, the `l2mesh` module).
The mesh transport, crypto and addressing are shared fabric — see
[The AS211024 L2 mesh](../../networking.md#the-as211024-l2-mesh) in networking.md.
### WireGuard tunnels
Point-to-point tunnels terminated here as networkd `wireguard` netdevs (private
keys from agenix); each SNATs out its own interface address:
| Tunnel | Port | Prefix | Notes |
|---|---|---|---|
| `kelder` | 51820 | — | to the remote `kelder` site; kelder's public `estuary` assignment is routed over the tunnel |
| `hillcrest` | 51822 | `prefixes.hillcrest.v4` (`10.100.5.0/30`) | /32 pair, estuary `.1` ↔ remote `.2` |
| `john-valorant` | 51823 | `prefixes.john-valorant.v4` (`10.100.5.4/30`) | /32 pair, estuary `.1` ↔ remote `.2` |
## Bandwidth management
[`bandwidth.nix`](../../../nixos/boxes/colony/vms/estuary/bandwidth.nix) implements a WAN shaper: a
token-bucket filter on `wan` (outbound) and on an `ifb-wan` IFB device that ingress traffic is
mirrored into (inbound), with
[`bandwidth.py`](../../../nixos/boxes/colony/vms/estuary/bandwidth.py) as a `bandwidth-limiter`
service that watches utilisation and can adjust the configured rate. **Currently disabled** — the
file is not in estuary's `imports` (only `dns.nix` and `bgp.nix` are), so no shaping is applied.
## Notable config files
- [`nixos/boxes/colony/vms/estuary/default.nix`](../../../nixos/boxes/colony/vms/estuary/default.nix) — system, networkd, firewall, WireGuard, mesh membership.
- [`nixos/boxes/colony/vms/estuary/bgp.nix`](../../../nixos/boxes/colony/vms/estuary/bgp.nix) — BIRD2 config.
- [`nixos/boxes/colony/vms/estuary/dns.nix`](../../../nixos/boxes/colony/vms/estuary/dns.nix) — PowerDNS auth + recursor.
- [`nixos/boxes/colony/vms/estuary/bandwidth.nix`](../../../nixos/boxes/colony/vms/estuary/bandwidth.nix) — WAN shaper (disabled, not imported).
+61
View File
@@ -0,0 +1,61 @@
# git
The Gitea VM — source hosting and CI for the boxes (`git.nul.ie`).
- **Source:** [`nixos/boxes/colony/vms/git/`](../../../nixos/boxes/colony/vms/git)
(`default.nix`, `gitea.nix`, `gitea-actions.nix`)
- **Host:** VM on `colony`
- **nixpkgs:** `mine`
## Role
### Gitea
The Git forge at `git.nul.ie` (self-registration disabled), configured in
[`gitea.nix`](../../../nixos/boxes/colony/vms/git/gitea.nix).
- Backed by PostgreSQL on [`colony-psql`](shill/containers/colony-psql.md) (waited on via
`lib.my.systemdAwaitPostgres`).
- LFS enabled; all object storage (incl. LFS and packages) is on MinIO at `s3.nul.ie` (bucket
`gitea`, on [`object`](shill/containers/object.md)) — the secret is spliced into `app.ini` at
startup.
- Mail goes out via `mail.nul.ie`, including the issue-reply incoming-mail poller.
### Gitea Actions runner
One Docker-mode instance (`main-docker`) on podman (privileged, `podman` network), configured in
[`gitea-actions.nix`](../../../nixos/boxes/colony/vms/git/gitea-actions.nix).
- Labels for `node:24-trixie` and the self-built `git.nul.ie/dev/actions-ubuntu:26.04` images.
- Runs as a fixed `gitea-runner` user (not `DynamicUser`) so it can read its token; jobs have a
configured timeout.
- The action cache lives on a dedicated disk (`/var/cache/gitea-runner`).
- Executes the repo's own `.gitea/workflows/ci.yaml`.
### nginx
Terminates TLS for `git.nul.ie` (and a default vhost) and proxies to Gitea on `:3000`. ACME
(Let's Encrypt, production) issues `nul.ie` + `*.nul.ie` via the Cloudflare DNS-01 challenge.
### podman
Local container backend for the runner; `/var/lib/containers` is an XFS data disk, and the default
`10.88.0.0/16` podman subnet is allowed to forward.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `git`).
## Storage
- `/var/lib/gitea` — the `git` LV (repositories, config).
- `/var/cache/gitea-runner` — the `gitea-actions-cache` LV.
- `/var/lib/containers` — the `oci` LV (XFS with project quotas). Despite the
name this is local to the `git` VM and unrelated to `whale2`'s `oci`
network.
## Notable config files
- [`nixos/boxes/colony/vms/git/default.nix`](../../../nixos/boxes/colony/vms/git/default.nix) — VM config, nginx + ACME, podman, firewall.
- [`nixos/boxes/colony/vms/git/gitea.nix`](../../../nixos/boxes/colony/vms/git/gitea.nix) — Gitea itself.
- [`nixos/boxes/colony/vms/git/gitea-actions.nix`](../../../nixos/boxes/colony/vms/git/gitea-actions.nix) — the Actions runner.
+45
View File
@@ -0,0 +1,45 @@
# mail
A Debian VM running [mailcow](https://mailcow.email/) (`mail.nul.ie`) — the
mail server for `nul.ie`. Declared in `colony`'s `my.vms.instances` but **not
a NixOS system**: everything inside the VM is configured out of band.
- **Source (host-side only):** the `mail` instance in
[`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix)
and the `90-vm-mail` network in
[`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
- **Host:** VM on `colony`
- **nixpkgs:** not applicable (Debian guest; host-side definition uses `colony`'s `mine-stable`)
## Role
- Runs the full mailcow stack (Postfix/Dovecot/SOGo/Rspamd) for `nul.ie`.
Other colony services send through it as `mail.nul.ie` (e.g. Gitea, and the
disabled Mastodon config).
- `root` and `data` LVM disks; the `vm-mail-data` LV is included in `colony`'s `borgthin` backups.
## Network assignments
This guest is not a NixOS system, so its host-routed addresses are not rows in the generated
[network assignments](../../networking.md#box-assignments) table.
### Link and addressing
The VM attaches to a dedicated, unbridged TAP (`vm-mail`). `colony` puts the point-to-point
`custRouting.mail-vm` address on the host side, link-routes public
`94.142.241.227/32` down the TAP, and advertises `2a0e:97c0:4d2:2000::/64` with RAs.
- DNS: `mail-vm.ams1.int.nul.ie` (and `mail.nul.ie` publicly, incl. the PTR in
estuary's reverse zone). `estuary` accepts traffic to the customer prefixes
without per-port filtering; `colony` forwards it on ("trust for now").
## Notes
- ACME certificates are issued on `middleman` and pushed to the VM over SSH
(`acme@mail.nul.ie mailcow-ssl-reload`, key `middleman/mailcow-ssh.key`);
the VM's SSH host key is pinned at `.keys/mail-vm-host.pub`.
## Notable config files
- [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix) — VM definition.
- [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix) — host-side network and routing.
+84
View File
@@ -0,0 +1,84 @@
# shill
The colony NixOS container host — most colony applications run as
`systemd-nspawn` containers on this VM.
- **Source:** [`nixos/boxes/colony/vms/shill/`](../../../../nixos/boxes/colony/vms/shill)
(`default.nix`, `containers-ext.nix`, `containers/`)
- **Host:** VM on `colony`
- **nixpkgs:** `mine`
## Role
### Container hosting
`my.containers.instances` runs the colony containers on the `ctrs` bridge. Each is a full NixOS
system rendered through `my.asContainer` and deployed as a profile on `shill`; containers are not
standalone deploy targets. The shared container module supplies the nspawn units, `/persist` and
store binds.
### Shared storage
LVM-backed host volumes are bind-mounted into the consumers: `/mnt/media` is read-only in
`middleman` and read-write in `jackflix`; `/mnt/minio` and `/mnt/nix-cache` are read-write in
`object`.
### Routing
`shill` routes between `vms` and `ctrs`, advertises `estuary` as DNS on `ctrs`, and routes Tailscale
through `waffletail` and the `qclk` prefix through `qclk`. It applies the shared `firewallForwards`
DNAT for `estuary`'s public IP; a connection-mark-based SNAT rule keeps replies symmetric.
### Host tuning
The box has a larger conntrack table and ephemeral-port range for high connection counts. Netdata
listens on port 19999.
## Network assignments
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `shill`).
## Containers
Defined under
[`shill/containers/`](../../../../nixos/boxes/colony/vms/shill/containers) and
wired up in `shill`'s `my.containers.instances`. The generated
[network assignments](../../../networking.md#box-assignments) table is the source of truth for
their current addresses. Each container has its own page:
| Container | Role |
|---|---|
| [`middleman`](containers/middleman.md) | Reverse proxy, ACME, nginx-sso, librespeed |
| [`vaultwarden`](containers/vaultwarden.md) | Password manager |
| [`colony-psql`](containers/colony-psql.md) | Shared PostgreSQL (14) |
| [`chatterbox`](containers/chatterbox.md) | Matrix Synapse + bridges |
| [`jackflix`](containers/jackflix.md) | Media stack |
| [`object`](containers/object.md) | MinIO, Harmonia Nix cache, Sharry, HedgeDoc, wastebin |
| [`toot`](containers/toot.md) | Bluesky PDS (Mastodon disabled) |
| [`waffletail`](containers/waffletail.md) | Tailscale subnet router / exit node |
| [`qclk`](containers/qclk.md) | WireGuard management appliance |
| [`gam`](containers/gam.md) | Terraria server |
### `jam`
A one-off: [`containers-ext.nix`](../../../../nixos/boxes/colony/vms/shill/containers-ext.nix)
runs a raw `systemd-nspawn` container (not a `my.containers` instance, not
NixOS) with its root on the `jam` LV, private user namespaces and a `ve-jam`
veth. It gets the `jam` customer prefix (`prefixes.jam`, `jam-cust` in DNS)
and SSH is forwarded to it from `shill`'s public IP port 60022.
## Notes
- `nix.settings.substituters` is forced to just `https://cache.nixos.org`
`shill` sits next to the S3 cache on `object`, so it doesn't use it.
- [`hercules.nix`](../../../../nixos/boxes/colony/vms/shill/hercules.nix)
(Hercules CI agent + the `nix-cache-gc` timer for the S3 binary cache)
exists but is **currently disabled**: the file is not imported by
`shill/default.nix`.
## Notable config files
- [`nixos/boxes/colony/vms/shill/default.nix`](../../../../nixos/boxes/colony/vms/shill/default.nix) — VM config, networkd, firewall, `my.containers.instances`.
- [`nixos/boxes/colony/vms/shill/containers/default.nix`](../../../../nixos/boxes/colony/vms/shill/containers/default.nix) — container imports.
- [`nixos/boxes/colony/vms/shill/containers-ext.nix`](../../../../nixos/boxes/colony/vms/shill/containers-ext.nix) — the `jam` nspawn container.
- [`nixos/modules/containers.nix`](../../../../nixos/modules/containers.nix) — the `my.containers` module.
@@ -0,0 +1,41 @@
# chatterbox
The Matrix homeserver for `nul.ie` (Synapse) and its bridges to other chat networks.
[middleman](middleman.md) fronts it as `matrix.nul.ie` for clients and on `:8448` for
federation.
- **Source:** [`shill/containers/chatterbox.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/chatterbox.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
## Role
### Synapse
`matrix-synapse` serves `nul.ie` at `https://matrix.nul.ie`, with Element at `element.nul.ie`.
Port 8008 carries client and federation traffic with forwarded headers, while a localhost manhole
uses port 9000. Registration and guest access are disabled; uploads, dynamic thumbnails and URL
previews are enabled, with preview fetching restricted to [middleman](middleman.md).
### Bridges
The box runs heisenbridge for IRC, `mautrix-whatsapp`, and two `mautrix-meta` instances for
Messenger and Instagram. The mautrix bridges use [colony-psql](colony-psql.md), default to
end-to-end encryption, and share the secret `doublepuppet.yaml` registration for double puppeting
onto `nul.ie`. The firewall admits ports 8008 and 8009 in addition to netdata.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `chatterbox`).
## Notes
- Synapse's real database config lives in the `chatterbox/synapse.yaml` age secret — options
only merge at the top level, so the base config carries a dummy `sqlite3` block to satisfy
the module defaults. The signing key is also an age secret.
- `olm-3.2.16` is allowed via `permittedInsecurePackages` (a nixpkgs E2EE library issue).
- The bridge services get `ffmpeg` on their `PATH` for GIF→video conversion.
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/chatterbox.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/chatterbox.nix) — container definition, Synapse settings and all bridge configuration
@@ -0,0 +1,39 @@
# colony-psql
The shared PostgreSQL instance for colony services. Rather than each service running its own
database, the containers (and the `git` VM) connect here over the `ctrs` network.
- **Source:** [`shill/containers/colony-psql.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/colony-psql.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
## Role
- **PostgreSQL 14** with TCP/IP enabled, reachable from the whole colony address space using `md5`
authentication. The firewall allows `5432`.
- Local `peer` auth maps `postgres`, `root`, `netdata` and `dev` to the `postgres` superuser via
the ident map.
- **netdata** with the Python PostgreSQL collector.
- Consumers wait for the database to accept connections with the `lib.my.systemdAwaitPostgres`
helper (e.g. `sharry`, `atticd`, `mastodon-init-db`, and `middleman`'s nginx as a DNS
bootstrap hack).
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `colony-psql`).
The assignment also has the alt name `colony-psql` (no `-ctr` suffix), which is what consumers
use as the database hostname.
## Consumers
- [object](object.md) — `sharry` and `hedgedoc` (and `atticd` when enabled) over
`colony-psql:5432`
- [toot](toot.md) — Mastodon's database (Mastodon currently disabled)
- [chatterbox](chatterbox.md) — the mautrix bridges (WhatsApp, Messenger, Instagram) via
Postgres URIs in their secret env files
- `git` VM — Gitea
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/colony-psql.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/colony-psql.nix) — container definition and PostgreSQL configuration
+29
View File
@@ -0,0 +1,29 @@
# gam
A game-server container currently dedicated to Terraria.
- **Source:** [`shill/containers/gam.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/gam.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
## Role
Runs lightweight game servers directly as NixOS services, rather than as OCI containers on
[`whale2`](../../whale2.md).
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `gam`).
## Terraria server
`services.terraria` runs the dedicated server with its world at
`/var/lib/terraria/NotWorld.wld`. It creates large worlds automatically, uses the MOTD
"sup gamers", and disables UPnP. Additional settings such as the password come from the
`gam/terraria.conf` age secret used as the service configuration file.
`openFirewall` is enabled, and `estuary` forwards TCP and UDP port `7777` to the container.
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/gam.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/gam.nix) — container definition and the Terraria service
@@ -0,0 +1,56 @@
# jackflix
The media stack — acquisition, library, streaming and photos. Torrent traffic is routed through
an AirVPN WireGuard tunnel so downloads only flow while the VPN is up.
- **Source:** [`shill/containers/jackflix/`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix)
(`default.nix`, `networking.nix`)
- **Host:** NixOS container on [`shill`](../README.md) (bind-mounts `/mnt/media` read-write)
- **nixpkgs:** `mine`
## Role
| Service | Port | Purpose |
| --- | --- | --- |
| Jellyfin | `8096` | streaming, published as `jackflix.nul.ie` |
| Transmission | `9091` | BitTorrent client (`transmission_4`), published as `torrents.nul.ie` (SSO) |
| Jackett | `9117` | indexer aggregator, `jackett.nul.ie` (SSO) |
| FlareSolverr | — | Cloudflare challenge solver for Jackett |
| Radarr | `7878` | movies, `radarr.nul.ie` (SSO) |
| Sonarr | `8989` | TV, `sonarr.nul.ie` (SSO) |
| Jellyseerr (`seerr`) | `5055` | request portal, `gib.nul.ie` (`openFirewall` on) |
| PhotoPrism | `2342` | photos, `photos.nul.ie`; password auth, sqlite DB, originals/import under `/mnt/media/photoprism` |
| copyparty | `3923` | file sharing, `stuff.nul.ie`; serves `/mnt/media/public` (read-only to everyone) and `/priv``/mnt/media/stuff` (admin for `dev`), share creation, indexing (`e2dsa`/`e2t`), file-magic checks |
All published through [middleman](middleman.md) as shown. A shared `media` group plus a group-writable
umask on Radarr/Sonarr gives the apps coordinated access to the media volume.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `jackflix`).
## VPN download path
[`networking.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix/networking.nix)
defines a `vpn` WireGuard netdev to **AirVPN NL** using a key and PSK from age secrets:
- Policy routing keeps colony traffic on the main table while everything else falls through to the
VPN's dedicated table, so the services stay reachable on the `ctrs` network while outbound
torrent traffic exits via AirVPN. `DNSDefaultRoute` is disabled on `host0`; the VPN provides DNS.
- `transmission` and `jackett` `bindsTo` `systemd-networkd-wait-online@vpn.service` — they only
run while the tunnel is up.
- AirVPN forwards a peer port to Transmission (`peer-port`); the firewall accepts it and drops
other new inbound TCP from `vpn`, while non-VPN input is limited to the service ports
(netdata, Transmission, Jackett, Radarr, Sonarr, Jellyfin, PhotoPrism) plus copyparty's `3923`
from the base config and Jellyseerr's `5055`.
## Storage
Media lives on the shared `/mnt/media` volume (bind-mounted read-write from `shill`); Transmission
downloads into `/mnt/media/downloads/torrents` with a `.incomplete` directory and configured
bandwidth and seed-ratio limits.
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/jackflix/default.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix/default.nix) — container definition and the media services
- [`nixos/boxes/colony/vms/shill/containers/jackflix/networking.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix/networking.nix) — AirVPN WireGuard netdev, policy routing and VPN firewall rules
@@ -0,0 +1,96 @@
# middleman
The front-end reverse proxy for the colony's public web services — the single ingress that
`estuary` DNATs HTTP/HTTPS (and Matrix federation on `:8448`) to. Terminates TLS with wildcard
certificates it issues itself, provides nginx-sso for gated vhosts, and runs a librespeed
backend.
- **Source:** [`shill/containers/middleman/`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman)
(`default.nix`, `vhosts.nix`)
- **Host:** NixOS container on [`shill`](../README.md) (`my.containers` ephemeral nspawn on the
`ctrs` bridge; bind-mounts `/mnt/media` read-only for the static file vhosts)
- **nixpkgs:** `mine`
## Role
### nginx
The reverse proxy enables `vts`, `fancyindex`, Brotli, kTLS and a proxy cache. Its dynamic resolver
points at `estuary`, allowing upstreams named under `ams1.int.nul.ie` to resolve again at runtime.
All vhosts live in
[`vhosts.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix). nginx
also waits for `colony-psql` through `systemdAwaitPostgres`, avoiding an early-boot DNS stall.
### ACME
`middleman` issues certificates for its own vhosts; it is not a shared CA for other boxes.
- `ams1.int.nul.ie` and its wildcard use a lego `exec` challenge that SSHes to
`pdns-file-records@estuary-vm`. This is the default `useACMEHost` certificate internally.
- `nul.ie`, `*.nul.ie` and `*.s3.nul.ie` use Cloudflare DNS. A `postRun` hook copies renewed
material to the `mail` VM and runs `mailcow-ssl-reload` there.
- Renewal reloads nginx; the `acme` group owns the secret files and includes the nginx user.
### nginx-sso
The `generic` SSO instance at `sso.nul.ie` uses Google OAuth by default and also offers a simple
username/password provider. Its cookie domain is `.nul.ie`; gated vhosts include the generated
`server-generic.conf` / `location-generic.conf` snippets from `/etc/nginx/includes/sso/`.
### librespeed
The frontend and backend are published as `speed.nul.ie` and `librespeed.ams1.int.nul.ie`, both
proxied to `localhost:8989`.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `middleman`).
The firewall allows `http`, `https` and `8448` (Matrix federation). A small nftables SNAT rule
rewrites outbound IPv6 to the container's own address on `host0`.
## Published vhosts
Everything is under `*.nul.ie` with the public wildcard cert unless noted; defaults applied to
all vhosts are `onlySSL`, kTLS and HTTP/2. "SSO" = gated behind nginx-sso (`generic` instance).
| Host | Upstream | Notes |
| --- | --- | --- |
| `nul.ie` (`_`, default) | static | landing page (`index.html`, CV PDF, SSH pubkey); serves Matrix `.well-known`s and redirects `webfinger`/`nodeinfo`/`host-meta``toot.nul.ie`, `atproto-did``pds.nul.ie`; `forceSSL` (plain HTTP redirects to HTTPS) |
| `localhost` | — | loopback-only VTS status page at `/status` (scraped by netdata); plain HTTP |
| `sso.nul.ie` | `localhost:8082` | nginx-sso endpoint |
| `netdata-colony.nul.ie` | `<host>.ams1.int.nul.ie:19999` | netdata fan-out over `vm`, `fw`, `ctr`, `oci`, `http`, `jackflix-ctr`, `chatterbox-ctr`, `colony-psql-ctr`; **SSO** |
| `pass.nul.ie` | `vaultwarden-ctr:8080` | [vaultwarden](vaultwarden.md); `/notifications/hub` proxied with websockets |
| `matrix.nul.ie` | `chatterbox-ctr:8008` | [chatterbox](chatterbox.md) Synapse client + federation; also listens on `:8448` as federation `default_server`; `= /` redirects to Element; serves Matrix `.well-known`s |
| `element.nul.ie` | static `element-web` | Element configured for the `nul.ie` homeserver |
| `torrents.nul.ie` | `jackflix-ctr:9091` | Transmission ([jackflix](jackflix.md)); **SSO** |
| `jackett.nul.ie` | `jackflix-ctr:9117` | **SSO** |
| `radarr.nul.ie` | `jackflix-ctr:7878` | **SSO**; websockets |
| `sonarr.nul.ie` | `jackflix-ctr:8989` | **SSO**; websockets |
| `gib.nul.ie` | `jackflix-ctr:5055` | Jellyseerr requests |
| `jackflix.nul.ie` | `jackflix-ctr:8096` | Jellyfin; `/socket` websockets; `/` redirects to `/web/` |
| `toot.nul.ie` | `toot-ctr:80` | Mastodon — **upstream currently disabled**, see [toot](toot.md) |
| `pds.nul.ie` | `toot-ctr:3000` | Bluesky PDS ([toot](toot.md)); websockets |
| `share.nul.ie` | `object-ctr:9090` | Sharry ([object](object.md)); websockets |
| `stuff.nul.ie` | `jackflix-ctr:3923` | copyparty |
| `public.nul.ie` (+ alias `p.nul.ie`) | static `/mnt/media/public` | fancyindex file listing; `addSSL` so plain HTTP also works |
| `mc-map.nul.ie` | `simpcraft-oci:8100` | Minecraft map (OCI container on [`whale2`](../../whale2.md#game-servers)) |
| `mc-rail.nul.ie` | `simpcraft-oci:3876` | Minecraft railway map ([`whale2`](../../whale2.md#game-servers)) |
| `mc-map-kink.nul.ie` | `kinkcraft-oci:8100` | Minecraft map ([`whale2`](../../whale2.md#game-servers)) |
| `speed.nul.ie` | `localhost:8989` | librespeed |
| `librespeed.ams1.int.nul.ie` | `localhost:8989` | librespeed on the internal domain (internal wildcard cert) |
| `md.nul.ie` | `object-ctr:3000` | HedgeDoc; websockets |
| `pb.nul.ie` | `object-ctr:8088` | wastebin |
| `photos.nul.ie` | `jackflix-ctr:2342` | PhotoPrism; websockets |
| `pront.nul.ie` | `stream-hi.h.nul.ie:5000` | OctoPrint on the home network ([`stream`](../../../home/stream.md)); `/webcam/``:5050`; **SSO** |
| `hass.nul.ie` | `hass-ctr.h.nul.ie:8123` | [Home Assistant](../../../home/sfh/containers/hass.md) (home network); websockets |
| `hass-john.nul.ie` | `john-valorant-tun.ams1.int.nul.ie:8123` | remote Home Assistant over the point-to-point tunnel; websockets |
| `minio.nul.ie` | `object-ctr:9001` | MinIO console; `/ws` websockets |
| `s3.nul.ie` (+ `*.s3.nul.ie`) | `object-ctr:9000` | MinIO S3 API (virtual-host style via the `*.s3` wildcard cert); `/gitea/packages/` has a hack forcing the correct `Content-Type` for Docker image manifests |
| `nix-cache.nul.ie` | `object-ctr:5000` | Harmonia Nix binary cache; `.narinfo`/`nar/`/`serve/` paths get immutable `Cache-Control`/`Expires` headers |
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/middleman/default.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman/default.nix) — container definition: nginx, ACME, nginx-sso, librespeed, secrets
- [`nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix) — all virtual hosts, the SSO include helpers, and the `.well-known` tree
- [`nixos/boxes/colony/vms/shill/containers/middleman/default.html`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman/default.html) — default vhost landing page
@@ -0,0 +1,39 @@
# object
Object storage and the Nix binary cache, plus a few small self-hosted web apps (Sharry,
HedgeDoc, wastebin).
- **Source:** [`shill/containers/object.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/object.nix)
- **Host:** NixOS container on [`shill`](../README.md) (bind-mounts `/mnt/minio` and
`/mnt/nix-cache` read-write)
- **nixpkgs:** `mine`
## Role
| Service | Port | Purpose |
| --- | --- | --- |
| MinIO | `9000` (S3) / `9001` (console) | S3-compatible object storage, `s3.nul.ie` + `*.s3.nul.ie` (virtual-host style via `MINIO_DOMAIN`), console at `minio.nul.ie`; region `eu-central-1`; data on the `/mnt/minio` XFS volume |
| Harmonia | `5000` | Nix binary cache at `nix-cache.nul.ie``harmonia-dev` cache serves `shill`'s `/nix/store` out of a dedicated store view rooted at `/var/lib/harmonia` (bind-mounted from `/mnt/nix-cache`), signed with the `nix-cache.key` secret; a `harmonia` user with authorized keys exists for cache pushes |
| Sharry | `9090` | file sharing at `share.nul.ie`; Postgres on [colony-psql](colony-psql.md), files stored in the `share` MinIO bucket; fixed `dev` account + invite signup; mail via `mail.nul.ie`; configured share-size limit |
| HedgeDoc | `3000` | collaborative markdown notes at `md.nul.ie`; Postgres on [colony-psql](colony-psql.md); anonymous edits but no anonymous notes, email login, no open email registration |
| wastebin | `8088` | pastebin at `pb.nul.ie` |
| atticd | `8069` | **currently disabled** (`services.atticd.enable = false`) — an alternative Nix cache that would store locally and sit behind `nix-cache.nul.ie`; config (including the `object/atticd.env` secret) is kept around |
Everything public is fronted by [middleman](middleman.md) (see its vhost table). The
`minio-client` is installed and the user's `~/.mc/config.json` points at an age-secret config.
`minio-2025-10-15T17-29-55Z` is allowlisted via `permittedInsecurePackages` (flagged as a TODO).
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `object`).
## Backing services
- [colony-psql](colony-psql.md) — Sharry and HedgeDoc databases (atticd too, when enabled).
- MinIO buckets back other boxes' services: Gitea LFS/packages (with the `middleman` MIME hack
for Docker manifests), Mastodon's `mastodon` bucket and the Bluesky PDS `pds` bucket on
[toot](toot.md), and Sharry's `share` bucket.
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/object.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/object.nix) — container definition and all services
@@ -0,0 +1,34 @@
# qclk
A WireGuard management appliance for the `qclk` network — it terminates the `management`
tunnel and routes/NATs the `qclk` prefix. No service daemon is currently defined in the config;
the container provides the network plumbing and opens the API port.
- **Source:** [`shill/containers/qclk/`](../../../../../nixos/boxes/colony/vms/shill/containers/qclk)
(`default.nix`)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
## Role
- **WireGuard `management` interface** — listens on UDP `51821` (`lib.my.c.colony.qclk.wgPort`,
allowed through the firewall; `estuary` port-forwards it here) with the private key from the
`qclk/wg.key` age secret. Managed devices are static peers, each pinned to its own address in the
`qclk` prefix; the peer list currently has a single entry.
- **Routing/NAT** — `shill` routes the `qclk` prefix to this container, and outbound traffic from
`host0` into `management` is
SNATed to the container's `qclk` address. Forwarding into `management` is accepted from the
AS211024 trusted IPv4 ranges (`lib.my.c.as211024.trusted.v4`).
- **API port** — TCP `8080` is accepted on the `management` interface (`apiPort`), but note
`services = { }`: whatever serves the qclk API is not defined in this configuration today.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `qclk`).
Two assignments: `internal` on the `ctrs` network like the other containers, and `qclk` on the
`management` WireGuard interface (IPv4 only, no DNS name).
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/qclk/default.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/qclk/default.nix) — container definition: WireGuard netdev, peer list and firewall rules
@@ -0,0 +1,47 @@
# toot
Federated-social container. Despite the name, the only service actually running is a **Bluesky
PDS** — the Mastodon instance ("toots") is **currently disabled**.
- **Source:** [`shill/containers/toot.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/toot.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
## Role
### Bluesky PDS
The active service listens on port 3000 as `pds.nul.ie`, fronted by [middleman](middleman.md),
which also redirects `/.well-known/atproto-did` here. It requires invites, applies an upload-size
limit, and stores blobs in [object](object.md)'s `pds` MinIO bucket. Federation uses the standard
Bluesky services and crawlers. `toot/pds.env` supplies secrets, including S3 credentials, and mail
comes from `pds@nul.ie`.
### Mastodon
Mastodon is disabled with `services.mastodon.enable = false`, but its configuration remains:
- `nul.ie` is the local domain and `toot.nul.ie` the web domain.
- PostgreSQL runs on [colony-psql](colony-psql.md), with local Redis and SMTP through `mail.nul.ie`.
- Media uses the `mastodon` MinIO bucket, a configured streaming-process pool, and periodic cleanup.
- [middleman](middleman.md) still proxies the dead vhost and `.well-known` endpoints.
The removed `otpSecretFile` option must be addressed before the service can return.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `toot`).
The firewall allows `http` (the Mastodon nginx vhost) and the PDS port `3000` besides netdata.
## Notes
- The local nginx still carries the Mastodon virtual host (`toot.nul.ie`) with proxy-header
overrides for being behind `middleman` — part of the preserved-but-disabled Mastodon setup.
- `mastodon-init-dirs` appends the S3 secret key to Mastodon's `.secrets_env` (the module has no
option for a secret-key file), and `mastodon-init-db` waits for `colony-psql` — moot while
Mastodon is disabled.
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/toot.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/toot.nix) — container definition; active PDS config and the preserved (disabled) Mastodon config
@@ -0,0 +1,33 @@
# vaultwarden
[Vaultwarden](https://github.com/dani-garcia/vaultwarden), a Bitwarden-compatible password
manager, published as `pass.nul.ie` through [middleman](middleman.md).
- **Source:** [`shill/containers/vaultwarden.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/vaultwarden.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
## Role
- **vaultwarden** — HTTP on `[::]:8080`, WebSocket notifications on `3012` (both allowed through
the firewall). Web vault enabled, signups disabled, Bitwarden push notifications enabled
(`PUSH_ENABLED`). `DOMAIN` is `https://pass.nul.ie`.
- **SMTP** via `mail.nul.ie:587` (STARTTLS) as `pass@nul.ie`; credentials and other sensitive
settings come from the `vaultwarden/config.env` age secret.
- **Backups** — a `borgbackup` job pushes `/var/lib/vaultwarden` to rsync.net
using a repokey-encrypted repository; its passphrase and SSH key come from secrets, and the job
keeps daily, weekly and monthly archives according to its configured retention policy.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `vaultwarden`).
## Persistence
`/var/lib/vaultwarden` is persisted through `my.tmproot.persistence` — like the other
`shill` containers the root is ephemeral and real state lives under `/persist` (bind-mounted
from the host's `/persist/containers/vaultwarden`).
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/vaultwarden.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/vaultwarden.nix) — container definition, service config and the borgbackup job
@@ -0,0 +1,34 @@
# waffletail
The colony Tailscale node: a subnet router and exit node that advertises the colony prefixes
into the tailnet.
- **Source:** [`shill/containers/waffletail.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/waffletail.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
## Role
### Tailscale
The node authenticates to `hs.nul.ie` with the secret `tailscale-auth.key`. It disables Tailscale's
netfilter management, advertises itself as an exit node and advertises the colony IPv4/IPv6 ranges,
but does not accept routes. UDP port 41641 is open and forwarded here by `estuary`.
### Routing and firewall
`shill` routes the Tailscale prefixes to this container.
The repository's nftables rules trust `tailscale0`, permit colony-sourced forwarding into the
tailnet, and SNAT tailnet traffic leaving through `host0` unless its destination is already within
colony.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `waffletail`).
Two assignments: `internal` on the `ctrs` network like the other containers, and `tailscale` for
its addresses on the tailnet itself (no DNS name).
## Notable config files
- [`nixos/boxes/colony/vms/shill/containers/waffletail.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/waffletail.nix) — container definition, Tailscale setup and forward/NAT rules
+80
View File
@@ -0,0 +1,80 @@
# whale2
The colony podman/OCI host, dedicated to game servers (kept off `shill` so
container churn and resource use stay isolated).
- **Source:** [`nixos/boxes/colony/vms/whale2/`](../../../nixos/boxes/colony/vms/whale2)
(`default.nix`, `valheim.nix`, `minecraft/`, `enshrouded.nix`)
- **Host:** VM on `colony`
- **nixpkgs:** `mine`
## Role
### Container runtime
OCI containers run under podman (`virtualisation.oci-containers`) with the netavark backend and
`firewall_driver = "none"`, leaving firewall management to `my.firewall`.
### Routable game servers
Each server gets an address from `extraAssignments` (`valheim-oci`, `simpcraft-oci`, …) on the
`colony` netavark network. That network is backed by the `oci` interface and `prefixes.oci` v4/v6
ranges; `lib.my.dockerNetAssignment` supplies the address through `--network=colony:ip=…`.
`estuary` forwards the public game ports, while IPv6 reaches the containers directly.
### Storage
`/var/lib/containers` is a dedicated XFS disk with project quotas.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `whale2`).
## Game servers
The OCI containers are documented here (they have no pages of their own).
Their per-container `extraAssignments` on the `oci` network are listed in the generated
[network assignments](../../networking.md#box-assignments) table. Ports below are the public ones
forwarded by `estuary`.
| Container | Ports | Status |
|---|---|---|
| `valheim` | `2456-2457`/udp | running |
| `simpcraft` | `25565` tcp+udp | running |
| `simpcraft-staging` | `25566` tcp | **disabled** (commented out) |
| `enshrouded` | `15636-15637`/udp | **disabled** (`enshrouded.nix` not imported) |
| `kevcraft` | `25567` tcp+udp | running |
| `kinkcraft` | `25568` tcp+udp | running |
| `graeme` | `25569` tcp+udp | running |
- **valheim** ([`valheim.nix`](../../../nixos/boxes/colony/vms/whale2/valheim.nix)) —
`lloesche/valheim-server`, public server "amogus sus", world `simpland2`,
allow-listed Steam IDs, password from agenix.
- **simpcraft** ([`minecraft/`](../../../nixos/boxes/colony/vms/whale2/minecraft)) —
`itzg/minecraft-server` (self-built `git.nul.ie/dev/craftblock` image),
Modrinth "Simpcraft" modpack, whitelist + ops.
- **simpcraft-staging** — the same setup pinned to an older pack version, currently commented out.
- **kevcraft** — vanilla Minecraft 1.20.1, extra op.
- **kinkcraft** — same Simpcraft modpack as `simpcraft`.
- **graeme** — vanilla Minecraft on hard difficulty with its own whitelist.
- **enshrouded** ([`enshrouded.nix`](../../../nixos/boxes/colony/vms/whale2/enshrouded.nix)) —
`sknnr/enshrouded-dedicated-server` ("UWUshrouded"); the file exists but is
commented out of `whale2`'s `imports`, so the server is down (its forwards
and DNS records remain).
The Minecraft containers share one whitelist/ops list and agenix env file
(`whale2/simpcraft.env`, which also carries the RCON password).
## Backups
A local borg job (`services.borgbackup.jobs.simpcraft`) archives the `simpcraft` world frequently,
offset from its autosave timer, into `/var/lib/containers/backup/simpcraft`. It uses `mcrcon` to
`save-off`/`save-on` around each run and keeps a short history for quick world rollback rather than
disaster recovery; the `oci` LV itself is covered by `colony`'s `borgthin`.
## Notable config files
- [`nixos/boxes/colony/vms/whale2/default.nix`](../../../nixos/boxes/colony/vms/whale2/default.nix) — VM config, podman/netavark setup, `extraAssignments`.
- [`nixos/boxes/colony/vms/whale2/valheim.nix`](../../../nixos/boxes/colony/vms/whale2/valheim.nix) — Valheim server.
- [`nixos/boxes/colony/vms/whale2/minecraft/default.nix`](../../../nixos/boxes/colony/vms/whale2/minecraft/default.nix) — the Minecraft servers + world backup job.
- [`nixos/boxes/colony/vms/whale2/enshrouded.nix`](../../../nixos/boxes/colony/vms/whale2/enshrouded.nix) — Enshrouded server (disabled, not imported).
+45
View File
@@ -0,0 +1,45 @@
# Home site
The home network (domain `h.nul.ie`): a redundant pair of routers in front of a VM host, an
NVMe-oF storage target, an IoT container host, and a workstation. The two routers — `river` (a VM)
and `stream` (a physical box) — are built from one shared
[`routing-common`](../../../nixos/boxes/home/routing-common) definition as an active/backup VRRP
pair, and everything clients touch (gateway, DNS) is a floating VIP that follows the master.
- **Source:** [`nixos/boxes/home/`](../../../nixos/boxes/home)
## Boxes
| Box | Role | Host |
|---|---|---|
| [`palace`](palace.md) | VM host | physical |
| [`river`](river.md) | Primary router (VRRP pair with `stream`) | VM on `palace` |
| [`stream`](stream.md) | Secondary router (VRRP pair with `river`) | physical |
| [`cellar`](cellar.md) | NVMe-oF / SPDK storage target | VM on `palace` |
| [`sfh`](sfh/README.md) | NixOS container host (containers on its page) | VM on `palace` |
| [`castle`](castle.md) | Workstation / gaming desktop | physical |
## Router VIPs
Clients use per-VLAN floating VIPs as their gateway and DNS server; `keepalived` moves them between
`river` and `stream`. The addresses, DHCP/RA behavior and failover mechanics are documented once in
[Router VIPs](../../networking.md#router-vips) and [Router HA](../../networking.md#router-ha).
## Networks
The site separates core management, high-MTU trusted traffic, general trusted traffic, untrusted
clients and the two WAN paths. VLAN IDs, prefixes, MTUs and router addressing live in the canonical
[`home` section of networking.md](../../networking.md#home).
## Switch fabric
The boxes hang off three hand-configured switches — `jim` and `dave` (MikroTik, RouterOS) and
`brian` (Ubiquiti, UniFi) — which are **not** managed by this flake. The physical topology, VLAN
map, the Digiweb WAN path (trunked VLAN 10 + PVID 140 at the ONT edge), and the multi-ONT plan are
documented in [switches.md](switches.md).
## Wireless APs
The Wi-Fi APs — `vibe` (MikroTik cAP ax) and `wave` (Cudy AX3000 on OpenWrt) — are dumb APs, also
**not** managed by this flake. The shared VLAN-trunk design, SSIDs, per-AP management addressing,
and the OpenWrt flash/config for `wave` are in [aps.md](aps.md).
+151
View File
@@ -0,0 +1,151 @@
# Home wireless APs
Reference for the home Wi-Fi access points. Like the switches ([switches.md](switches.md)), these
are **not** managed by this flake — they are configured on-device (RouterOS on the MikroTik,
OpenWrt/UCI on the Cudy). This file documents the shared VLAN/trunk design and each AP.
Only the DNS records live in the flake
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix),
`h.nul.ie` zone). Everything else here is applied by hand on the device.
## The APs
| | vibe | wave |
|---|---|---|
| Model | MikroTik **cAP ax** (`cAPGi-5HaxD2HaxD`) | Cudy **AX3000** (OpenWrt id `cudy,ap3000-v1`) |
| OS | RouterOS 7.x | OpenWrt 25.12.x (MT7981B / Filogic 820) |
| Radio | 2×2 both bands | 2×2 both bands (2 spatial streams) |
| Uplink | trunk (multi-port; `ether2` is a wired LAN port) | single **2.5 GbE** trunk |
| Management | core/hi/lo `.15` | core/lo `.14` (**not** `hi`; see below) |
Both mirror the same two SSIDs. `wave` replaced an older AP of the same name; the new one is the
Cudy running OpenWrt.
> **Note on "AX3000":** MT7981B is 2×2 (2 spatial streams). The "3000" is *aggregate* Mbps —
> 574 (2.4 GHz 2ss) + 2402 (5 GHz 2ss @ **160 MHz**) — not three streams. `iw` confirms 2×2
> (`Available Antennas TX/RX 0x3`; "3 streams: not supported"). 160 MHz is what earns the "3000".
## Shared design (dumb AP)
Every AP is a **dumb AP**: it bridges wireless clients onto the right VLAN and does **no** routing,
DHCP, RA or firewalling. The home routers (`river`/`stream`) own DHCP/RA/gateway (per-VLAN VRRP VIPs)
and firewalling. The uplink is a **tagged trunk**:
| VLAN | `lib.my.c.home.vlans` | Role | AP use |
|---|---|---|---|
| — (native) | `core` | switch/fabric management (1500) | backup management (untagged) |
| 100 | `hi` | trusted LAN, **high-MTU** (jumbo 9000) | `vibe` management (it's jumbo-capable) |
| 110 | `lo` | trusted LAN (1500) | main SSID `wlan0`; `wave` management |
| 120 | `untrusted` | guest network | guest SSID `wlan1` |
`hi` and `lo` are both **trusted** client VLANs — the only difference is MTU (`hi` carries jumbo
9000, `lo` is standard 1500). An AP puts its own management on whichever it can do: `vibe` (jumbo)
sits on `hi`, `wave` (eth0 capped at 2026) sits on `lo`. The main SSID lands on `lo` because Wi-Fi
clients are 1500 regardless.
### SSIDs
| SSID | Bands | Security | VLAN |
|---|---|---|---|
| `wlan0` (main) | 5 GHz + 2.4 GHz | WPA2/WPA3-PSK (`sae-mixed`) | 110 (`lo`) |
| `wlan1` (guest) | 2.4 GHz | WPA2-PSK (`psk2`) | 120 (`untrusted`) |
**Passphrases are never stored in this repo.** `vibe` is the source of truth; read them out-of-band
with `ssh admin@vibe '/interface wifi export show-sensitive'` (`.passphrase=` prints **unquoted**).
## vibe (MikroTik cAP ax)
RouterOS, one hardware-offloaded bridge `main` with `vlan-filtering=yes`. Access: `ssh admin@vibe`
— key auth for `admin` is installed (`~/.ssh/id_rsa`), with `admin`/`admin` as a fallback.
- **Uplink `ether1`** — trunk, tagged VLANs 100/110/120; native/untagged is the default VLAN 1
(PVID, no IP). `ether2` is a wired **access port** on VLAN 110 (`lo`). `l2mtu 9214`.
- **Radios** — `wifi1` (5 GHz, 20/40/80) + `wifi2` (2.4 GHz, 20/40) both broadcast `wlan0`
(WPA2/WPA3-PSK), untagged onto VLAN 110. `wifi3` is a virtual AP on `wifi2` broadcasting `wlan1`
(WPA2-PSK), untagged onto VLAN 120. `country=Ireland`.
- **Bridge VLANs** — 100 tagged `main,ether1`; 110 tagged `main,ether1` + untagged
`ether2,wifi1,wifi2`; 120 tagged `main,ether1` + untagged `wifi3`.
- **Roaming** — 802.11k/v via a `/interface wifi steering` profile (`rrm=yes wnm=yes`,
`neighbor-group=home-aps`) assigned to `wifi1`/`wifi2`/`wifi3`.
- **Resolver** — the hi VIP `192.168.71.254` / `2a0e:97c0:4d0:1::ffff`.
### Management
Management uses host `.15`: `192.168.64.15` on native/core as a backup,
`192.168.68.15/22` + `2a0e:97c0:4d0:1::1:6` on `hi` VLAN 100, and
`192.168.72.15/21` + `2a0e:97c0:4d0:2::1:6` on `lo` VLAN 110. The `hi` address holds the default
route through its VIP; `untrusted` has no address. With `l2mtu 9214`, `vibe` can use the jumbo
`hi` network unlike `wave`.
## wave (Cudy AX3000, OpenWrt)
Single-port AP, so the port is a VLAN **trunk** carrying management + both SSIDs.
### Management addressing
`wave` takes host `.14`, on **`lo` (primary) and `core` (backup)** — deliberately **not** `hi`,
unlike the switches and `vibe`. `hi` is the jumbo (9000) VLAN, but `wave`'s eth0 caps at 2026 (see
MTU note), so there's no reason to put it there; `lo` is 1500 with a proper VRRP VIP for the default
route + resolver, and `core` has no VIP/v6 so it can only be a backup. No IP on `untrusted`. Records
in `dns.nix`:
| Name | VLAN | Address |
|---|---|---|
| `wave-core` | core (native/untagged) | `192.168.64.14/24` — backup, like the switches (no VIP → backup only) |
| `wave` | lo 110 | `192.168.72.14/21`, `2a0e:97c0:4d0:2::1:5` — primary; holds the default route + resolver (lo VIP `192.168.79.254` / `2a0e:97c0:4d0:2::ffff`) |
**Firewall:** management (SSH/LuCI) reachable from `core`/`lo` only; `untrusted` is a separate
zone with `input REJECT` (and `wave` has no IP there) — **no management via the guest VLAN**.
### brian switch port
`wave` hangs off **brian** (UniFi). Its port is a **trunk**: tagged VLAN **110/120** (`lo` + guest),
and **native/untagged = core** (the fabric's management VLAN, carrying `wave-core`). VLAN 100 (`hi`)
is **not** needed here — `wave` isn't on `hi` (see Management addressing). Configure via the UniFi
controller (brian has no CLI); see [switches.md](switches.md).
### Flashing OpenWrt (Cudy AX3000 / `cudy_ap3000-v1`)
Hardware: MT7981B, 512 MB RAM, 256 MB SPI-NAND, 1× 2.5 GbE (RTL8221B), 2×2 WiFi 6.
> ⚠️ **Serial caveat:** units with a serial starting `2543…` (post ~Nov 2025) use a different flash
> chip and can brick with older firmware. Match firmware to the unit.
OpenWrt can't be flashed directly over stock. Two-stage, via a Cudy **transition** firmware (Cudy
OpenWrt download page / `support@cudy.com`; `warnning.txt` in that bundle has the steps):
1. Stock Cudy UI: update to **≥ 2.4.7** (adds TFTP `recovery.bin` recovery), then flash the Cudy
**intermediate** firmware (`cudy_ap3000-v1-sysupgrade_*.bin`), "keep settings" **unchecked**.
It reboots into an OpenWrt-based build at `192.168.1.1` (SSH `root`, empty password).
2. From there, `sysupgrade -n` to vanilla OpenWrt (`…-cudy_ap3000-v1-squashfs-sysupgrade.bin` from
`downloads.openwrt.org`; this release ships **no** factory image — sysupgrade only).
Stock default (out of box) is a DHCP client falling back to **`192.168.10.254`**; the stock UI is a
customised LuCI (only 80/443, no SSH) with a first-boot "create admin password" wizard — so the
stock-side flashing is done from a browser, not headless.
### On-device config notes
#### Wireless packages
The package manager is `apk`, not `opkg`. Wi-Fi uses the full `wpad-mbedtls` package so all SSIDs
can enable 802.11k/v (`ieee80211k` + `bss_transition`). Replacing `wpad-basic-mbedtls` live leaves
the mac80211 interfaces in a start/teardown loop that reloads cannot recover; reboot after
`apk add wpad-mbedtls`.
- Radios: `radio0` = 2.4 GHz, `radio1` = 5 GHz (keyed by `band`, don't assume). 5 GHz is pinned to
**channel 36 / HE160** (any 160 MHz block in IE is DFS; ch36 has the shortest ~60 s CAC).
- Bridge: `br-lan` with `vlan_filtering`, single port `eth0` — tagged `110/120`, untagged/PVID
VLAN 1 (= native/core). SSIDs attach via `network` = `lo`/`untrusted` (= `br-lan.110`/`.120`).
- Dumb-AP: no DHCP pools, `odhcpd.maindhcp=0`, `delegate=0` on the L3 interfaces.
- **MTU:** all interfaces are **1500**. The `mtk_eth_soc` 2.5 GbE (`eth0`) caps at **2026 bytes**
(`ip link set eth0 mtu 9000``SIOCSIFMTU: Invalid argument`), so `wave` can't join `hi`'s jumbo
(9000) fabric like `vibe` does — which is precisely **why `wave` is managed on `lo`, not `hi`**
(see Management addressing). Nothing on `wave` needs > 1500.
- **LuCI:** enabled, login `root` / `admin`. **SSH:** key-only (`PasswordAuth`/`RootPasswordAuth off`).
- `iperf3` installed for throughput testing.
### Access
- SSH: `ssh root@wave` (key-only; `wave`/`wave-core` resolve once `dns.nix` is deployed).
- LuCI: `http://192.168.72.14/` (or `http://wave/`), `root` / `admin`.
+63
View File
@@ -0,0 +1,63 @@
# castle
The home workstation / gaming desktop. Diskless-style: it netboots from `river` and keeps its
root storage on NVMe-oF volumes from `cellar`.
- **Source:** [`nixos/boxes/home/castle/`](../../../nixos/boxes/home/castle) (`default.nix`)
- **Host:** physical
- **nixpkgs:** `mine`
## Hardware
| Component | Inventory |
|---|---|
| Platform | ASUS ProArt X670E-CREATOR WIFI |
| CPU | AMD Ryzen 9 7950X (16 cores / 32 threads) |
| Memory | 64 GiB |
| Graphics | Integrated AMD Radeon graphics |
| Network | Mellanox ConnectX-4 100G, Aquantia AQC113CS 10G, Intel I225-V 2.5G and MediaTek MT7922 Wi-Fi 6E controllers |
| System storage | No local root disk; the box netboots and uses the SPDK NVMe-oF namespace exported by `cellar` |
## Role
### Desktop
The AMD desktop runs the GUI stack (`my.gui.enable`, Sway/Wayland via home-manager), low-latency
PipeWire, Bluetooth and Thunderbolt. Local `libvirtd`/`virt-manager` and the IOMMU are enabled, but
the box has no VFIO or GPU-passthrough configuration.
### Netboot
With `my.netboot.client.enable`, the firmware iPXE-boots from the 2.5G NIC. Kea matches
`et2.5g`'s MAC and directs it to `boot.h.nul.ie` on [`river`](river.md).
### NVMe-oF root
`/nix`, `/persist` and `/home` are `/dev/nvmeof/*` LVs in `cellar`'s
`nqn.2016-06.io.spdk:castle` namespace (`my.nvme.boot`, RDMA). The initrd brings
up `et100g`/`lan-hi` with `roceBootModules`; the running network keeps
`KeepConfiguration=static` so networkd does not drop the storage address. The root filesystem is a
size-limited tmpfs (`my.tmproot`).
### Other configuration
Both firewalls are disabled on this trusted `hi` desktop. Other settings include `binfmt`
emulation for `aarch64-linux`/`armv7l-linux`, `recursive-nix`, Wireshark and `rdma-core`/`qperf`;
a `drm-amd-display` flicker patch remains commented out.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `castle`).
## Networking
- `et100g` (100G, MTU 9000) carries `lan-hi` (the `hi` assignment, also pinned by a kea
reservation on its MAC) and `lan-lo`.
- `lan-lo` is a secondary network attachment: DHCPv4 with `UseGateway`/`UseDNS` off and RAs
accepted with gateway/DNS use off — present for reaching `lo` devices, never a default route.
- `et2.5g` (netboot) and `et10g` are renamed but carry no network config.
## Notable config files
- [`nixos/boxes/home/castle/default.nix`](../../../nixos/boxes/home/castle/default.nix) — box
config: netboot/NVMe-oF boot, 100G networking, GUI/audio, virtualisation.
+48
View File
@@ -0,0 +1,48 @@
# cellar
The home storage target. A VM on `palace` that drives three passed-through NVMe disks with SPDK
and exports them over NVMe-oF/RDMA — `river`, `sfh` and `castle` all run their root storage off
it.
- **Source:** [`nixos/boxes/home/palace/vms/cellar/`](../../../nixos/boxes/home/palace/vms/cellar)
(`default.nix`, `spdk.nix`)
- **Host:** VM on `palace`
- **nixpkgs:** `mine`
## Role
- Runs an **SPDK userspace target** (`my.spdk`, [`spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix)):
the kernel `nvme` driver is blacklisted so SPDK can claim the three NVMe controllers directly
(host BDFs `41:00.0``43:00.0`, attached in the guest as `02:00.0``04:00.0`).
- Builds a **RAID-0** (`NVMeRaid`) across the three drives and exports one
partition per consumer as an **NVMe-oF subsystem over RDMA** (port 4420) on the `hi` network:
| Bdev | NQN | Consumer |
|---|---|---|
| `NVMeRaidp1` | `nqn.2016-06.io.spdk:river` | [`river`](river.md) |
| `NVMeRaidp2` | `nqn.2016-06.io.spdk:castle` | [`castle`](castle.md) |
| `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh/README.md) |
Each subsystem is pinned to its consumer's `hostnqn` (the `my.nvme.uuid` on the client side).
- `spdk-tgt` is ordered after `lan-hi` is online; the RDMA listener binds the `hi` assignment on
port 4420. The VM itself is pinned to NUMA node 1 on `palace` and gets SR-IOV VF 0.
- `netdata` (port 19999 allowed in the firewall) and `fstrim`.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `cellar`).
## Notes
- The `ublk_*` calls in `my.spdk.debugCommands` are only a debugging aid — they create a local
ublk device so the RAID can be mounted and inspected on `cellar` itself. Client exports are the
`nvmf` subsystems above.
## Notable config files
- [`nixos/boxes/home/palace/vms/cellar/default.nix`](../../../nixos/boxes/home/palace/vms/cellar/default.nix) —
box config (assignment, networking, netdata).
- [`nixos/boxes/home/palace/vms/cellar/spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix) —
SPDK target: RAID-0, NVMe-oF/RDMA subsystems.
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
the VM definition on `palace` (VF 0, NVMe passthrough, NUMA pinning).
+70
View File
@@ -0,0 +1,70 @@
# palace
The physical VM host for the home site. Runs the `river`, `cellar` and `sfh` VMs and feeds them
SR-IOV VFs, PCI NVMe drives and LVM disks.
- **Source:** [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix)
(VM definitions in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
- **Host:** physical
- **nixpkgs:** `mine-stable`
## Hardware
| Component | Inventory |
|---|---|
| Platform | Gigabyte X399 DESIGNARE EX |
| CPU | AMD Ryzen Threadripper 1950X (16 cores / 32 threads) |
| Memory | 128 GiB |
| Host storage | 500 GB Samsung SSD 860 EVO containing the EFI partition and the `main` LVM thin pool |
| Bulk storage | Three 8 TB Seagate IronWolf disks in the `hdds` VG, providing the RAID-backed `hdd-storage` and `frigate` LVs |
| NVMe storage | Three 2 TB Samsung NVMe devices passed through to `cellar`; SPDK combines them as the `NVMeRaid` RAID 0 device |
| Network / graphics | Mellanox ConnectX-4 100G adapter with four SR-IOV VFs, two Intel I211 Gigabit Ethernet controllers, and an AMD Radeon RX 550/560-family GPU |
## Role
- Home hypervisor: VMs are declared in `my.vms.instances`
([`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix)); disks are LVs in
the `main` thin pool (`services.lvm.boot.thin.enable`).
- AMD box (`kvm-amd`, `amd_iommu=on`, microcode updates); the kernel is built with
`ACPI_APEI_PCIEAER`/`PCIEAER` for the PCIe passthrough work below.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `palace`).
## Networking
100G `et100g` (mlx5, MTU 9000) uplinks to the `dave` switch and carries `lan-hi` (VLAN 100, the
`hi` assignment). A udev rule creates four SR-IOV VFs on the PF:
| VF | Consumer | VLAN handling |
|---|---|---|
| 0 | `cellar` | `hi` |
| 1 | `river` | untagged VF; `river` tags its LAN and WAN VLANs |
| 2 | `sfh` | `hi` |
| 3 | `sfh` container MACVLAN parent | `hi` |
- `lan-core` is a bridge with the `core` assignment (no gateway); the 1G
`lan-core-phy` and the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used
for VM netboot and `lo` clients.
- The 1G `et1g0` (igb) is exported to `river` as a passthru-mode macvtap (`vm-et1g0`) — river sees
it as `wan-old`.
## VMs
| VM | vCPUs | RAM | Passthrough | Notes |
|---|---|---|---|---|
| `cellar` | 8c × 2t | 16 GiB | VF 0 (`44:00.1`); NVMe `41:00.0``43:00.0` | pinned to NUMA node 1; split IRQ chip + vIOMMU |
| `river` | 3c × 2t | 4 GiB | VF 1 (`44:00.2`); macvtap `vm-et1g0` | only an ESP local disk (plus an installer ISO) — root is NVMe-oF from `cellar` |
| `sfh` | 8c × 2t | 32 GiB | VF 2 (`44:00.3`), VF 3 (`44:00.4`); two USB host ports | no boot disk — netboots; gets the `hdds/frigate` LV |
Boot ordering is enforced with systemd dependencies: `vm@river` waits for `cellar`'s SSH port (and
for the `vm-et1g0` device), and `vm@sfh` waits for `river` — storage first, then the router, then
everything that boots off both.
## Notable config files
- [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix) — host
hardware, networkd (links/bridges/SR-IOV), LVM.
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
VM instances and boot ordering.
+79
View File
@@ -0,0 +1,79 @@
# river
A home router VM on `palace` with a Digiweb PPPoE WAN on the ISP's VLAN 10. It forms the redundant
router pair with [`stream`](stream.md).
- **Source:** [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix)
(shared router config: [`routing-common`](../../../nixos/boxes/home/routing-common), index 0)
- **Host:** VM on `palace`
- **nixpkgs:** `mine`
## Role
At `routing-common` index 0, `river` normally holds the primary position in the router pair.
Pair-wide addressing, VIP, DHCP/DNS and failover behavior is documented in the
[`home` networking overview](../../networking.md#home) and [Router HA](../../networking.md#router-ha).
This page covers `river`'s Digiweb WAN, VM platform, storage and netboot duties.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `river`).
## WAN (Digiweb PPPoE)
### Link and addressing
`services.pppd` peer `digiweb` attaches directly to `wan-pon-isp`, the raw ISP VLAN 10
(`vlans.pon-isp`) trunked untranslated from the ONT through `brian` and `dave`; see
[switches.md](switches.md). The netdev has no L3 configuration and uses `MTUBytes=1508`, leaving a
clean `mtu`/`mru 1500` after PPPoE overhead.
The static `84.203.124.128` address is requested through IPCP. The provider-wide credentials are
deliberately not secret; the peer persists indefinitely with LCP echo monitoring and ignores
Digiweb DNS in favor of the local recursor.
### Management subnet
`wan-pon-ont` (VLAN 140, PVID'd at `brian`) holds `192.168.100.100/24`, reaching the ONT UI at
`192.168.100.1`. The `.100` address follows `stream`'s modem-management convention.
### WAN readiness
The `pppd` hooks drive this shared gate with `DefaultDependencies=false`: `ip-up` installs the
link-scoped default route and starts the target, while `ip-down` stops it and removes the route.
`ipsec` and `ipv6-clear-default-route` attach through `wantedBy` + `partOf`, so they reload after
every WAN flap.
### Traffic shaping
The shared `wan-ifb` ingress-shaping pieces are inert here. CAKE is specific to `stream`, and
`networkd-dispatcher` is `mkForce false` pending scheduling tests.
## Platform
### Virtual machine and network attachment
The 100G `lan` NIC is VF 1 of `palace`'s `et100g` (MTU 9000), with every router VLAN tagged on top
as `55-lan`. A macvtap of `palace`'s 1G `et1g0` remains as the old `wan-old` path without L3
configuration. Deployments use the `hi` assignment.
### Storage
The VM's local disk holds only an ESP; `/nix` and `/persist` are LVs on `cellar`'s
`nqn.2016-06.io.spdk:river` namespace over RDMA. The initrd brings up `lan-hi`
with `roceBootModules`, and `KeepConfiguration=static` prevents networkd from dropping the address
during reconfiguration. An installer ISO remains attached.
## Netboot
`my.netboot.server` serves iPXE/TFTP for `sfh` and `castle` at `boot.h.nul.ie` from the `lo`
assignment, restricted to the `hi` and `lo` prefixes.
## Notable config files
- [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix) — box
config: pppd, WAN VLANs, `wan-online.target` hooks, netboot server, NVMe-oF boot.
- [`nixos/boxes/home/routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix) —
shared router definition (assignments, firewall/NAT, `as211024`).
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
the VM definition on `palace` (VF 1, macvtap, ESP disk).
+59
View File
@@ -0,0 +1,59 @@
# sfh
"Services for home" — the NixOS container host for the home site. A VM on `palace` that netboots
from `river` and runs its root off NVMe-oF from `cellar`.
- **Source:** [`nixos/boxes/home/palace/vms/sfh/`](../../../../nixos/boxes/home/palace/vms/sfh)
(`default.nix`, `containers/`)
- **Host:** VM on `palace`
- **nixpkgs:** `mine`
## Role
- Runs the home NixOS containers via `my.containers.instances` (systemd-nspawn); each container is
its own `nixos.systems.*` entry rendered through `my.asContainer`.
- **Netboot client** (`my.netboot.client.enable`): the VM has no boot disk — its `netboot` NIC
on palace's `lan-lo` bridge is matched by a kea client-class on [`river`](../river.md) and
iPXE-boots from `boot.h.nul.ie`.
- **Root on NVMe-oF**: `my.nvme.boot` connects to `nqn.2016-06.io.spdk:sfh`
([`cellar`](../cellar.md), RDMA) from the initrd (`lan-hi` up + `roceBootModules`); `/nix` and
`/persist` are LVs on that volume. `KeepConfiguration=static` on `lan-hi` protects the
NVMe-oF address from networkd reconfigures.
- **Frigate footage disk**: palace passes the `hdds/frigate` LVM LV through as a virtio disk;
sfh mounts it at `/mnt/frigate` (by label) and bind-mounts it into the `hass` container at
`/var/lib/frigate`.
- USB: two host ports are passed to the VM (qemu flags) for the Zigbee coordinator and webcam used
by `hass`; the nspawn unit gets `DeviceAllow` for `char-ttyUSB` and `char-video4linux`.
## Network assignments
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `sfh`).
## Networking
Four NICs, all MTU 9000 where jumbo-capable:
- `lan-hi` — SR-IOV VF 2, the box's own `hi` assignment.
- `lan-hi-ctrs` — SR-IOV VF 3, no L3: the MACVLAN parent for the containers' `hi` interfaces
(`host0` inside each container).
- `lan-core-ctrs` / `lan-lo-ctrs` — virtio NICs (bridged to palace's `lan-core` / `lan-lo`), no
L3: MACVLAN parents for containers that need a `core` or `lo` interface.
The per-container MACVLAN wiring lives in `systemd.nspawn.*.networkConfig` in
[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix).
## Containers
| Container | Role |
|---|---|
| [`hass`](containers/hass.md) | Home Assistant + Frigate + MQTT |
| [`unifi`](containers/unifi.md) | UniFi controller |
## Notable config files
- [`nixos/boxes/home/palace/vms/sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix) —
box config: netboot/NVMe-oF boot, container instances, MACVLAN plumbing, Frigate disk.
- [`nixos/boxes/home/palace/vms/sfh/containers/`](../../../../nixos/boxes/home/palace/vms/sfh/containers) —
the container system definitions.
- [`nixos/boxes/home/palace/vms/default.nix`](../../../../nixos/boxes/home/palace/vms/default.nix) —
the VM definition on `palace` (VFs, USB passthrough, netboot NIC, `hdds/frigate` disk).
+59
View File
@@ -0,0 +1,59 @@
# hass
Home automation container: Home Assistant plus its supporting services (MQTT, camera restreaming,
Frigate NVR), running on [`sfh`](../README.md).
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/hass.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/hass.nix)
- **Host:** NixOS container on `sfh`
- **nixpkgs:** `mine`
## Role
### Home Assistant
`services.home-assistant` uses declarative configuration (`configWritable = false`). It enables
the `esphome`, `zha`, `denonavr`, `webostv`, `androidtv_remote`, `heos`, `mqtt`, `wled`, `met` and
`google_translate` components, plus custom `alarmo`, `frigate`, `west_wood_club` and Irish Rail
integrations. A `hass-cli` wrapper uses a token from `my.secrets` to reach the local server.
- **mosquitto** — MQTT broker (anonymous local listener; port 1883 allowed, alongside HTTP).
- **go2rtc** — restreams the Reolink living-room camera (RTSP from `reolink-living-room`, on the
`lo` network) and the office USB webcam (`/dev/video0` via ffmpeg).
- **Frigate** (`services.frigate`, `frigate.h.nul.ie` — the `frigate` alt name on the `hi`
assignment) — records both restreamed cameras with a short retention policy; detection is
disabled.
- External access is via `https://hass.nul.ie` through the `middleman` reverse proxy
(`trusted_proxies`); internally it's `hass-ctr.h.nul.ie`.
- Not a deploy-rs target (`my.deploy.enable = false`) — it's rendered via `my.asContainer` and
started by `sfh`'s `my.containers.instances`.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `hass`).
## Storage
Frigate footage lives on a **separate HDD LV**: `palace` passes the `hdds/frigate` LVM LV to the
`sfh` VM, sfh mounts it at `/mnt/frigate`, and the container bind-mounts it at `/var/lib/frigate`
(read-write). This keeps recording churn off the NVMe-oF root.
## Devices
Passed through from `sfh` (USB host ports on `palace`):
- Nabu Casa Connect ZBT-1 Zigbee coordinator → `/dev/ttyUSB0` (used by `zha`).
- USB webcam → `/dev/video0` (go2rtc's `webcam_office` stream).
- The matching raw USB device node allowed through from `sfh`.
## Networking
MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment,
alt name `frigate`, default gateway via the VIP) and `lan-lo` on `lan-lo-ctrs` (the `lo`
assignment, no gateway) — the `lo` interface reaches the IoT devices (the Reolink camera lives there).
## Notable config files
- [`nixos/boxes/home/palace/vms/sfh/containers/hass.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/hass.nix) —
container system: Home Assistant, Frigate, mosquitto, go2rtc.
- [`nixos/boxes/home/palace/vms/sfh/default.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/default.nix) —
the `sfh` side: bind mounts, MACVLAN wiring, `DeviceAllow`.
+42
View File
@@ -0,0 +1,42 @@
# unifi
The UniFi network controller, running as a container on [`sfh`](../README.md). It manages the
home UniFi switch `brian` (see [switches.md](../../switches.md)).
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/unifi.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/unifi.nix)
- **Host:** NixOS container on `sfh`
- **nixpkgs:** `mine`
## Role
- **UniFi controller** (`services.unifi`, `pkgs.unifi` on `mongodb-7_0`, firewall open; TCP 8443
allowed).
- Not a deploy-rs target (`my.deploy.enable = false`) — it's rendered via `my.asContainer` and
started by `sfh`'s `my.containers.instances`.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `unifi`).
## Status
**Currently enabled.** The container spent a while disabled — its import was commented out of
[`containers/default.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/default.nix)
while there was no UniFi gear to manage — and was re-enabled when the UniFi switch `brian` was
added, gaining a `core` interface (`unifi-ctr-core`) at the same time so it can reach the switch on its
management network. It is imported, listed in `sfh`'s `my.containers.instances`, and
`services.unifi.enable = true`.
## Networking
Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` carries the `hi`
assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `lan-core-ctrs` carries
the gatewayless `core` assignment (`unifi-ctr-core`). The `core` interface is how the controller
talks to `brian` and the other switches on their management network.
## Notable config files
- [`nixos/boxes/home/palace/vms/sfh/containers/unifi.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/unifi.nix) —
container system: UniFi service, assignments.
- [`nixos/boxes/home/palace/vms/sfh/default.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/default.nix) —
the `sfh` side: container instance, MACVLAN wiring.
+98
View File
@@ -0,0 +1,98 @@
# stream
A physical Intel home router with a DHCP WAN from the Virgin Media cable modem. It forms the
redundant router pair with [`river`](river.md) and is dual-homed to both switches.
- **Source:** [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) (shared router
config: [`routing-common`](../../../nixos/boxes/home/routing-common), index 1)
- **Host:** physical
- **nixpkgs:** `mine`
## Hardware
| Component | Inventory |
|---|---|
| Platform | BROUNION R86S |
| CPU | Intel Celeron N5105 (4 cores / 4 threads) |
| Memory | 16 GiB |
| Storage | 512 GB Samsung SSD 970 PRO NVMe containing `/boot`, `/nix` and `/persist`; integrated 128 GB eMMC is present but is not used by the declared filesystems |
| Network | Three Intel `igc` interfaces and a dual-port Mellanox `mlx4_en` adapter; `wan`, `lan-jim` and `lan-dave` use three of these ports |
The platform configuration enables `kvm-intel`, `intel_iommu=on` and Intel microcode updates.
## Role
At `routing-common` index 1, `stream` normally holds the secondary position in the router pair.
Pair-wide addressing, VIP, DHCP/DNS and failover behavior is documented in the
[`home` networking overview](../../networking.md#home) and [Router HA](../../networking.md#router-ha).
This page covers `stream`'s Virgin Media WAN, physical platform and redundant switch attachment.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `stream`).
## WAN (Virgin Media DHCP)
### Link and addressing
`wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) towards the cable modem. The modem segment is
switch VLAN 130; `jim` handles the tag, so the box interface is untagged. See
[switches.md](switches.md) for the fabric side.
`DHCP=ipv4` pulls the public lease. `dhcpV4Config.UseDNS=false` points resolution at the local
recursor, and `IPv6AcceptRA=false` because public IPv6 arrives over the tunnel rather than this WAN.
### Management subnet
The static `192.168.0.100/24` address (host `.100` of `prefixes.modem.v4`) sits on `wan` without a
gateway, keeping the modem UI reachable alongside the DHCP lease.
### WAN readiness
`wan-wait-online.service` polls until the DHCP default route exists, then satisfies
`wan-online.target`. The route is the gate because the permanent modem address would make
networkd's wait-online report success before the public lease arrives, allowing `ipsec` to start
without its public `left=` address.
### Traffic shaping
Egress is shaped at the `wan` root qdisc. A `routing-common` `networkd-dispatcher` rule redirects
ingress through `tc`/`mirred` into `wan-ifb`; each direction has its own configured bandwidth and
uses the DOCSIS overhead preset.
### Per-box `routing-common` options
The modem's management subnet shares the `wan` interface, which `routing-common` itself knows
nothing about — it declares two per-box options
([`routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix)) that this
box sets:
- `my.homeRouter.dns.wanSkipBroadcasts` — skip the modem subnet's broadcast address when
auto-selecting the router's own `wan` A record for the zone's LUA record.
- `my.homeRouter.firewall.untrustedRejectV4` — reject untrusted clients from
reaching the modem subnet (needed only because it shares `wan`; WAN egress is otherwise
accepted).
## Switching (RSTP)
`stream` is dual-homed to both switches: `lan-jim` (igc) and `lan-dave` (mlx4_en), both MTU 9000,
are enslaved to the `lan` bridge with `STP=true`. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
runs a patched `mstpd` and forces RSTP on `lan` once it's routable, so exactly one uplink carries
traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfigured.)
## Deployment
`my.deploy.node.hostname` is currently commented out.
## Disabled printer services
`octoprint` and `mjpg-streamer` are defined but disabled (`enable = false`).
## Notable config files
- [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) — box config: DHCP WAN,
modem management, CAKE, `wan-online.target` gate, STP bridge.
- [`nixos/boxes/home/routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix) —
shared router definition (index 1).
- [`nixos/boxes/home/routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix) —
RSTP on the `lan` bridge.
+206
View File
@@ -0,0 +1,206 @@
# Home switches
Reference for the two MikroTik switches on the home network — **jim** and **dave** — plus the
Ubiquiti switch **brian**, and how the home boxes and the Digiweb WAN hang off them. These switches
are **not** managed by this flake; they are configured by hand (RouterOS on jim/dave, UniFi on
brian). It covers the physical topology, the VLAN map, and how the Digiweb WAN reaches river.
In short: the Digiweb ISP VLAN (10) is trunked straight through to river (which runs PPPoE on it),
and the ONT's untagged management is PVID'd onto VLAN 140 at brian, its edge switch. VLAN 10 is
carried untranslated because a single ONT makes it unique on the fabric — see
[the WAN path](#the-digiweb-wan-path-trunked-vlan-10--pvid-140) and
[why not translation](#why-not-translation-for-one-ont). The router side lives in
[river.md](river.md); the logical network map in [networking.md](../../networking.md). The Wi-Fi
APs that hang off these switches are in [aps.md](aps.md).
## The switches
| | jim | dave | brian |
|---|---|---|---|
| Identity | `jim-sw` | `dave-sw` | (UniFi) |
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti Switch Pro XG 8 PoE |
| Switch chip | Marvell 98DX3236 | Marvell 98DX4310 (+ Atheros 8227 for the 1G mgmt port) | — |
| OS | RouterOS | RouterOS | UniFi |
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 (100G, breakout-capable) + 1G mgmt | 8×10GBASE-T PoE + 2×10G SFP+ |
| Bridge | `main`, `vlan-filtering=yes` | `main`, `vlan-filtering=yes` | UniFi VLAN profiles |
jim and dave run a single hardware-offloaded bridge (`main`) with VLAN filtering. Access to the
MikroTiks is SSH as `admin` / `admin` by short hostname (see [Accessing the switches](#accessing-the-switches)).
Only jim and dave can do hardware VLAN translation (`/interface ethernet switch rule` on the Marvell
chips); brian cannot rewrite tags, only trunk/PVID them.
## Physical topology
The two WAN sources enter at the top: the Virgin Media modem lands on **jim** (VLAN 130), and the
Digiweb **ONT** lands on **brian**. Both `jim` and `brian` are edge switches that uplink down into
the **dave** core; the home boxes hang off dave's 100G ports, with backup links up to jim. jim's
`wan-pon-in` (`sfp-sfpplus2`) is a spare SFP+ port, unused today.
```
Virgin Media cable modem Digiweb ONT
stream WAN, VLAN 130 river WAN, management + VLAN 10
| |
jim brian
| 10G trunk 802.3ad LAG |
+--------------------+ +---------------+
| |
+---+----------+---+
| dave |
+--------+---------+
|
+---------------------+---------------------+
| | |
palace (100G) castle (100G) stream
river VM NVMe-oF root second router
Backup links to jim (normally idle):
* palace: 1G
* stream: 1G; STP selects the active link
* castle: 2.5G, normally down; no live failover
```
Notes:
- **river** runs as a VM on the **palace** host; its uplink is dave's 100G `palace` port. jim also
has 1G `palace`/`stream` ports, but those are secondary links and do **not** carry the WAN.
- **stream** (the second router box) is dual-homed to both jim and dave (STP picks the active path).
### Castle storage dependency
`castle` is dual-homed without STP: its primary link is dave's 100G `castle` port (`et100g`), while
the 2.5G link to jim (`et2.5g`) is normally down and provides no live failover. Its root disk is
NVMe-oF over `et100g` and dave, so interrupting either freezes `castle` mid-I/O. Do dave maintenance
from a box that does not depend on it, or power `castle` off cleanly first.
## VLANs
| VLAN | Name | Purpose |
|---|---|---|
| — (native) | core | Switch management, `192.168.64.0/24` (jim `.10`, dave `.11`, brian `.13`) |
| 100 | hi | High-performance / jumbo network (MTU 9000) |
| 110 | lo | Standard LAN |
| 120 | untrusted | Guest / untrusted network |
| 130 | wan | **stream's WAN** — Virgin Media cable modem (untagged on jim's `wan1`/`wan2`/`wan-in`) |
| 140 | wan-pon-ont | ONT management, `192.168.100.0/24` (PVID'd at the ONT edge) |
| 10 | pon-isp | Digiweb ISP transport — **trunked straight through** to river, PPPoE runs on it |
| 141 | wan-pon-isp | **Reserved** — the translated ISP VLAN for the future multi-ONT design |
Switch L3 presence (`/interface vlan` on `main`) exists **only** for VLANs the switch is managed
from — `hi` (100) and `lo` (110), plus native core. WAN and guest VLANs deliberately have no switch
L3 interface.
## The Digiweb WAN path (trunked VLAN 10 + PVID 140)
The ONT presents two things on one wire:
- **untagged** management traffic (`192.168.100.x`), and
- **tagged VLAN 10** carrying the Digiweb ISP session (the BRAS requires VLAN 10).
With a **single ONT** there's no reason to translate anything — VLAN 10 is unique on the fabric, so
we just carry it end to end and let river run PPPoE directly on it:
1. **Untagged mgmt → VLAN 140, at the ONT's edge switch (brian).** brian sets the ONT port's PVID to
140 so the untagged management traffic becomes VLAN 140, and allows tagged VLAN 10 through the
same port. river takes `192.168.100.100/24` on VLAN 140 (matching stream's modem-mgmt `.100`) to
reach the ONT web UI at `192.168.100.1`. Doing the PVID at the ONT-facing edge keeps it clean —
the untagged frames never share a domain with anything else.
2. **VLAN 10 (ISP) trunked straight through, untranslated.** brian → dave → palace carry tagged
VLAN 10 by ordinary bridge-VLAN membership. No `/interface ethernet switch rule`, no pinning, no
asymmetric-learning issues — it's just a normal tagged VLAN. river attaches PPPoE to VLAN 10
directly (`wan-pon-isp` netdev = VLAN `pon-isp` = 10; baby-jumbo MTU 1508 so PPP nets a clean
1500).
Net result: **river runs PPPoE single-tagged on VLAN 10 and holds a VLAN 140 address to reach the
ONT.** See [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix)
for the river side.
```
ONT -- untagged + VLAN 10 -- brian -- VLAN 140 + VLAN 10 -- dave -- palace -- river
|
+-- ONT port PVID 140; VLAN 10 remains tagged
```
### Why not translation (for one ONT)?
Translation would swap VLAN 10 → 141 with two pinned hardware ACL rules to keep VLAN 10 off the rest
of the fabric. That buys nothing with a single ONT — VLAN 10 is already unique, so trunking it is
simpler and rule-free. Translation only earns its keep when **two** ONTs both deliver VLAN 10 and
would collide (below).
## Switch configuration
How each switch is set up for the Digiweb WAN path. **Confirm any change on the box before applying**
(see [Accessing the switches](#accessing-the-switches)).
**brian (UniFi)** — hosts the ONT:
- The ONT port has **native/untagged network = VLAN 140** (PVID) and is a **tagged member of VLAN 10**,
so the ONT's untagged management lands on 140 and its tagged ISP frames pass through.
- The `brian-downlink` LAG up to dave trunks **tagged 140 + tagged 10** (alongside the LAN VLANs).
**dave (RouterOS)** — trunks both WAN-pon VLANs to `brian-downlink` and `palace`. The ISP VLAN 10 row:
```
/interface bridge vlan add bridge=main vlan-ids=10 tagged=brian-downlink,palace
```
VLAN 140 also spans `brian-downlink,palace` (it carries a few other members too). No switch rules —
this is plain tagged bridging.
**jim (RouterOS)** — carries **none** of the Digiweb WAN path: no translation rules, and no VLAN
10/140/141 rows. `wan-pon-in` (`sfp-sfpplus2`) sits at `pvid=1` as a spare port. jim only handles
stream's VLAN-130 WAN and the LAN VLANs.
## Future: multiple ONTs (per-port VLAN translation)
If a second ONT arrives (e.g. a Digiweb line for stream, or a second river), trunking breaks: both
ONTs deliver **tagged VLAN 10**, and plain bridge-VLAN filtering can't tell them apart. That's when
translation earns its place — a switch rule matches on the **ingress port**, so each ONT's VLAN 10
becomes a *distinct* fabric VLAN:
- ONT-A port: VLAN 10 → **141** (→ river)
- ONT-B port: VLAN 10 → **142** (→ stream / second river)
- mgmt: PVID each ONT port onto its own VLAN (140, 143, …) so both ONTs' `192.168.100.1` stay in
separate L2/L3 domains.
The forward direction isolates naturally (each ONT maps to a different fabric VLAN). The **return**
direction is where port targeting is mandatory: both translate *back* to VLAN 10, so bridge VLAN 10
now has two members and a plain FDB-miss flood would leak one ONT's upstream to the other. Each
return must be pinned to its port with `new-dst-ports`:
```
# ONT-A: 141 in on palace → 10, forced out ONT-A's port
# ONT-B: 142 in on stream → 10, forced out ONT-B's port
```
Each ONT port must also be a tagged member of bridge VLAN 10 for correct egress tagging (the missing
piece that otherwise shows up as pppd "Timeout waiting for PADO"). The pins bypass the FDB, so the
two ISP sessions never mix.
**Why a new switch:** jim (the only box with spare SFP+ *and* the translation feature) has just
**one** free SFP+ port, so it can't host two ONTs. The plan is a dedicated
**CRS305-1G-4S+** (4×SFP+, same Marvell rule support) to land multiple ONTs and do the per-port
translation there, feeding distinct fabric VLANs up to dave.
## Accessing the switches
The switches resolve by **short hostname** on the home network — the home routers serve their
records in the home zone
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix):
`jim` → hi `.10`, `dave` → hi `.11`, `brian` → core `.13`). From a box on the home network just
`ssh admin@jim` / `admin@dave`.
**Key auth** for `admin` is installed on jim/dave (and the `vibe` AP) — `ssh -i ~/.ssh/id_rsa
admin@jim` works keyless (imported via `/user ssh-keys import`). Password `admin`/`admin` remains as
a fallback. Non-interactive password pattern (avoids the ssh-agent hang) if the key isn't available:
```
sshpass -p admin ssh -o IdentityAgent=none -o PubkeyAuthentication=no \
-o PreferredAuthentications=password -o StrictHostKeyChecking=accept-new \
-o UserKnownHostsFile=/tmp/sw_known_hosts admin@jim
```
**Always confirm config changes on the switch** (print the affected menu, apply, re-verify). brian
is UniFi — configured through its controller, not RouterOS CLI.
## Management IPs
| | core (`192.168.64.0/24`) | hi (`192.168.68.0/22`) | lo (`192.168.72.0/21`) |
|---|---|---|---|
| jim | `.10` (on `main`) | `.10` | `.10` |
| dave | `.11` (on `management`, the 1G Atheros port) | `.11` | `.11` |
| brian | `.13` (core) | — | — |
+7
View File
@@ -49,6 +49,13 @@
copyparty.inputs.nixpkgs.follows = "nixpkgs-unstable";
hass-west-wood.url = "github:devplayer0/hass-west-wood";
hass-west-wood.inputs.nixpkgs.follows = "nixpkgs-unstable";
# Disabled: `pi-coding-agent-bun` breaks `nix flake check` in CI. Its bun2nix `fetchBunDeps`
# calls `builtins.filterSource` on subpaths of the pi.nix flake source (coding-agent/bun.nix),
# which requires that source derivation to be realised in the local store. On a fresh CI runner
# it isn't, so eval aborts with `path '…-source.drv' is not valid`. Works locally only because
# the source is already realised there. Re-enable (and revisit upstream) if we start using pi.
# pi-agent.url = "github:lukasl-dev/pi.nix";
# pi-agent.inputs.nixpkgs.follows = "nixpkgs-unstable";
};
outputs =
+6
View File
@@ -8,6 +8,9 @@ let
inherit (lib.my) mkOpt' dummyOption;
in
{
# Disabled: breaks `nix flake check` in CI (see flake.nix pi-agent note).
# imports = [ inputs.pi-agent.homeModules.default ];
options = with lib.types; {
my = {
isStandalone = mkOption {
@@ -27,6 +30,7 @@ in
};
};
};
config = mkMerge [
{
my = {
@@ -213,6 +217,7 @@ in
jq
yq-go
nix-tree
treemd
];
sessionVariables = {
@@ -229,6 +234,7 @@ in
inputs.deploy-rs.overlays.default
inputs.boardie.overlays.default
inputs.nixGL.overlays.default
# inputs.pi-agent.overlays.default
];
config = {
allowUnfree = true;
+14
View File
@@ -71,8 +71,10 @@ in
python3Packages.python-lsp-server
nil # nix language server
nixd # another nix language server
zls # zig language server
rust-analyzer
pyright
cowsay
fortune
@@ -85,6 +87,18 @@ in
ffmpeg-full
xournalpp
# Disabled: breaks `nix flake check` in CI (see flake.nix pi-agent note).
# (pkgs.symlinkJoin {
# name = "pi-coding-agent";
# buildInputs = [ pkgs.makeWrapper ];
# paths = [ pkgs.pi-coding-agent-bun ];
# postBuild = ''
# wrapProgram $out/bin/pi \
# --set NPM_CONFIG_PREFIX ${config.home.homeDirectory}/.pi/npm/ \
# --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.nodejs_latest ]}
# '';
# })
];
};
+14 -1
View File
@@ -312,6 +312,16 @@ rec {
lo = 110;
untrusted = 120;
wan = 130;
# Digiweb delivers the ISP VLAN (pon-isp, 10) single-tagged at the ONT alongside the ONT's
# own untagged management traffic. With a single ONT we trunk pon-isp (10) straight through
# the switches to river (PPPoE runs directly on it), and the switch at the ONT edge PVIDs the
# untagged management port onto wan-pon-ont (140). wan-pon-isp (141) is reserved for the
# future multi-ONT case, where per-port VLAN translation on a dedicated switch swaps each
# ONT's VLAN 10 to a distinct fabric VLAN (see docs/sites/home/switches.md).
pon-isp = 10;
wan-pon-ont = 140;
wan-pon-isp = 141;
};
hiMTU = 9000;
routers = [
@@ -319,7 +329,7 @@ rec {
"stream"
];
routersPubV4 = [
"109.255.108.88"
"84.203.124.128" # river: Digiweb static
"109.255.108.121"
];
@@ -327,6 +337,9 @@ rec {
modem = {
v4 = "192.168.0.0/24";
};
ont = {
v4 = "192.168.100.0/24";
};
all = {
v4 = "192.168.64.0/18";
v6 = "2a0e:97c0:4d0::/60";
+5 -4
View File
@@ -175,11 +175,11 @@ rec {
};
vm = rec {
lvmDisk' = name: lv: {
lvmDisk'' = name: vg: lv: {
inherit name;
backend = {
driver = "host_device";
filename = "/dev/main/${lv}";
filename = "/dev/${vg}/${lv}";
# It appears this needs to be set on the backend _and_ the format
discard = "unmap";
};
@@ -189,8 +189,9 @@ rec {
};
frontend = "virtio-blk";
};
lvmDisk = lv: lvmDisk' lv lv;
disk = vm: lv: lvmDisk' lv "vm-${vm}-${lv}";
lvmDisk' = vg: lv: lvmDisk'' lv vg lv;
lvmDisk = lvmDisk' "main";
disk = vm: lv: lvmDisk'' lv "main" "vm-${vm}-${lv}";
};
deploy-rs =
+2 -2
View File
@@ -212,8 +212,8 @@
cleanShutdown.timeout = 120;
drives = [
(mkMerge [ (vm.disk "darts" "root") { frontendOpts.bootindex = 0; } ])
(vm.lvmDisk' "media" "darts-media")
(vm.lvmDisk' "ext" "darts-ext")
(vm.lvmDisk'' "media" "main" "darts-media")
(vm.lvmDisk'' "ext" "main" "darts-ext")
];
};
};
+5 -1
View File
@@ -114,8 +114,12 @@ in
];
also-notify = [ "127.0.0.1" ];
allow-axfr-ips = [
"127.0.0.0/8" "::1/128"
"216.218.133.2" "2001:470:600::2"
];
]
++ lib.my.c.home.routersPubV4
++ lib.my.c.as211024.trusted.v4
++ lib.my.c.as211024.trusted.v6;
enable-lua-records = true;
#loglevel = 7;
#log-dns-queries = true;
@@ -95,6 +95,7 @@ in
shell = pkgs.bashInteractive;
openssh.authorizedKeys.keyFiles = [
lib.my.c.sshKeyFiles.harmonia
lib.my.c.sshKeyFiles.me
];
};
};
+15 -3
View File
@@ -53,7 +53,7 @@ in
};
};
});
kernelModules = [ "kvm-amd" ];
kernelModules = [ "dm-raid" "kvm-amd" ];
kernelParams = [ "amd_iommu=on" ];
initrd = {
availableKernelModules = [ "xhci_pci" "ahci" "usb_storage" "usbhid" "sd_mod" "sr_mod" ];
@@ -126,9 +126,9 @@ in
};
linkConfig.Name = "et1g0";
};
"10-lan-core" = {
"10-lan-core-phy" = {
matchConfig.PermanentMACAddress = "e0:d5:5e:68:0c:70";
linkConfig.Name = "lan-core";
linkConfig.Name = "lan-core-phy";
};
"10-et100g" = {
matchConfig = {
@@ -145,6 +145,12 @@ in
netdevs = mkMerge [
(mkVLAN "lan-hi" vlans.hi)
(mkVLAN "lan-lo-phy" vlans.lo)
{
"25-lan-core".netdevConfig = {
Name = "lan-core";
Kind = "bridge";
};
}
{
"25-lan-lo".netdevConfig = {
Name = "lan-lo";
@@ -199,6 +205,12 @@ in
};
"60-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
"50-lan-core-phy" = {
matchConfig.Name = "lan-core-phy";
networkConfig = {
Bridge = "lan-core";
} // networkd.noL3;
};
"50-lan-lo-phy" = {
matchConfig.Name = "lan-lo-phy";
networkConfig = {
+12 -1
View File
@@ -172,12 +172,23 @@
};
memory = 32768;
cleanShutdown.timeout = 120;
networks.netboot = {
networks = {
netboot = {
bridge = "lan-lo";
waitOnline = "carrier";
mac = "52:54:00:a5:7e:93";
extraOptions.bootindex = 1;
};
core = {
bridge = "lan-core";
ifname = "vm-sfh-core";
waitOnline = "carrier";
mac = "52:54:00:72:67:51";
};
};
drives = [
(vm.lvmDisk' "hdds" "frigate")
];
hostDevices = {
et100g0vf2 = {
index = 0;
+113 -5
View File
@@ -8,9 +8,23 @@
configuration = { lib, modulesPath, pkgs, config, assignments, allAssignments, ... }:
let
inherit (lib.my) networkdAssignment mkVLAN;
inherit (builtins) elemAt;
inherit (lib) mkForce mkMerge;
inherit (lib.my) net networkdAssignment mkVLAN;
inherit (lib.my.c) networkd;
inherit (lib.my.c.home) vlans domain prefixes roceBootModules;
inherit (lib.my.c.home) vlans domain prefixes roceBootModules routersPubV4;
# river reaches the ONT over its 100G `lan` uplink to the dave switch (which downlinks to
# brian, where the ONT lands). Digiweb delivers the ISP VLAN (pon-isp, 10) single-tagged at
# the ONT alongside the ONT's untagged management traffic. With a single ONT there's no VLAN
# collision, so the switches simply trunk the ISP's VLAN 10 straight through to river (PPPoE
# runs directly on it) and PVID the ONT's untagged management port onto wan-pon-ont (140).
# river takes .100 in the ONT's /24 (matching stream's modem-mgmt .100) to reach its web
# UI at 192.168.100.1. (See docs/sites/home/switches.md for the switch side and the multi-ONT plan.)
ontV4 = net.cidr.host 100 prefixes.ont.v4;
# river is routing-common index 0; the Digiweb static IP we request via IPCP
pubV4 = elemAt routersPubV4 0;
in
{
imports = [
@@ -71,11 +85,75 @@
dmeventd.enable = true;
};
fstrim.enable = true;
# TODO: re-enable once scheduling is tested
networkd-dispatcher.enable = mkForce false;
pppd = {
enable = true;
peers.digiweb = {
autostart = true;
enable = true;
# Password is shared across all Digiweb customers, so no need for a secret
config = ''
plugin pppoe.so wan-pon-isp
name "digiweb@nga.digiweb.ie"
password "digiweb"
# request our static IP as the local address in IPCP (local:remote, remote left open)
${pubV4}:
# no usepeerdns: we ignore Digiweb's resolvers and use the local recursive resolver
lcp-echo-interval 1
lcp-echo-failure 4
noauth
persist
maxfail 0
holdoff 5
mtu 1500
mru 1500
noaccomp
default-asyncmap
ifname wan
'';
};
};
};
# PPPoE WAN (Digiweb): pppd owns the `wan` interface on top of wan-pon-isp (the switch's
# swap of the ISP's VLAN 10), and its ip-up/ip-down hooks toggle the shared
# wan-online.target. Nothing else Wants the target, so it stays inactive until the link
# is actually up.
systemd.targets.wan-online.unitConfig.DefaultDependencies = false;
environment.etc = {
ppp-up = {
target = "ppp/ip-up";
mode = "0755";
text = ''
#!${pkgs.runtimeShell}
${pkgs.iproute2}/bin/ip route add default dev wan scope link metric 100
${config.systemd.package}/bin/systemctl --no-block start wan-online.target
'';
};
ppp-down = {
target = "ppp/ip-down";
mode = "0755";
text = ''
#!${pkgs.runtimeShell}
${config.systemd.package}/bin/systemctl --no-block stop wan-online.target
${pkgs.iproute2}/bin/ip route del default dev wan scope link metric 100
'';
};
};
systemd.network = {
netdevs = mkMerge [
(mkVLAN "wan-pon-ont" vlans.wan-pon-ont)
# The ISP VLAN is trunked through untranslated, so this is the raw pon-isp (10)
(mkVLAN "wan-pon-isp" vlans.pon-isp)
];
links = {
"10-wan" = {
"10-wan-old" = {
matchConfig = {
# Matching against MAC address seems to break VLAN interfaces
# (since they share the same MAC address)
@@ -83,7 +161,7 @@
PermanentMACAddress = "e0:d5:5e:68:0c:6e";
};
linkConfig = {
Name = "wan";
Name = "wan-old";
RxBufferSize = 4096;
TxBufferSize = 4096;
};
@@ -101,8 +179,38 @@
};
};
networks = {
"55-lan" = {
# both WAN VLANs arrive single-tagged on the 100G uplink to dave: wan-pon-ont (140,
# the ONT's management, PVID-tagged at the brian edge) and the ISP's VLAN 10, trunked
# straight through
vlan = [ "wan-pon-ont" "wan-pon-isp" ];
};
# So we don't drop the IP we use to connect to NVMe-oF!
networks."60-lan-hi".networkConfig.KeepConfiguration = "static";
"60-lan-hi".networkConfig.KeepConfiguration = "static";
# ONT management: the brian edge PVIDs the ONT's untagged port onto wan-pon-ont, so give
# ourselves an address in its /24 to reach the ONT's web UI at 192.168.100.1.
"70-wan-pon-ont" = {
matchConfig.Name = "wan-pon-ont";
address = [ "${ontV4}/24" ];
linkConfig = {
RequiredForOnline = "no";
MTUBytes = "1500";
};
};
# pppd attaches PPPoE to this; just needs to be up with no L3. This is the ISP's
# VLAN 10 trunked straight through from the ONT (no switch translation; see "55-lan").
"71-wan-pon-isp" = {
matchConfig.Name = "wan-pon-isp";
linkConfig = {
RequiredForOnline = "no";
# baby jumbo: PPPoE's 8B overhead leaves a clean 1500 on ppp
MTUBytes = "1508";
};
networkConfig = networkd.noL3;
};
};
};
my = {
@@ -1,6 +1,6 @@
{
imports = [
# ./unifi.nix
./unifi.nix
./hass.nix
];
}
@@ -24,6 +24,15 @@ in
address = net.cidr.host (65536*5+1) prefixes.hi.v6;
};
};
core = {
inherit domain;
name = "unifi-ctr-core";
mtu = 1500;
ipv4 = {
address = net.cidr.host 21 prefixes.core.v4;
gateway = null;
};
};
};
configuration = { lib, config, pkgs, assignments, ... }:
@@ -48,7 +57,10 @@ in
};
systemd = {
network.networks."80-container-host0" = networkdAssignment "host0" assignments.hi;
network.networks = {
"80-container-host0" = networkdAssignment "host0" assignments.hi;
"80-lan-core" = networkdAssignment "lan-core" assignments.core;
};
};
services = {
+26 -1
View File
@@ -72,6 +72,10 @@ in
fsType = "ext4";
neededForBoot = true;
};
"/mnt/frigate" = {
device = "/dev/disk/by-label/frigate";
fsType = "ext4";
};
};
networking = { inherit domain; };
@@ -111,6 +115,13 @@ in
MTUBytes = toString lib.my.c.home.hiMTU;
};
};
"10-lan-core-ctrs" = {
matchConfig = {
Driver = "virtio_net";
PermanentMACAddress = "52:54:00:72:67:51";
};
linkConfig.Name = "lan-core-ctrs";
};
"10-lan-lo-ctrs" = {
matchConfig = {
Driver = "virtio_net";
@@ -131,6 +142,11 @@ in
linkConfig.RequiredForOnline = "no";
networkConfig = networkd.noL3;
};
"30-lan-core-ctrs" = {
matchConfig.Name = "lan-core-ctrs";
linkConfig.RequiredForOnline = "no";
networkConfig = networkd.noL3;
};
"30-lan-lo-ctrs" = {
matchConfig.Name = "lan-lo-ctrs";
linkConfig.RequiredForOnline = "no";
@@ -145,6 +161,11 @@ in
MACVLAN = mkForce "lan-hi-ctrs:host0 lan-lo-ctrs:lan-lo";
};
};
unifi = {
networkConfig = {
MACVLAN = mkForce "lan-hi-ctrs:host0 lan-core-ctrs:lan-core";
};
};
};
systemd.services = {
@@ -174,9 +195,13 @@ in
containers.instances =
let
instances = {
# unifi = {};
unifi = {};
hass = {
bindMounts = {
"/mnt/frigate" = {
mountPoint = "/var/lib/frigate";
readOnly = false;
};
"/dev/bus/usb/001/002".readOnly = false;
"/dev/video0".readOnly = false;
"/dev/serial/by-id/usb-Nabu_Casa_Home_Assistant_Connect_ZBT-1_ce549704fe38ef11a2c2e5d154516304-if00-port0" = {

Some files were not shown because too many files have changed in this diff Show More