dev/nixfiles
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Compare commits

merge into: dev/nixfiles:master
Branches Tags
dev/nixfiles:master dev/nixfiles:docs-kimi-new dev/nixfiles:docs dev/nixfiles:fastback-ssh
dev/nixfiles:installer
..
pull from: dev/nixfiles:docs-kimi-new
Branches Tags
dev/nixfiles:master dev/nixfiles:docs-kimi-new dev/nixfiles:docs dev/nixfiles:fastback-ssh
dev/nixfiles:installer

2 Commits
master .. docs-kimi-new

Author SHA1 Message Date
jackos1998 f89ad33c26 ci: Add docs assignment-table auto-updater
Update docs assignments / update (push) Successful in 57s
Details 
Regenerate the marked network-assignment tables in docs/ from
nixos.allAssignments (nix run .#update-docs-assignments, registered in
pkgs/default.nix), preserving hand-written Notes cells. The workflow
runs on pushes to the docs branch and commits any table updates back;
flip its trigger to master when the docs branch merges.
 2026-07-26 19:16:43 +01:00
jackos1998 d51f2d62b6 docs: Document the deployment 
Add a top-level README mapping the boxes and a full docs/ tree: topic
pages (architecture, networking, deployment), per-site box pages for
colony and home with containers nested under their hosts, remote and
mobile boxes, the installer, and the home switch fabric reference
(folded in from home-switches.md, with AGENTS.md and code comments
retargeted to its new home). Box pages carry marked assignment tables
that CI regenerates from nixos.allAssignments.

AGENTS.md points at the new docs and keeps its terse agent version of
the mechanics, referring to the topic pages for depth.
 2026-07-26 19:16:43 +01:00
112 changed files with 1999 additions and 5601 deletions
Expand all files Collapse all files
.agents/skills/upgrade-nixpkgs/SKILL.md
-33
View File
@@ -1,33 +0,0 @@
---
name: upgrade-nixpkgs
description: >-
Upgrade all four nixpkgs channels (unstable, stable, mine, mine-stable) and home-manager for this
flake: check for a NixOS stable bump, rebase the devplayer0 nixpkgs fork, update kernel and
release metadata, refresh pins, sweep version-gated TODOs, and review other inputs. Use when the
user wants to update/bump nixpkgs, refresh the pins, or do the periodic nixpkgs/home-manager
upgrade.
---
# Upgrade nixpkgs
Read [`docs/nixpkgs-upgrade.md`](../../../docs/nixpkgs-upgrade.md), the canonical procedure, and
follow its phases in order.
Key reminders (see the doc for the full steps):
- It is **guided, not automated** — do the mechanical and investigative work but stop at the ⏸
points: pushing the fork, resolving rebase conflicts, applying a stable-channel bump, choosing a
new release codename, and deleting version guards. Report the findings and let the user decide.
- **Check the current NixOS stable first** (Phase 1) — the fork's `devplayer0-stable` rebase target
and the `flake.nix` stable pins must agree on one release.
- **Re-verify the patch stack against freshly fetched upstream**, not stale refs — enumerate it with
`git log`; don't assume a remembered list.
- After pushing the rebased fork branches, **wait for the GitHub mirror to catch up** before
refreshing flake pins. The `nixpkgs-mine*` inputs fetch from GitHub, not the fork's primary
remote; verify both GitHub branch tips match the pushed local tips first.
- After refreshing the pins, update `lib/constants.nix` to the current explicit LTS and latest
kernel package attributes, and update the `lib/default.nix` version overlay's `YY.MM` prefix to
the current month. Change its codename only when the stable channel advances.
- After the cheap evaluations pass, build the actual devshell and one representative NixOS system
(prefer the local box). `nix flake check --no-build` does not expose dependency build failures;
this is especially important when updating build-tool inputs such as Determinate Nix.
.claude/settings.json
-18
View File
@@ -1,18 +0,0 @@
{
"permissions": {
"allow": [
"Bash(nix eval:*)",
"Bash(nix flake check:*)",
"Bash(nix build:*)",
"Bash(check-system:*)",
"Bash(build-system:*)",
"Bash(build-home:*)",
"Bash(git status:*)",
"Bash(git diff:*)",
"Bash(git log:*)",
"Bash(git show:*)",
"Bash(drill:*)",
"Bash(ping:*)"
]
}
}
.claude/skills/flash-openwrt/SKILL.md
-37
View File
@@ -1,37 +0,0 @@
---
name: flash-openwrt
description: >-
Flash a flake-built OpenWrt image onto one of the OpenWrt boxes (currently fergal): build the
image, pre-flight the box, back up its config, validate and stage the image, run sysupgrade, and
verify what came back. Use when the user wants to flash, reflash, upgrade or sysupgrade an OpenWrt
box, or after changing its baked-in package list.
---
# Flash an OpenWrt box
The canonical, agent-agnostic procedure lives in the repo at
[`docs/openwrt-flash.md`](../../../docs/openwrt-flash.md). Read it and follow the phases in order.
Key reminders (see the doc for the full steps):
- **Stop at the ⏸ before Phase 5.** Flashing reboots the box and cannot be interrupted partway.
Confirm with the user, and confirm a serial console is reachable, *before* writing anything.
- **Packages are baked into the image**, so a package change means a reflash. Edit the box's list in
[`openwrt/default.nix`](../../../openwrt/default.nix), rebuild, and check the built `.manifest` —
a package name that doesn't exist is not a build error, it just isn't in the image.
- **`scp` does not work** on these boxes (no `sftp-server`). Move files with
`ssh <box> 'cat > /dev/…' < file` and `ssh <box> 'cat …' > file`.
- **Detach the upgrade with `setsid`**, not `nohup` (absent on busybox). `sysupgrade` kills the SSH
session mid-run, and an attached run dies with it — possibly after the firmware is erased.
- **Never reach for `sysupgrade -c`.** It needs `/overlay/upper/etc` and aborts *after* erasing the
firmware when that is missing, which is exactly the initramfs case. Plain `sysupgrade` already
keeps everything in `/lib/upgrade/keep.d/`.
- **Poll SSH to detect the reboot, never ping.** Successful pings return in milliseconds, so a
"wait for down" loop completes instantly and reports nonsense. Sleep between probes; expect about
three minutes.
- **Verify after**, don't assume: revision, management address, package count against the manifest,
and that the new packages are present and running.
The images are declared in [`openwrt/default.nix`](../../../openwrt/default.nix); background on the
outputs and the pinned package feeds is in
[`docs/deployment.md`](../../../docs/deployment.md#openwrt-images).
.claude/skills/install-box/SKILL.md
-40
View File
@@ -1,40 +0,0 @@
---
name: install-box
description: >-
Install a new NixOS box into this flake, from bare hardware booted into the custom installer
through to a deployable system: probe the hardware, partition and format the disks, write the box
config and flake entry, run do-install, and document the box. Use when the user wants to install,
bootstrap, provision or add a new box/host/machine.
---
# Install a box
The canonical, agent-agnostic procedure lives in the repo at
[`docs/install-box.md`](../../../docs/install-box.md). Read it and follow the phases in order.
Key reminders (see the doc for the full steps):
- It is **guided, not automated** — stop at the ⏸ points: settling what the box actually is
(Phase 1), wiping and partitioning disks (Phase 3), and running `do-install` (Phase 6). The user
often wants to do the install step by hand.
- **Phase 1 is not derivable from the hardware.** Name, site, role, channel and whether the box gets
assignments now all have to come from the user. Ask before writing files.
- **`show-hw-config` is a shell alias**, so it needs `installer-shell bash -lic show-hw-config`.
Run it twice: once early for the kernel-module lists, once after mounting for the filesystems.
- **`git add` the new box directory before evaluating** — the flake reads through git, and an
untracked path fails as "Path … is not tracked by Git" rather than as a Nix error.
- **Validate with `check-system <host>`**, not `build-system` — evaluation catches module and option
errors cheaply.
- **Seed the SSH host key from the installer** (Phase 3) by copying `/etc/ssh/ssh_host_*` onto the
persist volume. The installer regenerates them each boot, so they are safe to adopt, and it means
`my.secrets.key` can be set and secrets encrypted before the install rather than after first boot.
- **Every box declares a secret even when its own config declares none** — `my.user` pulls in
`user-passwd.txt` by default — so setting `my.secrets.key` always requires
`ragenix --rekey-one secrets/user-passwd.txt.age`. Check with
`nix eval .#nixosConfigurations.<host>.config.age.secrets --apply builtins.attrNames` rather than
assuming there is nothing to do. Re-encrypt selectively; `ragenix --rekey` rewrites every secret
in `secrets/` and drowns the real change in churn.
- Take **everything** useful out of `show-hw-config`, not just the modules and filesystems — drop an
option only when a nixfiles module already sets it.
- Finish with Phase 8: box page, site index row, `networking.md` prose. Don't hand-edit anything
between `<!-- ... -->` markers.
.claude/skills/upgrade-nixpkgs
-1
View File
@@ -1 +0,0 @@
../../.agents/skills/upgrade-nixpkgs
.envrc
+1 -3
View File
@@ -1,4 +1,2 @@
watch_file devshell/{default,commands,install,vm-tasks}.nix
# --accept-flake-config trusts the flake's nixConfig (our Harmonia cache) non-interactively, so
# direnv doesn't stall on the trust prompt.
use flake . --accept-flake-config
use flake
.gitea/workflows/ci.yaml
+5 -14
View File
@@ -10,22 +10,17 @@ jobs:
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v6
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: cachix/install-nix-action@v31
with:
# Gitea will supply a token in GITHUB_TOKEN, which this action passes to
# Nix (as access-tokens) when downloading from GitHub
github-token: ${{ secrets.GH_PULL_TOKEN }}
extra-conf: |
# Gitea will supply a token in GITHUB_TOKEN, which this action will
# try to pass to Nix when downloading from GitHub
github_access_token: ${{ secrets.GH_PULL_TOKEN }}
extra_nix_config: |
# Make sure we're using sandbox
sandbox-fallback = false
# Big C++ projects fill up memory...
cores = 6
# Determinate performance features
lazy-trees = true
eval-cores = 0
accept-flake-config = true
extra-substituters = https://nix-cache.nul.ie
extra-trusted-public-keys = nix-cache.nul.ie-1:BzH5yMfF4HbzY1C977XzOxoPhEc9Zbu39ftPkUbH+m4=
@@ -37,10 +32,6 @@ jobs:
env:
HARMONIA_SSH_KEY: ${{ secrets.HARMONIA_SSH_KEY }}
run: |
echo "::group::Collect cache garbage"
ci/push-to-cache.sh --gc
echo "::endgroup::"
nix eval --json --apply "builtins.attrNames" .#ci.x86_64-linux | jq -cr '.[]' | while read job; do
echo "::group::Build $job"
nix build --no-link .#ci.x86_64-linux."$job"
.gitea/workflows/installer.yaml
+12 -14
View File
@@ -7,22 +7,20 @@ on:
jobs:
installer:
name: Build installer
runs-on: ubuntu-26.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v6
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: actions/checkout@v4
- name: Set up Go
uses: https://github.com/actions/setup-go@v4
with:
# Gitea will supply a token in GITHUB_TOKEN, which this action passes to
# Nix (as access-tokens) when downloading from GitHub
github-token: ${{ secrets.GH_PULL_TOKEN }}
extra-conf: |
go-version: '>=1.20.1'
- uses: cachix/install-nix-action@v27
with:
github_access_token: ${{ secrets.GH_PULL_TOKEN }}
extra_nix_config: |
# Make sure we're using sandbox
sandbox-fallback = false
# Determinate performance features
lazy-trees = true
eval-cores = 0
extra-substituters = https://nix-cache.nul.ie
extra-trusted-public-keys = nix-cache.nul.ie-1:BzH5yMfF4HbzY1C977XzOxoPhEc9Zbu39ftPkUbH+m4=
@@ -42,10 +40,10 @@ jobs:
jackos-installer-netboot-${{ steps.setup.outputs.short_rev }}.tar.zst
- name: Create release
uses: https://gitea.com/actions/gitea-release-action@main
uses: https://gitea.com/actions/release-action@main
with:
name: Latest installer
token: '${{ secrets.RELEASE_TOKEN }}'
title: Latest installer
api_key: '${{ secrets.RELEASE_TOKEN }}'
files: |
jackos-installer-${{ steps.setup.outputs.short_rev }}.iso
jackos-installer-netboot-${{ steps.setup.outputs.short_rev }}.tar.zst
.gitea/workflows/update-docs.yaml
+5 -18
View File
@@ -1,12 +1,12 @@
name: Update docs
name: Update docs assignments
on:
push:
branches: [master]
branches: [docs-kimi-new]
jobs:
update:
if: "!contains(github.event.head_commit.message, 'docs: Update generated references')"
if: "!contains(github.event.head_commit.message, 'docs: update assignment tables')"
runs-on: ubuntu-26.04
permissions:
contents: write
@@ -23,29 +23,16 @@ jobs:
- name: Update assignment tables
run: nix run .#update-docs-assignments
- name: Update option reference
run: nix run .#update-docs-options
- name: Update DNS reference
run: >
nix run .#update-docs-dns --
ams1.int.nul.ie
100.10.in-addr.arpa
2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa
h.nul.ie
168.192.in-addr.arpa
0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa
- name: Commit and push if changed
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
REPO_URL: ${{ gitea.repositoryUrl }}
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
REPO_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}"
git remote set-url origin "${REPO_URL/https:\/\//https:\/\/oauth2:${GITEA_TOKEN}@}"
git add docs/
if ! git diff --cached --quiet; then
git commit -m "docs: Update generated references"
git commit -m "docs: update assignment tables"
git push
fi
.gitignore
-1
View File
@@ -4,4 +4,3 @@ result*
!/.vms/.gitkeep
/.keys/*.key
*.swp
/.claude/settings.local.json
AGENTS.md
+45 -145
View File
@@ -6,24 +6,13 @@ This file provides guidance to coding agents when working with code in this repo
tools refuse to write through a symlink and will error on `CLAUDE.md`).
**Prefer this file over agent memory.** When you learn something durable about this repo — a
convention, a workflow gotcha, a design rationale — record it here (or in a repo doc this file
points to, e.g. `docs/sites/home/switches.md`), not in agent memory. AGENTS.md is versioned and
shared; memory is not.
Keep this file lean: it should contain only rules, workflows, safety constraints and sharp gotchas
an agent needs before starting work. Detailed inventories, topology, service operation and design
rationale belong in the canonical document under `docs/`; summarize only the essential constraint
here and link to that document instead of maintaining a second copy.
Claude Code permissions live in two files: `.claude/settings.json` (versioned, shared — the
committed allow list of safe-to-auto-approve commands) and `.claude/settings.local.json` (personal,
gitignored — where interactive "always allow" grants accumulate). Put durable, generally-safe
commands in the shared file; leave one-off or workstation-specific grants in the local one.
convention, a workflow gotcha, a design rationale — record it here (or in a repo doc this file points
to, e.g. `docs/sites/home/switches.md`), not in agent memory. AGENTS.md is versioned and shared; memory is not.
## Overview
Personal Nix flake managing NixOS systems and home-manager configurations for a set of
**boxes**, never a "fleet". It is built around a **custom module
machines — always called **"boxes"**, never "fleet". It is built around a **custom module
system** layered on top of NixOS/home-manager, not the stock flake `nixosConfigurations` pattern.
## Commands
@@ -46,10 +35,7 @@ Common ones:
Pass the flake-qualified node, e.g. `deploy .#git`. The deploy node name is **always** the system
name (`deploy-rs.nix` keys nodes directly off `nixos.systems` / `home-manager.homes`); a system is
only a deploy target when `config.my.deploy.enable` is true (defaults true; auto-disabled for dev
VMs and containers). A container is **not** its own deploy node — it is generated as a
`container-<name>` profile on its **host** node. So `deploy .#<host>` deploys the host's `system`
profile and every one of its containers, whereas `deploy .#<host>.container-<name>` targets a
single container (e.g. `deploy .#shill.container-middleman`). Pass `--boot` to stage a config as the boot default **without** live-switching
VMs and containers). Pass `--boot` to stage a config as the boot default **without** live-switching
(`deploy --boot .#<host>`) — the box keeps running its current generation until it reboots. Use this
when a live `switch` would break connectivity mid-change (e.g. a router's WAN VLAN rework), then
reboot to cut over.
@@ -61,32 +47,13 @@ Common ones:
`SSH_AUTH_SOCK= ssh-machine …` (or add `-o IdentityAgent=none` to a raw `ssh`).
- `ragenix` — edit age secrets using `.keys/dev.key` as identity (see Secrets).
- `repl` — `nix repl .#`.
- `installer-shell` / `do-install <system>` — drive an install against a booted installer at
`$INSTALLER`. For bringing up a new box end to end follow the guided procedure in
[`docs/install-box.md`](docs/install-box.md).
- `update-nixpkgs` / `update-home-manager` — bump pinned inputs. For the full periodic upgrade
(rebasing the `devplayer0` nixpkgs fork, stable-release bumps, kernel and release-metadata
refreshes, version-gate sweep, input review) follow the guided procedure in
[`docs/nixpkgs-upgrade.md`](docs/nixpkgs-upgrade.md).
- `update-nixpkgs` / `update-home-manager` — bump pinned inputs.
Use the narrowest relevant evaluation while iterating: `check-system <host>` for a box config,
`nix eval .#nixfiles.config.nixos.allAssignments --json` for assignment generation, or
`nix build --no-link .#nixfiles.config.nixos.optionsDoc` for the option reference. Reserve
`nix flake check --no-build` for final broad validation or reproducing CI.
Check everything (what CI runs): `nix flake check --no-build`.
CI builds each attr of `.#ci.x86_64-linux` (systems, homes, packages, shell) and pushes to the
Harmonia binary cache; see `.gitea/workflows/ci.yaml` and `ci/push-to-cache.sh`. A separate
workflow (`.gitea/workflows/update-docs.yaml`) regenerates the network assignments, NixOS option
reference and live DNS reference under `docs/`.
For DNS lookups use **`drill`** (ldns) — `dig` isn't installed in this environment (it fails with
exit 127, which is easy to miss if stderr is redirected). E.g. `drill -Q @<resolver> <name> A`.
For privilege escalation use **`doas`**, not `sudo` — the boxes don't install `sudo` (it fails with
`command not found`). E.g. `doas ip link set <if> up`.
For ad-hoc packages prefer **`pkgs#<name>`** over `nixpkgs#<name>` (e.g. `nix run pkgs#python3`):
`pkgs` is a system registry alias for the same pinned nixpkgs the boxes are built from, so it
resolves from the local store instead of fetching a different channel.
workflow (`.gitea/workflows/update-docs.yaml`) regenerates the network-assignment tables in `docs/`
via `nix run .#update-docs-assignments`.
## Architecture
@@ -147,30 +114,45 @@ via the `vms` / `containers` modules and the `l2mesh` VXLAN module.
For a human-readable map of what is actually deployed (per-box roles, services and networking),
see `README.md` and `docs/` (index at `docs/README.md`; box pages under `docs/sites/`,
`docs/remote/`, `docs/mobile/`). Keep these in sync when adding, removing or repurposing a box or
service. The network-assignment tables are consolidated in `docs/networking.md` (one per site,
CI-generated from `allAssignments` between `<!-- assignments: <site> -->` markers); box pages
link to that section. Write the prose and let the updater refresh the tables.
service. The network-assignment tables in box pages are CI-generated from `allAssignments` (the
`<!-- assignments: <name> -->` markers) — write the prose and let the updater refresh the tables.
### Home routers (`nixos/boxes/home/routing-common`)
`river` and `stream` share `routing-common`, a function of an `index`; keep common HA behavior there
and box-specific WAN behavior in the respective box file. Client-facing gateway and DNS services
must use the floating VIPs. `routing-common` only declares the inert `wan-online.target`; each box
wires its own reachability mechanism. See `docs/networking.md#router-ha` and the router box pages
for the design and operational detail. Keep pair-wide behavior in `docs/networking.md`; router
pages should document only their WAN, platform and other box-specific responsibilities.
The two home routers, `river` and `stream`, share `routing-common`, which is a **function of an
`index`** (`import ../../routing-common 0` for river, `1` for stream). The index derives per-box
addresses, keepalived VRRP priorities/state, DNS `ns` numbering, etc., so the two boxes are an
active/backup HA pair from one definition. They differ where hardware/uplink differ: `stream` has a
DHCP WAN, `river` runs PPPoE (`services.pppd`, Digiweb) — box-specific bits live in the respective
box file, not `routing-common`.
The easy-to-mix-up implementation details are that `lib.my.networkdAssignment` and
`lib.my.mkVLAN` live under `lib.my`, networkd snippet constants such as `networkd.noL3` live under
`lib.my.c`, and interface MTU belongs in the `.network`'s `linkConfig.MTUBytes`.
- **HA is VRRP (`keepalived`).** Per-VLAN floating **VIPs** (`lib.my.c.home.vips`) are what clients
use as both gateway *and* DNS server. `kea` (DHCP) and `radvd` (RAs; started only on the master)
hand out the VIP, and `pdns-recursor` binds the VIPs (with `net.ipv*.ip_nonlocal_bind` so the
backup can pre-bind). Point client-facing services at the VIP, not a box's real address, so
failover follows the master instead of relying on client resolver timeouts.
- **`wan-online.target`** is a shared abstract target meaning "the public WAN/IPv4 route is up".
`routing-common` only declares it; each box wires *how it is reached* (`stream`: a oneshot that
waits for the DHCP default route; `river`: the pppd `ip-up`/`ip-down` hooks). Services that need
the WAN attach **to** it via `wantedBy` + `partOf` + `after` (not `requires`/`wants`), so an empty
target is never pulled in and prematurely activated, and they re-load on WAN flap.
- networkd helpers used heavily here: `lib.my.networkdAssignment` and `lib.my.mkVLAN` live under
**`lib.my`**, while networkd snippet constants like `networkd.noL3` live under **`lib.my.c`** —
easy to mix up. Set an interface MTU via the `.network`'s `linkConfig.MTUBytes` (`[Link]`), not
`netdevConfig` (`[NetDev]` rejects `MTUBytes`).
### Home switches (`jim` / `dave` / `brian`)
These switches are not managed by the flake, and changing them is out-of-band and hard to revert.
Read `docs/sites/home/switches.md` before WAN or VLAN work, always confirm before applying a switch
change, and keep the documented layout and `lib.my.c.home.vlans` in sync.
### Home wireless APs (`vibe` / `wave`)
These APs are not managed by the flake; only their DNS records are. Read and update
`docs/sites/home/aps.md` when changing an AP or its VLAN layout.
The home boxes and the Digiweb WAN hang off hand-configured switches that are **not** managed by
this flake: `jim` and `dave` (MikroTik, RouterOS) and `brian` (Ubiquiti, UniFi). The full topology,
VLAN map, and the ONT/WAN path live in **`docs/sites/home/switches.md`** — read it before
touching anything WAN/VLAN-related, and update it when the switch layout changes.
- **Access:** the switches resolve by **short hostname** on the home network (the routers serve
their records in the home zone — `routing-common/dns.nix`: `jim`/`dave`/`brian`). From a home box,
SSH to the MikroTiks as `admin`/`admin` (e.g. `ssh admin@jim`); `brian` is configured via the
UniFi controller, not a CLI.
- **Changing switch config is out-of-band and hard to revert — always confirm before applying:**
print the affected menu, make the change, then re-verify. The nix config and the switches must
agree on VLAN numbering (e.g. `lib.my.c.home.vlans`), so a switch-side change usually pairs with a
box change; `docs/sites/home/switches.md` documents the switch layout and per-switch config for the WAN design.
## Secrets
@@ -181,14 +163,9 @@ recipient key list (always including `.keys/dev.pub`). Edit secrets with the `ra
command, which supplies `.keys/dev.key` as the identity. The `.keys/` directory (dev + deploy
private keys) is required for editing secrets, deploying, and running dev VMs.
When a recipient list changes, re-encrypt selectively with `ragenix --rekey-one <file>` for each
affected secret. `ragenix --rekey` rewrites **every** secret in `secrets/`, burying the real change
in churn.
## Conventions
- Format with `nixpkgs-fmt` (`fmt`). 2-space indent, `inherit (...)` blocks at the top of `let` —
prefer `inherit (lib) mkOption ...;` (and bare use) over qualifying inline as `lib.mkOption`.
- Format with `nixpkgs-fmt` (`fmt`). 2-space indent, `inherit (...)` blocks at the top of `let`.
**Ask before running `fmt`** — some files aren't canonically formatted, so `fmt` can reindent a
whole file and bury a logical change in whitespace churn. Match the surrounding style by hand and
leave formatting to the user unless they ask.
@@ -203,83 +180,6 @@ in churn.
as `overlays.default`.
- In prose and commit messages, quote code-like identifiers (commands, options, paths, package and
attribute names) in backticks.
- Keep docs and prose plain — avoid vague AI-tell filler. In particular use "layout"/"structure"
rather than "shape", and "General" rather than "cross-cutting topics"; name things directly
instead of reaching for umbrella words.
- For network connectivity, use the concrete term — "interface", "connection", "network
attachment" or the network's name — rather than calling it a "leg".
- Call the systems **"boxes"**, never "machines" or "fleet", except where "machine" is part of a
command, option or upstream technical term such as QEMU's machine type.
- Call the machines **"boxes"**, never "fleet".
- Commit subjects follow `area/scope: Capitalized summary` (e.g. `nixos/home: ...`); keep logically
distinct changes in separate commits. Aim for 50-character subjects and do not exceed 72
characters. Hard-wrap commit body lines at 72 characters; Git preserves an unwrapped `-m`
argument as one long line, so include literal line breaks or use a commit-message file. Before
reporting a commit, inspect `git show -s --format=%B HEAD` and amend it if any line exceeds 72
characters. A concise body describing the change and its rationale is welcome when the subject
alone does not provide enough context — keep it to the essentials rather than restating the diff.
`Co-Authored-By` is the only trailer used here; do **not** add a `Claude-Session` link (or any
other session/tooling trailer).
- **"Logically distinct" means unrelated** — two different applications, two boxes that have nothing
to do with each other, a drive-by fix that happens to sit in a file you were editing anyway. One
piece of work stays in one commit even when it touches a config, several docs and a switch: if the
parts only make sense together, splitting them just makes each half unreviewable. Err towards one
commit and split when a reader would ask why two things arrived together.
## Documentation
Human-readable docs live under `docs/` (index: `docs/README.md`). Box pages are under
`docs/sites/<site>/`, `docs/remote/`, `docs/mobile/`. A box that itself hosts sub-systems
(containers/nested VMs) gets a **directory named for it** with a `README.md` and the child pages
beneath it (e.g. `shill/README.md` + `shill/containers/*.md`, `sfh/README.md`). When a site and its
physical box share a name, the site keeps the `README.md` and the box page stays alongside it (for
example `sites/colony/README.md` and `sites/colony/colony.md`).
**Box page layout** (match the existing pages): H1 + a one-line intro; a short bullet list of
`Source` / `Host` / `nixpkgs`; an optional hardware inventory or VPS resource-allocation section;
`## Role`; `## Network assignments` that **links** to
[`networking.md#box-assignments`](docs/networking.md#box-assignments) (never inline the table); one
`##` section per topic; `## Notable config files` last. Keep non-hardware platform details in their
topical sections rather than moving them with the inventory. A box without static assignments still
gets the section with a short explanation instead of a generated-table link.
**Structure and layout:**
- Use **tables** for lists of structured items (BGP peers, forwarded ports, vhosts, containers,
VMs, VLANs). Make the item **name the link**; don't add a separate `Page`/`Docs` column.
- Break up **long prose**: any bullet running past ~4 lines over several distinct facts becomes a
`###` subheading (lead sentence + nested bullets). Don't leave walls of long bullets.
- Prefer durable concepts and named configuration identifiers over copying exact numbers that are
tunable, generated or likely to drift. Exact values are appropriate when they are useful parts of
the inventory or interface: VM resource allocations, hardware-fixed properties, network and
protocol identifiers, exposed service ports, and safety or recovery values. Describe mutable
implementation tunables by purpose and link to their source instead of duplicating the current
value.
- **Per-site index pages** (`docs/sites/*/README.md`, `docs/remote/README.md`, and
`docs/mobile/README.md`) use one table listing **boxes only**; don't repeat that inventory in a
diagram. A box's containers / nested VMs are tabulated on that box's own page. The global
`docs/README.md` is the exception: it may show them in high-level site diagrams, but should not
add a second detailed inventory outside those diagrams.
**De-dup by ownership** — each fact has one home:
- Shared cross-box **fabric** (the AS211024 mesh, Tailscale/headscale topology, the BGP overview,
the WireGuard-tunnel summary) is documented once in `docs/networking.md`; box pages give a
one-line summary and link to it.
- Site-wide network definitions (prefixes, VLANs, router VIPs and router HA) also live in
`docs/networking.md`; site pages summarize and link to the canonical section.
- **Box-specific** detail (a box's own BGP peers, the WireGuard tunnels it terminates, its
services) lives on the box page; `networking.md` carries only a per-box summary that links out.
- Addresses represented by generated `assignments` or `extraAssignments` live only in the
assignment tables; handwritten box and workload inventories link there instead of copying them.
Addresses outside that data model (such as external peers or service endpoints) stay with the
topic that owns them.
**Generated content:** the network-assignment tables in `networking.md`, the option reference
(`docs/reference/nixos-options.md`) and the live DNS tables (`docs/reference/dns.md`) are
CI-generated by the corresponding `update-docs-*` packages; the workflow supplies the DNS zones.
Don't hand-edit content between `<!-- ... -->` markers; write the surrounding prose or source
configuration and let the updater refresh the tables. The option-reference file is generated in
full.
**Keep docs current:** when you add, remove or repurpose a box or service, update its box page, the
relevant site-index `README.md`, and any affected prose in `networking.md` (the assignment/option
tables refresh via CI). This is the same "prefer docs over agent memory" rule from the top of this
file.
distinct changes in separate commits.
README.md
+55 -31
View File
@@ -1,14 +1,14 @@
# nixfiles
Personal Nix flake managing every box I run: hosted servers, home
Personal Nix flake managing every machine I run: hosted servers, home
infrastructure, routers, a remote site, VPSes and personal workstations. It is
built around a **custom module system** layered on top of NixOS and
home-manager rather than the stock per-host `nixosConfigurations` pattern.
For the module-system internals, see [`docs/architecture.md`](docs/architecture.md); for day-to-day
commands, deployment and secrets, see [`docs/deployment.md`](docs/deployment.md). This README is the
map of **what is actually deployed**; the per-box details live under [`docs/`](docs) (start at
[`docs/README.md`](docs/README.md)).
For day-to-day commands and a deeper explanation of the module system,
conventions and secrets, see [`AGENTS.md`](AGENTS.md). This README is the map of
**what is actually deployed**; the per-machine details live under [`docs/`](docs)
(start at [`docs/README.md`](docs/README.md)).
> **Note:** This documentation (the README and everything under `docs/`) is a
> work in progress and was **agent-generated** from the repository. It may be
@@ -16,50 +16,74 @@ map of **what is actually deployed**; the per-box details live under [`docs/`](d
## The boxes at a glance
Boxes are grouped by deployment/location. Each group has its own directory
under `docs/` with a `README.md` overview and one page per box.
Machines are grouped by deployment/location. Each group has its own directory
under `docs/` with a `README.md` overview and one file per machine.
| Group | What it is |
| --- | --- |
| [**colony**](docs/sites/colony) | Hosted dedicated server in Amsterdam (`ams1`). A VM host running the public-facing infrastructure: routing, web, git, media, object storage, chat, game servers. |
| [**home**](docs/sites/home) | Home network: a VM host (`palace`), the home routers, storage, Home Assistant, and a workstation — plus the hand-configured switches and wireless APs tying it together. |
| [**remote**](docs/remote) | Edge VPSes (`britway`, `britnet`) and the remote `kelder` site. |
| [**mobile**](docs/mobile) | The `tower` laptop. |
| Group | What it is | Docs |
| --- | --- | --- |
| **colony** | Hosted dedicated server in Amsterdam (`ams1`). A VM host running the public-facing infrastructure: routing, web, git, media, object storage, chat, game servers. | [`docs/sites/colony/`](docs/sites/colony) |
| **home** | Home network: a VM host (`palace`), the home routers, storage, Home Assistant, and personal desktops — plus the hand-configured switches tying it together. | [`docs/sites/home/`](docs/sites/home) |
| **remote** | Edge VPSes (`britway`, `britnet`) and the remote `kelder` site. | [`docs/remote/`](docs/remote) |
| **mobile** | The `tower` laptop. | [`docs/mobile/`](docs/mobile) |
The custom installer image is documented at [`docs/misc/installer.md`](docs/misc/installer.md).
The general topics — the module system, network and deployment — live next to the index:
[`docs/architecture.md`](docs/architecture.md), [`docs/networking.md`](docs/networking.md) and
[`docs/deployment.md`](docs/deployment.md). The generated custom-module option reference is at
[`docs/reference/nixos-options.md`](docs/reference/nixos-options.md).
Cross-cutting topics — the module system, the network, deployment — live next to
the index: [`docs/architecture.md`](docs/architecture.md),
[`docs/networking.md`](docs/networking.md), [`docs/deployment.md`](docs/deployment.md).
The high-level site diagrams and box hierarchy live in [`docs/README.md`](docs/README.md).
Networking is largely defined by per-box `assignments` plus the AS211024 L2 VXLAN mesh; see
[`docs/networking.md`](docs/networking.md) for the full picture and
[`docs/architecture.md`](docs/architecture.md) for the implementation mechanics.
## The "big machine" pattern
The larger sites (`colony`, `home`) all follow the same shape:
```
physical host (VM host)
├── VM ── thing impractical to containerise (router, storage, podman host, …)
├── VM ── container host ──┬── NixOS container ── application
│ ├── NixOS container ──┬── application
│ │ ├── application
│ │ └── application
│ └── …
└── VM ── …
```
- A **physical host** (`colony`, `palace`) does little itself beyond running
**VMs** via the custom `my.vms` module (QEMU + systemd units, LVM-backed
disks).
- **VMs** exist for things that are impractical to put in a container — kernel
features, separate networking, podman/OCI workloads, foreign OSes.
- One VM is usually a **container host** (`shill` on colony, `sfh` on home; and
`kelder` directly). It runs **NixOS containers** via the custom `my.containers`
module (systemd-nspawn based, each with its own address on a bridge).
- **Most applications live in those NixOS containers.** A container isn't limited
to a single application — it commonly hosts a **group of related applications**
that belong together (e.g. `jackflix` runs Jellyfin, the *arr stack,
Transmission, PhotoPrism and copyparty; `object` runs MinIO, Harmonia, HedgeDoc
and wastebin). Each container is a first-class entry in the docs.
Networking between everything is largely defined by per-system `assignments`
(IPs/prefixes) plus an L2 VXLAN mesh (`my.vpns.l2`, AS211024) that ties the edge
routers together. See [`docs/networking.md`](docs/networking.md) for the full
picture and [`AGENTS.md`](AGENTS.md) for the mechanics.
## Repo layout
```
README.md <- you are here
nixos/
boxes/ per-box configuration
boxes/ per-machine configuration ("boxes")
colony/ colony host + its VMs (vms/) + shill's containers
home/ palace host + its VMs, routing-common, plus stream, castle
britway/ London VPS
kelder/ remote box + its containers
tower/ laptop
britnet.nix Birmingham VPS
installer.nix installer-image configuration
britway/ britnet.nix, kelder/, tower/, installer.nix …
modules/ shared NixOS modules (my.* options); registered in _list.nix
home-manager/ home-manager modules + configs
lib/ lib.my helpers, constants (lib.my.c), net/dns helpers
pkgs/ custom packages (overlays.default)
secrets/ age-encrypted secrets (ragenix)
devshell/ devshell commands (build/deploy/check/ssh helpers)
ci/ CI helpers (binary-cache push, docs generators)
ci/ CI helpers (binary-cache push, docs assignment-table updater)
docs/ deployment documentation (index at docs/README.md)
```
A box is wired into the flake by adding its config file to the `configs` list
in `flake.nix`. See [the top-level evaluation](docs/architecture.md#the-top-level-evaluation) for
how `evalModules` turns these into `nixosConfigurations`, `homeConfigurations` and `deploy` nodes.
A machine is wired into the flake by adding its box file to the `configs` list
in `flake.nix`. See [`AGENTS.md`](AGENTS.md#architecture) for how `evalModules`
turns these into `nixosConfigurations`, `homeConfigurations` and `deploy` nodes.
ci/push-to-cache.sh
+3 -14
View File
@@ -10,14 +10,6 @@ remote_cmd() {
ssh -i "$SSH_KEY" "$SSH_HOST" env HOME=/run/harmonia NIX_REMOTE="$REMOTE_STORE" "$@"
}
collect_garbage() {
echo "Collecting garbage..."
remote_cmd nix-env \
-p "$REMOTE_STORE"/nix/var/nix/profiles/nixfiles \
--delete-generations 60d
remote_cmd nix-collect-garbage
}
umask_old=$(umask)
umask 0066
echo "$HARMONIA_SSH_KEY" | base64 -d > "$SSH_KEY"
@@ -25,12 +17,6 @@ umask $umask_old
mkdir -p ~/.ssh
cp ci/known_hosts ~/.ssh/
if [ "${1-}" = "--gc" ]; then
collect_garbage
exit
fi
path="$1"
echo "Pushing $path to cache..."
@@ -39,4 +25,7 @@ nix copy --no-check-sigs --to "$STORE_URI" "$path"
if [ -n "$UPDATE_PROFILE" ]; then
echo "Updating profile..."
remote_cmd nix-env -p "$REMOTE_STORE"/nix/var/nix/profiles/nixfiles --set "$path"
echo "Collecting garbage..."
remote_cmd nix-collect-garbage --delete-older-than 60d
fi
ci/update-docs-assignments.py
Executable → Regular
+76 -142
View File
@@ -1,10 +1,5 @@
#!/usr/bin/env python3
"""Update the consolidated network-assignment tables in docs/networking.md.
Reads nixos.allAssignments from the flake and renders assignment-key subgroups for each site
(colony / home / remote / other) between per-site `<!-- assignments: <site> -->` markers. The
hand-written Notes column is preserved, keyed by (box, assignment).
"""
"""Update assignment tables in docs/ from nixos.allAssignments."""
import json
import re
@@ -12,167 +7,102 @@ import subprocess
import sys
from pathlib import Path
DOCS = Path("docs")
TARGET = DOCS / "networking.md"
HEADER = "| Box | IPv4 | IPv6 | Domain | Notes |"
SEP = "|---|---|---|---|---|"
# Table order; also the set of valid `<!-- assignments: <site> -->` tokens.
SITE_ORDER = ["colony", "home", "remote", "other"]
# Per-site internal domains of the remote boxes (britway=lon1, britnet=bhx1, kelder=hentai).
REMOTE_DOMAINS = ("lon1.int.nul.ie", "bhx1.int.nul.ie", "hentai.engineer")
def site_of(assignments: dict) -> str:
"""Classify a box into a site table by its assignment domain.
Prefer the `internal` assignment's domain, else the first assignment that has one.
"""
dom = (assignments.get("internal") or {}).get("domain")
if not dom:
for a in assignments.values():
if a.get("domain"):
dom = a["domain"]
break
if not dom:
return "other"
if dom.endswith("ams1.int.nul.ie"):
return "colony"
if dom == "h.nul.ie" or dom.endswith(".h.nul.ie"):
return "home"
if any(dom.endswith(s) for s in REMOTE_DOMAINS):
return "remote"
return "other"
DOCS_DIR = Path("docs")
HEADER = "| Name | Assignment | IPv4 | IPv6 | Domain | Notes |"
def fmt_ip(ip: dict) -> str:
addr = ip.get("address")
if addr is None:
return "—"
parts = [f"{addr}/{ip.get('mask')}"]
if ip.get("gateway") is not None:
parts.append(f"gw {ip['gateway']}")
mask = ip.get("mask")
gateway = ip.get("gateway")
parts = [f"{addr}/{mask}"]
if gateway is not None:
parts.append(f"gw {gateway}")
return f"`{' '.join(parts)}`"
def find_page(box: str) -> str | None:
"""Doc page for a box (relative to docs/), if one exists."""
for pattern in (f"{box}.md", f"{box}/README.md"):
for p in sorted(DOCS.rglob(pattern)):
return p.relative_to(DOCS).as_posix()
return None
def box_cell(box: str) -> str:
page = find_page(box)
return f"[`{box}`]({page})" if page else f"`{box}`"
def box_name_from_cell(cell: str) -> str:
m = re.match(r"\[`?([^`\]]+)`?\]", cell)
return m.group(1) if m else cell.strip("`")
def parse_notes(lines: list[str]) -> dict[tuple[str, str], str]:
"""Existing Notes indexed by (box, assignment), from either table layout."""
notes: dict[tuple[str, str], str] = {}
assignment = None
def parse_notes(lines: list[str]) -> dict[str, str]:
"""Extract existing Notes indexed by Assignment from a marked table."""
notes: dict[str, str] = {}
for line in lines:
stripped = line.strip()
heading = re.match(r"^####\s+`([^`]+)`$", stripped)
if heading:
assignment = heading.group(1)
continue
if not stripped.startswith("|"):
continue
cells = [c.strip() for c in stripped.split("|")]
# "| a | b | ... |" splits to ['', 'a', 'b', ..., '']
if len(cells) >= 7 and assignment is not None:
# Current layout: one table under each assignment heading.
box, row_assignment, note = cells[1], assignment, cells[5]
elif len(cells) >= 8:
# Previous layout: one site table with an Assignment column.
box, row_assignment, note = cells[1], cells[2], cells[6]
else:
# Splitting "| a | b |" produces ['', 'a', 'b', '']
if len(cells) < 8:
continue
# Skip header and separator rows.
if box == "Box" or set(box) <= {"-"}:
key = cells[2]
if key == "Assignment":
continue
notes[(box_name_from_cell(box), row_assignment)] = note
notes[key] = cells[6]
return notes
def render_site(boxes: list[str], all_assignments: dict, notes: dict) -> list[str]:
groups: dict[str, list[tuple[str, dict]]] = {}
for box in sorted(boxes):
for key, a in all_assignments[box].items():
groups.setdefault(key, []).append((box, a))
lines: list[str] = []
group_order = sorted(groups, key=lambda key: (key != "internal", key))
for key in group_order:
if lines:
lines.append("")
lines.extend([f"#### `{key}`", "", HEADER, SEP])
for box, a in groups[key]:
lines.append(
"| "
+ " | ".join(
[
box_cell(box),
fmt_ip(a.get("ipv4", {})),
fmt_ip(a.get("ipv6", {})),
a.get("domain") or "—",
notes.get((box, key), ""),
]
)
+ " |"
def render_table(assignments: dict, notes: dict[str, str]) -> list[str]:
lines = [
HEADER,
"|---|---|---|---|---|---|",
]
for key, a in assignments.items():
name = a.get("name", key)
alt = ", ".join(a.get("altNames", []))
if alt:
name = f"{name} ({alt})"
lines.append(
"| "
+ " | ".join(
[
name,
key,
fmt_ip(a.get("ipv4", {})),
fmt_ip(a.get("ipv6", {})),
a.get("domain") or "—",
notes.get(key, ""),
]
)
+ " |"
)
return lines
def update_target(all_assignments: dict) -> bool:
by_site: dict[str, list[str]] = {s: [] for s in SITE_ORDER}
for box, assignments in all_assignments.items():
if not assignments:
continue
by_site[site_of(assignments)].append(box)
def process_file(path: Path, all_assignments: dict) -> bool:
box_name = path.stem
if box_name not in all_assignments:
return False
text = TARGET.read_text()
text = path.read_text()
lines = text.splitlines()
marker_re = re.compile(r"^<!--\s*assignments:\s*(\S+)\s*-->$")
for i, line in enumerate(lines):
m = marker_re.match(line.strip())
if m and m.group(1) == box_name:
end_idx = None
for j in range(i + 1, len(lines)):
if lines[j].strip() == "<!-- assignments-end -->":
end_idx = j
break
if end_idx is None:
return False
# Walk the file line by line; at each `<!-- assignments: <site> -->` marker, replace
# everything up to the matching `<!-- assignments-end -->` with a freshly rendered table
# (carrying the hand-written Notes over), leaving all other lines untouched.
out: list[str] = []
i = 0
while i < len(lines):
m = marker_re.match(lines[i].strip())
if not m:
out.append(lines[i])
i += 1
continue
site = m.group(1)
end = i + 1
while end < len(lines) and lines[end].strip() != "<!-- assignments-end -->":
end += 1
if end >= len(lines):
out.append(lines[i])
i += 1
continue
notes = parse_notes(lines[i + 1 : end])
out.append(lines[i])
out.append("<!-- assignments-start -->")
out.extend(render_site(by_site.get(site, []), all_assignments, notes))
out.append("<!-- assignments-end -->")
i = end + 1
old_inner = lines[i + 1 : end_idx]
notes = parse_notes(old_inner)
new_table = render_table(all_assignments[box_name], notes)
new_lines = (
lines[: i + 1]
+ ["<!-- assignments-start -->"]
+ new_table
+ ["<!-- assignments-end -->"]
+ lines[end_idx + 1 :]
)
if new_lines != lines:
path.write_text("\n".join(new_lines) + "\n")
return True
return False
new_text = "\n".join(out) + "\n"
if new_text != text:
TARGET.write_text(new_text)
return True
return False
@@ -185,9 +115,13 @@ def main() -> int:
)
all_assignments = json.loads(result.stdout)
if update_target(all_assignments):
print(f"updated {TARGET}")
return 0
changed = False
for path in sorted(DOCS_DIR.rglob("*.md")):
if process_file(path, all_assignments):
print(f"updated {path}")
changed = True
return 1 if changed else 0
if __name__ == "__main__":
ci/update-docs-dns.py
-362
View File
@@ -1,362 +0,0 @@
#!/usr/bin/env python3
"""Render docs/reference/dns.md from live authoritative DNS zone transfers.
The authoritative servers are queried directly over AXFR. Records owned by Kea are
identified by DHCID records and omitted together with their forward and reverse data.
"""
import argparse
import ipaddress
import re
import socket
import sys
from dataclasses import dataclass
from pathlib import Path
import dns.exception
import dns.name
import dns.query
import dns.resolver
import dns.rdatatype
OUT = Path("docs/reference/dns.md")
DEFAULT_PORT = 53
IGNORED_TYPES = {"DHCID", "SOA"}
POWERDNS_ALIAS = 65401
POWERDNS_LUA = 65402
@dataclass(frozen=True)
class Record:
zone: str
owner: str
type: str
value: str
def normalize_name(name: str) -> str:
return name.rstrip(".").lower()
def record_data(rdtype: int, rdata) -> tuple[str, str]:
if rdtype == POWERDNS_ALIAS:
target, _used = dns.name.from_wire(rdata.data, 0)
return "ALIAS", target.to_text()
if rdtype == POWERDNS_LUA:
logical_type = int.from_bytes(rdata.data[:2], byteorder="big")
return "LUA", dns.rdatatype.to_text(logical_type)
return dns.rdatatype.to_text(rdtype), rdata.to_text()
def server_addresses(server: str, port: int) -> list[str]:
addresses = []
try:
for result in socket.getaddrinfo(server, port, type=socket.SOCK_STREAM):
address = result[4][0]
if address not in addresses:
addresses.append(address)
except socket.gaierror as error:
raise RuntimeError(f"cannot resolve nameserver {server}: {error}") from error
return addresses
def transfer(server: str, port: int, zone: str) -> list[Record]:
addresses = server_addresses(server, port)
errors = []
for address in addresses:
try:
records = []
for message in dns.query.xfr(
address, zone, port=port, lifetime=60, relativize=False
):
for rrset in message.answer:
for rdata in rrset:
record_type, value = record_data(rrset.rdtype, rdata)
records.append(
Record(
zone=normalize_name(zone),
owner=normalize_name(rrset.name.to_text()),
type=record_type,
value=value,
)
)
if not any(record.type == "SOA" for record in records):
raise RuntimeError("transfer returned no SOA record")
return records
except (dns.exception.DNSException, OSError, RuntimeError) as error:
errors.append(f"{address}: {error}")
raise RuntimeError(f"AXFR of {zone} from {server} failed ({'; '.join(errors)})")
def system_nameservers(domain: str) -> list[str]:
try:
answer = dns.resolver.resolve(domain, "NS", lifetime=30)
except dns.exception.DNSException as error:
message = f"cannot discover authoritative servers for {domain}: {error}"
raise RuntimeError(message) from error
return [rdata.target.to_text() for rdata in answer]
def nameservers_via(server: str, port: int, domain: str) -> list[str]:
errors = []
for address in server_addresses(server, port):
resolver = dns.resolver.Resolver(configure=False)
resolver.nameservers = [address]
resolver.port = port
try:
answer = resolver.resolve(domain, "NS", lifetime=15, search=False)
return [rdata.target.to_text() for rdata in answer]
except dns.exception.DNSException as error:
errors.append(f"{address}: {error}")
raise RuntimeError(f"NS query for {domain} via {server} failed ({'; '.join(errors)})")
def discover_nameservers(domains: list[str], port: int) -> dict[str, list[str]]:
discovered = {}
unresolved = {}
candidates = []
for domain in domains:
try:
servers = system_nameservers(domain)
discovered[domain] = servers
for server in servers:
if server not in candidates:
candidates.append(server)
except RuntimeError as error:
unresolved[domain] = [str(error)]
for domain, errors in list(unresolved.items()):
for server in candidates:
try:
discovered[domain] = nameservers_via(server, port, domain)
del unresolved[domain]
break
except RuntimeError as error:
errors.append(str(error))
if unresolved:
details = "; ".join(
f"{domain}: {'; '.join(errors)}" for domain, errors in unresolved.items()
)
raise RuntimeError(details)
return discovered
def transfer_domain(
port: int, domain: str, servers: list[str], fallback: list[str] = ()
) -> list[Record]:
# A zone may be delegated publicly to servers that refuse AXFR (e.g. HE serving
# reverse DNS) while our own authoritative servers, discovered for other zones,
# will transfer it. Try the delegated servers first, then fall back to those.
ordered = list(servers)
for server in fallback:
if server not in ordered:
ordered.append(server)
errors = []
for server in ordered:
try:
return transfer(server, port, domain)
except RuntimeError as error:
errors.append(str(error))
raise RuntimeError(f"no authoritative server allowed AXFR for {domain} ({'; '.join(errors)})")
def dynamic_names(records: list[Record]) -> set[str]:
return {record.owner for record in records if record.type == "DHCID"}
def dynamic_addresses(records: list[Record], names: set[str]) -> set[str]:
return {
record.value.rstrip(".").lower()
for record in records
if record.owner in names and record.type in {"A", "AAAA"}
}
def reverse_address(owner: str) -> str | None:
if owner.endswith(".in-addr.arpa"):
labels = owner.removesuffix(".in-addr.arpa").split(".")
if len(labels) != 4:
return None
try:
return str(ipaddress.IPv4Address(".".join(reversed(labels))))
except ValueError:
return None
if owner.endswith(".ip6.arpa"):
labels = owner.removesuffix(".ip6.arpa").split(".")
if len(labels) != 32:
return None
try:
value = int("".join(reversed(labels)), 16)
return str(ipaddress.IPv6Address(value))
except ValueError:
return None
return None
def static_records(records: list[Record]) -> list[Record]:
names = dynamic_names(records)
addresses = dynamic_addresses(records, names)
static = []
for record in records:
if record.type in IGNORED_TYPES or record.owner in names:
continue
if record.type == "PTR":
target = normalize_name(record.value.split()[0])
address = reverse_address(record.owner)
if target in names or address in addresses:
continue
static.append(record)
return static
def relative_name(owner: str, zone: str) -> str:
if owner == zone:
return "@"
suffix = f".{zone}"
return owner[: -len(suffix)] if owner.endswith(suffix) else owner
def markdown_code(value: str) -> str:
escaped = value.replace("|", "\\|")
return f"`{escaped}`"
def display_record(record: Record) -> tuple[str, str]:
if record.type != "LUA":
return record.type, record.value
return f"{record.value} (LUA)", "generated at query time"
def render_forward(domain: str, records: list[Record]) -> list[str]:
domain = normalize_name(domain)
rows = []
for record in records:
if record.zone != domain or record.type == "PTR":
continue
record_type, value = display_record(record)
rows.append((relative_name(record.owner, record.zone), record_type, value))
rows.sort(key=lambda row: (row[0] != "@", row[0], row[1], row[2]))
lines = ["| Name | Type | Value |", "|---|---|---|"]
lines.extend(
f"| {markdown_code(name)} | {markdown_code(record_type)} | {markdown_code(value)} |"
for name, record_type, value in rows
)
return lines
def render_reverse(domain: str, records: list[Record]) -> list[str]:
domain = normalize_name(domain)
rows = []
for record in records:
if record.zone != domain or record.type != "PTR":
continue
rows.append((reverse_address(record.owner) or record.owner, record.value))
rows.sort(key=lambda row: ipaddress.ip_address(row[0]))
lines = ["| Address | Name |", "|---|---|"]
lines.extend(
f"| {markdown_code(address)} | {markdown_code(name)} |" for address, name in rows
)
return lines
def is_reverse(domain: str) -> bool:
domain = normalize_name(domain)
return domain.endswith(".in-addr.arpa") or domain.endswith(".ip6.arpa")
def rendered_zones(transferred: list[tuple[str, list[Record]]]) -> dict[str, list[str]]:
records = static_records([record for _domain, zone in transferred for record in zone])
rendered = {}
for domain, _raw_records in transferred:
if is_reverse(domain):
rendered[normalize_name(domain)] = render_reverse(domain, records)
else:
rendered[normalize_name(domain)] = render_forward(domain, records)
return rendered
def update_target(transferred: list[tuple[str, list[Record]]], target: Path) -> bool:
rendered = rendered_zones(transferred)
text = target.read_text()
lines = text.splitlines()
marker_re = re.compile(r"^<!--\s*dns:\s*(\S+)\s*-->$")
found = set()
output = []
i = 0
while i < len(lines):
match = marker_re.match(lines[i].strip())
if not match or normalize_name(match.group(1)) not in rendered:
output.append(lines[i])
i += 1
continue
domain = normalize_name(match.group(1))
end = i + 1
while end < len(lines) and lines[end].strip() != "<!-- dns-end -->":
end += 1
if end >= len(lines):
raise RuntimeError(f"missing <!-- dns-end --> for {domain}")
output.extend(
[
lines[i],
"<!-- dns-start -->",
*rendered[domain],
"<!-- dns-end -->",
]
)
found.add(domain)
i = end + 1
missing = rendered.keys() - found
if missing:
raise RuntimeError(f"missing DNS markers for: {', '.join(sorted(missing))}")
new = "\n".join(output) + "\n"
if new == text:
return False
target.write_text(new)
return True
def main() -> int:
parser = argparse.ArgumentParser(prog="update-docs-dns", description=__doc__)
parser.add_argument("domain", nargs="+", help="DNS zone to transfer")
parser.add_argument("--port", type=int, default=DEFAULT_PORT)
parser.add_argument("--output", type=Path, default=OUT)
args = parser.parse_args()
try:
nameservers = discover_nameservers(args.domain, args.port)
fallback = []
for servers in nameservers.values():
for server in servers:
if server not in fallback:
fallback.append(server)
transferred = [
(domain, transfer_domain(args.port, domain, nameservers[domain], fallback))
for domain in args.domain
]
except RuntimeError as error:
print(f"update-docs-dns: {error}", file=sys.stderr)
return 1
try:
changed = update_target(transferred, args.output)
except (OSError, RuntimeError) as error:
print(f"update-docs-dns: {error}", file=sys.stderr)
return 1
if changed:
print(f"updated {args.output}")
return 0
if __name__ == "__main__":
sys.exit(main())
ci/update-docs-options.py
-108
View File
@@ -1,108 +0,0 @@
#!/usr/bin/env python3
"""Render docs/reference/nixos-options.md from the `my.*` module options.
Builds the `optionsDoc` flake output (a `nixosOptionsDoc` JSON dump of the custom
`my.*` options) and renders it to a per-module table. The prose lives in the option
descriptions in `nixos/modules/`; this file is generated.
"""
import json
import re
import subprocess
import sys
from pathlib import Path
OPTIONS_DOC_ATTR = ".#nixfiles.config.nixos.optionsDoc"
OUT = Path("docs/reference/nixos-options.md")
HEADER = "| Option | Type | Default | Description |"
SEP = "|---|---|---|---|"
def collapse(text: str) -> str:
# Strip the flake's own /nix/store/<hash>-source/ prefix from rendered values so
# defaults like `./.keys/deploy.pub` are stable (the hash changes every time this
# generated file — part of the source tree — is rewritten).
text = re.sub(r"/nix/store/[a-z0-9]{32}-source/", "", text)
text = re.sub(
r'<link\s+xlink:href="([^"]+)">([\s\S]+?)</link>',
r"[\2](\1)",
text,
)
return re.sub(r"\s+", " ", text).strip().replace("|", "\\|")
def render_default(entry: dict) -> str:
if "default" not in entry:
return "—"
d = entry["default"]
# A dict default is a `defaultText` / `literalExpression` (pre-rendered Nix in `text`);
# anything else is a literal value we JSON-encode.
if isinstance(d, dict):
text = d.get("text")
if text is None:
text = json.dumps(d.get("value"))
else:
text = json.dumps(d)
text = collapse(text)
return f"`{text}`" if text else "—"
def build_options_json() -> dict:
# `optionsDoc` builds to a derivation containing share/doc/nixos/options.json; the last
# printed out-path is the derivation root.
out = subprocess.run(
["nix", "build", "--no-link", "--print-out-paths", OPTIONS_DOC_ATTR],
capture_output=True,
text=True,
check=True,
).stdout.strip().splitlines()[-1]
return json.loads((Path(out) / "share/doc/nixos/options.json").read_text())
def main() -> int:
opts = build_options_json()
by_module: dict[str, list[tuple[str, dict]]] = {}
for key, entry in opts.items():
decls = entry.get("declarations") or []
# Only document options declared in this repo's modules. Some `my.*` options
# (e.g. `my.buildAs`, which aliases `options.system.build`) pull in sub-options
# declared in nixpkgs; their declaration paths don't exist here — skip them.
if not decls or not Path(decls[0]).exists():
continue
by_module.setdefault(decls[0], []).append((key, entry))
lines = [
"# NixOS module options (`my.*`)",
"",
"> **Generated** from the module option descriptions by `nix run .#update-docs-options`;",
"> CI keeps it current. The source of truth is the `mkOpt'` / `mkBoolOpt'` declarations in",
"> `nixos/modules/` — edit those, not this file. For what each module is *for*, see the",
"> [shared-modules overview](../architecture.md#shared-modules).",
"",
]
for decl in sorted(by_module):
mod = Path(decl).stem
if decl != "(unknown)":
lines.append(f"## `{mod}` — [`{decl}`](../../{decl})")
else:
lines.append(f"## `{mod}`")
lines += ["", HEADER, SEP]
for key, entry in sorted(by_module[decl], key=lambda kv: kv[0]):
lines.append(
f"| `{key}` | {collapse(entry.get('type', ''))} | "
f"{render_default(entry)} | {collapse(entry.get('description', ''))} |"
)
lines.append("")
OUT.parent.mkdir(parents=True, exist_ok=True)
new = "\n".join(lines).rstrip() + "\n"
old = OUT.read_text() if OUT.exists() else ""
if new != old:
OUT.write_text(new)
print(f"updated {OUT}")
return 0
if __name__ == "__main__":
sys.exit(main())
devshell/default.nix
+1 -4
View File
@@ -12,9 +12,6 @@ in
NIX_USER_CONF_FILES = toString (pkgs.writeText "nix.conf"
''
experimental-features = nix-command flakes ca-derivations
lazy-trees = true
eval-cores = 0
accept-flake-config = true
connect-timeout = 5
fallback = true
${lib.my.c.nix.cache.conf}
@@ -25,7 +22,7 @@ in
packages = with pkgs; [
coreutils
determinate-nix
nixVersions.stable
rage
wireguard-tools
(pkgs.writeShellScriptBin "deploy" ''
devshell/install.nix
+1 -1
View File
@@ -29,7 +29,7 @@ let
coreutils
gnugrep
openssh
determinate-nix
nixVersions.stable
jq
];
text =
docs/README.md
+18 -25
View File
@@ -18,22 +18,13 @@ physical host (VM host)
Not every box fits this pattern, but **colony** and **home** are organised this way.
## General
## Cross-cutting topics
- [`architecture.md`](architecture.md) — the custom module system, `my.*` namespace, multiple
nixpkgs channels, shared module inventory.
- [`networking.md`](networking.md) — network assignments, domains, site topologies, router HA,
the AS211024 L2 mesh, BGP, WireGuard, Tailscale.
- [`deployment.md`](deployment.md) — deploy-rs, devshell commands, secrets workflow, CI.
- [`nixpkgs-upgrade.md`](nixpkgs-upgrade.md) — guided procedure for the periodic upgrade of the four
nixpkgs channels and home-manager (fork rebase, stable bumps, input review).
- [`install-box.md`](install-box.md) — guided procedure for installing a new box, from the booted
installer through partitioning, the box config, `do-install` and documentation.
- [`openwrt-flash.md`](openwrt-flash.md) — guided procedure for flashing a flake-built image onto an
OpenWrt box, from the build through pre-flight, `sysupgrade` and verification.
- [`reference/dns.md`](reference/dns.md) — generated forward and reverse DNS record reference.
- [`reference/nixos-options.md`](reference/nixos-options.md) — generated per-option reference for
the custom `my.*` NixOS modules.
## Site: colony (Amsterdam)
@@ -47,7 +38,7 @@ colony (physical VM host, ams1)
│ ├── colony-psql (shared PostgreSQL)
│ ├── chatterbox (Matrix Synapse + bridges)
│ ├── jackflix (media stack)
│ ├── object (MinIO, Harmonia Nix cache, HedgeDoc, wastebin)
│ ├── object (MinIO, Harmonia Nix cache, Sharry, HedgeDoc, wastebin)
│ ├── toot (Bluesky PDS; Mastodon disabled)
│ ├── waffletail (Tailscale subnet router / exit node)
│ ├── qclk (WireGuard management appliance)
@@ -57,25 +48,20 @@ colony (physical VM host, ams1)
├── git ────── Gitea + Gitea Actions runner
├── mail ───── Debian VM running mailcow (not NixOS)
└── darts ──── third-party/customer VM (opaque, not NixOS)
portcullis (bare-metal edge box for Nikhef — staged, not yet in service)
└── fergal OpenWrt SFP+ switch, staged and moving with it
```
## Site: home
Redundant routers, VM host, storage, IoT containers and the workstation — see
[`sites/home/README.md`](sites/home/README.md). The hand-configured switch fabric (jim/dave/brian)
and the Digiweb WAN path are documented in [`sites/home/switches.md`](sites/home/switches.md); the
5G modem being evaluated as a replacement for `stream`'s WAN is in
[`sites/home/wwan.md`](sites/home/wwan.md).
and the Digiweb WAN path are documented in [`sites/home/switches.md`](sites/home/switches.md).
```
h.nul.ie
├── palace (physical VM host — AMD, 100G, SR-IOV)
│ ├── river ── primary router VM (PPPoE / Digiweb WAN)
│ ├── cellar ─ NVMe-oF / SPDK storage target VM
│ └── sfh ──── container host VM ("shill from home")
│ └── sfh ──── container host VM ("services for home")
│ ├── hass ── Home Assistant + Frigate + MQTT (container)
│ └── unifi ─ UniFi controller (container)
├── stream (physical secondary router — Virgin Media WAN)
@@ -84,11 +70,19 @@ h.nul.ie
## Remote boxes
The edge VPSes and remote `kelder` site are indexed in [`remote/README.md`](remote/README.md).
Edge VPSes and the remote `kelder` site — see [`remote/README.md`](remote/README.md).
- [`remote/britway.md`](remote/britway.md) — London VPS: headscale control plane, Tailscale exit
node, BGP edge.
- [`remote/britnet.md`](remote/britnet.md) — Birmingham VPS: Tailscale exit node, WireGuard hub.
- [`remote/kelder.md`](remote/kelder.md) — secondary home server: container host, Nextcloud,
Samba, WireGuard to `estuary`.
- [`remote/kelder-acquisition.md`](remote/kelder-acquisition.md) — media acquisition container.
- [`remote/kelder-spoder.md`](remote/kelder-spoder.md) — Nextcloud + nginx container.
## Mobile boxes
The laptop is indexed in [`mobile/README.md`](mobile/README.md).
- [`mobile/tower.md`](mobile/tower.md) — laptop workstation.
## Misc
@@ -96,8 +90,7 @@ The laptop is indexed in [`mobile/README.md`](mobile/README.md).
## A note on the assignment tables
The consolidated [`Box assignments`](networking.md#box-assignments) tables in
[`networking.md`](networking.md) (one per site, between `<!-- assignments: <site> -->` markers)
are **generated from the flake** (`nixos.allAssignments`) by `nix run .#update-docs-assignments` —
CI refreshes them on push. Individual box pages link to that section rather than carrying their
own table. Only the Notes column is hand-written; don't hand-edit the other cells.
The "Network assignments" tables in box pages (between `<!-- assignments: … -->` markers) are
**generated from the flake** (`nixos.allAssignments`) by `nix run .#update-docs-assignments` —
CI refreshes them on push. Hand-written prose and the table's Notes column are preserved; don't
hand-edit the other cells.
docs/architecture.md
+33 -57
View File
@@ -73,11 +73,11 @@ Each entry of `nixos.systems` is a submodule defined by `systemOpts` in
`mkSystem` does the real work:
1. It selects the channel's nixpkgs flake (`pkgsFlakes.${config'.nixpkgs}`).
- It imports `nixos/lib/eval-config.nix` **by hand** because the flake force-sets `lib`;
otherwise `eval-config.nix` would import its own unextended copy.
- The supplied library is the channel's `pkgs.lib` extended with `lib.my` and a
`versionOverlay` that stamps `system.nixos` with the flake revision and channel.
1. It selects the channel's nixpkgs flake (`pkgsFlakes.${config'.nixpkgs}`) and imports
`nixos/lib/eval-config.nix` **by hand**, because the flake force-sets `lib` and
`eval-config.nix` would otherwise import its own unextended one. The lib it passes is the
channel's `pkgs.lib` extended with `lib.my` plus a `versionOverlay` that stamps
`system.nixos` with the flake revision and channel.
2. `specialArgs` gives every module access to `self`, `inputs`, `pkgsFlakes`, the box's own
`pkgsFlake`, `allAssignments` (every box's assignments), and `systems` (all of
`nixos.systems`). Passing these via `specialArgs` (rather than module `imports`) avoids
@@ -96,12 +96,12 @@ Each entry of `nixos.systems` is a submodule defined by `systemOpts` in
- `nixpkgs.system` is set and the overlays computed at flake level (lib overlay + custom
packages) are passed through, so `pkgs` is imported modularly with config and overlays
applied.
- `home-manager.useGlobalPkgs` defaults to true when the system and home-manager channels
match; a warning is emitted when they deliberately differ.
- `home-manager.sharedModules` includes every module in `home-manager.modules` plus an inline
module. The inline module passes `pkgsPath` / `pkgs'`, disables the release check, and pins
`home.stateVersion` through `homeStateVersion` (`22.11` for stable-flavoured channels,
`23.05` otherwise).
- home-manager integration: `useGlobalPkgs` defaults to true when the system and
home-manager channels match (a warning is emitted when they deliberately differ);
`sharedModules` is every module in `home-manager.modules` plus an inline module that
passes `pkgsPath` / `pkgs'` into home-manager, disables the release check, and pins
`home.stateVersion` via `homeStateVersion` (currently `22.11` for stable-flavoured
channels, `23.05` otherwise).
5. Finally `applyAssertions` throws with all failed assertion messages (and shows warnings)
when the merged configuration is forced.
@@ -136,8 +136,8 @@ etc. Inside a module, `pkgs` is the selected channel and `pkgs'` is the attrset
flake-level `lib` (`pkgsFlakes.unstable.lib` so extended) is for platform-independent flake
use only; each box gets its own channel's lib extended the same way. Contents:
- **Option helpers** — `mkOpt'`, `mkBoolOpt'`, `nullOrOpt'`, `mkDefault'` (slightly stronger than
`mkDefault`), `mkVMOverride'`, `inlineModule'` / `inlineModule`
- **Option helpers** — `mkOpt'`, `mkBoolOpt'`, `nullOrOpt'`, `mkDefault'` (`mkOverride 900`,
slightly stronger than `mkDefault`), `mkVMOverride'`, `inlineModule'` / `inlineModule`
(attach `_file` provenance), `commonOpts` (the shared `system`/`nixpkgs`/`home-manager`
options and the `moduleType` used for exported modules), `applyAssertions`, `duplicates`.
- **`lib.my.net`** — CIDR/IP math (`net.cidr.host`, `net.cidr.subnet`, `net.types.ipv4`, …)
@@ -152,14 +152,11 @@ use only; each box gets its own channel's lib extended the same way. Contents:
(`ifaceA`, `lookupIP`) used by the home routers.
- **networkd helpers** — `networkdAssignment` (assignment → `systemd.network` network, see
[networking.md](networking.md)), `mkVLAN`, `dockerNetAssignment`.
### Misc
The remaining helpers are `mkDefaultSystemsPkgs`, `flakePackageOverlay` (wrap another flake's
package as an overlay), `isIPv6` / `parseIPPort` / `netBroadcast`, `nft` chain-name helpers,
`systemdAwaitPostgres`, `vm.*` LVM disk descriptors for the `vms` module, the typed `deploy-rs`
option definitions (`lib.my.deploy-rs`), `netbootKeaClientClasses` (iPXE/EFI DHCP client classes),
and `homeStateVersion`.
- **Misc** — `mkDefaultSystemsPkgs`, `flakePackageOverlay` (wrap another flake's package as an
overlay), `isIPv6` / `parseIPPort` / `netBroadcast`, `nft` chain-name helpers,
`systemdAwaitPostgres`, `vm.*` LVM disk descriptors for the `vms` module, the typed
`deploy-rs` option definitions (`lib.my.deploy-rs`), `netbootKeaClientClasses` (iPXE/EFI
DHCP client classes), and `homeStateVersion`.
Custom NixOS / home-manager modules declare their options under the `my.*` namespace (the root
`options.my` is declared in `nixos/modules/common.nix` / `home-manager/modules/common.nix`) —
@@ -169,13 +166,11 @@ e.g. `my.secrets`, `my.build`, `my.tmproot`, `my.firewall`, `my.server`, `my.dep
## Shared modules
Registered in [`nixos/modules/_list.nix`](../nixos/modules/_list.nix) and applied to **every**
system (box files opt in per-feature via `my.*` options). The table below is the overview — what
each module is *for*; for the exhaustive, CI-generated per-option reference (types, defaults,
descriptions) see [`reference/nixos-options.md`](reference/nixos-options.md).
system (box files opt in per-feature via `my.*` options):
| Module | Provides |
|---|---|
| `common` | Baseline for all boxes: imports the impermanence, ragenix (age), copyparty and harmonia NixOS modules; pins `system.stateVersion`; `doas` instead of `sudo`; immutable users; nix settings (flakes, `ca-derivations`, the `nix-cache.nul.ie` substituter); declares the `my` option root. |
| `common` | Baseline for all boxes: imports the impermanence, ragenix (age), sharry, copyparty and harmonia NixOS modules; pins `system.stateVersion`; `doas` instead of `sudo`; immutable users; nix settings (flakes, `ca-derivations`, the `nix-cache.nul.ie` substituter); declares the `my` option root. |
| `user` | `my.user` — the primary user: `users.users` + matching `home-manager.users` entry, wheel/doas, SSH authorized key from `.keys/me.pub`, shell taken from the home config, home persistence under tmproot. |
| `build` | `my.build` — alternate build targets via `extendModules`: `my.buildAs.devVM` (QEMU dev VM), `iso`, `container`, `kexecTree`, `netbootTree`/`netbootArchive`; `my.build.isDevVM` marker; `allHardware` profile toggle. |
| `dynamic-motd` | `my.dynamic-motd` — runs a script via `pam_exec` to generate the MOTD on login/ssh. |
@@ -229,38 +224,19 @@ home (including the per-user homes attached to systems via `my.user.homeConfig`)
## Home-manager
Home-manager follows the same option-definition and evaluation split as NixOS.
### `homeOpts`
[`home-manager/default.nix`](../home-manager/default.nix) defines each entry with these options:
| Option | Meaning |
|---|---|
| `system` | Target platform. |
| `nixpkgs` | nixpkgs channel used for the home packages. |
| `home-manager` | home-manager channel. |
| `homeDirectory` / `username` | User identity and home path. |
| `configuration` | The home-manager module definitions evaluated by `mkHome`. |
### `mkHome`
1. Calls the selected channel's `lib.homeManagerConfiguration`.
2. Supplies that channel's `pkgs'` with `config` emptied, allowing home-manager to apply package
configuration and overlays through its module system.
3. Passes `inputs`, `pkgsFlakes` and the selected `pkgsFlake` through `extraSpecialArgs`.
4. Loads every module in `home-manager.modules` plus an inline module that sets
`home.homeDirectory` / `home.username` and exposes every channel through the `pkgs'` module
argument.
5. Pins `home.stateVersion` through `homeStateVersion`.
### Standalone homes
Home-manager has the same two-level structure: `homeOpts` (in
[`home-manager/default.nix`](../home-manager/default.nix)) defines `system`, `nixpkgs`,
`home-manager`, `homeDirectory`, `username`, and `configuration`; `mkHome` calls the selected
channel's `lib.homeManagerConfiguration`, passing the channel's `pkgs'` (with `config` emptied
so home-manager applies overlays/config modularly itself), `extraSpecialArgs` (`inputs`,
`pkgsFlakes`, `pkgsFlake`), every module in `home-manager.modules`, an inline module that sets
`home.homeDirectory` / `home.username` and provides `pkgs'` (all channels) as a module arg,
and the pinned `home.stateVersion`.
Standalone homes live in [`home-manager/configs/`](../home-manager/configs) and are named
`<user>@<host>` (deploy-rs mangles the `@` to `-at-`). `macsimum` is a home-manager-only
config — an `x86_64-darwin` macOS box with no NixOS side.
No standalone homes are currently wired into the flake: `home-manager/configs/macsimum.nix` is
commented out of `configs`, while `home-manager/configs/castle.nix` is not listed. Consequently,
`nixfiles.config.home-manager.homes` evaluates to `{}`. Most user configuration instead accompanies
NixOS boxes through `my.user.homeConfig`.
config — an `x86_64-darwin` macOS box with no NixOS side at all. Note that **no homes are
currently wired into the flake**: the `home-manager/configs/macsimum.nix` entry in `configs` is
commented out, and `home-manager/configs/castle.nix` exists on disk but is not listed either,
so `nixfiles.config.home-manager.homes` currently evaluates to `{}`. Most day-to-day user
configuration instead rides along with systems through `my.user.homeConfig`.
docs/deployment.md
+63 -159
View File
@@ -32,16 +32,19 @@ The shared [`nixos/modules/deploy-rs.nix`](../nixos/modules/deploy-rs.nix) modul
`sudo = "doas -u"` (or `sudo -u`), `sshOpts` = the box's first OpenSSH port. The module
creates the `deploy` system user (bash shell, wheel, keys from `my.deploy.authorizedKeys`,
which defaults to `.keys/deploy.pub`).
Generated profiles deploy in this order:
1. **`system`** activates `config.system.build.toplevel` with `switch-to-configuration switch`,
applies the `/tmp` cwd and systemd-boot `loader.conf` workarounds, then prunes old generations.
`keepGenerations` controls how much history is retained; zero disables pruning.
2. **`container-<name>`** activates each `my.containers.instances` entry into
`/nix/var/nix/profiles/per-container/<name>/system`. With `hotReload` (the default), it reloads
`systemd-nspawn@<name>` and restarts only a stopped container or one still running the dummy
init; otherwise it restarts the unit. The same generation cleanup applies.
- Generated profiles:
- **`system`** — activates `config.system.build.toplevel` with
`switch-to-configuration switch`, with workarounds for the `/tmp` cwd issue
(NixOS/nixpkgs#73404) and the systemd-boot `loader.conf` default-entry issue
(deploy-rs#31), then prunes old generations (`nix-env --delete-generations +10`,
tunable via `keepGenerations`, 0 disables).
- **`container-<name>`** — one per `my.containers.instances` entry, activating the
container's `my.buildAs.container` toplevel into
`/nix/var/nix/profiles/per-container/<name>/system`. With `hotReload` (default) the
profile *reloads* `systemd-nspawn@<name>` (restarting only if the container is down or
still running the placeholder "dummy" init); otherwise it restarts it. Generation cleanup
applies here too.
- Profiles deploy in order `system`, then the containers.
### Usage
@@ -52,16 +55,13 @@ Generated profiles deploy in this order:
(deploy-rs's `--boot` maps to the custom activation's `boot` phase, which also does the
generation cleanup). Use it when a live `switch` would cut the box off mid-change (e.g. a
router WAN rework), then reboot to cut over.
- `ssh-machine <name> [cmd]` — resolves `user@host` and the merged global+node `sshOpts` from
`.#deploy` (same `my.deploy.enable` gate as `deploy`, same `@` → `-at-` mangling), then
execs `ssh`. Boxes default to the `fish` login shell; pipe multi-statement remote scripts
through `bash` (`ssh-machine <name> bash -s < script.sh`). If outbound SSH hangs at the
publickey step (flaky agent), use `SSH_AUTH_SOCK= ssh-machine …`.
- `nix flake check` includes deploy-rs's own `deployChecks` for the whole `deploy` attrset.
#### `ssh-machine`
`ssh-machine <name> [cmd]` resolves `user@host` and the merged global/node `sshOpts` from
`.#deploy`, using the same `my.deploy.enable` gate and `@` → `-at-` mangling as `deploy`, then runs
`ssh`. Boxes default to `fish`, so pipe multi-statement remote scripts through `bash`, for example
`ssh-machine <name> bash -s < script.sh`. If a flaky agent stalls public-key authentication, use
`SSH_AUTH_SOCK= ssh-machine …`.
## Devshell commands
The repo ships a `numtide/devshell` ([`devshell/`](../devshell), entered via `direnv`). Run a
@@ -110,56 +110,27 @@ VM's unix sockets from `/run/vms/<vm>/` on `<host>` over SSH):
| `vm-monitor <host> <vm>` | QEMU monitor socket in `minicom`. |
| `vm-viewer <host> <vm>` | SPICE display in `virt-viewer` (not on Darwin). |
## Nix implementation
Every context uses **Determinate Nix** as its `nix.package`, for its performance features
(parallel evaluation and lazy trees) — not `determinate-nixd`; the daemon and `nix.conf` model
are unchanged, and the Determinate NixOS module is deliberately not imported.
- **Input and package.** The [`determinate-nix`](../flake.nix) input is the `nix-src` flake
(`flakehub.com/f/DeterminateSystems/nix-src`), with `nixpkgs.follows = "nixpkgs-unstable"`. We
build it ourselves against our pinned nixpkgs — FlakeHub's own cache needs authentication, so
there is nothing to gain from leaving it unpinned — and it then flows through the Harmonia cache
like everything else. `determinateOverlay` exposes it under the stable attr `determinate-nix`,
added to both the devshell `pkgs'` and the config `configPkgs'` overlay lists, so systems, homes
and the devshell all resolve the same package (`pkgs'.mine.determinate-nix`).
- **Settings.** `lib.my.c.nix.determinateSettings` (`lazy-trees`, `eval-cores = 0`) is merged into
`nix.settings` for systems and homes and into the devshell's `nix.conf`. These keys are only
understood by the Determinate binary.
- **Consumers follow automatically.** Everything that shells out to Nix references
`config.nix.package` (deploy-rs, containers, `build`, netboot, Harmonia), so they inherit
Determinate without further change.
- **`accept-flake-config`.** Set true only in the devshell `nix.conf`, `.envrc` (as
`--accept-flake-config`, for direnv) and CI — the contexts that build this flake — so its
`nixConfig` (the Harmonia cache) is trusted without an interactive prompt. It is deliberately not
set system-wide: boxes already trust that cache via `nix.settings`, so a global setting would only
blanket-trust every flake's `nixConfig` for no gain.
## Secrets
Secrets are age-encrypted files in [`secrets/`](../secrets), managed with **ragenix** (a fork
with a rekey flag, from the flake inputs).
### Per-box declarations
Each box declares `my.secrets.key`, the host public key its secrets encrypt to, and
`my.secrets.files.<name>`, whose values merge settings such as `owner` and `mode` into the agenix
secret. At runtime, identity paths come from the box's OpenSSH host keys. On tmproot boxes they are
read from the persistence directory because agenix runs before the persisted keys would otherwise
be available.
### Recipient rules
[`secrets.nix`](../secrets.nix) is generated from every system's `my.secrets.files` and `key`.
Each recipient list also includes `.keys/dev.pub`, so the development key can open every secret.
Run `ragenix -r` after adding a box or secret to re-key the files.
### Local keys and editing
The `ragenix` devshell command wraps `ragenix --identity .keys/dev.key`. The `.keys/` directory
contains that development key, the deploy key authorized for every box's `deploy` user, and other
keys referenced by `lib.my.c.sshKeyFiles`; it is required for editing secrets, deploying and running
development VMs.
- Each box declares `my.secrets.key` (the host public key its secrets encrypt to) and
`my.secrets.files.<name>` (files to decrypt; attribute values merge into the agenix secret,
e.g. `owner`/`mode`). At runtime the `secrets` module decrypts via
`age.secrets."<name>".path`, with identity paths derived from the box's OpenSSH host keys —
read from the tmproot persistence dir when there is one, since agenix runs before persisted
keys would otherwise be available.
- [`secrets.nix`](../secrets.nix) (the ragenix rules file at the repo root) is **generated**:
it evaluates the flake, collects every system's `my.secrets.files` + `key`, and emits each
secret with its recipient list — always including `.keys/dev.pub` so the dev key can open
everything. Re-running `ragenix -r` re-keys after adding a box or secret.
- The `ragenix` devshell command wraps `ragenix --identity .keys/dev.key`; use it to
edit/rekey secrets.
- The `.keys/` directory holds the dev key (`dev.key`/`dev.pub`), the deploy key
(`deploy.key`/`deploy.pub`, authorized on every box's `deploy` user), and assorted other
keys referenced by `lib.my.c.sshKeyFiles`. It is required for editing secrets, deploying,
and running dev VMs.
## Dev VMs
@@ -171,104 +142,37 @@ default `/tmp/xchg/dev.key`), so dev VMs can decrypt the boxes' secrets without
keys. Dev VMs also get DHCP on `eth0`, an SSH port forward (host 2222 → guest 22), and are
automatically excluded from deploy targets.
## OpenWrt images
The OpenWrt boxes are not NixOS and are not deployed by this flake, but their firmware is built
here. [`openwrt/default.nix`](../openwrt/default.nix) declares one image per box and packages it
through [`astro/nix-openwrt-imagebuilder`](https://github.com/astro/nix-openwrt-imagebuilder),
which drives OpenWrt's official ImageBuilder — prebuilt target packages assembled into a sysupgrade
image, with no cross-toolchain involved.
| Output | Box | Release |
|---|---|---|
| `openwrt-fergal` | [fergal](sites/colony/fergal.md) | `snapshot` |
| `openwrt-fergal-release` | The same, on the release branch | pinned in `openwrt/default.nix` |
Both are in `ci`, so images are built and pushed to the Harmonia cache like everything else. Build
one with `nix build .#openwrt-fergal`; the result holds the `-squashfs-sysupgrade.bin` to flash,
plus a package manifest and an SBOM. Getting it onto the box is a guided procedure of its own —
see [`openwrt-flash.md`](openwrt-flash.md).
Packages are baked into the image rather than installed on the box. OpenWrt's package server keeps
only the current build of each feed, so a box that installs packages at runtime stops being able to
do so as soon as the feed moves on from the firmware it is running. Adding a package means editing
the image's `packages` list and reflashing.
### The feed pin
OpenWrt's download server is never at rest: snapshot is rebuilt daily, and
`releases/<version>/packages/` is a symlink to the rolling `packages-<major>` feed shared by every
point release. Building straight against it fails on hash mismatches and, worse, resolves the
package list by import-from-derivation — which would drag *evaluation* of this flake onto the
network and let an OpenWrt feed rebuild break `check-system` for unrelated boxes.
The `openwrt-feeds` input exists to stop that. It holds expanded per-package metadata and vendors
the repository indexes themselves, so every `.apk` is a plain pinned `fetchurl`, image builds read
the indexes from the flake rather than OpenWrt's mutable URLs, and no import-from-derivation is
involved. Its generated files run to hundreds of thousands of lines and are rewritten wholesale on
each refresh, which is why they live in their own repository rather than here. Regenerate and push
that repository with `nix run .#update`, then refresh this flake's pin with
`nix flake update openwrt-feeds`; adding a release or target means adding it to that repository's
`pins` first.
## CI
GitHub/Gitea Actions workflows live in [`.gitea/workflows/`](../.gitea/workflows).
GitHub/Gitea Actions workflows live in [`.gitea/workflows/`](../.gitea/workflows):
### `ci.yaml`
- **`ci.yaml`** (push to `master`): `nix flake check --no-build`, then for every attribute of
`.#ci.x86_64-linux` (each system as `system-<name>`, each home as `home-<name>` with `@` →
`-at-`, each package as `package-<name>`, plus the dev `shell`) it builds and pushes the
result to the Harmonia binary cache with [`ci/push-to-cache.sh`](../ci/push-to-cache.sh).
Finally it builds `.#ciDrv.x86_64-linux` (a `linkFarm` of all CI attrs) and pushes it with
`UPDATE_PROFILE=1`, which updates the `nixfiles` profile on the cache box and garbage
collects paths older than 60 days. The cache is `ssh-ng://harmonia@object-ctr.ams1.int.nul.ie`
(remote store `/var/lib/harmonia`), keyed by the `HARMONIA_SSH_KEY` secret with a pinned
`ci/known_hosts`; clients consume it as `https://nix-cache.nul.ie` (see `lib.my.c.nix.cache`).
- **`installer.yaml`** (push of the `installer` tag; `update-installer` refreshes it): builds
the installer's `my.buildAs.iso` and `my.buildAs.netbootArchive` and attaches both to a
release.
- **`update-docs.yaml`** (push to the docs branch, skipping its own commits): runs
`nix run .#update-docs-assignments` and commits any changes as
`docs: update assignment tables`.
On pushes to `master`, this installs Determinate Nix on the runner (via
`DeterminateSystems/determinate-nix-action`, configured with the same performance settings and
Harmonia substituter as the boxes), runs `nix flake check --no-build`, then builds every attribute
of `.#ci.x86_64-linux`: systems as `system-<name>`, homes as `home-<name>` (with `@` changed to
`-at-`), packages as `package-<name>`, and the development `shell`. Each result is pushed to the
Harmonia cache with [`ci/push-to-cache.sh`](../ci/push-to-cache.sh). Before the first push, the
workflow deletes cache-profile generations older than its retention period, then collects
unreachable paths so a full cache cannot prevent collection from being reached.
### The docs assignment-table updater
It then builds `.#ciDrv.x86_64-linux`, a `linkFarm` of all CI attributes, and pushes it with
`UPDATE_PROFILE=1`. That updates the `nixfiles` profile on the cache box. The SSH store uses
`/var/lib/harmonia`, `HARMONIA_SSH_KEY`, and pinned `ci/known_hosts`; clients use
`https://nix-cache.nul.ie` through `lib.my.c.nix.cache`.
### `installer.yaml`
Pushing the `installer` tag (refreshed by `update-installer`) builds `my.buildAs.iso` and
`my.buildAs.netbootArchive`, then attaches both to a release.
### `update-docs.yaml`
On pushes to `master`, excluding its own commits, this runs the assignment, option and DNS
reference generators and commits changed outputs as `docs: Update generated references`.
### The docs generators
The generators are registered in [`pkgs/default.nix`](../pkgs/default.nix) as wrappers around
Python scripts under [`ci/`](../ci). They leave the worktree unchanged when their output is current;
the workflow stages `docs/` and uses `git diff --cached --quiet` to decide whether to commit.
`update-docs-assignments` ([`ci/update-docs-assignments.py`](../ci/update-docs-assignments.py))
evaluates `.#nixfiles.config.nixos.allAssignments` to JSON and rewrites the consolidated
[`Box assignments`](networking.md#box-assignments) tables in `docs/networking.md` — one table
per site (`colony` / `home` / `remote`), each between a `<!-- assignments: <site> -->` marker
and a closing `<!-- assignments-end -->` line. Boxes are grouped by site from their assignment
domain, and each `Box` cell links to that box's page when one exists. Hand-written text in the
**Notes** column is preserved across runs (keyed by box + assignment), so notes survive
regeneration. Individual box pages don't carry tables; they link to the consolidated section.
`update-docs-options` ([`ci/update-docs-options.py`](../ci/update-docs-options.py)) builds the
`nixos.optionsDoc` output (declared in [`nixos/default.nix`](../nixos/default.nix)) — a
`nixosOptionsDoc` JSON dump of the custom `my.*` options (declared with `mkOpt'` / `mkBoolOpt'` in
`nixos/modules/`). It's evaluated against a **minimal synthetic system**, since the shared modules
apply to every box, so defaults don't pick up a real host's values. The renderer writes
[`docs/reference/nixos-options.md`](reference/nixos-options.md), one table per module file. The
whole file is generated; edit the option descriptions in the modules, not the reference. The
internal `asX` build-target options are marked `internal = true` so they're excluded.
`update-docs-dns` ([`ci/update-docs-dns.py`](../ci/update-docs-dns.py)) accepts forward and reverse
zone names, discovers their authoritative nameservers through NS queries, and transfers each zone
over AXFR. If a private reverse zone is not visible through the configured recursive resolver, it
asks the authoritative servers discovered for the other requested zones. It updates only the
matching `<!-- dns: <zone> -->` blocks in the [`DNS records`](reference/dns.md) reference; the page's
headings and prose remain handwritten. Kea-managed owners are identified by `DHCID` records and
removed together with their A, AAAA and PTR records; SOA records and TTLs are also omitted.
`update-docs-assignments` is registered in [`pkgs/default.nix`](../pkgs/default.nix) (a
`writeShellScriptBin` wrapping [`ci/update-docs-assignments.py`](../ci/update-docs-assignments.py)).
It evaluates `.#nixfiles.config.nixos.allAssignments` to JSON, walks `docs/**/*.md`, and for
each file looks for a marker line `<!-- assignments: <box> -->` where `<box>` is the **file's
own stem** (e.g. `river.md` → `river`) and names a real box. The generated table (columns
`Name | Assignment | IPv4 | IPv6 | Domain | Notes`) is (re)written between the marker and a
closing `<!-- assignments-end -->` line, with an `<!-- assignments-start -->` line inserted
after the marker on the first run. Hand-written text in the **Notes** column is preserved
across runs (keyed by the Assignment cell), so notes survive regeneration. Pages that want a
table only need the marker + end marker; pages without markers (including the three top-level
docs) are left alone. The script exits non-zero when it changed something, which is how the
workflow knows whether to commit.
docs/install-box.md
-213
View File
@@ -1,213 +0,0 @@
# Installing a box
Procedure for bringing a new NixOS box into this flake, from bare hardware booted into the custom
installer through to a deployable system. Written to be followed by a person or any coding agent; a
Claude Code entry point exists at `.claude/skills/install-box/` but the steps below are the
canonical source.
The install is **guided, not automated**: the mechanical steps (probing hardware, partitioning,
writing the config, evaluating it) can be done straight through, but stop at the judgment points
(marked ⏸) — what the box actually is, wiping disks, and running `do-install` itself. Keep a running
summary and present it before any destructive step.
For the installer image itself — what it contains, how it is built and released — see
[`misc/installer.md`](misc/installer.md).
## Setup facts
- **Installer access:** the box boots the custom installer (ISO, kexec or netboot) and is reached
over SSH as `root` with `.keys/deploy.key`. The devshell sets
`INSTALLER_SSH_OPTS = "-i .keys/deploy.key"`; set `INSTALLER` to the address, and
`INSTALLER_SSH_PORT` if it is not 22.
- **Devshell commands** (from [`devshell/install.nix`](../devshell/install.nix)):
`installer-shell [cmd]` and `do-install [--no-bootloader] [--no-substitute] <system>`.
- **`INSTALL_ROOT`** is `/mnt` in the installer's environment — everything is mounted under it and
`do-install` reads it from the installer rather than assuming.
- **`show-hw-config`** is a shell *alias* in the installer (wrapping
`nixos-generate-config --show-hardware-config --root $INSTALL_ROOT`), so it needs an interactive
shell: `installer-shell bash -lic show-hw-config`. A plain `installer-shell show-hw-config` will
not find it.
- **Validation:** `check-system <host>` evaluates a system without building it — use it while
iterating. Only `build-system` when you need the artifact.
- Nix reads the flake through git, so **`git add` new files before evaluating** — an untracked
box directory fails with "Path … is not tracked by Git", not a Nix error.
## Phase 1 — Establish what the box is
⏸ Settle these before writing anything; they decide where every file goes and they are not
recoverable from the hardware:
1. **Name and site** — the box name doubles as the `nixos.systems.<name>` attribute, the deploy node
name and the docs page name. The site decides the directory (`nixos/boxes/<site>/`), the
constants block in [`lib/constants.nix`](../lib/constants.nix) it draws prefixes from, and the
docs directory (`docs/sites/<site>/`, `docs/remote/`, `docs/mobile/`).
2. **Role** — what it does, which decides its modules, networking and firewall config.
3. **nixpkgs channel** — `unstable` / `stable` / `mine` / `mine-stable`; match the site's other
boxes unless there is a reason not to.
4. **Networking** — whether it gets `assignments` now, or bootstraps on DHCP because it is being
staged somewhere other than its final home. A box with no assignment still needs a reachable
`my.deploy.node.hostname`, since the default (`config.networking.fqdn`) will not resolve.
## Phase 2 — Reach the installer and inventory the hardware
With the box booted into the installer and `INSTALLER` set:
1. Confirm you are talking to the right thing — `installer-shell hostname` reports `installer`, and
`/etc/os-release` carries `VARIANT_ID=installer`.
2. Collect the inventory you will need for both the config and the docs page: `lscpu`, `free -h`,
`lsblk -o NAME,SIZE,TYPE,FSTYPE,MODEL,SERIAL`, `ip -br link`, `ip -br addr`,
`lspci -nn | grep -Ei 'ethernet|network|nvme|sata|raid'`, and whether `/sys/firmware/efi` exists.
3. Record every NIC's **permanent MAC** against its PCI address — interface naming in Phase 5 pins
names to MACs, and the PCI order tells you which physical port is which.
4. Run `installer-shell bash -lic show-hw-config` now for the kernel-module lists. Filesystems are
not mounted yet, so run it again in Phase 4 for those.
## Phase 3 — Partition, format and mount
⏸ Destructive. Check the target disks are the ones you think they are and that nothing on them is
wanted, then show the exact command sequence and get confirmation before running it.
The house layout is a tmpfs root (`my.tmproot`) with three mounts: an ESP at `/boot`, `/nix`, and
`/persist` (`neededForBoot = true`). Use **`sgdisk`** for partitioning and put `/nix` and `/persist`
on **LVM** so they can be resized later:
```sh
sgdisk -Z /dev/<disk>
sgdisk \
-n 1:0:+2G -t 1:ef00 -c 1:esp \
-n 2:0:0 -t 2:8e00 -c 2:lvm \
/dev/<disk>
partprobe /dev/<disk>
pvcreate /dev/<disk>p2
vgcreate main /dev/<disk>p2
lvcreate -L 48G -n <host>-nix main
lvcreate -l 100%FREE -n <host>-persist main
mkfs.vfat -n ESP /dev/<disk>p1
mkfs.ext4 -L nix /dev/main/<host>-nix
mkfs.ext4 -L persist /dev/main/<host>-persist
```
Conventions worth keeping: volume group `main`, logical volumes `<host>-nix` / `<host>-persist`,
and ext4 filesystem labels `nix` and `persist`. Size the ESP and `/nix` to the box — 2 GiB and
48 GiB suit a small single-disk box.
Then mount everything under `$INSTALL_ROOT`, with a tmpfs standing in for the eventual tmpfs root:
```sh
mount -t tmpfs -o size=2G tmpfs "$INSTALL_ROOT"
mkdir -p "$INSTALL_ROOT"/{nix,persist,boot}
mount /dev/main/<host>-nix "$INSTALL_ROOT/nix"
mount /dev/main/<host>-persist "$INSTALL_ROOT/persist"
mount /dev/<disk>p1 "$INSTALL_ROOT/boot"
```
### Seed the SSH host key
The installer generates fresh host keys on every boot, so adopt them as the box's own rather than
letting it generate another set on first boot. Copy them onto the persist volume now:
```sh
install -d -m 0755 "$INSTALL_ROOT/persist/etc/ssh"
for t in ed25519 rsa; do
install -m 0600 "/etc/ssh/ssh_host_${t}_key" "$INSTALL_ROOT/persist/etc/ssh/ssh_host_${t}_key"
install -m 0644 "/etc/ssh/ssh_host_${t}_key.pub" "$INSTALL_ROOT/persist/etc/ssh/ssh_host_${t}_key.pub"
done
```
`my.tmproot` persists `services.openssh.hostKeys` at exactly those paths, so the installed system
picks them up. This means the box's key is known **before** it first boots, so `my.secrets.key` can
be set and its secrets encrypted as part of the same pass — no install, boot, re-encrypt, re-deploy
round trip. (For a box already up, the `ssh-get-ed25519 <host>` devshell command prints the same
value in the form `my.secrets.key` wants.)
## Phase 4 — Capture the hardware config
Re-run `installer-shell bash -lic show-hw-config` with the filesystems mounted.
Read the whole generated file and carry over **anything** in it that the flake does not already
provide — it reflects what was actually detected on this hardware, and the list below is just what
usually shows up, not a limit:
- `boot.initrd.availableKernelModules` and `boot.initrd.kernelModules` (LVM adds `dm-snapshot`)
- `boot.kernelModules` (`kvm-intel` / `kvm-amd`) and the microcode attribute
- the ESP's `by-uuid` device, and the device paths for `/nix` and `/persist`
- anything else it emits — `boot.extraModulePackages`, `hardware.*` attributes, `swapDevices`,
additional detected filesystems, `imports` such as `not-detected.nix`
The test is conflict, not familiarity: drop an option only when a nixfiles module already sets it,
and keep it otherwise. The flake's own modules cover the bootloader, `initrd.systemd`,
`initrd.services.lvm`, the kernel package and `nixpkgs.hostPlatform` (see
[`nixos/modules/common.nix`](../nixos/modules/common.nix) and
[`nixos/default.nix`](../nixos/default.nix)), so those are the ones to leave out. Don't paste the
file in wholesale either — translate it into the box's own style, and reference LVM volumes as
`/dev/main/<host>-nix` rather than the generated `/dev/mapper/main-<host>--nix`.
## Phase 5 — Write the box config
Create `nixos/boxes/<site>/<host>/default.nix` (a directory, so per-topic files can be added
alongside it later) declaring `nixos.systems.<host>`, and add its path to the `configs` list in
[`flake.nix`](../flake.nix). Then `git add` it.
The minimum is `system`, `nixpkgs`, `home-manager` and a `configuration` with the hardware from
Phase 4, the three filesystems, and networking. Beyond that:
- **Interface naming:** pin names to hardware with `.link` files matching `PermanentMACAddress`,
named for speed and index — `et1g0`, `et2g5-0`, `et10g-1`. Never rely on predictable-interface
names in the `.network` files.
- **Servers** set `my.server.enable = true`.
- **Secrets:** set `my.secrets.key` to the ed25519 public key seeded in Phase 3 (the key only, no
`root@installer` comment). Note that **every box declares at least one secret** even if its own
config declares none: [`nixos/modules/user.nix`](../nixos/modules/user.nix) adds
`user-passwd.txt` whenever `my.user.enable` is on, which is the default. So setting
`my.secrets.key` always adds the box to that file's recipients, and
`ragenix --rekey-one secrets/user-passwd.txt.age` is required — skip it and the box cannot
decrypt its user password on first boot. Confirm what the box actually declares with
`nix eval .#nixosConfigurations.<host>.config.age.secrets --apply builtins.attrNames`, and
re-encrypt each of those files the same way. Create any new secrets with `ragenix -e <path>`.
Never use `--rekey`, which rewrites every secret in `secrets/`.
- **A box staged away from its final home** gets a bootstrap `.network` taking DHCP, plus
`systemd.network.wait-online.anyInterface = true` so boot does not block on unpatched ports, and
an explicit `my.deploy.node.hostname`. Comment it as temporary and say what replaces it.
Validate with `check-system <host>` and fix eval errors before going near the target.
## Phase 6 — Install
⏸ The maintainer may want to run this step themselves; ask rather than assume.
`do-install <host>` builds the system's `toplevel`, `nix copy`s the closure into the installer's
`$INSTALL_ROOT` store, points `/nix/var/nix/profiles/system` at it, touches `/etc/NIXOS`, and runs
`switch-to-configuration boot` with `NIXOS_INSTALL_BOOTLOADER=1`. It prompts for confirmation and
prints the target it resolved.
- `--no-bootloader` skips the bootloader install (for a box that boots by other means).
- `--no-substitute` copies everything from the local store instead of letting the target substitute.
## Phase 7 — First boot and post-install
1. Reboot the box off the installer and confirm it comes up: it should get its address, and
`hostname` should be the system name. Its SSH host key is the one seeded in Phase 3, so it
presents the same fingerprint the installer did.
2. **Secrets.** If Phase 5 set `my.secrets.key`, they already decrypt. [`secrets.nix`](../secrets.nix)
computes the ragenix recipient list from that key at evaluation time, so nothing needs
regenerating — but any secret added to the box later must be re-encrypted for the new recipient
list with `ragenix --rekey-one <path>`, one file at a time. Never reach for `ragenix --rekey`:
it rewrites every secret in `secrets/` and buries the actual change in churn.
3. **Deploy.** `deploy .#<host>` should now work over the `deploy` user. If the box is staged
somewhere without its final DNS name, `deploy --hostname <address> .#<host>` overrides the node
hostname for one run.
## Phase 8 — Document it
Per [`AGENTS.md`](../AGENTS.md), a new box means:
- a box page under the right docs directory, following the standard layout (H1 + one-line intro;
`Source` / `Host` / `nixpkgs` bullets; hardware inventory; `## Role`; `## Network assignments`
linking to [`networking.md#box-assignments`](networking.md#box-assignments), or a short
explanation if it has none yet; one `##` per topic; `## Notable config files` last);
- a row in the site index `README.md` boxes table;
- affected prose in [`networking.md`](networking.md) — the assignment tables themselves are
CI-generated, so write the prose and leave the tables alone;
- the site diagram in [`README.md`](README.md) if the box changes its layout.
docs/misc/installer.md
+5 -9
View File
@@ -24,20 +24,16 @@ The custom NixOS installer image used to bootstrap new boxes.
`installer-<hex>` hostname is set at boot.
- `INSTALL_ROOT=/mnt` in the session environment, plus a `show-hw-config` alias wrapping
`nixos-generate-config --show-hardware-config --root $INSTALL_ROOT`.
- NixOS documentation enabled, NetworkManager available but not started at boot (run
`systemctl start NetworkManager`, then `nmtui`), GC and memory-overcommit tuning for low-memory
targets, LVM thin and NFS support.
- Identifies itself as `VARIANT_ID=installer` in `/etc/os-release`, and leaves the target's
persistent pstore entries alone (`Unlink=no`) so an install doesn't evacuate them.
- NixOS documentation enabled, `wpa_supplicant` available but not started, aggressive GC
(`GC_INITIAL_HEAP_SIZE=1M`) and `vm.overcommit_memory=1` for low-memory targets, LVM thin
and NFS support.
- No regular user (`my.user.enable = false`), no tmpfs-root management, no NAT, and not a
deploy target (`my.deploy.enable = false`).
## Installing a box
The end-to-end procedure — hardware inventory, partitioning, writing the box config, installing and
documenting it — is in [`install-box.md`](../install-box.md). The devshell's installer commands
([`devshell/install.nix`](../../devshell/install.nix)) drive an install over SSH against a booted
installer reachable at `$INSTALLER`:
The devshell's installer commands ([`devshell/install.nix`](../../devshell/install.nix)) drive
an install over SSH against a booted installer reachable at `$INSTALLER`:
- `installer-shell` — get a shell on the installer.
- `do-install <system>` — builds the system's toplevel, `nix copy`s the closure to the
docs/mobile/README.md
+3 -3
View File
@@ -4,6 +4,6 @@ Portable workstations that move between networks — currently just `tower`. Mob
no static network assignments; they use DHCP/NetworkManager and reach other boxes over the
tailnet.
| Box | What it is |
| --- | --- |
| [`tower`](tower.md) | Framework Laptop 13 workstation |
| Box | What it is | Docs |
| --- | --- | --- |
| `tower` | Framework Laptop 13 workstation | [tower.md](tower.md) |
docs/mobile/tower.md
+8 -20
View File
@@ -4,22 +4,6 @@ Portable workstation — a Framework Laptop 13 (Intel), running the full GUI env
- **Source:** [`nixos/boxes/tower/default.nix`](../../nixos/boxes/tower/default.nix)
- **Host:** physical (laptop)
- **nixpkgs:** `mine`
## Hardware / platform
| Component | Inventory |
|---|---|
| Platform | Framework Laptop 13 (12th Gen Intel Core) |
| CPU | Intel Core i5-1240P (12 cores / 16 threads) |
| Memory | 32 GiB |
| Storage | 2 TB WD Black SN850 NVMe SSD |
| Graphics | Integrated Intel Iris Xe |
| Connectivity | Intel AX210 Wi-Fi 6E, Bluetooth and Thunderbolt 4 |
The configuration enables Intel microcode updates, `kvm-intel`, `intel_iommu=on`,
`intel-media-driver` and the latest kernel (`lib.my.c.kernel.latest`). Thunderbolt security
(`bolt`), the fingerprint reader (`fprintd`) and `tlp` power management are also enabled.
## Role
@@ -27,16 +11,20 @@ The configuration enables Intel microcode updates, `kvm-intel`, `intel_iommu=on`
- Joins the tailnet through the headscale on [`britway`](../remote/britway.md) (fish abbr
`tsup` = `doas tailscale up --login-server=https://hs.nul.ie --accept-routes`).
## Network assignments
## Hardware / platform
`tower` has no static assignment; it uses DHCP through NetworkManager and reaches the other boxes
over Tailscale.
- Intel platform: microcode updates, `kvm-intel`, `intel_iommu=on`, `intel-media-driver` for
graphics; latest kernel (`lib.my.c.kernel.latest`).
- Thunderbolt security (`bolt`), Bluetooth (`blueman` + tray applet), fingerprint reader
(`fprintd`; `doas` persists auth for `wheel`).
- `tlp` power management, including battery charge thresholds (start 90% / stop 97%).
## Storage
- Two LUKS-encrypted partitions, `persist` and `home` (both `allowDiscards`); `/nix` is a
separate ext4 filesystem, `/boot` is vfat.
- Persistent `/home` (`my.user.tmphome = false`) with a size-limited tmpfs root (`my.tmproot.size`).
- Persistent `/home` (`my.user.tmphome = false`) with an 8G tmpfs root
(`my.tmproot.size = "8G"`).
## Networking / services
docs/networking.md
+68 -359
View File
@@ -8,7 +8,7 @@ sites together. Switch-level home topology (jim/dave/brian, the ONT path) lives
## Assignments
Every box declares `nixos.systems.<name>.assignments`, an attrset of *assignments* (one per
network the box is attached to). The option definition (`assignmentOpts` in
network the box is attached to). The option shape (`assignmentOpts` in
[`nixos/default.nix`](../nixos/default.nix)):
- `name` (defaults to the attribute name) and `altNames` — DNS names for the assignment.
@@ -41,197 +41,6 @@ Two pieces of machinery consume assignments:
the system name) and `networking.domain` to `assignments.internal.domain`. The shared
`network` module sets a fallback domain of `int.nul.ie` for boxes without one.
## Box assignments
Every box's `assignments` (plus the routers' floating VIP `extraAssignments`), grouped first by site
and then by assignment key. Generated from `nixos.allAssignments` by
`nix run .#update-docs-assignments`; CI keeps it current. Only the **Notes** column is hand-written —
edit prose there, never the other generated cells.
### colony
<!-- assignments: colony -->
<!-- assignments-start -->
#### `internal`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`chatterbox`](sites/colony/shill/containers/chatterbox.md) | `10.100.2.5/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::5/64` | ams1.int.nul.ie | |
| [`colony`](sites/colony/colony.md) | `94.142.241.224/32` | `2a0e:97c0:4d2:10::2/64` | ams1.int.nul.ie | |
| [`colony-psql`](sites/colony/shill/containers/colony-psql.md) | `10.100.2.4/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::4/64` | ams1.int.nul.ie | |
| `enshrouded-oci` | `10.100.3.5/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::5/64` | ams1.int.nul.ie | Enshrouded OCI container on [`whale2`](sites/colony/whale2.md#game-servers); disabled |
| [`estuary`](sites/colony/estuary.md) | `94.142.240.44/24 gw 94.142.240.254` | `2a02:898:0:20::329:1/64 gw 2a02:898:0:20::1` | ams1.int.nul.ie | |
| [`gam`](sites/colony/shill/containers/gam.md) | `10.100.2.11/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::b/64` | ams1.int.nul.ie | |
| [`git`](sites/colony/git.md) | `94.142.241.117/32` | `2a0e:97c0:4d2:11::4/64` | ams1.int.nul.ie | |
| `graeme-oci` | `10.100.3.8/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::8/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| [`jackflix`](sites/colony/shill/containers/jackflix.md) | `10.100.2.6/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::6/64` | ams1.int.nul.ie | |
| `kevcraft-oci` | `10.100.3.6/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::6/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| `kinkcraft-oci` | `10.100.3.7/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::7/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| [`middleman`](sites/colony/shill/containers/middleman.md) | `10.100.2.2/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::2/64` | ams1.int.nul.ie | |
| [`object`](sites/colony/shill/containers/object.md) | `10.100.2.7/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::7/64` | ams1.int.nul.ie | |
| [`qclk`](sites/colony/shill/containers/qclk.md) | `10.100.2.10/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::a/64` | ams1.int.nul.ie | |
| [`shill`](sites/colony/shill/README.md) | `94.142.241.225/32` | `2a0e:97c0:4d2:11::2/64` | ams1.int.nul.ie | |
| `simpcraft-oci` | `10.100.3.3/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::3/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| `simpcraft-staging-oci` | `10.100.3.4/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::4/64` | ams1.int.nul.ie | Minecraft staging OCI container on [`whale2`](sites/colony/whale2.md#game-servers); disabled |
| [`toot`](sites/colony/shill/containers/toot.md) | `10.100.2.8/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::8/64` | ams1.int.nul.ie | |
| `valheim-oci` | `10.100.3.2/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::2/64` | ams1.int.nul.ie | Valheim OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
| [`vaultwarden`](sites/colony/shill/containers/vaultwarden.md) | `10.100.2.3/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::3/64` | ams1.int.nul.ie | |
| [`waffletail`](sites/colony/shill/containers/waffletail.md) | `10.100.2.9/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::9/64` | ams1.int.nul.ie | |
| [`whale2`](sites/colony/whale2.md) | `94.142.241.226/32` | `2a0e:97c0:4d2:11::3/64` | ams1.int.nul.ie | |
#### `as211024`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`estuary`](sites/colony/estuary.md) | `10.100.50.1/24` | `2a0e:97c0:4df::1/64` | — | |
#### `base`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`estuary`](sites/colony/estuary.md) | `10.100.0.1/24` | `2a0e:97c0:4d2:10::1/64` | ams1.int.nul.ie | |
#### `ctrs`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`shill`](sites/colony/shill/README.md) | `10.100.2.1/24` | `2a0e:97c0:4d2:12::1/64` | ams1.int.nul.ie | |
#### `oci`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`whale2`](sites/colony/whale2.md) | `10.100.3.1/24` | `2a0e:97c0:4d2:13::1/64` | ams1.int.nul.ie | |
#### `qclk`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`qclk`](sites/colony/shill/containers/qclk.md) | `10.100.4.1/24` | — | — | |
#### `routing`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`colony`](sites/colony/colony.md) | `10.100.0.2/24 gw 10.100.0.1` | — | ams1.int.nul.ie | |
| [`git`](sites/colony/git.md) | `10.100.1.4/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
| [`shill`](sites/colony/shill/README.md) | `10.100.1.2/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
| [`whale2`](sites/colony/whale2.md) | `10.100.1.3/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
#### `tailscale`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`waffletail`](sites/colony/shill/containers/waffletail.md) | `100.64.0.5/32` | `fd7a:115c:a1e0::5/128` | — | |
#### `vms`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`colony`](sites/colony/colony.md) | `10.100.1.1/24` | `2a0e:97c0:4d2:11::1/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
### home
<!-- assignments: home -->
<!-- assignments-start -->
#### `as211024`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`river`](sites/home/river.md) | `10.100.50.2/24` | `2a0e:97c0:4df:0:1::1/64 gw 2a0e:97c0:4df:0:2::1` | — | |
| [`stream`](sites/home/stream.md) | `10.100.50.3/24` | `2a0e:97c0:4df:0:1::2/64 gw 2a0e:97c0:4df:0:2::1` | — | |
#### `core`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`palace`](sites/home/palace.md) | `192.168.64.20/24` | — | h.nul.ie | |
| [`river`](sites/home/river.md) | `192.168.64.1/24` | — | h.nul.ie | |
| [`stream`](sites/home/stream.md) | `192.168.64.2/24` | — | h.nul.ie | |
| [`unifi`](sites/home/sfh/containers/unifi.md) | `192.168.64.21/24` | — | h.nul.ie | |
#### `hi`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`castle`](sites/home/castle.md) | `192.168.68.40/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::3:1/64` | h.nul.ie | |
| [`cellar`](sites/home/cellar.md) | `192.168.68.80/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:1/64` | h.nul.ie | |
| [`hass`](sites/home/sfh/containers/hass.md) | `192.168.68.103/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:3/64` | h.nul.ie | |
| [`palace`](sites/home/palace.md) | `192.168.68.22/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::2:1/64` | h.nul.ie | |
| [`portcullis`](sites/colony/portcullis.md) | `192.168.68.41/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::6:1/64` | h.nul.ie | |
| [`river`](sites/home/river.md) | `192.168.68.1/22` | `2a0e:97c0:4d0:1::1/64` | h.nul.ie | |
| `router-hi` | `192.168.71.254/22 gw 192.168.68.1` | `2a0e:97c0:4d0:1::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
| [`sfh`](sites/home/sfh/README.md) | `192.168.68.81/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:2/64` | h.nul.ie | |
| [`stream`](sites/home/stream.md) | `192.168.68.2/22` | `2a0e:97c0:4d0:1::2/64` | h.nul.ie | |
| [`unifi`](sites/home/sfh/containers/unifi.md) | `192.168.68.100/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:1/64` | h.nul.ie | |
#### `lo`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`hass`](sites/home/sfh/containers/hass.md) | `192.168.72.103/21` | `2a0e:97c0:4d0:2::5:3/64` | h.nul.ie | |
| [`river`](sites/home/river.md) | `192.168.72.1/21` | `2a0e:97c0:4d0:2::1/64` | h.nul.ie | |
| `router-lo` | `192.168.79.254/21 gw 192.168.72.1` | `2a0e:97c0:4d0:2::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
| [`stream`](sites/home/stream.md) | `192.168.72.2/21` | `2a0e:97c0:4d0:2::2/64` | h.nul.ie | |
#### `untrusted`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`river`](sites/home/river.md) | `192.168.80.1/24` | `2a0e:97c0:4d0:3::1/64` | h.nul.ie | |
| `router-ut` | `192.168.80.254/24 gw 192.168.80.1` | `2a0e:97c0:4d0:3::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
| [`stream`](sites/home/stream.md) | `192.168.80.2/24` | `2a0e:97c0:4d0:3::2/64` | h.nul.ie | |
<!-- assignments-end -->
### remote
<!-- assignments: remote -->
<!-- assignments-start -->
#### `internal`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`kelder-acquisition`](remote/kelder/containers/kelder-acquisition.md) | `172.16.64.2/24 gw 172.16.64.1` | — | hentai.engineer | |
| [`kelder-spoder`](remote/kelder/containers/kelder-spoder.md) | `172.16.64.3/24 gw 172.16.64.1` | — | hentai.engineer | |
#### `allhost`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`britnet`](remote/britnet.md) | `77.74.199.67/24 gw 77.74.199.1` | `2a12:ab46:5344:99::a/64 gw 2a12:ab46:5344::1` | bhx1.int.nul.ie | |
#### `as211024`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`britway`](remote/britway.md) | `10.100.50.5/24` | `2a0e:97c0:4df:0:2::1/64` | — | |
#### `ctrs`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`kelder`](remote/kelder/README.md) | `172.16.64.1/24` | — | hentai.engineer | |
#### `estuary`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`kelder`](remote/kelder/README.md) | `94.142.242.254/32` | — | — | |
#### `vpn`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`britnet`](remote/britnet.md) | `10.200.0.1/24` | `fdfb:5ebf:6e84::1/64` | — | |
#### `vultr`
| Box | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|
| [`britway`](remote/britway.md) | `45.76.141.188/23 gw 45.76.140.1` | `2001:19f0:7402:128b::1/64` | lon1.int.nul.ie | |
<!-- assignments-end -->
## Domains
The public domain is `nul.ie` (`lib.my.c.pubDomain`). Each site has its own internal domain
@@ -267,11 +76,6 @@ On top of that: `p2pTunnels` (`10.100.5.0/24`) holds point-to-point tunnel /30s
public blocks and the per-customer `mail` / `darts` / `jam` prefixes carry customer-facing
services with their own public addresses (announced by BGP, routed via the host).
This layout is expected to change: [`portcullis`](sites/colony/portcullis.md) is bare-metal edge
hardware headed for Nikhef that will take over most of `estuary`'s routing. It has no colony
assignments yet (only a home `hi` one, from being staged at home) and the replacement topology is
still being designed.
## home
The home site prefixes (`lib.my.c.home.prefixes`) come from `192.168.64.0/18` and
@@ -314,58 +118,36 @@ which the other sites use as their next-hop into the home prefixes.
### Router HA
#### VRRP
[`routing-common/keepalived.nix`](../nixos/boxes/home/routing-common/keepalived.nix) defines separate
v4/v6 instances (router IDs 51/52) on `lan-core`. Index 0 (`river`) starts as `MASTER`, priorities
are `255 - index`, and track scripts demote a router whose WAN checks fail. All VIPs of an address
family move together.
#### Client gateway and DNS
`kea` hands out `vips.<vlan>.v4` as both `routers` and `domain-name-servers`, with the two routers
serving disjoint pool halves. `radvd` advertises the v6 VIP as RDNSS (`untrusted` gets Cloudflare)
and keepalived's `notify_master`/`notify_backup` hooks ensure that only the master sends RAs.
Statically-addressed boxes (the servers on `hi`) don't run DHCP, so they'd otherwise learn a
resolver only from the v6 RA RDNSS — which vanishes when v6 is disabled, taking DNS with it. They
instead anchor DNS on the VIPs via the shared `lib.my.c.home.vlanDns "<vlan>"` fragment, which sets
`DNS` to `vips.<vlan>.{v4,v6}` and `Domains` to the advertised search list; the always-present
static v4 VIP keeps resolution working even with v6 down.
#### DNS binding
`pdns-recursor` binds the VIPs directly; see
[`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix). The
`net.ipv4.ip_nonlocal_bind` / `net.ipv6.ip_nonlocal_bind` settings let the backup listen before it
owns the addresses, so failover does not depend on client resolver timeouts. The recursor forwards
the site's zones to authoritative PowerDNS on `127.0.0.1:5353`. The generated
[DNS reference](reference/dns.md) lists the live forward and reverse records; the authoritative
servers allow its AXFRs from the shared internal prefixes and the colony site's egress address.
#### `wan-online.target`
This shared, initially inert systemd target means "the public IPv4 WAN route is up".
`routing-common` only declares it: `stream` gates it on a oneshot that waits for the DHCP default
route on `wan`, while `river`'s `pppd` hooks start and stop it. Consumers such as `ipsec` attach with
`wantedBy` + `partOf` + `after`, never `requires`/`wants`, so they cannot pull the target in early
and they reload on WAN flap.
- **VRRP via `keepalived`** ([`routing-common/keepalived.nix`](../nixos/boxes/home/routing-common/keepalived.nix)):
separate v4/v6 instances (router IDs 51/52) on the `lan-core` link; index 0 (river) starts
as `MASTER`, priorities are `255 - index`. Track scripts ping public v4/v6 anycast targets
and demote a router whose WAN is down. All VIPs of a family move together.
- **Clients get the VIP as gateway *and* DNS.** `kea` hands out `vips.<vlan>.v4` as both
`routers` and `domain-name-servers`, with the two routers serving disjoint pool halves;
`radvd` advertises the v6 VIP as RDNSS (the `untrusted` VLAN gets Cloudflare instead) and
is started/stopped by keepalived's `notify_master`/`notify_backup` hooks so only the master
sends RAs.
- **`pdns-recursor` binds the VIPs directly** ([`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix)),
with `net.ipv4.ip_nonlocal_bind` / `net.ipv6.ip_nonlocal_bind` so the backup can listen on
addresses it doesn't currently hold — failover doesn't depend on client resolver timeouts.
The recursor forwards the site's zones to the local authoritative PowerDNS on `127.0.0.1:5353`.
- **`wan-online.target`** is a shared, initially-inert systemd target meaning "the public
IPv4 WAN route is up". `routing-common` only declares it; each box wires how it's reached —
stream gates it on a oneshot that waits for the DHCP default route on `wan`, river's `pppd`
`ip-up`/`ip-down` hooks start/stop it. Consumers (e.g. `ipsec`, the RA-default-route
cleanup) attach **to** it with `wantedBy` + `partOf` + `after`, never `requires`/`wants`,
so the target is never pulled in early and services re-load on WAN flap.
### WAN paths (summary)
#### `river`
The VM on `palace` runs Digiweb PPPoE directly on VLAN 10 (`vlans.pon-isp`), trunked untranslated
through the switches. The carrying interface is named `wan-pon-isp`; VLAN ID 141 with that name is
reserved for a future translation scheme. MTU 1508 preserves a 1500-byte PPP session, IPCP requests
the static address, and the `pppd` hooks own `wan-online.target`. `wan-pon-ont` (VLAN 140) reaches
the ONT management subnet at `192.168.100.0/24`, where `river` takes `.100`.
#### `stream`
The bare-metal backup uses DHCP from the Virgin Media modem on `wan` (VLAN 130), keeps a static
`192.168.0.100/24` management address beside the public lease, and applies CAKE shaping through
`wan-ifb`.
- **river** (a VM on `palace`): PPPoE to Digiweb via `services.pppd`, running directly on the
ISP's VLAN 10 (`wan-pon-isp`, trunked untranslated through the switches; baby-jumbo MTU 1508
so the PPP session is a clean 1500). The static IP is requested in IPCP; the pppd hooks own
`wan-online.target`. The ONT's management subnet (`192.168.100.0/24`) is reached on
`wan-pon-ont` (VLAN 140, PVID'd at the brian switch), where river takes `.100`.
- **stream** (bare metal): DHCP on the Virgin Media cable modem (VLAN 130) on `wan`, with a
static modem-management address (`192.168.0.100/24`) alongside the public lease, and CAKE
egress/ingress shaping via `wan-ifb`.
The full fabric story — which switch port carries what, why VLAN 10 is trunked untranslated,
and the multi-ONT plan — is in [sites/home/switches.md](sites/home/switches.md); the
@@ -378,130 +160,57 @@ The edge routers are joined by a layer-2 mesh, defined once as `nixos.vpns.l2.as
[`nixos/boxes/colony/vms/estuary/default.nix`](../nixos/boxes/colony/vms/estuary/default.nix)
and realised on each member by the [`l2mesh` module](../nixos/modules/l2mesh.nix):
### Members
`estuary`, `river`, `stream` and `britway` peer on their public addresses.
### Transport
VXLAN (VNI 211024, UDP port 4789) uses static per-peer FDB entries and UDP-encapsulated IPsec in
Libreswan transport mode. It authenticates without encryption by default; `security.encrypt`
switches ESP from `null-sha256` to AES-GCM. The shared `l2mesh/as211024.key` PSK is expanded into
`/run/l2mesh.secrets` when `ipsec` starts.
### ESP throughput
ESP is the mesh's throughput limit rather than VXLAN — encapsulation is close to free, and on a
small box the crypto is what costs. The kernel processes a single SA on a single core, so per-peer
throughput is capped by one core however many the box has. That limit binds per peer rather than
per box, so a router spreads naturally across its peers.
Two things follow for configuration:
- The `esp4_offload` / `esp6_offload` modules provide GSO/GRO batching for ESP and are **not**
autoloaded when an SA is created. The [`l2mesh` module](../nixos/modules/l2mesh.nix) loads the
one matching each secured mesh's underlay family; without them throughput is around a third
lower, for more CPU.
- `security.encrypt = false` does not save CPU on hardware with AES-NI — it measured slower than
AES-GCM. GCM resolves to a single fused accelerated implementation, while the authenticate-only
path falls back to a generic `authenc(hmac(sha256),ecb(cipher_null))` composition.
#### NIC crypto offload
No mesh uses it, and the `esp4_offload` / `esp6_offload` modules above are unrelated to it — those
are software GSO/GRO batching. Hardware ESP offload is a separate XFRM feature that Libreswan only
requests for connections setting `nic-offload=yes`, which the
[`l2mesh` module](../nixos/modules/l2mesh.nix) does not.
[`portcullis`](sites/colony/portcullis.md)'s 82599ES ports advertise `esp-hw-offload` and the
offload does work, but not for anything the meshes could use. Measured on the box by installing
SAs directly with `ip xfrm` and watching `ixgbe`'s `tx_ipsec` counter:
| SA | Result |
|---|---|
| `et10g-0`, transport, AES-GCM-128 | offload active — `mode crypto` against the physical port |
| `et10g-0`, transport, AES-GCM-256 | rejected: *"IPsec hw offload only supports keys up to 128 bits with a 32 bit salt"* |
| `et10g-0`, **tunnel** mode | rejected: *"Unsupported mode for ipsec offload"* |
| `lan-hi` (a VLAN on `et10g-0`) | **accepted with the offload silently dropped** — software crypto |
| `et2g5-0` (I226-V) | accepted, offload silently dropped — `igc` has none |
Two traps are worth knowing. Binding an SA to a device that cannot offload is **not** an error:
`xfrm_dev_state_add` returns success having cleared the device, so the SA looks fine and quietly
runs in software. And a VLAN interface never offloads — it carries no `xfrmdev_ops`, and
`esp-hw-offload` reads `off [fixed]` on it even when its parent supports the feature.
The second trap is the decisive one here. Even with the offload genuinely active against
`et10g-0`, driving traffic through the SA left `tx_ipsec` at zero, because the packets egress
`lan-hi` and the kernel only offloads when the SA's device matches the egress device. Every address
`portcullis` holds is on a VLAN, so an SA would have to be bound to an untagged physical port to
see the hardware at all.
So adopting it would mean dropping to a 128-bit key to suit one NIC family, keeping the underlay
off VLANs, and forgoing `udpEncapsulation` — `xfrm_dev_offload_ok` refuses any SA carrying
`encap`. That is not a trade worth making for a mesh that has to run across boxes with no offload
at all.
#### pcrypt
`pcrypt` parallelises an SA's crypto across cores via padata and does lift the per-SA ceiling. It
is not enabled on any box here, and is recorded as an option rather than a recommendation:
- It cannot be named in the SA — `ip xfrm` and the kernel both validate AEAD names against a fixed
list. It is engaged instead by registering a `pcrypt(...)`-wrapped instance under the standard
algorithm name at a higher priority, over `NETLINK_CRYPTO` (`CONFIG_CRYPTO_USER`). `crconf` is
the usual tool for that and is not packaged in nixpkgs.
- The registration is global: it redirects every user of that algorithm on the box, not just the
mesh, and would have to run before `ipsec` starts.
- A single-threaded submit path remains, so it does not scale with core count, and it trades
latency and packet ordering for throughput.
### Overlay addressing
The overlay uses `10.100.50.0/24` / `2a0e:97c0:4df::/64`. Each router holds `10.100.50.<n>`:
`estuary` `.1`, `river` `.2`, `stream` `.3`, and `britway` `.5`. Interface MTU is calculated from
the physical MTU minus VXLAN/UDP/IPsec overhead.
### Routing
The home routers route colony prefixes via `estuary` and Tailscale prefixes via `britway`;
`estuary` and `britway` route home prefixes through the `10.100.50.4` VIP. The home IPv6 default
route also crosses the mesh through `britway`, which is why the recursor pins its upstreams to IPv4
as noted in [`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix). The `nftTrust`
snippet in `lib.my.c.as211024` admits trusted colony, home, mesh and Tailscale prefixes at the edge
firewalls.
- **Members**: `estuary`, `river`, `stream`, `britway`, peering on their public addresses.
- **Transport**: VXLAN (VNI 211024, UDP port 4789) with static per-peer FDB entries and
UDP-encapsulated IPsec in transport mode via Libreswan (authentication-only by default;
`security.encrypt` would switch ESP from `null-sha256` to AES-GCM). The PSK is the shared
`l2mesh/as211024.key` secret, expanded into `/run/l2mesh.secrets` at `ipsec` start.
- **Overlay addressing**: `10.100.50.0/24` / `2a0e:97c0:4df::/64`; the interface MTU is
computed from the physical MTU minus VXLAN/UDP/IPsec overhead. Each router holds
`10.100.50.<n>` (estuary `.1`, river `.2`, stream `.3`, britway `.5`).
- **Routing over it**: the home routers route the colony prefixes via estuary and the
Tailscale prefixes via britway; estuary and britway route the home prefixes via the
`10.100.50.4` VIP. The home IPv6 **default** route also runs over the mesh, via britway
(hence the recursor's IPv4-only upstream pinning noted in
[`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix)). The `nftTrust`
snippet in `lib.my.c.as211024` lets trusted inter-site traffic (colony, home, mesh,
Tailscale prefixes) through the edge firewalls.
## BGP
Both edge routers run `bird2` as AS211024.
Both edge routers run `bird2` as AS211024:
### `estuary`
The colony edge takes a full table from ColoClue, IPv6 transit from iFog and Hurricane Electric,
and peers through the Frys-IX, NL-ix and FogIXP route servers. It also has direct and monitoring
sessions; the complete peer table and originated routes live in [estuary.md](sites/colony/estuary.md#bgp).
### `britway`
The London edge uses secret-backed MD5 authentication for Vultr transit (AS64515), connects to
`bgp.tools`, and originates the internal, colony and home IPv6 prefixes. See
[britway.md](remote/britway.md) for its box-specific routing detail.
- **estuary** ([`bgp.nix`](../nixos/boxes/colony/vms/estuary/bgp.nix)): full table from
ColoClue (AS8283) over two sessions per family, plus IPv6 transit from iFog and Hurricane
Electric; peering at the Frys-IX, NL-ix and FogIXP route servers and with various networks
(Meta, Cloudflare, Apple, LUJE.net, …); a `bgp.tools` monitoring session. Static routes pull
the customer VIP blocks out of the `base` network and the internal/home IPv6 prefixes out of
the mesh.
- **britway** ([`bgp.nix`](../nixos/boxes/britway/bgp.nix)): Vultr transit (AS64515,
MD5-passworded from a secret) and a `bgp.tools` session, originating the internal, colony
and home IPv6 prefixes.
## WireGuard point-to-point tunnels
Separate from the mesh, a few boxes run their own WireGuard (private keys in per-box secrets):
Separate from the mesh, estuary terminates several networkd-managed WireGuard tunnels (private
keys in per-box secrets):
- **estuary** terminates point-to-point tunnels to `kelder`, `hillcrest` and `john-valorant`,
addressed out of `p2pTunnels` — see
[estuary.md](sites/colony/estuary.md#wireguard-tunnels) for the per-tunnel ports and prefixes.
- **`qclk`** (container on `shill`) runs its own WireGuard on port 51821 out of `10.100.4.0/24`
— see [qclk.md](sites/colony/shill/containers/qclk.md).
- **`britnet`** hosts a road-warrior WireGuard VPN on port 51820 serving `10.200.0.0/24` /
`fdfb:5ebf:6e84::/64` — see [britnet.md](remote/britnet.md).
| Tunnel | Port | Remote / prefix | Notes |
|---|---|---|---|
| estuary ↔ kelder | 51820 | kelder holds `94.142.242.254/32` | kelder's public presence is a colony /32 routed over the tunnel |
| estuary ↔ hillcrest | 51822 | `10.100.5.0/30` | point-to-point /30 out of `p2pTunnels` |
| estuary ↔ john-valorant | 51823 | `10.100.5.4/30` | point-to-point /30 out of `p2pTunnels` |
Additionally, the `qclk` container on `shill` runs its own WireGuard instances on port 51821
out of `10.100.4.0/24`, and `britnet` hosts a road-warrior style WireGuard VPN on port 51820
serving `10.200.0.0/24` / `fdfb:5ebf:6e84::/64`.
## Tailscale / headscale
Tailscale runs against a self-hosted **headscale** control plane on britway at `https://hs.nul.ie`
(OIDC, MagicDNS, split DNS — see [britway.md](remote/britway.md)). The tailnet prefixes are
Tailscale runs against a self-hosted **headscale** control plane on britway
([`tailscale.nix`](../nixos/boxes/britway/tailscale.nix)) at `https://hs.nul.ie`: Google OIDC
login, SQLite state, MagicDNS under `ts.nul.ie`, and split DNS that resolves the colony and
home internal domains through their site resolvers. The tailnet prefixes are
`100.64.0.0/10` / `fd7a:115c:a1e0::/48`.
Notable nodes:
docs/nixpkgs-upgrade.md
-161
View File
@@ -1,161 +0,0 @@
# Upgrading nixpkgs
Procedure for the periodic upgrade of all four nixpkgs channels (`unstable`, `stable`, `mine`,
`mine-stable`) and home-manager. Written to be followed by a person or any coding agent; a
shared agent-skill entry point exists at `.agents/skills/upgrade-nixpkgs/`, but the steps below are
the canonical source.
The upgrade is **guided, not automated**: do the mechanical and investigative steps, but stop at
the judgment points (marked ⏸) — pushing the fork, resolving rebase conflicts, editing the
stable-channel configuration, choosing a new release codename, and deleting version guards. Report
findings and let the maintainer decide. Keep a running summary and present it before any push or
commit.
Work the phases in order; skip one only if explicitly scoped to a subset.
## Setup facts
- **Fork checkout:** `~/documents/projects/nixpkgs` — remotes `origin` (`devplayer0/nixpkgs`) and
`upstream` (`NixOS/nixpkgs`). Confirm the path exists; if not, ask.
- **Fork branches:** `devplayer0` (tracks `nixos-unstable`) and `devplayer0-stable` (tracks the
current NixOS stable). Each is a small stack of local patches rebased onto upstream.
- **Flake pins** in `flake.nix` (`inputs`):
- `nixpkgs-unstable.url = "nixpkgs/nixos-unstable"`
- `nixpkgs-stable.url = "nixpkgs/nixos-<STABLE>"` (e.g. `nixos-26.05`)
- `nixpkgs-mine.url = "github:devplayer0/nixpkgs/devplayer0"`
- `nixpkgs-mine-stable.url = "github:devplayer0/nixpkgs/devplayer0-stable"`
- `home-manager-unstable.url = "home-manager"`
- `home-manager-stable.url = "home-manager/release-<STABLE>"`
- **Devshell commands:** `update-nixpkgs` = `nix flake update nixpkgs-{unstable,stable,mine,mine-stable}`;
`update-home-manager` = `nix flake update home-manager-{unstable,stable}`.
- **Validation:** prefer `check-system <host>` and `nix flake check --no-build` over full builds.
## Phase 1 — Determine the current NixOS stable
Do this first: everything downstream (the fork's `devplayer0-stable` rebase target, the `flake.nix`
stable pins) has to agree on one NixOS stable release, so establish it up front.
1. Find the latest NixOS stable release branch — check `git branch -r` on `upstream` for the newest
`release-YY.NN`, or the NixOS release schedule.
2. Compare it to `<STABLE>` in the `flake.nix` `nixpkgs-stable` / `home-manager-stable` URLs.
3. **If they already match** (no new stable): note "stable is current" and carry `<STABLE>` into the
later phases.
4. ⏸ **If a newer stable has cut:** stop and report the coordinated change set before proceeding —
the pieces must all move to the same release together:
- Rebase `devplayer0-stable` onto the new `upstream/release-YY.NN` (Phase 2 uses this target).
- Edit `flake.nix`: `nixpkgs-stable.url` and `home-manager-stable.url` → the new release.
- Choose a new `lib/default.nix` `versionOverlay` codename (Phase 4 updates it).
- Bump each system's `stateVersion` / `home.stateVersion` only if the maintainer explicitly
wants to — that is a separate, deliberate decision; never auto-bump.
Don't edit `flake.nix` here without confirmation.
## Phase 2 — Rebase the nixpkgs fork
For **both** branches — `devplayer0` onto `upstream/nixos-unstable`, and `devplayer0-stable` onto
`upstream/release-<STABLE>` (the release established in Phase 1):
1. In `~/documents/projects/nixpkgs`, confirm a clean working tree (`git status`). If dirty, stop
and report — don't stash silently.
2. `git fetch upstream --prune` and `git fetch origin --prune`. If the checkout has been idle a
long time this fetch can be large and slow; let it finish.
3. Enumerate the patch stack before rebasing:
`git log --oneline upstream/nixos-unstable..origin/devplayer0` (and the stable equivalent
against `upstream/release-<STABLE>`). For each commit, check whether it has landed upstream or
been superseded — e.g. `git log --oneline upstream/nixos-unstable -- <path>` or grep the
upstream tree for the package/option. Note any patch that now looks redundant.
4. Rebase: `git switch devplayer0 && git rebase upstream/nixos-unstable` (and the stable branch
onto `upstream/release-<STABLE>`).
- ⏸ **Conflicts:** stop. Report which patch conflicts and against what upstream change; let the
maintainer resolve, or drop the patch if it has been upstreamed.
- Clean rebase: continue.
5. Summarize: which patches still apply, which are now redundant (candidate to drop), which
conflicted.
6. ⏸ **Push:** only after confirmation. `git push --force-with-lease origin devplayer0
devplayer0-stable` (force needed — rebase rewrites history).
7. Wait for the GitHub mirror used by the flake inputs to catch up with the primary fork remote.
Compare the local branch tips with
`git ls-remote https://github.com/devplayer0/nixpkgs.git refs/heads/devplayer0
refs/heads/devplayer0-stable` and do not continue until both match. Updating sooner can leave
`nixpkgs-mine` and `nixpkgs-mine-stable` pinned to the pre-rebase commits even though the push
succeeded.
## Phase 3 — Update the pinned inputs
Run together (they move as a set):
```
update-nixpkgs
update-home-manager
```
Then show the `flake.lock` diff for the nixpkgs/home-manager entries so the old→new revisions are
visible.
## Phase 4 — Refresh kernels and release metadata
Update the repository values that deliberately move with nixpkgs upgrades:
1. In `lib/constants.nix`, inspect the kernel attributes available from the refreshed nixpkgs pins
and update both explicit selections:
- `kernel.lts` → the newest upstream long-term-support kernel carried by nixpkgs.
- `kernel.latest` → the newest kernel series carried by nixpkgs.
Keep explicit `pkgs.linuxKernel.packages.linux_X_Y` attributes rather than replacing them with
moving aliases. Confirm both attributes exist in the unstable and stable package sets used by
the boxes; if the newest choice is unavailable on stable, report that instead of breaking the
shared constant.
2. In `lib/default.nix`, update the `versionOverlay` values:
- Set the leading `YY.MM` in `trivial.release` to the current year and month. Preserve the
`:u-${prev.trivial.release}` suffix.
- If Phase 1 found a new NixOS stable release, ⏸ ask the maintainer to choose or approve a new
`trivial.codeName`, then update it as part of the coordinated stable bump. Otherwise retain the
existing codename.
Show these edits alongside the input changes in the upgrade summary.
## Phase 5 — Sweep version-gated behavior
The repo carries branch-conditional logic and TODOs keyed to specific nixpkgs versions; some become
removable after an upgrade, especially after a stable bump. Surface them:
```
grep -rn "versionAtLeast\|versionOlder\|when 2[0-9]\.[0-9][0-9]\|TODO.*2[0-9]\.[0-9][0-9]" \
--include=*.nix nixos home-manager lib pkgs flake.nix
```
Known example: `nixos/modules/common.nix` carries a `# TODO: Remove if-else when 26.11 releases`
guard. For each hit, evaluate whether the now-current versions make the guard removable and list
candidates. ⏸ Don't delete guards without confirmation — some protect the still-supported stable.
## Phase 6 — Review remaining flake inputs
Don't blanket-update. Walk the other inputs deliberately:
1. List inputs and locked revisions from `flake.lock` (or `nix flake metadata`).
2. For each meaningful input (`libnetRepo`, `devshell`, `determinate-nix`, `ragenix`, `deploy-rs`,
`impermanence`, and the packaged apps like `boardie`, `harmonia`, and `copyparty`),
compare the locked revision to upstream and summarize notable changes (breaking changes,
relevant fixes). Many inputs `follows` `nixpkgs-unstable` and already moved in Phase 3.
3. Propose a per-input update list with reasons; update the approved ones with targeted
`nix flake update <input>`, not a global update.
## Phase 7 — Validate
1. `nix flake check --no-build` (broad eval; reproduces CI's cheap checks).
2. `check-system <host>` on a representative box, and one exercising the stable channel if the
boxes mix channels. This must exercise the refreshed kernel constants on both channels.
3. Build the actual devshell with
`nix build --no-link --print-out-paths .#devShells.x86_64-linux.default`. Evaluation does not
build its dependencies, so it cannot catch packaging conflicts introduced by inputs such as
Determinate Nix.
4. After the evaluations pass, run `build-system <host>` for one representative NixOS box. Prefer
the local box when it is managed by this flake: its full closure is likely to exercise the most
relevant packages, home-manager configuration and upgraded kernel. Build only; do not switch.
5. Report eval/build results honestly. On failure, surface the error and stop rather than papering
over it.
## Wrap-up
Present a final summary: fork rebase outcome (patches kept/dropped/conflicted), whether a stable
bump is pending or was applied, kernel and release-metadata changes, the lock diff, version-gate
cleanup candidates, inputs updated, and validation results. Leave committing to the maintainer
unless asked; if committing, follow the repo's `area/scope: Capitalized summary` convention.
docs/openwrt-flash.md
-103
View File
@@ -1,103 +0,0 @@
# Flashing an OpenWrt box
Guided procedure for putting a flake-built OpenWrt image onto a box. The images themselves are
declared in [`openwrt/default.nix`](../openwrt/default.nix) and described in
[`deployment.md`](deployment.md#openwrt-images); the boxes are listed on their site pages (today
that is [fergal](sites/colony/fergal.md)).
Packages are baked into the image, so this runs whenever the package list changes — not only for
version upgrades. Work through the phases in order; ⏸ marks the point to stop and confirm.
## Phase 1 — Build
```sh
nix build .#openwrt-<box>
```
The result holds the `-squashfs-sysupgrade.bin` to flash, plus a `.manifest` listing every package
in the image and an SBOM. Check the manifest for the packages the change was meant to add — an
unknown package name is not an error at build time, it just silently isn't there.
## Phase 2 — Pre-flight
Confirm on the box:
```sh
grep -E 'RELEASE|REVISION' /etc/openwrt_release # what is running now
mount | grep -E ' / | /overlay | /rom ' # flash or RAM? (see below)
uci get network.lan.ipaddr # will it come back reachable?
cat /lib/upgrade/keep.d/* # what survives the flash
df -h /tmp # room for the image
```
**Flash or RAM matters.** A box booted normally shows a squashfs `/rom` plus a jffs2 `/overlay`;
one booted from an initramfs has `/` on tmpfs. The initramfs case has its own hazards — see
[Flashing from an initramfs](sites/colony/fergal.md#flashing-notes).
**Check the address is in UCI**, not just present on the interface. An address added by hand with
`ip` disappears on reboot and the box comes back unreachable.
`keep.d` normally lists `/etc/config/`, `/etc/dropbear/authorized_keys` and the dropbear host keys,
so an ordinary flash preserves both access and identity. Verify rather than assume — losing
`authorized_keys` on a box reachable only over SSH means a serial console recovery.
## Phase 3 — Back up
```sh
sysupgrade -b /tmp/<box>-config-backup.tar.gz
```
Fetch it with `ssh <box> 'cat /tmp/…' > local.tar.gz`. **`scp` does not work** — these boxes have no
`/usr/libexec/sftp-server`, so it fails with `Connection closed`. (`scp -O` forces the legacy
protocol if you prefer it.)
For a box being flashed off its **vendor** firmware for the first time, back up the whole flash
first — the vendor partitions hold per-unit MAC addresses and licence data that cannot be
regenerated. See [fergal's flash layout](sites/colony/fergal.md#flash-layout).
## Phase 4 — Stage and validate
```sh
ssh <box> 'cat > /tmp/sysupgrade.bin' < <image>.bin
ssh <box> 'sha256sum /tmp/sysupgrade.bin; sysupgrade -T /tmp/sysupgrade.bin'
```
Compare the sha256 against the local file, and require `sysupgrade -T` to exit 0. `-T` validates the
image and its device-compatibility metadata without writing anything, which is the last cheap chance
to catch a wrong-profile image.
## Phase 5 — Flash ⏸
Confirm before this point. It reboots the box and is not interruptible.
```sh
ssh <box> 'setsid sh -c "sleep 2; sysupgrade -v /tmp/sysupgrade.bin" \
</dev/null >/tmp/upgrade.log 2>&1 & echo detached'
```
**Detaching matters.** `sysupgrade` kills the SSH session partway through; without `setsid` the
upgrade dies with it, potentially after the flash has been erased. `nohup` is not available on these
boxes' busybox — use `setsid`.
Plain `sysupgrade` keeps the config in `keep.d`. Do not reach for `-c` out of caution: it needs
`/overlay/upper/etc` and aborts *after* erasing the firmware if that is missing.
## Phase 6 — Wait and verify
Poll SSH, not ping. A successful ping returns in milliseconds, so a naive "wait for it to go down"
loop finishes before the box has even started rebooting. Sleep between probes and wait on something
that only succeeds once userspace is up:
```sh
for i in $(seq 1 40); do
sleep 15
ssh -o ConnectTimeout=5 -o BatchMode=yes <box> 'grep REVISION /etc/openwrt_release' && break
done
```
Expect roughly three minutes. Then confirm the revision changed, the management address returned,
the package count matches the manifest, and the new packages are actually present and running.
Connecting without host-key overrides also confirms the host keys survived.
If the box does not return, it needs the serial console — have that confirmed as reachable *before*
Phase 5, not after.
docs/reference/dns.md
-319
View File
@@ -1,319 +0,0 @@
# DNS records
The tables on this page are generated from live authoritative zone transfers by
`nix run .#update-docs-dns -- <zones...>`; CI keeps them current. The Nix DNS configuration is
the source of truth. Edit the prose and headings here, but not content between the DNS markers.
DHCP-managed records are excluded. The generator identifies them by `DHCID` and removes the
corresponding forward and reverse records. SOA records and TTLs are also omitted because they are
operational metadata rather than useful inventory.
## Colony
These zones are served by [`estuary`](../sites/colony/estuary.md); their source configuration is
[`estuary/dns.nix`](../../nixos/boxes/colony/vms/estuary/dns.nix).
### Forward zone: `ams1.int.nul.ie`
<!-- dns: ams1.int.nul.ie -->
<!-- dns-start -->
| Name | Type | Value |
|---|---|---|
| `@` | `ALIAS` | `estuary-vm.ams1.int.nul.ie.` |
| `@` | `NS` | `ns.ams1.int.nul.ie.` |
| `_acme-challenge` | `TXT (LUA)` | `generated at query time` |
| `andrey-cust` | `A` | `94.142.242.254` |
| `chatterbox-ctr` | `A` | `10.100.2.5` |
| `chatterbox-ctr` | `AAAA` | `2a0e:97c0:4d2:12::5` |
| `colony` | `A` | `94.142.241.224` |
| `colony` | `AAAA` | `2a0e:97c0:4d2:10::2` |
| `colony-psql` | `CNAME` | `colony-psql-ctr.ams1.int.nul.ie.` |
| `colony-psql-ctr` | `A` | `10.100.2.4` |
| `colony-psql-ctr` | `AAAA` | `2a0e:97c0:4d2:12::4` |
| `colony-routing` | `A` | `10.100.0.2` |
| `colony-vms` | `A` | `10.100.1.1` |
| `colony-vms` | `AAAA` | `2a0e:97c0:4d2:11::1` |
| `ctr` | `CNAME` | `shill-vm.ams1.int.nul.ie.` |
| `darts-cust` | `A` | `94.142.242.255` |
| `darts-cust` | `AAAA` | `2a0e:97c0:4d2:2001::1` |
| `enshrouded` | `A` | `94.142.240.44` |
| `enshrouded-oci` | `A` | `10.100.3.5` |
| `enshrouded-oci` | `AAAA` | `2a0e:97c0:4d2:13::5` |
| `estuary-vm` | `A` | `94.142.240.44` |
| `estuary-vm` | `AAAA` | `2a02:898:0:20::329:1` |
| `estuary-vm-base` | `A` | `10.100.0.1` |
| `estuary-vm-base` | `AAAA` | `2a0e:97c0:4d2:10::1` |
| `fw` | `CNAME` | `estuary-vm.ams1.int.nul.ie.` |
| `gam-ctr` | `A` | `10.100.2.11` |
| `gam-ctr` | `AAAA` | `2a0e:97c0:4d2:12::b` |
| `git-vm` | `A` | `94.142.241.117` |
| `git-vm` | `AAAA` | `2a0e:97c0:4d2:11::4` |
| `git-vm-routing` | `A` | `10.100.1.4` |
| `graeme` | `A` | `94.142.240.44` |
| `graeme` | `AAAA` | `2a0e:97c0:4d2:13::8` |
| `graeme-oci` | `A` | `10.100.3.8` |
| `graeme-oci` | `AAAA` | `2a0e:97c0:4d2:13::8` |
| `hillcrest-tun` | `A` | `10.100.5.2` |
| `http` | `A` | `94.142.240.44` |
| `http` | `AAAA` | `2a0e:97c0:4d2:12::2` |
| `jackflix-ctr` | `A` | `10.100.2.6` |
| `jackflix-ctr` | `AAAA` | `2a0e:97c0:4d2:12::6` |
| `jam-cust` | `A` | `10.100.100.4` |
| `jam-cust` | `AAAA` | `2a0e:97c0:4d2:2002::1` |
| `jam-fwd` | `A` | `94.142.241.225` |
| `john-valorant-tun` | `A` | `10.100.5.6` |
| `kevcraft` | `A` | `94.142.240.44` |
| `kevcraft` | `AAAA` | `2a0e:97c0:4d2:13::6` |
| `kevcraft-oci` | `A` | `10.100.3.6` |
| `kevcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::6` |
| `kinkcraft` | `A` | `94.142.240.44` |
| `kinkcraft` | `AAAA` | `2a0e:97c0:4d2:13::7` |
| `kinkcraft-oci` | `A` | `10.100.3.7` |
| `kinkcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::7` |
| `librespeed` | `CNAME` | `http.ams1.int.nul.ie.` |
| `mail-vm` | `A` | `94.142.241.227` |
| `mail-vm` | `AAAA` | `2a0e:97c0:4d2:2000::1` |
| `middleman-ctr` | `A` | `10.100.2.2` |
| `middleman-ctr` | `AAAA` | `2a0e:97c0:4d2:12::2` |
| `ns` | `ALIAS` | `estuary-vm.ams1.int.nul.ie.` |
| `object-ctr` | `A` | `10.100.2.7` |
| `object-ctr` | `AAAA` | `2a0e:97c0:4d2:12::7` |
| `oci` | `CNAME` | `whale-vm.ams1.int.nul.ie.` |
| `qclk-ctr` | `A` | `10.100.2.10` |
| `qclk-ctr` | `AAAA` | `2a0e:97c0:4d2:12::a` |
| `shill-vm` | `A` | `94.142.241.225` |
| `shill-vm` | `AAAA` | `2a0e:97c0:4d2:11::2` |
| `shill-vm-ctrs` | `A` | `10.100.2.1` |
| `shill-vm-ctrs` | `AAAA` | `2a0e:97c0:4d2:12::1` |
| `shill-vm-routing` | `A` | `10.100.1.2` |
| `simpcraft` | `A` | `94.142.240.44` |
| `simpcraft` | `AAAA` | `2a0e:97c0:4d2:13::3` |
| `simpcraft-oci` | `A` | `10.100.3.3` |
| `simpcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::3` |
| `simpcraft-staging` | `A` | `94.142.240.44` |
| `simpcraft-staging` | `AAAA` | `2a0e:97c0:4d2:13::4` |
| `simpcraft-staging-oci` | `A` | `10.100.3.4` |
| `simpcraft-staging-oci` | `AAAA` | `2a0e:97c0:4d2:13::4` |
| `terraria` | `A` | `94.142.240.44` |
| `terraria` | `AAAA` | `2a0e:97c0:4d2:12::b` |
| `toot-ctr` | `A` | `10.100.2.8` |
| `toot-ctr` | `AAAA` | `2a0e:97c0:4d2:12::8` |
| `valheim` | `A` | `94.142.240.44` |
| `valheim` | `AAAA` | `2a0e:97c0:4d2:13::2` |
| `valheim-oci` | `A` | `10.100.3.2` |
| `valheim-oci` | `AAAA` | `2a0e:97c0:4d2:13::2` |
| `vaultwarden-ctr` | `A` | `10.100.2.3` |
| `vaultwarden-ctr` | `AAAA` | `2a0e:97c0:4d2:12::3` |
| `vm` | `CNAME` | `colony.ams1.int.nul.ie.` |
| `waffletail-ctr` | `A` | `10.100.2.9` |
| `waffletail-ctr` | `AAAA` | `2a0e:97c0:4d2:12::9` |
| `whale-vm` | `A` | `94.142.241.226` |
| `whale-vm` | `AAAA` | `2a0e:97c0:4d2:11::3` |
| `whale-vm-routing` | `A` | `10.100.1.3` |
<!-- dns-end -->
### IPv4 reverse zone: `100.10.in-addr.arpa`
<!-- dns: 100.10.in-addr.arpa -->
<!-- dns-start -->
| Address | Name |
|---|---|
| `10.100.0.1` | `estuary-vm-base.ams1.int.nul.ie.` |
| `10.100.0.2` | `colony-routing.ams1.int.nul.ie.` |
| `10.100.1.1` | `colony-vms.ams1.int.nul.ie.` |
| `10.100.1.2` | `shill-vm-routing.ams1.int.nul.ie.` |
| `10.100.1.3` | `whale-vm-routing.ams1.int.nul.ie.` |
| `10.100.1.4` | `git-vm-routing.ams1.int.nul.ie.` |
| `10.100.2.1` | `shill-vm-ctrs.ams1.int.nul.ie.` |
| `10.100.2.2` | `middleman-ctr.ams1.int.nul.ie.` |
| `10.100.2.3` | `vaultwarden-ctr.ams1.int.nul.ie.` |
| `10.100.2.4` | `colony-psql-ctr.ams1.int.nul.ie.` |
| `10.100.2.5` | `chatterbox-ctr.ams1.int.nul.ie.` |
| `10.100.2.6` | `jackflix-ctr.ams1.int.nul.ie.` |
| `10.100.2.7` | `object-ctr.ams1.int.nul.ie.` |
| `10.100.2.8` | `toot-ctr.ams1.int.nul.ie.` |
| `10.100.2.9` | `waffletail-ctr.ams1.int.nul.ie.` |
| `10.100.2.10` | `qclk-ctr.ams1.int.nul.ie.` |
| `10.100.2.11` | `gam-ctr.ams1.int.nul.ie.` |
| `10.100.3.2` | `valheim-oci.ams1.int.nul.ie.` |
| `10.100.3.3` | `simpcraft-oci.ams1.int.nul.ie.` |
| `10.100.3.4` | `simpcraft-staging-oci.ams1.int.nul.ie.` |
| `10.100.3.5` | `enshrouded-oci.ams1.int.nul.ie.` |
| `10.100.3.6` | `kevcraft-oci.ams1.int.nul.ie.` |
| `10.100.3.7` | `kinkcraft-oci.ams1.int.nul.ie.` |
| `10.100.3.8` | `graeme-oci.ams1.int.nul.ie.` |
<!-- dns-end -->
### IPv6 reverse zone: `2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa`
<!-- dns: 2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa -->
<!-- dns-start -->
| Address | Name |
|---|---|
| `2a0e:97c0:4d2:10::1` | `estuary-vm-base.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:10::2` | `colony.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:11::1` | `colony-vms.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:11::2` | `shill-vm.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:11::3` | `whale-vm.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:11::4` | `git-vm.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::1` | `shill-vm-ctrs.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::2` | `middleman-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::3` | `vaultwarden-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::4` | `colony-psql-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::5` | `chatterbox-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::6` | `jackflix-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::7` | `object-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::8` | `toot-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::9` | `waffletail-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::a` | `qclk-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:12::b` | `gam-ctr.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::2` | `valheim-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::3` | `simpcraft-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::4` | `simpcraft-staging-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::5` | `enshrouded-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::6` | `kevcraft-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::7` | `kinkcraft-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:13::8` | `graeme-oci.ams1.int.nul.ie.` |
| `2a0e:97c0:4d2:2000::1` | `mail.nul.ie.` |
| `2a0e:97c0:4d2:2001::1` | `darts-cust.ams1.int.nul.ie.` |
<!-- dns-end -->
## Home
These zones are served by [`river`](../sites/home/river.md) and
[`stream`](../sites/home/stream.md); their shared source configuration is
[`routing-common/dns.nix`](../../nixos/boxes/home/routing-common/dns.nix).
### Forward zone: `h.nul.ie`
<!-- dns: h.nul.ie -->
<!-- dns-start -->
| Name | Type | Value |
|---|---|---|
| `@` | `NS` | `ns1.h.nul.ie.` |
| `@` | `NS` | `ns2.h.nul.ie.` |
| `boot` | `CNAME` | `river-hi.h.nul.ie.` |
| `brian` | `A` | `192.168.64.13` |
| `castle` | `A` | `192.168.68.40` |
| `castle` | `AAAA` | `2a0e:97c0:4d0:1::3:1` |
| `cellar` | `A` | `192.168.68.80` |
| `cellar` | `AAAA` | `2a0e:97c0:4d0:1::4:1` |
| `dave` | `A` | `192.168.68.11` |
| `dave` | `AAAA` | `2a0e:97c0:4d0:1::1:2` |
| `dave-core` | `A` | `192.168.64.11` |
| `dave-lo` | `A` | `192.168.72.11` |
| `dave-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:2` |
| `dyn` | `NS` | `ns1.dyn.h.nul.ie.` |
| `dyn` | `NS` | `ns2.dyn.h.nul.ie.` |
| `frigate` | `CNAME` | `hass-ctr.h.nul.ie.` |
| `hass-ctr` | `A` | `192.168.68.103` |
| `hass-ctr` | `AAAA` | `2a0e:97c0:4d0:1::5:3` |
| `hass-ctr-lo` | `A` | `192.168.72.103` |
| `hass-ctr-lo` | `AAAA` | `2a0e:97c0:4d0:2::5:3` |
| `jim` | `A` | `192.168.68.10` |
| `jim` | `AAAA` | `2a0e:97c0:4d0:1::1:1` |
| `jim-core` | `A` | `192.168.64.10` |
| `jim-lo` | `A` | `192.168.72.10` |
| `jim-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:1` |
| `nixlight` | `A` | `192.168.72.46` |
| `ns1` | `ALIAS` | `river.h.nul.ie.` |
| `ns1.dyn` | `ALIAS` | `river.h.nul.ie.` |
| `ns2` | `ALIAS` | `stream.h.nul.ie.` |
| `ns2.dyn` | `ALIAS` | `stream.h.nul.ie.` |
| `palace` | `A` | `192.168.68.22` |
| `palace` | `AAAA` | `2a0e:97c0:4d0:1::2:1` |
| `palace-core` | `A` | `192.168.64.20` |
| `palace-kvm` | `A` | `192.168.72.21` |
| `reolink-living-room` | `A` | `192.168.72.45` |
| `river` | `A (LUA)` | `generated at query time` |
| `river` | `AAAA` | `2a0e:97c0:4df:0:1::1` |
| `river-core` | `A` | `192.168.64.1` |
| `river-hi` | `A` | `192.168.68.1` |
| `river-hi` | `AAAA` | `2a0e:97c0:4d0:1::1` |
| `river-lo` | `A` | `192.168.72.1` |
| `river-lo` | `AAAA` | `2a0e:97c0:4d0:2::1` |
| `river-ut` | `A` | `192.168.80.1` |
| `river-ut` | `AAAA` | `2a0e:97c0:4d0:3::1` |
| `router-hi` | `A` | `192.168.71.254` |
| `router-hi` | `AAAA` | `2a0e:97c0:4d0:1::ffff` |
| `router-lo` | `A` | `192.168.79.254` |
| `router-lo` | `AAAA` | `2a0e:97c0:4d0:2::ffff` |
| `router-ut` | `A` | `192.168.80.254` |
| `router-ut` | `AAAA` | `2a0e:97c0:4d0:3::ffff` |
| `sfh` | `A` | `192.168.68.81` |
| `sfh` | `AAAA` | `2a0e:97c0:4d0:1::4:2` |
| `shytzel` | `A` | `192.168.64.12` |
| `stream` | `A (LUA)` | `generated at query time` |
| `stream` | `AAAA` | `2a0e:97c0:4df:0:1::2` |
| `stream-core` | `A` | `192.168.64.2` |
| `stream-hi` | `A` | `192.168.68.2` |
| `stream-hi` | `AAAA` | `2a0e:97c0:4d0:1::2` |
| `stream-lo` | `A` | `192.168.72.2` |
| `stream-lo` | `AAAA` | `2a0e:97c0:4d0:2::2` |
| `stream-ut` | `A` | `192.168.80.2` |
| `stream-ut` | `AAAA` | `2a0e:97c0:4d0:3::2` |
| `unifi-ctr` | `A` | `192.168.68.100` |
| `unifi-ctr` | `AAAA` | `2a0e:97c0:4d0:1::5:1` |
| `unifi-ctr-core` | `A` | `192.168.64.21` |
| `ups` | `A` | `192.168.72.20` |
| `vibe` | `A` | `192.168.68.15` |
| `vibe` | `AAAA` | `2a0e:97c0:4d0:1::1:6` |
| `vibe-core` | `A` | `192.168.64.15` |
| `vibe-lo` | `A` | `192.168.72.15` |
| `vibe-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:6` |
| `wave` | `A` | `192.168.72.14` |
| `wave` | `AAAA` | `2a0e:97c0:4d0:2::1:5` |
| `wave-core` | `A` | `192.168.64.14` |
<!-- dns-end -->
### IPv4 reverse zone: `168.192.in-addr.arpa`
<!-- dns: 168.192.in-addr.arpa -->
<!-- dns-start -->
| Address | Name |
|---|---|
| `192.168.64.1` | `river-core.h.nul.ie.` |
| `192.168.64.2` | `stream-core.h.nul.ie.` |
| `192.168.64.20` | `palace-core.h.nul.ie.` |
| `192.168.64.21` | `unifi-ctr-core.h.nul.ie.` |
| `192.168.68.1` | `river-hi.h.nul.ie.` |
| `192.168.68.2` | `stream-hi.h.nul.ie.` |
| `192.168.68.22` | `palace.h.nul.ie.` |
| `192.168.68.40` | `castle.h.nul.ie.` |
| `192.168.68.80` | `cellar.h.nul.ie.` |
| `192.168.68.81` | `sfh.h.nul.ie.` |
| `192.168.68.100` | `unifi-ctr.h.nul.ie.` |
| `192.168.68.103` | `hass-ctr.h.nul.ie.` |
| `192.168.71.254` | `router-hi.h.nul.ie.` |
| `192.168.72.1` | `river-lo.h.nul.ie.` |
| `192.168.72.2` | `stream-lo.h.nul.ie.` |
| `192.168.72.103` | `hass-ctr-lo.h.nul.ie.` |
| `192.168.79.254` | `router-lo.h.nul.ie.` |
| `192.168.80.1` | `river-ut.h.nul.ie.` |
| `192.168.80.2` | `stream-ut.h.nul.ie.` |
| `192.168.80.254` | `router-ut.h.nul.ie.` |
<!-- dns-end -->
### IPv6 reverse zone: `0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa`
<!-- dns: 0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa -->
<!-- dns-start -->
| Address | Name |
|---|---|
| `2a0e:97c0:4d0:1::1` | `river-hi.h.nul.ie.` |
| `2a0e:97c0:4d0:1::2` | `stream-hi.h.nul.ie.` |
| `2a0e:97c0:4d0:1::ffff` | `router-hi.h.nul.ie.` |
| `2a0e:97c0:4d0:1::2:1` | `palace.h.nul.ie.` |
| `2a0e:97c0:4d0:1::3:1` | `castle.h.nul.ie.` |
| `2a0e:97c0:4d0:1::4:1` | `cellar.h.nul.ie.` |
| `2a0e:97c0:4d0:1::4:2` | `sfh.h.nul.ie.` |
| `2a0e:97c0:4d0:1::5:1` | `unifi-ctr.h.nul.ie.` |
| `2a0e:97c0:4d0:1::5:3` | `hass-ctr.h.nul.ie.` |
| `2a0e:97c0:4d0:2::1` | `river-lo.h.nul.ie.` |
| `2a0e:97c0:4d0:2::2` | `stream-lo.h.nul.ie.` |
| `2a0e:97c0:4d0:2::ffff` | `router-lo.h.nul.ie.` |
| `2a0e:97c0:4d0:2::5:3` | `hass-ctr-lo.h.nul.ie.` |
| `2a0e:97c0:4d0:3::1` | `river-ut.h.nul.ie.` |
| `2a0e:97c0:4d0:3::2` | `stream-ut.h.nul.ie.` |
| `2a0e:97c0:4d0:3::ffff` | `router-ut.h.nul.ie.` |
<!-- dns-end -->
docs/reference/nixos-options.md
-286
View File
@@ -1,286 +0,0 @@
# NixOS module options (`my.*`)
> **Generated** from the module option descriptions by `nix run .#update-docs-options`;
> CI keeps it current. The source of truth is the `mkOpt'` / `mkBoolOpt'` declarations in
> `nixos/modules/` — edit those, not this file. For what each module is *for*, see the
> [shared-modules overview](../architecture.md#shared-modules).
## `borgthin` — [`nixos/modules/borgthin.nix`](../../nixos/modules/borgthin.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.borgthin.enable` | boolean | `false` | Whether to enable borgthin jobs |
| `my.borgthin.jobs` | attribute set of (submodule) | `{ }` | borgthin jobs |
| `my.borgthin.jobs.<name>.archivePrefix` | string | `"${config.networking.hostName}-${name}-"` | Prefix to start new archives with |
| `my.borgthin.jobs.<name>.compression` | string | `"zstd,3"` | Compression options |
| `my.borgthin.jobs.<name>.dateFormat` | string | `"+%Y-%m-%dT%H:%M:%S"` | Format passed to the date command |
| `my.borgthin.jobs.<name>.environment` | attribute set of string | `{ }` | Extra environment variables to pass to borg |
| `my.borgthin.jobs.<name>.extraArgs` | list of string | `[ "--iec" ]` | Extra args to pass to all borg commands |
| `my.borgthin.jobs.<name>.extraCreateArgs` | list of string | `[ ]` | Extra args to pass to tcreate command |
| `my.borgthin.jobs.<name>.lvs` | list of string | `null` | Thin LVs to backup (vg/lv format) |
| `my.borgthin.jobs.<name>.passFile` | null or string | `null` | Path to file containing passphrase |
| `my.borgthin.jobs.<name>.prune.keep` | attribute set of (signed integer or string) | `{ }` | Borg pruning params |
| `my.borgthin.jobs.<name>.prune.pattern` | string | `"sh:${config.archivePrefix}*"` | Borg pattern to select archives for pruning |
| `my.borgthin.jobs.<name>.repo` | string | `null` | borg repository URL |
| `my.borgthin.jobs.<name>.timer.at` | string or list of string | `"5:00"` | systemd calendar time(s) to run backup at |
| `my.borgthin.jobs.<name>.timer.persistent` | boolean | `false` | Persistent systemd timer |
| `my.borgthin.lvmPackage` | package | `<derivation lvm2-2.03.41>` | Packge containing LVM tools |
| `my.borgthin.package` | package | `inputs.borgthin.packages.${system}.borgthin` | borgthin package |
| `my.borgthin.thinToolsPackage` | package | `<derivation thin-provisioning-tools-1.3.2>` | Package containing thin-provisioning-tools |
## `build` — [`nixos/modules/build.nix`](../../nixos/modules/build.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.build.allHardware` | boolean | `false` | Whether to enable a lot of firmware and kernel modules for a wide range of hardware.Only applies to some build targets. |
| `my.build.isDevVM` | boolean | `false` | Whether the system is a development VM. |
| `my.buildAs` | open submodule of lazy attribute set of unspecified value | `{ }` | Attribute set of derivations used to set up the system. |
## `common` — [`nixos/modules/common.nix`](../../nixos/modules/common.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.tmproot.persistence.config.users` | attribute set of (submodule) | `{ }` | A set of user submodules listing the files and directories to link to their respective user's home directories. Each attribute name should be the name of the user. For detailed usage, check the [documentation](https://github.com/nix-community/impermanence). |
## `containers` — [`nixos/modules/containers.nix`](../../nixos/modules/containers.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.containers.instances` | attribute set of (submodule) | `{ }` | Individual containers. |
| `my.containers.instances.<name>.autoStart` | boolean | `true` | Whether to start the container automatically at boot. |
| `my.containers.instances.<name>.bindMounts` | attribute set of (submodule) | `{ }` | An extra list of directories that is bound to the container. |
| `my.containers.instances.<name>.bindMounts.<name>.hostPath` | null or string | `"‹name›"` | Location of the host path to be mounted. |
| `my.containers.instances.<name>.bindMounts.<name>.mountPoint` | string | `"‹name›"` | Mount point on the container file system. |
| `my.containers.instances.<name>.bindMounts.<name>.readOnly` | boolean | `true` | Determine whether the mounted path will be accessed in read-only mode. |
| `my.containers.instances.<name>.containerSystem` | absolute path | `"/nix/var/nix/profiles/system"` | Path to NixOS system configuration from within container. |
| `my.containers.instances.<name>.hotReload` | boolean | `true` | Whether to apply new configuration by running `switch-to-configuration` instead of rebooting the container. |
| `my.containers.instances.<name>.networking.bridge` | null or string | `null` | Network bridge to connect to. |
| `my.containers.instances.<name>.networking.macVLAN` | null or string | `null` | Network interface to make MACVLAN interface from. |
| `my.containers.instances.<name>.system` | absolute path | `"/nix/var/nix/profiles/per-container/‹name›/system"` | Path to NixOS system configuration. |
| `my.containers.persistDir` | string | `"/persist/containers"` | Where to store container persistence data. |
## `deploy-rs` — [`nixos/modules/deploy-rs.nix`](../../nixos/modules/deploy-rs.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.deploy.authorizedKeys.keyFiles` | list of absolute path | `[ .keys/deploy.pub ]` | SSH public key files to add to the default deployment user. |
| `my.deploy.authorizedKeys.keys` | list of (optionally newline-terminated) single-line string | `[ ]` | SSH public keys to add to the default deployment user. |
| `my.deploy.enable` | boolean | `true` | Whether to expose deploy-rs configuration for this system. |
| `my.deploy.generate.containers.enable` | boolean | `true` | Whether to generate deploy-rs profiles for this system's containers. |
| `my.deploy.generate.containers.keepGenerations` | unsigned integer, meaning >=0 | `10` | Number of generations to keep when cleaning up old deployments (0 to disable deletion on deployment). |
| `my.deploy.generate.system.enable` | boolean | `true` | Whether to generate a deploy-rs profile for this system's config. |
| `my.deploy.generate.system.keepGenerations` | unsigned integer, meaning >=0 | `10` | Number of generations to keep when cleaning up old deployments (0 to disable deletion on deployment). |
| `my.deploy.generate.system.mode` | string | `"switch"` | switch-to-configuration mode. |
| `my.deploy.node` | submodule | `{ }` | deploy-rs node configuration. |
| `my.deploy.node.autoRollback` | null or boolean | `null` | Whether to roll back the profile if activation fails. |
| `my.deploy.node.confirmTimeout` | null or 16 bit unsigned integer; between 0 and 65535 (both inclusive) | `null` | Timeout for confirming activation succeeded. |
| `my.deploy.node.fastConnection` | null or boolean | `null` | Whether to copy the whole closure instead of using substitution. |
| `my.deploy.node.hostname` | string | `""` | Hostname deploy-rs will connect to. |
| `my.deploy.node.magicRollback` | null or boolean | `null` | Whether to roll back the profile if connectivity to the deployer is lost. |
| `my.deploy.node.profiles` | attribute set of (submodule) | `{ }` | Profiles to deploy. |
| `my.deploy.node.profiles.<name>.autoRollback` | null or boolean | `null` | Whether to roll back the profile if activation fails. |
| `my.deploy.node.profiles.<name>.confirmTimeout` | null or 16 bit unsigned integer; between 0 and 65535 (both inclusive) | `null` | Timeout for confirming activation succeeded. |
| `my.deploy.node.profiles.<name>.fastConnection` | null or boolean | `null` | Whether to copy the whole closure instead of using substitution. |
| `my.deploy.node.profiles.<name>.magicRollback` | null or boolean | `null` | Whether to roll back the profile if connectivity to the deployer is lost. |
| `my.deploy.node.profiles.<name>.path` | package | `""` | Derivation to build (should include activation script). |
| `my.deploy.node.profiles.<name>.profilePath` | null or string | `null` | Path to profile location |
| `my.deploy.node.profiles.<name>.sshOpts` | list of string | `[ ]` | Options deploy-rs will pass to ssh. Note: overriding at a lower level _merges_ options. |
| `my.deploy.node.profiles.<name>.sshUser` | null or string | `null` | Username deploy-rs will deploy with. |
| `my.deploy.node.profiles.<name>.sudo` | null or string | `null` | Command to elevate privileges with (used if the deployment user != profile user). |
| `my.deploy.node.profiles.<name>.tempPath` | null or string | `null` | Path that deploy-rs will use for temporary files. |
| `my.deploy.node.profiles.<name>.user` | null or string | `null` | Username deploy-rs will deploy with. |
| `my.deploy.node.profilesOrder` | null or (list of string) | `null` | Order to deploy profiles in (remainder will be deployed in arbitrary order). |
| `my.deploy.node.sshOpts` | list of string | `[ ]` | Options deploy-rs will pass to ssh. Note: overriding at a lower level _merges_ options. |
| `my.deploy.node.sshUser` | null or string | `null` | Username deploy-rs will deploy with. |
| `my.deploy.node.sudo` | null or string | `null` | Command to elevate privileges with (used if the deployment user != profile user). |
| `my.deploy.node.tempPath` | null or string | `null` | Path that deploy-rs will use for temporary files. |
| `my.deploy.node.user` | null or string | `null` | Username deploy-rs will deploy with. |
## `dynamic-motd` — [`nixos/modules/dynamic-motd.nix`](../../nixos/modules/dynamic-motd.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.dynamic-motd.enable` | boolean | `true` | Whether to enable the dynamic message of the day PAM module. |
| `my.dynamic-motd.script` | null or strings concatenated with "\n" | `null` | Script that generates message of the day. |
| `my.dynamic-motd.services` | list of string | `[ "login" "sshd" ]` | PAM services to enable the dynamic message of the day module for. |
## `firewall` — [`nixos/modules/firewall.nix`](../../nixos/modules/firewall.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.firewall.enable` | boolean | `true` | Whether to enable the nftables-based firewall. |
| `my.firewall.extraRules` | strings concatenated with "\n" | `""` | Arbitrary additional nftables rules. |
| `my.firewall.nat.enable` | boolean | `true` | Whether to enable IP forwarding and NAT. |
| `my.firewall.nat.externalInterface` | null or string | `null` | The name of the external network interface. |
| `my.firewall.nat.forwardPorts` | (list of (submodule)) or attribute set of list of (submodule) | `[ ]` | IPv4 port forwards |
| `my.firewall.tcp.allowed` | list of (16 bit unsigned integer; between 0 and 65535 (both inclusive) or string) | `[ ]` | TCP ports to open. |
| `my.firewall.trustedInterfaces` | list of string | `[ ]` | Traffic coming in from these interfaces will be accepted unconditionally. Traffic from the loopback (lo) interface will always be accepted. |
| `my.firewall.udp.allowTraceroute` | boolean | `true` | Whethor or not to add a rule to accept UDP traceroute packets. |
| `my.firewall.udp.allowed` | list of (16 bit unsigned integer; between 0 and 65535 (both inclusive) or string) | `[ ]` | UDP ports to open. |
## `gui` — [`nixos/modules/gui`](../../nixos/modules/gui)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.gui.enable` | boolean | `true` | Whether to enable GUI system options. |
## `l2mesh` — [`nixos/modules/l2mesh.nix`](../../nixos/modules/l2mesh.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.vpns.l2.pskFiles` | attribute set of string | `{ }` | PSK files for secured meshes. |
## `librespeed` — [`nixos/modules/librespeed`](../../nixos/modules/librespeed)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.librespeed.backend.enable` | boolean | `false` | Whether to enable librespeed backend. |
| `my.librespeed.backend.extraSettingsFile` | null or string | `null` | Extra settings file. |
| `my.librespeed.backend.settings` | attribute set of unspecified value | `{ }` | Backend settings. |
| `my.librespeed.frontend.servers` | list of attribute set of unspecified value | `{ }` | Server configs. |
| `my.librespeed.frontend.webroot` | package | — | Frontend webroot. |
## `netboot` — [`nixos/modules/netboot`](../../nixos/modules/netboot)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.netboot.client.configurationLimit` | unsigned integer, meaning >=0 | `10` | Max generations to show in boot menu. |
| `my.netboot.client.enable` | boolean | `false` | Whether network booting should be enabled. |
| `my.netboot.server.allowedPrefixes` | list of string | `null` | Prefixes clients should be allowed to connect from (NFS). |
| `my.netboot.server.enable` | boolean | `false` | Whether a netboot server should be enabled. |
| `my.netboot.server.host` | string | `config.networking.fqdn` | Hostname clients should connect to over HTTP / NFS. |
| `my.netboot.server.installer.storeSize` | string | `"16GiB"` | Total allowed writable size of store. |
| `my.netboot.server.instances` | list of string | `[ ]` | Systems to hold boot files for. |
| `my.netboot.server.ip` | string | `null` | IP clients should connect to via TFTP. |
## `nginx-sso` — [`nixos/modules/nginx-sso.nix`](../../nixos/modules/nginx-sso.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.nginx-sso.configuration` | attribute set of unspecified value | `{ }` | nginx-sso configuration. |
| `my.nginx-sso.enable` | boolean | `false` | Whether to enable custom nginx-sso. |
| `my.nginx-sso.extraConfigFile` | null or string | `null` | Path to configuration (e.g. for secrets). |
| `my.nginx-sso.includes.baseURL` | string | `null` | Base URL for redirects. |
| `my.nginx-sso.includes.endpoint` | string | `"http://localhost:8082"` | Upstream for proxied auth requests. |
| `my.nginx-sso.includes.instances` | attribute set of (submodule) | `{ }` | nginx includes instances. |
| `my.nginx-sso.includes.instances.<name>.auth.path` | string | `"/sso-auth"` | HTTP path for SSO auth. |
| `my.nginx-sso.includes.instances.<name>.auth.redirect` | string | `"$scheme://$http_host$request_uri"` | URL to redirect to upon successful login. |
| `my.nginx-sso.includes.instances.<name>.logout.path` | string | `"/sso-logout"` | HTTP path for SSO logout. |
| `my.nginx-sso.includes.instances.<name>.logout.redirect` | string | `"$scheme://$http_host/"` | URL to redirect to upon successful logout. |
| `my.nginx-sso.package` | package | `<derivation nginx-sso-0.27.8>` | nginx-sso package to use. |
## `nvme` — [`nixos/modules/nvme`](../../nixos/modules/nvme)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.nvme.boot.address` | string | `null` | Address of NVMe-oF target. |
| `my.nvme.boot.nqn` | null or string | `null` | NQN to connect to on boot |
| `my.nvme.uuid` | null or string | `null` | NVMe host ID |
## `pdns` — [`nixos/modules/pdns.nix`](../../nixos/modules/pdns.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.pdns.auth.bind.file-records.sshKey` | null or string | `null` | SSH public key for file record update user. |
| `my.pdns.auth.bind.options.also-notify` | list of string | `[ ]` | List of additional address to send DNS NOTIFY messages to. |
| `my.pdns.auth.bind.zones` | attribute set of (submodule) | `{ }` | BIND-style zones definitions. |
| `my.pdns.auth.bind.zones.<name>.also-notify` | list of string | `[ ]` | List of additional address to send DNS NOTIFY messages to. |
| `my.pdns.auth.bind.zones.<name>.masters` | list of string | `[ ]` | List of masters to retrieve data from (as slave). |
| `my.pdns.auth.bind.zones.<name>.path` | absolute path | `null` | Path to zone file. |
| `my.pdns.auth.bind.zones.<name>.template` | boolean | `true` | Whether to run the zone contents through a template for post-processing. |
| `my.pdns.auth.bind.zones.<name>.text` | null or strings concatenated with "\n" | `null` | Inline content of the zone file. |
| `my.pdns.auth.bind.zones.<name>.type` | one of "master", "slave", "native" | `"native"` | Zone type. |
| `my.pdns.auth.enable` | boolean | `false` | Whether to enable PowerDNS authoritative nameserver. |
| `my.pdns.auth.extraSettingsFile` | null or string | `null` | Path to extra settings (e.g. for secrets). |
| `my.pdns.auth.settings` | attribute set of (null or signed integer or string or boolean or absolute path or list of (signed integer or string or boolean or absolute path)) | `{ }` | Authoritative server settings. |
| `my.pdns.recursor.enable` | boolean | `false` | Whether to enable PowerDNS recursive nameserver. |
| `my.pdns.recursor.extraSettingsFile` | null or string | `null` | Path to extra settings (e.g. for secrets). |
## `secrets` — [`nixos/modules/secrets.nix`](../../nixos/modules/secrets.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.secrets.files` | attribute set of unspecified value | `{ }` | Secrets to decrypt with agenix. |
| `my.secrets.key` | null or string | `null` | Public key that secrets for this system should be encrypted for. |
| `my.secrets.vmKeyPath` | string | `"/tmp/xchg/dev.key"` | Path to dev key when in a dev VM. |
## `server` — [`nixos/modules/server.nix`](../../nixos/modules/server.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.server.enable` | boolean | `false` | Whether to enable common configuration for servers. |
## `spdk` — [`nixos/modules/spdk.nix`](../../nixos/modules/spdk.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.spdk.config.subsystems` | attribute set of list of (submodule) | `{ }` | Subsystem config / RPCs. |
| `my.spdk.config.subsystems.<name>.*.method` | string | `null` | RPC method name. |
| `my.spdk.config.subsystems.<name>.*.params` | attribute set of unspecified value | `{ }` | RPC params |
| `my.spdk.debugCommands` | strings concatenated with "\n" | `""` | Commands to run with the spdk-debug script. |
| `my.spdk.enable` | boolean | `false` | Whether to enable SPDK target. |
| `my.spdk.extraArgs` | string | `""` | Extra arguments to pass to spdk_tgt. |
## `tmproot` — [`nixos/modules/tmproot.nix`](../../nixos/modules/tmproot.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.tmproot.enable` | boolean | `true` | Whether to enable tmproot. |
| `my.tmproot.persistence.config` | submodule | `{ }` | Persistence configuration |
| `my.tmproot.persistence.dir` | null or string | `"/persist"` | Path where persisted files are stored. |
| `my.tmproot.size` | string | `"2G"` | Size of tmpfs root |
| `my.tmproot.unsaved.ignore` | list of string | `[ ]` | Path prefixes to ignore if unsaved. |
| `my.tmproot.unsaved.showMotd` | boolean | `true` | Whether to show unsaved files with `dynamic-motd`. |
## `user` — [`nixos/modules/user.nix`](../../nixos/modules/user.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.user.config` | submodule | `{ }` | User definition (as `users.users.*`). |
| `my.user.enable` | boolean | `true` | Whether to create a primary user. |
| `my.user.homeConfig` | Home Manager module | `{ }` | Home configuration (as `home-manager.users.*`) |
| `my.user.passwordSecret` | null or string | `"user-passwd.txt"` | Name of user password secret. |
| `my.user.tmphome` | boolean | `true` | Whether to persist home directory files under tmproot |
## `vms` — [`nixos/modules/vms.nix`](../../nixos/modules/vms.nix)
| Option | Type | Default | Description |
|---|---|---|---|
| `my.vms.instances` | attribute set of (submodule) | `{ }` | VM instances. |
| `my.vms.instances.<name>.autoStart` | boolean | `true` | Whether to start the VM automatically at boot. |
| `my.vms.instances.<name>.boot` | string or (attribute set of unspecified value) convertible to it | `{ menu = "on"; splash-time = 5000; }` | Boot options. |
| `my.vms.instances.<name>.cleanShutdown.enabled` | boolean | `true` | Whether to attempt to cleanly shut down the guest. |
| `my.vms.instances.<name>.cleanShutdown.timeout` | unsigned integer, meaning >=0 | `30` | Clean shutdown timeout (in seconds). |
| `my.vms.instances.<name>.cpu` | string | `"host"` | QEMU CPU model. |
| `my.vms.instances.<name>.drives` | list of (submodule) | `[ ]` | Drives to attach to VM. |
| `my.vms.instances.<name>.drives.*.backend` | string or (attribute set of unspecified value) convertible to it | `{ }` | Backend blockdev options. |
| `my.vms.instances.<name>.drives.*.format` | string or (attribute set of unspecified value) convertible to it | `{ }` | Format blockdev options. |
| `my.vms.instances.<name>.drives.*.formatBackendProp` | string | `"file"` | Property that references the backend blockdev. |
| `my.vms.instances.<name>.drives.*.frontend` | string | `"virtio-blk"` | Frontend device driver. |
| `my.vms.instances.<name>.drives.*.frontendOpts` | string or (attribute set of unspecified value) convertible to it | `{ }` | Frontend device options. |
| `my.vms.instances.<name>.drives.*.name` | string | `null` | Drive name. |
| `my.vms.instances.<name>.enableKVM` | boolean | `true` | Whether to enable KVM. |
| `my.vms.instances.<name>.enableUEFI` | boolean | `true` | Whether to enable UEFI. |
| `my.vms.instances.<name>.hostDevices` | attribute set of (submodule) | `{ }` | Host PCI devices to pass to the VM. |
| `my.vms.instances.<name>.hostDevices.<name>.bindVFIO` | boolean | `true` | Whether to automatically bind the device to vfio-pci. |
| `my.vms.instances.<name>.hostDevices.<name>.extraOptions` | string or (attribute set of unspecified value) convertible to it | `{ }` | Extra QEMU options for the vfio-pci QEMU device. |
| `my.vms.instances.<name>.hostDevices.<name>.hostBDF` | string | `null` | PCI BDF of host device. |
| `my.vms.instances.<name>.hostDevices.<name>.index` | unsigned integer, meaning >=0 | `null` | Index of device in guest (for root port chassis and slot). |
| `my.vms.instances.<name>.machine` | string | `"q35"` | QEMU machine type. |
| `my.vms.instances.<name>.memory` | unsigned integer, meaning >=0 | `1024` | Amount of RAM (mebibytes). |
| `my.vms.instances.<name>.networks` | attribute set of (submodule) | `{ }` | Networks to attach VM to. |
| `my.vms.instances.<name>.networks.<name>.bridge` | null or string | `"‹name›"` | Network bridge to connect to (null to not attach to bridge). |
| `my.vms.instances.<name>.networks.<name>.extraOptions` | string or (attribute set of unspecified value) convertible to it | `{ }` | Extra QEMU options to set for the NIC. |
| `my.vms.instances.<name>.networks.<name>.ifname` | string | `"vm-‹name›"` | TAP device to create / use. |
| `my.vms.instances.<name>.networks.<name>.mac` | string | `null` | Guest MAC address. |
| `my.vms.instances.<name>.networks.<name>.model` | string | `"virtio-net"` | Device type for network interface. |
| `my.vms.instances.<name>.networks.<name>.tapFD` | null or (unsigned integer, meaning >=0) | `null` | FD to use to pass existing TAP device. |
| `my.vms.instances.<name>.networks.<name>.waitOnline` | boolean or string | `true` | Whether to wait for networkd to consider the bridge / existing TAP device online. Pass a string to set the OPERSTATE will wait for. |
| `my.vms.instances.<name>.qemuBin` | absolute path | `"/nix/store/2lkr0v29a67jybn8ckjawpg08y0yizfp-qemu-host-cpu-only-11.1.0/bin/qemu-kvm"` | Path to QEMU executable. |
| `my.vms.instances.<name>.qemuFlags` | list of string | `[ ]` | Additional flags to pass to QEMU. |
| `my.vms.instances.<name>.smp.cpus` | unsigned integer, meaning >=0 | `1` | Number of CPU cores. |
| `my.vms.instances.<name>.smp.threads` | unsigned integer, meaning >=0 | `1` | Number of threads per core. |
| `my.vms.instances.<name>.spice.enable` | boolean | `true` | Whether to enable SPICE. |
| `my.vms.instances.<name>.uuid` | string | `null` | QEMU machine UUID. |
| `my.vms.instances.<name>.vga` | string | `"virtio"` | VGA card type. |
| `my.vms.ovmfPackage` | package | `<derivation OVMF-202605>` | OVMF package. |
docs/remote/README.md
+7 -5
View File
@@ -5,8 +5,10 @@ edge VPSes (`britway` in London, `britnet` in Birmingham) and the `kelder` site
server at a remote location, linked back to colony over WireGuard and acting as a NixOS
container host.
| Box | What it is |
| --- | --- |
| [`britway`](britway.md) | Vultr VPS (London, `lon1`): Headscale control plane, Tailscale exit node, BGP edge, nginx |
| [`britnet`](britnet.md) | VPS (Birmingham, `bhx1`): Tailscale exit node / WireGuard hub |
| [`kelder`](kelder/README.md) | Secondary home server (`hentai.engineer`): container host, Samba, DDNS (containers on its page) |
| Box | What it is | Docs |
| --- | --- | --- |
| `britway` | Vultr VPS (London, `lon1`): Headscale control plane, Tailscale exit node, BGP edge, nginx | [britway.md](britway.md) |
| `britnet` | VPS (Birmingham, `bhx1`): Tailscale exit node / WireGuard hub | [britnet.md](britnet.md) |
| `kelder` | Secondary home server (`hentai.engineer`): container host, Samba, DDNS | [kelder.md](kelder.md) |
| `kelder-acquisition` | Media stack container on `kelder` (Transmission over VPN, *arr, Jellyfin) | [kelder-acquisition.md](kelder-acquisition.md) |
| `kelder-spoder` | Web container on `kelder` (Nextcloud + nginx reverse proxy) | [kelder-spoder.md](kelder-spoder.md) |
docs/remote/britnet.md
+12 -14
View File
@@ -5,33 +5,31 @@ narrower gateway role than [`britway`](britway.md) (no control plane, no BGP).
- **Source:** [`nixos/boxes/britnet.nix`](../../nixos/boxes/britnet.nix)
- **Host:** VPS (Birmingham, `bhx1`; provider uplink assignment `allhost`)
- **nixpkgs:** `mine`
## Platform
| Component | Allocation |
|---|---|
| Virtualisation | KVM/QEMU guest |
| Compute | 2 vCPUs and 2 GiB RAM |
| Storage | 32 GiB virtio disk with separate ext4 filesystems for `/boot`, `/nix` and `/persist`; root is tmpfs |
## Role
- **Tailscale exit node** — logs into the headscale on [`britway`](britway.md)
(`--login-server=https://hs.nul.ie`) with `--advertise-exit-node`.
- **WireGuard hub** — `wg0` listens on UDP 51820 on the `vpn` assignment, with a single static
peer.
- **WireGuard hub** — `wg0` listens on UDP 51820 on the `vpn` network
(`10.200.0.0/24` / `fdfb:5ebf:6e84::/64`), with a single peer at `10.200.0.10` /
`fdfb:5ebf:6e84::10`.
- **NAT gateway** — traffic arriving on `tailscale0`/`wg0` is forwarded out `veth0` and SNATed
to the `allhost` v4/v6 addresses.
## Network assignments
See the consolidated [network assignments](../networking.md#box-assignments) table (this box: `britnet`).
<!-- assignments: britnet -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| britnet | allhost | `77.74.199.67/24 gw 77.74.199.1` | `2a12:ab46:5344:99::a/64 gw 2a12:ab46:5344::1` | bhx1.int.nul.ie | |
| britnet | vpn | `10.200.0.1/24` | `fdfb:5ebf:6e84::1/64` | — | |
<!-- assignments-end -->
## Networking
- The provider interface is renamed to `veth0` by MAC. Its IPv6 default gateway sits off-subnet, so
a link-scope route is added to reach it.
- The provider interface is renamed to `veth0` by MAC. The v6 default gateway
(`2a12:ab46:5344::1`) sits off-subnet, so a link-scope route is added to reach it.
- `wg0` is a networkd WireGuard netdev keyed from `britnet/wg.key`; RA is disabled on it.
- Upstream DNS is hardcoded to Cloudflare (`1.1.1.1` / `1.0.0.1`).
- `iperf3` runs with an open port for bandwidth testing.
docs/remote/britway.md
+11 -14
View File
@@ -5,15 +5,6 @@ control plane, a tailnet exit node, and the BGP speaker for AS211024.
- **Source:** [`nixos/boxes/britway/`](../../nixos/boxes/britway)
- **Host:** VPS at Vultr (London, `lon1`)
- **nixpkgs:** `mine`
## Platform
| Component | Allocation |
|---|---|
| Virtualisation | Vultr VC2 virtual guest on a QEMU-compatible platform |
| Compute | 2 vCPUs and 2 GiB RAM |
| Storage | 65 GiB virtio disk with separate ext4 filesystems for `/boot`, `/nix` and `/persist`; root is tmpfs |
## Role
@@ -27,21 +18,27 @@ control plane, a tailnet exit node, and the BGP speaker for AS211024.
- **BGP edge** — `bird2` speaks BGP as AS211024 to Vultr transit (AS64515, separate v4/v6
sessions authenticated with a password from `britway/bgp-password-vultr.conf`) and exports
everything to a `bgp.tools` monitoring session. It originates the internal, colony and home
IPv6 prefixes documented in [networking](../networking.md#domains).
v6 prefixes (`2a0e:97c0:4df::/48`, `2a0e:97c0:4d2::/48`, `2a0e:97c0:4d0::/48`).
- **nginx** — reverse proxy fronting headscale (`hs.nul.ie` → `localhost` headscale port),
with a wildcard ACME cert for `nul.ie` issued via Cloudflare DNS.
## Network assignments
See the consolidated [network assignments](../networking.md#box-assignments) table (this box: `britway`).
<!-- assignments: britway -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| britway | as211024 | `10.100.50.5/24` | `2a0e:97c0:4df:0:2::1/64` | — | |
| britway | vultr | `45.76.141.188/23 gw 45.76.140.1` | `2001:19f0:7402:128b::1/64` | lon1.int.nul.ie | |
<!-- assignments-end -->
## Networking
- Two assignments: `vultr` on the provider interface `veth0` (renamed by MAC), and `as211024`
on the `l2mesh` VXLAN interface (`my.vpns.l2`) — member of the shared mesh (see
[The AS211024 L2 mesh](../networking.md#the-as211024-l2-mesh)).
on the l2mesh VXLAN interface (`my.vpns.l2`, a mesh shared with `estuary` and the home
routers `river`/`stream`).
- Static routes steer colony/home v4 traffic over the `as211024` mesh. A separate `ts-extra`
routing table with a policy rule on `tailscale0` ingress sends Tailscale-sourced v6
routing table (1337) with a policy rule on `tailscale0` ingress sends Tailscale-sourced v6
traffic for colony via `estuary`, while the box's own v6 uses WAN.
- The firewall trusts the `as211024` prefixes (`lib.my.c.as211024.nftTrust`) and
`tailscale0`; `iperf3` runs with an open port for bandwidth testing.
docs/remote/kelder-acquisition.md
+48
View File
@@ -0,0 +1,48 @@
# kelder-acquisition
The media acquisition stack for the kelder site — Transmission (forced over VPN), the *arrs
and Jellyfin in one NixOS container.
- **Source:** [`nixos/boxes/kelder/containers/acquisition/`](../../nixos/boxes/kelder/containers/acquisition)
- **Host:** NixOS container on [`kelder`](kelder.md)
## Role
- **Transmission** (`transmission_4`) — BitTorrent client bound to the VPN
(`bindsTo systemd-networkd-wait-online@vpn.service`); peer port 26180 (forwarded in the
AirVPN config), 20 MiB/s down / 1 MiB/s up limits, ratio limit 2.0. Downloads land in
`/mnt/media/downloads/torrents` (`/mnt/media` is bind-mounted from the host's
`/mnt/storage/media`).
- **Jackett, Radarr, Sonarr** — indexer + media managers, in the shared `media` group with
`UMask 0002`.
- **Jellyfin** — streaming with Intel hardware transcoding: `/dev/dri` is bind-mounted from
the host, `intel-vaapi-driver`/`intel-ocl` are installed and the `jellyfin` user is in the
`render` group.
## Network assignments
<!-- assignments: kelder-acquisition -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| acquisition-ctr | internal | `172.16.64.2/24 gw 172.16.64.1` | — | hentai.engineer | |
<!-- assignments-end -->
## Networking
- `internal` assignment (name `acquisition-ctr`) on the host's `ctrs` bridge, MTU 1460 to
match the site WAN.
- All non-site traffic goes over an AirVPN WireGuard tunnel (`vpn` netdev, MTU 1320, AirVPN
IE endpoint) using fwmark 42 / route table 51820: policy rules keep traffic to and from the
kelder prefixes on the main table and push everything else via the VPN.
- An nftables input chain drops new TCP connections from the VPN interface except the
Transmission peer port; the web UI ports (9091 Transmission, 9117 Jackett, 7878 Radarr,
8989 Sonarr, 8096 Jellyfin) are accepted from the site. When built as a dev VM, those ports
are forwarded to the host.
- Sonarr still needs the EOL .NET 6 runtime, allowed via
`nixpkgs.config.permittedInsecurePackages`.
## Notable config files
- [`nixos/boxes/kelder/containers/acquisition/default.nix`](../../nixos/boxes/kelder/containers/acquisition/default.nix) — services and users.
- [`nixos/boxes/kelder/containers/acquisition/networking.nix`](../../nixos/boxes/kelder/containers/acquisition/networking.nix) — AirVPN tunnel + firewall.
docs/remote/kelder/containers/kelder-spoder.md → docs/remote/kelder-spoder.md
+12 -8
View File
@@ -3,9 +3,8 @@
The web container for the kelder site: Nextcloud plus an nginx (OpenResty) reverse proxy for
the site's services.
- **Source:** [`nixos/boxes/kelder/containers/spoder/`](../../../../nixos/boxes/kelder/containers/spoder)
- **Host:** NixOS container on [`kelder`](../README.md)
- **nixpkgs:** `mine`
- **Source:** [`nixos/boxes/kelder/containers/spoder/`](../../nixos/boxes/kelder/containers/spoder)
- **Host:** NixOS container on [`kelder`](kelder.md)
## Role
@@ -19,7 +18,12 @@ the site's services.
## Network assignments
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `kelder-spoder`).
<!-- assignments: kelder-spoder -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| spoder-ctr | internal | `172.16.64.3/24 gw 172.16.64.1` | — | hentai.engineer | |
<!-- assignments-end -->
## Proxy vhosts
@@ -35,8 +39,8 @@ All under `hentai.engineer`, each with a `*-local` alias:
| `sonarr` | Sonarr on `kelder-acquisition` (:8989) | `htpasswd` |
| `cloud` | Nextcloud (local) | — |
An `init_worker_by_lua` timer polls `v4.ident.me` periodically to track the site's public IP; it
feeds `localRedirect` rewrites (bounce public-IP clients to the `*-local` name) that
An `init_worker_by_lua` timer polls `v4.ident.me` every 5 minutes to track the site's public
IP; it feeds `localRedirect` rewrites (bounce public-IP clients to the `*-local` name) that
are currently **disabled** (commented out — Virgin Media filters DNS answers containing local
IPs, so the split doesn't work as intended).
@@ -46,5 +50,5 @@ IPs, so the split doesn't work as intended).
## Notable config files
- [`nixos/boxes/kelder/containers/spoder/default.nix`](../../../../nixos/boxes/kelder/containers/spoder/default.nix) — Nextcloud + ACME.
- [`nixos/boxes/kelder/containers/spoder/nginx.nix`](../../../../nixos/boxes/kelder/containers/spoder/nginx.nix) — reverse proxy vhosts.
- [`nixos/boxes/kelder/containers/spoder/default.nix`](../../nixos/boxes/kelder/containers/spoder/default.nix) — Nextcloud + ACME.
- [`nixos/boxes/kelder/containers/spoder/nginx.nix`](../../nixos/boxes/kelder/containers/spoder/nginx.nix) — reverse proxy vhosts.
docs/remote/kelder.md
+67
View File
@@ -0,0 +1,67 @@
# kelder
Secondary home server at a remote site, domain `hentai.engineer`. Linked back to colony over
WireGuard and acting as a NixOS container host (like `shill`/`sfh`).
- **Source:** [`nixos/boxes/kelder/`](../../nixos/boxes/kelder)
- **Host:** physical (Intel; LTS kernel, `kvm-intel`, IOMMU on)
## Role
- **Container host** — runs two NixOS containers on the `ctrs` bridge
(`my.containers.instances`): `kelder-acquisition` and `kelder-spoder` (below).
- **Public services via colony** — a WireGuard tunnel (`estuary` netdev) connects to colony's
`estuary` box, which DNATs public traffic to kelder's `/32`; policy routing (table 51820 +
connmark) sends replies back through the tunnel while ordinary traffic uses the LAN.
kelder's own NAT forwards `http`/`https` on to `kelder-spoder`.
- **Nextcloud host** — served from the `kelder-spoder` container.
- **Samba** — the `storage` share backed by `/mnt/storage`, with `nmbd` and `samba-wsdd` for
Windows discovery.
- **DDNS** — a `ddns-update` timer (every 10 min) runs `dns_update.py` to sync the
`hentai.engineer` and `kelder-local.hentai.engineer` Cloudflare records with the address on
`et1g0`.
## Network assignments
<!-- assignments: kelder -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| kelder-ctrs | ctrs | `172.16.64.1/24` | — | hentai.engineer | |
| kelder | estuary | `94.142.242.254/32` | — | — | |
<!-- assignments-end -->
## Containers
| Container | Role | Docs |
| --- | --- | --- |
| `kelder-acquisition` | Media stack (Transmission over AirVPN, Jackett/Radarr/Sonarr, Jellyfin) | [kelder-acquisition.md](kelder-acquisition.md) |
| `kelder-spoder` | Nextcloud + nginx reverse proxy | [kelder-spoder.md](kelder-spoder.md) |
The containers are not deploy targets (`my.deploy.enable = false`); they're managed through
the host.
## Networking
- LAN on `et1g0` (renamed by MAC) with DHCP and MTU 1460 (`lib.my.c.kelder.ipv4MTU`); the
kelder v4 prefixes are masqueraded out of it.
- The `estuary` WireGuard peer (`AllowedIPs = 0.0.0.0/0`, keepalive 25s) is combined with
rules that keep LAN traffic on the main table and only route tunnel-marked/owned traffic
via table 51820.
## Services
- `netdata` (proxied as `monitor.hentai.engineer` by `kelder-spoder`), `smartd`, `fstrim`,
LVM thin provisioning.
- `minecraft-server` is present but **disabled** (`enable = false`); the firewall still opens
25565 tcp/udp.
- Primary user `kontent` (in the `storage`/`media` groups).
- Sets `system.nixos.distroName = "KelderOS"`, a custom Plymouth theme and an `amogus-beep`
boot jingle ([`boot.nix`](../../nixos/boxes/kelder/boot.nix)).
## Notable config files
- [`nixos/boxes/kelder/default.nix`](../../nixos/boxes/kelder/default.nix) — system, assignments, tunnel, NAT, containers.
- [`nixos/boxes/kelder/boot.nix`](../../nixos/boxes/kelder/boot.nix) — Plymouth theme + boot beep.
- [`nixos/boxes/kelder/containers/`](../../nixos/boxes/kelder/containers) — the two container definitions.
- [`nixos/boxes/kelder/dns_update.py`](../../nixos/boxes/kelder/dns_update.py) — Cloudflare DDNS script.
docs/remote/kelder/README.md
-61
View File
@@ -1,61 +0,0 @@
# kelder
Secondary home server at a remote site, domain `hentai.engineer`. Linked back to colony over
WireGuard and acting as a NixOS container host (like `shill`/`sfh`).
- **Source:** [`nixos/boxes/kelder/`](../../../nixos/boxes/kelder)
- **Host:** physical (Intel; LTS kernel, `kvm-intel`, IOMMU on)
- **nixpkgs:** `mine`
## Role
- **Container host** — runs two NixOS containers on the `ctrs` bridge
(`my.containers.instances`): `kelder-acquisition` and `kelder-spoder` (below).
- **Public services via colony** — a WireGuard tunnel (`estuary` netdev) connects to colony's
`estuary` box, which DNATs public traffic to kelder's tunneled assignment; connection-mark-based
policy routing sends replies back through the tunnel while ordinary traffic uses the LAN.
kelder's own NAT forwards `http`/`https` on to `kelder-spoder`.
- **Nextcloud host** — served from the `kelder-spoder` container.
- **Samba** — the `storage` share backed by `/mnt/storage`, with `nmbd` and `samba-wsdd` for
Windows discovery.
- **DDNS** — a `ddns-update` timer runs `dns_update.py` periodically to sync the
`hentai.engineer` and `kelder-local.hentai.engineer` Cloudflare records with the address on
`et1g0`.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `kelder`).
## Containers
| Container | Role |
| --- | --- |
| [`kelder-acquisition`](containers/kelder-acquisition.md) | Media stack (Transmission over AirVPN, Jackett/Radarr/Sonarr, Jellyfin) |
| [`kelder-spoder`](containers/kelder-spoder.md) | Nextcloud + nginx reverse proxy |
The containers are not deploy targets (`my.deploy.enable = false`); they're managed through
the host.
## Networking
- LAN on `et1g0` (renamed by MAC) with DHCP and MTU 1460 (`lib.my.c.kelder.ipv4MTU`); the
kelder v4 prefixes are masqueraded out of it.
- The `estuary` WireGuard peer is combined with rules that keep LAN traffic on the main table and
only route tunnel-marked or owned traffic through the tunnel's dedicated table.
## Services
- `netdata` (proxied as `monitor.hentai.engineer` by `kelder-spoder`), `smartd`, `fstrim`,
LVM thin provisioning.
- `minecraft-server` is present but **disabled** (`enable = false`); the firewall still opens
25565 tcp/udp.
- Primary user `kontent` (in the `storage`/`media` groups).
- Sets `system.nixos.distroName = "KelderOS"`, a custom Plymouth theme and an `amogus-beep`
boot jingle ([`boot.nix`](../../../nixos/boxes/kelder/boot.nix)).
## Notable config files
- [`nixos/boxes/kelder/default.nix`](../../../nixos/boxes/kelder/default.nix) — system, assignments, tunnel, NAT, containers.
- [`nixos/boxes/kelder/boot.nix`](../../../nixos/boxes/kelder/boot.nix) — Plymouth theme + boot beep.
- [`nixos/boxes/kelder/containers/`](../../../nixos/boxes/kelder/containers) — the two container definitions.
- [`nixos/boxes/kelder/dns_update.py`](../../../nixos/boxes/kelder/dns_update.py) — Cloudflare DDNS script.
docs/remote/kelder/containers/kelder-acquisition.md
-45
View File
@@ -1,45 +0,0 @@
# kelder-acquisition
The media acquisition stack for the kelder site — Transmission (forced over VPN), the *arrs
and Jellyfin in one NixOS container.
- **Source:** [`nixos/boxes/kelder/containers/acquisition/`](../../../../nixos/boxes/kelder/containers/acquisition)
- **Host:** NixOS container on [`kelder`](../README.md)
- **nixpkgs:** `mine`
## Role
### Transmission
`transmission_4` is bound to the VPN wait-online unit and uses AirVPN's forwarded peer port. Upload,
download and seed-ratio limits are configured in the service source. Downloads use
`/mnt/media/downloads/torrents`, backed by the host's `/mnt/storage/media`.
### Media services
Jackett, Radarr and Sonarr share the `media` group with a group-writable umask. Jellyfin uses the
host's bind-mounted `/dev/dri` with `intel-vaapi-driver` / `intel-ocl`; its user belongs to
`render`.
## Network assignments
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `kelder-acquisition`).
## Networking
- `internal` assignment (name `acquisition-ctr`) on the host's `ctrs` bridge, MTU 1460 to
match the site WAN.
- All non-site traffic goes over an AirVPN WireGuard tunnel (`vpn` netdev, AirVPN IE endpoint).
Policy rules keep traffic to and from the kelder prefixes on the main table and push everything
else through the VPN's dedicated table.
- An nftables input chain drops new TCP connections from the VPN interface except the
Transmission peer port; the web UI ports (9091 Transmission, 9117 Jackett, 7878 Radarr,
8989 Sonarr, 8096 Jellyfin) are accepted from the site. When built as a dev VM, those ports
are forwarded to the host.
- Sonarr still needs the EOL .NET 6 runtime, allowed via
`nixpkgs.config.permittedInsecurePackages`.
## Notable config files
- [`nixos/boxes/kelder/containers/acquisition/default.nix`](../../../../nixos/boxes/kelder/containers/acquisition/default.nix) — services and users.
- [`nixos/boxes/kelder/containers/acquisition/networking.nix`](../../../../nixos/boxes/kelder/containers/acquisition/networking.nix) — AirVPN tunnel + firewall.
docs/sites/colony/README.md
+75 -24
View File
@@ -7,33 +7,84 @@ the boxes: almost everything reachable from the internet lives here.
- **Public domain:** `nul.ie` — public services are published as `*.nul.ie`
- **Source:** [`nixos/boxes/colony/`](../../../nixos/boxes/colony)
## Networking
## Shape
`colony` separates the host, VMs, `shill` containers and `whale2` OCI workloads onto dedicated
networks behind [`estuary`](estuary.md), which terminates the public addressing. The canonical
prefixes and routing overview are in the [`colony` section of networking.md](../../networking.md#colony).
`colony` is the physical VM host. It runs the VMs below; `shill` is itself a
NixOS container host where most applications run.
## Boxes
```
colony (physical VM host, ams1)
├── estuary ── edge router: WAN, firewall/NAT, DNS, BGP (AS211024), WireGuard
├── shill ──── NixOS container host ──┬── middleman (reverse proxy, ACME, nginx-sso, librespeed)
│ ├── vaultwarden (password manager)
│ ├── colony-psql (shared PostgreSQL)
│ ├── chatterbox (Matrix Synapse + bridges)
│ ├── jackflix (media stack)
│ ├── object (MinIO, Harmonia Nix cache, Sharry, HedgeDoc, wastebin)
│ ├── toot (Bluesky PDS; Mastodon disabled)
│ ├── waffletail (Tailscale subnet router / exit node)
│ ├── qclk (WireGuard management appliance)
│ ├── gam (Terraria server)
│ └── jam (raw nspawn customer container)
├── whale2 ─── podman/OCI host for game servers
├── git ────── Gitea + Gitea Actions runner
├── mail ───── Debian VM running mailcow (not NixOS)
└── darts ──── third-party/customer VM (opaque, not NixOS)
```
| Box | Role |
|---|---|
| [`colony`](colony.md) | Physical VM host (AMD, KVM, LVM-thin, `borgthin` backups → rsync.net) |
| [`estuary`](estuary.md) | Edge router: WAN, firewall/NAT, DNS, BGP (AS211024), WireGuard |
| [`shill`](shill/README.md) | NixOS container host (most applications; per-container pages under `shill/`) |
| [`whale2`](whale2.md) | podman/OCI game-server host |
| [`git`](git.md) | Gitea + Gitea Actions runner |
| [`mail`](mail.md) | Debian VM running mailcow (not NixOS) |
| [`darts`](darts.md) | Third-party/customer VM (not NixOS) |
| [`portcullis`](portcullis.md) | Bare-metal edge box for Nikhef; being staged, not yet in service |
## Networks
The applications running on `shill` are listed on its own page — see
[shill/README.md](shill/README.md#containers).
All internal space is carved out of `10.100.0.0/16` and `2a0e:97c0:4d2:10::/60`
(`lib.my.c.colony.prefixes`):
`mail` and `darts` are host-defined VMs whose guest operating systems are managed out of band; their
pages document only what this repository controls.
| Network | IPv4 CIDR | IPv6 CIDR | Purpose |
|---|---|---|---|
| `base` | `10.100.0.0/24` | `2a0e:97c0:4d2:10::/64` | Base LAN shared by `colony` and `estuary` |
| `vms` | `10.100.1.0/24` | `2a0e:97c0:4d2:11::/64` | VM network |
| `ctrs` | `10.100.2.0/24` | `2a0e:97c0:4d2:12::/64` | `shill` container network |
| `oci` | `10.100.3.0/24` | `2a0e:97c0:4d2:13::/64` | `whale2` podman network |
`portcullis` is new hardware headed for Nikhef that will take over most of `estuary`'s edge routing.
It is not deployed yet and the resulting topology is still being worked out. It travels with
[`fergal`](fergal.md), an OpenWrt SFP+ switch whose firmware this flake builds; both are staged at
home for now, borrowing the home fabric through
[jim](../home/switches.md#fergal-portculliss-switch).
Public addressing — the WAN /24 (`94.142.240.44`), the `vip*` ranges shared by
the VMs, and the customer /32s for `mail` / `darts` — terminates on `estuary`;
see [estuary.md](estuary.md).
## Machines
| Machine | Role | Page |
|---|---|---|
| `colony` | Physical VM host (AMD, KVM, LVM-thin, `borgthin` backups → rsync.net) | [colony.md](colony.md) |
| `estuary` | Edge router: WAN, firewall/NAT, DNS, BGP (AS211024), WireGuard | [estuary.md](estuary.md) |
| `shill` | NixOS container host (most applications) | [shill.md](shill.md) |
| `whale2` | podman/OCI game-server host | [whale2.md](whale2.md) |
| `git` | Gitea + Gitea Actions runner | [git.md](git.md) |
| `mail` | Debian VM running mailcow (not NixOS) | [mail.md](mail.md) |
| `darts` | Third-party/customer VM (not NixOS) | [darts.md](darts.md) |
### `shill` containers
Each has its own page under `shill/containers/`:
| Container | Role | Page |
|---|---|---|
| `middleman` | Front-end nginx reverse proxy, ACME, nginx-sso, librespeed | [middleman](shill/containers/middleman.md) |
| `vaultwarden` | Vaultwarden password manager | [vaultwarden](shill/containers/vaultwarden.md) |
| `colony-psql` | Shared PostgreSQL for colony services | [colony-psql](shill/containers/colony-psql.md) |
| `chatterbox` | Matrix homeserver + bridges | [chatterbox](shill/containers/chatterbox.md) |
| `jackflix` | Media stack (Jellyfin, *arr, Transmission, PhotoPrism, copyparty) | [jackflix](shill/containers/jackflix.md) |
| `object` | MinIO (S3), Harmonia Nix cache, Sharry, HedgeDoc, wastebin | [object](shill/containers/object.md) |
| `toot` | Bluesky PDS (Mastodon disabled) | [toot](shill/containers/toot.md) |
| `waffletail` | Tailscale subnet router / exit node | [waffletail](shill/containers/waffletail.md) |
| `qclk` | WireGuard management appliance | [qclk](shill/containers/qclk.md) |
| `gam` | Terraria server | [gam](shill/containers/gam.md) |
## Non-NixOS VMs
Two VMs are declared in `colony`'s `my.vms.instances` — so `colony` runs them
and routes/firewalls their traffic — but they are **not** managed as NixOS
systems by this repo:
- **`mail`** — a Debian VM running [mailcow](https://mailcow.email/)
(`mail.nul.ie`). ACME certificates are pushed to it from `middleman`.
See [mail.md](mail.md).
- **`darts`** — an opaque third-party/customer VM, given a routed public /32
and IPv6 /64 and otherwise left alone. See [darts.md](darts.md).
docs/sites/colony/colony.md
+37 -61
View File
@@ -8,64 +8,39 @@ everything at the colony site.
- **Host:** bare metal (this *is* the physical box)
- **nixpkgs:** `mine-stable`
## Hardware
| Component | Inventory |
|---|---|
| Platform | ASRock Rack X570D4U server board |
| CPU | AMD Ryzen 9 5950X (16 cores / 32 threads) |
| Memory | 128 GiB |
| NVMe storage | Three 2 TB Samsung SSD 980 PRO devices providing the NVMe-backed LVM thin pool and data LVs |
| Bulk storage | Three 12 TB WD120EDBZ disks and one 18 TB WD180EDGZ disk for the bulk LVM volumes |
| Boot | SanDisk USB device holding the EFI system partition |
| Network / management | Two Intel I210 Gigabit Ethernet controllers, one passed through to `estuary`; ASPEED BMC graphics and console |
## Role
Bare-metal AMD host. It does little application work itself — its job is to run
the VMs and provide them with storage, networking and backups.
### Virtualisation
The `my.vms` module drives QEMU/KVM with `kvm-amd` and IOMMU enabled. Each `my.vms.instances` entry
becomes a `vm@<name>` systemd service with UEFI, QMP/monitor sockets under `/run/vms/<name>/`, TAP
networking and optional `hostDevices` passthrough through `vfio-pci`. `estuary` receives the WAN NIC
this way.
### Storage
All three NVMe SSDs and all four SATA HDDs are physical volumes in one `main` VG. Separate pools
and standalone RAID LVs keep workloads on the appropriate media:
| Layer | Physical layout | Main consumers |
|---|---|---|
| `nvme-tpool` | Thin-pool data is RAID 0 across the three NVMe SSDs; thin metadata is RAID 1 on two of them. The data itself has no redundancy. | Host `/nix` and `/persist`; VM system and persistence disks; fast data LVs such as `minio`, `oci`, `git`, `gitea-actions-cache`, `nix-cache` and `jam` |
| `hdds-tpool` | Thin-pool data is RAID 5 across the three 12 TB HDDs; thin metadata is RAID 1 on two NVMe SSDs. | `media`, passed to `shill`, and `backup`, mounted by the host at `/mnt/backup` |
| `darts-media` | Standalone RAID 5 LV spanning all four HDDs. | Bulk storage passed to `darts` |
| `darts-ext` | Standalone linear LV using the remaining capacity of the 18 TB HDD. | Expansion storage passed to `darts` |
`media` and `backup` are thin-provisioned, so their virtual capacities are not additive physical
capacity and overcommit `hdds-tpool`. VM disks built with `lib.my.vm.disk` are named
`vm-<name>-<disk>` in `main`; `lib.my.vm.lvmDisk` attaches the named data LVs.
The initrd activates only `colony-nix` and `colony-persist`. This avoids checking and activating all
of the storage before switching root; `lvm-activate-main.service` activates the remaining LVs before
local filesystems and VMs need them.
### Backups
`my.borgthin` job `main` snapshots host and VM persistence/data LVs into `/mnt/backup/main`.
`borgthin-rsync.service` copies the repository to rsync.net and `rsync-lvm-meta.service` sends the
LVM metadata; both run at idle priority after the Borg job.
### Monitoring
Netdata uses FreeIPMI while ignoring the VCCM sensor. The box also runs `smartd` with logs under
`/var/log/smartd`, `rasdaemon`, and `fstrim` before the backup job.
- **Virtualisation:** QEMU/KVM (`kvm-amd`, IOMMU on) driven by the `my.vms`
module: each entry in `my.vms.instances` becomes a `vm@<name>` systemd
service running `qemu-kvm` with UEFI, a QMP/monitor socket under
`/run/vms/<name>/`, TAP networking and optional PCI passthrough
(`hostDevices`, bound to `vfio-pci`). `estuary` gets the WAN NIC this way.
- **Storage:** LVM-thin (`services.lvm.boot.thin`) in the `main` VG; VM disks
are logical volumes (`vm-<name>-<disk>`, see the `lib.my.vm.disk` /
`lvmDisk` helpers). `/persist` holds host state, `/mnt/backup` the local
borg repo. Only the boot-critical LVs are activated in the initrd; the rest
come up via `lvm-activate-main.service`.
- **Backups:** `my.borgthin` job `main` snapshots the persist/data LVs of the
host and its VMs into `/mnt/backup/main`; `borgthin-rsync.service` then
rsyncs the repo to rsync.net and `rsync-lvm-meta.service` ships the LVM
metadata alongside (both run idle-priority, after the borg job).
- **Monitoring/health:** netdata (freeipmi, ignoring the VCCM sensor), `smartd`
(logging to `/var/log/smartd`), `rasdaemon`, `fstrim` at 04:45 (before the
05:00 backup).
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `colony`).
<!-- assignments: colony -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| colony (vm) | internal | `94.142.241.224/32` | `2a0e:97c0:4d2:10::2/64` | ams1.int.nul.ie | |
| colony-routing | routing | `10.100.0.2/24 gw 10.100.0.1` | — | ams1.int.nul.ie | |
| colony-vms | vms | `10.100.1.1/24` | `2a0e:97c0:4d2:11::1/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
## Networking
@@ -75,8 +50,9 @@ See the consolidated [network assignments](../../networking.md#box-assignments)
- `colony` sends RAs on `vms` (DNS = `estuary`'s base address) and carries
static routes for the downstream prefixes: `ctrs` via `shill`, `oci` via
`whale2`, plus the Tailscale, `qclk` and `jam` prefixes via `shill`.
- `estuary` is the default gateway (via the `base` bridge); `colony`'s own public-facing address is
its `internal` assignment (alt name `vm`).
- `estuary` is the default gateway (via the `base` bridge); `colony`'s own
public-facing address is its `internal` assignment (a `vip1` /32, alt name
`vm`).
- The customer VMs attach to dedicated TAP devices (`vm-mail`, `vm-darts`)
which are **not** bridged: networkd puts the point-to-point /32
(`lib.my.c.colony.custRouting`) and the customer's IPv6 /64 on each, sends
@@ -90,18 +66,18 @@ See the consolidated [network assignments](../../networking.md#box-assignments)
Declared in `my.vms.instances` (`cpus`/`threads` are QEMU `smp` values):
| VM | Cores | Threads | Memory | Network | Disks |
| VM | Cores | Threads | Memory | MAC | Disks |
|---|---|---|---|---|---|
| `estuary` | 2 | 2 | 3 GiB | `base` | `esp` / `nix` / `persist` LVs + WAN NIC passthrough |
| `shill` | 12 | 2 | 40 GiB | `vms` | `esp` / `nix` / `persist` + `media` / `minio` / `nix-cache` / `jam` LVs |
| `whale2` | 8 | 2 | 16 GiB | `vms` | `esp` / `nix` / `persist` + `oci` LV |
| `git` | 12 | 2 | 40 GiB | `vms` | `esp` / `nix` / `persist` / `oci` + `git` / `gitea-actions-cache` LVs |
| `mail` | 3 | 2 | 6 GiB | `vm-mail` tap | `root` / `data` LVs |
| `darts` | 4 | 2 | 16 GiB | `vm-darts` tap | `root` LV + `darts-media` / `darts-ext` LVs |
| `estuary` | 2 | 2 | 3 GiB | `52:54:00:15:1a:53` (`base`) | `esp` / `nix` / `persist` LVs + WAN NIC passthrough |
| `shill` | 12 | 2 | 40 GiB | `52:54:00:27:3d:5c` (`vms`) | `esp` / `nix` / `persist` + `media` / `minio` / `nix-cache` / `jam` LVs |
| `whale2` | 8 | 2 | 16 GiB | `52:54:00:d5:d9:c6` (`vms`) | `esp` / `nix` / `persist` + `oci` LV |
| `git` | 12 | 2 | 40 GiB | `52:54:00:75:78:a8` (`vms`) | `esp` / `nix` / `persist` / `oci` + `git` / `gitea-actions-cache` LVs |
| `mail` | 3 | 2 | 6 GiB | `52:54:00:a8:d1:03` (`vm-mail` tap) | `root` / `data` LVs |
| `darts` | 4 | 2 | 16 GiB | `52:54:00:a8:29:cd` (`vm-darts` tap) | `root` LV + `darts-media` / `darts-ext` LVs |
`estuary`, `shill`, `whale2` and `git` are NixOS systems with their own pages
(see the [README](README.md#boxes)); [`mail`](mail.md) and [`darts`](darts.md) have host-side
definitions here, but their guest operating systems are managed out of band.
(see the [README](README.md#machines)); `mail` and `darts` are not (see
[Non-NixOS VMs](README.md#non-nixos-vms)).
## Notable config files
docs/sites/colony/darts.md
+4 -14
View File
@@ -9,29 +9,19 @@ and doesn't manage it.
and the `90-vm-darts` network in
[`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
- **Host:** VM on `colony`
- **nixpkgs:** not applicable (unmanaged guest; host-side definition uses `colony`'s `mine-stable`)
## Role
- Customer/dedicated VM, left alone beyond hosting and connectivity.
- A thin-provisioned `root` LV on the NVMe pool, plus `darts-media` (RAID 5 across the four HDDs)
and `darts-ext` (linear storage on the 18 TB HDD) in the `main` VG. See
[`colony`'s storage layout](colony.md#storage).
- 4 cores, 16 GiB RAM; a `root` LV plus `darts-media` and `darts-ext` LVs from
the `media`/`ext` volume groups.
## Network assignments
This guest is not a NixOS system, so its host-routed addresses are not rows in the generated
[network assignments](../../networking.md#box-assignments) table.
## Networking
- Same customer-VM pattern as [`mail`](mail.md): dedicated unbridged TAP
(`vm-darts`), point-to-point address
(`vm-darts`, MAC `52:54:00:a8:29:cd`), point-to-point address
(`custRouting.darts-vm`) on the host side, link-routed public /32
`94.142.242.255`, and the IPv6 /64 `2a0e:97c0:4d2:2001::/64` with RAs.
- DNS: `darts-cust.ams1.int.nul.ie`. Like the other customer prefixes, its
inbound traffic is accepted by `estuary` without per-port filtering and
forwarded on by `colony`.
## Notable config files
- [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix) — VM definition.
- [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix) — host-side network and routing.
docs/sites/colony/estuary.md
+67 -88
View File
@@ -10,18 +10,30 @@ and connects everything else at the site to the internet.
## Role
- **Edge routing / firewall / NAT** — owns the colony public IPv4/IPv6 assignments, NATs outbound
traffic, and port-forwards inbound services (see [Firewall and NAT](#firewall-and-nat)).
- **DNS** — PowerDNS authoritative server *and* recursor (see [DNS](#dns)).
- **BGP** — BIRD2 speaking AS211024 with upstreams, IXP route servers and
direct peers (see [BGP](#bgp)).
- **VPNs** — the `as211024` L2 VXLAN mesh plus three point-to-point WireGuard
tunnels (see [VPNs](#vpns)).
- **Misc** — `iperf3` server, netdata.
- **Edge routing / firewall / NAT:** owns the colony public IPv4/IPv6
(`94.142.240.44/24`, `2a02:898:0:20::329:1/64`), NATs outbound traffic, and
port-forwards inbound services (`my.firewall.nat.forwardPorts` driven by the
shared `lib.my.c.colony.firewallForwards` list): HTTP/S and Matrix
federation to `middleman`, git to `git`, game ports to the OCI servers on
`whale2` and to `gam`, Tailscale to `waffletail`, and the `qclk` WireGuard
port.
- **DNS:** PowerDNS authoritative server *and* recursor (see below).
- **BGP:** BIRD2 speaking AS211024 with upstreams, IXP route servers and
direct peers (see below).
- **VPNs:** member of the `as211024` L2 VXLAN mesh and endpoint for three
point-to-point WireGuard tunnels (see below).
- **Misc:** `iperf3` server, netdata.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `estuary`).
<!-- assignments: estuary -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| estuary | as211024 | `10.100.50.1/24` | `2a0e:97c0:4df::1/64` | — | |
| estuary-vm-base | base | `10.100.0.1/24` | `2a0e:97c0:4d2:10::1/64` | ams1.int.nul.ie | |
| estuary-vm (fw) | internal | `94.142.240.44/24 gw 94.142.240.254` | `2a02:898:0:20::329:1/64 gw 2a02:898:0:20::1` | ams1.int.nul.ie | |
<!-- assignments-end -->
## WAN and IXP VLANs
@@ -48,18 +60,7 @@ chain rejects non-IP/ARP ethertypes in both directions.
## Firewall and NAT
`my.firewall` (nftables). Inbound port forwards (`my.firewall.nat.forwardPorts`,
driven by the shared `lib.my.c.colony.firewallForwards` list):
| Service | Forwarded to |
|---|---|
| HTTP/S, Matrix federation | `middleman` |
| Git | `git` |
| Game servers | OCI servers on `whale2`, `gam` |
| Tailscale | `waffletail` |
| `qclk` WireGuard | `qclk` |
Besides the forwards, `extraRules` defines:
`my.firewall` (nftables). Besides the port forwards, `extraRules` defines:
- `routing-tcp` / `routing-udp` chains — the inbound allow-list for new
connections from `wan`/`as211024`/IXPs towards internal services (SSH
@@ -75,88 +76,66 @@ Besides the forwards, `extraRules` defines:
## DNS
Both halves are PowerDNS ([`dns.nix`](../../../nixos/boxes/colony/vms/estuary/dns.nix)). The live
forward and reverse records are listed in the generated [DNS reference](../../reference/dns.md).
Both halves are PowerDNS ([`dns.nix`](../../../nixos/boxes/colony/vms/estuary/dns.nix)):
### Authoritative
`my.pdns.auth`, listening on `0.0.0.0:5353` / `[::]:5353`. Primary for
`ams1.int.nul.ie`, `100.10.in-addr.arpa` and the `2a0e:97c0:4d2::/48` reverse
zone.
- Zone contents are largely generated from `allAssignments`
(`lib.my.dns.fwdRecords` / `ptrRecords` / `ptr6Records`); `ALIAS` records
(with `expand-alias`) point the zone apex at estuary itself.
- AXFR is allowed to HE.net's secondary and the trusted internal/site-egress sources used by the
generated DNS reference.
- `_acme-challenge` is a LUA `TXT` record answered from a file (DNS-01 issuance).
- Reached publicly via the NAT redirect of port 53 → 5353; the `base` side also
accepts DNS directly.
### Recursor
`my.pdns.recursor` (`pdns-recursor`), listening on localhost and the `base`
addresses, serving `prefixes.all` and the Tailscale prefixes.
- Authoritative zones are forwarded back to `127.0.0.1:5353` (with NOTIFY
support, so changes show up immediately).
- A small Lua `preresolve` hook rewrites `nix-cache.nul.ie` →
`http.ams1.int.nul.ie` so cache traffic stays on-site.
- **Authoritative** (`my.pdns.auth`) listens on `0.0.0.0:5353` / `[::]:5353` as
primary for `ams1.int.nul.ie`, `100.10.in-addr.arpa` and the
`2a0e:97c0:4d2::/48` reverse zone. Zone contents are largely generated from
`allAssignments` (`lib.my.dns.fwdRecords` / `ptrRecords` / `ptr6Records`);
`ALIAS` records (with `expand-alias`) point the zone apex at estuary itself.
AXFR is allowed to HE.net's secondary (`216.218.133.2` / `2001:470:600::2`),
and `_acme-challenge` is a LUA `TXT` record answered from a file (used for
DNS-01 issuance). Public DNS reaches it via the NAT redirect of port 53 to
5353; the `base` side also accepts DNS directly.
- **Recursor** (`my.pdns.recursor`, `pdns-recursor`) listens on localhost and
the `base` addresses, serving `prefixes.all` and the Tailscale prefixes. The
authoritative zones are forwarded back to `127.0.0.1:5353` (with NOTIFY
support so changes show up immediately), and a small Lua `preresolve` hook
rewrites `nix-cache.nul.ie` to `http.ams1.int.nul.ie` so cache traffic stays
on-site.
## BGP
BIRD2 ([`bgp.nix`](../../../nixos/boxes/colony/vms/estuary/bgp.nix)) speaking **AS211024**:
| Peer | ASN | Role | Where / notes |
|---|---|---|---|
| ColoClue | AS8283 | Upstream | `euNetworks` 2/3, v4+v6 |
| iFog | AS34927 | Upstream | IPv6 transit |
| Hurricane Electric | AS6939 | Upstream | IPv6 over Frys-IX |
| Frys-IX | AS56393 | IXP route server | |
| NL-ix | AS34307 | IXP route server | lower preference |
| FogIXP | AS47498 | IXP route server | |
| LUJE.net | AS212855 | Direct peer | ColoClue/Frys-IX/FogIXP + multihop labs |
| jurrian | AS212635 | Direct peer | |
| Meta | AS32934 | Direct peer | Frys-IX/NL-ix |
| Cloudflare | AS13335 | Direct peer | Frys-IX |
| Apple | AS714 | Direct peer | NL-ix |
| HE | AS6939 | Direct peer | Frys-IX (v4) |
| bgp.tools | AS212232 | Monitoring | multihop collector, `add paths` |
Statics originate the site prefixes: the `vip*` ranges via `base`, the
ams1/internal v6 prefixes via `base`/`as211024`, and the home /48 towards the
home routers' `as211024` VIP.
- **Upstreams:** ColoClue (AS8283, `euNetworks` 2/3, v4+v6); iFog IPv6 transit
(AS34927); Hurricane Electric IPv6 over Frys-IX (AS6939).
- **IXP route servers:** Frys-IX (AS56393), NL-ix (AS34307, depref'd by 1),
FogIXP (AS47498).
- **Direct peers:** LUJE.net (AS212855, on ColoClue/Frys-IX/FogIXP + multihop
labs), jurrian (AS212635), Meta (AS32934, Frys-IX/NL-ix), Cloudflare
(AS13335, Frys-IX), Apple (AS714, NL-ix), HE (AS6939, Frys-IX v4).
- **Monitoring:** a multihop session to the bgp.tools collector (AS212232)
exporting everything with `add paths`.
- Statics originate the site prefixes: the `vip*` ranges via `base`, the
ams1/internal v6 prefixes via `base`/`as211024`, and the home /48 towards
the home routers' `as211024` VIP.
Currently disabled (commented out): efero transit over FogIXP ("not working so
well lately") and the NL-ix Cloudflare sessions.
## VPNs
### `as211024` L2 mesh
Member alongside `river`/`stream`/`britway` (`my.vpns.l2`, the `l2mesh` module).
The mesh transport, crypto and addressing are shared fabric — see
[The AS211024 L2 mesh](../../networking.md#the-as211024-l2-mesh) in networking.md.
### WireGuard tunnels
Point-to-point tunnels terminated here as networkd `wireguard` netdevs (private
keys from agenix); each SNATs out its own interface address:
| Tunnel | Port | Prefix | Notes |
|---|---|---|---|
| `kelder` | 51820 | — | to the remote `kelder` site; kelder's public `estuary` assignment is routed over the tunnel |
| `hillcrest` | 51822 | `prefixes.hillcrest.v4` (`10.100.5.0/30`) | /32 pair, estuary `.1` ↔ remote `.2` |
| `john-valorant` | 51823 | `prefixes.john-valorant.v4` (`10.100.5.4/30`) | /32 pair, estuary `.1` ↔ remote `.2` |
- **`as211024` L2 mesh** (`my.vpns.l2`, the `l2mesh` module): VXLAN (VNI
211024, UDP-encapsulated) secured with libreswan IPsec, meshing estuary with
the home routers `river`/`stream` and `britway`. This carries the AS211024
anycast-ish internal address space between sites.
- **WireGuard endpoints** (networkd `wireguard` netdevs, keys from agenix):
- `kelder` — tunnel to the remote `kelder` site, port `51820`.
- `hillcrest` — port `51822`, point-to-point /32 pair out of
`prefixes.hillcrest.v4`.
- `john-valorant` — port `51823`, same pattern out of
`prefixes.john-valorant.v4`.
## Bandwidth management
[`bandwidth.nix`](../../../nixos/boxes/colony/vms/estuary/bandwidth.nix) implements a WAN shaper: a
token-bucket filter on `wan` (outbound) and on an `ifb-wan` IFB device that ingress traffic is
mirrored into (inbound), with
[`bandwidth.py`](../../../nixos/boxes/colony/vms/estuary/bandwidth.py) as a `bandwidth-limiter`
service that watches utilisation and can adjust the configured rate. **Currently disabled** — the
file is not in estuary's `imports` (only `dns.nix` and `bgp.nix` are), so no shaping is applied.
[`bandwidth.nix`](../../../nixos/boxes/colony/vms/estuary/bandwidth.nix)
implements a ~95% WAN shaper: a 245 Mbit token-bucket filter on `wan`
(outbound) and on an `ifb-wan` IFB device that ingress traffic is mirrored
into (inbound), with [`bandwidth.py`](../../../nixos/boxes/colony/vms/estuary/bandwidth.py)
as a `bandwidth-limiter` service that watches/utilises the link and can adjust
the rate. **Currently disabled** — the file is not in estuary's `imports`
(only `dns.nix` and `bgp.nix` are), so no shaping is applied.
## Notable config files
docs/sites/colony/fergal.md
-104
View File
@@ -1,104 +0,0 @@
# fergal
An 8-port SFP+ switch running OpenWrt, bought to sit in front of
[`portcullis`](portcullis.md) at Nikhef. It is physically at home for now, on the bench alongside
`portcullis` while that box is staged.
- **Source:** firmware built by this flake — [`openwrt/default.nix`](../../../openwrt/default.nix)
- **Host:** bare metal
- **OS:** OpenWrt (snapshot), configured through UCI rather than RouterOS or a UniFi controller
## Hardware
| Component | Inventory |
|---|---|
| Platform | XikeStor SKS8300-8X; the board itself is branded ONTi ONT-S508CL-8S |
| SoC | Realtek RTL9303 (MIPS 34Kc) |
| Memory | 512 MB |
| Storage | 32 MiB SPI NOR (`spi0.0`) |
| Network | 8×SFP+ (`lan1`…`lan8`) |
## Role
`portcullis`'s 10G switch. Nothing else depends on it, and it is not part of the home fabric — it
is expected to travel to Nikhef with `portcullis` rather than stay behind.
While staged at home it hangs off jim's spare SFP+ port, so `portcullis` can reach the home `hi`
VLAN over 10G: `lan1` uplinks to jim's `sfp-spare`, `lan2` goes to `portcullis`, and the other six
cages are empty. See [the home switches](../home/switches.md) for the fabric it borrows.
## Network assignments
fergal has no assignments — it is not managed by the flake. Its management address is
`192.168.64.30` on the home `core` VLAN, set in UCI as `network.lan`, with no DNS record; reach it
as `ssh root@192.168.64.30`.
## VLAN configuration
One bridge (`switch`), with VLAN 1 as the untagged PVID on every port — that's the native VLAN on
jim's `sfp-spare`, and `switch.1` is where fergal's own management address lives. `hi` (100) and
`lo` (110) are **tagged** members of every port, so a box on any cage can pick them up:
```
uci show network | grep bridge-vlan
```
Tagging all eight rather than just `lan1`/`lan2` keeps a spare cage usable without a reconfigure;
there is nothing sensitive behind it while fergal is on the bench.
**Jumbo frames pass, despite what `ip link` says.** Every DSA port and the `switch` bridge read
`mtu 1500`, but the RTL9303 forwards between ports in hardware and isn't bound by those — a
`ping -M do -s 8972` from `portcullis` to the `hi` VIP crosses fergal intact, which is what makes
the 9000-MTU `hi` VLAN usable over this path. The 1500 does apply to traffic punted to the CPU,
i.e. fergal's own management on `switch.1`.
## Firmware
The image is built by this flake — see [OpenWrt images](../../deployment.md#openwrt-images) for the
outputs and the feed pin. Packages are baked into the image, so adding tooling means editing
[`openwrt/default.nix`](../../../openwrt/default.nix) and reflashing rather than installing on the
box.
### Flash layout
A single 32 MiB SPI NOR chip (`spi0.0`, 64 KiB erase blocks). `kernel` and `rootfs` are
sub-partitions of `firmware`, and OpenWrt adds `rootfs_data` as the JFFS2 overlay after a real
flash.
| Partition | Device | Offset | Size |
|---|---|---|---|
| `u-boot` | `mtd0` | `0x000000` | 1 MiB |
| `board-info` | `mtd1` | `0x100000` | 192 KiB |
| `syslog` | `mtd2` | `0x130000` | 832 KiB |
| `firmware` | `mtd3` | `0x200000` | 30 MiB |
**`board-info` is irreplaceable.** It holds the unit's MAC addresses (`[vlanmac]` / `[cpumac]`), its
`[license]` hash, the stock boot pointers and an SSH host key — only about 1.3 KiB of it is
non-blank, and none of it can be regenerated. A full dump of all four partitions, taken before
OpenWrt was flashed, is kept outside this repo — 33 MB of images, with per-partition checksums and
restore notes. Never write `u-boot` or `board-info` without a confirmed serial/TFTP recovery path.
### Flashing notes
The procedure itself is in [`openwrt-flash.md`](../../openwrt-flash.md); what follows is specific to
this board.
Stock u-boot boots `flash:/nos.img` from a JFFS2 filesystem, so OpenWrt's sysupgrade image is
itself a JFFS2 image containing `nos.img` rather than a raw kernel + squashfs. Two things bite when
flashing from an initramfs, as during the initial install:
- **`sysupgrade -c` does not work.** It needs `/overlay/upper/etc`, which doesn't exist when running
from RAM, and it aborts *after* `mtd erase firmware` has already run — leaving the box with no
bootable firmware until the job is finished. Pass the config as an explicit tarball instead
(`tar czf`, then `sysupgrade -f <tarball> …`).
- **The working management address may not be in UCI.** If it was set by hand with `ip` while UCI
still held the stock address, the box comes back unreachable. Write it into `network.lan` and
commit before flashing.
Neither applies to an ordinary flash-to-flash upgrade, where `sysupgrade` keeps `/etc/config` and
the files listed in `/lib/upgrade/keep.d/` by default. Dropbear host keys are regenerated by a flash
that doesn't preserve them, so clear the old `known_hosts` entry afterwards.
## Notable config files
- [`openwrt/default.nix`](../../../openwrt/default.nix) — image definition and baked-in package list.
docs/sites/colony/git.md
+29 -33
View File
@@ -9,42 +9,38 @@ The Gitea VM — source hosting and CI for the boxes (`git.nul.ie`).
## Role
### Gitea
The Git forge at `git.nul.ie` (self-registration disabled), configured in
[`gitea.nix`](../../../nixos/boxes/colony/vms/git/gitea.nix).
- Backed by PostgreSQL on [`colony-psql`](shill/containers/colony-psql.md) (waited on via
`lib.my.systemdAwaitPostgres`).
- LFS enabled; all object storage (incl. LFS and packages) is on MinIO at `s3.nul.ie` (bucket
`gitea`, on [`object`](shill/containers/object.md)) — the secret is spliced into `app.ini` at
startup.
- Mail goes out via `mail.nul.ie`, including the issue-reply incoming-mail poller.
### Gitea Actions runner
One Docker-mode instance (`main-docker`) on podman (privileged, `podman` network), configured in
[`gitea-actions.nix`](../../../nixos/boxes/colony/vms/git/gitea-actions.nix).
- Labels for `node:24-trixie` and the self-built `git.nul.ie/dev/actions-ubuntu:26.04` images.
- Runs as a fixed `gitea-runner` user (not `DynamicUser`) so it can read its token; jobs have a
configured timeout.
- The action cache lives on a dedicated disk (`/var/cache/gitea-runner`).
- Executes the repo's own `.gitea/workflows/ci.yaml`.
### nginx
Terminates TLS for `git.nul.ie` (and a default vhost) and proxies to Gitea on `:3000`. ACME
(Let's Encrypt, production) issues `nul.ie` + `*.nul.ie` via the Cloudflare DNS-01 challenge.
### podman
Local container backend for the runner; `/var/lib/containers` is an XFS data disk, and the default
`10.88.0.0/16` podman subnet is allowed to forward.
- **Gitea** ([`gitea.nix`](../../../nixos/boxes/colony/vms/git/gitea.nix)) — the
Git forge at `git.nul.ie` (self-registration disabled). Backed by PostgreSQL
on `colony-psql` (waiting on it via `lib.my.systemdAwaitPostgres`), LFS
enabled, with all object storage (incl. LFS and packages) on MinIO at
`s3.nul.ie` (bucket `gitea`, on `object`; the secret is spliced into
`app.ini` at startup). Mail goes out via `mail.nul.ie`, including the
issue-reply incoming-mail poller.
- **Gitea Actions runner**
([`gitea-actions.nix`](../../../nixos/boxes/colony/vms/git/gitea-actions.nix)) —
one Docker-mode instance (`main-docker`) on podman (privileged, `podman`
network), with labels for `node:24-trixie` and the self-built
`git.nul.ie/dev/actions-ubuntu:26.04` images. Runs as a fixed
`gitea-runner` user (not `DynamicUser`) so it can read its token; 8 h job
timeout; the action cache lives on a dedicated disk
(`/var/cache/gitea-runner`). This runner executes the repo's own
`.gitea/workflows/ci.yaml`.
- **nginx** — terminates TLS for `git.nul.ie` (and a default vhost) and
proxies to Gitea on `:3000`. ACME (Let's Encrypt, production) issues
`nul.ie` + `*.nul.ie` via the Cloudflare DNS-01 challenge.
- **podman** — local container backend for the runner;
`/var/lib/containers` is an XFS data disk, and the default
`10.88.0.0/16` podman subnet is allowed to forward.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `git`).
<!-- assignments: git -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| git-vm | internal | `94.142.241.117/32` | `2a0e:97c0:4d2:11::4/64` | ams1.int.nul.ie | |
| git-vm-routing | routing | `10.100.1.4/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
<!-- assignments-end -->
## Storage
docs/sites/colony/mail.md
+8 -17
View File
@@ -9,26 +9,22 @@ a NixOS system**: everything inside the VM is configured out of band.
and the `90-vm-mail` network in
[`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
- **Host:** VM on `colony`
- **nixpkgs:** not applicable (Debian guest; host-side definition uses `colony`'s `mine-stable`)
## Role
- Runs the full mailcow stack (Postfix/Dovecot/SOGo/Rspamd) for `nul.ie`.
Other colony services send through it as `mail.nul.ie` (e.g. Gitea, and the
disabled Mastodon config).
- `root` and `data` LVM disks; the `vm-mail-data` LV is included in `colony`'s `borgthin` backups.
- 3 cores, 6 GiB RAM; `root` and `data` LVM disks (the `vm-mail-data` LV is
included in `colony`'s `borgthin` backups).
## Network assignments
This guest is not a NixOS system, so its host-routed addresses are not rows in the generated
[network assignments](../../networking.md#box-assignments) table.
### Link and addressing
The VM attaches to a dedicated, unbridged TAP (`vm-mail`). `colony` puts the point-to-point
`custRouting.mail-vm` address on the host side, link-routes public
`94.142.241.227/32` down the TAP, and advertises `2a0e:97c0:4d2:2000::/64` with RAs.
## Networking
- Attaches to a dedicated, unbridged TAP (`vm-mail`, MAC
`52:54:00:a8:d1:03`). `colony` puts the point-to-point address
(`custRouting.mail-vm`) on the host side, link-routes the public /32
`94.142.241.227` down the tap, and serves the IPv6 /64
`2a0e:97c0:4d2:2000::/64` with RAs.
- DNS: `mail-vm.ams1.int.nul.ie` (and `mail.nul.ie` publicly, incl. the PTR in
estuary's reverse zone). `estuary` accepts traffic to the customer prefixes
without per-port filtering; `colony` forwards it on ("trust for now").
@@ -38,8 +34,3 @@ The VM attaches to a dedicated, unbridged TAP (`vm-mail`). `colony` puts the poi
- ACME certificates are issued on `middleman` and pushed to the VM over SSH
(`acme@mail.nul.ie mailcow-ssl-reload`, key `middleman/mailcow-ssh.key`);
the VM's SSH host key is pinned at `.keys/mail-vm-host.pub`.
## Notable config files
- [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix) — VM definition.
- [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix) — host-side network and routing.
docs/sites/colony/portcullis.md
-120
View File
@@ -1,120 +0,0 @@
# portcullis
A bare-metal box destined for Nikhef, intended to take over most of the colony edge
routing currently done by the [`estuary`](estuary.md) VM.
- **Source:** [`nixos/boxes/colony/portcullis/`](../../../nixos/boxes/colony/portcullis)
- **Host:** bare metal
- **nixpkgs:** `mine-stable`
## Hardware
| Component | Inventory |
|---|---|
| Platform | Mini PC (no vendor DMI strings) |
| CPU | Intel N150 (4 cores / 4 threads) |
| Memory | 8 GiB |
| Storage | One 128 GB NVMe SSD (`nvme0n1`), partitioned as a 2 GiB ESP plus an LVM PV holding the `nix` and `persist` volumes |
| Network | Four Intel I226-V 2.5 GbE ports (`et2g5-0`…`et2g5-3`) and one dual-port Intel 82599ES 10 GbE SFP+ card (`et10g-0`, `et10g-1`) |
| Management | JetKVM (HDMI/USB KVM with virtual media) |
The PCIe layout constrains what the cards can reach. The 82599ES sits behind a gen2 x4 link giving
16 Gb/s for **both** its ports together, so one port runs at line rate but the pair is
oversubscribed. The NVMe is on a x1 root port, capped near 7.9 Gb/s regardless of the drive. Each
I226-V has its own x1 link and is not constrained.
## Role
Not yet in service. The eventual job is to be the physical edge for the colony site at Nikhef,
taking over most of what `estuary` does today — WAN termination, firewalling and NAT, BGP for
AS211024 and DNS. Some of that functionality stays on `estuary`, and the surrounding network
topology will change with the move, so the split is not settled yet. Until it is, the config in
this repository covers only what is needed to boot and reach the box.
## Network assignments
`portcullis` has no colony assignments yet — those land alongside the routing config once the
topology is decided. While it is staged at home it holds a single home `hi` assignment, listed in
[`networking.md#box-assignments`](../../networking.md#box-assignments).
## Networking
- The four I226-V ports are named `et2g5-0`…`et2g5-3` and the 82599ES SFP+ ports `et10g-0` /
`et10g-1`, pinned by permanent MAC address in `.link` files.
- Bootstrap: a single `.network` matches every `et2g5-*` port and takes DHCP on the home `lo` VLAN,
so whichever port happens to be patched in brings the box up. `wait-online.anyInterface` keeps
boot from blocking on the unpatched ports.
- kea registers the DHCP hostname, so while staged the box also answers to `portcullis.dyn.h.nul.ie`.
- `my.deploy.node.hostname` is the `hi` address, taken from the assignment rather than written out,
since there is no colony FQDN for the box yet.
### 10G to the home `hi` VLAN
`et10g-0` runs over fibre to [`fergal`](fergal.md), which uplinks to jim's `sfp-spare` port. That
uplink is untagged VLAN 1, so `hi` is carried tagged on a `lan-hi` VLAN interface rather than on the
port itself; the physical link takes the `hi` jumbo MTU so the whole path is consistent with the
rest of the VLAN. `lan-hi` carries the static assignment, resolves through the router VIPs like
every other `hi` client, and its gateway route outranks the DHCP default, so the 10G path is
preferred while the 2.5G one stays as a fallback.
Both jim and `fergal` tag `hi` and `lo` along that path. It exists only while the box is staged at
home — `fergal` goes to Nikhef with it.
The other SFP+ port, `et10g-1`, is unused.
### Interface tuning
Every port takes router-sized 4096-entry rings rather than the driver defaults, matching the other
routers here, and enables `GenericReceiveOffloadUDPForwarding` so GRO batching survives forwarding
once the box carries UDP-encapsulated traffic. Both are `.link` settings, so they apply on the next
device event rather than at switch time — a reboot is the reliable way to land a change to them.
Interrupt coalescing is deliberately left alone. `igc` reports `rx-usecs` 3 and `ixgbe` reports 1,
which are the drivers' markers for dynamic ITR rather than literal microseconds; writing a
plausible-looking value there replaces adaptive moderation with a fixed one.
### I226-V erratum
The I226-V link-drop erratum is driven by PCIe ASPM, Energy Efficient Ethernet and stale NIC
firmware. ASPM, the dominant cause, is off across the whole box for the reason in
[Power](#power) below. EEE is held off by a udev rule invoking `ethtool`, as `systemd.link` has no
knob for it. `igc` already leaves EEE off on these ports, so the rule pins a driver default rather
than correcting one, and keeps it from drifting on a kernel bump. Firmware is the remaining item:
the ports report NVM `2.13` (EEPROM version word `0x2013`, which `igc` prints as the `2013` in
`ethtool -i`), behind the `2.29`/`2.32` images that circulate. Intel does not publish the I226-V
NVM image, so updating means third-party firmware and is best attempted while the box is at home
and the JetKVM is attached.
## Power
The SoC side is already at its floor and needs no tuning: the package draws around 0.75 W idle with
cores in C10 essentially all the time, under `intel_pstate` on the `powersave` governor.
Platform idle is capped instead, and deliberately left that way. The ACPI FADT declares that the
system does not support PCIe ASPM, so the OS defers to firmware, every root port advertises ASPM as
unsupported and every endpoint sits with it disabled. Deep package C-states need every PCIe link in
L1, so the package never leaves C3. `pcie_aspm=force` is the usual answer and is **not** used here:
the 82599ES advertises only L0s with an unlimited exit latency, so no amount of forcing reaches the
deep states while that card is fitted, and the only links it would actually change are the four
I226-V ones — the exact configuration behind the erratum above. Recovering that power is a firmware
question for the mini PC, and only worthwhile once the 82599ES is gone.
`iommu=pt` puts host devices in passthrough so the forwarding path does not pay DMA translation,
while leaving the IOMMU available.
## Storage
A single NVMe SSD, following the usual tmpfs-root layout: a 2 GiB ESP at `/boot`, then one LVM PV
in volume group `main` carrying `portcullis-nix` (48 GiB, `/nix`) and `portcullis-persist` (the
remainder, `/persist`).
## Secrets
`my.secrets.key` is the SSH host key adopted from the installer session at install time (seeded onto
the persist volume before first boot), so secrets could be encrypted for the box without waiting for
it to come up. The box declares nothing of its own yet — only the default `user-passwd.txt` that
`my.user` brings in.
## Notable config files
- [`nixos/boxes/colony/portcullis/default.nix`](../../../nixos/boxes/colony/portcullis/default.nix) — hardware, filesystems and bootstrap networking.
docs/sites/colony/shill.md
+83
View File
@@ -0,0 +1,83 @@
# shill
The colony NixOS container host — most colony applications run as
`systemd-nspawn` containers on this VM.
- **Source:** [`nixos/boxes/colony/vms/shill/`](../../../nixos/boxes/colony/vms/shill)
(`default.nix`, `containers-ext.nix`, `containers/`)
- **Host:** VM on `colony` (large: 12 cores, 40 GiB RAM)
- **nixpkgs:** `mine`
## Role
- Runs the colony containers via `my.containers.instances`, each attached to
the `ctrs` bridge with its own address. The containers are full NixOS
systems rendered via `my.asContainer` and deployed as container profiles on
`shill` (`my.deploy.enable = false` — they are not standalone deploy
targets); the `my.containers` module wires up the nspawn units, `/persist`
bind mounts and store binds.
- Provides shared data volumes to containers via bind mounts from LVM-backed
disks: `/mnt/media` (→ `middleman` read-only, `jackflix` read-write),
`/mnt/minio` and `/mnt/nix-cache` (→ `object`, both read-write).
- Routes between the `vms` network and the `ctrs` container network: sends RAs
on `ctrs` (DNS = `estuary`'s base address) and routes the Tailscale prefixes
via `waffletail` and the `qclk` prefix via `qclk`. Applies the shared
`firewallForwards` DNAT for traffic addressed to `estuary`'s public IP, with
an nftables `ct mark 0x1337` SNAT hack so forwarded return traffic stays
symmetric.
- Tuned for high connection counts (larger conntrack table, wider ephemeral
port range); netdata on 19999.
## Network assignments
<!-- assignments: shill -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| shill-vm-ctrs | ctrs | `10.100.2.1/24` | `2a0e:97c0:4d2:12::1/64` | ams1.int.nul.ie | |
| shill-vm (ctr) | internal | `94.142.241.225/32` | `2a0e:97c0:4d2:11::2/64` | ams1.int.nul.ie | |
| shill-vm-routing | routing | `10.100.1.2/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
<!-- assignments-end -->
## Containers
Defined under
[`shill/containers/`](../../../nixos/boxes/colony/vms/shill/containers) and
wired up in `shill`'s `my.containers.instances`. Each has its own page:
| Container | IPv4 | IPv6 | Role | Page |
|---|---|---|---|---|
| `middleman` | `10.100.2.2` | `2a0e:97c0:4d2:12::2` | Reverse proxy, ACME, nginx-sso, librespeed | [middleman](shill/containers/middleman.md) |
| `vaultwarden` | `10.100.2.3` | `2a0e:97c0:4d2:12::3` | Password manager | [vaultwarden](shill/containers/vaultwarden.md) |
| `colony-psql` | `10.100.2.4` | `2a0e:97c0:4d2:12::4` | Shared PostgreSQL (14) | [colony-psql](shill/containers/colony-psql.md) |
| `chatterbox` | `10.100.2.5` | `2a0e:97c0:4d2:12::5` | Matrix Synapse + bridges | [chatterbox](shill/containers/chatterbox.md) |
| `jackflix` | `10.100.2.6` | `2a0e:97c0:4d2:12::6` | Media stack | [jackflix](shill/containers/jackflix.md) |
| `object` | `10.100.2.7` | `2a0e:97c0:4d2:12::7` | MinIO, Harmonia Nix cache, Sharry, HedgeDoc, wastebin | [object](shill/containers/object.md) |
| `toot` | `10.100.2.8` | `2a0e:97c0:4d2:12::8` | Bluesky PDS (Mastodon disabled) | [toot](shill/containers/toot.md) |
| `waffletail` | `10.100.2.9` | `2a0e:97c0:4d2:12::9` | Tailscale subnet router / exit node | [waffletail](shill/containers/waffletail.md) |
| `qclk` | `10.100.2.10` | `2a0e:97c0:4d2:12::a` | WireGuard management appliance | [qclk](shill/containers/qclk.md) |
| `gam` | `10.100.2.11` | `2a0e:97c0:4d2:12::b` | Terraria server | [gam](shill/containers/gam.md) |
### `jam`
A one-off: [`containers-ext.nix`](../../../nixos/boxes/colony/vms/shill/containers-ext.nix)
runs a raw `systemd-nspawn` container (not a `my.containers` instance, not
NixOS) with its root on the `jam` LV, private user namespaces and a `ve-jam`
veth. It gets the `jam` customer prefix (`prefixes.jam`, `jam-cust` in DNS)
and SSH is forwarded to it from `shill`'s public IP port 60022.
## Notes
- `nix.settings.substituters` is forced to just `https://cache.nixos.org` —
`shill` sits next to the S3 cache on `object`, so it doesn't use it.
- [`hercules.nix`](../../../nixos/boxes/colony/vms/shill/hercules.nix)
(Hercules CI agent + the `nix-cache-gc` timer for the S3 binary cache)
exists but is **currently disabled**: the file is not imported by
`shill/default.nix`.
## Notable config files
- [`nixos/boxes/colony/vms/shill/default.nix`](../../../nixos/boxes/colony/vms/shill/default.nix) — VM config, networkd, firewall, `my.containers.instances`.
- [`nixos/boxes/colony/vms/shill/containers/default.nix`](../../../nixos/boxes/colony/vms/shill/containers/default.nix) — container imports.
- [`nixos/boxes/colony/vms/shill/containers-ext.nix`](../../../nixos/boxes/colony/vms/shill/containers-ext.nix) — the `jam` nspawn container.
- [`nixos/modules/containers.nix`](../../../nixos/modules/containers.nix) — the `my.containers` module.
docs/sites/colony/shill/README.md
-84
View File
@@ -1,84 +0,0 @@
# shill
The colony NixOS container host — most colony applications run as
`systemd-nspawn` containers on this VM.
- **Source:** [`nixos/boxes/colony/vms/shill/`](../../../../nixos/boxes/colony/vms/shill)
(`default.nix`, `containers-ext.nix`, `containers/`)
- **Host:** VM on `colony`
- **nixpkgs:** `mine`
## Role
### Container hosting
`my.containers.instances` runs the colony containers on the `ctrs` bridge. Each is a full NixOS
system rendered through `my.asContainer` and deployed as a profile on `shill`; containers are not
standalone deploy targets. The shared container module supplies the nspawn units, `/persist` and
store binds.
### Shared storage
LVM-backed host volumes are bind-mounted into the consumers: `/mnt/media` is read-only in
`middleman` and read-write in `jackflix`; `/mnt/minio` and `/mnt/nix-cache` are read-write in
`object`.
### Routing
`shill` routes between `vms` and `ctrs`, advertises `estuary` as DNS on `ctrs`, and routes Tailscale
through `waffletail` and the `qclk` prefix through `qclk`. It applies the shared `firewallForwards`
DNAT for `estuary`'s public IP; a connection-mark-based SNAT rule keeps replies symmetric.
### Host tuning
The box has a larger conntrack table and ephemeral-port range for high connection counts. Netdata
listens on port 19999.
## Network assignments
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `shill`).
## Containers
Defined under
[`shill/containers/`](../../../../nixos/boxes/colony/vms/shill/containers) and
wired up in `shill`'s `my.containers.instances`. The generated
[network assignments](../../../networking.md#box-assignments) table is the source of truth for
their current addresses. Each container has its own page:
| Container | Role |
|---|---|
| [`middleman`](containers/middleman.md) | Reverse proxy, ACME, nginx-sso, librespeed |
| [`vaultwarden`](containers/vaultwarden.md) | Password manager |
| [`colony-psql`](containers/colony-psql.md) | Shared PostgreSQL (14) |
| [`chatterbox`](containers/chatterbox.md) | Matrix Synapse + bridges |
| [`jackflix`](containers/jackflix.md) | Media stack |
| [`object`](containers/object.md) | MinIO, Harmonia Nix cache, HedgeDoc, wastebin |
| [`toot`](containers/toot.md) | Bluesky PDS (Mastodon disabled) |
| [`waffletail`](containers/waffletail.md) | Tailscale subnet router / exit node |
| [`qclk`](containers/qclk.md) | WireGuard management appliance |
| [`gam`](containers/gam.md) | Terraria server |
### `jam`
A one-off: [`containers-ext.nix`](../../../../nixos/boxes/colony/vms/shill/containers-ext.nix)
runs a raw `systemd-nspawn` container (not a `my.containers` instance, not
NixOS) with its root on the `jam` LV, private user namespaces and a `ve-jam`
veth. It gets the `jam` customer prefix (`prefixes.jam`, `jam-cust` in DNS)
and SSH is forwarded to it from `shill`'s public IP port 60022.
## Notes
- `nix.settings.substituters` is forced to just `https://cache.nixos.org` —
`shill` sits next to the S3 cache on `object`, so it doesn't use it.
- [`hercules.nix`](../../../../nixos/boxes/colony/vms/shill/hercules.nix)
(Hercules CI agent + the `nix-cache-gc` timer for the S3 binary cache)
exists but is **currently disabled**: the file is not imported by
`shill/default.nix`.
## Notable config files
- [`nixos/boxes/colony/vms/shill/default.nix`](../../../../nixos/boxes/colony/vms/shill/default.nix) — VM config, networkd, firewall, `my.containers.instances`.
- [`nixos/boxes/colony/vms/shill/containers/default.nix`](../../../../nixos/boxes/colony/vms/shill/containers/default.nix) — container imports.
- [`nixos/boxes/colony/vms/shill/containers-ext.nix`](../../../../nixos/boxes/colony/vms/shill/containers-ext.nix) — the `jam` nspawn container.
- [`nixos/modules/containers.nix`](../../../../nixos/modules/containers.nix) — the `my.containers` module.
docs/sites/colony/shill/containers/chatterbox.md
+20 -16
View File
@@ -5,28 +5,32 @@ The Matrix homeserver for `nul.ie` (Synapse) and its bridges to other chat netwo
federation.
- **Source:** [`shill/containers/chatterbox.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/chatterbox.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
- **Host:** NixOS container on [`shill`](../../shill.md)
## Role
### Synapse
`matrix-synapse` serves `nul.ie` at `https://matrix.nul.ie`, with Element at `element.nul.ie`.
Port 8008 carries client and federation traffic with forwarded headers, while a localhost manhole
uses port 9000. Registration and guest access are disabled; uploads, dynamic thumbnails and URL
previews are enabled, with preview fetching restricted to [middleman](middleman.md).
### Bridges
The box runs heisenbridge for IRC, `mautrix-whatsapp`, and two `mautrix-meta` instances for
Messenger and Instagram. The mautrix bridges use [colony-psql](colony-psql.md), default to
end-to-end encryption, and share the secret `doublepuppet.yaml` registration for double puppeting
onto `nul.ie`. The firewall admits ports 8008 and 8009 in addition to netdata.
- **matrix-synapse** — `server_name = "nul.ie"`, `public_baseurl = https://matrix.nul.ie`,
Element at `element.nul.ie` as the web client. Listens on `[::]:8008` (client + federation
resources, `x_forwarded`) with a localhost manhole on `:9000`. Registration and guest access
are disabled; uploads up to 1024M with dynamic thumbnails and URL previews enabled (previews
are limited to [middleman](middleman.md)'s addresses as the fetch proxy).
- **heisenbridge** — IRC bridge, owner `@dev:nul.ie`, exclusive `@irc_*` user namespace.
- **mautrix-whatsapp** — WhatsApp bridge (appservice `whatsapp2`, `!wa` commands).
- **mautrix-meta** — two instances, `messenger` (`fbm2_*`, `!fbm`) and `instagram` (`ig_*`,
`!ig`), both with backfill enabled.
- All three mautrix bridges use Postgres on [colony-psql](colony-psql.md) (URIs in their secret
env files), require end-to-end encryption by default, and double-puppet onto `nul.ie` via the
shared `doublepuppet.yaml` appservice registration (an age secret).
- The firewall allows `8008` (Synapse) and `8009` besides netdata.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `chatterbox`).
<!-- assignments: chatterbox -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| chatterbox-ctr | internal | `10.100.2.5/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::5/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
## Notes
docs/sites/colony/shill/containers/colony-psql.md
+11 -7
View File
@@ -4,30 +4,34 @@ The shared PostgreSQL instance for colony services. Rather than each service run
database, the containers (and the `git` VM) connect here over the `ctrs` network.
- **Source:** [`shill/containers/colony-psql.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/colony-psql.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
- **Host:** NixOS container on [`shill`](../../shill.md)
## Role
- **PostgreSQL 14** with TCP/IP enabled, reachable from the whole colony address space using `md5`
authentication. The firewall allows `5432`.
- **PostgreSQL 14** with TCP/IP enabled, reachable from the whole colony (`10.100.0.0/16` and
`2a0e:97c0:4d2:10::/60`, `md5` auth). The firewall allows `5432`.
- Local `peer` auth maps `postgres`, `root`, `netdata` and `dev` to the `postgres` superuser via
the ident map.
- **netdata** with the Python PostgreSQL collector.
- Consumers wait for the database to accept connections with the `lib.my.systemdAwaitPostgres`
helper (e.g. `atticd`, `mastodon-init-db`, and `middleman`'s nginx as a DNS
helper (e.g. `sharry`, `atticd`, `mastodon-init-db`, and `middleman`'s nginx as a DNS
bootstrap hack).
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `colony-psql`).
<!-- assignments: colony-psql -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| colony-psql-ctr (colony-psql) | internal | `10.100.2.4/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::4/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
The assignment also has the alt name `colony-psql` (no `-ctr` suffix), which is what consumers
use as the database hostname.
## Consumers
- [object](object.md) — `hedgedoc` (and `atticd` when enabled) over
- [object](object.md) — `sharry` and `hedgedoc` (and `atticd` when enabled) over
`colony-psql:5432`
- [toot](toot.md) — Mastodon's database (Mastodon currently disabled)
- [chatterbox](chatterbox.md) — the mautrix bridges (WhatsApp, Messenger, Instagram) via
docs/sites/colony/shill/containers/gam.md
+13 -15
View File
@@ -1,28 +1,26 @@
# gam
A game-server container currently dedicated to Terraria.
A game-server container — the lightweight counterpart to the OCI game servers on `whale2`,
running servers directly as NixOS services. Currently it runs a single Terraria server.
- **Source:** [`shill/containers/gam.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/gam.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
- **Host:** NixOS container on [`shill`](../../shill.md)
## Role
Runs lightweight game servers directly as NixOS services, rather than as OCI containers on
[`whale2`](../../whale2.md).
- **terraria** — dedicated server (`services.terraria`): world at
`/var/lib/terraria/NotWorld.wld`, auto-created large worlds, MOTD "sup gamers", UPnP off.
Extra settings (e.g. password) come from the `gam/terraria.conf` age secret used as the config
file. `openFirewall` is on, and `estuary` port-forwards TCP and UDP `7777` to this container.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `gam`).
## Terraria server
`services.terraria` runs the dedicated server with its world at
`/var/lib/terraria/NotWorld.wld`. It creates large worlds automatically, uses the MOTD
"sup gamers", and disables UPnP. Additional settings such as the password come from the
`gam/terraria.conf` age secret used as the service configuration file.
`openFirewall` is enabled, and `estuary` forwards TCP and UDP port `7777` to the container.
<!-- assignments: gam -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| gam-ctr | internal | `10.100.2.11/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::b/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
## Notable config files
docs/sites/colony/shill/containers/jackflix.md
+19 -13
View File
@@ -5,8 +5,7 @@ an AirVPN WireGuard tunnel so downloads only flow while the VPN is up.
- **Source:** [`shill/containers/jackflix/`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix)
(`default.nix`, `networking.nix`)
- **Host:** NixOS container on [`shill`](../README.md) (bind-mounts `/mnt/media` read-write)
- **nixpkgs:** `mine`
- **Host:** NixOS container on [`shill`](../../shill.md) (bind-mounts `/mnt/media` read-write)
## Role
@@ -22,33 +21,40 @@ an AirVPN WireGuard tunnel so downloads only flow while the VPN is up.
| PhotoPrism | `2342` | photos, `photos.nul.ie`; password auth, sqlite DB, originals/import under `/mnt/media/photoprism` |
| copyparty | `3923` | file sharing, `stuff.nul.ie`; serves `/mnt/media/public` (read-only to everyone) and `/priv` → `/mnt/media/stuff` (admin for `dev`), share creation, indexing (`e2dsa`/`e2t`), file-magic checks |
All published through [middleman](middleman.md) as shown. A shared `media` group plus a group-writable
umask on Radarr/Sonarr gives the apps coordinated access to the media volume.
All published through [middleman](middleman.md) as shown. A shared `media` group (gid 2000) plus
`UMask=0002` on Radarr/Sonarr gives the apps coordinated access to the media volume.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `jackflix`).
<!-- assignments: jackflix -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| jackflix-ctr | internal | `10.100.2.6/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::6/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
## VPN download path
[`networking.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix/networking.nix)
defines a `vpn` WireGuard netdev to **AirVPN NL** using a key and PSK from age secrets:
defines a `vpn` WireGuard netdev to **AirVPN NL** (key + PSK from age secrets, MTU 1320 per
AirVPN, fwmark `42`, route table `51820`):
- Policy routing keeps colony traffic on the main table while everything else falls through to the
VPN's dedicated table, so the services stay reachable on the `ctrs` network while outbound
torrent traffic exits via AirVPN. `DNSDefaultRoute` is disabled on `host0`; the VPN provides DNS.
- Policy routing keeps colony traffic on the main table (from/to `prefixes.all` v4/v6 rules at
priority 100, plus a default-route suppression) while everything else falls through to the VPN
table — so the services stay reachable on the `ctrs` network while outbound torrent traffic
exits via AirVPN. `DNSDefaultRoute` is disabled on `host0`; the VPN provides DNS.
- `transmission` and `jackett` `bindsTo` `systemd-networkd-wait-online@vpn.service` — they only
run while the tunnel is up.
- AirVPN forwards a peer port to Transmission (`peer-port`); the firewall accepts it and drops
other new inbound TCP from `vpn`, while non-VPN input is limited to the service ports
- AirVPN forwards peer port `28457` to Transmission (`peer-port`); the firewall accepts it and
drops other new inbound TCP from `vpn`, while non-VPN input is limited to the service ports
(netdata, Transmission, Jackett, Radarr, Sonarr, Jellyfin, PhotoPrism) plus copyparty's `3923`
from the base config and Jellyseerr's `5055`.
## Storage
Media lives on the shared `/mnt/media` volume (bind-mounted read-write from `shill`); Transmission
downloads into `/mnt/media/downloads/torrents` with a `.incomplete` directory and configured
bandwidth and seed-ratio limits.
downloads into `/mnt/media/downloads/torrents` with a `.incomplete` dir, 28 MB/s up/down limits
and a seed ratio limit of 2.0.
## Notable config files
docs/sites/colony/shill/containers/middleman.md
+35 -36
View File
@@ -7,44 +7,42 @@ backend.
- **Source:** [`shill/containers/middleman/`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman)
(`default.nix`, `vhosts.nix`)
- **Host:** NixOS container on [`shill`](../README.md) (`my.containers` ephemeral nspawn on the
- **Host:** NixOS container on [`shill`](../../shill.md) (`my.containers` ephemeral nspawn on the
`ctrs` bridge; bind-mounts `/mnt/media` read-only for the static file vhosts)
- **nixpkgs:** `mine`
## Role
### nginx
The reverse proxy enables `vts`, `fancyindex`, Brotli, kTLS and a proxy cache. Its dynamic resolver
points at `estuary`, allowing upstreams named under `ams1.int.nul.ie` to resolve again at runtime.
All vhosts live in
[`vhosts.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix). nginx
also waits for `colony-psql` through `systemdAwaitPostgres`, avoiding an early-boot DNS stall.
### ACME
`middleman` issues certificates for its own vhosts; it is not a shared CA for other boxes.
- `ams1.int.nul.ie` and its wildcard use a lego `exec` challenge that SSHes to
`pdns-file-records@estuary-vm`. This is the default `useACMEHost` certificate internally.
- `nul.ie`, `*.nul.ie` and `*.s3.nul.ie` use Cloudflare DNS. A `postRun` hook copies renewed
material to the `mail` VM and runs `mailcow-ssl-reload` there.
- Renewal reloads nginx; the `acme` group owns the secret files and includes the nginx user.
### nginx-sso
The `generic` SSO instance at `sso.nul.ie` uses Google OAuth by default and also offers a simple
username/password provider. Its cookie domain is `.nul.ie`; gated vhosts include the generated
`server-generic.conf` / `location-generic.conf` snippets from `/etc/nginx/includes/sso/`.
### librespeed
The frontend and backend are published as `speed.nul.ie` and `librespeed.ams1.int.nul.ie`, both
proxied to `localhost:8989`.
- **nginx** reverse proxy with the `vts` and `fancyindex` modules, Brotli, kTLS, a proxy cache,
and a dynamic resolver pointed at `estuary` (`proxyResolveWhileRunning`) so upstreams re-resolve
at runtime. All vhosts are declared in
[`vhosts.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix);
upstreams are addressed by their internal container/VM hostnames under `ams1.int.nul.ie`.
nginx is made to wait for `colony-psql` (`systemdAwaitPostgres`) before starting — a hack for
it getting stuck on DNS early in boot.
- **ACME** — issues the wildcard certificates its own vhosts are served with (it is not a shared
CA for the other boxes):
- `ams1.int.nul.ie` + `*.ams1.int.nul.ie` via a lego `exec` DNS challenge that SSHes to
`pdns-file-records@estuary-vm` to add/remove records in the internal PowerDNS zone. This is
the default cert (`useACMEHost`) for vhosts on the internal domain.
- `nul.ie` + `*.nul.ie` + `*.s3.nul.ie` via the Cloudflare DNS provider. A `postRun` hook
`scp`s the renewed key/cert to the `mail` (Mailcow) VM and runs `mailcow-ssl-reload` there.
- Certs reload `nginx` on renewal; the `acme` group owns the secret files and `nginx` is a
member.
- **nginx-sso** — single sign-on at `sso.nul.ie` (one instance, `generic`), with Google OAuth as
the default method and a simple username/password provider. Cookie domain is `.nul.ie`.
SSO-gated vhosts below include `server-generic.conf` / `location-generic.conf` from
`/etc/nginx/includes/sso/`.
- **librespeed** — speed test, backend + frontend, published as `speed.nul.ie` and
`librespeed.ams1.int.nul.ie` (both proxy to `localhost:8989`).
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `middleman`).
<!-- assignments: middleman -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| middleman-ctr | internal | `10.100.2.2/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::2/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
The firewall allows `http`, `https` and `8448` (Matrix federation). A small nftables SNAT rule
rewrites outbound IPv6 to the container's own address on `host0`.
@@ -71,18 +69,19 @@ all vhosts are `onlySSL`, kTLS and HTTP/2. "SSO" = gated behind nginx-sso (`gene
| `jackflix.nul.ie` | `jackflix-ctr:8096` | Jellyfin; `/socket` websockets; `/` redirects to `/web/` |
| `toot.nul.ie` | `toot-ctr:80` | Mastodon — **upstream currently disabled**, see [toot](toot.md) |
| `pds.nul.ie` | `toot-ctr:3000` | Bluesky PDS ([toot](toot.md)); websockets |
| `share.nul.ie` | `object-ctr:9090` | Sharry ([object](object.md)); websockets |
| `stuff.nul.ie` | `jackflix-ctr:3923` | copyparty |
| `public.nul.ie` (+ alias `p.nul.ie`) | static `/mnt/media/public` | fancyindex file listing; `addSSL` so plain HTTP also works |
| `mc-map.nul.ie` | `simpcraft-oci:8100` | Minecraft map (OCI container on [`whale2`](../../whale2.md#game-servers)) |
| `mc-rail.nul.ie` | `simpcraft-oci:3876` | Minecraft railway map ([`whale2`](../../whale2.md#game-servers)) |
| `mc-map-kink.nul.ie` | `kinkcraft-oci:8100` | Minecraft map ([`whale2`](../../whale2.md#game-servers)) |
| `mc-map.nul.ie` | `simpcraft-oci:8100` | Minecraft map (OCI container on `whale2`) |
| `mc-rail.nul.ie` | `simpcraft-oci:3876` | Minecraft railway map |
| `mc-map-kink.nul.ie` | `kinkcraft-oci:8100` | Minecraft map |
| `speed.nul.ie` | `localhost:8989` | librespeed |
| `librespeed.ams1.int.nul.ie` | `localhost:8989` | librespeed on the internal domain (internal wildcard cert) |
| `md.nul.ie` | `object-ctr:3000` | HedgeDoc; websockets |
| `pb.nul.ie` | `object-ctr:8088` | wastebin |
| `photos.nul.ie` | `jackflix-ctr:2342` | PhotoPrism; websockets |
| `pront.nul.ie` | `stream-hi.h.nul.ie:5000` | OctoPrint on the home network ([`stream`](../../../home/stream.md)); `/webcam/` → `:5050`; **SSO** |
| `hass.nul.ie` | `hass-ctr.h.nul.ie:8123` | [Home Assistant](../../../home/sfh/containers/hass.md) (home network); websockets |
| `pront.nul.ie` | `stream-hi.h.nul.ie:5000` | OctoPrint on the home network; `/webcam/` → `:5050`; **SSO** |
| `hass.nul.ie` | `hass-ctr.h.nul.ie:8123` | Home Assistant (home network); websockets |
| `hass-john.nul.ie` | `john-valorant-tun.ams1.int.nul.ie:8123` | remote Home Assistant over the point-to-point tunnel; websockets |
| `minio.nul.ie` | `object-ctr:9001` | MinIO console; `/ws` websockets |
| `s3.nul.ie` (+ `*.s3.nul.ie`) | `object-ctr:9000` | MinIO S3 API (virtual-host style via the `*.s3` wildcard cert); `/gitea/packages/` has a hack forcing the correct `Content-Type` for Docker image manifests |
docs/sites/colony/shill/containers/object.md
+13 -7
View File
@@ -1,18 +1,19 @@
# object
Object storage and the Nix binary cache, plus HedgeDoc and wastebin.
Object storage and the Nix binary cache, plus a few small self-hosted web apps (Sharry,
HedgeDoc, wastebin).
- **Source:** [`shill/containers/object.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/object.nix)
- **Host:** NixOS container on [`shill`](../README.md) (bind-mounts `/mnt/minio` and
- **Host:** NixOS container on [`shill`](../../shill.md) (bind-mounts `/mnt/minio` and
`/mnt/nix-cache` read-write)
- **nixpkgs:** `mine`
## Role
| Service | Port | Purpose |
| --- | --- | --- |
| MinIO | `9000` (S3) / `9001` (console) | S3-compatible object storage, `s3.nul.ie` + `*.s3.nul.ie` (virtual-host style via `MINIO_DOMAIN`), console at `minio.nul.ie`; region `eu-central-1`; data on the `/mnt/minio` XFS volume |
| Harmonia | `5000` | Nix binary cache at `nix-cache.nul.ie` — `harmonia-dev` cache serves `shill`'s `/nix/store` out of a dedicated store view rooted at `/var/lib/harmonia` (bind-mounted from `/mnt/nix-cache`), signed with the `nix-cache.key` secret; a `harmonia` user with authorized keys exists for cache pushes |
| Harmonia | `5000` | Nix binary cache at `nix-cache.nul.ie` — `harmonia-dev` cache serves `shill`'s `/nix/store` out of a dedicated store view rooted at `/var/lib/harmonia` (bind-mounted from `/mnt/nix-cache`), signed with the `nix-cache.key` secret; a `harmonia` user with authorized keys exists for cache pushes; priority 30 |
| Sharry | `9090` | file sharing at `share.nul.ie`; Postgres on [colony-psql](colony-psql.md), files stored in the `share` MinIO bucket; fixed `dev` account + invite signup; mail via `mail.nul.ie`; max share size 128G |
| HedgeDoc | `3000` | collaborative markdown notes at `md.nul.ie`; Postgres on [colony-psql](colony-psql.md); anonymous edits but no anonymous notes, email login, no open email registration |
| wastebin | `8088` | pastebin at `pb.nul.ie` |
| atticd | `8069` | **currently disabled** (`services.atticd.enable = false`) — an alternative Nix cache that would store locally and sit behind `nix-cache.nul.ie`; config (including the `object/atticd.env` secret) is kept around |
@@ -23,14 +24,19 @@ Everything public is fronted by [middleman](middleman.md) (see its vhost table).
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `object`).
<!-- assignments: object -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| object-ctr | internal | `10.100.2.7/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::7/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
## Backing services
- [colony-psql](colony-psql.md) — HedgeDoc's database (atticd too, when enabled).
- [colony-psql](colony-psql.md) — Sharry and HedgeDoc databases (atticd too, when enabled).
- MinIO buckets back other boxes' services: Gitea LFS/packages (with the `middleman` MIME hack
for Docker manifests), Mastodon's `mastodon` bucket and the Bluesky PDS `pds` bucket on
[toot](toot.md).
[toot](toot.md), and Sharry's `share` bucket.
## Notable config files
docs/sites/colony/shill/containers/qclk.md
+14 -9
View File
@@ -6,17 +6,16 @@ the container provides the network plumbing and opens the API port.
- **Source:** [`shill/containers/qclk/`](../../../../../nixos/boxes/colony/vms/shill/containers/qclk)
(`default.nix`)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
- **Host:** NixOS container on [`shill`](../../shill.md)
## Role
- **WireGuard `management` interface** — listens on UDP `51821` (`lib.my.c.colony.qclk.wgPort`,
allowed through the firewall; `estuary` port-forwards it here) with the private key from the
`qclk/wg.key` age secret. Managed devices are static peers, each pinned to its own address in the
`qclk` prefix; the peer list currently has a single entry.
- **Routing/NAT** — `shill` routes the `qclk` prefix to this container, and outbound traffic from
`host0` into `management` is
`qclk/wg.key` age secret. Managed devices are static peers, each pinned to its own address in
the `qclk` prefix (`10.100.4.0/24`); the peer list currently has a single entry (host 2).
- **Routing/NAT** — the container itself is host 1 of the `qclk` prefix. `shill` routes
`10.100.4.0/24` to this container, and outbound traffic from `host0` into `management` is
SNATed to the container's `qclk` address. Forwarding into `management` is accepted from the
AS211024 trusted IPv4 ranges (`lib.my.c.as211024.trusted.v4`).
- **API port** — TCP `8080` is accepted on the `management` interface (`apiPort`), but note
@@ -24,10 +23,16 @@ the container provides the network plumbing and opens the API port.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `qclk`).
<!-- assignments: qclk -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| qclk-ctr | internal | `10.100.2.10/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::a/64` | ams1.int.nul.ie | |
| qclk | qclk | `10.100.4.1/24` | — | — | |
<!-- assignments-end -->
Two assignments: `internal` on the `ctrs` network like the other containers, and `qclk` on the
`management` WireGuard interface (IPv4 only, no DNS name).
Two assignments: `internal` on the `ctrs` network like the other containers, and `qclk` — host 1
of the `qclk` prefix on the `management` WireGuard interface (IPv4 only, no DNS name).
## Notable config files
docs/sites/colony/shill/containers/toot.md
+22 -21
View File
@@ -4,33 +4,34 @@ Federated-social container. Despite the name, the only service actually running
PDS** — the Mastodon instance ("toots") is **currently disabled**.
- **Source:** [`shill/containers/toot.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/toot.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
- **Host:** NixOS container on [`shill`](../../shill.md)
## Role
### Bluesky PDS
The active service listens on port 3000 as `pds.nul.ie`, fronted by [middleman](middleman.md),
which also redirects `/.well-known/atproto-did` here. It requires invites, applies an upload-size
limit, and stores blobs in [object](object.md)'s `pds` MinIO bucket. Federation uses the standard
Bluesky services and crawlers. `toot/pds.env` supplies secrets, including S3 credentials, and mail
comes from `pds@nul.ie`.
### Mastodon
Mastodon is disabled with `services.mastodon.enable = false`, but its configuration remains:
- `nul.ie` is the local domain and `toot.nul.ie` the web domain.
- PostgreSQL runs on [colony-psql](colony-psql.md), with local Redis and SMTP through `mail.nul.ie`.
- Media uses the `mastodon` MinIO bucket, a configured streaming-process pool, and periodic cleanup.
- [middleman](middleman.md) still proxies the dead vhost and `.well-known` endpoints.
The removed `otpSecretFile` option must be addressed before the service can return.
- **bluesky-pds** — the active service. `pds.nul.ie` on port `3000`, fronted by
[middleman](middleman.md) (which also redirects `/.well-known/atproto-did` here). Invites
required; blob store is the `pds` bucket on [object](object.md)'s MinIO (`s3.nul.ie`,
`eu-central-1`), upload limit 50 MiB; federation settings point at the stock Bluesky
infrastructure (`plc.directory`, `api.bsky.app`, `mod.bsky.app`, `bsky.network` crawlers).
Secrets (including the S3 credentials) come from the `toot/pds.env` age secret. Email from
`pds@nul.ie`.
- **mastodon** — **disabled** (`services.mastodon.enable = false`). The full config is still
present: `LOCAL_DOMAIN = nul.ie` with `WEB_DOMAIN = toot.nul.ie`, Postgres on
[colony-psql](colony-psql.md), local Redis, SMTP via `mail.nul.ie`, media in the `mastodon`
MinIO bucket (`S3_ALIAS_HOST = mastodon.s3.nul.ie`), 4 streaming processes, and media
auto-cleanup after 30 days. [middleman](middleman.md) still proxies `toot.nul.ie` →
`toot-ctr:80` and redirects the `webfinger`/`nodeinfo`/`host-meta` well-knowns there, but with
the service off those endpoints are dead. The config notes the removed `otpSecretFile` option
would need addressing before Mastodon can come back.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `toot`).
<!-- assignments: toot -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| toot-ctr | internal | `10.100.2.8/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::8/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
The firewall allows `http` (the Mastodon nginx vhost) and the PDS port `3000` besides netdata.
docs/sites/colony/shill/containers/vaultwarden.md
+9 -5
View File
@@ -4,8 +4,7 @@
manager, published as `pass.nul.ie` through [middleman](middleman.md).
- **Source:** [`shill/containers/vaultwarden.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/vaultwarden.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
- **Host:** NixOS container on [`shill`](../../shill.md)
## Role
@@ -15,12 +14,17 @@ manager, published as `pass.nul.ie` through [middleman](middleman.md).
- **SMTP** via `mail.nul.ie:587` (STARTTLS) as `pass@nul.ie`; credentials and other sensitive
settings come from the `vaultwarden/config.env` age secret.
- **Backups** — a `borgbackup` job pushes `/var/lib/vaultwarden` to rsync.net
using a repokey-encrypted repository; its passphrase and SSH key come from secrets, and the job
keeps daily, weekly and monthly archives according to its configured retention policy.
(`zh2855@zh2855.rsync.net:borg/vaultwarden2`), repokey-encrypted (passphrase and SSH key from
secrets), `zstd,10` compression, keeping 7 daily / 4 weekly / all monthly archives.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `vaultwarden`).
<!-- assignments: vaultwarden -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| vaultwarden-ctr | internal | `10.100.2.3/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::3/64` | ams1.int.nul.ie | |
<!-- assignments-end -->
## Persistence
docs/sites/colony/shill/containers/waffletail.md
+23 -17
View File
@@ -4,30 +4,36 @@ The colony Tailscale node: a subnet router and exit node that advertises the col
into the tailnet.
- **Source:** [`shill/containers/waffletail.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/waffletail.nix)
- **Host:** NixOS container on [`shill`](../README.md)
- **nixpkgs:** `mine`
- **Host:** NixOS container on [`shill`](../../shill.md)
## Role
### Tailscale
The node authenticates to `hs.nul.ie` with the secret `tailscale-auth.key`. It disables Tailscale's
netfilter management, advertises itself as an exit node and advertises the colony IPv4/IPv6 ranges,
but does not accept routes. UDP port 41641 is open and forwarded here by `estuary`.
### Routing and firewall
`shill` routes the Tailscale prefixes to this container.
The repository's nftables rules trust `tailscale0`, permit colony-sourced forwarding into the
tailnet, and SNAT tailnet traffic leaving through `host0` unless its destination is already within
colony.
- **tailscale** — joins via an auth key from secrets (`tailscale-auth.key`) against the
self-hosted Headscale control plane at `hs.nul.ie`. Runs with `--netfilter-mode=off` (firewall
is managed by the repo's own nftables rules), `--advertise-exit-node`, and
`--advertise-routes` covering the whole colony — `10.100.0.0/16` and `2a0e:97c0:4d2:10::/60`.
Does **not** accept routes itself. Listens on UDP `41641` (`openFirewall`), which `estuary`
port-forwards to this container.
- `shill` routes the Tailscale prefixes (`100.64.0.0/10`, `fd7a:115c:a1e0::/48`) to this
container, so colony hosts can reach tailnet clients and vice versa.
- nftables: `tailscale0` is a trusted interface; forwarding from `host0` into Tailscale is
allowed for the colony source ranges, and tailnet-sourced traffic leaving via `host0` is
SNATed to the container's colony addresses (except when destined to the colony ranges
themselves).
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `waffletail`).
<!-- assignments: waffletail -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| waffletail-ctr | internal | `10.100.2.9/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::9/64` | ams1.int.nul.ie | |
| waffletail | tailscale | `100.64.0.5/32` | `fd7a:115c:a1e0::5/128` | — | |
<!-- assignments-end -->
Two assignments: `internal` on the `ctrs` network like the other containers, and `tailscale` for
its addresses on the tailnet itself (no DNS name).
Two assignments: `internal` on the `ctrs` network like the other containers, and `tailscale` —
its addresses on the tailnet itself (host 5 of `100.64.0.0/10` and `fd7a:115c:a1e0::/48`; no
DNS name).
## Notable config files
docs/sites/colony/whale2.md
+40 -36
View File
@@ -10,52 +10,54 @@ container churn and resource use stay isolated).
## Role
### Container runtime
OCI containers run under podman (`virtualisation.oci-containers`) with the netavark backend and
`firewall_driver = "none"`, leaving firewall management to `my.firewall`.
### Routable game servers
Each server gets an address from `extraAssignments` (`valheim-oci`, `simpcraft-oci`, …) on the
`colony` netavark network. That network is backed by the `oci` interface and `prefixes.oci` v4/v6
ranges; `lib.my.dockerNetAssignment` supplies the address through `--network=colony:ip=…`.
`estuary` forwards the public game ports, while IPv6 reaches the containers directly.
### Storage
`/var/lib/containers` is a dedicated XFS disk with project quotas.
- Runs OCI containers via podman (`virtualisation.oci-containers`, netavark
backend, `firewall_driver = "none"` so podman doesn't fight `my.firewall`).
- Each game server gets its own routable address on the `colony` netavark
network (defined in `/etc/containers/networks/colony.json`), which is backed
by the `oci` interface and the `prefixes.oci` v4/v6 ranges; per-game
addresses come from `extraAssignments` (`valheim-oci`, `simpcraft-oci`, …)
and are passed to podman with `--network=colony:ip=…` (`lib.my.dockerNetAssignment`).
- `estuary` forwards the game ports in (see `firewallForwards`), so the
servers are reachable on the public IP as well as directly over IPv6.
- `/var/lib/containers` is a dedicated XFS disk (project quotas).
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `whale2`).
<!-- assignments: whale2 -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| whale-vm (oci) | internal | `94.142.241.226/32` | `2a0e:97c0:4d2:11::3/64` | ams1.int.nul.ie | |
| whale-vm-oci | oci | `10.100.3.1/24` | `2a0e:97c0:4d2:13::1/64` | ams1.int.nul.ie | |
| whale-vm-routing | routing | `10.100.1.3/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
<!-- assignments-end -->
## Game servers
The OCI containers are documented here (they have no pages of their own).
Their per-container `extraAssignments` on the `oci` network are listed in the generated
[network assignments](../../networking.md#box-assignments) table. Ports below are the public ones
forwarded by `estuary`.
Addresses are the per-container `extraAssignments` on the `oci` network;
ports are the public ones forwarded by `estuary`.
| Container | Ports | Status |
|---|---|---|
| `valheim` | `2456-2457`/udp | running |
| `simpcraft` | `25565` tcp+udp | running |
| `simpcraft-staging` | `25566` tcp | **disabled** (commented out) |
| `enshrouded` | `15636-15637`/udp | **disabled** (`enshrouded.nix` not imported) |
| `kevcraft` | `25567` tcp+udp | running |
| `kinkcraft` | `25568` tcp+udp | running |
| `graeme` | `25569` tcp+udp | running |
| Container | Address (v4 / v6 host) | Ports | Status |
|---|---|---|---|
| `valheim` | `10.100.3.2` / `2a0e:97c0:4d2:13::2` | `2456-2457`/udp | running |
| `simpcraft` | `10.100.3.3` / `2a0e:97c0:4d2:13::3` | `25565` tcp+udp | running |
| `simpcraft-staging` | `10.100.3.4` / `2a0e:97c0:4d2:13::4` | `25566` tcp | **disabled** (commented out) |
| `enshrouded` | `10.100.3.5` / `2a0e:97c0:4d2:13::5` | `15636-15637`/udp | **disabled** (`enshrouded.nix` not imported) |
| `kevcraft` | `10.100.3.6` / `2a0e:97c0:4d2:13::6` | `25567` tcp+udp | running |
| `kinkcraft` | `10.100.3.7` / `2a0e:97c0:4d2:13::7` | `25568` tcp+udp | running |
| `graeme` | `10.100.3.8` / `2a0e:97c0:4d2:13::8` | `25569` tcp+udp | running |
- **valheim** ([`valheim.nix`](../../../nixos/boxes/colony/vms/whale2/valheim.nix)) —
`lloesche/valheim-server`, public server "amogus sus", world `simpland2`,
allow-listed Steam IDs, password from agenix.
- **simpcraft** ([`minecraft/`](../../../nixos/boxes/colony/vms/whale2/minecraft)) —
`itzg/minecraft-server` (self-built `git.nul.ie/dev/craftblock` image),
Modrinth "Simpcraft" modpack, whitelist + ops.
- **simpcraft-staging** — the same setup pinned to an older pack version, currently commented out.
- **kevcraft** — vanilla Minecraft 1.20.1, extra op.
- **kinkcraft** — same Simpcraft modpack as `simpcraft`.
Modrinth "Simpcraft" modpack, whitelist + ops, 8 GiB heap.
**simpcraft-staging** is the same setup pinned to an older pack version,
currently commented out.
- **kevcraft** — vanilla Minecraft 1.20.1, 4 GiB heap, extra op.
- **kinkcraft** — same Simpcraft modpack as `simpcraft`, 6 GiB heap.
- **graeme** — vanilla Minecraft on hard difficulty with its own whitelist.
- **enshrouded** ([`enshrouded.nix`](../../../nixos/boxes/colony/vms/whale2/enshrouded.nix)) —
`sknnr/enshrouded-dedicated-server` ("UWUshrouded"); the file exists but is
@@ -67,10 +69,12 @@ The Minecraft containers share one whitelist/ops list and agenix env file
## Backups
A local borg job (`services.borgbackup.jobs.simpcraft`) archives the `simpcraft` world frequently,
offset from its autosave timer, into `/var/lib/containers/backup/simpcraft`. It uses `mcrcon` to
`save-off`/`save-on` around each run and keeps a short history for quick world rollback rather than
disaster recovery; the `oci` LV itself is covered by `colony`'s `borgthin`.
A local borg job (`services.borgbackup.jobs.simpcraft`) archives the
`simpcraft` world every ~15 minutes (offset from the usual 5-minute autosave
ticks) into `/var/lib/containers/backup/simpcraft`, using `mcrcon` to
`save-off`/`save-on` around each run. Retention is short (12 h + 48 hourly) —
this is for quick world rollback, not disaster recovery (the `oci` LV itself
is covered by `colony`'s `borgthin`).
## Notable config files
docs/sites/home/README.md
+57 -27
View File
@@ -8,28 +8,70 @@ pair, and everything clients touch (gateway, DNS) is a floating VIP that follows
- **Source:** [`nixos/boxes/home/`](../../../nixos/boxes/home)
## Boxes
## Hierarchy
| Box | Role | Host |
|---|---|---|
| [`palace`](palace.md) | VM host | physical |
| [`river`](river.md) | Primary router (VRRP pair with `stream`) | VM on `palace` |
| [`stream`](stream.md) | Secondary router (VRRP pair with `river`) | physical |
| [`cellar`](cellar.md) | NVMe-oF / SPDK storage target | VM on `palace` |
| [`sfh`](sfh/README.md) | NixOS container host (containers on its page) | VM on `palace` |
| [`castle`](castle.md) | Workstation / gaming desktop | physical |
```
h.nul.ie
├── palace (physical VM host — AMD, 100G, SR-IOV)
│ ├── river ── primary router VM (PPPoE / Digiweb WAN)
│ ├── cellar ─ NVMe-oF / SPDK storage target VM
│ └── sfh ──── container host VM ("services for home")
│ ├── hass ── Home Assistant + Frigate + MQTT (container)
│ └── unifi ─ UniFi controller (container)
├── stream (physical secondary router — Virgin Media WAN)
└── castle (workstation / gaming desktop — netboot, NVMe-oF root)
```
## Machines
| Box | Role | Host | Page |
|---|---|---|---|
| `palace` | VM host | physical | [palace.md](palace.md) |
| `river` | Primary router (VRRP pair with `stream`) | VM on `palace` | [river.md](river.md) |
| `stream` | Secondary router (VRRP pair with `river`) | physical | [stream.md](stream.md) |
| `cellar` | NVMe-oF / SPDK storage target | VM on `palace` | [cellar.md](cellar.md) |
| `sfh` | NixOS container host | VM on `palace` | [sfh.md](sfh.md) |
| `castle` | Workstation / gaming desktop | physical | [castle.md](castle.md) |
| `hass` | Home Assistant + Frigate + MQTT | container on `sfh` | [sfh/containers/hass.md](sfh/containers/hass.md) |
| `unifi` | UniFi controller | container on `sfh` | [sfh/containers/unifi.md](sfh/containers/unifi.md) |
## Router VIPs
Clients use per-VLAN floating VIPs as their gateway and DNS server; `keepalived` moves them between
`river` and `stream`. The addresses, DHCP/RA behavior and failover mechanics are documented once in
[Router VIPs](../../networking.md#router-vips) and [Router HA](../../networking.md#router-ha).
Clients never use a router's real address: `kea` (DHCP) and `radvd` (RAs) hand out the per-VLAN
floating VIPs as both gateway and DNS server, and `keepalived` moves them between `river` and
`stream`. They are declared as pseudo-systems (`router-hi` / `router-lo` / `router-ut`) in
`routing-common`'s `extraAssignments`, with addresses from `lib.my.c.home.vips`
([`lib/constants.nix`](../../../lib/constants.nix)); `pdns-recursor` binds them on both routers
(`ip_nonlocal_bind`) so DNS follows the master instead of relying on client resolver timeouts.
| Pseudo-system | Network | IPv4 | IPv6 |
|---|---|---|---|
| `router-hi` | `hi` (VLAN 100) | `192.168.71.254/22` | `2a0e:97c0:4d0:1::ffff/64` |
| `router-lo` | `lo` (VLAN 110) | `192.168.79.254/21` | `2a0e:97c0:4d0:2::ffff/64` |
| `router-ut` | `untrusted` (VLAN 120) | `192.168.80.254/24` | `2a0e:97c0:4d0:3::ffff/64` |
Notes:
- The IPv6 gateway clients learn from RAs is the link-local `fe80::1` on each VLAN (a
`virtual_ipaddress_excluded` VIP; `radvd` advertises from it), not the global VIP above.
- `keepalived` also floats a VIP on the `as211024` mesh interface (`10.100.50.4`,
`2a0e:97c0:4df:0:1::ffff`) — it has no pseudo-system because nothing client-facing uses it.
- The `untrusted` VLAN is an exception to "DNS follows the master": its DHCP/RA options hand out
Cloudflare resolvers (`1.1.1.1` / `2606:4700:4700::1111`), not the VIP.
## Networks
The site separates core management, high-MTU trusted traffic, general trusted traffic, untrusted
clients and the two WAN paths. VLAN IDs, prefixes, MTUs and router addressing live in the canonical
[`home` section of networking.md](../../networking.md#home).
| Network | VLAN | IPv4 | IPv6 | MTU |
|---|---|---|---|---|
| `core` | native | `192.168.64.0/24` | — | 1500 |
| `hi` | 100 | `192.168.68.0/22` | `2a0e:97c0:4d0:1::/64` | 9000 |
| `lo` | 110 | `192.168.72.0/21` | `2a0e:97c0:4d0:2::/64` | 1500 |
| `untrusted` | 120 | `192.168.80.0/24` | `2a0e:97c0:4d0:3::/64` | 1500 |
| `wan` (stream) | 130 | DHCP public lease + modem mgmt `192.168.0.0/24` | — | 1500 |
| `pon-isp` (river) | 10 | PPPoE transport (no L3) | — | 1508 |
| `wan-pon-ont` (river) | 140 | ONT mgmt `192.168.100.0/24` | — | 1500 |
The logical network map lives in [networking.md](../../networking.md).
## Switch fabric
@@ -37,15 +79,3 @@ The boxes hang off three hand-configured switches — `jim` and `dave` (MikroTik
`brian` (Ubiquiti, UniFi) — which are **not** managed by this flake. The physical topology, VLAN
map, the Digiweb WAN path (trunked VLAN 10 + PVID 140 at the ONT edge), and the multi-ONT plan are
documented in [switches.md](switches.md).
## Wireless APs
The Wi-Fi APs — `vibe` (MikroTik cAP ax) and `wave` (Cudy AX3000 on OpenWrt) — are dumb APs, also
**not** managed by this flake. The shared VLAN-trunk design, SSIDs, per-AP management addressing,
and the OpenWrt flash/config for `wave` are in [aps.md](aps.md).
## 5G WWAN
A Quectel RM500U-EA USB modem with a GoMo SIM is being evaluated as a replacement for `stream`'s
Virgin Media WAN. It is bench-tested only and not yet referenced by the flake; the module settings
it needs, the APN gotcha and the CGNAT consequences are in [wwan.md](wwan.md).
docs/sites/home/aps.md
-157
View File
@@ -1,157 +0,0 @@
# Home wireless APs
Reference for the home Wi-Fi access points. Like the switches ([switches.md](switches.md)), these
are **not** managed by this flake — they are configured on-device (RouterOS on the MikroTik,
OpenWrt/UCI on the Cudy). This file documents the shared VLAN/trunk design and each AP.
Only the DNS records live in the flake
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix),
`h.nul.ie` zone). Everything else here is applied by hand on the device.
## The APs
| | vibe | wave |
|---|---|---|
| Model | MikroTik **cAP ax** (`cAPGi-5HaxD2HaxD`) | Cudy **AX3000** (OpenWrt id `cudy,ap3000-v1`) |
| OS | RouterOS 7.x | OpenWrt 25.12.x (MT7981B / Filogic 820) |
| Radio | 2×2 both bands | 2×2 both bands (2 spatial streams) |
| Uplink | trunk (multi-port; `ether2` is a wired LAN port) | single **2.5 GbE** trunk |
| Management | core/hi/lo `.15` | core/lo `.14` (**not** `hi`; see below) |
Both mirror the same two SSIDs. `wave` replaced an older AP of the same name; the new one is the
Cudy running OpenWrt.
> **Note on "AX3000":** MT7981B is 2×2 (2 spatial streams). The "3000" is *aggregate* Mbps —
> 574 (2.4 GHz 2ss) + 2402 (5 GHz 2ss @ **160 MHz**) — not three streams. `iw` confirms 2×2
> (`Available Antennas TX/RX 0x3`; "3 streams: not supported"). 160 MHz is what earns the "3000".
## Shared design (dumb AP)
Every AP is a **dumb AP**: it bridges wireless clients onto the right VLAN and does **no** routing,
DHCP, RA or firewalling. The home routers (`river`/`stream`) own DHCP/RA/gateway (per-VLAN VRRP VIPs)
and firewalling. The uplink is a **tagged trunk**:
| VLAN | `lib.my.c.home.vlans` | Role | AP use |
|---|---|---|---|
| — (native) | `core` | switch/fabric management (1500) | backup management (untagged) |
| 100 | `hi` | trusted LAN, **high-MTU** (jumbo 9000) | `vibe` management (it's jumbo-capable) |
| 110 | `lo` | trusted LAN (1500) | main SSID `wlan0`; `wave` management |
| 120 | `untrusted` | guest network | guest SSID `wlan1` |
`hi` and `lo` are both **trusted** client VLANs — the only difference is MTU (`hi` carries jumbo
9000, `lo` is standard 1500). An AP puts its own management on whichever it can do: `vibe` (jumbo)
sits on `hi`, `wave` (eth0 capped at 2026) sits on `lo`. The main SSID lands on `lo` because Wi-Fi
clients are 1500 regardless.
### SSIDs
| SSID | Bands | Security | VLAN |
|---|---|---|---|
| `wlan0` (main) | 5 GHz + 2.4 GHz | WPA2/WPA3-PSK (`sae-mixed`) | 110 (`lo`) |
| `wlan1` (guest) | 2.4 GHz | WPA2-PSK (`psk2`) | 120 (`untrusted`) |
**Passphrases are never stored in this repo.** `vibe` is the source of truth; read them out-of-band
with `ssh admin@vibe '/interface wifi export show-sensitive'` (`.passphrase=` prints **unquoted**).
## vibe (MikroTik cAP ax)
RouterOS, one hardware-offloaded bridge `main` with `vlan-filtering=yes`. Access: `ssh admin@vibe`
— key auth for `admin` is installed (`~/.ssh/id_rsa`), with `admin`/`admin` as a fallback.
- **Uplink `ether1`** — trunk, tagged VLANs 100/110/120; native/untagged is the default VLAN 1
(PVID, no IP). `ether2` is a wired **access port** on VLAN 110 (`lo`). `l2mtu 9214`.
- **Radios** — `wifi1` (5 GHz, 20/40/80) + `wifi2` (2.4 GHz, 20/40) both broadcast `wlan0`
(WPA2/WPA3-PSK), untagged onto VLAN 110. `wifi3` is a virtual AP on `wifi2` broadcasting `wlan1`
(WPA2-PSK), untagged onto VLAN 120. `country=Ireland`.
- **Bridge VLANs** — 100 tagged `main,ether1`; 110 tagged `main,ether1` + untagged
`ether2,wifi1,wifi2`; 120 tagged `main,ether1` + untagged `wifi3`.
- **Roaming** — 802.11k/v via a `/interface wifi steering` profile (`rrm=yes wnm=yes`,
`neighbor-group=home-aps`) assigned to `wifi1`/`wifi2`/`wifi3`.
- **Resolver** — the hi VIP `192.168.71.254` / `2a0e:97c0:4d0:1::ffff`.
### Management
Management uses host `.15`: `192.168.64.15` on native/core as a backup,
`192.168.68.15/22` + `2a0e:97c0:4d0:1::1:6` on `hi` VLAN 100, and
`192.168.72.15/21` + `2a0e:97c0:4d0:2::1:6` on `lo` VLAN 110. The `hi` address holds the default
route through its VIP; `untrusted` has no address. With `l2mtu 9214`, `vibe` can use the jumbo
`hi` network unlike `wave`.
Like the switches, `vibe` is **pure L2 and must not route** — `ip-forward=no`, IPv6 `forward=no`,
`accept-router-advertisements=no`, `ra-lifetime=0`. RouterOS ships these forwarding-on and then
advertises itself as an IPv6 router; re-check after any upgrade. See
[switches.md#switches-must-not-route](switches.md#switches-must-not-route) for the rationale and
commands.
## wave (Cudy AX3000, OpenWrt)
Single-port AP, so the port is a VLAN **trunk** carrying management + both SSIDs.
### Management addressing
`wave` takes host `.14`, on **`lo` (primary) and `core` (backup)** — deliberately **not** `hi`,
unlike the switches and `vibe`. `hi` is the jumbo (9000) VLAN, but `wave`'s eth0 caps at 2026 (see
MTU note), so there's no reason to put it there; `lo` is 1500 with a proper VRRP VIP for the default
route + resolver, and `core` has no VIP/v6 so it can only be a backup. No IP on `untrusted`. Records
in `dns.nix`:
| Name | VLAN | Address |
|---|---|---|
| `wave-core` | core (native/untagged) | `192.168.64.14/24` — backup, like the switches (no VIP → backup only) |
| `wave` | lo 110 | `192.168.72.14/21`, `2a0e:97c0:4d0:2::1:5` — primary; holds the default route + resolver (lo VIP `192.168.79.254` / `2a0e:97c0:4d0:2::ffff`) |
**Firewall:** management (SSH/LuCI) reachable from `core`/`lo` only; `untrusted` is a separate
zone with `input REJECT` (and `wave` has no IP there) — **no management via the guest VLAN**.
### brian switch port
`wave` hangs off **brian** (UniFi). Its port is a **trunk**: tagged VLAN **110/120** (`lo` + guest),
and **native/untagged = core** (the fabric's management VLAN, carrying `wave-core`). VLAN 100 (`hi`)
is **not** needed here — `wave` isn't on `hi` (see Management addressing). Configure via the UniFi
controller (brian has no CLI); see [switches.md](switches.md).
### Flashing OpenWrt (Cudy AX3000 / `cudy_ap3000-v1`)
Hardware: MT7981B, 512 MB RAM, 256 MB SPI-NAND, 1× 2.5 GbE (RTL8221B), 2×2 WiFi 6.
> ⚠️ **Serial caveat:** units with a serial starting `2543…` (post ~Nov 2025) use a different flash
> chip and can brick with older firmware. Match firmware to the unit.
OpenWrt can't be flashed directly over stock. Two-stage, via a Cudy **transition** firmware (Cudy
OpenWrt download page / `support@cudy.com`; `warnning.txt` in that bundle has the steps):
1. Stock Cudy UI: update to **≥ 2.4.7** (adds TFTP `recovery.bin` recovery), then flash the Cudy
**intermediate** firmware (`cudy_ap3000-v1-sysupgrade_*.bin`), "keep settings" **unchecked**.
It reboots into an OpenWrt-based build at `192.168.1.1` (SSH `root`, empty password).
2. From there, `sysupgrade -n` to vanilla OpenWrt (`…-cudy_ap3000-v1-squashfs-sysupgrade.bin` from
`downloads.openwrt.org`; this release ships **no** factory image — sysupgrade only).
Stock default (out of box) is a DHCP client falling back to **`192.168.10.254`**; the stock UI is a
customised LuCI (only 80/443, no SSH) with a first-boot "create admin password" wizard — so the
stock-side flashing is done from a browser, not headless.
### On-device config notes
#### Wireless packages
The package manager is `apk`, not `opkg`. Wi-Fi uses the full `wpad-mbedtls` package so all SSIDs
can enable 802.11k/v (`ieee80211k` + `bss_transition`). Replacing `wpad-basic-mbedtls` live leaves
the mac80211 interfaces in a start/teardown loop that reloads cannot recover; reboot after
`apk add wpad-mbedtls`.
- Radios: `radio0` = 2.4 GHz, `radio1` = 5 GHz (keyed by `band`, don't assume). 5 GHz is pinned to
**channel 36 / HE160** (any 160 MHz block in IE is DFS; ch36 has the shortest ~60 s CAC).
- Bridge: `br-lan` with `vlan_filtering`, single port `eth0` — tagged `110/120`, untagged/PVID
VLAN 1 (= native/core). SSIDs attach via `network` = `lo`/`untrusted` (= `br-lan.110`/`.120`).
- Dumb-AP: no DHCP pools, `odhcpd.maindhcp=0`, `delegate=0` on the L3 interfaces.
- **MTU:** all interfaces are **1500**. The `mtk_eth_soc` 2.5 GbE (`eth0`) caps at **2026 bytes**
(`ip link set eth0 mtu 9000` → `SIOCSIFMTU: Invalid argument`), so `wave` can't join `hi`'s jumbo
(9000) fabric like `vibe` does — which is precisely **why `wave` is managed on `lo`, not `hi`**
(see Management addressing). Nothing on `wave` needs > 1500.
- **LuCI:** enabled, login `root` / `admin`. **SSH:** key-only (`PasswordAuth`/`RootPasswordAuth off`).
- `iperf3` installed for throughput testing.
### Access
- SSH: `ssh root@wave` (key-only; `wave`/`wave-core` resolve once `dns.nix` is deployed).
- LuCI: `http://192.168.72.14/` (or `http://wave/`), `root` / `admin`.
docs/sites/home/castle.md
+27 -41
View File
@@ -5,56 +5,42 @@ root storage on NVMe-oF volumes from `cellar`.
- **Source:** [`nixos/boxes/home/castle/`](../../../nixos/boxes/home/castle) (`default.nix`)
- **Host:** physical
- **nixpkgs:** `mine`
## Hardware
| Component | Inventory |
|---|---|
| Platform | ASUS ProArt X670E-CREATOR WIFI |
| CPU | AMD Ryzen 9 7950X (16 cores / 32 threads) |
| Memory | 64 GiB |
| Graphics | Integrated AMD Radeon graphics |
| Network | Mellanox ConnectX-4 100G, Aquantia AQC113CS 10G, Intel I225-V 2.5G and MediaTek MT7922 Wi-Fi 6E controllers |
| System storage | No local root disk; the box netboots and uses the SPDK NVMe-oF namespace exported by `cellar` |
## Role
### Desktop
The AMD desktop runs the GUI stack (`my.gui.enable`, Sway/Wayland via home-manager), low-latency
PipeWire, Bluetooth and Thunderbolt. Local `libvirtd`/`virt-manager` and the IOMMU are enabled, but
the box has no VFIO or GPU-passthrough configuration.
### Netboot
With `my.netboot.client.enable`, the firmware iPXE-boots from the 2.5G NIC. Kea matches
`et2.5g`'s MAC and directs it to `boot.h.nul.ie` on [`river`](river.md).
### NVMe-oF root
`/nix`, `/persist` and `/home` are `/dev/nvmeof/*` LVs in `cellar`'s
`nqn.2016-06.io.spdk:castle` namespace (`my.nvme.boot`, RDMA). The initrd brings
up `et100g`/`lan-hi` with `roceBootModules`; the running network keeps
`KeepConfiguration=static` so networkd does not drop the storage address. The root filesystem is a
size-limited tmpfs (`my.tmproot`).
### Other configuration
Both firewalls are disabled on this trusted `hi` desktop. Other settings include `binfmt`
emulation for `aarch64-linux`/`armv7l-linux`, `recursive-nix`, Wireshark and `rdma-core`/`qperf`;
a `drm-amd-display` flicker patch remains commented out.
- AMD desktop running the GUI stack (`my.gui.enable`, Sway/Wayland via home-manager), PipeWire
(low-latency `quantum 128`, EasyEffects, jacktrip), Bluetooth, Thunderbolt (`bolt`).
- **Netboot client** (`my.netboot.client.enable`): the firmware iPXE-boots off the 2.5G NIC —
kea's client-class for `castle` matches `et2.5g`'s MAC (`c8:7f:54:6e:17:0f`) and points at
`boot.h.nul.ie` on [`river`](river.md).
- **Root on NVMe-oF**: `/nix`, `/persist` and `/home` are `/dev/nvmeof/*` LVs on the
`nqn.2016-06.io.spdk:castle` namespace from [`cellar`](cellar.md) (`my.nvme.boot`,
`192.168.68.80`, RDMA). The initrd brings up `et100g`/`lan-hi` plus `roceBootModules` to reach
it, and the running system keeps `KeepConfiguration=static` on `lan-hi` so networkd never drops
the NVMe-oF address. The root itself is a 24 GiB tmpfs (`my.tmproot`).
- Local virtualisation: `libvirtd` + `virt-manager` are enabled and the IOMMU is on
(`amd_iommu=on`), but no VFIO/GPU-passthrough is configured in the box config today.
- Both firewalls are off (`networking.firewall.enable` and `my.firewall.enable` — it's a trusted
desktop on `hi`).
- Misc: `binfmt` emulation for `aarch64-linux`/`armv7l-linux`, `recursive-nix`, Wireshark,
`rdma-core`/`qperf` for the RoCE link. A `drm-amd-display` flicker patch sits commented out in
`kernelPatches`.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `castle`).
<!-- assignments: castle -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| castle | hi | `192.168.68.40/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::3:1/64` | h.nul.ie | |
<!-- assignments-end -->
## Networking
- `et100g` (100G, MTU 9000) carries `lan-hi` (the `hi` assignment, also pinned by a kea
reservation on its MAC) and `lan-lo`.
- `lan-lo` is a secondary network attachment: DHCPv4 with `UseGateway`/`UseDNS` off and RAs
accepted with gateway/DNS use off — present for reaching `lo` devices, never a default route.
- `et100g` (100G, MTU 9000) carries `lan-hi` (the `hi` assignment, statically `.40` — also pinned
by a kea reservation on its MAC) and `lan-lo`.
- `lan-lo` is a secondary leg: DHCPv4 with `UseGateway`/`UseDNS` off and RAs accepted with
gateway/DNS use off — present for reaching `lo` devices, never a default route.
- `et2.5g` (netboot) and `et10g` are renamed but carry no network config.
## Notable config files
docs/sites/home/cellar.md
+11 -6
View File
@@ -7,30 +7,35 @@ it.
- **Source:** [`nixos/boxes/home/palace/vms/cellar/`](../../../nixos/boxes/home/palace/vms/cellar)
(`default.nix`, `spdk.nix`)
- **Host:** VM on `palace`
- **nixpkgs:** `mine`
## Role
- Runs an **SPDK userspace target** (`my.spdk`, [`spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix)):
the kernel `nvme` driver is blacklisted so SPDK can claim the three NVMe controllers directly
(host BDFs `41:00.0`–`43:00.0`, attached in the guest as `02:00.0`–`04:00.0`).
- Builds a **RAID-0** (`NVMeRaid`) across the three drives and exports one
- Builds a **RAID-0** (`NVMeRaid`, 64 KiB strip) across the three drives and exports one
partition per consumer as an **NVMe-oF subsystem over RDMA** (port 4420) on the `hi` network:
| Bdev | NQN | Consumer |
|---|---|---|
| `NVMeRaidp1` | `nqn.2016-06.io.spdk:river` | [`river`](river.md) |
| `NVMeRaidp2` | `nqn.2016-06.io.spdk:castle` | [`castle`](castle.md) |
| `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh/README.md) |
| `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh.md) |
Each subsystem is pinned to its consumer's `hostnqn` (the `my.nvme.uuid` on the client side).
- `spdk-tgt` is ordered after `lan-hi` is online; the RDMA listener binds the `hi` assignment on
port 4420. The VM itself is pinned to NUMA node 1 on `palace` and gets SR-IOV VF 0.
- `spdk-tgt` is ordered after `lan-hi` is online; the RDMA listener binds the `hi` address
(`192.168.68.80:4420`). The VM itself is pinned to NUMA node 1 on `palace` and gets SR-IOV VF 0.
- `netdata` (port 19999 allowed in the firewall) and `fstrim`.
- Deploy address `192.168.68.80` (`my.deploy.node.hostname`).
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `cellar`).
<!-- assignments: cellar -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| cellar | hi | `192.168.68.80/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:1/64` | h.nul.ie | |
<!-- assignments-end -->
## Notes
docs/sites/home/palace.md
+14 -25
View File
@@ -6,19 +6,6 @@ SR-IOV VFs, PCI NVMe drives and LVM disks.
- **Source:** [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix)
(VM definitions in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
- **Host:** physical
- **nixpkgs:** `mine-stable`
## Hardware
| Component | Inventory |
|---|---|
| Platform | Gigabyte X399 DESIGNARE EX |
| CPU | AMD Ryzen Threadripper 1950X (16 cores / 32 threads) |
| Memory | 128 GiB |
| Host storage | 500 GB Samsung SSD 860 EVO containing the EFI partition and the `main` LVM thin pool |
| Bulk storage | Three 8 TB Seagate IronWolf disks in the `hdds` VG, providing the RAID-backed `hdd-storage` and `frigate` LVs |
| NVMe storage | Three 2 TB Samsung NVMe devices passed through to `cellar`; SPDK combines them as the `NVMeRaid` RAID 0 device |
| Network / graphics | Mellanox ConnectX-4 100G adapter with four SR-IOV VFs, two Intel I211 Gigabit Ethernet controllers, and an AMD Radeon RX 550/560-family GPU |
## Role
@@ -27,24 +14,26 @@ SR-IOV VFs, PCI NVMe drives and LVM disks.
the `main` thin pool (`services.lvm.boot.thin.enable`).
- AMD box (`kvm-amd`, `amd_iommu=on`, microcode updates); the kernel is built with
`ACPI_APEI_PCIEAER`/`PCIEAER` for the PCIe passthrough work below.
- Deploy address `192.168.68.22` (`my.deploy.node.hostname`).
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `palace`).
<!-- assignments: palace -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| palace-core | core | `192.168.64.20/24` | — | h.nul.ie | |
| palace | hi | `192.168.68.22/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::2:1/64` | h.nul.ie | |
<!-- assignments-end -->
## Networking
100G `et100g` (mlx5, MTU 9000) uplinks to the `dave` switch and carries `lan-hi` (VLAN 100, the
`hi` assignment). A udev rule creates four SR-IOV VFs on the PF:
| VF | Consumer | VLAN handling |
|---|---|---|
| 0 | `cellar` | `hi` |
| 1 | `river` | untagged VF; `river` tags its LAN and WAN VLANs |
| 2 | `sfh` | `hi` |
| 3 | `sfh` container MACVLAN parent | `hi` |
- `lan-core` is a bridge with the `core` assignment (no gateway); the 1G
- 100G `et100g` (mlx5, MTU 9000) uplinks to the `dave` switch and carries `lan-hi` (VLAN 100,
the `hi` assignment). A udev rule creates **4 SR-IOV VFs** on the PF, distributed in
`50-et100g`'s `[SR-IOV]` sections: VF 0 → `cellar` (VLAN hi), VF 1 → `river` (no VLAN — river
tags all of its own VLANs, including both WAN VLANs, on this trunk), VF 2 → `sfh` (VLAN hi),
VF 3 → `sfh`'s container MACVLAN parent (VLAN hi).
- `lan-core` is a bridge with the `core` assignment (`192.168.64.20`, no gateway); the 1G
`lan-core-phy` and the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used
for VM netboot and `lo` clients.
- The 1G `et1g0` (igb) is exported to `river` as a passthru-mode macvtap (`vm-et1g0`) — river sees
docs/sites/home/river.md
+57 -55
View File
@@ -1,73 +1,75 @@
# river
A home router VM on `palace` with a Digiweb PPPoE WAN on the ISP's VLAN 10. It forms the redundant
router pair with [`stream`](stream.md).
The primary home router. A VM on `palace`, built from `routing-common` at index 0; it runs PPPoE
to Digiweb directly on the ISP's VLAN 10 and normally holds the master side of the VRRP pair with
[`stream`](stream.md).
- **Source:** [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix)
(shared router config: [`routing-common`](../../../nixos/boxes/home/routing-common), index 0)
- **Host:** VM on `palace`
- **nixpkgs:** `mine`
## Role
At `routing-common` index 0, `river` normally holds the primary position in the router pair.
Pair-wide addressing, VIP, DHCP/DNS and failover behavior is documented in the
[`home` networking overview](../../networking.md#home) and [Router HA](../../networking.md#router-ha).
This page covers `river`'s Digiweb WAN, VM platform, storage and netboot duties.
- Primary of the router pair: `routing-common` index 0 → keepalived starts `MASTER` (priority
255), kea serves the lower-half DHCP pools, the zone's SOA/`ns1` point here.
- Everything from `routing-common`: keepalived/VRRP ([`keepalived.nix`](../../../nixos/boxes/home/routing-common/keepalived.nix)),
PowerDNS recursor + authoritative with a blocklist ([`dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix)),
kea DHCP + DDNS ([`kea.nix`](../../../nixos/boxes/home/routing-common/kea.nix)), radvd
([`radvd.nix`](../../../nixos/boxes/home/routing-common/radvd.nix)), NAT/firewall, the
`as211024` L2 mesh link back to colony, `iperf3`, `nginx`. See
[networking.md](../../networking.md) for the logical view and [README.md](README.md#router-vips)
for the floating VIPs.
- **Netboot server** for `sfh` and `castle` (`my.netboot.server`): iPXE/TFTP at
`boot.h.nul.ie` (a CNAME to `river-hi`), served from the `lo` address `192.168.72.1` and
restricted to the hi/lo prefixes.
- **NVMe-oF client of `cellar`**: the VM's only local disk is an ESP (an installer ISO is also
still attached); `/nix` and `/persist` are LVs on the `nqn.2016-06.io.spdk:river` namespace
exported by `cellar` (`192.168.68.80`, RDMA).
The initrd brings up `lan-hi` with the RoCE modules (`roceBootModules`), and
`KeepConfiguration=static` on `lan-hi` stops networkd from dropping the NVMe-oF address on
reconfigure.
- **SR-IOV VF passthrough**: the 100G `lan` NIC is VF 1 of palace's `et100g` (MAC
`52:54:00:8a:8a:f2`, MTU 9000). All router VLANs — hi/lo/untrusted plus both WAN VLANs — are
tagged on top of it (`55-lan`).
- Also carries a macvtap passthrough of palace's 1G `et1g0`, renamed `wan-old` — the pre-100G WAN
path, kept around with no L3 config today.
- Deploy address `192.168.68.1` (`my.deploy.node.hostname`).
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `river`).
<!-- assignments: river -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| river | as211024 | `10.100.50.2/24` | `2a0e:97c0:4df:0:1::1/64 gw 2a0e:97c0:4df:0:2::1` | — | |
| river-core | core | `192.168.64.1/24` | — | h.nul.ie | |
| river-hi | hi | `192.168.68.1/22` | `2a0e:97c0:4d0:1::1/64` | h.nul.ie | |
| river-lo | lo | `192.168.72.1/21` | `2a0e:97c0:4d0:2::1/64` | h.nul.ie | |
| river-ut | untrusted | `192.168.80.1/24` | `2a0e:97c0:4d0:3::1/64` | h.nul.ie | |
<!-- assignments-end -->
## WAN (Digiweb PPPoE)
### Link and addressing
`services.pppd` peer `digiweb` attaches directly to `wan-pon-isp`, the raw ISP VLAN 10
(`vlans.pon-isp`) trunked untranslated from the ONT through `brian` and `dave`; see
[switches.md](switches.md). The netdev has no L3 configuration and uses `MTUBytes=1508`, leaving a
clean `mtu`/`mru 1500` after PPPoE overhead.
The static `84.203.124.128` address is requested through IPCP. The provider-wide credentials are
deliberately not secret; the peer persists indefinitely with LCP echo monitoring and ignores
Digiweb DNS in favor of the local recursor.
### Management subnet
`wan-pon-ont` (VLAN 140, PVID'd at `brian`) holds `192.168.100.100/24`, reaching the ONT UI at
`192.168.100.1`. The `.100` address follows `stream`'s modem-management convention.
### WAN readiness
The `pppd` hooks drive this shared gate with `DefaultDependencies=false`: `ip-up` installs the
link-scoped default route and starts the target, while `ip-down` stops it and removes the route.
`ipsec` and `ipv6-clear-default-route` attach through `wantedBy` + `partOf`, so they reload after
every WAN flap.
### Traffic shaping
The shared `wan-ifb` ingress-shaping pieces are inert here. CAKE is specific to `stream`, and
`networkd-dispatcher` is `mkForce false` pending scheduling tests.
## Platform
### Virtual machine and network attachment
The 100G `lan` NIC is VF 1 of `palace`'s `et100g` (MTU 9000), with every router VLAN tagged on top
as `55-lan`. A macvtap of `palace`'s 1G `et1g0` remains as the old `wan-old` path without L3
configuration. Deployments use the `hi` assignment.
### Storage
The VM's local disk holds only an ESP; `/nix` and `/persist` are LVs on `cellar`'s
`nqn.2016-06.io.spdk:river` namespace over RDMA. The initrd brings up `lan-hi`
with `roceBootModules`, and `KeepConfiguration=static` prevents networkd from dropping the address
during reconfiguration. An installer ISO remains attached.
## Netboot
`my.netboot.server` serves iPXE/TFTP for `sfh` and `castle` at `boot.h.nul.ie` from the `lo`
assignment, restricted to the `hi` and `lo` prefixes.
- `services.pppd` peer `digiweb` attaches PPPoE directly to `wan-pon-isp`
(`plugin pppoe.so wan-pon-isp`) — the **raw ISP VLAN 10** (`vlans.pon-isp`), trunked
untranslated from the ONT through `brian` and `dave` (the fabric side is in
[switches.md](switches.md)). The netdev is up with no L3; `MTUBytes=1508` (baby jumbo) absorbs
PPPoE's 8-byte overhead so the `wan` ppp interface gets a clean `mtu`/`mru 1500`.
- The Digiweb static IP (`84.203.124.128`, `elemAt routersPubV4 0`) is requested as the local
address in IPCP. Auth is the across-all-customers shared `digiweb@nga.digiweb.ie` / `digiweb`
(deliberately not a secret); `persist`, `maxfail 0`, 1s LCP echoes. `usepeerdns` is absent on
purpose — Digiweb's resolvers are ignored in favour of the local recursor.
- **ONT management**: `wan-pon-ont` (VLAN 140, PVID'd at the `brian` edge) holds
`192.168.100.100/24` so the ONT web UI at `192.168.100.1` is reachable — the `.100` mirrors
stream's modem-management convention.
- **`wan-online.target`** is the shared "public WAN is up" gate declared by `routing-common`;
here it is driven by the pppd hooks (`DefaultDependencies=false`, so nothing pulls it in
early): `ip-up` installs `default dev wan scope link metric 100` and starts the target,
`ip-down` stops it and deletes the route. Consumers (`ipsec`, `ipv6-clear-default-route`)
attach with `wantedBy` + `partOf`, so they re-load on every WAN flap.
- The `wan-ifb` ingress-shaping pieces from `routing-common` are inert on this box: the CAKE
config itself is stream's, and `networkd-dispatcher` (which installs the `tc` mirror) is
`mkForce false` here pending scheduling testing.
## Notable config files
docs/sites/home/sfh/README.md → docs/sites/home/sfh.md
+23 -19
View File
@@ -1,22 +1,21 @@
# sfh
"Shill from home" — the NixOS container host for the home site. A VM on `palace` that netboots
"Services for home" — the NixOS container host for the home site. A VM on `palace` that netboots
from `river` and runs its root off NVMe-oF from `cellar`.
- **Source:** [`nixos/boxes/home/palace/vms/sfh/`](../../../../nixos/boxes/home/palace/vms/sfh)
- **Source:** [`nixos/boxes/home/palace/vms/sfh/`](../../../nixos/boxes/home/palace/vms/sfh)
(`default.nix`, `containers/`)
- **Host:** VM on `palace`
- **nixpkgs:** `mine`
## Role
- Runs the home NixOS containers via `my.containers.instances` (systemd-nspawn); each container is
its own `nixos.systems.*` entry rendered through `my.asContainer`.
- **Netboot client** (`my.netboot.client.enable`): the VM has no boot disk — its `netboot` NIC
on palace's `lan-lo` bridge is matched by a kea client-class on [`river`](../river.md) and
iPXE-boots from `boot.h.nul.ie`.
- **Root on NVMe-oF**: `my.nvme.boot` connects to `nqn.2016-06.io.spdk:sfh`
([`cellar`](../cellar.md), RDMA) from the initrd (`lan-hi` up + `roceBootModules`); `/nix` and
(MAC `52:54:00:a5:7e:93`, on palace's `lan-lo` bridge, `bootindex=1`) is matched by a kea
client-class on [`river`](river.md) and iPXE-boots from `boot.h.nul.ie`.
- **Root on NVMe-oF**: `my.nvme.boot` connects to `nqn.2016-06.io.spdk:sfh` at `192.168.68.80`
([`cellar`](cellar.md), RDMA) from the initrd (`lan-hi` up + `roceBootModules`); `/nix` and
`/persist` are LVs on that volume. `KeepConfiguration=static` on `lan-hi` protects the
NVMe-oF address from networkd reconfigures.
- **Frigate footage disk**: palace passes the `hdds/frigate` LVM LV through as a virtio disk;
@@ -27,33 +26,38 @@ from `river` and runs its root off NVMe-oF from `cellar`.
## Network assignments
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `sfh`).
<!-- assignments: sfh -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| sfh | hi | `192.168.68.81/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:2/64` | h.nul.ie | |
<!-- assignments-end -->
## Networking
Four NICs, all MTU 9000 where jumbo-capable:
- `lan-hi` — SR-IOV VF 2, the box's own `hi` assignment.
- `lan-hi-ctrs` — SR-IOV VF 3, no L3: the MACVLAN parent for the containers' `hi` interfaces
- `lan-hi` — SR-IOV VF 2, the box's own `hi` assignment (`192.168.68.81`).
- `lan-hi-ctrs` — SR-IOV VF 3, no L3: the MACVLAN parent for the containers' `hi` legs
(`host0` inside each container).
- `lan-core-ctrs` / `lan-lo-ctrs` — virtio NICs (bridged to palace's `lan-core` / `lan-lo`), no
L3: MACVLAN parents for containers that need a `core` or `lo` interface.
L3: MACVLAN parents for containers that need a `core` or `lo` leg.
The per-container MACVLAN wiring lives in `systemd.nspawn.*.networkConfig` in
[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix).
[`sfh/default.nix`](../../../nixos/boxes/home/palace/vms/sfh/default.nix).
## Containers
| Container | Role |
|---|---|
| [`hass`](containers/hass.md) | Home Assistant + Frigate + MQTT |
| [`unifi`](containers/unifi.md) | UniFi controller |
| Container | Role | Page |
|---|---|---|
| `hass` | Home Assistant + Frigate + MQTT | [sfh/containers/hass.md](sfh/containers/hass.md) |
| `unifi` | UniFi controller | [sfh/containers/unifi.md](sfh/containers/unifi.md) |
## Notable config files
- [`nixos/boxes/home/palace/vms/sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix) —
- [`nixos/boxes/home/palace/vms/sfh/default.nix`](../../../nixos/boxes/home/palace/vms/sfh/default.nix) —
box config: netboot/NVMe-oF boot, container instances, MACVLAN plumbing, Frigate disk.
- [`nixos/boxes/home/palace/vms/sfh/containers/`](../../../../nixos/boxes/home/palace/vms/sfh/containers) —
- [`nixos/boxes/home/palace/vms/sfh/containers/`](../../../nixos/boxes/home/palace/vms/sfh/containers) —
the container system definitions.
- [`nixos/boxes/home/palace/vms/default.nix`](../../../../nixos/boxes/home/palace/vms/default.nix) —
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
the VM definition on `palace` (VFs, USB passthrough, netboot NIC, `hdds/frigate` disk).
docs/sites/home/sfh/containers/hass.md
+18 -16
View File
@@ -1,27 +1,23 @@
# hass
Home automation container: Home Assistant plus its supporting services (MQTT, camera restreaming,
Frigate NVR), running on [`sfh`](../README.md).
Frigate NVR), running on [`sfh`](../../sfh.md).
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/hass.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/hass.nix)
- **Host:** NixOS container on `sfh`
- **nixpkgs:** `mine`
## Role
### Home Assistant
`services.home-assistant` uses declarative configuration (`configWritable = false`). It enables
the `esphome`, `zha`, `denonavr`, `webostv`, `androidtv_remote`, `heos`, `mqtt`, `wled`, `met` and
`google_translate` components, plus custom `alarmo`, `frigate`, `west_wood_club` and Irish Rail
integrations. A `hass-cli` wrapper uses a token from `my.secrets` to reach the local server.
- **Home Assistant** (`services.home-assistant`) — declarative config (`configWritable = false`),
components `esphome`, `zha`, `denonavr`, `webostv`, `androidtv_remote`, `heos`, `mqtt`, `wled`,
`met`, `google_translate`; custom components `alarmo`, `frigate`, `west_wood_club`; a custom
Irish Rail sensor (Glenageary ↔ Dublin Connolly). A `hass-cli` wrapper is on the box, wired to
the local server with a token from `my.secrets`.
- **mosquitto** — MQTT broker (anonymous local listener; port 1883 allowed, alongside HTTP).
- **go2rtc** — restreams the Reolink living-room camera (RTSP from `reolink-living-room`, on the
`lo` network) and the office USB webcam (`/dev/video0` via ffmpeg).
`lo` leg) and the office USB webcam (`/dev/video0` via ffmpeg).
- **Frigate** (`services.frigate`, `frigate.h.nul.ie` — the `frigate` alt name on the `hi`
assignment) — records both restreamed cameras with a short retention policy; detection is
disabled.
assignment) — records both restreamed cameras with 1-day retention; detection is disabled.
- External access is via `https://hass.nul.ie` through the `middleman` reverse proxy
(`trusted_proxies`); internally it's `hass-ctr.h.nul.ie`.
- Not a deploy-rs target (`my.deploy.enable = false`) — it's rendered via `my.asContainer` and
@@ -29,7 +25,13 @@ integrations. A `hass-cli` wrapper uses a token from `my.secrets` to reach the l
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `hass`).
<!-- assignments: hass -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| hass-ctr (frigate) | hi | `192.168.68.103/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:3/64` | h.nul.ie | |
| hass-ctr-lo | lo | `192.168.72.103/21` | `2a0e:97c0:4d0:2::5:3/64` | h.nul.ie | |
<!-- assignments-end -->
## Storage
@@ -43,13 +45,13 @@ Passed through from `sfh` (USB host ports on `palace`):
- Nabu Casa Connect ZBT-1 Zigbee coordinator → `/dev/ttyUSB0` (used by `zha`).
- USB webcam → `/dev/video0` (go2rtc's `webcam_office` stream).
- The matching raw USB device node allowed through from `sfh`.
- `/dev/bus/usb/001/002` (raw USB device node).
## Networking
MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment,
MACVLAN legs created from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment,
alt name `frigate`, default gateway via the VIP) and `lan-lo` on `lan-lo-ctrs` (the `lo`
assignment, no gateway) — the `lo` interface reaches the IoT devices (the Reolink camera lives there).
assignment, no gateway) — the `lo` leg reaches the IoT devices (the Reolink camera lives there).
## Notable config files
docs/sites/home/sfh/containers/unifi.md
+21 -16
View File
@@ -1,11 +1,19 @@
# unifi
The UniFi network controller, running as a container on [`sfh`](../README.md). It manages the
The UniFi network controller, running as a container on [`sfh`](../../sfh.md). It manages the
home UniFi switch `brian` (see [switches.md](../../switches.md)).
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/unifi.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/unifi.nix)
- **Host:** NixOS container on `sfh`
- **nixpkgs:** `mine`
## Status
**Currently enabled.** The container spent a while disabled — its import was commented out of
[`containers/default.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/default.nix)
while there was no UniFi gear to manage — and was re-enabled when the UniFi switch `brian` was
added, gaining a `core` leg (`unifi-ctr-core`) at the same time so it can reach the switch on its
management network. It is imported, listed in `sfh`'s `my.containers.instances`, and
`services.unifi.enable = true`.
## Role
@@ -16,23 +24,20 @@ home UniFi switch `brian` (see [switches.md](../../switches.md)).
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `unifi`).
## Status
**Currently enabled.** The container spent a while disabled — its import was commented out of
[`containers/default.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/default.nix)
while there was no UniFi gear to manage — and was re-enabled when the UniFi switch `brian` was
added, gaining a `core` interface (`unifi-ctr-core`) at the same time so it can reach the switch on its
management network. It is imported, listed in `sfh`'s `my.containers.instances`, and
`services.unifi.enable = true`.
<!-- assignments: unifi -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| unifi-ctr-core | core | `192.168.64.21/24` | — | h.nul.ie | |
| unifi-ctr | hi | `192.168.68.100/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:1/64` | h.nul.ie | |
<!-- assignments-end -->
## Networking
Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` carries the `hi`
assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `lan-core-ctrs` carries
the gatewayless `core` assignment (`unifi-ctr-core`). The `core` interface is how the controller
talks to `brian` and the other switches on their management network.
Two MACVLAN legs from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment —
`unifi-ctr`, `192.168.68.100`, default gateway via the VIP) and `lan-core` on `lan-core-ctrs`
(the `core` assignment — `unifi-ctr-core`, `192.168.64.21/24`, no gateway). The `core` leg is
how the controller talks to the switches: `brian` lives at `192.168.64.13` on `core`.
## Notable config files
docs/sites/home/stream.md
+44 -65
View File
@@ -1,98 +1,77 @@
# stream
A physical Intel home router with a DHCP WAN from the Virgin Media cable modem. It forms the
redundant router pair with [`river`](river.md) and is dual-homed to both switches.
The secondary home router. A physical Intel box, built from `routing-common` at index 1; its WAN
is a DHCP lease from the Virgin Media cable modem, and it is dual-homed to both switches.
- **Source:** [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) (shared router
config: [`routing-common`](../../../nixos/boxes/home/routing-common), index 1)
- **Host:** physical
- **nixpkgs:** `mine`
## Hardware
| Component | Inventory |
|---|---|
| Platform | BROUNION R86S |
| CPU | Intel Celeron N5105 (4 cores / 4 threads) |
| Memory | 16 GiB |
| Storage | 512 GB Samsung SSD 970 PRO NVMe containing `/boot`, `/nix` and `/persist`; integrated 128 GB eMMC is present but is not used by the declared filesystems |
| Network | Three Intel `igc` interfaces and a dual-port Mellanox `mlx4_en` adapter; `wan`, `lan-jim` and `lan-dave` use three of these ports |
The platform configuration enables `kvm-intel`, `intel_iommu=on` and Intel microcode updates.
## Role
At `routing-common` index 1, `stream` normally holds the secondary position in the router pair.
Pair-wide addressing, VIP, DHCP/DNS and failover behavior is documented in the
[`home` networking overview](../../networking.md#home) and [Router HA](../../networking.md#router-ha).
This page covers `stream`'s Virgin Media WAN, physical platform and redundant switch attachment.
- Backup of the router pair: `routing-common` index 1 → keepalived starts `BACKUP` (priority
254), kea serves the upper-half DHCP pools, the zone's `ns2` points here. Takes over all VIPs
when [`river`](river.md) is down — see [README.md](README.md#router-vips).
- Runs the same `routing-common` services as river: keepalived/VRRP, PowerDNS recursor +
authoritative, kea DHCP + DDNS, radvd, NAT/firewall, the `as211024` L2 mesh, `iperf3`, `nginx`.
- Intel box (`kvm-intel`, `intel_iommu=on`, microcode updates).
- `octoprint` and `mjpg-streamer` (3D-printer services) are defined but **disabled**
(`enable = false`).
- `my.deploy.node.hostname` is currently commented out (it was `192.168.68.2`).
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `stream`).
<!-- assignments: stream -->
<!-- assignments-start -->
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|---|---|---|---|---|---|
| stream | as211024 | `10.100.50.3/24` | `2a0e:97c0:4df:0:1::2/64 gw 2a0e:97c0:4df:0:2::1` | — | |
| stream-core | core | `192.168.64.2/24` | — | h.nul.ie | |
| stream-hi | hi | `192.168.68.2/22` | `2a0e:97c0:4d0:1::2/64` | h.nul.ie | |
| stream-lo | lo | `192.168.72.2/21` | `2a0e:97c0:4d0:2::2/64` | h.nul.ie | |
| stream-ut | untrusted | `192.168.80.2/24` | `2a0e:97c0:4d0:3::2/64` | h.nul.ie | |
<!-- assignments-end -->
## WAN (Virgin Media DHCP)
A Quectel RM500U-EA 5G modem is being evaluated as a replacement for this WAN; it is bench-tested
only and nothing here depends on it yet. Note that its SIM is CGNAT, so it cannot carry the public
lease this section assumes — see [wwan.md](wwan.md).
- `wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) towards the cable modem. The modem segment is
switch **VLAN 130** — tagging is handled on the switch (`jim`), so the box interface itself is
untagged. The fabric side is in [switches.md](switches.md).
- `DHCP=ipv4` pulls the public lease; `dhcpV4Config.UseDNS=false` and the interface DNS points at
the local recursor. `IPv6AcceptRA=false` — this is an IPv4-only WAN (public IPv6 arrives over
the tunnel, not this link).
- A **static modem-management address** (`192.168.0.100/24`, host `.100` of `prefixes.modem.v4`)
sits on `wan` alongside the DHCP lease so the modem's web UI stays reachable; it has no
gateway.
- **`wan-online.target`** wiring: `wan-wait-online.service` (a oneshot) polls until the DHCP
default route exists, and the target `requires`/`after`s it (`wantedBy multi-user.target`).
The route — not networkd's wait-online — is the gate because the permanent static modem address
would otherwise report "online" before the public lease arrives, letting `ipsec` start
unoriented (`left=` is the public IP) and never connect.
- **CAKE QoS**: egress is shaped at the `wan` root qdisc (`Bandwidth=48M`); ingress is redirected
by `tc` (`mirred`, installed by the `networkd-dispatcher` rule in `routing-common`) into the
`wan-ifb` IFB at `Bandwidth=490M` with the DOCSIS overhead preset.
### Link and addressing
`wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) towards the cable modem. The modem segment is
switch VLAN 130; `jim` handles the tag, so the box interface is untagged. See
[switches.md](switches.md) for the fabric side.
`DHCP=ipv4` pulls the public lease. `dhcpV4Config.UseDNS=false` points resolution at the local
recursor, and `IPv6AcceptRA=false` because public IPv6 arrives over the tunnel rather than this WAN.
### Management subnet
The static `192.168.0.100/24` address (host `.100` of `prefixes.modem.v4`) sits on `wan` without a
gateway, keeping the modem UI reachable alongside the DHCP lease.
### WAN readiness
`wan-wait-online.service` polls until the DHCP default route exists, then satisfies
`wan-online.target`. The route is the gate because the permanent modem address would make
networkd's wait-online report success before the public lease arrives, allowing `ipsec` to start
without its public `left=` address.
### Traffic shaping
Egress is shaped at the `wan` root qdisc. A `routing-common` `networkd-dispatcher` rule redirects
ingress through `tc`/`mirred` into `wan-ifb`; each direction has its own configured bandwidth and
uses the DOCSIS overhead preset.
### Per-box `routing-common` options
### Modem specifics (de-shared from `routing-common`)
The modem's management subnet shares the `wan` interface, which `routing-common` itself knows
nothing about — it declares two per-box options
([`routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix)) that this
box sets:
- `my.homeRouter.dns.wanSkipBroadcasts` — skip the modem subnet's broadcast address when
- `my.homeRouter.dns.wanSkipBroadcasts = [ 192.168.0.255 ]` — skip the modem subnet when
auto-selecting the router's own `wan` A record for the zone's LUA record.
- `my.homeRouter.firewall.untrustedRejectV4` — reject untrusted clients from
- `my.homeRouter.firewall.untrustedRejectV4 = [ 192.168.0.0/24 ]` — reject untrusted clients from
reaching the modem subnet (needed only because it shares `wan`; WAN egress is otherwise
accepted).
## Switching (RSTP)
## Switching (STP)
`stream` is dual-homed to both switches: `lan-jim` (igc) and `lan-dave` (mlx4_en), both MTU 9000,
are enslaved to the `lan` bridge with `STP=true`. The explicit bridge-port costs prefer
`lan-dave` at 10 over `lan-jim` at 100. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
runs a patched `mstpd` and forces RSTP on `lan` once it is configured, so exactly one uplink carries
are enslaved to the `lan` bridge with `STP=true`. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
runs a patched `mstpd` and forces RSTP on `lan` once it's routable, so exactly one uplink carries
traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfigured.)
## Deployment
`my.deploy.node.hostname` is currently commented out.
## Disabled printer services
`octoprint` and `mjpg-streamer` are defined but disabled (`enable = false`).
## Notable config files
- [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) — box config: DHCP WAN,
docs/sites/home/switches.md
+46 -96
View File
@@ -10,19 +10,17 @@ and the ONT's untagged management is PVID'd onto VLAN 140 at brian, its edge swi
carried untranslated because a single ONT makes it unique on the fabric — see
[the WAN path](#the-digiweb-wan-path-trunked-vlan-10--pvid-140) and
[why not translation](#why-not-translation-for-one-ont). The router side lives in
[river.md](river.md); the logical network map in [networking.md](../../networking.md). The Wi-Fi
APs that hang off these switches are in [aps.md](aps.md). A fourth switch, **fergal**, hangs off jim
but belongs to the colony site — see [fergal](#fergal-portculliss-switch).
[river.md](river.md); the logical network map in [networking.md](../../networking.md).
## The switches
| | jim | dave | brian |
|---|---|---|---|
| Identity | `jim-sw` | `dave-sw` | (UniFi) |
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti Switch Pro XG 8 PoE |
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti 10GBASE-T |
| Switch chip | Marvell 98DX3236 | Marvell 98DX4310 (+ Atheros 8227 for the 1G mgmt port) | — |
| OS | RouterOS | RouterOS | UniFi |
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 (100G, breakout-capable) + 1G mgmt | 8×10GBASE-T PoE + 2×10G SFP+ |
| OS | RouterOS 7.18 | RouterOS 7.18 | UniFi |
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 (100G, breakout-capable) + 1G mgmt | 10GBASE-T |
| Bridge | `main`, `vlan-filtering=yes` | `main`, `vlan-filtering=yes` | UniFi VLAN profiles |
jim and dave run a single hardware-offloaded bridge (`main`) with VLAN filtering. Access to the
@@ -32,46 +30,47 @@ chips); brian cannot rewrite tags, only trunk/PVID them.
## Physical topology
The two WAN sources enter at the top: the Virgin Media modem lands on **jim** (VLAN 130), and the
Digiweb **ONT** lands on **brian**. Both `jim` and `brian` are edge switches that uplink down into
the **dave** core; the home boxes hang off dave's 100G ports, with backup links up to jim. jim's
second SFP+ port (`sfp-spare`, `sfp-sfpplus2`) feeds [fergal](#fergal-portculliss-switch), which
[`portcullis`](../colony/portcullis.md) hangs off while it is staged at home.
The ONT terminates on brian; jim's `wan-pon-in` (`sfp-sfpplus2`) is a spare SFP+ port.
```
Virgin Media cable modem Digiweb ONT
stream WAN, VLAN 130 river WAN, management + VLAN 10
| |
jim ---- 10G ---- fergal ---- portcullis brian
| 10G trunk 802.3ad LAG |
+--------------------+ +---------------+
| |
+---+----------+---+
| dave |
+--------+---------+
|
+---------------------+---------------------+
| | |
palace (100G) castle (100G) stream
river VM NVMe-oF root second router
Backup links to jim (normally idle):
* palace: 1G
* stream: 1G; STP selects the active link
* castle: 2.5G, normally down; no live failover
Virgin Media cable modem
│
│ (VLAN 130)
│ wan1/wan2/wan-in
┌──────────────────────────────────────────────────┴──┐
│ jim CRS326-24G-2S+ (Marvell 98DX3236) │
│ 1G edge ports: castle, fort, pronter, laptop-dock, │
│ palace-kvm, ups, ether15-20, wan1/wan2/wan-in │
│ wan-pon-in (= sfp-sfpplus2) ← spare SFP+ │
└───────┬───────────────────────────────────┬─────────┘
dave-uplink ┤ (= sfp-sfpplus1) │ stream, palace (1G secondaries)
10G trunk│ └── stream (box, dual-homed)
┌─────────┴───────────────────────────────────────────────┐
│ dave CRS504-4XQ (Marvell 98DX4310) │
│ jim-downlink = qsfp28-3-1 │
└──┬───────────┬──────────┬───────────────────┬────────────┘
palace(100G) │ castle │ stream │ brian-downlink
= qsfp28-1-1 │ │ │ (802.3ad LAG: brian1+brian2)
┌────────────────┴─┐ … │
│ palace host │ Digiweb (PPPoE) via ONT
│ └ river (VM) │ │
└──────────────────┘ ┌─────┴─────┐
↑ river's WAN + LAN ride the 100G link │ ONT │ untagged mgmt 192.168.100.1
└─────┬─────┘ + tagged VLAN 10 (ISP)
│
┌────────┴────────┐
│ brian (UniFi) │ PVID 140 on the ONT port,
│ 10GBASE-T │ tagged VLAN 10 allowed through
└────────┬────────┘
└── brian-downlink LAG up to dave
```
Notes:
- **river** runs as a VM on the **palace** host; its uplink is dave's 100G `palace` port. jim also
has 1G `palace`/`stream` ports, but those are secondary links and do **not** carry the WAN.
- **stream** (the second router box) is dual-homed to both jim and dave (STP picks the active path).
### Castle storage dependency
`castle` is dual-homed without STP: its primary link is dave's 100G `castle` port (`et100g`), while
the 2.5G link to jim (`et2.5g`) is normally down and provides no live failover. Its root disk is
NVMe-oF over `et100g` and dave, so interrupting either freezes `castle` mid-I/O. Do dave maintenance
from a box that does not depend on it, or power `castle` off cleanly first.
- **brian** is a Ubiquiti 10GBASE-T switch, downlinked from dave over an **802.3ad LAG**
(`brian-downlink` = `brian1` + `brian2`, layer-2 hash). It hosts the ONT.
## VLANs
@@ -88,9 +87,7 @@ from a box that does not depend on it, or power `castle` off cleanly first.
Switch L3 presence (`/interface vlan` on `main`) exists **only** for VLANs the switch is managed
from — `hi` (100) and `lo` (110), plus native core. WAN and guest VLANs deliberately have no switch
L3 interface. jim and dave carry a static IPv4 and **global IPv6** address on `hi`/`lo` (plus a
static default route on each stack) purely for management — they are **pure L2, never routers**. See
[Switches must not route](#switches-must-not-route).
L3 interface.
## The Digiweb WAN path (trunked VLAN 10 + PVID 140)
@@ -118,9 +115,9 @@ ONT.** See [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/p
for the river side.
```
ONT -- untagged + VLAN 10 -- brian -- VLAN 140 + VLAN 10 -- dave -- palace -- river
|
+-- ONT port PVID 140; VLAN 10 remains tagged
ONT ──(untagged + VLAN10)── brian ──(VLAN140 + VLAN10)── dave ──(VLAN140 + VLAN10)── river
ONT port │ PVID140 + tagged 10 │ plain bridging
└─ brian-downlink LAG ── dave ┘
```
### Why not translation (for one ONT)?
@@ -148,40 +145,8 @@ VLAN 140 also spans `brian-downlink,palace` (it carries a few other members too)
this is plain tagged bridging.
**jim (RouterOS)** — carries **none** of the Digiweb WAN path: no translation rules, and no VLAN
10/140/141 rows. jim only handles stream's VLAN-130 WAN and the LAN VLANs. `sfp-spare`
(`sfp-sfpplus2`) stays at `pvid=1` — the switch feeding `portcullis` is reached over VLAN 1
untagged — and is a **tagged** member of `hi` (100) and `lo` (110) so those reach `portcullis`:
```
/interface bridge vlan set [find bridge=main vlan-ids=100] tagged=...,sfp-spare
/interface bridge vlan set [find bridge=main vlan-ids=110] tagged=...,sfp-spare
```
## Switches must not route
jim and dave (and the `vibe` AP) are **pure L2** — river/stream do all routing. Their per-stack
management addresses and static default routes exist only so the boxes themselves can be reached and
reach out; they must **never** forward traffic or advertise themselves as routers. RouterOS defaults
work against this: `ip-forward` and IPv6 `forward` ship **on**, and with IPv6 forwarding on RouterOS
also emits Router Advertisements (`ra-lifetime=30m`) on every L3 interface — so a switch silently
becomes a competing IPv6 default router. This surfaced after the 7.18 → 7.23 upgrade, when clients
picked up dave/jim as default routers alongside river.
The required config on each RouterOS box:
```
/ip settings set ip-forward=no
/ipv6 settings set forward=no accept-router-advertisements=no
/ipv6 nd set [find] ra-lifetime=0
```
- `ip-forward=no` / `forward=no` — no L3 forwarding on either stack; IPv6 `forward=no` also stops RA
emission at the source.
- `accept-router-advertisements=no` — with forwarding off RouterOS would otherwise start *accepting*
RAs; this keeps the box on its deterministic **static** default route.
- `ra-lifetime=0` — belt-and-suspenders: even if forwarding is ever re-enabled the box advertises
router-lifetime 0 (i.e. "not a default router"). Setting it also emits a withdrawal RA that
actively clears the rogue default from clients (they otherwise cache it for up to ~30 min).
**After any RouterOS upgrade, re-check `/ip settings` and `/ipv6 settings`** — an upgrade can reset
these to the forwarding-on defaults. brian (UniFi) is not a RouterOS box and was not affected.
10/140/141 rows. `wan-pon-in` (`sfp-sfpplus2`) sits at `pvid=1` as a spare port. jim only handles
stream's VLAN-130 WAN and the LAN VLANs.
## Future: multiple ONTs (per-port VLAN translation)
@@ -207,33 +172,18 @@ Each ONT port must also be a tagged member of bridge VLAN 10 for correct egress
piece that otherwise shows up as pppd "Timeout waiting for PADO"). The pins bypass the FDB, so the
two ISP sessions never mix.
**Why a new switch:** jim (the only box with spare SFP+ *and* the translation feature) had just
**one** free SFP+ port — now taken by fergal — so it can't host two ONTs. The plan is a dedicated
**Why a new switch:** jim (the only box with spare SFP+ *and* the translation feature) has just
**one** free SFP+ port, so it can't host two ONTs. The plan is a dedicated
**CRS305-1G-4S+** (4×SFP+, same Marvell rule support) to land multiple ONTs and do the per-port
translation there, feeding distinct fabric VLANs up to dave.
## fergal (portcullis's switch)
**fergal** is an 8-port SFP+ switch running OpenWrt, hanging off jim's `sfp-spare` port. It belongs
to [`portcullis`](../colony/portcullis.md) rather than to the home fabric — it is here only while
that box is staged at home, and goes to Nikhef with it. Nothing in the home fabric depends on it.
What it borrows from home is VLAN 1 untagged on the jim uplink (fergal's own management sits on it,
at `192.168.64.30` on core) plus tagged `hi` (100) and `lo` (110), so `portcullis` can reach those
over 10G. The switch itself — VLAN layout, flash layout, firmware and flashing notes — is
documented in [sites/colony/fergal.md](../colony/fergal.md).
## Accessing the switches
The switches resolve by **short hostname** on the home network — the home routers serve their
records in the home zone
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix):
`jim` → hi `.10`, `dave` → hi `.11`, `brian` → core `.13`). From a box on the home network just
`ssh admin@jim` / `admin@dave`.
**Key auth** for `admin` is installed on jim/dave (and the `vibe` AP) — `ssh -i ~/.ssh/id_rsa
admin@jim` works keyless (imported via `/user ssh-keys import`). Password `admin`/`admin` remains as
a fallback. Non-interactive password pattern (avoids the ssh-agent hang) if the key isn't available:
`ssh admin@jim` / `admin@dave`. Non-interactive pattern (password auth, avoids the ssh-agent hang):
```
sshpass -p admin ssh -o IdentityAgent=none -o PubkeyAuthentication=no \
docs/sites/home/wwan.md
-139
View File
@@ -1,139 +0,0 @@
# Home 5G WWAN modem
Reference for the Quectel **RM500U-EA** USB 5G modem, bought to eventually replace `stream`'s
Virgin Media cable WAN ([stream.md](stream.md)). Like the switches ([switches.md](switches.md)) and
the APs ([aps.md](aps.md)), it is **not** managed by this flake: the module's own settings live in
its NVRAM and are applied out-of-band over AT, and nothing in the repo references it yet.
As of 2026-08-05 it has only been bench-tested on `tower`; `stream` is untouched. The notes below
are the working knowledge from that session — what the module needs in order to connect at all, and
what is still unresolved.
## The hardware
| | |
|---|---|
| Model | Quectel **RM500U-EA** |
| Platform | UNISOC-based (not Qualcomm) — its AT set is the `AT+QCFG`/`AT+QNETDEVCTL` router-firmware family, not the QMI one |
| USB | `2c7c:0900`, SuperSpeed (USB 3.1) |
| Firmware | `RM500UEAAAR03A13M2G` |
| SIM | **GoMo**, which rides eir's network (MCC/MNC **272-03**, AS**15751** Meteor Mobile) |
The SIM's **PIN lock has been disabled** on the SIM itself, so no PIN needs to be entered at boot
and no age secret is required for it. Card identifiers and the PIN are deliberately not recorded
here; read them from the modem with `mmcli` if needed.
## The module must be in MBIM mode
This is the single most important setting. The module ships in **NCM** mode (`AT+QCFG="usbnet",5`),
and in that mode it does not work:
- The PDP context comes up correctly — `AT+CGPADDR` reports a real address and `AT+CGCONTRDP`
reports the APN, DNS servers and prefix — but the `cdc_ncm` interface **never raises carrier**, so
no traffic can leave the box. No combination of `AT+QNETDEVCTL` modes (the `(0-3)` operations,
per profile) changed that.
- ModemManager also mis-reports the module's capability as `gsm-umts` only, and cannot read signal
quality (it sits at 0% while the radio is registered and attached).
Switching to **MBIM** fixes both:
```
AT+QCFG="usbnet",2
AT+CFUN=1,1 # reset so the new USB composition takes effect
```
It then enumerates as `cdc_mbim` with `/dev/cdc-wdm0` and a `wwp*` interface, ModemManager reports
`gsm-umts, lte, 5gnr`, signal quality works, and carrier follows the bearer. The other `usbnet`
values the module advertises are `(1,2,3,5,11,13,15)` — `1` ECM, `2` MBIM, `3` RNDIS, `5` NCM.
To reach the AT ports (`ttyUSB2` and `ttyUSB3` are the AT ones; ModemManager claims them, so stop it
first), any serial terminal works — `minicom -D /dev/ttyUSB2`, or a raw `stty`/`exec` pair on the
device node.
### Router-mode features are not in use
The firmware is the router variant: it can do its own NAT (`AT+QCFG="nat"`) and hand the host a
lease off a private LAN prefix (`AT+QCFG="lanip"`, default `192.168.42.0/24`). It is currently in
bridge mode (`nat=0`) so the host gets the real WAN address. NAT mode was not needed once MBIM
worked, and would mean double NAT.
## Connecting: the APN must be the network-expanded form
The documented consumer APNs (`gomo.ie`, `data.myeirmobile.ie`) **fail**. The connect only succeeds
with the fully expanded name the network itself uses, and only as **IPv4**:
```
mmcli -m <n> --simple-connect="apn=data.myeirmobile.ie.mnc003.mcc272.gprs,ip-type=ipv4"
```
The module has `AT+QCFG="autoapn",1`, so it selects an APN by itself during attach and brings up an
initial EPS bearer regardless. That bearer is where the expanded name comes from: list the modem's
bearers and read the one whose type is `default-attach`.
```
mmcli -m <n> # note the bearer paths and the initial bearer path
mmcli -b <n> # the default-attach bearer carries the real APN
```
Failure modes are worth distinguishing, since they look similar from `mmcli`:
| Symptom | Meaning |
|---|---|
| `MBIM status error: Failure`, immediate | The APN reached the network and was rejected — usually the wrong APN string |
| `Network timeout`, after a long wait | The APN never resolved to anything; wrong name entirely |
| `No valid data port found` | Already connected — the single data port is in use by an existing bearer |
## Bench result on tower
Measured 2026-08-05 on `tower`, indoors, with **no external antennas** and a weak signal
(RSSI around −85 dBm):
| | |
|---|---|
| Access technology | `lte, 5gnr` — 5G **NSA** |
| Throughput | ~64 Mbit/s down, ~26 Mbit/s up |
| Latency | ~76 ms to `1.1.1.1` |
| Bearer-negotiated rates | 150 Mbit/s down, 50 Mbit/s up |
Treat the throughput as a floor, not a characterisation — antennas and siting were both worst-case.
## The address is CGNAT, and the prefix length is a trap
Two separate consequences of how the bearer addresses the host.
### No public IP
The bearer address is in `100.64.0.0/10` and egress is carrier-NAT'd (`*.cgn1.srl.meteor.ie`,
AS15751). There is **no inbound reachability and no public address**. `stream` currently takes a
*public* DHCP lease on `wan` and publishes it — see [stream.md](stream.md#wan-virgin-media-dhcp),
which also drives `my.homeRouter.dns.wanSkipBroadcasts`. Replacing that WAN with this SIM therefore
drops port forwards, inbound WireGuard and anything resolving to `stream`'s WAN address. Making
this a real WAN needs either a public/static IP from the carrier, or `stream`'s inbound
reachability moved onto the AS211024 mesh or a tunnel from `britway`
([networking.md](../../networking.md)).
### The bearer reports a /8
ModemManager reports the address with a **`/8` prefix**, i.e. `100.0.0.0/8`. Configuring that
literally would install a route covering **Tailscale's `100.64.0.0/10`** and break it. Any
configuration for this modem must add the address as a `/32` with an explicit on-link route to the
gateway, and never use the bearer's own prefix length.
For a throwaway test that cannot disturb the box, put the address and default route in their own
routing table behind an `ip rule` matching the source address, and drive traffic onto it with
`ping -I <addr>` / `curl --interface <addr>`.
## Still open
- **IPv6.** GoMo is expected to provide it, but `ip-type=ipv4v6` fails to connect and only
`ip-type=ipv4` works. In NCM mode the module *did* report an IPv6 address and IPv6 DNS servers
(`2001:bb0::11`/`::12`) on the context, so the network clearly offers it — this looks like an APN
or MBIM-session problem rather than a carrier one. Worth retrying with a separate IPv6-only
context, or with the initial EPS bearer settings pinned via
`mmcli --3gpp-set-initial-eps-bearer-settings`.
- **A public or static IP** from GoMo/eir, without which this cannot replace `stream`'s WAN
unchanged (see above).
- **Antenna siting**, and whether 5G **SA** is reachable rather than the NSA seen so far.
- **Flake integration** — nothing exists yet. It would need the MBIM interface configured under
`stream`'s networkd, a `wan-online.target` mechanism equivalent to the current DHCP-route gate,
and a decision on whether ModemManager or a plain `mbimcli` connect script drives the bearer.
flake.lock
Generated
+301 -210
View File
@@ -67,6 +67,34 @@
"type": "github"
}
},
"bun2nix": {
"inputs": {
"flake-parts": "flake-parts",
"nixpkgs": [
"pi-agent",
"nixpkgs"
],
"systems": [
"pi-agent",
"systems"
],
"treefmt-nix": "treefmt-nix_2"
},
"locked": {
"lastModified": 1778446047,
"narHash": "sha256-oQvcadh2BCkrog+SGrG6YffKJrveYpjj3TdQJWaKhaM=",
"owner": "nix-community",
"repo": "bun2nix",
"rev": "f2bc12af1a6369648aac41041ceeaa0b866599c6",
"type": "github"
},
"original": {
"owner": "nix-community",
"ref": "2.1.0",
"repo": "bun2nix",
"type": "github"
}
},
"copyparty": {
"inputs": {
"flake-utils": "flake-utils_5",
@@ -75,11 +103,11 @@
]
},
"locked": {
"lastModified": 1787524125,
"narHash": "sha256-P48TOQdIbB0PKMn4FTk6X0utbf0LemNlJ+bFcSbveGA=",
"lastModified": 1781351267,
"narHash": "sha256-86HFs1K+LRlx8t4AjaMdU5qlg4O7kLz1VlnNapKZIuY=",
"owner": "9001",
"repo": "copyparty",
"rev": "9de090265f8d063056320f41d984830839017a2f",
"rev": "90639de9840d7dcc2d9000026fe547f666c1d550",
"type": "github"
},
"original": {
@@ -90,11 +118,11 @@
},
"crane": {
"locked": {
"lastModified": 1787326676,
"narHash": "sha256-lWhBbBvC05/xwivKBBiM2YNizpmgqCgyOIzomvRuwxs=",
"lastModified": 1780532242,
"narHash": "sha256-D+BsdpxmtUwtqGoY0IXPhHgTlmqgcZKCEo1oMyn7ep0=",
"owner": "ipetkov",
"repo": "crane",
"rev": "692f7e9ef2ece8125b466f66f2af532b3edaed0d",
"rev": "59a82a1222dd3b2080b5cc52a1a2e8d5f1b77f37",
"type": "github"
},
"original": {
@@ -150,11 +178,11 @@
"utils": "utils"
},
"locked": {
"lastModified": 1786361680,
"narHash": "sha256-IxaZkb9rCGEZ+yGndxKXONeIEcKMzoFUsvLTB5G/caw=",
"lastModified": 1781023725,
"narHash": "sha256-Gt+qFANcrDRjl3xzidLYrAUQCd3808iuAsLwZbYYAEU=",
"owner": "serokell",
"repo": "deploy-rs",
"rev": "16901271e5b30b591e56f7a84f25f186fb20f3e1",
"rev": "2ce9051767ee4d1a3c43b52ba327431783bfd463",
"type": "github"
},
"original": {
@@ -163,27 +191,6 @@
"type": "github"
}
},
"determinate-nix": {
"inputs": {
"flake-parts": "flake-parts",
"git-hooks-nix": "git-hooks-nix",
"nixpkgs": "nixpkgs_4",
"nixpkgs-23-11": "nixpkgs-23-11",
"nixpkgs-regression": "nixpkgs-regression"
},
"locked": {
"lastModified": 1787334067,
"narHash": "sha256-wmwgSBcAGJe/e+FrLwJxlghYV12F7UkIodm0j6cosYg=",
"rev": "c407745c8b9b616bebf7288697699c45794e31ac",
"revCount": 27248,
"type": "tarball",
"url": "https://api.flakehub.com/f/pinned/DeterminateSystems/nix-src/3.22.2/01a02595-e77f-7e43-a616-5bbc77a2dc07/source.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://flakehub.com/f/DeterminateSystems/nix-src/%2A"
}
},
"devshell": {
"inputs": {
"flake-utils": "flake-utils",
@@ -203,6 +210,25 @@
"type": "github"
}
},
"devshell-tools": {
"inputs": {
"flake-utils": "flake-utils_10",
"nixpkgs": "nixpkgs_5"
},
"locked": {
"lastModified": 1710099997,
"narHash": "sha256-WmBKTLdth6I/D+0//9enbIXohGsBjepbjIAm9pCYj0U=",
"owner": "eikek",
"repo": "devshell-tools",
"rev": "e82faf976d318b3829f6f7f6785db6f3c7b65267",
"type": "github"
},
"original": {
"owner": "eikek",
"repo": "devshell-tools",
"type": "github"
}
},
"devshell_2": {
"inputs": {
"flake-utils": "flake-utils_3",
@@ -258,55 +284,20 @@
"type": "github"
}
},
"flake-compat_2": {
"flake": false,
"locked": {
"lastModified": 1767039857,
"narHash": "sha256-vNpUSpF5Nuw8xvDLj2KCwwksIbjua2LZCqhV1LNRDns=",
"owner": "NixOS",
"repo": "flake-compat",
"rev": "5edf11c44bc78a0d334f6334cdaf7d60d732daab",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "flake-compat",
"type": "github"
}
},
"flake-parts": {
"inputs": {
"nixpkgs-lib": [
"determinate-nix",
"pi-agent",
"bun2nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1782949081,
"narHash": "sha256-vp6Y/Grm98ESt6ceOkWiHWyZRDV3J1RID4w+6NWK9yA=",
"rev": "17c9d6cdfc60c64f4ee8d306f9bc0b4ccb51481e",
"revCount": 480,
"type": "tarball",
"url": "https://api.flakehub.com/f/pinned/hercules-ci/flake-parts/0.1.480%2Brev-17c9d6cdfc60c64f4ee8d306f9bc0b4ccb51481e/019f2195-dee5-7233-9747-eca0c27f7406/source.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://flakehub.com/f/hercules-ci/flake-parts/0.1"
}
},
"flake-parts_2": {
"inputs": {
"nixpkgs-lib": [
"openwrt-imagebuilder",
"nixpkgs"
]
},
"locked": {
"lastModified": 1772408722,
"narHash": "sha256-rHuJtdcOjK7rAHpHphUb1iCvgkU3GpfvicLMwwnfMT0=",
"lastModified": 1777988971,
"narHash": "sha256-qIoWPDs+0/8JecyYgE3gpKQxW/4bLW/gp45vow9ioCQ=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "f20dc5d9b8027381c474144ecabc9034d6a839a3",
"rev": "0678d8986be1661af6bb555f3489f2fdfc31f6ff",
"type": "github"
},
"original": {
@@ -333,6 +324,57 @@
"type": "github"
}
},
"flake-utils_10": {
"inputs": {
"systems": "systems_10"
},
"locked": {
"lastModified": 1709126324,
"narHash": "sha256-q6EQdSeUZOG26WelxqkmR7kArjgWCdw5sfJVHPH/7j8=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "d465f4819400de7c8d874d50b982301f28a84605",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"flake-utils_11": {
"inputs": {
"systems": "systems_11"
},
"locked": {
"lastModified": 1705309234,
"narHash": "sha256-uNRRNRKmJyCRC/8y1RqBkqWBLM034y4qN7EprSdmgyA=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "1ef2e671c3b0c19053962c07dbda38332dcebf26",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"flake-utils_12": {
"locked": {
"lastModified": 1667395993,
"narHash": "sha256-nuEHfE/LcWyuSWnS8t12N1wc105Qtau+/OdUAjtQ0rA=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "5aed5285a952e0b949eb3ba02c12fa4fcfef535f",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"flake-utils_2": {
"inputs": {
"systems": "systems_2"
@@ -468,27 +510,6 @@
"type": "github"
}
},
"git-hooks-nix": {
"inputs": {
"flake-compat": "flake-compat_2",
"nixpkgs": [
"determinate-nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1784288435,
"narHash": "sha256-ReRHaLgr/uVqdD8afFSn+myXIfpHeOhP0yYe0TJqAA8=",
"rev": "43b3c1ab9d40fb1dbb008f451988a91e375825e9",
"revCount": 1231,
"type": "tarball",
"url": "https://api.flakehub.com/f/pinned/cachix/git-hooks.nix/0.1.1231%2Brev-43b3c1ab9d40fb1dbb008f451988a91e375825e9/019f7135-8fdf-76f0-b1a1-d2c67e91af8d/source.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://flakehub.com/f/cachix/git-hooks.nix/0.1.941"
}
},
"harmonia": {
"inputs": {
"crane": "crane",
@@ -499,11 +520,11 @@
"treefmt-nix": "treefmt-nix"
},
"locked": {
"lastModified": 1787502072,
"narHash": "sha256-K5sKCAV3kPbUW0evsqpWrlQRsa2t0jfkduSZ+lRWAA8=",
"lastModified": 1781128165,
"narHash": "sha256-97WpKZkaNAL5g7MtASLwqnrJrvrLpQRr6cXWiRNLiXQ=",
"owner": "nix-community",
"repo": "harmonia",
"rev": "7c1ef262e324bbf61201fe92a73849eb3d6fd9e2",
"rev": "f0dd1094cdc8d72e038cf9347cacfa9272a8f72d",
"type": "github"
},
"original": {
@@ -520,11 +541,11 @@
]
},
"locked": {
"lastModified": 1781447016,
"narHash": "sha256-bxZ8XTdUFQRWsh6rZn7fCui/SV4ox7dUAiSg4zYJuDg=",
"lastModified": 1781402797,
"narHash": "sha256-pBdDca7xv1nuP0kj+gC5g5AcR/DV+9Zy3CS6uDOMdJ4=",
"owner": "devplayer0",
"repo": "hass-west-wood",
"rev": "fd43bede6e1175d9118c42507b737041b8923787",
"rev": "3e6ef7a9084e4053c82dea20127a775e7bcf77a5",
"type": "github"
},
"original": {
@@ -562,11 +583,11 @@
]
},
"locked": {
"lastModified": 1787377438,
"narHash": "sha256-Sxu1NLTD/Ern6hFGLlZmtKCSct3YQXZI/lls8RE1XeM=",
"lastModified": 1781319724,
"narHash": "sha256-ZGuxexEMo4Xv28KJ0dX/m/PHN4oZIOnxHZpNTyrvx4M=",
"owner": "nix-community",
"repo": "home-manager",
"rev": "65258d5c65a250189fde2e35f490d15e064c4c62",
"rev": "8355f0a16b2dbb06a97959a918af5b239bbe05ae",
"type": "github"
},
"original": {
@@ -582,11 +603,11 @@
]
},
"locked": {
"lastModified": 1787487906,
"narHash": "sha256-zIdM+8teujHm5hc5MIPDnV7k2UeOOT/pFyFtWjOCwsY=",
"lastModified": 1781305496,
"narHash": "sha256-g8Vv4Qfc7n+lgov97REu3X6BeJtvYY0hlSUZR1GrGQQ=",
"owner": "nix-community",
"repo": "home-manager",
"rev": "cfba7ad5886b342b8dd63ba74354b3853ea4cfc9",
"rev": "c87a39aa979acc4848016d2220c6238390d84779",
"type": "github"
},
"original": {
@@ -599,7 +620,7 @@
"home-manager": [
"home-manager-unstable"
],
"nixpkgs": "nixpkgs_5"
"nixpkgs": "nixpkgs_4"
},
"locked": {
"lastModified": 1769548169,
@@ -615,14 +636,29 @@
"type": "github"
}
},
"jail-nix": {
"locked": {
"lastModified": 1776230864,
"narHash": "sha256-YsEjjdOsGEzTeD+iT7ONh071BqWAOQWpzYVei3okAXE=",
"owner": "~alexdavid",
"repo": "jail.nix",
"rev": "404e7da9da5ab9aa643666682b2ba1312fa5fbe8",
"type": "sourcehut"
},
"original": {
"owner": "~alexdavid",
"repo": "jail.nix",
"type": "sourcehut"
}
},
"libnetRepo": {
"flake": false,
"locked": {
"lastModified": 1781446676,
"narHash": "sha256-b3rJDKxzsf7p4wI698iBi2PInDPRH3KwjdqOk/SahKk=",
"lastModified": 1776595118,
"narHash": "sha256-6bIEi8q5hXCHU9nApTbQXvpljMWldg3QipCD+jkOGK8=",
"owner": "oddlama",
"repo": "nixos-extra-modules",
"rev": "f097b474fcb5db7dfd52263c055c9e6caeb13d62",
"rev": "84207afebb794be7b53cfc9768730f37c64f4a13",
"type": "github"
},
"original": {
@@ -634,11 +670,11 @@
"nix": {
"flake": false,
"locked": {
"lastModified": 1787394889,
"narHash": "sha256-qtDusLx9yn0aME9D9Oe5QhFnmDUaARwMJo/vt4+DtIU=",
"lastModified": 1780652321,
"narHash": "sha256-o/6YXRB6AbeL4SYtSHlJ9oEROl6Wmf7yheJNa3fAv2I=",
"owner": "nixos",
"repo": "nix",
"rev": "88b09c64fbea076a0376830d98e5331f70ed31a3",
"rev": "d1f04a798cf4276da59567c07a3bf4a628669288",
"type": "github"
},
"original": {
@@ -684,29 +720,13 @@
"type": "github"
}
},
"nixpkgs-23-11": {
"locked": {
"lastModified": 1717159533,
"narHash": "sha256-oamiKNfr2MS6yH64rUn99mIZjc45nGJlj9eGth/3Xuw=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a62e6edd6d5e1fa0329b8653c801147986f8d446",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a62e6edd6d5e1fa0329b8653c801147986f8d446",
"type": "github"
}
},
"nixpkgs-mine": {
"locked": {
"lastModified": 1787612836,
"narHash": "sha256-25KxhEJHYVZXAwsHQbXpyaG9/WpWGo9EmGe7kzMc25Y=",
"lastModified": 1781356656,
"narHash": "sha256-Ygkl3ZBJ434/WhwdK1FyvPMeHvNPAopg3KE/1HtcJuk=",
"owner": "devplayer0",
"repo": "nixpkgs",
"rev": "c92598bc3fd46ff4d23407045091eea206120979",
"rev": "a15e20705db295f621cb5bb63613f03a9373323f",
"type": "github"
},
"original": {
@@ -718,11 +738,11 @@
},
"nixpkgs-mine-stable": {
"locked": {
"lastModified": 1787523195,
"narHash": "sha256-NI87OKi5hXSZlIgh5Gwjjca52MAJnwRaU/+Su99fMqg=",
"lastModified": 1781356876,
"narHash": "sha256-s8ed+zuk5wrbyhtDQpkxycAcLmhQH9umGRuVRBNKUbU=",
"owner": "devplayer0",
"repo": "nixpkgs",
"rev": "2a058ae98b603146eae51e6a268854ce0ad035a1",
"rev": "2eb8bacf9f641d4510fc43ba7fc0eea7dfdf5b24",
"type": "github"
},
"original": {
@@ -732,29 +752,13 @@
"type": "github"
}
},
"nixpkgs-regression": {
"locked": {
"lastModified": 1643052045,
"narHash": "sha256-uGJ0VXIhWKGXxkeNnq4TvV3CIOkUJ3PAoLZ3HMzNVMw=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
}
},
"nixpkgs-stable": {
"locked": {
"lastModified": 1787414105,
"narHash": "sha256-WncT27+3BOkgTaJZLnCsf3LcYf9RXMuR9ONSN4rzQ7s=",
"lastModified": 1780902259,
"narHash": "sha256-q8yYEC5f1mFlQO9RGna4LTc9QrcvWunX6FYp83munkQ=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a9e6d84f9c2f9012f5fe7d964a7851352300e61a",
"rev": "bd0ff2d3eac24699c3664d5966b9ef36f388e2ca",
"type": "github"
},
"original": {
@@ -765,11 +769,11 @@
},
"nixpkgs-unstable": {
"locked": {
"lastModified": 1787360063,
"narHash": "sha256-dt4WdcvsA8/RCe+VZZwqU0X+XMM3wBbGCWA0/sFWzGo=",
"lastModified": 1781074563,
"narHash": "sha256-md8WlXOlfnIeHeOScMTTHFyf2d6iaTwPl2apR5EQ3P4=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "2c423e03bbafcff28bfadc6781a4a8257f205cb5",
"rev": "9ae611a455b90cf061d8f332b977e387bda8e1ca",
"type": "github"
},
"original": {
@@ -809,22 +813,6 @@
}
},
"nixpkgs_4": {
"locked": {
"lastModified": 1784160687,
"narHash": "sha256-iYL/bixrb6FlHFu/gIuBYzq6c6lM5AAXsXNSWXtIgQc=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "4382ed2b7a6839d4280a9b386db49cbc5907414d",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "4382ed2b7a6839d4280a9b386db49cbc5907414d",
"type": "github"
}
},
"nixpkgs_5": {
"locked": {
"lastModified": 1768564909,
"narHash": "sha256-Kell/SpJYVkHWMvnhqJz/8DqQg2b6PguxVWOuadbHCc=",
@@ -840,48 +828,58 @@
"type": "github"
}
},
"openwrt-feeds": {
"inputs": {
"nixpkgs": [
"nixpkgs-unstable"
],
"openwrt-imagebuilder": [
"openwrt-imagebuilder"
]
},
"nixpkgs_5": {
"locked": {
"lastModified": 1787522666,
"narHash": "sha256-Ev4X1HCx6aV4L5GtMQX9DJRgWQ7K914rT11nL7mIkmw=",
"owner": "devplayer0",
"repo": "openwrt-feeds",
"rev": "53720becf1e89473660e25107ae5d23bf1465621",
"lastModified": 1709309926,
"narHash": "sha256-VZFBtXGVD9LWTecGi6eXrE0hJ/mVB3zGUlHImUs2Qak=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "79baff8812a0d68e24a836df0a364c678089e2c7",
"type": "github"
},
"original": {
"owner": "devplayer0",
"repo": "openwrt-feeds",
"owner": "NixOS",
"ref": "nixos-23.11",
"repo": "nixpkgs",
"type": "github"
}
},
"openwrt-imagebuilder": {
"nixpkgs_6": {
"locked": {
"lastModified": 1674990008,
"narHash": "sha256-4zOyp+hFW2Y7imxIpZqZGT8CEqKmDjwgfD6BzRUE0mQ=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "d2bbcbe6c626d339b25a4995711f07625b508214",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"pi-agent": {
"inputs": {
"flake-parts": "flake-parts_2",
"bun2nix": "bun2nix",
"jail-nix": "jail-nix",
"nixpkgs": [
"nixpkgs-unstable"
],
"systems": "systems_7"
},
"locked": {
"lastModified": 1787474509,
"narHash": "sha256-jL5RS/TbKk7HxjsGyFWeceHveyRgM8btvfY9Z77P9jM=",
"owner": "astro",
"repo": "nix-openwrt-imagebuilder",
"rev": "4371439b1e4e582266fc38345d1a01db1f8db6d6",
"lastModified": 1784984457,
"narHash": "sha256-y7jv+RTP0TkcJaDm7/D2xIFF9JDWKc6g5L3X3EUu3Eo=",
"owner": "lukasl-dev",
"repo": "pi.nix",
"rev": "fd2c62853f66a5803dae621769570f1165e59b80",
"type": "github"
},
"original": {
"owner": "astro",
"repo": "nix-openwrt-imagebuilder",
"owner": "lukasl-dev",
"repo": "pi.nix",
"type": "github"
}
},
@@ -937,7 +935,6 @@
"borgthin": "borgthin",
"copyparty": "copyparty",
"deploy-rs": "deploy-rs",
"determinate-nix": "determinate-nix",
"devshell": "devshell_3",
"flake-utils": "flake-utils_6",
"harmonia": "harmonia",
@@ -951,9 +948,9 @@
"nixpkgs-mine-stable": "nixpkgs-mine-stable",
"nixpkgs-stable": "nixpkgs-stable",
"nixpkgs-unstable": "nixpkgs-unstable",
"openwrt-feeds": "openwrt-feeds",
"openwrt-imagebuilder": "openwrt-imagebuilder",
"ragenix": "ragenix"
"pi-agent": "pi-agent",
"ragenix": "ragenix",
"sharry": "sharry"
}
},
"rust-overlay": {
@@ -964,11 +961,11 @@
]
},
"locked": {
"lastModified": 1787454509,
"narHash": "sha256-r4LDUF+zmJnkftvCVkCrUhSJazsf6EVJF+V2l4/MYbI=",
"lastModified": 1761791894,
"narHash": "sha256-myRIDh+PxaREz+z9LzbqBJF+SnTFJwkthKDX9zMyddY=",
"owner": "oxalica",
"repo": "rust-overlay",
"rev": "f60c1b57ff805a46b5175c76fc981fb4f81efbcc",
"rev": "59c45eb69d9222a4362673141e00ff77842cd219",
"type": "github"
},
"original": {
@@ -977,6 +974,48 @@
"type": "github"
}
},
"sbt": {
"inputs": {
"flake-utils": "flake-utils_12",
"nixpkgs": "nixpkgs_6"
},
"locked": {
"lastModified": 1698464090,
"narHash": "sha256-Pnej7WZIPomYWg8f/CZ65sfW85IfIUjYhphMMg7/LT0=",
"owner": "zaninime",
"repo": "sbt-derivation",
"rev": "6762cf2c31de50efd9ff905cbcc87239995a4ef9",
"type": "github"
},
"original": {
"owner": "zaninime",
"repo": "sbt-derivation",
"type": "github"
}
},
"sharry": {
"inputs": {
"devshell-tools": "devshell-tools",
"flake-utils": "flake-utils_11",
"nixpkgs": [
"nixpkgs-unstable"
],
"sbt": "sbt"
},
"locked": {
"lastModified": 1741328331,
"narHash": "sha256-OtsHm9ykxfAOMRcgFDsqFBBy5Wu0ag7eq1qmTIluVcw=",
"owner": "eikek",
"repo": "sharry",
"rev": "6203b90f9a76357d75c108a27ad00f323d45c1d0",
"type": "github"
},
"original": {
"owner": "eikek",
"repo": "sharry",
"type": "github"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
@@ -992,6 +1031,36 @@
"type": "github"
}
},
"systems_10": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"systems_11": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"systems_2": {
"locked": {
"lastModified": 1681028828,
@@ -1069,16 +1138,16 @@
},
"systems_7": {
"locked": {
"lastModified": 1680978846,
"narHash": "sha256-Gtqg8b/v49BFDpDetjclCYXm8mAnTrUzR0JnE2nv5aw=",
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "x86_64-linux",
"rev": "2ecfcac5e15790ba6ce360ceccddb15ad16d08a8",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "x86_64-linux",
"repo": "default",
"type": "github"
}
},
@@ -1120,11 +1189,33 @@
]
},
"locked": {
"lastModified": 1786901030,
"narHash": "sha256-WSFCsDSE5ffgD2MqzkM2CYjeFiKhRF/dJUN8uedb6YE=",
"lastModified": 1780220602,
"narHash": "sha256-eynAfOmbmxJnkp7YewvCEbShNnnYJ9gLLqkzsYtBPeM=",
"owner": "numtide",
"repo": "treefmt-nix",
"rev": "27b3b12a8e6375f28ebe122f07d230ca5459bbfa",
"rev": "db947814a175b7ca6ded66e21383d938df01c227",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "treefmt-nix",
"type": "github"
}
},
"treefmt-nix_2": {
"inputs": {
"nixpkgs": [
"pi-agent",
"bun2nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1775636079,
"narHash": "sha256-pc20NRoMdiar8oPQceQT47UUZMBTiMdUuWrYu2obUP0=",
"owner": "numtide",
"repo": "treefmt-nix",
"rev": "790751ff7fd3801feeaf96d7dc416a8d581265ba",
"type": "github"
},
"original": {
flake.nix
+6 -65
View File
@@ -1,19 +1,6 @@
{
description = "System configs";
# Offer our Harmonia cache when building the flake itself, so `nix develop` / `nix build` don't
# rebuild from source. Nix reads `nixConfig` before the flake evaluates and rejects any computed
# value (imports/thunks), so these must stay literal — keep them in sync with `lib.my.c.nix.cache`.
# Consumers must trust these (accept-flake-config / a trusted user) for them to take effect.
nixConfig = {
extra-substituters = [
"https://nix-cache.nul.ie"
];
extra-trusted-public-keys = [
"nix-cache.nul.ie-1:BzH5yMfF4HbzY1C977XzOxoPhEc9Zbu39ftPkUbH+m4="
];
};
inputs = {
flake-utils.url = "github:numtide/flake-utils";
# libnet.url = "github:reo101/nix-lib-net";
@@ -34,13 +21,6 @@
home-manager-stable.url = "home-manager/release-26.05";
home-manager-stable.inputs.nixpkgs.follows = "nixpkgs-stable";
# Determinate Nix, used as the common Nix implementation across systems, homes, the devshell and
# CI (see lib.my.c.nix). We build it ourselves (FlakeHub's cache needs auth), so it flows through
# our own Harmonia cache like everything else. Keep its tested nixpkgs pin: its packaging carries
# compatibility patches that can conflict with newer nixpkgs patches.
determinate-nix.url = "https://flakehub.com/f/DeterminateSystems/nix-src/*";
determinate-nix.inputs.nixpkgs.url = "github:NixOS/nixpkgs/4382ed2b7a6839d4280a9b386db49cbc5907414d";
# Stuff used by the flake for build / deployment
# ragenix.url = "github:yaxitech/ragenix";
ragenix.url = "github:devplayer0/ragenix/add-rekey-one-flag";
@@ -59,16 +39,9 @@
# harmonia.url = "github:devplayer0/harmonia/cache-config-daemon-store";
harmonia.inputs.nixpkgs.follows = "nixpkgs-unstable";
# Firmware building for the OpenWrt boxes, which aren't managed by this flake otherwise.
# `openwrt-feeds` pins the package feeds; without it, evaluation would reach the OpenWrt
# download server over import-from-derivation and break on hashes that upstream rotates daily.
openwrt-imagebuilder.url = "github:astro/nix-openwrt-imagebuilder";
openwrt-imagebuilder.inputs.nixpkgs.follows = "nixpkgs-unstable";
openwrt-feeds.url = "github:devplayer0/openwrt-feeds";
openwrt-feeds.inputs.nixpkgs.follows = "nixpkgs-unstable";
openwrt-feeds.inputs.openwrt-imagebuilder.follows = "openwrt-imagebuilder";
# Packages not in nixpkgs
sharry.url = "github:eikek/sharry";
sharry.inputs.nixpkgs.follows = "nixpkgs-unstable";
borgthin.url = "github:devplayer0/borg";
# TODO: Update borgthin so this works
# borgthin.inputs.nixpkgs.follows = "nixpkgs-mine";
@@ -76,13 +49,8 @@
copyparty.inputs.nixpkgs.follows = "nixpkgs-unstable";
hass-west-wood.url = "github:devplayer0/hass-west-wood";
hass-west-wood.inputs.nixpkgs.follows = "nixpkgs-unstable";
# Disabled: `pi-coding-agent-bun` breaks `nix flake check` in CI. Its bun2nix `fetchBunDeps`
# calls `builtins.filterSource` on subpaths of the pi.nix flake source (coding-agent/bun.nix),
# which requires that source derivation to be realised in the local store. On a fresh CI runner
# it isn't, so eval aborts with `path '…-source.drv' is not valid`. Works locally only because
# the source is already realised there. Re-enable (and revisit upstream) if we start using pi.
# pi-agent.url = "github:lukasl-dev/pi.nix";
# pi-agent.inputs.nixpkgs.follows = "nixpkgs-unstable";
pi-agent.url = "github:lukasl-dev/pi.nix";
pi-agent.inputs.nixpkgs.follows = "nixpkgs-unstable";
};
outputs =
@@ -110,22 +78,6 @@
};
pkgsLibOverlay = final: prev: { lib = prev.lib.extend libOverlay; };
myPkgsOverlay = final: prev: import ./pkgs { lib = final.lib; pkgs = prev; };
# Exposes Determinate Nix under a stable attr name so systems, homes and the devshell all
# resolve the exact same package (referenced as `pkgs'.mine.determinate-nix` in configs).
# `nix-util`'s `readLinkAt.works` unit test creates PATH_MAX-length symlinks, which our CI
# runner's XFS-backed build filesystem rejects (XFS hard-caps symlink targets at 1024 bytes).
# Skip just that test via gtest's GTEST_FILTER so the rest of the suite still gates the build.
determinateOverlay = final: prev: {
determinate-nix =
(inputs.determinate-nix.packages.${prev.stdenv.hostPlatform.system}.default).overrideAttrs (o: {
checkInputs = map
(drv:
if (drv.name or "") == "nix-util-tests-run"
then drv.overrideAttrs (_: { GTEST_FILTER = "-readLinkAt.*"; })
else drv)
o.checkInputs;
});
};
# Override the flake-level lib since we're going to use it for non-config specific stuff
pkgsFlakes = mapAttrs (_: pkgsFlake: pkgsFlake // { lib = pkgsFlake.lib.extend libOverlay; }) {
@@ -154,7 +106,6 @@
pkgsLibOverlay
myPkgsOverlay
determinateOverlay
inputs.devshell.overlays.default
inputs.ragenix.overlays.default
inputs.deploy-rs.overlays.default
@@ -170,7 +121,6 @@
pkgsLibOverlay
myPkgsOverlay
determinateOverlay
];
config = {
@@ -189,7 +139,6 @@
# Systems
nixos/installer.nix
nixos/boxes/colony
nixos/boxes/colony/portcullis
nixos/boxes/tower
nixos/boxes/home/stream.nix
nixos/boxes/home/palace
@@ -233,13 +182,7 @@
nixosModules = nixfiles.config.nixos.modules;
homeModules = nixfiles.config.home-manager.modules;
# Containers and the installer override `rendered` with a bare `extendModules` config
# (`my.asContainer` / `my.asISO`) that lacks the `pkgs`/`lib` attrs `eval-config` exposes on a
# normal system. Determinate Nix's flake schemas read `machine.pkgs.stdenv.system` for every
# `nixosConfigurations` entry, so re-attach them from the full system eval (`configuration`).
nixosConfigurations = mapAttrs
(_: s: s.rendered // { inherit (s.configuration) pkgs lib; })
nixfiles.config.nixos.systems;
nixosConfigurations = mapAttrs (_: s: s.rendered) nixfiles.config.nixos.systems;
homeConfigurations = mapAttrs (_: s: s.configuration) nixfiles.config.home-manager.homes;
deploy = nixfiles.config.deploy-rs.rendered;
@@ -267,9 +210,7 @@
deploy = recurseIntoAttrs (pkgs.deploy-rs.lib.deployChecks self.deploy);
};
packages = flattenTree (
(import ./pkgs { inherit lib pkgs; }) //
(import ./openwrt { inherit pkgs inputs; }));
packages = flattenTree (import ./pkgs { inherit lib pkgs; });
devShells.default = shell;
home-manager/modules/common.nix
+7 -8
View File
@@ -2,14 +2,13 @@
let
inherit (builtins) listToAttrs mapAttrs readFile;
inherit (lib)
optionalString nameValuePair concatMapStrings concatStringsSep optionalAttrs
optionalString nameValuePair concatMapStrings concatStringsSep optionalAttrs versionAtLeast
mapAttrsToList mkMerge mkIf mkDefault mkOption;
inherit (lib.hm) dag;
inherit (lib.my) mkOpt' dummyOption;
in
{
# Disabled: breaks `nix flake check` in CI (see flake.nix pi-agent note).
# imports = [ inputs.pi-agent.homeModules.default ];
imports = [ inputs.pi-agent.homeModules.default ];
options = with lib.types; {
my = {
@@ -50,7 +49,8 @@ in
};
nix = {
settings = with lib.my.c.nix; determinateSettings // {
package = mkIf (!(versionAtLeast config.home.stateVersion "22.11")) pkgs.nix;
settings = with lib.my.c.nix; {
experimental-features = [ "nix-command" "flakes" "ca-derivations" ];
max-jobs = mkDefault "auto";
@@ -216,7 +216,6 @@ in
jq
yq-go
nix-tree
treemd
];
sessionVariables = {
@@ -233,7 +232,7 @@ in
inputs.deploy-rs.overlays.default
inputs.boardie.overlays.default
inputs.nixGL.overlays.default
# inputs.pi-agent.overlays.default
inputs.pi-agent.overlays.default
];
config = {
allowUnfree = true;
@@ -256,13 +255,13 @@ in
ssh.authKeys.files = [ lib.my.c.sshKeyFiles.me ];
};
nix.package = pkgs'.mine.determinate-nix;
nix.package = mkIf (versionAtLeast config.home.stateVersion "22.05") pkgs.nix;
fonts.fontconfig.enable = true;
home = {
packages = with pkgs; [
pkgs'.mine.determinate-nix
pkgs'.mine.nix
];
# Without this, we are at the mercy of whatever version of nix is in $PATH...
home-manager/modules/gui/default.nix
+19 -38
View File
@@ -1,6 +1,6 @@
{ lib, pkgs', pkgs, config, ... }:
let
inherit (lib) genAttrs mkIf mkMerge mkForce mapAttrs mkOptionDefault mkDefault optional;
inherit (lib) genAttrs mkIf mkMerge mkForce mapAttrs mkOptionDefault mkDefault;
inherit (lib.my) mkOpt' mkBoolOpt';
inherit (lib.my.c) pubDomain;
@@ -45,9 +45,6 @@ let
chmod +x "$out"/bin/doomsaver
'';
doomsaver = doomsaver' cfg.screensaver.brainrotTextCommand;
firefoxMemoryControl = pkgs.firefox-memory-control.override {
inherit (cfg.firefoxMemoryControl) lowAvailableMiB highAvailableMiB pollIntervalMs minInactiveMs;
};
in
{
options.my.gui = with lib.types; {
@@ -55,26 +52,12 @@ in
manageGraphical = mkBoolOpt' false "Configure the graphical session";
standalone = mkBoolOpt' false "Enable settings for fully Nix managed systems";
screensaver.brainrotTextCommand = mkOpt' (either path str) genLipsum "Command to generate brainrot text.";
firefoxMemoryControl = {
enable = mkBoolOpt' pkgs.stdenv.isLinux "Enable memory-pressure tab unloading in Firefox";
lowAvailableMiB = mkOpt' ints.positive 2048 "Available memory threshold at which Firefox starts unloading tabs.";
highAvailableMiB = mkOpt' ints.positive 3072 "Available memory threshold at which Firefox stops unloading tabs.";
pollIntervalMs = mkOpt' ints.positive 1000 "Memory-pressure polling interval in milliseconds.";
minInactiveMs = mkOpt' ints.unsigned 300000 "Minimum tab inactivity before automatic unloading, in milliseconds.";
};
};
config = mkIf cfg.enable (mkMerge [
{
assertions = [
{
assertion = cfg.firefoxMemoryControl.highAvailableMiB > cfg.firefoxMemoryControl.lowAvailableMiB;
message = "`my.gui.firefoxMemoryControl.highAvailableMiB` must exceed `lowAvailableMiB`.";
}
];
home = {
packages = (with pkgs; [
packages = with pkgs; [
xdg-utils
font.package
@@ -105,18 +88,17 @@ in
ffmpeg-full
xournalpp
# Disabled: breaks `nix flake check` in CI (see flake.nix pi-agent note).
# (pkgs.symlinkJoin {
# name = "pi-coding-agent";
# buildInputs = [ pkgs.makeWrapper ];
# paths = [ pkgs.pi-coding-agent-bun ];
# postBuild = ''
# wrapProgram $out/bin/pi \
# --set NPM_CONFIG_PREFIX ${config.home.homeDirectory}/.pi/npm/ \
# --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.nodejs_latest ]}
# '';
# })
]) ++ optional cfg.firefoxMemoryControl.enable firefoxMemoryControl;
(pkgs.symlinkJoin {
name = "pi-coding-agent";
buildInputs = [ pkgs.makeWrapper ];
paths = [ pkgs.pi-coding-agent-bun ];
postBuild = ''
wrapProgram $out/bin/pi \
--set NPM_CONFIG_PREFIX ${config.home.homeDirectory}/.pi/npm/ \
--prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.nodejs_latest ]}
'';
})
];
};
programs = {
@@ -395,9 +377,8 @@ in
"XF86AudioRaiseVolume" = "exec ${pkgs.pamixer}/bin/pamixer -i 5";
"XF86AudioLowerVolume" = "exec ${pkgs.pamixer}/bin/pamixer -d 5";
# Some AVRCP devices alternate play and pause events independently of player state.
"XF86AudioPlay" = "exec ${pkgs.playerctl}/bin/playerctl play-pause";
"XF86AudioPause" = "exec ${pkgs.playerctl}/bin/playerctl play-pause";
"XF86AudioPlay" = "exec ${pkgs.playerctl}/bin/playerctl play";
"XF86AudioPause" = "exec ${pkgs.playerctl}/bin/playerctl pause";
"XF86AudioNext" = "exec ${pkgs.playerctl}/bin/playerctl next";
"XF86AudioPrev" = "exec ${pkgs.playerctl}/bin/playerctl previous";
};
@@ -423,12 +404,12 @@ in
gtk = {
enable = true;
theme = {
name = "Adwaita";
package = pkgs.gnome-themes-extra;
name = "Numix";
package = pkgs.numix-gtk-theme;
};
gtk4.theme = {
name = "Adwaita";
package = pkgs.gnome-themes-extra;
name = "Numix";
package = pkgs.numix-gtk-theme;
};
iconTheme = {
name = "Numix";
lib/constants.nix
+1 -20
View File
@@ -30,7 +30,7 @@ rec {
kernel = {
lts = pkgs: pkgs.linuxKernel.packages.linux_6_18;
latest = pkgs: pkgs.linuxKernel.packages.linux_7_2;
latest = pkgs: pkgs.linuxKernel.packages.linux_7_0;
};
nginx = rec {
@@ -111,13 +111,6 @@ rec {
extra-trusted-public-keys = ${concatStringsSep " " keys}
'';
};
# Determinate-specific settings enabling its performance features. Only understood by the
# Determinate Nix binary, so they must not be emitted for a base-Nix package.
determinateSettings = {
lazy-trees = true;
eval-cores = 0;
};
};
pubDomain = "nul.ie";
@@ -314,8 +307,6 @@ rec {
home = rec {
domain = "h.${pubDomain}";
# Search domains advertised to clients (radvd DNSSL / kea domain-search)
searchDomains = [ domain "dyn.${domain}" colony.domain britway.domain ];
vlans = {
hi = 100;
lo = 110;
@@ -392,16 +383,6 @@ rec {
};
};
# networkConfig fragment anchoring a VLAN client's DNS on the router pair's
# VRRP VIPs rather than the RA RDNSS. v6 addresses here are RA/token-derived,
# so when RA is absent (e.g. v6 disabled) there is no v6 and no RDNSS at all;
# the always-present static v4 VIP keeps name resolution working, with the v6
# VIP as a bonus when v6 is up. Merge into the VLAN network's networkConfig.
vlanDns = vlan: {
DNS = [ vips.${vlan}.v4 vips.${vlan}.v6 ];
Domains = searchDomains;
};
roceBootModules = [ "ib_core" "ib_uverbs" "mlx5_core" "mlx5_ib" ];
};
lib/default.nix
+3 -7
View File
@@ -107,7 +107,7 @@ rec {
then throw "\nFailed assertions:\n${concatStringsSep "\n" (map (x: "- ${x}") failedAssertions)}"
else showWarnings config.warnings res;
homeStateVersion' = hmBranch: "23.05";
homeStateVersion' = hmBranch: (if (hmBranch == "stable" || hmBranch == "mine-stable") then "22.11" else "23.05");
homeStateVersion = hmBranch: {
# The flake passes a default setting, but we don't care about that
home.stateVersion = mkForce (homeStateVersion' hmBranch);
@@ -149,10 +149,6 @@ rec {
(optional (a.ipv6.gateway != null) a.ipv6.gateway);
networkConfig = {
IPv6AcceptRA = a.ipv6.gateway == null || a.ipv6.iid != null;
# These are servers: temporary (privacy) addresses only rotate our stable source
# address out from under long-lived connected sockets (e.g. nginx's resolver, which
# wedges permanently when the address it bound to expires).
IPv6PrivacyExtensions = "no";
# NOTE: LLDP emission / reception is ignored on bridge interfaces
LLDP = true;
EmitLLDP = "customer-bridge";
@@ -195,7 +191,7 @@ rec {
};
lvmDisk' = vg: lv: lvmDisk'' lv vg lv;
lvmDisk = lvmDisk' "main";
disk = vm: lv: lvmDisk'' lv "main" "vm-${vm}-${lv}";
disk = vm: lv: lvmDisk' lv "vm-${vm}-${lv}";
};
deploy-rs =
@@ -253,7 +249,7 @@ rec {
in
{
trivial = prev.trivial // {
release = "26.08:u-${prev.trivial.release}";
release = "26.06:u-${prev.trivial.release}";
codeName = "Irritating";
revisionWithDefault = default: self.rev or default;
versionSuffix = ".${date}.${revCode self}:u-${revCode pkgsFlake}";
nixos/boxes/colony/portcullis/default.nix
-175
View File
@@ -1,175 +0,0 @@
{ lib, ... }:
let
inherit (lib.my) net;
inherit (lib.my.c.colony) domain;
home = lib.my.c.home;
in
{
nixos.systems.portcullis = {
system = "x86_64-linux";
nixpkgs = "mine-stable";
home-manager = "mine-stable";
assignments = {
# Staging-only: the 10G link lands on the home hi VLAN until portcullis is racked.
hi = {
domain = home.domain;
mtu = home.hiMTU;
ipv4 = {
address = net.cidr.host 41 home.prefixes.hi.v4;
mask = 22;
gateway = home.vips.hi.v4;
};
ipv6 = {
iid = "::6:1";
address = net.cidr.host (65536*6+1) home.prefixes.hi.v6;
};
};
};
configuration = { lib, pkgs, config, assignments, ... }:
let
inherit (lib) mkMerge;
inherit (lib.my) mkVLAN networkdAssignment;
inherit (lib.my.c) networkd;
# Router-sized rings rather than the driver defaults, and GRO kept across
# forwarding so UDP-encapsulated traffic stays batched.
nicTuning = {
RxBufferSize = 4096;
TxBufferSize = 4096;
GenericReceiveOffloadUDPForwarding = true;
};
in
{
hardware = {
enableRedistributableFirmware = true;
cpu = {
intel.updateMicrocode = true;
};
};
boot = {
kernelModules = [ "kvm-intel" ];
# Passthrough mode keeps the IOMMU available without paying DMA translation
# on the forwarding path.
kernelParams = [ "intel_iommu=on" "iommu=pt" ];
initrd = {
availableKernelModules = [ "xhci_pci" "nvme" "usb_storage" "usbhid" "sd_mod" "sr_mod" ];
kernelModules = [ "dm-snapshot" ];
};
};
fileSystems = {
"/boot" = {
device = "/dev/disk/by-uuid/1A70-EBCB";
fsType = "vfat";
options = [ "fmask=0022" "dmask=0022" ];
};
"/nix" = {
device = "/dev/main/portcullis-nix";
fsType = "ext4";
};
"/persist" = {
device = "/dev/main/portcullis-persist";
fsType = "ext4";
neededForBoot = true;
};
};
networking = { inherit domain; };
# The I226-V link-drop erratum is driven by EEE as well as ASPM. The driver already
# leaves EEE off, so this pins a default rather than changing one; systemd.link has
# no knob for it.
services.udev.extraRules = ''
ACTION=="add", SUBSYSTEM=="net", DRIVERS=="igc", RUN+="${pkgs.ethtool}/bin/ethtool --set-eee $name eee off"
'';
environment.systemPackages = with pkgs; [
pciutils
usbutils
ethtool
lm_sensors
smartmontools
];
systemd.network = {
# Only some ports are patched in while the box is being staged, so don't block
# boot on the others coming up.
wait-online.anyInterface = true;
netdevs = mkVLAN "lan-hi" home.vlans.hi;
links = {
"10-et2g5-0" = {
matchConfig.PermanentMACAddress = "00:d0:b4:05:ed:48";
linkConfig = nicTuning // { Name = "et2g5-0"; };
};
"10-et2g5-1" = {
matchConfig.PermanentMACAddress = "00:d0:b4:05:ed:49";
linkConfig = nicTuning // { Name = "et2g5-1"; };
};
"10-et2g5-2" = {
matchConfig.PermanentMACAddress = "00:d0:b4:05:ed:4a";
linkConfig = nicTuning // { Name = "et2g5-2"; };
};
"10-et2g5-3" = {
matchConfig.PermanentMACAddress = "00:d0:b4:05:ed:4b";
linkConfig = nicTuning // { Name = "et2g5-3"; };
};
"11-et10g-0" = {
matchConfig.PermanentMACAddress = "60:be:b4:2e:9b:a2";
linkConfig = nicTuning // { Name = "et10g-0"; };
};
"11-et10g-1" = {
matchConfig.PermanentMACAddress = "60:be:b4:2e:9b:a3";
linkConfig = nicTuning // { Name = "et10g-1"; };
};
};
networks = {
# TODO: replace with the colony assignments and routing config once portcullis is
# racked at Nikhef. Until then it is staged at home, so every 2.5G port takes DHCP on
# the lo VLAN and whichever one is patched in provides connectivity. kea registers
# the DHCP hostname, making the box reachable as `portcullis.dyn.h.nul.ie`.
"80-bootstrap" = {
matchConfig.Name = "et2g5-*";
DHCP = "yes";
networkConfig.IPv6PrivacyExtensions = "no";
linkConfig.RequiredForOnline = "routable";
};
# 10G up to jim's spare SFP+ port via an intermediary switch. That uplink is
# untagged VLAN 1, so hi has to be tagged on its own interface.
"81-et10g-0" = {
matchConfig.Name = "et10g-0";
vlan = [ "lan-hi" ];
networkConfig = networkd.noL3;
linkConfig = {
# The carrier has to allow hi's jumbo frames before lan-hi can take that MTU
MTUBytes = toString home.hiMTU;
RequiredForOnline = "no";
};
};
"82-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
{ networkConfig = home.vlanDns "hi"; }
];
};
};
my = {
# As above: no colony assignment yet, so deploy over the staging hi address.
deploy.node.hostname = assignments.hi.ipv4.address;
secrets = {
key = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAUolR93Byg+Daw8pUYHVpQ34ioxSc2C8vzj9F4KbqMs";
};
server.enable = true;
};
};
};
}
nixos/boxes/colony/vms/default.nix
+2 -2
View File
@@ -212,8 +212,8 @@
cleanShutdown.timeout = 120;
drives = [
(mkMerge [ (vm.disk "darts" "root") { frontendOpts.bootindex = 0; } ])
(vm.lvmDisk'' "media" "main" "darts-media")
(vm.lvmDisk'' "ext" "main" "darts-ext")
(vm.lvmDisk' "media" "darts-media")
(vm.lvmDisk' "ext" "darts-ext")
];
};
};
nixos/boxes/colony/vms/estuary/dns.nix
+1 -5
View File
@@ -114,12 +114,8 @@ in
];
also-notify = [ "127.0.0.1" ];
allow-axfr-ips = [
"127.0.0.0/8" "::1/128"
"216.218.133.2" "2001:470:600::2"
]
++ lib.my.c.home.routersPubV4
++ lib.my.c.as211024.trusted.v4
++ lib.my.c.as211024.trusted.v6;
];
enable-lua-records = true;
#loglevel = 7;
#log-dns-queries = true;
nixos/boxes/colony/vms/git/default.nix
+2 -4
View File
@@ -38,8 +38,6 @@ in
let
inherit (lib) mkMerge;
inherit (lib.my) networkdAssignment;
podmanSubnet = "10.88.0.0/16";
in
{
imports = [
@@ -160,7 +158,7 @@ in
oci-containers = {
backend = "podman";
};
containers.containersConf.settings.network.default_subnet = podmanSubnet;
containers.containersConf.settings.network.default_subnet = "10.88.0.0/16";
};
systemd.network = {
@@ -197,7 +195,7 @@ in
extraRules = ''
table inet filter {
chain forward {
ip saddr ${podmanSubnet} accept
ip saddr 10.88.0.0/16 accept
}
}
'';
nixos/boxes/colony/vms/git/gitea-actions.nix
-21
View File
@@ -1,14 +1,7 @@
{ lib, pkgs, config, ... }:
let
inherit (lib) mkForce;
inherit (lib.my) net;
inherit (lib.my.c) pubDomain;
# The podman bridge gateway (first host of the default subnet); job
# containers reach the runner's artifact cache server here, through a single
# fixed port opened in the firewall below.
podmanGateway = net.cidr.host 1 config.virtualisation.containers.containersConf.settings.network.default_subnet;
cachePort = 34567;
in
{
config = {
@@ -41,11 +34,6 @@ in
cache = {
enabled = true;
dir = "/var/cache/gitea-runner";
# Announce the podman bridge gateway rather than let act_runner
# autodetect the box's outbound address, which containers can't
# route back to.
host = podmanGateway;
port = cachePort;
};
};
};
@@ -85,15 +73,6 @@ in
group = "gitea-runner";
};
};
# Let job containers reach the runner's artifact cache server on the host.
firewall.extraRules = ''
table inet filter {
chain input {
iifname "podman0" tcp dport ${toString cachePort} accept
}
}
'';
};
};
}
nixos/boxes/colony/vms/git/gitea.nix
-10
View File
@@ -43,19 +43,9 @@ in
(umask 027; gitea_extra_setup)
'';
# Uploaded release assets are buffered through a temp file before being stored.
# The default /tmp is on the small tmpfs root, so keep them on the state volume.
environment.TMPDIR = "${config.services.gitea.stateDir}/tmp";
}
];
};
tmpfiles.settings."10-gitea-tmp"."${config.services.gitea.stateDir}/tmp".d = {
user = config.services.gitea.user;
group = config.services.gitea.group;
mode = "0700";
};
};
services = {
nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix
+9
View File
@@ -336,6 +336,15 @@ in
useACMEHost = pubDomain;
};
"share.${pubDomain}" = {
locations."/" = {
proxyPass = "http://object-ctr.${domain}:9090";
proxyWebsockets = true;
extraConfig = proxyHeaders;
};
useACMEHost = pubDomain;
};
"stuff.${pubDomain}" = {
locations."/" = {
proxyPass = "http://jackflix-ctr.${domain}:3923";
nixos/boxes/colony/vms/shill/containers/object.nix
+68
View File
@@ -47,6 +47,10 @@ in
key = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFdHbZErWLmTPO/aEWB1Fup/aGMf31Un5Wk66FJwTz/8";
files = {
"object/minio.env" = {};
"object/sharry.conf" = {
owner = "sharry";
group = "sharry";
};
"object/minio-client-config.json" = {
owner = config.my.user.config.name;
group = config.my.user.config.group;
@@ -61,6 +65,7 @@ in
firewall = {
tcp.allowed = [
9000 9001
config.services.sharry.config.bind.port
8069
5000
config.services.hedgedoc.settings.port
@@ -117,6 +122,8 @@ in
};
};
sharry = awaitPostgres;
atticd = mkMerge [
awaitPostgres
{
@@ -168,6 +175,66 @@ in
dataDir = [ "/mnt/minio" ];
};
sharry = {
enable = true;
configOverridesFile = config.age.secrets."object/sharry.conf".path;
config = {
base-url = "https://share.${lib.my.c.pubDomain}";
bind.address = "::";
alias-member-enabled = true;
webapp = {
chunk-size = "64M";
};
backend = {
auth = {
fixed = {
enabled = true;
user = "dev";
};
internal = {
enabled = true;
order = 50;
};
};
jdbc = {
url = "jdbc:postgresql://colony-psql:5432/sharry";
user = "sharry";
};
files = {
default-store = "minio";
stores = {
database.enabled = false;
minio = {
enabled = true;
type = "s3";
endpoint = "https://s3.nul.ie";
access-key = "share";
bucket = "share";
};
};
};
compute-checksum.parallel = 4;
signup.mode = "invite";
share = {
max-size = "128G";
max-validity = "3650 days";
};
mail = {
enabled = true;
smtp = {
host = "mail.nul.ie";
port = 587;
user = "sharry@nul.ie";
ssl-type = "starttls";
default-from = "Sharry <sharry@nul.ie>";
timeout = "30 seconds";
};
};
};
};
};
atticd = {
enable = false;
environmentFile = config.age.secrets."object/atticd.env".path;
@@ -241,6 +308,7 @@ in
forwardPorts = [
{ from = "host"; host.port = 9000; guest.port = 9000; }
{ from = "host"; host.port = 9001; guest.port = 9001; }
{ from = "host"; guest.port = config.services.sharry.config.bind.port; }
];
};
})
nixos/boxes/colony/vms/shill/containers/toot.nix
+2 -3
View File
@@ -48,9 +48,8 @@ in
"s3-secret-key.txt"
])
(_: with config.services.mastodon; {
# user doesn't exist any more, so this breaks on deploy
# owner = user;
# inherit group;
owner = user;
inherit group;
})) // {
"toot/pds.env" = {
owner = "pds";
nixos/boxes/home/castle/default.nix
+3 -10
View File
@@ -2,7 +2,7 @@
let
inherit (lib.my) net;
inherit (lib.my.c) networkd;
inherit (lib.my.c.home) domain vlans prefixes vips vlanDns roceBootModules;
inherit (lib.my.c.home) domain vlans prefixes vips roceBootModules;
in
{
nixos.systems.castle = {
@@ -189,18 +189,11 @@ in
};
"40-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
{
networkConfig = vlanDns "hi" // {
# So we don't drop the IP we use to connect to NVMe-oF!
KeepConfiguration = "static";
};
}
# So we don't drop the IP we use to connect to NVMe-oF!
{ networkConfig.KeepConfiguration = "static"; }
];
"45-lan-lo" = {
matchConfig.Name = "lan-lo";
# The parent carries hi's jumbo frames, but lo runs at the standard MTU;
# without this the VLAN would inherit the parent's larger one.
linkConfig.MTUBytes = "1500";
networkConfig = {
DHCP = "ipv4";
IPv6AcceptRA = true;
nixos/boxes/home/palace/default.nix
+2 -5
View File
@@ -2,7 +2,7 @@
let
inherit (lib.my) net mkVLAN;
inherit (lib.my.c) pubDomain;
inherit (lib.my.c.home) domain vlans prefixes vips vlanDns hiMTU;
inherit (lib.my.c.home) domain vlans prefixes vips hiMTU;
in
{
imports = [ ./vms ];
@@ -203,10 +203,7 @@ in
MACAddress=52:54:00:90:34:95
'';
};
"60-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
{ networkConfig = vlanDns "hi"; }
];
"60-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
"50-lan-core-phy" = {
matchConfig.Name = "lan-core-phy";
nixos/boxes/home/palace/vms/cellar/default.nix
+2 -5
View File
@@ -2,7 +2,7 @@
let
inherit (lib.my) net;
inherit (lib.my.c) pubDomain;
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
inherit (lib.my.c.home) domain prefixes vips hiMTU;
in
{
nixos.systems.cellar = {
@@ -79,10 +79,7 @@ in
};
networks = {
"80-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
{ networkConfig = vlanDns "hi"; }
];
"80-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
};
};
nixos/boxes/home/palace/vms/river.nix
-9
View File
@@ -145,15 +145,6 @@
};
};
# networkd's wait-online knows nothing about the pppd-owned `wan` interface, so
# network-online.target is reached long before there's a route off-site. Gate the
# installer fetch on the WAN instead, and retry it whenever the link returns.
systemd.services.netboot-update = {
after = [ "wan-online.target" ];
wantedBy = mkForce [ "wan-online.target" ];
partOf = [ "wan-online.target" ];
};
systemd.network = {
netdevs = mkMerge [
(mkVLAN "wan-pon-ont" vlans.wan-pon-ont)
nixos/boxes/home/palace/vms/sfh/containers/hass.nix
+30 -11
View File
@@ -2,7 +2,7 @@
let
inherit (lib.my) net;
inherit (lib.my.c) pubDomain;
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
inherit (lib.my.c.home) domain prefixes vips hiMTU;
in
{
nixos.systems.hass = { config, ... }: {
@@ -47,9 +47,8 @@ in
inherit (lib) mkMerge mkIf mkForce;
inherit (lib.my) networkdAssignment;
hassPort = 8123;
hassCli = pkgs.writeShellScriptBin "hass-cli" ''
export HASS_SERVER="http://localhost:${toString hassPort}"
export HASS_SERVER="http://localhost:${toString config.services.home-assistant.config.http.server_port}"
export HASS_TOKEN="$(< ${config.age.secrets."hass/cli-token.txt".path})"
exec ${pkgs.home-assistant-cli}/bin/hass-cli "$@"
'';
@@ -70,7 +69,7 @@ in
};
firewall = {
tcp.allowed = [ "http" hassPort 1883 ];
tcp.allowed = [ "http" 1883 ];
};
};
@@ -83,10 +82,7 @@ in
systemd = {
network.networks = {
"80-container-host0" = mkMerge [
(networkdAssignment "host0" assignments.hi)
{ networkConfig = vlanDns "hi"; }
];
"80-container-host0" = networkdAssignment "host0" assignments.hi;
"80-container-lan-lo" = networkdAssignment "lan-lo" assignments.lo;
};
};
@@ -165,7 +161,29 @@ in
};
};
home-assistant = {
home-assistant =
let
cfg = config.services.home-assistant;
pyirishrail = ps: ps.buildPythonPackage rec {
pname = "pyirishrail";
version = "0.0.2";
src = pkgs.fetchFromGitHub {
owner = "ttroy50";
repo = "pyirishrail";
tag = version;
hash = "sha256-NgARqhcXP0lgGpgBRiNtQaSn9JcRNtCcZPljcL7t3Xc=";
};
dependencies = with ps; [
requests
];
pyproject = true;
build-system = [ ps.setuptools ];
};
in
{
enable = true;
extraComponents = [
@@ -187,6 +205,7 @@ in
isal
gtts
(pyirishrail python3Packages)
];
customComponents = with pkgs.home-assistant-custom-components; [
alarmo
@@ -195,6 +214,7 @@ in
];
configWritable = false;
openFirewall = true;
config = {
default_config = {};
homeassistant = {
@@ -204,10 +224,9 @@ in
country = "IE";
time_zone = "Europe/Dublin";
external_url = "https://hass.${pubDomain}";
internal_url = "http://hass-ctr.${domain}:${toString hassPort}";
internal_url = "http://hass-ctr.${domain}:${toString cfg.config.http.server_port}";
};
http = {
server_port = hassPort;
use_x_forwarded_for = true;
trusted_proxies = with allAssignments.middleman.internal; [
ipv4.address
nixos/boxes/home/palace/vms/sfh/containers/unifi.nix
+2 -5
View File
@@ -1,7 +1,7 @@
{ lib, ... }:
let
inherit (lib.my) net;
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
inherit (lib.my.c.home) domain prefixes vips hiMTU;
in
{
nixos.systems.unifi = { config, ... }: {
@@ -58,10 +58,7 @@ in
systemd = {
network.networks = {
"80-container-host0" = mkMerge [
(networkdAssignment "host0" assignments.hi)
{ networkConfig = vlanDns "hi"; }
];
"80-container-host0" = networkdAssignment "host0" assignments.hi;
"80-lan-core" = networkdAssignment "lan-core" assignments.core;
};
};
nixos/boxes/home/palace/vms/sfh/default.nix
+3 -5
View File
@@ -1,7 +1,7 @@
{ lib, ... }:
let
inherit (lib.my) net;
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU roceBootModules;
inherit (lib.my.c.home) domain prefixes vips hiMTU roceBootModules;
in
{
imports = [ ./containers ];
@@ -134,10 +134,8 @@ in
networks = {
"30-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
{
# So we don't drop the IP we use to connect to NVMe-oF!
networkConfig = vlanDns "hi" // { KeepConfiguration = "static"; };
}
# So we don't drop the IP we use to connect to NVMe-oF!
{ networkConfig.KeepConfiguration = "static"; }
];
"30-lan-hi-ctrs" = {
matchConfig.Name = "lan-hi-ctrs";
nixos/boxes/home/routing-common/dns.nix
+4 -14
View File
@@ -170,12 +170,6 @@ in
"0.0.0.0:5353" "[::]:5353"
];
also-notify = [ "127.0.0.1" ];
allow-axfr-ips = [
"127.0.0.0/8" "::1/128"
allAssignments.estuary.internal.ipv4.address
]
++ lib.my.c.as211024.trusted.v4
++ lib.my.c.as211024.trusted.v6;
enable-lua-records = true;
# loglevel = 7;
# log-dns-queries = true;
@@ -248,14 +242,10 @@ in
shytzel IN A ${net.cidr.host 12 prefixes.core.v4}
brian IN A ${net.cidr.host 13 prefixes.core.v4}
vibe-core IN A ${net.cidr.host 15 prefixes.core.v4}
vibe IN A ${net.cidr.host 15 prefixes.hi.v4}
vibe IN AAAA ${net.cidr.host (65536+6) prefixes.hi.v6}
vibe-lo IN A ${net.cidr.host 15 prefixes.lo.v4}
vibe-lo IN AAAA ${net.cidr.host (65536+6) prefixes.lo.v6}
wave-core IN A ${net.cidr.host 14 prefixes.core.v4}
wave IN A ${net.cidr.host 14 prefixes.lo.v4}
wave IN AAAA ${net.cidr.host (65536+5) prefixes.lo.v6}
wave IN A ${net.cidr.host 12 prefixes.hi.v4}
; wave IN AAAA ${net.cidr.host (65536+3) prefixes.hi.v6}
vibe IN A ${net.cidr.host 13 prefixes.hi.v4}
vibe IN AAAA ${net.cidr.host (65536+4) prefixes.hi.v6}
ups IN A ${net.cidr.host 20 prefixes.lo.v4}
palace-kvm IN A ${net.cidr.host 21 prefixes.lo.v4}
nixos/boxes/home/routing-common/kea.nix
+3 -3
View File
@@ -1,8 +1,8 @@
index: { lib, pkgs, config, assignments, allAssignments, ... }:
let
inherit (lib) mkForce concatStringsSep;
inherit (lib) mkForce;
inherit (lib.my) net netbootKeaClientClasses;
inherit (lib.my.c.home) domain searchDomains prefixes vips hiMTU;
inherit (lib.my.c.home) domain prefixes vips hiMTU;
dns-servers = [
{
@@ -59,7 +59,7 @@ in
}
{
name = "domain-search";
data = concatStringsSep ", " searchDomains;
data = "${domain}, dyn.${domain}, ${lib.my.c.colony.domain}, ${lib.my.c.britway.domain}";
always-send = true;
}
];
nixos/boxes/home/routing-common/mstpd.nix
+1 -1
View File
@@ -24,7 +24,7 @@ in
services = {
networkd-dispatcher.rules = {
configure-mstpd = {
onState = [ "configured" ];
onState = [ "routable" ];
script = ''
#!${pkgs.runtimeShell}
if [ "$IFACE" = "lan" ]; then
nixos/boxes/home/routing-common/radvd.nix
+3 -3
View File
@@ -1,8 +1,8 @@
index: { lib, pkgs, ... }:
let
inherit (lib) mkForce concatMapStringsSep concatStringsSep;
inherit (lib) mkForce concatMapStringsSep;
inherit (lib.my) net;
inherit (lib.my.c.home) domain searchDomains prefixes vips;
inherit (lib.my.c.home) domain prefixes vips;
# untrusted uses external (Cloudflare) resolvers, matching the v4 kea config;
# trusted VLANs use the internal recursor via its floating VRRP VIP
@@ -18,7 +18,7 @@ let
AdvLinkMTU ${toString prefixes."${name}".mtu};
prefix ${prefixes."${name}".v6} {};
RDNSS ${rdnss name} {};
DNSSL ${concatStringsSep " " searchDomains} {};
DNSSL ${domain} dyn.${domain} ${lib.my.c.colony.domain} ${lib.my.c.britway.domain} {};
};
'';
in
nixos/boxes/home/stream.nix
+4 -3
View File
@@ -115,7 +115,10 @@
Name = "lan";
Kind = "bridge";
};
bridgeConfig.STP = true;
extraConfig = ''
[Bridge]
STP=true
'';
};
};
links = {
@@ -172,12 +175,10 @@
"50-lan-jim" = {
matchConfig.Name = "lan-jim";
networkConfig.Bridge = "lan";
bridgeConfig.Cost = 100;
};
"50-lan-dave" = {
matchConfig.Name = "lan-dave";
networkConfig.Bridge = "lan";
bridgeConfig.Cost = 10;
};
"50-wan-ifb" = {
nixos/default.nix
+3 -45
View File
@@ -100,40 +100,6 @@ let
] ++ defs;
};
# JSON dump of the custom `my.*` options, rendered to docs/ by `update-docs-options`.
# Built from a minimal synthetic system (the shared modules are applied to every system,
# so any eval exposes the same option declarations) rather than a real box, so defaults
# don't pick up a specific host's values.
optionsDoc =
let
eval = mkSystem {
name = "options-doc";
config' = {
system = "x86_64-linux";
nixpkgs = "mine";
home-manager = "mine";
hmNixpkgs = "mine";
docCustom = false;
assignments = { };
};
defs = [ ];
};
pkgs = pkgs'.mine."x86_64-linux";
in
(pkgs.nixosOptionsDoc {
options = { my = eval.options.my; };
warningsAreErrors = false;
# Rewrite store-path declarations to repo-relative so the reference can link module
# files. The internal `asX` build-target options are dropped by nixosOptionsDoc itself.
transformOptions = opt: opt // {
declarations = map
(decl:
let m = builtins.match ".*-source/(.*)" (toString decl); in
if m != null then builtins.head m else decl)
opt.declarations;
};
}).optionsJSON;
assignmentOpts = with lib.types; { name, config, ... }: {
options = {
name = mkOpt' str name "Name of assignment.";
@@ -188,11 +154,8 @@ let
options = {
inherit (commonOpts) system nixpkgs home-manager;
hmNixpkgs = commonOpts.nixpkgs;
# Routes the custom modules into `baseModules` so the NixOS manual documents them. The old
# infinite-recursion is gone, but enabling this makes every system build regenerate the
# manual, and it documents everything the modules transitively import — including third-party
# modules that aren't doc-clean. Prefer the generated
# `nixos.optionsDoc` reference (`docs/reference/nixos-options.md`) instead.
# This causes a (very slow) docs rebuild on every change to a module's options it seems
# TODO: Currently broken with infinite recursion...
docCustom = mkBoolOpt' false "Whether to document nixfiles' custom NixOS modules.";
assignments = mkOpt' (attrsOf (submoduleWith {
@@ -234,11 +197,6 @@ in
description = "All network assignments.";
readOnly = true;
};
optionsDoc = mkOption {
type = package;
description = "nixosOptionsDoc JSON dump of the custom `my.*` module options.";
readOnly = true;
};
vpns = {
l2 = mkOpt' (attrsOf (submodule l2MeshOpts)) { } "Layer 2 meshes.";
};
@@ -266,7 +224,7 @@ in
];
nixos = {
inherit allAssignments optionsDoc;
inherit allAssignments;
};
};
}
nixos/installer.nix
+5 -24
View File
@@ -32,7 +32,7 @@
};
image = {
baseName = mkForce "jackos-installer";
baseName = "jackos-installer";
};
isoImage = {
volumeID = "jackos-${config.system.nixos.release}-${pkgs.stdenv.hostPlatform.uname.processor}";
@@ -97,17 +97,10 @@
documentation.enable = mkForce true;
documentation.nixos.enable = mkForce true;
system.nixos.variant_id = mkDefault "installer";
# Enable NetworkManager, but don't start it by default.
networking.networkmanager.enable = true;
systemd.services = {
NetworkManager.wantedBy = mkForce [];
NetworkManager-wait-online.wantedBy = mkForce [];
NetworkManager-dispatcher.wantedBy = mkForce [];
# NetworkManager's wireless backend, D-Bus activated on demand
wpa_supplicant.wantedBy = mkForce [];
};
# Enable wpa_supplicant, but don't start it by default.
networking.wireless.enable = mkDefault true;
networking.wireless.userControlled = true;
systemd.services.wpa_supplicant.wantedBy = mkForce [];
# Tell the Nix evaluator to garbage collect more aggressively.
# This is desirable in memory-constrained environments that don't
@@ -120,18 +113,6 @@
# download-using-manifests.pl from forking even if there is
# plenty of free memory.
boot.kernel.sysctl."vm.overcommit_memory" = "1";
# Prevent installation media from evacuating persistent storage, as their
# var directory is not persistent and it would thus result in deletion of
# those entries.
environment.etc."systemd/pstore.conf".text = ''
[PStore]
Unlink=no
'';
# Remove warning about unset mail
boot.swraid.mdadmConf = "PROGRAM ${pkgs.coreutils}/bin/true";
services.lvm.boot.thin.enable = true;
};
};

Some files were not shown because too many files have changed in this diff Show More