Compare commits
24 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| b9bcb1eaeb | |||
| f9f61e19ad | |||
| 14af217880 | |||
| 26e6870337 | |||
| 000c03d3c1 | |||
| 4f6ea35ee4 | |||
| de681f33da | |||
| ab2308e765 | |||
| 539a76a94f | |||
| 88d0d19239 | |||
| 8f9ca5e1c4 | |||
| 08605ab422 | |||
| c354998e11 | |||
| b44095f1ed | |||
| 4084dfe64e | |||
| f3d921bcdc | |||
| a2f3410e42 | |||
| 43cf35d54e | |||
| 390818ffdb | |||
| 43b5bedb48 | |||
| e47e30df19 | |||
| e077c6e30c | |||
| 6f8d499ec2 | |||
| d0859d4c5c |
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"permissions": {
|
||||||
|
"allow": [
|
||||||
|
"Bash(nix eval:*)",
|
||||||
|
"Bash(nix flake check:*)",
|
||||||
|
"Bash(nix build:*)",
|
||||||
|
"Bash(check-system:*)",
|
||||||
|
"Bash(build-system:*)",
|
||||||
|
"Bash(build-home:*)",
|
||||||
|
"Bash(git status:*)",
|
||||||
|
"Bash(git diff:*)",
|
||||||
|
"Bash(git log:*)",
|
||||||
|
"Bash(git show:*)",
|
||||||
|
"Bash(drill:*)",
|
||||||
|
"Bash(ping:*)"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,12 +1,12 @@
|
|||||||
name: Update docs assignments
|
name: Update docs
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: [docs-kimi-new]
|
branches: [master]
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
update:
|
update:
|
||||||
if: "!contains(github.event.head_commit.message, 'docs: update assignment tables')"
|
if: "!contains(github.event.head_commit.message, 'docs: Update generated references')"
|
||||||
runs-on: ubuntu-26.04
|
runs-on: ubuntu-26.04
|
||||||
permissions:
|
permissions:
|
||||||
contents: write
|
contents: write
|
||||||
@@ -23,6 +23,19 @@ jobs:
|
|||||||
- name: Update assignment tables
|
- name: Update assignment tables
|
||||||
run: nix run .#update-docs-assignments
|
run: nix run .#update-docs-assignments
|
||||||
|
|
||||||
|
- name: Update option reference
|
||||||
|
run: nix run .#update-docs-options
|
||||||
|
|
||||||
|
- name: Update DNS reference
|
||||||
|
run: >
|
||||||
|
nix run .#update-docs-dns --
|
||||||
|
ams1.int.nul.ie
|
||||||
|
100.10.in-addr.arpa
|
||||||
|
2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa
|
||||||
|
h.nul.ie
|
||||||
|
168.192.in-addr.arpa
|
||||||
|
0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa
|
||||||
|
|
||||||
- name: Commit and push if changed
|
- name: Commit and push if changed
|
||||||
env:
|
env:
|
||||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
@@ -33,6 +46,6 @@ jobs:
|
|||||||
git remote set-url origin "${REPO_URL/https:\/\//https:\/\/oauth2:${GITEA_TOKEN}@}"
|
git remote set-url origin "${REPO_URL/https:\/\//https:\/\/oauth2:${GITEA_TOKEN}@}"
|
||||||
git add docs/
|
git add docs/
|
||||||
if ! git diff --cached --quiet; then
|
if ! git diff --cached --quiet; then
|
||||||
git commit -m "docs: update assignment tables"
|
git commit -m "docs: Update generated references"
|
||||||
git push
|
git push
|
||||||
fi
|
fi
|
||||||
|
|||||||
@@ -4,3 +4,4 @@ result*
|
|||||||
!/.vms/.gitkeep
|
!/.vms/.gitkeep
|
||||||
/.keys/*.key
|
/.keys/*.key
|
||||||
*.swp
|
*.swp
|
||||||
|
/.claude/settings.local.json
|
||||||
|
|||||||
@@ -6,13 +6,24 @@ This file provides guidance to coding agents when working with code in this repo
|
|||||||
tools refuse to write through a symlink and will error on `CLAUDE.md`).
|
tools refuse to write through a symlink and will error on `CLAUDE.md`).
|
||||||
|
|
||||||
**Prefer this file over agent memory.** When you learn something durable about this repo — a
|
**Prefer this file over agent memory.** When you learn something durable about this repo — a
|
||||||
convention, a workflow gotcha, a design rationale — record it here (or in a repo doc this file points
|
convention, a workflow gotcha, a design rationale — record it here (or in a repo doc this file
|
||||||
to, e.g. `docs/sites/home/switches.md`), not in agent memory. AGENTS.md is versioned and shared; memory is not.
|
points to, e.g. `docs/sites/home/switches.md`), not in agent memory. AGENTS.md is versioned and
|
||||||
|
shared; memory is not.
|
||||||
|
|
||||||
|
Keep this file lean: it should contain only rules, workflows, safety constraints and sharp gotchas
|
||||||
|
an agent needs before starting work. Detailed inventories, topology, service operation and design
|
||||||
|
rationale belong in the canonical document under `docs/`; summarize only the essential constraint
|
||||||
|
here and link to that document instead of maintaining a second copy.
|
||||||
|
|
||||||
|
Claude Code permissions live in two files: `.claude/settings.json` (versioned, shared — the
|
||||||
|
committed allow list of safe-to-auto-approve commands) and `.claude/settings.local.json` (personal,
|
||||||
|
gitignored — where interactive "always allow" grants accumulate). Put durable, generally-safe
|
||||||
|
commands in the shared file; leave one-off or workstation-specific grants in the local one.
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Personal Nix flake managing NixOS systems and home-manager configurations for a set of
|
Personal Nix flake managing NixOS systems and home-manager configurations for a set of
|
||||||
machines — always called **"boxes"**, never "fleet". It is built around a **custom module
|
**boxes**, never a "fleet". It is built around a **custom module
|
||||||
system** layered on top of NixOS/home-manager, not the stock flake `nixosConfigurations` pattern.
|
system** layered on top of NixOS/home-manager, not the stock flake `nixosConfigurations` pattern.
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
@@ -35,7 +46,10 @@ Common ones:
|
|||||||
Pass the flake-qualified node, e.g. `deploy .#git`. The deploy node name is **always** the system
|
Pass the flake-qualified node, e.g. `deploy .#git`. The deploy node name is **always** the system
|
||||||
name (`deploy-rs.nix` keys nodes directly off `nixos.systems` / `home-manager.homes`); a system is
|
name (`deploy-rs.nix` keys nodes directly off `nixos.systems` / `home-manager.homes`); a system is
|
||||||
only a deploy target when `config.my.deploy.enable` is true (defaults true; auto-disabled for dev
|
only a deploy target when `config.my.deploy.enable` is true (defaults true; auto-disabled for dev
|
||||||
VMs and containers). Pass `--boot` to stage a config as the boot default **without** live-switching
|
VMs and containers). A container is **not** its own deploy node — it is generated as a
|
||||||
|
`container-<name>` profile on its **host** node. So `deploy .#<host>` deploys the host's `system`
|
||||||
|
profile and every one of its containers, whereas `deploy .#<host>.container-<name>` targets a
|
||||||
|
single container (e.g. `deploy .#shill.container-middleman`). Pass `--boot` to stage a config as the boot default **without** live-switching
|
||||||
(`deploy --boot .#<host>`) — the box keeps running its current generation until it reboots. Use this
|
(`deploy --boot .#<host>`) — the box keeps running its current generation until it reboots. Use this
|
||||||
when a live `switch` would break connectivity mid-change (e.g. a router's WAN VLAN rework), then
|
when a live `switch` would break connectivity mid-change (e.g. a router's WAN VLAN rework), then
|
||||||
reboot to cut over.
|
reboot to cut over.
|
||||||
@@ -49,11 +63,24 @@ Common ones:
|
|||||||
- `repl` — `nix repl .#`.
|
- `repl` — `nix repl .#`.
|
||||||
- `update-nixpkgs` / `update-home-manager` — bump pinned inputs.
|
- `update-nixpkgs` / `update-home-manager` — bump pinned inputs.
|
||||||
|
|
||||||
Check everything (what CI runs): `nix flake check --no-build`.
|
Use the narrowest relevant evaluation while iterating: `check-system <host>` for a box config,
|
||||||
|
`nix eval .#nixfiles.config.nixos.allAssignments --json` for assignment generation, or
|
||||||
|
`nix build --no-link .#nixfiles.config.nixos.optionsDoc` for the option reference. Reserve
|
||||||
|
`nix flake check --no-build` for final broad validation or reproducing CI.
|
||||||
CI builds each attr of `.#ci.x86_64-linux` (systems, homes, packages, shell) and pushes to the
|
CI builds each attr of `.#ci.x86_64-linux` (systems, homes, packages, shell) and pushes to the
|
||||||
Harmonia binary cache; see `.gitea/workflows/ci.yaml` and `ci/push-to-cache.sh`. A separate
|
Harmonia binary cache; see `.gitea/workflows/ci.yaml` and `ci/push-to-cache.sh`. A separate
|
||||||
workflow (`.gitea/workflows/update-docs.yaml`) regenerates the network-assignment tables in `docs/`
|
workflow (`.gitea/workflows/update-docs.yaml`) regenerates the network assignments, NixOS option
|
||||||
via `nix run .#update-docs-assignments`.
|
reference and live DNS reference under `docs/`.
|
||||||
|
|
||||||
|
For DNS lookups use **`drill`** (ldns) — `dig` isn't installed in this environment (it fails with
|
||||||
|
exit 127, which is easy to miss if stderr is redirected). E.g. `drill -Q @<resolver> <name> A`.
|
||||||
|
|
||||||
|
For privilege escalation use **`doas`**, not `sudo` — the boxes don't install `sudo` (it fails with
|
||||||
|
`command not found`). E.g. `doas ip link set <if> up`.
|
||||||
|
|
||||||
|
For ad-hoc packages prefer **`pkgs#<name>`** over `nixpkgs#<name>` (e.g. `nix run pkgs#python3`):
|
||||||
|
`pkgs` is a system registry alias for the same pinned nixpkgs the boxes are built from, so it
|
||||||
|
resolves from the local store instead of fetching a different channel.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
@@ -114,45 +141,30 @@ via the `vms` / `containers` modules and the `l2mesh` VXLAN module.
|
|||||||
For a human-readable map of what is actually deployed (per-box roles, services and networking),
|
For a human-readable map of what is actually deployed (per-box roles, services and networking),
|
||||||
see `README.md` and `docs/` (index at `docs/README.md`; box pages under `docs/sites/`,
|
see `README.md` and `docs/` (index at `docs/README.md`; box pages under `docs/sites/`,
|
||||||
`docs/remote/`, `docs/mobile/`). Keep these in sync when adding, removing or repurposing a box or
|
`docs/remote/`, `docs/mobile/`). Keep these in sync when adding, removing or repurposing a box or
|
||||||
service. The network-assignment tables in box pages are CI-generated from `allAssignments` (the
|
service. The network-assignment tables are consolidated in `docs/networking.md` (one per site,
|
||||||
`<!-- assignments: <name> -->` markers) — write the prose and let the updater refresh the tables.
|
CI-generated from `allAssignments` between `<!-- assignments: <site> -->` markers); box pages
|
||||||
|
link to that section. Write the prose and let the updater refresh the tables.
|
||||||
|
|
||||||
### Home routers (`nixos/boxes/home/routing-common`)
|
### Home routers (`nixos/boxes/home/routing-common`)
|
||||||
The two home routers, `river` and `stream`, share `routing-common`, which is a **function of an
|
`river` and `stream` share `routing-common`, a function of an `index`; keep common HA behavior there
|
||||||
`index`** (`import ../../routing-common 0` for river, `1` for stream). The index derives per-box
|
and box-specific WAN behavior in the respective box file. Client-facing gateway and DNS services
|
||||||
addresses, keepalived VRRP priorities/state, DNS `ns` numbering, etc., so the two boxes are an
|
must use the floating VIPs. `routing-common` only declares the inert `wan-online.target`; each box
|
||||||
active/backup HA pair from one definition. They differ where hardware/uplink differ: `stream` has a
|
wires its own reachability mechanism. See `docs/networking.md#router-ha` and the router box pages
|
||||||
DHCP WAN, `river` runs PPPoE (`services.pppd`, Digiweb) — box-specific bits live in the respective
|
for the design and operational detail. Keep pair-wide behavior in `docs/networking.md`; router
|
||||||
box file, not `routing-common`.
|
pages should document only their WAN, platform and other box-specific responsibilities.
|
||||||
|
|
||||||
- **HA is VRRP (`keepalived`).** Per-VLAN floating **VIPs** (`lib.my.c.home.vips`) are what clients
|
The easy-to-mix-up implementation details are that `lib.my.networkdAssignment` and
|
||||||
use as both gateway *and* DNS server. `kea` (DHCP) and `radvd` (RAs; started only on the master)
|
`lib.my.mkVLAN` live under `lib.my`, networkd snippet constants such as `networkd.noL3` live under
|
||||||
hand out the VIP, and `pdns-recursor` binds the VIPs (with `net.ipv*.ip_nonlocal_bind` so the
|
`lib.my.c`, and interface MTU belongs in the `.network`'s `linkConfig.MTUBytes`.
|
||||||
backup can pre-bind). Point client-facing services at the VIP, not a box's real address, so
|
|
||||||
failover follows the master instead of relying on client resolver timeouts.
|
|
||||||
- **`wan-online.target`** is a shared abstract target meaning "the public WAN/IPv4 route is up".
|
|
||||||
`routing-common` only declares it; each box wires *how it is reached* (`stream`: a oneshot that
|
|
||||||
waits for the DHCP default route; `river`: the pppd `ip-up`/`ip-down` hooks). Services that need
|
|
||||||
the WAN attach **to** it via `wantedBy` + `partOf` + `after` (not `requires`/`wants`), so an empty
|
|
||||||
target is never pulled in and prematurely activated, and they re-load on WAN flap.
|
|
||||||
- networkd helpers used heavily here: `lib.my.networkdAssignment` and `lib.my.mkVLAN` live under
|
|
||||||
**`lib.my`**, while networkd snippet constants like `networkd.noL3` live under **`lib.my.c`** —
|
|
||||||
easy to mix up. Set an interface MTU via the `.network`'s `linkConfig.MTUBytes` (`[Link]`), not
|
|
||||||
`netdevConfig` (`[NetDev]` rejects `MTUBytes`).
|
|
||||||
|
|
||||||
### Home switches (`jim` / `dave` / `brian`)
|
### Home switches (`jim` / `dave` / `brian`)
|
||||||
The home boxes and the Digiweb WAN hang off hand-configured switches that are **not** managed by
|
These switches are not managed by the flake, and changing them is out-of-band and hard to revert.
|
||||||
this flake: `jim` and `dave` (MikroTik, RouterOS) and `brian` (Ubiquiti, UniFi). The full topology,
|
Read `docs/sites/home/switches.md` before WAN or VLAN work, always confirm before applying a switch
|
||||||
VLAN map, and the ONT/WAN path live in **`docs/sites/home/switches.md`** — read it before
|
change, and keep the documented layout and `lib.my.c.home.vlans` in sync.
|
||||||
touching anything WAN/VLAN-related, and update it when the switch layout changes.
|
|
||||||
- **Access:** the switches resolve by **short hostname** on the home network (the routers serve
|
### Home wireless APs (`vibe` / `wave`)
|
||||||
their records in the home zone — `routing-common/dns.nix`: `jim`/`dave`/`brian`). From a home box,
|
These APs are not managed by the flake; only their DNS records are. Read and update
|
||||||
SSH to the MikroTiks as `admin`/`admin` (e.g. `ssh admin@jim`); `brian` is configured via the
|
`docs/sites/home/aps.md` when changing an AP or its VLAN layout.
|
||||||
UniFi controller, not a CLI.
|
|
||||||
- **Changing switch config is out-of-band and hard to revert — always confirm before applying:**
|
|
||||||
print the affected menu, make the change, then re-verify. The nix config and the switches must
|
|
||||||
agree on VLAN numbering (e.g. `lib.my.c.home.vlans`), so a switch-side change usually pairs with a
|
|
||||||
box change; `docs/sites/home/switches.md` documents the switch layout and per-switch config for the WAN design.
|
|
||||||
|
|
||||||
## Secrets
|
## Secrets
|
||||||
|
|
||||||
@@ -165,7 +177,8 @@ private keys) is required for editing secrets, deploying, and running dev VMs.
|
|||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
- Format with `nixpkgs-fmt` (`fmt`). 2-space indent, `inherit (...)` blocks at the top of `let`.
|
- Format with `nixpkgs-fmt` (`fmt`). 2-space indent, `inherit (...)` blocks at the top of `let` —
|
||||||
|
prefer `inherit (lib) mkOption ...;` (and bare use) over qualifying inline as `lib.mkOption`.
|
||||||
**Ask before running `fmt`** — some files aren't canonically formatted, so `fmt` can reindent a
|
**Ask before running `fmt`** — some files aren't canonically formatted, so `fmt` can reindent a
|
||||||
whole file and bury a logical change in whitespace churn. Match the surrounding style by hand and
|
whole file and bury a logical change in whitespace churn. Match the surrounding style by hand and
|
||||||
leave formatting to the user unless they ask.
|
leave formatting to the user unless they ask.
|
||||||
@@ -180,6 +193,73 @@ private keys) is required for editing secrets, deploying, and running dev VMs.
|
|||||||
as `overlays.default`.
|
as `overlays.default`.
|
||||||
- In prose and commit messages, quote code-like identifiers (commands, options, paths, package and
|
- In prose and commit messages, quote code-like identifiers (commands, options, paths, package and
|
||||||
attribute names) in backticks.
|
attribute names) in backticks.
|
||||||
- Call the machines **"boxes"**, never "fleet".
|
- Keep docs and prose plain — avoid vague AI-tell filler. In particular use "layout"/"structure"
|
||||||
|
rather than "shape", and "General" rather than "cross-cutting topics"; name things directly
|
||||||
|
instead of reaching for umbrella words.
|
||||||
|
- For network connectivity, use the concrete term — "interface", "connection", "network
|
||||||
|
attachment" or the network's name — rather than calling it a "leg".
|
||||||
|
- Call the systems **"boxes"**, never "machines" or "fleet", except where "machine" is part of a
|
||||||
|
command, option or upstream technical term such as QEMU's machine type.
|
||||||
- Commit subjects follow `area/scope: Capitalized summary` (e.g. `nixos/home: ...`); keep logically
|
- Commit subjects follow `area/scope: Capitalized summary` (e.g. `nixos/home: ...`); keep logically
|
||||||
distinct changes in separate commits.
|
distinct changes in separate commits. Aim for 50-character subjects and do not exceed 72
|
||||||
|
characters. Wrap commit bodies at 72 columns. A concise body describing the change and its
|
||||||
|
rationale is welcome when the subject alone does not provide enough context.
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
Human-readable docs live under `docs/` (index: `docs/README.md`). Box pages are under
|
||||||
|
`docs/sites/<site>/`, `docs/remote/`, `docs/mobile/`. A box that itself hosts sub-systems
|
||||||
|
(containers/nested VMs) gets a **directory named for it** with a `README.md` and the child pages
|
||||||
|
beneath it (e.g. `shill/README.md` + `shill/containers/*.md`, `sfh/README.md`). When a site and its
|
||||||
|
physical box share a name, the site keeps the `README.md` and the box page stays alongside it (for
|
||||||
|
example `sites/colony/README.md` and `sites/colony/colony.md`).
|
||||||
|
|
||||||
|
**Box page layout** (match the existing pages): H1 + a one-line intro; a short bullet list of
|
||||||
|
`Source` / `Host` / `nixpkgs`; an optional hardware inventory or VPS resource-allocation section;
|
||||||
|
`## Role`; `## Network assignments` that **links** to
|
||||||
|
[`networking.md#box-assignments`](docs/networking.md#box-assignments) (never inline the table); one
|
||||||
|
`##` section per topic; `## Notable config files` last. Keep non-hardware platform details in their
|
||||||
|
topical sections rather than moving them with the inventory. A box without static assignments still
|
||||||
|
gets the section with a short explanation instead of a generated-table link.
|
||||||
|
|
||||||
|
**Structure and layout:**
|
||||||
|
- Use **tables** for lists of structured items (BGP peers, forwarded ports, vhosts, containers,
|
||||||
|
VMs, VLANs). Make the item **name the link**; don't add a separate `Page`/`Docs` column.
|
||||||
|
- Break up **long prose**: any bullet running past ~4 lines over several distinct facts becomes a
|
||||||
|
`###` subheading (lead sentence + nested bullets). Don't leave walls of long bullets.
|
||||||
|
- Prefer durable concepts and named configuration identifiers over copying exact numbers that are
|
||||||
|
tunable, generated or likely to drift. Exact values are appropriate when they are useful parts of
|
||||||
|
the inventory or interface: VM resource allocations, hardware-fixed properties, network and
|
||||||
|
protocol identifiers, exposed service ports, and safety or recovery values. Describe mutable
|
||||||
|
implementation tunables by purpose and link to their source instead of duplicating the current
|
||||||
|
value.
|
||||||
|
- **Per-site index pages** (`docs/sites/*/README.md`, `docs/remote/README.md`, and
|
||||||
|
`docs/mobile/README.md`) use one table listing **boxes only**; don't repeat that inventory in a
|
||||||
|
diagram. A box's containers / nested VMs are tabulated on that box's own page. The global
|
||||||
|
`docs/README.md` is the exception: it may show them in high-level site diagrams, but should not
|
||||||
|
add a second detailed inventory outside those diagrams.
|
||||||
|
|
||||||
|
**De-dup by ownership** — each fact has one home:
|
||||||
|
- Shared cross-box **fabric** (the AS211024 mesh, Tailscale/headscale topology, the BGP overview,
|
||||||
|
the WireGuard-tunnel summary) is documented once in `docs/networking.md`; box pages give a
|
||||||
|
one-line summary and link to it.
|
||||||
|
- Site-wide network definitions (prefixes, VLANs, router VIPs and router HA) also live in
|
||||||
|
`docs/networking.md`; site pages summarize and link to the canonical section.
|
||||||
|
- **Box-specific** detail (a box's own BGP peers, the WireGuard tunnels it terminates, its
|
||||||
|
services) lives on the box page; `networking.md` carries only a per-box summary that links out.
|
||||||
|
- Addresses represented by generated `assignments` or `extraAssignments` live only in the
|
||||||
|
assignment tables; handwritten box and workload inventories link there instead of copying them.
|
||||||
|
Addresses outside that data model (such as external peers or service endpoints) stay with the
|
||||||
|
topic that owns them.
|
||||||
|
|
||||||
|
**Generated content:** the network-assignment tables in `networking.md`, the option reference
|
||||||
|
(`docs/reference/nixos-options.md`) and the live DNS tables (`docs/reference/dns.md`) are
|
||||||
|
CI-generated by the corresponding `update-docs-*` packages; the workflow supplies the DNS zones.
|
||||||
|
Don't hand-edit content between `<!-- ... -->` markers; write the surrounding prose or source
|
||||||
|
configuration and let the updater refresh the tables. The option-reference file is generated in
|
||||||
|
full.
|
||||||
|
|
||||||
|
**Keep docs current:** when you add, remove or repurpose a box or service, update its box page, the
|
||||||
|
relevant site-index `README.md`, and any affected prose in `networking.md` (the assignment/option
|
||||||
|
tables refresh via CI). This is the same "prefer docs over agent memory" rule from the top of this
|
||||||
|
file.
|
||||||
|
|||||||
@@ -1,14 +1,14 @@
|
|||||||
# nixfiles
|
# nixfiles
|
||||||
|
|
||||||
Personal Nix flake managing every machine I run: hosted servers, home
|
Personal Nix flake managing every box I run: hosted servers, home
|
||||||
infrastructure, routers, a remote site, VPSes and personal workstations. It is
|
infrastructure, routers, a remote site, VPSes and personal workstations. It is
|
||||||
built around a **custom module system** layered on top of NixOS and
|
built around a **custom module system** layered on top of NixOS and
|
||||||
home-manager rather than the stock per-host `nixosConfigurations` pattern.
|
home-manager rather than the stock per-host `nixosConfigurations` pattern.
|
||||||
|
|
||||||
For day-to-day commands and a deeper explanation of the module system,
|
For the module-system internals, see [`docs/architecture.md`](docs/architecture.md); for day-to-day
|
||||||
conventions and secrets, see [`AGENTS.md`](AGENTS.md). This README is the map of
|
commands, deployment and secrets, see [`docs/deployment.md`](docs/deployment.md). This README is the
|
||||||
**what is actually deployed**; the per-machine details live under [`docs/`](docs)
|
map of **what is actually deployed**; the per-box details live under [`docs/`](docs) (start at
|
||||||
(start at [`docs/README.md`](docs/README.md)).
|
[`docs/README.md`](docs/README.md)).
|
||||||
|
|
||||||
> **Note:** This documentation (the README and everything under `docs/`) is a
|
> **Note:** This documentation (the README and everything under `docs/`) is a
|
||||||
> work in progress and was **agent-generated** from the repository. It may be
|
> work in progress and was **agent-generated** from the repository. It may be
|
||||||
@@ -16,74 +16,50 @@ conventions and secrets, see [`AGENTS.md`](AGENTS.md). This README is the map of
|
|||||||
|
|
||||||
## The boxes at a glance
|
## The boxes at a glance
|
||||||
|
|
||||||
Machines are grouped by deployment/location. Each group has its own directory
|
Boxes are grouped by deployment/location. Each group has its own directory
|
||||||
under `docs/` with a `README.md` overview and one file per machine.
|
under `docs/` with a `README.md` overview and one page per box.
|
||||||
|
|
||||||
| Group | What it is | Docs |
|
| Group | What it is |
|
||||||
| --- | --- | --- |
|
| --- | --- |
|
||||||
| **colony** | Hosted dedicated server in Amsterdam (`ams1`). A VM host running the public-facing infrastructure: routing, web, git, media, object storage, chat, game servers. | [`docs/sites/colony/`](docs/sites/colony) |
|
| [**colony**](docs/sites/colony) | Hosted dedicated server in Amsterdam (`ams1`). A VM host running the public-facing infrastructure: routing, web, git, media, object storage, chat, game servers. |
|
||||||
| **home** | Home network: a VM host (`palace`), the home routers, storage, Home Assistant, and personal desktops — plus the hand-configured switches tying it together. | [`docs/sites/home/`](docs/sites/home) |
|
| [**home**](docs/sites/home) | Home network: a VM host (`palace`), the home routers, storage, Home Assistant, and a workstation — plus the hand-configured switches and wireless APs tying it together. |
|
||||||
| **remote** | Edge VPSes (`britway`, `britnet`) and the remote `kelder` site. | [`docs/remote/`](docs/remote) |
|
| [**remote**](docs/remote) | Edge VPSes (`britway`, `britnet`) and the remote `kelder` site. |
|
||||||
| **mobile** | The `tower` laptop. | [`docs/mobile/`](docs/mobile) |
|
| [**mobile**](docs/mobile) | The `tower` laptop. |
|
||||||
|
|
||||||
The custom installer image is documented at [`docs/misc/installer.md`](docs/misc/installer.md).
|
The custom installer image is documented at [`docs/misc/installer.md`](docs/misc/installer.md).
|
||||||
Cross-cutting topics — the module system, the network, deployment — live next to
|
The general topics — the module system, network and deployment — live next to the index:
|
||||||
the index: [`docs/architecture.md`](docs/architecture.md),
|
[`docs/architecture.md`](docs/architecture.md), [`docs/networking.md`](docs/networking.md) and
|
||||||
[`docs/networking.md`](docs/networking.md), [`docs/deployment.md`](docs/deployment.md).
|
[`docs/deployment.md`](docs/deployment.md). The generated custom-module option reference is at
|
||||||
|
[`docs/reference/nixos-options.md`](docs/reference/nixos-options.md).
|
||||||
|
|
||||||
## The "big machine" pattern
|
The high-level site diagrams and box hierarchy live in [`docs/README.md`](docs/README.md).
|
||||||
|
Networking is largely defined by per-box `assignments` plus the AS211024 L2 VXLAN mesh; see
|
||||||
The larger sites (`colony`, `home`) all follow the same shape:
|
[`docs/networking.md`](docs/networking.md) for the full picture and
|
||||||
|
[`docs/architecture.md`](docs/architecture.md) for the implementation mechanics.
|
||||||
```
|
|
||||||
physical host (VM host)
|
|
||||||
├── VM ── thing impractical to containerise (router, storage, podman host, …)
|
|
||||||
├── VM ── container host ──┬── NixOS container ── application
|
|
||||||
│ ├── NixOS container ──┬── application
|
|
||||||
│ │ ├── application
|
|
||||||
│ │ └── application
|
|
||||||
│ └── …
|
|
||||||
└── VM ── …
|
|
||||||
```
|
|
||||||
|
|
||||||
- A **physical host** (`colony`, `palace`) does little itself beyond running
|
|
||||||
**VMs** via the custom `my.vms` module (QEMU + systemd units, LVM-backed
|
|
||||||
disks).
|
|
||||||
- **VMs** exist for things that are impractical to put in a container — kernel
|
|
||||||
features, separate networking, podman/OCI workloads, foreign OSes.
|
|
||||||
- One VM is usually a **container host** (`shill` on colony, `sfh` on home; and
|
|
||||||
`kelder` directly). It runs **NixOS containers** via the custom `my.containers`
|
|
||||||
module (systemd-nspawn based, each with its own address on a bridge).
|
|
||||||
- **Most applications live in those NixOS containers.** A container isn't limited
|
|
||||||
to a single application — it commonly hosts a **group of related applications**
|
|
||||||
that belong together (e.g. `jackflix` runs Jellyfin, the *arr stack,
|
|
||||||
Transmission, PhotoPrism and copyparty; `object` runs MinIO, Harmonia, HedgeDoc
|
|
||||||
and wastebin). Each container is a first-class entry in the docs.
|
|
||||||
|
|
||||||
Networking between everything is largely defined by per-system `assignments`
|
|
||||||
(IPs/prefixes) plus an L2 VXLAN mesh (`my.vpns.l2`, AS211024) that ties the edge
|
|
||||||
routers together. See [`docs/networking.md`](docs/networking.md) for the full
|
|
||||||
picture and [`AGENTS.md`](AGENTS.md) for the mechanics.
|
|
||||||
|
|
||||||
## Repo layout
|
## Repo layout
|
||||||
|
|
||||||
```
|
```
|
||||||
README.md <- you are here
|
README.md <- you are here
|
||||||
nixos/
|
nixos/
|
||||||
boxes/ per-machine configuration ("boxes")
|
boxes/ per-box configuration
|
||||||
colony/ colony host + its VMs (vms/) + shill's containers
|
colony/ colony host + its VMs (vms/) + shill's containers
|
||||||
home/ palace host + its VMs, routing-common, plus stream, castle
|
home/ palace host + its VMs, routing-common, plus stream, castle
|
||||||
britway/ britnet.nix, kelder/, tower/, installer.nix …
|
britway/ London VPS
|
||||||
|
kelder/ remote box + its containers
|
||||||
|
tower/ laptop
|
||||||
|
britnet.nix Birmingham VPS
|
||||||
|
installer.nix installer-image configuration
|
||||||
modules/ shared NixOS modules (my.* options); registered in _list.nix
|
modules/ shared NixOS modules (my.* options); registered in _list.nix
|
||||||
home-manager/ home-manager modules + configs
|
home-manager/ home-manager modules + configs
|
||||||
lib/ lib.my helpers, constants (lib.my.c), net/dns helpers
|
lib/ lib.my helpers, constants (lib.my.c), net/dns helpers
|
||||||
pkgs/ custom packages (overlays.default)
|
pkgs/ custom packages (overlays.default)
|
||||||
secrets/ age-encrypted secrets (ragenix)
|
secrets/ age-encrypted secrets (ragenix)
|
||||||
devshell/ devshell commands (build/deploy/check/ssh helpers)
|
devshell/ devshell commands (build/deploy/check/ssh helpers)
|
||||||
ci/ CI helpers (binary-cache push, docs assignment-table updater)
|
ci/ CI helpers (binary-cache push, docs generators)
|
||||||
docs/ deployment documentation (index at docs/README.md)
|
docs/ deployment documentation (index at docs/README.md)
|
||||||
```
|
```
|
||||||
|
|
||||||
A machine is wired into the flake by adding its box file to the `configs` list
|
A box is wired into the flake by adding its config file to the `configs` list
|
||||||
in `flake.nix`. See [`AGENTS.md`](AGENTS.md#architecture) for how `evalModules`
|
in `flake.nix`. See [the top-level evaluation](docs/architecture.md#the-top-level-evaluation) for
|
||||||
turns these into `nixosConfigurations`, `homeConfigurations` and `deploy` nodes.
|
how `evalModules` turns these into `nixosConfigurations`, `homeConfigurations` and `deploy` nodes.
|
||||||
|
|||||||
Regular → Executable
+132
-66
@@ -1,5 +1,10 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Update assignment tables in docs/ from nixos.allAssignments."""
|
"""Update the consolidated network-assignment tables in docs/networking.md.
|
||||||
|
|
||||||
|
Reads nixos.allAssignments from the flake and renders assignment-key subgroups for each site
|
||||||
|
(colony / home / remote / other) between per-site `<!-- assignments: <site> -->` markers. The
|
||||||
|
hand-written Notes column is preserved, keyed by (box, assignment).
|
||||||
|
"""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import re
|
import re
|
||||||
@@ -7,60 +12,119 @@ import subprocess
|
|||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
DOCS_DIR = Path("docs")
|
DOCS = Path("docs")
|
||||||
HEADER = "| Name | Assignment | IPv4 | IPv6 | Domain | Notes |"
|
TARGET = DOCS / "networking.md"
|
||||||
|
HEADER = "| Box | IPv4 | IPv6 | Domain | Notes |"
|
||||||
|
SEP = "|---|---|---|---|---|"
|
||||||
|
|
||||||
|
# Table order; also the set of valid `<!-- assignments: <site> -->` tokens.
|
||||||
|
SITE_ORDER = ["colony", "home", "remote", "other"]
|
||||||
|
|
||||||
|
# Per-site internal domains of the remote boxes (britway=lon1, britnet=bhx1, kelder=hentai).
|
||||||
|
REMOTE_DOMAINS = ("lon1.int.nul.ie", "bhx1.int.nul.ie", "hentai.engineer")
|
||||||
|
|
||||||
|
|
||||||
|
def site_of(assignments: dict) -> str:
|
||||||
|
"""Classify a box into a site table by its assignment domain.
|
||||||
|
|
||||||
|
Prefer the `internal` assignment's domain, else the first assignment that has one.
|
||||||
|
"""
|
||||||
|
dom = (assignments.get("internal") or {}).get("domain")
|
||||||
|
if not dom:
|
||||||
|
for a in assignments.values():
|
||||||
|
if a.get("domain"):
|
||||||
|
dom = a["domain"]
|
||||||
|
break
|
||||||
|
if not dom:
|
||||||
|
return "other"
|
||||||
|
if dom.endswith("ams1.int.nul.ie"):
|
||||||
|
return "colony"
|
||||||
|
if dom == "h.nul.ie" or dom.endswith(".h.nul.ie"):
|
||||||
|
return "home"
|
||||||
|
if any(dom.endswith(s) for s in REMOTE_DOMAINS):
|
||||||
|
return "remote"
|
||||||
|
return "other"
|
||||||
|
|
||||||
|
|
||||||
def fmt_ip(ip: dict) -> str:
|
def fmt_ip(ip: dict) -> str:
|
||||||
addr = ip.get("address")
|
addr = ip.get("address")
|
||||||
if addr is None:
|
if addr is None:
|
||||||
return "—"
|
return "—"
|
||||||
mask = ip.get("mask")
|
parts = [f"{addr}/{ip.get('mask')}"]
|
||||||
gateway = ip.get("gateway")
|
if ip.get("gateway") is not None:
|
||||||
parts = [f"{addr}/{mask}"]
|
parts.append(f"gw {ip['gateway']}")
|
||||||
if gateway is not None:
|
|
||||||
parts.append(f"gw {gateway}")
|
|
||||||
return f"`{' '.join(parts)}`"
|
return f"`{' '.join(parts)}`"
|
||||||
|
|
||||||
|
|
||||||
def parse_notes(lines: list[str]) -> dict[str, str]:
|
def find_page(box: str) -> str | None:
|
||||||
"""Extract existing Notes indexed by Assignment from a marked table."""
|
"""Doc page for a box (relative to docs/), if one exists."""
|
||||||
notes: dict[str, str] = {}
|
for pattern in (f"{box}.md", f"{box}/README.md"):
|
||||||
|
for p in sorted(DOCS.rglob(pattern)):
|
||||||
|
return p.relative_to(DOCS).as_posix()
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def box_cell(box: str) -> str:
|
||||||
|
page = find_page(box)
|
||||||
|
return f"[`{box}`]({page})" if page else f"`{box}`"
|
||||||
|
|
||||||
|
|
||||||
|
def box_name_from_cell(cell: str) -> str:
|
||||||
|
m = re.match(r"\[`?([^`\]]+)`?\]", cell)
|
||||||
|
return m.group(1) if m else cell.strip("`")
|
||||||
|
|
||||||
|
|
||||||
|
def parse_notes(lines: list[str]) -> dict[tuple[str, str], str]:
|
||||||
|
"""Existing Notes indexed by (box, assignment), from either table layout."""
|
||||||
|
notes: dict[tuple[str, str], str] = {}
|
||||||
|
assignment = None
|
||||||
for line in lines:
|
for line in lines:
|
||||||
stripped = line.strip()
|
stripped = line.strip()
|
||||||
|
heading = re.match(r"^####\s+`([^`]+)`$", stripped)
|
||||||
|
if heading:
|
||||||
|
assignment = heading.group(1)
|
||||||
|
continue
|
||||||
if not stripped.startswith("|"):
|
if not stripped.startswith("|"):
|
||||||
continue
|
continue
|
||||||
cells = [c.strip() for c in stripped.split("|")]
|
cells = [c.strip() for c in stripped.split("|")]
|
||||||
# Splitting "| a | b |" produces ['', 'a', 'b', '']
|
# "| a | b | ... |" splits to ['', 'a', 'b', ..., '']
|
||||||
if len(cells) < 8:
|
if len(cells) >= 7 and assignment is not None:
|
||||||
|
# Current layout: one table under each assignment heading.
|
||||||
|
box, row_assignment, note = cells[1], assignment, cells[5]
|
||||||
|
elif len(cells) >= 8:
|
||||||
|
# Previous layout: one site table with an Assignment column.
|
||||||
|
box, row_assignment, note = cells[1], cells[2], cells[6]
|
||||||
|
else:
|
||||||
continue
|
continue
|
||||||
key = cells[2]
|
# Skip header and separator rows.
|
||||||
if key == "Assignment":
|
if box == "Box" or set(box) <= {"-"}:
|
||||||
continue
|
continue
|
||||||
notes[key] = cells[6]
|
notes[(box_name_from_cell(box), row_assignment)] = note
|
||||||
return notes
|
return notes
|
||||||
|
|
||||||
|
|
||||||
def render_table(assignments: dict, notes: dict[str, str]) -> list[str]:
|
def render_site(boxes: list[str], all_assignments: dict, notes: dict) -> list[str]:
|
||||||
lines = [
|
groups: dict[str, list[tuple[str, dict]]] = {}
|
||||||
HEADER,
|
for box in sorted(boxes):
|
||||||
"|---|---|---|---|---|---|",
|
for key, a in all_assignments[box].items():
|
||||||
]
|
groups.setdefault(key, []).append((box, a))
|
||||||
for key, a in assignments.items():
|
|
||||||
name = a.get("name", key)
|
lines: list[str] = []
|
||||||
alt = ", ".join(a.get("altNames", []))
|
group_order = sorted(groups, key=lambda key: (key != "internal", key))
|
||||||
if alt:
|
for key in group_order:
|
||||||
name = f"{name} ({alt})"
|
if lines:
|
||||||
|
lines.append("")
|
||||||
|
lines.extend([f"#### `{key}`", "", HEADER, SEP])
|
||||||
|
for box, a in groups[key]:
|
||||||
lines.append(
|
lines.append(
|
||||||
"| "
|
"| "
|
||||||
+ " | ".join(
|
+ " | ".join(
|
||||||
[
|
[
|
||||||
name,
|
box_cell(box),
|
||||||
key,
|
|
||||||
fmt_ip(a.get("ipv4", {})),
|
fmt_ip(a.get("ipv4", {})),
|
||||||
fmt_ip(a.get("ipv6", {})),
|
fmt_ip(a.get("ipv6", {})),
|
||||||
a.get("domain") or "—",
|
a.get("domain") or "—",
|
||||||
notes.get(key, ""),
|
notes.get((box, key), ""),
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
+ " |"
|
+ " |"
|
||||||
@@ -68,43 +132,49 @@ def render_table(assignments: dict, notes: dict[str, str]) -> list[str]:
|
|||||||
return lines
|
return lines
|
||||||
|
|
||||||
|
|
||||||
def process_file(path: Path, all_assignments: dict) -> bool:
|
def update_target(all_assignments: dict) -> bool:
|
||||||
box_name = path.stem
|
by_site: dict[str, list[str]] = {s: [] for s in SITE_ORDER}
|
||||||
if box_name not in all_assignments:
|
for box, assignments in all_assignments.items():
|
||||||
return False
|
if not assignments:
|
||||||
|
continue
|
||||||
|
by_site[site_of(assignments)].append(box)
|
||||||
|
|
||||||
text = path.read_text()
|
text = TARGET.read_text()
|
||||||
lines = text.splitlines()
|
lines = text.splitlines()
|
||||||
|
|
||||||
marker_re = re.compile(r"^<!--\s*assignments:\s*(\S+)\s*-->$")
|
marker_re = re.compile(r"^<!--\s*assignments:\s*(\S+)\s*-->$")
|
||||||
for i, line in enumerate(lines):
|
|
||||||
m = marker_re.match(line.strip())
|
|
||||||
if m and m.group(1) == box_name:
|
|
||||||
end_idx = None
|
|
||||||
for j in range(i + 1, len(lines)):
|
|
||||||
if lines[j].strip() == "<!-- assignments-end -->":
|
|
||||||
end_idx = j
|
|
||||||
break
|
|
||||||
if end_idx is None:
|
|
||||||
return False
|
|
||||||
|
|
||||||
old_inner = lines[i + 1 : end_idx]
|
# Walk the file line by line; at each `<!-- assignments: <site> -->` marker, replace
|
||||||
notes = parse_notes(old_inner)
|
# everything up to the matching `<!-- assignments-end -->` with a freshly rendered table
|
||||||
new_table = render_table(all_assignments[box_name], notes)
|
# (carrying the hand-written Notes over), leaving all other lines untouched.
|
||||||
new_lines = (
|
out: list[str] = []
|
||||||
lines[: i + 1]
|
i = 0
|
||||||
+ ["<!-- assignments-start -->"]
|
while i < len(lines):
|
||||||
+ new_table
|
m = marker_re.match(lines[i].strip())
|
||||||
+ ["<!-- assignments-end -->"]
|
if not m:
|
||||||
+ lines[end_idx + 1 :]
|
out.append(lines[i])
|
||||||
)
|
i += 1
|
||||||
if new_lines != lines:
|
continue
|
||||||
path.write_text("\n".join(new_lines) + "\n")
|
site = m.group(1)
|
||||||
|
end = i + 1
|
||||||
|
while end < len(lines) and lines[end].strip() != "<!-- assignments-end -->":
|
||||||
|
end += 1
|
||||||
|
if end >= len(lines):
|
||||||
|
out.append(lines[i])
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
notes = parse_notes(lines[i + 1 : end])
|
||||||
|
out.append(lines[i])
|
||||||
|
out.append("<!-- assignments-start -->")
|
||||||
|
out.extend(render_site(by_site.get(site, []), all_assignments, notes))
|
||||||
|
out.append("<!-- assignments-end -->")
|
||||||
|
i = end + 1
|
||||||
|
|
||||||
|
new_text = "\n".join(out) + "\n"
|
||||||
|
if new_text != text:
|
||||||
|
TARGET.write_text(new_text)
|
||||||
return True
|
return True
|
||||||
return False
|
return False
|
||||||
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
@@ -115,13 +185,9 @@ def main() -> int:
|
|||||||
)
|
)
|
||||||
all_assignments = json.loads(result.stdout)
|
all_assignments = json.loads(result.stdout)
|
||||||
|
|
||||||
changed = False
|
if update_target(all_assignments):
|
||||||
for path in sorted(DOCS_DIR.rglob("*.md")):
|
print(f"updated {TARGET}")
|
||||||
if process_file(path, all_assignments):
|
return 0
|
||||||
print(f"updated {path}")
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
return 1 if changed else 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
@@ -0,0 +1,362 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render docs/reference/dns.md from live authoritative DNS zone transfers.
|
||||||
|
|
||||||
|
The authoritative servers are queried directly over AXFR. Records owned by Kea are
|
||||||
|
identified by DHCID records and omitted together with their forward and reverse data.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import ipaddress
|
||||||
|
import re
|
||||||
|
import socket
|
||||||
|
import sys
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import dns.exception
|
||||||
|
import dns.name
|
||||||
|
import dns.query
|
||||||
|
import dns.resolver
|
||||||
|
import dns.rdatatype
|
||||||
|
|
||||||
|
|
||||||
|
OUT = Path("docs/reference/dns.md")
|
||||||
|
DEFAULT_PORT = 53
|
||||||
|
IGNORED_TYPES = {"DHCID", "SOA"}
|
||||||
|
POWERDNS_ALIAS = 65401
|
||||||
|
POWERDNS_LUA = 65402
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Record:
|
||||||
|
zone: str
|
||||||
|
owner: str
|
||||||
|
type: str
|
||||||
|
value: str
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_name(name: str) -> str:
|
||||||
|
return name.rstrip(".").lower()
|
||||||
|
|
||||||
|
|
||||||
|
def record_data(rdtype: int, rdata) -> tuple[str, str]:
|
||||||
|
if rdtype == POWERDNS_ALIAS:
|
||||||
|
target, _used = dns.name.from_wire(rdata.data, 0)
|
||||||
|
return "ALIAS", target.to_text()
|
||||||
|
if rdtype == POWERDNS_LUA:
|
||||||
|
logical_type = int.from_bytes(rdata.data[:2], byteorder="big")
|
||||||
|
return "LUA", dns.rdatatype.to_text(logical_type)
|
||||||
|
return dns.rdatatype.to_text(rdtype), rdata.to_text()
|
||||||
|
|
||||||
|
|
||||||
|
def server_addresses(server: str, port: int) -> list[str]:
|
||||||
|
addresses = []
|
||||||
|
try:
|
||||||
|
for result in socket.getaddrinfo(server, port, type=socket.SOCK_STREAM):
|
||||||
|
address = result[4][0]
|
||||||
|
if address not in addresses:
|
||||||
|
addresses.append(address)
|
||||||
|
except socket.gaierror as error:
|
||||||
|
raise RuntimeError(f"cannot resolve nameserver {server}: {error}") from error
|
||||||
|
return addresses
|
||||||
|
|
||||||
|
|
||||||
|
def transfer(server: str, port: int, zone: str) -> list[Record]:
|
||||||
|
addresses = server_addresses(server, port)
|
||||||
|
|
||||||
|
errors = []
|
||||||
|
for address in addresses:
|
||||||
|
try:
|
||||||
|
records = []
|
||||||
|
for message in dns.query.xfr(
|
||||||
|
address, zone, port=port, lifetime=60, relativize=False
|
||||||
|
):
|
||||||
|
for rrset in message.answer:
|
||||||
|
for rdata in rrset:
|
||||||
|
record_type, value = record_data(rrset.rdtype, rdata)
|
||||||
|
records.append(
|
||||||
|
Record(
|
||||||
|
zone=normalize_name(zone),
|
||||||
|
owner=normalize_name(rrset.name.to_text()),
|
||||||
|
type=record_type,
|
||||||
|
value=value,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if not any(record.type == "SOA" for record in records):
|
||||||
|
raise RuntimeError("transfer returned no SOA record")
|
||||||
|
return records
|
||||||
|
except (dns.exception.DNSException, OSError, RuntimeError) as error:
|
||||||
|
errors.append(f"{address}: {error}")
|
||||||
|
raise RuntimeError(f"AXFR of {zone} from {server} failed ({'; '.join(errors)})")
|
||||||
|
|
||||||
|
|
||||||
|
def system_nameservers(domain: str) -> list[str]:
|
||||||
|
try:
|
||||||
|
answer = dns.resolver.resolve(domain, "NS", lifetime=30)
|
||||||
|
except dns.exception.DNSException as error:
|
||||||
|
message = f"cannot discover authoritative servers for {domain}: {error}"
|
||||||
|
raise RuntimeError(message) from error
|
||||||
|
return [rdata.target.to_text() for rdata in answer]
|
||||||
|
|
||||||
|
|
||||||
|
def nameservers_via(server: str, port: int, domain: str) -> list[str]:
|
||||||
|
errors = []
|
||||||
|
for address in server_addresses(server, port):
|
||||||
|
resolver = dns.resolver.Resolver(configure=False)
|
||||||
|
resolver.nameservers = [address]
|
||||||
|
resolver.port = port
|
||||||
|
try:
|
||||||
|
answer = resolver.resolve(domain, "NS", lifetime=15, search=False)
|
||||||
|
return [rdata.target.to_text() for rdata in answer]
|
||||||
|
except dns.exception.DNSException as error:
|
||||||
|
errors.append(f"{address}: {error}")
|
||||||
|
raise RuntimeError(f"NS query for {domain} via {server} failed ({'; '.join(errors)})")
|
||||||
|
|
||||||
|
|
||||||
|
def discover_nameservers(domains: list[str], port: int) -> dict[str, list[str]]:
|
||||||
|
discovered = {}
|
||||||
|
unresolved = {}
|
||||||
|
candidates = []
|
||||||
|
for domain in domains:
|
||||||
|
try:
|
||||||
|
servers = system_nameservers(domain)
|
||||||
|
discovered[domain] = servers
|
||||||
|
for server in servers:
|
||||||
|
if server not in candidates:
|
||||||
|
candidates.append(server)
|
||||||
|
except RuntimeError as error:
|
||||||
|
unresolved[domain] = [str(error)]
|
||||||
|
|
||||||
|
for domain, errors in list(unresolved.items()):
|
||||||
|
for server in candidates:
|
||||||
|
try:
|
||||||
|
discovered[domain] = nameservers_via(server, port, domain)
|
||||||
|
del unresolved[domain]
|
||||||
|
break
|
||||||
|
except RuntimeError as error:
|
||||||
|
errors.append(str(error))
|
||||||
|
|
||||||
|
if unresolved:
|
||||||
|
details = "; ".join(
|
||||||
|
f"{domain}: {'; '.join(errors)}" for domain, errors in unresolved.items()
|
||||||
|
)
|
||||||
|
raise RuntimeError(details)
|
||||||
|
return discovered
|
||||||
|
|
||||||
|
|
||||||
|
def transfer_domain(
|
||||||
|
port: int, domain: str, servers: list[str], fallback: list[str] = ()
|
||||||
|
) -> list[Record]:
|
||||||
|
# A zone may be delegated publicly to servers that refuse AXFR (e.g. HE serving
|
||||||
|
# reverse DNS) while our own authoritative servers, discovered for other zones,
|
||||||
|
# will transfer it. Try the delegated servers first, then fall back to those.
|
||||||
|
ordered = list(servers)
|
||||||
|
for server in fallback:
|
||||||
|
if server not in ordered:
|
||||||
|
ordered.append(server)
|
||||||
|
|
||||||
|
errors = []
|
||||||
|
for server in ordered:
|
||||||
|
try:
|
||||||
|
return transfer(server, port, domain)
|
||||||
|
except RuntimeError as error:
|
||||||
|
errors.append(str(error))
|
||||||
|
raise RuntimeError(f"no authoritative server allowed AXFR for {domain} ({'; '.join(errors)})")
|
||||||
|
|
||||||
|
|
||||||
|
def dynamic_names(records: list[Record]) -> set[str]:
|
||||||
|
return {record.owner for record in records if record.type == "DHCID"}
|
||||||
|
|
||||||
|
|
||||||
|
def dynamic_addresses(records: list[Record], names: set[str]) -> set[str]:
|
||||||
|
return {
|
||||||
|
record.value.rstrip(".").lower()
|
||||||
|
for record in records
|
||||||
|
if record.owner in names and record.type in {"A", "AAAA"}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def reverse_address(owner: str) -> str | None:
|
||||||
|
if owner.endswith(".in-addr.arpa"):
|
||||||
|
labels = owner.removesuffix(".in-addr.arpa").split(".")
|
||||||
|
if len(labels) != 4:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return str(ipaddress.IPv4Address(".".join(reversed(labels))))
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
if owner.endswith(".ip6.arpa"):
|
||||||
|
labels = owner.removesuffix(".ip6.arpa").split(".")
|
||||||
|
if len(labels) != 32:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
value = int("".join(reversed(labels)), 16)
|
||||||
|
return str(ipaddress.IPv6Address(value))
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def static_records(records: list[Record]) -> list[Record]:
|
||||||
|
names = dynamic_names(records)
|
||||||
|
addresses = dynamic_addresses(records, names)
|
||||||
|
static = []
|
||||||
|
for record in records:
|
||||||
|
if record.type in IGNORED_TYPES or record.owner in names:
|
||||||
|
continue
|
||||||
|
if record.type == "PTR":
|
||||||
|
target = normalize_name(record.value.split()[0])
|
||||||
|
address = reverse_address(record.owner)
|
||||||
|
if target in names or address in addresses:
|
||||||
|
continue
|
||||||
|
static.append(record)
|
||||||
|
return static
|
||||||
|
|
||||||
|
|
||||||
|
def relative_name(owner: str, zone: str) -> str:
|
||||||
|
if owner == zone:
|
||||||
|
return "@"
|
||||||
|
suffix = f".{zone}"
|
||||||
|
return owner[: -len(suffix)] if owner.endswith(suffix) else owner
|
||||||
|
|
||||||
|
|
||||||
|
def markdown_code(value: str) -> str:
|
||||||
|
escaped = value.replace("|", "\\|")
|
||||||
|
return f"`{escaped}`"
|
||||||
|
|
||||||
|
|
||||||
|
def display_record(record: Record) -> tuple[str, str]:
|
||||||
|
if record.type != "LUA":
|
||||||
|
return record.type, record.value
|
||||||
|
return f"{record.value} (LUA)", "generated at query time"
|
||||||
|
|
||||||
|
|
||||||
|
def render_forward(domain: str, records: list[Record]) -> list[str]:
|
||||||
|
domain = normalize_name(domain)
|
||||||
|
rows = []
|
||||||
|
for record in records:
|
||||||
|
if record.zone != domain or record.type == "PTR":
|
||||||
|
continue
|
||||||
|
record_type, value = display_record(record)
|
||||||
|
rows.append((relative_name(record.owner, record.zone), record_type, value))
|
||||||
|
rows.sort(key=lambda row: (row[0] != "@", row[0], row[1], row[2]))
|
||||||
|
|
||||||
|
lines = ["| Name | Type | Value |", "|---|---|---|"]
|
||||||
|
lines.extend(
|
||||||
|
f"| {markdown_code(name)} | {markdown_code(record_type)} | {markdown_code(value)} |"
|
||||||
|
for name, record_type, value in rows
|
||||||
|
)
|
||||||
|
return lines
|
||||||
|
|
||||||
|
|
||||||
|
def render_reverse(domain: str, records: list[Record]) -> list[str]:
|
||||||
|
domain = normalize_name(domain)
|
||||||
|
rows = []
|
||||||
|
for record in records:
|
||||||
|
if record.zone != domain or record.type != "PTR":
|
||||||
|
continue
|
||||||
|
rows.append((reverse_address(record.owner) or record.owner, record.value))
|
||||||
|
rows.sort(key=lambda row: ipaddress.ip_address(row[0]))
|
||||||
|
|
||||||
|
lines = ["| Address | Name |", "|---|---|"]
|
||||||
|
lines.extend(
|
||||||
|
f"| {markdown_code(address)} | {markdown_code(name)} |" for address, name in rows
|
||||||
|
)
|
||||||
|
return lines
|
||||||
|
|
||||||
|
|
||||||
|
def is_reverse(domain: str) -> bool:
|
||||||
|
domain = normalize_name(domain)
|
||||||
|
return domain.endswith(".in-addr.arpa") or domain.endswith(".ip6.arpa")
|
||||||
|
|
||||||
|
|
||||||
|
def rendered_zones(transferred: list[tuple[str, list[Record]]]) -> dict[str, list[str]]:
|
||||||
|
records = static_records([record for _domain, zone in transferred for record in zone])
|
||||||
|
rendered = {}
|
||||||
|
for domain, _raw_records in transferred:
|
||||||
|
if is_reverse(domain):
|
||||||
|
rendered[normalize_name(domain)] = render_reverse(domain, records)
|
||||||
|
else:
|
||||||
|
rendered[normalize_name(domain)] = render_forward(domain, records)
|
||||||
|
return rendered
|
||||||
|
|
||||||
|
|
||||||
|
def update_target(transferred: list[tuple[str, list[Record]]], target: Path) -> bool:
|
||||||
|
rendered = rendered_zones(transferred)
|
||||||
|
text = target.read_text()
|
||||||
|
lines = text.splitlines()
|
||||||
|
marker_re = re.compile(r"^<!--\s*dns:\s*(\S+)\s*-->$")
|
||||||
|
found = set()
|
||||||
|
output = []
|
||||||
|
i = 0
|
||||||
|
while i < len(lines):
|
||||||
|
match = marker_re.match(lines[i].strip())
|
||||||
|
if not match or normalize_name(match.group(1)) not in rendered:
|
||||||
|
output.append(lines[i])
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
domain = normalize_name(match.group(1))
|
||||||
|
end = i + 1
|
||||||
|
while end < len(lines) and lines[end].strip() != "<!-- dns-end -->":
|
||||||
|
end += 1
|
||||||
|
if end >= len(lines):
|
||||||
|
raise RuntimeError(f"missing <!-- dns-end --> for {domain}")
|
||||||
|
|
||||||
|
output.extend(
|
||||||
|
[
|
||||||
|
lines[i],
|
||||||
|
"<!-- dns-start -->",
|
||||||
|
*rendered[domain],
|
||||||
|
"<!-- dns-end -->",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
found.add(domain)
|
||||||
|
i = end + 1
|
||||||
|
|
||||||
|
missing = rendered.keys() - found
|
||||||
|
if missing:
|
||||||
|
raise RuntimeError(f"missing DNS markers for: {', '.join(sorted(missing))}")
|
||||||
|
|
||||||
|
new = "\n".join(output) + "\n"
|
||||||
|
if new == text:
|
||||||
|
return False
|
||||||
|
target.write_text(new)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(prog="update-docs-dns", description=__doc__)
|
||||||
|
parser.add_argument("domain", nargs="+", help="DNS zone to transfer")
|
||||||
|
parser.add_argument("--port", type=int, default=DEFAULT_PORT)
|
||||||
|
parser.add_argument("--output", type=Path, default=OUT)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
try:
|
||||||
|
nameservers = discover_nameservers(args.domain, args.port)
|
||||||
|
fallback = []
|
||||||
|
for servers in nameservers.values():
|
||||||
|
for server in servers:
|
||||||
|
if server not in fallback:
|
||||||
|
fallback.append(server)
|
||||||
|
transferred = [
|
||||||
|
(domain, transfer_domain(args.port, domain, nameservers[domain], fallback))
|
||||||
|
for domain in args.domain
|
||||||
|
]
|
||||||
|
except RuntimeError as error:
|
||||||
|
print(f"update-docs-dns: {error}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
try:
|
||||||
|
changed = update_target(transferred, args.output)
|
||||||
|
except (OSError, RuntimeError) as error:
|
||||||
|
print(f"update-docs-dns: {error}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
if changed:
|
||||||
|
print(f"updated {args.output}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Executable
+108
@@ -0,0 +1,108 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render docs/reference/nixos-options.md from the `my.*` module options.
|
||||||
|
|
||||||
|
Builds the `optionsDoc` flake output (a `nixosOptionsDoc` JSON dump of the custom
|
||||||
|
`my.*` options) and renders it to a per-module table. The prose lives in the option
|
||||||
|
descriptions in `nixos/modules/`; this file is generated.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
OPTIONS_DOC_ATTR = ".#nixfiles.config.nixos.optionsDoc"
|
||||||
|
OUT = Path("docs/reference/nixos-options.md")
|
||||||
|
HEADER = "| Option | Type | Default | Description |"
|
||||||
|
SEP = "|---|---|---|---|"
|
||||||
|
|
||||||
|
|
||||||
|
def collapse(text: str) -> str:
|
||||||
|
# Strip the flake's own /nix/store/<hash>-source/ prefix from rendered values so
|
||||||
|
# defaults like `./.keys/deploy.pub` are stable (the hash changes every time this
|
||||||
|
# generated file — part of the source tree — is rewritten).
|
||||||
|
text = re.sub(r"/nix/store/[a-z0-9]{32}-source/", "", text)
|
||||||
|
text = re.sub(
|
||||||
|
r'<link\s+xlink:href="([^"]+)">([\s\S]+?)</link>',
|
||||||
|
r"[\2](\1)",
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
return re.sub(r"\s+", " ", text).strip().replace("|", "\\|")
|
||||||
|
|
||||||
|
|
||||||
|
def render_default(entry: dict) -> str:
|
||||||
|
if "default" not in entry:
|
||||||
|
return "—"
|
||||||
|
d = entry["default"]
|
||||||
|
# A dict default is a `defaultText` / `literalExpression` (pre-rendered Nix in `text`);
|
||||||
|
# anything else is a literal value we JSON-encode.
|
||||||
|
if isinstance(d, dict):
|
||||||
|
text = d.get("text")
|
||||||
|
if text is None:
|
||||||
|
text = json.dumps(d.get("value"))
|
||||||
|
else:
|
||||||
|
text = json.dumps(d)
|
||||||
|
text = collapse(text)
|
||||||
|
return f"`{text}`" if text else "—"
|
||||||
|
|
||||||
|
|
||||||
|
def build_options_json() -> dict:
|
||||||
|
# `optionsDoc` builds to a derivation containing share/doc/nixos/options.json; the last
|
||||||
|
# printed out-path is the derivation root.
|
||||||
|
out = subprocess.run(
|
||||||
|
["nix", "build", "--no-link", "--print-out-paths", OPTIONS_DOC_ATTR],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=True,
|
||||||
|
).stdout.strip().splitlines()[-1]
|
||||||
|
return json.loads((Path(out) / "share/doc/nixos/options.json").read_text())
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
opts = build_options_json()
|
||||||
|
|
||||||
|
by_module: dict[str, list[tuple[str, dict]]] = {}
|
||||||
|
for key, entry in opts.items():
|
||||||
|
decls = entry.get("declarations") or []
|
||||||
|
# Only document options declared in this repo's modules. Some `my.*` options
|
||||||
|
# (e.g. `my.buildAs`, which aliases `options.system.build`) pull in sub-options
|
||||||
|
# declared in nixpkgs; their declaration paths don't exist here — skip them.
|
||||||
|
if not decls or not Path(decls[0]).exists():
|
||||||
|
continue
|
||||||
|
by_module.setdefault(decls[0], []).append((key, entry))
|
||||||
|
|
||||||
|
lines = [
|
||||||
|
"# NixOS module options (`my.*`)",
|
||||||
|
"",
|
||||||
|
"> **Generated** from the module option descriptions by `nix run .#update-docs-options`;",
|
||||||
|
"> CI keeps it current. The source of truth is the `mkOpt'` / `mkBoolOpt'` declarations in",
|
||||||
|
"> `nixos/modules/` — edit those, not this file. For what each module is *for*, see the",
|
||||||
|
"> [shared-modules overview](../architecture.md#shared-modules).",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
for decl in sorted(by_module):
|
||||||
|
mod = Path(decl).stem
|
||||||
|
if decl != "(unknown)":
|
||||||
|
lines.append(f"## `{mod}` — [`{decl}`](../../{decl})")
|
||||||
|
else:
|
||||||
|
lines.append(f"## `{mod}`")
|
||||||
|
lines += ["", HEADER, SEP]
|
||||||
|
for key, entry in sorted(by_module[decl], key=lambda kv: kv[0]):
|
||||||
|
lines.append(
|
||||||
|
f"| `{key}` | {collapse(entry.get('type', ''))} | "
|
||||||
|
f"{render_default(entry)} | {collapse(entry.get('description', ''))} |"
|
||||||
|
)
|
||||||
|
lines.append("")
|
||||||
|
|
||||||
|
OUT.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
new = "\n".join(lines).rstrip() + "\n"
|
||||||
|
old = OUT.read_text() if OUT.exists() else ""
|
||||||
|
if new != old:
|
||||||
|
OUT.write_text(new)
|
||||||
|
print(f"updated {OUT}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
+12
-16
@@ -18,13 +18,16 @@ physical host (VM host)
|
|||||||
|
|
||||||
Not every box fits this pattern, but **colony** and **home** are organised this way.
|
Not every box fits this pattern, but **colony** and **home** are organised this way.
|
||||||
|
|
||||||
## Cross-cutting topics
|
## General
|
||||||
|
|
||||||
- [`architecture.md`](architecture.md) — the custom module system, `my.*` namespace, multiple
|
- [`architecture.md`](architecture.md) — the custom module system, `my.*` namespace, multiple
|
||||||
nixpkgs channels, shared module inventory.
|
nixpkgs channels, shared module inventory.
|
||||||
- [`networking.md`](networking.md) — network assignments, domains, site topologies, router HA,
|
- [`networking.md`](networking.md) — network assignments, domains, site topologies, router HA,
|
||||||
the AS211024 L2 mesh, BGP, WireGuard, Tailscale.
|
the AS211024 L2 mesh, BGP, WireGuard, Tailscale.
|
||||||
- [`deployment.md`](deployment.md) — deploy-rs, devshell commands, secrets workflow, CI.
|
- [`deployment.md`](deployment.md) — deploy-rs, devshell commands, secrets workflow, CI.
|
||||||
|
- [`reference/dns.md`](reference/dns.md) — generated forward and reverse DNS record reference.
|
||||||
|
- [`reference/nixos-options.md`](reference/nixos-options.md) — generated per-option reference for
|
||||||
|
the custom `my.*` NixOS modules.
|
||||||
|
|
||||||
## Site: colony (Amsterdam)
|
## Site: colony (Amsterdam)
|
||||||
|
|
||||||
@@ -61,7 +64,7 @@ h.nul.ie
|
|||||||
├── palace (physical VM host — AMD, 100G, SR-IOV)
|
├── palace (physical VM host — AMD, 100G, SR-IOV)
|
||||||
│ ├── river ── primary router VM (PPPoE / Digiweb WAN)
|
│ ├── river ── primary router VM (PPPoE / Digiweb WAN)
|
||||||
│ ├── cellar ─ NVMe-oF / SPDK storage target VM
|
│ ├── cellar ─ NVMe-oF / SPDK storage target VM
|
||||||
│ └── sfh ──── container host VM ("services for home")
|
│ └── sfh ──── container host VM ("shill from home")
|
||||||
│ ├── hass ── Home Assistant + Frigate + MQTT (container)
|
│ ├── hass ── Home Assistant + Frigate + MQTT (container)
|
||||||
│ └── unifi ─ UniFi controller (container)
|
│ └── unifi ─ UniFi controller (container)
|
||||||
├── stream (physical secondary router — Virgin Media WAN)
|
├── stream (physical secondary router — Virgin Media WAN)
|
||||||
@@ -70,19 +73,11 @@ h.nul.ie
|
|||||||
|
|
||||||
## Remote boxes
|
## Remote boxes
|
||||||
|
|
||||||
Edge VPSes and the remote `kelder` site — see [`remote/README.md`](remote/README.md).
|
The edge VPSes and remote `kelder` site are indexed in [`remote/README.md`](remote/README.md).
|
||||||
|
|
||||||
- [`remote/britway.md`](remote/britway.md) — London VPS: headscale control plane, Tailscale exit
|
|
||||||
node, BGP edge.
|
|
||||||
- [`remote/britnet.md`](remote/britnet.md) — Birmingham VPS: Tailscale exit node, WireGuard hub.
|
|
||||||
- [`remote/kelder.md`](remote/kelder.md) — secondary home server: container host, Nextcloud,
|
|
||||||
Samba, WireGuard to `estuary`.
|
|
||||||
- [`remote/kelder-acquisition.md`](remote/kelder-acquisition.md) — media acquisition container.
|
|
||||||
- [`remote/kelder-spoder.md`](remote/kelder-spoder.md) — Nextcloud + nginx container.
|
|
||||||
|
|
||||||
## Mobile boxes
|
## Mobile boxes
|
||||||
|
|
||||||
- [`mobile/tower.md`](mobile/tower.md) — laptop workstation.
|
The laptop is indexed in [`mobile/README.md`](mobile/README.md).
|
||||||
|
|
||||||
## Misc
|
## Misc
|
||||||
|
|
||||||
@@ -90,7 +85,8 @@ Edge VPSes and the remote `kelder` site — see [`remote/README.md`](remote/READ
|
|||||||
|
|
||||||
## A note on the assignment tables
|
## A note on the assignment tables
|
||||||
|
|
||||||
The "Network assignments" tables in box pages (between `<!-- assignments: … -->` markers) are
|
The consolidated [`Box assignments`](networking.md#box-assignments) tables in
|
||||||
**generated from the flake** (`nixos.allAssignments`) by `nix run .#update-docs-assignments` —
|
[`networking.md`](networking.md) (one per site, between `<!-- assignments: <site> -->` markers)
|
||||||
CI refreshes them on push. Hand-written prose and the table's Notes column are preserved; don't
|
are **generated from the flake** (`nixos.allAssignments`) by `nix run .#update-docs-assignments` —
|
||||||
hand-edit the other cells.
|
CI refreshes them on push. Individual box pages link to that section rather than carrying their
|
||||||
|
own table. Only the Notes column is hand-written; don't hand-edit the other cells.
|
||||||
|
|||||||
+56
-32
@@ -73,11 +73,11 @@ Each entry of `nixos.systems` is a submodule defined by `systemOpts` in
|
|||||||
|
|
||||||
`mkSystem` does the real work:
|
`mkSystem` does the real work:
|
||||||
|
|
||||||
1. It selects the channel's nixpkgs flake (`pkgsFlakes.${config'.nixpkgs}`) and imports
|
1. It selects the channel's nixpkgs flake (`pkgsFlakes.${config'.nixpkgs}`).
|
||||||
`nixos/lib/eval-config.nix` **by hand**, because the flake force-sets `lib` and
|
- It imports `nixos/lib/eval-config.nix` **by hand** because the flake force-sets `lib`;
|
||||||
`eval-config.nix` would otherwise import its own unextended one. The lib it passes is the
|
otherwise `eval-config.nix` would import its own unextended copy.
|
||||||
channel's `pkgs.lib` extended with `lib.my` plus a `versionOverlay` that stamps
|
- The supplied library is the channel's `pkgs.lib` extended with `lib.my` and a
|
||||||
`system.nixos` with the flake revision and channel.
|
`versionOverlay` that stamps `system.nixos` with the flake revision and channel.
|
||||||
2. `specialArgs` gives every module access to `self`, `inputs`, `pkgsFlakes`, the box's own
|
2. `specialArgs` gives every module access to `self`, `inputs`, `pkgsFlakes`, the box's own
|
||||||
`pkgsFlake`, `allAssignments` (every box's assignments), and `systems` (all of
|
`pkgsFlake`, `allAssignments` (every box's assignments), and `systems` (all of
|
||||||
`nixos.systems`). Passing these via `specialArgs` (rather than module `imports`) avoids
|
`nixos.systems`). Passing these via `specialArgs` (rather than module `imports`) avoids
|
||||||
@@ -96,12 +96,12 @@ Each entry of `nixos.systems` is a submodule defined by `systemOpts` in
|
|||||||
- `nixpkgs.system` is set and the overlays computed at flake level (lib overlay + custom
|
- `nixpkgs.system` is set and the overlays computed at flake level (lib overlay + custom
|
||||||
packages) are passed through, so `pkgs` is imported modularly with config and overlays
|
packages) are passed through, so `pkgs` is imported modularly with config and overlays
|
||||||
applied.
|
applied.
|
||||||
- home-manager integration: `useGlobalPkgs` defaults to true when the system and
|
- `home-manager.useGlobalPkgs` defaults to true when the system and home-manager channels
|
||||||
home-manager channels match (a warning is emitted when they deliberately differ);
|
match; a warning is emitted when they deliberately differ.
|
||||||
`sharedModules` is every module in `home-manager.modules` plus an inline module that
|
- `home-manager.sharedModules` includes every module in `home-manager.modules` plus an inline
|
||||||
passes `pkgsPath` / `pkgs'` into home-manager, disables the release check, and pins
|
module. The inline module passes `pkgsPath` / `pkgs'`, disables the release check, and pins
|
||||||
`home.stateVersion` via `homeStateVersion` (currently `22.11` for stable-flavoured
|
`home.stateVersion` through `homeStateVersion` (`22.11` for stable-flavoured channels,
|
||||||
channels, `23.05` otherwise).
|
`23.05` otherwise).
|
||||||
5. Finally `applyAssertions` throws with all failed assertion messages (and shows warnings)
|
5. Finally `applyAssertions` throws with all failed assertion messages (and shows warnings)
|
||||||
when the merged configuration is forced.
|
when the merged configuration is forced.
|
||||||
|
|
||||||
@@ -136,8 +136,8 @@ etc. Inside a module, `pkgs` is the selected channel and `pkgs'` is the attrset
|
|||||||
flake-level `lib` (`pkgsFlakes.unstable.lib` so extended) is for platform-independent flake
|
flake-level `lib` (`pkgsFlakes.unstable.lib` so extended) is for platform-independent flake
|
||||||
use only; each box gets its own channel's lib extended the same way. Contents:
|
use only; each box gets its own channel's lib extended the same way. Contents:
|
||||||
|
|
||||||
- **Option helpers** — `mkOpt'`, `mkBoolOpt'`, `nullOrOpt'`, `mkDefault'` (`mkOverride 900`,
|
- **Option helpers** — `mkOpt'`, `mkBoolOpt'`, `nullOrOpt'`, `mkDefault'` (slightly stronger than
|
||||||
slightly stronger than `mkDefault`), `mkVMOverride'`, `inlineModule'` / `inlineModule`
|
`mkDefault`), `mkVMOverride'`, `inlineModule'` / `inlineModule`
|
||||||
(attach `_file` provenance), `commonOpts` (the shared `system`/`nixpkgs`/`home-manager`
|
(attach `_file` provenance), `commonOpts` (the shared `system`/`nixpkgs`/`home-manager`
|
||||||
options and the `moduleType` used for exported modules), `applyAssertions`, `duplicates`.
|
options and the `moduleType` used for exported modules), `applyAssertions`, `duplicates`.
|
||||||
- **`lib.my.net`** — CIDR/IP math (`net.cidr.host`, `net.cidr.subnet`, `net.types.ipv4`, …)
|
- **`lib.my.net`** — CIDR/IP math (`net.cidr.host`, `net.cidr.subnet`, `net.types.ipv4`, …)
|
||||||
@@ -152,11 +152,14 @@ use only; each box gets its own channel's lib extended the same way. Contents:
|
|||||||
(`ifaceA`, `lookupIP`) used by the home routers.
|
(`ifaceA`, `lookupIP`) used by the home routers.
|
||||||
- **networkd helpers** — `networkdAssignment` (assignment → `systemd.network` network, see
|
- **networkd helpers** — `networkdAssignment` (assignment → `systemd.network` network, see
|
||||||
[networking.md](networking.md)), `mkVLAN`, `dockerNetAssignment`.
|
[networking.md](networking.md)), `mkVLAN`, `dockerNetAssignment`.
|
||||||
- **Misc** — `mkDefaultSystemsPkgs`, `flakePackageOverlay` (wrap another flake's package as an
|
|
||||||
overlay), `isIPv6` / `parseIPPort` / `netBroadcast`, `nft` chain-name helpers,
|
### Misc
|
||||||
`systemdAwaitPostgres`, `vm.*` LVM disk descriptors for the `vms` module, the typed
|
|
||||||
`deploy-rs` option definitions (`lib.my.deploy-rs`), `netbootKeaClientClasses` (iPXE/EFI
|
The remaining helpers are `mkDefaultSystemsPkgs`, `flakePackageOverlay` (wrap another flake's
|
||||||
DHCP client classes), and `homeStateVersion`.
|
package as an overlay), `isIPv6` / `parseIPPort` / `netBroadcast`, `nft` chain-name helpers,
|
||||||
|
`systemdAwaitPostgres`, `vm.*` LVM disk descriptors for the `vms` module, the typed `deploy-rs`
|
||||||
|
option definitions (`lib.my.deploy-rs`), `netbootKeaClientClasses` (iPXE/EFI DHCP client classes),
|
||||||
|
and `homeStateVersion`.
|
||||||
|
|
||||||
Custom NixOS / home-manager modules declare their options under the `my.*` namespace (the root
|
Custom NixOS / home-manager modules declare their options under the `my.*` namespace (the root
|
||||||
`options.my` is declared in `nixos/modules/common.nix` / `home-manager/modules/common.nix`) —
|
`options.my` is declared in `nixos/modules/common.nix` / `home-manager/modules/common.nix`) —
|
||||||
@@ -166,7 +169,9 @@ e.g. `my.secrets`, `my.build`, `my.tmproot`, `my.firewall`, `my.server`, `my.dep
|
|||||||
## Shared modules
|
## Shared modules
|
||||||
|
|
||||||
Registered in [`nixos/modules/_list.nix`](../nixos/modules/_list.nix) and applied to **every**
|
Registered in [`nixos/modules/_list.nix`](../nixos/modules/_list.nix) and applied to **every**
|
||||||
system (box files opt in per-feature via `my.*` options):
|
system (box files opt in per-feature via `my.*` options). The table below is the overview — what
|
||||||
|
each module is *for*; for the exhaustive, CI-generated per-option reference (types, defaults,
|
||||||
|
descriptions) see [`reference/nixos-options.md`](reference/nixos-options.md).
|
||||||
|
|
||||||
| Module | Provides |
|
| Module | Provides |
|
||||||
|---|---|
|
|---|---|
|
||||||
@@ -224,19 +229,38 @@ home (including the per-user homes attached to systems via `my.user.homeConfig`)
|
|||||||
|
|
||||||
## Home-manager
|
## Home-manager
|
||||||
|
|
||||||
Home-manager has the same two-level structure: `homeOpts` (in
|
Home-manager follows the same option-definition and evaluation split as NixOS.
|
||||||
[`home-manager/default.nix`](../home-manager/default.nix)) defines `system`, `nixpkgs`,
|
|
||||||
`home-manager`, `homeDirectory`, `username`, and `configuration`; `mkHome` calls the selected
|
### `homeOpts`
|
||||||
channel's `lib.homeManagerConfiguration`, passing the channel's `pkgs'` (with `config` emptied
|
|
||||||
so home-manager applies overlays/config modularly itself), `extraSpecialArgs` (`inputs`,
|
[`home-manager/default.nix`](../home-manager/default.nix) defines each entry with these options:
|
||||||
`pkgsFlakes`, `pkgsFlake`), every module in `home-manager.modules`, an inline module that sets
|
|
||||||
`home.homeDirectory` / `home.username` and provides `pkgs'` (all channels) as a module arg,
|
| Option | Meaning |
|
||||||
and the pinned `home.stateVersion`.
|
|---|---|
|
||||||
|
| `system` | Target platform. |
|
||||||
|
| `nixpkgs` | nixpkgs channel used for the home packages. |
|
||||||
|
| `home-manager` | home-manager channel. |
|
||||||
|
| `homeDirectory` / `username` | User identity and home path. |
|
||||||
|
| `configuration` | The home-manager module definitions evaluated by `mkHome`. |
|
||||||
|
|
||||||
|
### `mkHome`
|
||||||
|
|
||||||
|
1. Calls the selected channel's `lib.homeManagerConfiguration`.
|
||||||
|
2. Supplies that channel's `pkgs'` with `config` emptied, allowing home-manager to apply package
|
||||||
|
configuration and overlays through its module system.
|
||||||
|
3. Passes `inputs`, `pkgsFlakes` and the selected `pkgsFlake` through `extraSpecialArgs`.
|
||||||
|
4. Loads every module in `home-manager.modules` plus an inline module that sets
|
||||||
|
`home.homeDirectory` / `home.username` and exposes every channel through the `pkgs'` module
|
||||||
|
argument.
|
||||||
|
5. Pins `home.stateVersion` through `homeStateVersion`.
|
||||||
|
|
||||||
|
### Standalone homes
|
||||||
|
|
||||||
Standalone homes live in [`home-manager/configs/`](../home-manager/configs) and are named
|
Standalone homes live in [`home-manager/configs/`](../home-manager/configs) and are named
|
||||||
`<user>@<host>` (deploy-rs mangles the `@` to `-at-`). `macsimum` is a home-manager-only
|
`<user>@<host>` (deploy-rs mangles the `@` to `-at-`). `macsimum` is a home-manager-only
|
||||||
config — an `x86_64-darwin` macOS box with no NixOS side at all. Note that **no homes are
|
config — an `x86_64-darwin` macOS box with no NixOS side.
|
||||||
currently wired into the flake**: the `home-manager/configs/macsimum.nix` entry in `configs` is
|
|
||||||
commented out, and `home-manager/configs/castle.nix` exists on disk but is not listed either,
|
No standalone homes are currently wired into the flake: `home-manager/configs/macsimum.nix` is
|
||||||
so `nixfiles.config.home-manager.homes` currently evaluates to `{}`. Most day-to-day user
|
commented out of `configs`, while `home-manager/configs/castle.nix` is not listed. Consequently,
|
||||||
configuration instead rides along with systems through `my.user.homeConfig`.
|
`nixfiles.config.home-manager.homes` evaluates to `{}`. Most user configuration instead accompanies
|
||||||
|
NixOS boxes through `my.user.homeConfig`.
|
||||||
|
|||||||
+91
-63
@@ -32,19 +32,16 @@ The shared [`nixos/modules/deploy-rs.nix`](../nixos/modules/deploy-rs.nix) modul
|
|||||||
`sudo = "doas -u"` (or `sudo -u`), `sshOpts` = the box's first OpenSSH port. The module
|
`sudo = "doas -u"` (or `sudo -u`), `sshOpts` = the box's first OpenSSH port. The module
|
||||||
creates the `deploy` system user (bash shell, wheel, keys from `my.deploy.authorizedKeys`,
|
creates the `deploy` system user (bash shell, wheel, keys from `my.deploy.authorizedKeys`,
|
||||||
which defaults to `.keys/deploy.pub`).
|
which defaults to `.keys/deploy.pub`).
|
||||||
- Generated profiles:
|
|
||||||
- **`system`** — activates `config.system.build.toplevel` with
|
Generated profiles deploy in this order:
|
||||||
`switch-to-configuration switch`, with workarounds for the `/tmp` cwd issue
|
|
||||||
(NixOS/nixpkgs#73404) and the systemd-boot `loader.conf` default-entry issue
|
1. **`system`** activates `config.system.build.toplevel` with `switch-to-configuration switch`,
|
||||||
(deploy-rs#31), then prunes old generations (`nix-env --delete-generations +10`,
|
applies the `/tmp` cwd and systemd-boot `loader.conf` workarounds, then prunes old generations.
|
||||||
tunable via `keepGenerations`, 0 disables).
|
`keepGenerations` controls how much history is retained; zero disables pruning.
|
||||||
- **`container-<name>`** — one per `my.containers.instances` entry, activating the
|
2. **`container-<name>`** activates each `my.containers.instances` entry into
|
||||||
container's `my.buildAs.container` toplevel into
|
`/nix/var/nix/profiles/per-container/<name>/system`. With `hotReload` (the default), it reloads
|
||||||
`/nix/var/nix/profiles/per-container/<name>/system`. With `hotReload` (default) the
|
`systemd-nspawn@<name>` and restarts only a stopped container or one still running the dummy
|
||||||
profile *reloads* `systemd-nspawn@<name>` (restarting only if the container is down or
|
init; otherwise it restarts the unit. The same generation cleanup applies.
|
||||||
still running the placeholder "dummy" init); otherwise it restarts it. Generation cleanup
|
|
||||||
applies here too.
|
|
||||||
- Profiles deploy in order `system`, then the containers.
|
|
||||||
|
|
||||||
### Usage
|
### Usage
|
||||||
|
|
||||||
@@ -55,13 +52,16 @@ The shared [`nixos/modules/deploy-rs.nix`](../nixos/modules/deploy-rs.nix) modul
|
|||||||
(deploy-rs's `--boot` maps to the custom activation's `boot` phase, which also does the
|
(deploy-rs's `--boot` maps to the custom activation's `boot` phase, which also does the
|
||||||
generation cleanup). Use it when a live `switch` would cut the box off mid-change (e.g. a
|
generation cleanup). Use it when a live `switch` would cut the box off mid-change (e.g. a
|
||||||
router WAN rework), then reboot to cut over.
|
router WAN rework), then reboot to cut over.
|
||||||
- `ssh-machine <name> [cmd]` — resolves `user@host` and the merged global+node `sshOpts` from
|
|
||||||
`.#deploy` (same `my.deploy.enable` gate as `deploy`, same `@` → `-at-` mangling), then
|
|
||||||
execs `ssh`. Boxes default to the `fish` login shell; pipe multi-statement remote scripts
|
|
||||||
through `bash` (`ssh-machine <name> bash -s < script.sh`). If outbound SSH hangs at the
|
|
||||||
publickey step (flaky agent), use `SSH_AUTH_SOCK= ssh-machine …`.
|
|
||||||
- `nix flake check` includes deploy-rs's own `deployChecks` for the whole `deploy` attrset.
|
- `nix flake check` includes deploy-rs's own `deployChecks` for the whole `deploy` attrset.
|
||||||
|
|
||||||
|
#### `ssh-machine`
|
||||||
|
|
||||||
|
`ssh-machine <name> [cmd]` resolves `user@host` and the merged global/node `sshOpts` from
|
||||||
|
`.#deploy`, using the same `my.deploy.enable` gate and `@` → `-at-` mangling as `deploy`, then runs
|
||||||
|
`ssh`. Boxes default to `fish`, so pipe multi-statement remote scripts through `bash`, for example
|
||||||
|
`ssh-machine <name> bash -s < script.sh`. If a flaky agent stalls public-key authentication, use
|
||||||
|
`SSH_AUTH_SOCK= ssh-machine …`.
|
||||||
|
|
||||||
## Devshell commands
|
## Devshell commands
|
||||||
|
|
||||||
The repo ships a `numtide/devshell` ([`devshell/`](../devshell), entered via `direnv`). Run a
|
The repo ships a `numtide/devshell` ([`devshell/`](../devshell), entered via `direnv`). Run a
|
||||||
@@ -115,22 +115,26 @@ VM's unix sockets from `/run/vms/<vm>/` on `<host>` over SSH):
|
|||||||
Secrets are age-encrypted files in [`secrets/`](../secrets), managed with **ragenix** (a fork
|
Secrets are age-encrypted files in [`secrets/`](../secrets), managed with **ragenix** (a fork
|
||||||
with a rekey flag, from the flake inputs).
|
with a rekey flag, from the flake inputs).
|
||||||
|
|
||||||
- Each box declares `my.secrets.key` (the host public key its secrets encrypt to) and
|
### Per-box declarations
|
||||||
`my.secrets.files.<name>` (files to decrypt; attribute values merge into the agenix secret,
|
|
||||||
e.g. `owner`/`mode`). At runtime the `secrets` module decrypts via
|
Each box declares `my.secrets.key`, the host public key its secrets encrypt to, and
|
||||||
`age.secrets."<name>".path`, with identity paths derived from the box's OpenSSH host keys —
|
`my.secrets.files.<name>`, whose values merge settings such as `owner` and `mode` into the agenix
|
||||||
read from the tmproot persistence dir when there is one, since agenix runs before persisted
|
secret. At runtime, identity paths come from the box's OpenSSH host keys. On tmproot boxes they are
|
||||||
keys would otherwise be available.
|
read from the persistence directory because agenix runs before the persisted keys would otherwise
|
||||||
- [`secrets.nix`](../secrets.nix) (the ragenix rules file at the repo root) is **generated**:
|
be available.
|
||||||
it evaluates the flake, collects every system's `my.secrets.files` + `key`, and emits each
|
|
||||||
secret with its recipient list — always including `.keys/dev.pub` so the dev key can open
|
### Recipient rules
|
||||||
everything. Re-running `ragenix -r` re-keys after adding a box or secret.
|
|
||||||
- The `ragenix` devshell command wraps `ragenix --identity .keys/dev.key`; use it to
|
[`secrets.nix`](../secrets.nix) is generated from every system's `my.secrets.files` and `key`.
|
||||||
edit/rekey secrets.
|
Each recipient list also includes `.keys/dev.pub`, so the development key can open every secret.
|
||||||
- The `.keys/` directory holds the dev key (`dev.key`/`dev.pub`), the deploy key
|
Run `ragenix -r` after adding a box or secret to re-key the files.
|
||||||
(`deploy.key`/`deploy.pub`, authorized on every box's `deploy` user), and assorted other
|
|
||||||
keys referenced by `lib.my.c.sshKeyFiles`. It is required for editing secrets, deploying,
|
### Local keys and editing
|
||||||
and running dev VMs.
|
|
||||||
|
The `ragenix` devshell command wraps `ragenix --identity .keys/dev.key`. The `.keys/` directory
|
||||||
|
contains that development key, the deploy key authorized for every box's `deploy` user, and other
|
||||||
|
keys referenced by `lib.my.c.sshKeyFiles`; it is required for editing secrets, deploying and running
|
||||||
|
development VMs.
|
||||||
|
|
||||||
## Dev VMs
|
## Dev VMs
|
||||||
|
|
||||||
@@ -144,35 +148,59 @@ automatically excluded from deploy targets.
|
|||||||
|
|
||||||
## CI
|
## CI
|
||||||
|
|
||||||
GitHub/Gitea Actions workflows live in [`.gitea/workflows/`](../.gitea/workflows):
|
GitHub/Gitea Actions workflows live in [`.gitea/workflows/`](../.gitea/workflows).
|
||||||
|
|
||||||
- **`ci.yaml`** (push to `master`): `nix flake check --no-build`, then for every attribute of
|
### `ci.yaml`
|
||||||
`.#ci.x86_64-linux` (each system as `system-<name>`, each home as `home-<name>` with `@` →
|
|
||||||
`-at-`, each package as `package-<name>`, plus the dev `shell`) it builds and pushes the
|
|
||||||
result to the Harmonia binary cache with [`ci/push-to-cache.sh`](../ci/push-to-cache.sh).
|
|
||||||
Finally it builds `.#ciDrv.x86_64-linux` (a `linkFarm` of all CI attrs) and pushes it with
|
|
||||||
`UPDATE_PROFILE=1`, which updates the `nixfiles` profile on the cache box and garbage
|
|
||||||
collects paths older than 60 days. The cache is `ssh-ng://harmonia@object-ctr.ams1.int.nul.ie`
|
|
||||||
(remote store `/var/lib/harmonia`), keyed by the `HARMONIA_SSH_KEY` secret with a pinned
|
|
||||||
`ci/known_hosts`; clients consume it as `https://nix-cache.nul.ie` (see `lib.my.c.nix.cache`).
|
|
||||||
- **`installer.yaml`** (push of the `installer` tag; `update-installer` refreshes it): builds
|
|
||||||
the installer's `my.buildAs.iso` and `my.buildAs.netbootArchive` and attaches both to a
|
|
||||||
release.
|
|
||||||
- **`update-docs.yaml`** (push to the docs branch, skipping its own commits): runs
|
|
||||||
`nix run .#update-docs-assignments` and commits any changes as
|
|
||||||
`docs: update assignment tables`.
|
|
||||||
|
|
||||||
### The docs assignment-table updater
|
On pushes to `master`, this runs `nix flake check --no-build`, then builds every attribute of
|
||||||
|
`.#ci.x86_64-linux`: systems as `system-<name>`, homes as `home-<name>` (with `@` changed to
|
||||||
|
`-at-`), packages as `package-<name>`, and the development `shell`. Each result is pushed to the
|
||||||
|
Harmonia cache with [`ci/push-to-cache.sh`](../ci/push-to-cache.sh).
|
||||||
|
|
||||||
`update-docs-assignments` is registered in [`pkgs/default.nix`](../pkgs/default.nix) (a
|
It then builds `.#ciDrv.x86_64-linux`, a `linkFarm` of all CI attributes, and pushes it with
|
||||||
`writeShellScriptBin` wrapping [`ci/update-docs-assignments.py`](../ci/update-docs-assignments.py)).
|
`UPDATE_PROFILE=1`. That updates the `nixfiles` profile on the cache box and collects old paths
|
||||||
It evaluates `.#nixfiles.config.nixos.allAssignments` to JSON, walks `docs/**/*.md`, and for
|
according to the workflow's retention setting. The SSH store uses `/var/lib/harmonia`,
|
||||||
each file looks for a marker line `<!-- assignments: <box> -->` where `<box>` is the **file's
|
`HARMONIA_SSH_KEY`, and pinned `ci/known_hosts`; clients use `https://nix-cache.nul.ie` through
|
||||||
own stem** (e.g. `river.md` → `river`) and names a real box. The generated table (columns
|
`lib.my.c.nix.cache`.
|
||||||
`Name | Assignment | IPv4 | IPv6 | Domain | Notes`) is (re)written between the marker and a
|
|
||||||
closing `<!-- assignments-end -->` line, with an `<!-- assignments-start -->` line inserted
|
### `installer.yaml`
|
||||||
after the marker on the first run. Hand-written text in the **Notes** column is preserved
|
|
||||||
across runs (keyed by the Assignment cell), so notes survive regeneration. Pages that want a
|
Pushing the `installer` tag (refreshed by `update-installer`) builds `my.buildAs.iso` and
|
||||||
table only need the marker + end marker; pages without markers (including the three top-level
|
`my.buildAs.netbootArchive`, then attaches both to a release.
|
||||||
docs) are left alone. The script exits non-zero when it changed something, which is how the
|
|
||||||
workflow knows whether to commit.
|
### `update-docs.yaml`
|
||||||
|
|
||||||
|
On pushes to `master`, excluding its own commits, this runs the assignment, option and DNS
|
||||||
|
reference generators and commits changed outputs as `docs: Update generated references`.
|
||||||
|
|
||||||
|
### The docs generators
|
||||||
|
|
||||||
|
The generators are registered in [`pkgs/default.nix`](../pkgs/default.nix) as wrappers around
|
||||||
|
Python scripts under [`ci/`](../ci). They leave the worktree unchanged when their output is current;
|
||||||
|
the workflow stages `docs/` and uses `git diff --cached --quiet` to decide whether to commit.
|
||||||
|
|
||||||
|
`update-docs-assignments` ([`ci/update-docs-assignments.py`](../ci/update-docs-assignments.py))
|
||||||
|
evaluates `.#nixfiles.config.nixos.allAssignments` to JSON and rewrites the consolidated
|
||||||
|
[`Box assignments`](networking.md#box-assignments) tables in `docs/networking.md` — one table
|
||||||
|
per site (`colony` / `home` / `remote`), each between a `<!-- assignments: <site> -->` marker
|
||||||
|
and a closing `<!-- assignments-end -->` line. Boxes are grouped by site from their assignment
|
||||||
|
domain, and each `Box` cell links to that box's page when one exists. Hand-written text in the
|
||||||
|
**Notes** column is preserved across runs (keyed by box + assignment), so notes survive
|
||||||
|
regeneration. Individual box pages don't carry tables; they link to the consolidated section.
|
||||||
|
|
||||||
|
`update-docs-options` ([`ci/update-docs-options.py`](../ci/update-docs-options.py)) builds the
|
||||||
|
`nixos.optionsDoc` output (declared in [`nixos/default.nix`](../nixos/default.nix)) — a
|
||||||
|
`nixosOptionsDoc` JSON dump of the custom `my.*` options (declared with `mkOpt'` / `mkBoolOpt'` in
|
||||||
|
`nixos/modules/`). It's evaluated against a **minimal synthetic system**, since the shared modules
|
||||||
|
apply to every box, so defaults don't pick up a real host's values. The renderer writes
|
||||||
|
[`docs/reference/nixos-options.md`](reference/nixos-options.md), one table per module file. The
|
||||||
|
whole file is generated; edit the option descriptions in the modules, not the reference. The
|
||||||
|
internal `asX` build-target options are marked `internal = true` so they're excluded.
|
||||||
|
|
||||||
|
`update-docs-dns` ([`ci/update-docs-dns.py`](../ci/update-docs-dns.py)) accepts forward and reverse
|
||||||
|
zone names, discovers their authoritative nameservers through NS queries, and transfers each zone
|
||||||
|
over AXFR. If a private reverse zone is not visible through the configured recursive resolver, it
|
||||||
|
asks the authoritative servers discovered for the other requested zones. It updates only the
|
||||||
|
matching `<!-- dns: <zone> -->` blocks in the [`DNS records`](reference/dns.md) reference; the page's
|
||||||
|
headings and prose remain handwritten. Kea-managed owners are identified by `DHCID` records and
|
||||||
|
removed together with their A, AAAA and PTR records; SOA records and TTLs are also omitted.
|
||||||
|
|||||||
@@ -24,9 +24,8 @@ The custom NixOS installer image used to bootstrap new boxes.
|
|||||||
`installer-<hex>` hostname is set at boot.
|
`installer-<hex>` hostname is set at boot.
|
||||||
- `INSTALL_ROOT=/mnt` in the session environment, plus a `show-hw-config` alias wrapping
|
- `INSTALL_ROOT=/mnt` in the session environment, plus a `show-hw-config` alias wrapping
|
||||||
`nixos-generate-config --show-hardware-config --root $INSTALL_ROOT`.
|
`nixos-generate-config --show-hardware-config --root $INSTALL_ROOT`.
|
||||||
- NixOS documentation enabled, `wpa_supplicant` available but not started, aggressive GC
|
- NixOS documentation enabled, `wpa_supplicant` available but not started, GC and memory-overcommit
|
||||||
(`GC_INITIAL_HEAP_SIZE=1M`) and `vm.overcommit_memory=1` for low-memory targets, LVM thin
|
tuning for low-memory targets, LVM thin and NFS support.
|
||||||
and NFS support.
|
|
||||||
- No regular user (`my.user.enable = false`), no tmpfs-root management, no NAT, and not a
|
- No regular user (`my.user.enable = false`), no tmpfs-root management, no NAT, and not a
|
||||||
deploy target (`my.deploy.enable = false`).
|
deploy target (`my.deploy.enable = false`).
|
||||||
|
|
||||||
|
|||||||
@@ -4,6 +4,6 @@ Portable workstations that move between networks — currently just `tower`. Mob
|
|||||||
no static network assignments; they use DHCP/NetworkManager and reach other boxes over the
|
no static network assignments; they use DHCP/NetworkManager and reach other boxes over the
|
||||||
tailnet.
|
tailnet.
|
||||||
|
|
||||||
| Box | What it is | Docs |
|
| Box | What it is |
|
||||||
| --- | --- | --- |
|
| --- | --- |
|
||||||
| `tower` | Framework Laptop 13 workstation | [tower.md](tower.md) |
|
| [`tower`](tower.md) | Framework Laptop 13 workstation |
|
||||||
|
|||||||
+20
-8
@@ -4,6 +4,22 @@ Portable workstation — a Framework Laptop 13 (Intel), running the full GUI env
|
|||||||
|
|
||||||
- **Source:** [`nixos/boxes/tower/default.nix`](../../nixos/boxes/tower/default.nix)
|
- **Source:** [`nixos/boxes/tower/default.nix`](../../nixos/boxes/tower/default.nix)
|
||||||
- **Host:** physical (laptop)
|
- **Host:** physical (laptop)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
|
## Hardware / platform
|
||||||
|
|
||||||
|
| Component | Inventory |
|
||||||
|
|---|---|
|
||||||
|
| Platform | Framework Laptop 13 (12th Gen Intel Core) |
|
||||||
|
| CPU | Intel Core i5-1240P (12 cores / 16 threads) |
|
||||||
|
| Memory | 32 GiB |
|
||||||
|
| Storage | 2 TB WD Black SN850 NVMe SSD |
|
||||||
|
| Graphics | Integrated Intel Iris Xe |
|
||||||
|
| Connectivity | Intel AX210 Wi-Fi 6E, Bluetooth and Thunderbolt 4 |
|
||||||
|
|
||||||
|
The configuration enables Intel microcode updates, `kvm-intel`, `intel_iommu=on`,
|
||||||
|
`intel-media-driver` and the latest kernel (`lib.my.c.kernel.latest`). Thunderbolt security
|
||||||
|
(`bolt`), the fingerprint reader (`fprintd`) and `tlp` power management are also enabled.
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
@@ -11,20 +27,16 @@ Portable workstation — a Framework Laptop 13 (Intel), running the full GUI env
|
|||||||
- Joins the tailnet through the headscale on [`britway`](../remote/britway.md) (fish abbr
|
- Joins the tailnet through the headscale on [`britway`](../remote/britway.md) (fish abbr
|
||||||
`tsup` = `doas tailscale up --login-server=https://hs.nul.ie --accept-routes`).
|
`tsup` = `doas tailscale up --login-server=https://hs.nul.ie --accept-routes`).
|
||||||
|
|
||||||
## Hardware / platform
|
## Network assignments
|
||||||
|
|
||||||
- Intel platform: microcode updates, `kvm-intel`, `intel_iommu=on`, `intel-media-driver` for
|
`tower` has no static assignment; it uses DHCP through NetworkManager and reaches the other boxes
|
||||||
graphics; latest kernel (`lib.my.c.kernel.latest`).
|
over Tailscale.
|
||||||
- Thunderbolt security (`bolt`), Bluetooth (`blueman` + tray applet), fingerprint reader
|
|
||||||
(`fprintd`; `doas` persists auth for `wheel`).
|
|
||||||
- `tlp` power management, including battery charge thresholds (start 90% / stop 97%).
|
|
||||||
|
|
||||||
## Storage
|
## Storage
|
||||||
|
|
||||||
- Two LUKS-encrypted partitions, `persist` and `home` (both `allowDiscards`); `/nix` is a
|
- Two LUKS-encrypted partitions, `persist` and `home` (both `allowDiscards`); `/nix` is a
|
||||||
separate ext4 filesystem, `/boot` is vfat.
|
separate ext4 filesystem, `/boot` is vfat.
|
||||||
- Persistent `/home` (`my.user.tmphome = false`) with an 8G tmpfs root
|
- Persistent `/home` (`my.user.tmphome = false`) with a size-limited tmpfs root (`my.tmproot.size`).
|
||||||
(`my.tmproot.size = "8G"`).
|
|
||||||
|
|
||||||
## Networking / services
|
## Networking / services
|
||||||
|
|
||||||
|
|||||||
+287
-68
@@ -8,7 +8,7 @@ sites together. Switch-level home topology (jim/dave/brian, the ONT path) lives
|
|||||||
## Assignments
|
## Assignments
|
||||||
|
|
||||||
Every box declares `nixos.systems.<name>.assignments`, an attrset of *assignments* (one per
|
Every box declares `nixos.systems.<name>.assignments`, an attrset of *assignments* (one per
|
||||||
network the box is attached to). The option shape (`assignmentOpts` in
|
network the box is attached to). The option definition (`assignmentOpts` in
|
||||||
[`nixos/default.nix`](../nixos/default.nix)):
|
[`nixos/default.nix`](../nixos/default.nix)):
|
||||||
|
|
||||||
- `name` (defaults to the attribute name) and `altNames` — DNS names for the assignment.
|
- `name` (defaults to the attribute name) and `altNames` — DNS names for the assignment.
|
||||||
@@ -41,6 +41,196 @@ Two pieces of machinery consume assignments:
|
|||||||
the system name) and `networking.domain` to `assignments.internal.domain`. The shared
|
the system name) and `networking.domain` to `assignments.internal.domain`. The shared
|
||||||
`network` module sets a fallback domain of `int.nul.ie` for boxes without one.
|
`network` module sets a fallback domain of `int.nul.ie` for boxes without one.
|
||||||
|
|
||||||
|
## Box assignments
|
||||||
|
|
||||||
|
Every box's `assignments` (plus the routers' floating VIP `extraAssignments`), grouped first by site
|
||||||
|
and then by assignment key. Generated from `nixos.allAssignments` by
|
||||||
|
`nix run .#update-docs-assignments`; CI keeps it current. Only the **Notes** column is hand-written —
|
||||||
|
edit prose there, never the other generated cells.
|
||||||
|
|
||||||
|
### colony
|
||||||
|
|
||||||
|
<!-- assignments: colony -->
|
||||||
|
<!-- assignments-start -->
|
||||||
|
#### `internal`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`chatterbox`](sites/colony/shill/containers/chatterbox.md) | `10.100.2.5/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::5/64` | ams1.int.nul.ie | |
|
||||||
|
| [`colony`](sites/colony/colony.md) | `94.142.241.224/32` | `2a0e:97c0:4d2:10::2/64` | ams1.int.nul.ie | |
|
||||||
|
| [`colony-psql`](sites/colony/shill/containers/colony-psql.md) | `10.100.2.4/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::4/64` | ams1.int.nul.ie | |
|
||||||
|
| `enshrouded-oci` | `10.100.3.5/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::5/64` | ams1.int.nul.ie | Enshrouded OCI container on [`whale2`](sites/colony/whale2.md#game-servers); disabled |
|
||||||
|
| [`estuary`](sites/colony/estuary.md) | `94.142.240.44/24 gw 94.142.240.254` | `2a02:898:0:20::329:1/64 gw 2a02:898:0:20::1` | ams1.int.nul.ie | |
|
||||||
|
| [`gam`](sites/colony/shill/containers/gam.md) | `10.100.2.11/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::b/64` | ams1.int.nul.ie | |
|
||||||
|
| [`git`](sites/colony/git.md) | `94.142.241.117/32` | `2a0e:97c0:4d2:11::4/64` | ams1.int.nul.ie | |
|
||||||
|
| `graeme-oci` | `10.100.3.8/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::8/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
|
||||||
|
| [`jackflix`](sites/colony/shill/containers/jackflix.md) | `10.100.2.6/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::6/64` | ams1.int.nul.ie | |
|
||||||
|
| `kevcraft-oci` | `10.100.3.6/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::6/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
|
||||||
|
| `kinkcraft-oci` | `10.100.3.7/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::7/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
|
||||||
|
| [`middleman`](sites/colony/shill/containers/middleman.md) | `10.100.2.2/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::2/64` | ams1.int.nul.ie | |
|
||||||
|
| [`object`](sites/colony/shill/containers/object.md) | `10.100.2.7/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::7/64` | ams1.int.nul.ie | |
|
||||||
|
| [`qclk`](sites/colony/shill/containers/qclk.md) | `10.100.2.10/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::a/64` | ams1.int.nul.ie | |
|
||||||
|
| [`shill`](sites/colony/shill/README.md) | `94.142.241.225/32` | `2a0e:97c0:4d2:11::2/64` | ams1.int.nul.ie | |
|
||||||
|
| `simpcraft-oci` | `10.100.3.3/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::3/64` | ams1.int.nul.ie | Minecraft OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
|
||||||
|
| `simpcraft-staging-oci` | `10.100.3.4/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::4/64` | ams1.int.nul.ie | Minecraft staging OCI container on [`whale2`](sites/colony/whale2.md#game-servers); disabled |
|
||||||
|
| [`toot`](sites/colony/shill/containers/toot.md) | `10.100.2.8/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::8/64` | ams1.int.nul.ie | |
|
||||||
|
| `valheim-oci` | `10.100.3.2/24 gw 10.100.3.1` | `2a0e:97c0:4d2:13::2/64` | ams1.int.nul.ie | Valheim OCI container on [`whale2`](sites/colony/whale2.md#game-servers) |
|
||||||
|
| [`vaultwarden`](sites/colony/shill/containers/vaultwarden.md) | `10.100.2.3/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::3/64` | ams1.int.nul.ie | |
|
||||||
|
| [`waffletail`](sites/colony/shill/containers/waffletail.md) | `10.100.2.9/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::9/64` | ams1.int.nul.ie | |
|
||||||
|
| [`whale2`](sites/colony/whale2.md) | `94.142.241.226/32` | `2a0e:97c0:4d2:11::3/64` | ams1.int.nul.ie | |
|
||||||
|
|
||||||
|
#### `as211024`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`estuary`](sites/colony/estuary.md) | `10.100.50.1/24` | `2a0e:97c0:4df::1/64` | — | |
|
||||||
|
|
||||||
|
#### `base`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`estuary`](sites/colony/estuary.md) | `10.100.0.1/24` | `2a0e:97c0:4d2:10::1/64` | ams1.int.nul.ie | |
|
||||||
|
|
||||||
|
#### `ctrs`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`shill`](sites/colony/shill/README.md) | `10.100.2.1/24` | `2a0e:97c0:4d2:12::1/64` | ams1.int.nul.ie | |
|
||||||
|
|
||||||
|
#### `oci`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`whale2`](sites/colony/whale2.md) | `10.100.3.1/24` | `2a0e:97c0:4d2:13::1/64` | ams1.int.nul.ie | |
|
||||||
|
|
||||||
|
#### `qclk`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`qclk`](sites/colony/shill/containers/qclk.md) | `10.100.4.1/24` | — | — | |
|
||||||
|
|
||||||
|
#### `routing`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`colony`](sites/colony/colony.md) | `10.100.0.2/24 gw 10.100.0.1` | — | ams1.int.nul.ie | |
|
||||||
|
| [`git`](sites/colony/git.md) | `10.100.1.4/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
|
||||||
|
| [`shill`](sites/colony/shill/README.md) | `10.100.1.2/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
|
||||||
|
| [`whale2`](sites/colony/whale2.md) | `10.100.1.3/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
|
||||||
|
|
||||||
|
#### `tailscale`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`waffletail`](sites/colony/shill/containers/waffletail.md) | `100.64.0.5/32` | `fd7a:115c:a1e0::5/128` | — | |
|
||||||
|
|
||||||
|
#### `vms`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`colony`](sites/colony/colony.md) | `10.100.1.1/24` | `2a0e:97c0:4d2:11::1/64` | ams1.int.nul.ie | |
|
||||||
|
<!-- assignments-end -->
|
||||||
|
|
||||||
|
### home
|
||||||
|
|
||||||
|
<!-- assignments: home -->
|
||||||
|
<!-- assignments-start -->
|
||||||
|
#### `as211024`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`river`](sites/home/river.md) | `10.100.50.2/24` | `2a0e:97c0:4df:0:1::1/64 gw 2a0e:97c0:4df:0:2::1` | — | |
|
||||||
|
| [`stream`](sites/home/stream.md) | `10.100.50.3/24` | `2a0e:97c0:4df:0:1::2/64 gw 2a0e:97c0:4df:0:2::1` | — | |
|
||||||
|
|
||||||
|
#### `core`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`palace`](sites/home/palace.md) | `192.168.64.20/24` | — | h.nul.ie | |
|
||||||
|
| [`river`](sites/home/river.md) | `192.168.64.1/24` | — | h.nul.ie | |
|
||||||
|
| [`stream`](sites/home/stream.md) | `192.168.64.2/24` | — | h.nul.ie | |
|
||||||
|
| [`unifi`](sites/home/sfh/containers/unifi.md) | `192.168.64.21/24` | — | h.nul.ie | |
|
||||||
|
|
||||||
|
#### `hi`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`castle`](sites/home/castle.md) | `192.168.68.40/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::3:1/64` | h.nul.ie | |
|
||||||
|
| [`cellar`](sites/home/cellar.md) | `192.168.68.80/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:1/64` | h.nul.ie | |
|
||||||
|
| [`hass`](sites/home/sfh/containers/hass.md) | `192.168.68.103/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:3/64` | h.nul.ie | |
|
||||||
|
| [`palace`](sites/home/palace.md) | `192.168.68.22/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::2:1/64` | h.nul.ie | |
|
||||||
|
| [`river`](sites/home/river.md) | `192.168.68.1/22` | `2a0e:97c0:4d0:1::1/64` | h.nul.ie | |
|
||||||
|
| `router-hi` | `192.168.71.254/22 gw 192.168.68.1` | `2a0e:97c0:4d0:1::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
|
||||||
|
| [`sfh`](sites/home/sfh/README.md) | `192.168.68.81/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:2/64` | h.nul.ie | |
|
||||||
|
| [`stream`](sites/home/stream.md) | `192.168.68.2/22` | `2a0e:97c0:4d0:1::2/64` | h.nul.ie | |
|
||||||
|
| [`unifi`](sites/home/sfh/containers/unifi.md) | `192.168.68.100/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:1/64` | h.nul.ie | |
|
||||||
|
|
||||||
|
#### `lo`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`hass`](sites/home/sfh/containers/hass.md) | `192.168.72.103/21` | `2a0e:97c0:4d0:2::5:3/64` | h.nul.ie | |
|
||||||
|
| [`river`](sites/home/river.md) | `192.168.72.1/21` | `2a0e:97c0:4d0:2::1/64` | h.nul.ie | |
|
||||||
|
| `router-lo` | `192.168.79.254/21 gw 192.168.72.1` | `2a0e:97c0:4d0:2::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
|
||||||
|
| [`stream`](sites/home/stream.md) | `192.168.72.2/21` | `2a0e:97c0:4d0:2::2/64` | h.nul.ie | |
|
||||||
|
|
||||||
|
#### `untrusted`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`river`](sites/home/river.md) | `192.168.80.1/24` | `2a0e:97c0:4d0:3::1/64` | h.nul.ie | |
|
||||||
|
| `router-ut` | `192.168.80.254/24 gw 192.168.80.1` | `2a0e:97c0:4d0:3::ffff/64` | h.nul.ie | Floating VIP shared by [`river`](sites/home/river.md) and [`stream`](sites/home/stream.md) |
|
||||||
|
| [`stream`](sites/home/stream.md) | `192.168.80.2/24` | `2a0e:97c0:4d0:3::2/64` | h.nul.ie | |
|
||||||
|
<!-- assignments-end -->
|
||||||
|
|
||||||
|
### remote
|
||||||
|
|
||||||
|
<!-- assignments: remote -->
|
||||||
|
<!-- assignments-start -->
|
||||||
|
#### `internal`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`kelder-acquisition`](remote/kelder/containers/kelder-acquisition.md) | `172.16.64.2/24 gw 172.16.64.1` | — | hentai.engineer | |
|
||||||
|
| [`kelder-spoder`](remote/kelder/containers/kelder-spoder.md) | `172.16.64.3/24 gw 172.16.64.1` | — | hentai.engineer | |
|
||||||
|
|
||||||
|
#### `allhost`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`britnet`](remote/britnet.md) | `77.74.199.67/24 gw 77.74.199.1` | `2a12:ab46:5344:99::a/64 gw 2a12:ab46:5344::1` | bhx1.int.nul.ie | |
|
||||||
|
|
||||||
|
#### `as211024`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`britway`](remote/britway.md) | `10.100.50.5/24` | `2a0e:97c0:4df:0:2::1/64` | — | |
|
||||||
|
|
||||||
|
#### `ctrs`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`kelder`](remote/kelder/README.md) | `172.16.64.1/24` | — | hentai.engineer | |
|
||||||
|
|
||||||
|
#### `estuary`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`kelder`](remote/kelder/README.md) | `94.142.242.254/32` | — | — | |
|
||||||
|
|
||||||
|
#### `vpn`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`britnet`](remote/britnet.md) | `10.200.0.1/24` | `fdfb:5ebf:6e84::1/64` | — | |
|
||||||
|
|
||||||
|
#### `vultr`
|
||||||
|
|
||||||
|
| Box | IPv4 | IPv6 | Domain | Notes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| [`britway`](remote/britway.md) | `45.76.141.188/23 gw 45.76.140.1` | `2001:19f0:7402:128b::1/64` | lon1.int.nul.ie | |
|
||||||
|
<!-- assignments-end -->
|
||||||
|
|
||||||
## Domains
|
## Domains
|
||||||
|
|
||||||
The public domain is `nul.ie` (`lib.my.c.pubDomain`). Each site has its own internal domain
|
The public domain is `nul.ie` (`lib.my.c.pubDomain`). Each site has its own internal domain
|
||||||
@@ -118,36 +308,58 @@ which the other sites use as their next-hop into the home prefixes.
|
|||||||
|
|
||||||
### Router HA
|
### Router HA
|
||||||
|
|
||||||
- **VRRP via `keepalived`** ([`routing-common/keepalived.nix`](../nixos/boxes/home/routing-common/keepalived.nix)):
|
#### VRRP
|
||||||
separate v4/v6 instances (router IDs 51/52) on the `lan-core` link; index 0 (river) starts
|
|
||||||
as `MASTER`, priorities are `255 - index`. Track scripts ping public v4/v6 anycast targets
|
[`routing-common/keepalived.nix`](../nixos/boxes/home/routing-common/keepalived.nix) defines separate
|
||||||
and demote a router whose WAN is down. All VIPs of a family move together.
|
v4/v6 instances (router IDs 51/52) on `lan-core`. Index 0 (`river`) starts as `MASTER`, priorities
|
||||||
- **Clients get the VIP as gateway *and* DNS.** `kea` hands out `vips.<vlan>.v4` as both
|
are `255 - index`, and track scripts demote a router whose WAN checks fail. All VIPs of an address
|
||||||
`routers` and `domain-name-servers`, with the two routers serving disjoint pool halves;
|
family move together.
|
||||||
`radvd` advertises the v6 VIP as RDNSS (the `untrusted` VLAN gets Cloudflare instead) and
|
|
||||||
is started/stopped by keepalived's `notify_master`/`notify_backup` hooks so only the master
|
#### Client gateway and DNS
|
||||||
sends RAs.
|
|
||||||
- **`pdns-recursor` binds the VIPs directly** ([`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix)),
|
`kea` hands out `vips.<vlan>.v4` as both `routers` and `domain-name-servers`, with the two routers
|
||||||
with `net.ipv4.ip_nonlocal_bind` / `net.ipv6.ip_nonlocal_bind` so the backup can listen on
|
serving disjoint pool halves. `radvd` advertises the v6 VIP as RDNSS (`untrusted` gets Cloudflare)
|
||||||
addresses it doesn't currently hold — failover doesn't depend on client resolver timeouts.
|
and keepalived's `notify_master`/`notify_backup` hooks ensure that only the master sends RAs.
|
||||||
The recursor forwards the site's zones to the local authoritative PowerDNS on `127.0.0.1:5353`.
|
|
||||||
- **`wan-online.target`** is a shared, initially-inert systemd target meaning "the public
|
Statically-addressed boxes (the servers on `hi`) don't run DHCP, so they'd otherwise learn a
|
||||||
IPv4 WAN route is up". `routing-common` only declares it; each box wires how it's reached —
|
resolver only from the v6 RA RDNSS — which vanishes when v6 is disabled, taking DNS with it. They
|
||||||
stream gates it on a oneshot that waits for the DHCP default route on `wan`, river's `pppd`
|
instead anchor DNS on the VIPs via the shared `lib.my.c.home.vlanDns "<vlan>"` fragment, which sets
|
||||||
`ip-up`/`ip-down` hooks start/stop it. Consumers (e.g. `ipsec`, the RA-default-route
|
`DNS` to `vips.<vlan>.{v4,v6}` and `Domains` to the advertised search list; the always-present
|
||||||
cleanup) attach **to** it with `wantedBy` + `partOf` + `after`, never `requires`/`wants`,
|
static v4 VIP keeps resolution working even with v6 down.
|
||||||
so the target is never pulled in early and services re-load on WAN flap.
|
|
||||||
|
#### DNS binding
|
||||||
|
|
||||||
|
`pdns-recursor` binds the VIPs directly; see
|
||||||
|
[`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix). The
|
||||||
|
`net.ipv4.ip_nonlocal_bind` / `net.ipv6.ip_nonlocal_bind` settings let the backup listen before it
|
||||||
|
owns the addresses, so failover does not depend on client resolver timeouts. The recursor forwards
|
||||||
|
the site's zones to authoritative PowerDNS on `127.0.0.1:5353`. The generated
|
||||||
|
[DNS reference](reference/dns.md) lists the live forward and reverse records; the authoritative
|
||||||
|
servers allow its AXFRs from the shared internal prefixes and the colony site's egress address.
|
||||||
|
|
||||||
|
#### `wan-online.target`
|
||||||
|
|
||||||
|
This shared, initially inert systemd target means "the public IPv4 WAN route is up".
|
||||||
|
`routing-common` only declares it: `stream` gates it on a oneshot that waits for the DHCP default
|
||||||
|
route on `wan`, while `river`'s `pppd` hooks start and stop it. Consumers such as `ipsec` attach with
|
||||||
|
`wantedBy` + `partOf` + `after`, never `requires`/`wants`, so they cannot pull the target in early
|
||||||
|
and they reload on WAN flap.
|
||||||
|
|
||||||
### WAN paths (summary)
|
### WAN paths (summary)
|
||||||
|
|
||||||
- **river** (a VM on `palace`): PPPoE to Digiweb via `services.pppd`, running directly on the
|
#### `river`
|
||||||
ISP's VLAN 10 (`wan-pon-isp`, trunked untranslated through the switches; baby-jumbo MTU 1508
|
|
||||||
so the PPP session is a clean 1500). The static IP is requested in IPCP; the pppd hooks own
|
The VM on `palace` runs Digiweb PPPoE directly on VLAN 10 (`vlans.pon-isp`), trunked untranslated
|
||||||
`wan-online.target`. The ONT's management subnet (`192.168.100.0/24`) is reached on
|
through the switches. The carrying interface is named `wan-pon-isp`; VLAN ID 141 with that name is
|
||||||
`wan-pon-ont` (VLAN 140, PVID'd at the brian switch), where river takes `.100`.
|
reserved for a future translation scheme. MTU 1508 preserves a 1500-byte PPP session, IPCP requests
|
||||||
- **stream** (bare metal): DHCP on the Virgin Media cable modem (VLAN 130) on `wan`, with a
|
the static address, and the `pppd` hooks own `wan-online.target`. `wan-pon-ont` (VLAN 140) reaches
|
||||||
static modem-management address (`192.168.0.100/24`) alongside the public lease, and CAKE
|
the ONT management subnet at `192.168.100.0/24`, where `river` takes `.100`.
|
||||||
egress/ingress shaping via `wan-ifb`.
|
|
||||||
|
#### `stream`
|
||||||
|
|
||||||
|
The bare-metal backup uses DHCP from the Virgin Media modem on `wan` (VLAN 130), keeps a static
|
||||||
|
`192.168.0.100/24` management address beside the public lease, and applies CAKE shaping through
|
||||||
|
`wan-ifb`.
|
||||||
|
|
||||||
The full fabric story — which switch port carries what, why VLAN 10 is trunked untranslated,
|
The full fabric story — which switch port carries what, why VLAN 10 is trunked untranslated,
|
||||||
and the multi-ONT plan — is in [sites/home/switches.md](sites/home/switches.md); the
|
and the multi-ONT plan — is in [sites/home/switches.md](sites/home/switches.md); the
|
||||||
@@ -160,57 +372,64 @@ The edge routers are joined by a layer-2 mesh, defined once as `nixos.vpns.l2.as
|
|||||||
[`nixos/boxes/colony/vms/estuary/default.nix`](../nixos/boxes/colony/vms/estuary/default.nix)
|
[`nixos/boxes/colony/vms/estuary/default.nix`](../nixos/boxes/colony/vms/estuary/default.nix)
|
||||||
and realised on each member by the [`l2mesh` module](../nixos/modules/l2mesh.nix):
|
and realised on each member by the [`l2mesh` module](../nixos/modules/l2mesh.nix):
|
||||||
|
|
||||||
- **Members**: `estuary`, `river`, `stream`, `britway`, peering on their public addresses.
|
### Members
|
||||||
- **Transport**: VXLAN (VNI 211024, UDP port 4789) with static per-peer FDB entries and
|
|
||||||
UDP-encapsulated IPsec in transport mode via Libreswan (authentication-only by default;
|
`estuary`, `river`, `stream` and `britway` peer on their public addresses.
|
||||||
`security.encrypt` would switch ESP from `null-sha256` to AES-GCM). The PSK is the shared
|
|
||||||
`l2mesh/as211024.key` secret, expanded into `/run/l2mesh.secrets` at `ipsec` start.
|
### Transport
|
||||||
- **Overlay addressing**: `10.100.50.0/24` / `2a0e:97c0:4df::/64`; the interface MTU is
|
|
||||||
computed from the physical MTU minus VXLAN/UDP/IPsec overhead. Each router holds
|
VXLAN (VNI 211024, UDP port 4789) uses static per-peer FDB entries and UDP-encapsulated IPsec in
|
||||||
`10.100.50.<n>` (estuary `.1`, river `.2`, stream `.3`, britway `.5`).
|
Libreswan transport mode. It authenticates without encryption by default; `security.encrypt`
|
||||||
- **Routing over it**: the home routers route the colony prefixes via estuary and the
|
switches ESP from `null-sha256` to AES-GCM. The shared `l2mesh/as211024.key` PSK is expanded into
|
||||||
Tailscale prefixes via britway; estuary and britway route the home prefixes via the
|
`/run/l2mesh.secrets` when `ipsec` starts.
|
||||||
`10.100.50.4` VIP. The home IPv6 **default** route also runs over the mesh, via britway
|
|
||||||
(hence the recursor's IPv4-only upstream pinning noted in
|
### Overlay addressing
|
||||||
[`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix)). The `nftTrust`
|
|
||||||
snippet in `lib.my.c.as211024` lets trusted inter-site traffic (colony, home, mesh,
|
The overlay uses `10.100.50.0/24` / `2a0e:97c0:4df::/64`. Each router holds `10.100.50.<n>`:
|
||||||
Tailscale prefixes) through the edge firewalls.
|
`estuary` `.1`, `river` `.2`, `stream` `.3`, and `britway` `.5`. Interface MTU is calculated from
|
||||||
|
the physical MTU minus VXLAN/UDP/IPsec overhead.
|
||||||
|
|
||||||
|
### Routing
|
||||||
|
|
||||||
|
The home routers route colony prefixes via `estuary` and Tailscale prefixes via `britway`;
|
||||||
|
`estuary` and `britway` route home prefixes through the `10.100.50.4` VIP. The home IPv6 default
|
||||||
|
route also crosses the mesh through `britway`, which is why the recursor pins its upstreams to IPv4
|
||||||
|
as noted in [`routing-common/dns.nix`](../nixos/boxes/home/routing-common/dns.nix). The `nftTrust`
|
||||||
|
snippet in `lib.my.c.as211024` admits trusted colony, home, mesh and Tailscale prefixes at the edge
|
||||||
|
firewalls.
|
||||||
|
|
||||||
## BGP
|
## BGP
|
||||||
|
|
||||||
Both edge routers run `bird2` as AS211024:
|
Both edge routers run `bird2` as AS211024.
|
||||||
|
|
||||||
- **estuary** ([`bgp.nix`](../nixos/boxes/colony/vms/estuary/bgp.nix)): full table from
|
### `estuary`
|
||||||
ColoClue (AS8283) over two sessions per family, plus IPv6 transit from iFog and Hurricane
|
|
||||||
Electric; peering at the Frys-IX, NL-ix and FogIXP route servers and with various networks
|
The colony edge takes a full table from ColoClue, IPv6 transit from iFog and Hurricane Electric,
|
||||||
(Meta, Cloudflare, Apple, LUJE.net, …); a `bgp.tools` monitoring session. Static routes pull
|
and peers through the Frys-IX, NL-ix and FogIXP route servers. It also has direct and monitoring
|
||||||
the customer VIP blocks out of the `base` network and the internal/home IPv6 prefixes out of
|
sessions; the complete peer table and originated routes live in [estuary.md](sites/colony/estuary.md#bgp).
|
||||||
the mesh.
|
|
||||||
- **britway** ([`bgp.nix`](../nixos/boxes/britway/bgp.nix)): Vultr transit (AS64515,
|
### `britway`
|
||||||
MD5-passworded from a secret) and a `bgp.tools` session, originating the internal, colony
|
|
||||||
and home IPv6 prefixes.
|
The London edge uses secret-backed MD5 authentication for Vultr transit (AS64515), connects to
|
||||||
|
`bgp.tools`, and originates the internal, colony and home IPv6 prefixes. See
|
||||||
|
[britway.md](remote/britway.md) for its box-specific routing detail.
|
||||||
|
|
||||||
## WireGuard point-to-point tunnels
|
## WireGuard point-to-point tunnels
|
||||||
|
|
||||||
Separate from the mesh, estuary terminates several networkd-managed WireGuard tunnels (private
|
Separate from the mesh, a few boxes run their own WireGuard (private keys in per-box secrets):
|
||||||
keys in per-box secrets):
|
|
||||||
|
|
||||||
| Tunnel | Port | Remote / prefix | Notes |
|
- **estuary** terminates point-to-point tunnels to `kelder`, `hillcrest` and `john-valorant`,
|
||||||
|---|---|---|---|
|
addressed out of `p2pTunnels` — see
|
||||||
| estuary ↔ kelder | 51820 | kelder holds `94.142.242.254/32` | kelder's public presence is a colony /32 routed over the tunnel |
|
[estuary.md](sites/colony/estuary.md#wireguard-tunnels) for the per-tunnel ports and prefixes.
|
||||||
| estuary ↔ hillcrest | 51822 | `10.100.5.0/30` | point-to-point /30 out of `p2pTunnels` |
|
- **`qclk`** (container on `shill`) runs its own WireGuard on port 51821 out of `10.100.4.0/24`
|
||||||
| estuary ↔ john-valorant | 51823 | `10.100.5.4/30` | point-to-point /30 out of `p2pTunnels` |
|
— see [qclk.md](sites/colony/shill/containers/qclk.md).
|
||||||
|
- **`britnet`** hosts a road-warrior WireGuard VPN on port 51820 serving `10.200.0.0/24` /
|
||||||
Additionally, the `qclk` container on `shill` runs its own WireGuard instances on port 51821
|
`fdfb:5ebf:6e84::/64` — see [britnet.md](remote/britnet.md).
|
||||||
out of `10.100.4.0/24`, and `britnet` hosts a road-warrior style WireGuard VPN on port 51820
|
|
||||||
serving `10.200.0.0/24` / `fdfb:5ebf:6e84::/64`.
|
|
||||||
|
|
||||||
## Tailscale / headscale
|
## Tailscale / headscale
|
||||||
|
|
||||||
Tailscale runs against a self-hosted **headscale** control plane on britway
|
Tailscale runs against a self-hosted **headscale** control plane on britway at `https://hs.nul.ie`
|
||||||
([`tailscale.nix`](../nixos/boxes/britway/tailscale.nix)) at `https://hs.nul.ie`: Google OIDC
|
(OIDC, MagicDNS, split DNS — see [britway.md](remote/britway.md)). The tailnet prefixes are
|
||||||
login, SQLite state, MagicDNS under `ts.nul.ie`, and split DNS that resolves the colony and
|
|
||||||
home internal domains through their site resolvers. The tailnet prefixes are
|
|
||||||
`100.64.0.0/10` / `fd7a:115c:a1e0::/48`.
|
`100.64.0.0/10` / `fd7a:115c:a1e0::/48`.
|
||||||
|
|
||||||
Notable nodes:
|
Notable nodes:
|
||||||
|
|||||||
@@ -0,0 +1,319 @@
|
|||||||
|
# DNS records
|
||||||
|
|
||||||
|
The tables on this page are generated from live authoritative zone transfers by
|
||||||
|
`nix run .#update-docs-dns -- <zones...>`; CI keeps them current. The Nix DNS configuration is
|
||||||
|
the source of truth. Edit the prose and headings here, but not content between the DNS markers.
|
||||||
|
|
||||||
|
DHCP-managed records are excluded. The generator identifies them by `DHCID` and removes the
|
||||||
|
corresponding forward and reverse records. SOA records and TTLs are also omitted because they are
|
||||||
|
operational metadata rather than useful inventory.
|
||||||
|
|
||||||
|
## Colony
|
||||||
|
|
||||||
|
These zones are served by [`estuary`](../sites/colony/estuary.md); their source configuration is
|
||||||
|
[`estuary/dns.nix`](../../nixos/boxes/colony/vms/estuary/dns.nix).
|
||||||
|
|
||||||
|
### Forward zone: `ams1.int.nul.ie`
|
||||||
|
|
||||||
|
<!-- dns: ams1.int.nul.ie -->
|
||||||
|
<!-- dns-start -->
|
||||||
|
| Name | Type | Value |
|
||||||
|
|---|---|---|
|
||||||
|
| `@` | `ALIAS` | `estuary-vm.ams1.int.nul.ie.` |
|
||||||
|
| `@` | `NS` | `ns.ams1.int.nul.ie.` |
|
||||||
|
| `_acme-challenge` | `TXT (LUA)` | `generated at query time` |
|
||||||
|
| `andrey-cust` | `A` | `94.142.242.254` |
|
||||||
|
| `chatterbox-ctr` | `A` | `10.100.2.5` |
|
||||||
|
| `chatterbox-ctr` | `AAAA` | `2a0e:97c0:4d2:12::5` |
|
||||||
|
| `colony` | `A` | `94.142.241.224` |
|
||||||
|
| `colony` | `AAAA` | `2a0e:97c0:4d2:10::2` |
|
||||||
|
| `colony-psql` | `CNAME` | `colony-psql-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `colony-psql-ctr` | `A` | `10.100.2.4` |
|
||||||
|
| `colony-psql-ctr` | `AAAA` | `2a0e:97c0:4d2:12::4` |
|
||||||
|
| `colony-routing` | `A` | `10.100.0.2` |
|
||||||
|
| `colony-vms` | `A` | `10.100.1.1` |
|
||||||
|
| `colony-vms` | `AAAA` | `2a0e:97c0:4d2:11::1` |
|
||||||
|
| `ctr` | `CNAME` | `shill-vm.ams1.int.nul.ie.` |
|
||||||
|
| `darts-cust` | `A` | `94.142.242.255` |
|
||||||
|
| `darts-cust` | `AAAA` | `2a0e:97c0:4d2:2001::1` |
|
||||||
|
| `enshrouded` | `A` | `94.142.240.44` |
|
||||||
|
| `enshrouded-oci` | `A` | `10.100.3.5` |
|
||||||
|
| `enshrouded-oci` | `AAAA` | `2a0e:97c0:4d2:13::5` |
|
||||||
|
| `estuary-vm` | `A` | `94.142.240.44` |
|
||||||
|
| `estuary-vm` | `AAAA` | `2a02:898:0:20::329:1` |
|
||||||
|
| `estuary-vm-base` | `A` | `10.100.0.1` |
|
||||||
|
| `estuary-vm-base` | `AAAA` | `2a0e:97c0:4d2:10::1` |
|
||||||
|
| `fw` | `CNAME` | `estuary-vm.ams1.int.nul.ie.` |
|
||||||
|
| `gam-ctr` | `A` | `10.100.2.11` |
|
||||||
|
| `gam-ctr` | `AAAA` | `2a0e:97c0:4d2:12::b` |
|
||||||
|
| `git-vm` | `A` | `94.142.241.117` |
|
||||||
|
| `git-vm` | `AAAA` | `2a0e:97c0:4d2:11::4` |
|
||||||
|
| `git-vm-routing` | `A` | `10.100.1.4` |
|
||||||
|
| `graeme` | `A` | `94.142.240.44` |
|
||||||
|
| `graeme` | `AAAA` | `2a0e:97c0:4d2:13::8` |
|
||||||
|
| `graeme-oci` | `A` | `10.100.3.8` |
|
||||||
|
| `graeme-oci` | `AAAA` | `2a0e:97c0:4d2:13::8` |
|
||||||
|
| `hillcrest-tun` | `A` | `10.100.5.2` |
|
||||||
|
| `http` | `A` | `94.142.240.44` |
|
||||||
|
| `http` | `AAAA` | `2a0e:97c0:4d2:12::2` |
|
||||||
|
| `jackflix-ctr` | `A` | `10.100.2.6` |
|
||||||
|
| `jackflix-ctr` | `AAAA` | `2a0e:97c0:4d2:12::6` |
|
||||||
|
| `jam-cust` | `A` | `10.100.100.4` |
|
||||||
|
| `jam-cust` | `AAAA` | `2a0e:97c0:4d2:2002::1` |
|
||||||
|
| `jam-fwd` | `A` | `94.142.241.225` |
|
||||||
|
| `john-valorant-tun` | `A` | `10.100.5.6` |
|
||||||
|
| `kevcraft` | `A` | `94.142.240.44` |
|
||||||
|
| `kevcraft` | `AAAA` | `2a0e:97c0:4d2:13::6` |
|
||||||
|
| `kevcraft-oci` | `A` | `10.100.3.6` |
|
||||||
|
| `kevcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::6` |
|
||||||
|
| `kinkcraft` | `A` | `94.142.240.44` |
|
||||||
|
| `kinkcraft` | `AAAA` | `2a0e:97c0:4d2:13::7` |
|
||||||
|
| `kinkcraft-oci` | `A` | `10.100.3.7` |
|
||||||
|
| `kinkcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::7` |
|
||||||
|
| `librespeed` | `CNAME` | `http.ams1.int.nul.ie.` |
|
||||||
|
| `mail-vm` | `A` | `94.142.241.227` |
|
||||||
|
| `mail-vm` | `AAAA` | `2a0e:97c0:4d2:2000::1` |
|
||||||
|
| `middleman-ctr` | `A` | `10.100.2.2` |
|
||||||
|
| `middleman-ctr` | `AAAA` | `2a0e:97c0:4d2:12::2` |
|
||||||
|
| `ns` | `ALIAS` | `estuary-vm.ams1.int.nul.ie.` |
|
||||||
|
| `object-ctr` | `A` | `10.100.2.7` |
|
||||||
|
| `object-ctr` | `AAAA` | `2a0e:97c0:4d2:12::7` |
|
||||||
|
| `oci` | `CNAME` | `whale-vm.ams1.int.nul.ie.` |
|
||||||
|
| `qclk-ctr` | `A` | `10.100.2.10` |
|
||||||
|
| `qclk-ctr` | `AAAA` | `2a0e:97c0:4d2:12::a` |
|
||||||
|
| `shill-vm` | `A` | `94.142.241.225` |
|
||||||
|
| `shill-vm` | `AAAA` | `2a0e:97c0:4d2:11::2` |
|
||||||
|
| `shill-vm-ctrs` | `A` | `10.100.2.1` |
|
||||||
|
| `shill-vm-ctrs` | `AAAA` | `2a0e:97c0:4d2:12::1` |
|
||||||
|
| `shill-vm-routing` | `A` | `10.100.1.2` |
|
||||||
|
| `simpcraft` | `A` | `94.142.240.44` |
|
||||||
|
| `simpcraft` | `AAAA` | `2a0e:97c0:4d2:13::3` |
|
||||||
|
| `simpcraft-oci` | `A` | `10.100.3.3` |
|
||||||
|
| `simpcraft-oci` | `AAAA` | `2a0e:97c0:4d2:13::3` |
|
||||||
|
| `simpcraft-staging` | `A` | `94.142.240.44` |
|
||||||
|
| `simpcraft-staging` | `AAAA` | `2a0e:97c0:4d2:13::4` |
|
||||||
|
| `simpcraft-staging-oci` | `A` | `10.100.3.4` |
|
||||||
|
| `simpcraft-staging-oci` | `AAAA` | `2a0e:97c0:4d2:13::4` |
|
||||||
|
| `terraria` | `A` | `94.142.240.44` |
|
||||||
|
| `terraria` | `AAAA` | `2a0e:97c0:4d2:12::b` |
|
||||||
|
| `toot-ctr` | `A` | `10.100.2.8` |
|
||||||
|
| `toot-ctr` | `AAAA` | `2a0e:97c0:4d2:12::8` |
|
||||||
|
| `valheim` | `A` | `94.142.240.44` |
|
||||||
|
| `valheim` | `AAAA` | `2a0e:97c0:4d2:13::2` |
|
||||||
|
| `valheim-oci` | `A` | `10.100.3.2` |
|
||||||
|
| `valheim-oci` | `AAAA` | `2a0e:97c0:4d2:13::2` |
|
||||||
|
| `vaultwarden-ctr` | `A` | `10.100.2.3` |
|
||||||
|
| `vaultwarden-ctr` | `AAAA` | `2a0e:97c0:4d2:12::3` |
|
||||||
|
| `vm` | `CNAME` | `colony.ams1.int.nul.ie.` |
|
||||||
|
| `waffletail-ctr` | `A` | `10.100.2.9` |
|
||||||
|
| `waffletail-ctr` | `AAAA` | `2a0e:97c0:4d2:12::9` |
|
||||||
|
| `whale-vm` | `A` | `94.142.241.226` |
|
||||||
|
| `whale-vm` | `AAAA` | `2a0e:97c0:4d2:11::3` |
|
||||||
|
| `whale-vm-routing` | `A` | `10.100.1.3` |
|
||||||
|
<!-- dns-end -->
|
||||||
|
|
||||||
|
### IPv4 reverse zone: `100.10.in-addr.arpa`
|
||||||
|
|
||||||
|
<!-- dns: 100.10.in-addr.arpa -->
|
||||||
|
<!-- dns-start -->
|
||||||
|
| Address | Name |
|
||||||
|
|---|---|
|
||||||
|
| `10.100.0.1` | `estuary-vm-base.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.0.2` | `colony-routing.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.1.1` | `colony-vms.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.1.2` | `shill-vm-routing.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.1.3` | `whale-vm-routing.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.1.4` | `git-vm-routing.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.1` | `shill-vm-ctrs.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.2` | `middleman-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.3` | `vaultwarden-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.4` | `colony-psql-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.5` | `chatterbox-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.6` | `jackflix-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.7` | `object-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.8` | `toot-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.9` | `waffletail-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.10` | `qclk-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.2.11` | `gam-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.3.2` | `valheim-oci.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.3.3` | `simpcraft-oci.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.3.4` | `simpcraft-staging-oci.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.3.5` | `enshrouded-oci.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.3.6` | `kevcraft-oci.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.3.7` | `kinkcraft-oci.ams1.int.nul.ie.` |
|
||||||
|
| `10.100.3.8` | `graeme-oci.ams1.int.nul.ie.` |
|
||||||
|
<!-- dns-end -->
|
||||||
|
|
||||||
|
### IPv6 reverse zone: `2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa`
|
||||||
|
|
||||||
|
<!-- dns: 2.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa -->
|
||||||
|
<!-- dns-start -->
|
||||||
|
| Address | Name |
|
||||||
|
|---|---|
|
||||||
|
| `2a0e:97c0:4d2:10::1` | `estuary-vm-base.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:10::2` | `colony.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:11::1` | `colony-vms.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:11::2` | `shill-vm.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:11::3` | `whale-vm.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:11::4` | `git-vm.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::1` | `shill-vm-ctrs.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::2` | `middleman-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::3` | `vaultwarden-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::4` | `colony-psql-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::5` | `chatterbox-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::6` | `jackflix-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::7` | `object-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::8` | `toot-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::9` | `waffletail-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::a` | `qclk-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:12::b` | `gam-ctr.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:13::2` | `valheim-oci.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:13::3` | `simpcraft-oci.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:13::4` | `simpcraft-staging-oci.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:13::5` | `enshrouded-oci.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:13::6` | `kevcraft-oci.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:13::7` | `kinkcraft-oci.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:13::8` | `graeme-oci.ams1.int.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:2000::1` | `mail.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d2:2001::1` | `darts-cust.ams1.int.nul.ie.` |
|
||||||
|
<!-- dns-end -->
|
||||||
|
|
||||||
|
## Home
|
||||||
|
|
||||||
|
These zones are served by [`river`](../sites/home/river.md) and
|
||||||
|
[`stream`](../sites/home/stream.md); their shared source configuration is
|
||||||
|
[`routing-common/dns.nix`](../../nixos/boxes/home/routing-common/dns.nix).
|
||||||
|
|
||||||
|
### Forward zone: `h.nul.ie`
|
||||||
|
|
||||||
|
<!-- dns: h.nul.ie -->
|
||||||
|
<!-- dns-start -->
|
||||||
|
| Name | Type | Value |
|
||||||
|
|---|---|---|
|
||||||
|
| `@` | `NS` | `ns1.h.nul.ie.` |
|
||||||
|
| `@` | `NS` | `ns2.h.nul.ie.` |
|
||||||
|
| `boot` | `CNAME` | `river-hi.h.nul.ie.` |
|
||||||
|
| `brian` | `A` | `192.168.64.13` |
|
||||||
|
| `castle` | `A` | `192.168.68.40` |
|
||||||
|
| `castle` | `AAAA` | `2a0e:97c0:4d0:1::3:1` |
|
||||||
|
| `cellar` | `A` | `192.168.68.80` |
|
||||||
|
| `cellar` | `AAAA` | `2a0e:97c0:4d0:1::4:1` |
|
||||||
|
| `dave` | `A` | `192.168.68.11` |
|
||||||
|
| `dave` | `AAAA` | `2a0e:97c0:4d0:1::1:2` |
|
||||||
|
| `dave-core` | `A` | `192.168.64.11` |
|
||||||
|
| `dave-lo` | `A` | `192.168.72.11` |
|
||||||
|
| `dave-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:2` |
|
||||||
|
| `dyn` | `NS` | `ns1.dyn.h.nul.ie.` |
|
||||||
|
| `dyn` | `NS` | `ns2.dyn.h.nul.ie.` |
|
||||||
|
| `frigate` | `CNAME` | `hass-ctr.h.nul.ie.` |
|
||||||
|
| `hass-ctr` | `A` | `192.168.68.103` |
|
||||||
|
| `hass-ctr` | `AAAA` | `2a0e:97c0:4d0:1::5:3` |
|
||||||
|
| `hass-ctr-lo` | `A` | `192.168.72.103` |
|
||||||
|
| `hass-ctr-lo` | `AAAA` | `2a0e:97c0:4d0:2::5:3` |
|
||||||
|
| `jim` | `A` | `192.168.68.10` |
|
||||||
|
| `jim` | `AAAA` | `2a0e:97c0:4d0:1::1:1` |
|
||||||
|
| `jim-core` | `A` | `192.168.64.10` |
|
||||||
|
| `jim-lo` | `A` | `192.168.72.10` |
|
||||||
|
| `jim-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:1` |
|
||||||
|
| `nixlight` | `A` | `192.168.72.46` |
|
||||||
|
| `ns1` | `ALIAS` | `river.h.nul.ie.` |
|
||||||
|
| `ns1.dyn` | `ALIAS` | `river.h.nul.ie.` |
|
||||||
|
| `ns2` | `ALIAS` | `stream.h.nul.ie.` |
|
||||||
|
| `ns2.dyn` | `ALIAS` | `stream.h.nul.ie.` |
|
||||||
|
| `palace` | `A` | `192.168.68.22` |
|
||||||
|
| `palace` | `AAAA` | `2a0e:97c0:4d0:1::2:1` |
|
||||||
|
| `palace-core` | `A` | `192.168.64.20` |
|
||||||
|
| `palace-kvm` | `A` | `192.168.72.21` |
|
||||||
|
| `reolink-living-room` | `A` | `192.168.72.45` |
|
||||||
|
| `river` | `A (LUA)` | `generated at query time` |
|
||||||
|
| `river` | `AAAA` | `2a0e:97c0:4df:0:1::1` |
|
||||||
|
| `river-core` | `A` | `192.168.64.1` |
|
||||||
|
| `river-hi` | `A` | `192.168.68.1` |
|
||||||
|
| `river-hi` | `AAAA` | `2a0e:97c0:4d0:1::1` |
|
||||||
|
| `river-lo` | `A` | `192.168.72.1` |
|
||||||
|
| `river-lo` | `AAAA` | `2a0e:97c0:4d0:2::1` |
|
||||||
|
| `river-ut` | `A` | `192.168.80.1` |
|
||||||
|
| `river-ut` | `AAAA` | `2a0e:97c0:4d0:3::1` |
|
||||||
|
| `router-hi` | `A` | `192.168.71.254` |
|
||||||
|
| `router-hi` | `AAAA` | `2a0e:97c0:4d0:1::ffff` |
|
||||||
|
| `router-lo` | `A` | `192.168.79.254` |
|
||||||
|
| `router-lo` | `AAAA` | `2a0e:97c0:4d0:2::ffff` |
|
||||||
|
| `router-ut` | `A` | `192.168.80.254` |
|
||||||
|
| `router-ut` | `AAAA` | `2a0e:97c0:4d0:3::ffff` |
|
||||||
|
| `sfh` | `A` | `192.168.68.81` |
|
||||||
|
| `sfh` | `AAAA` | `2a0e:97c0:4d0:1::4:2` |
|
||||||
|
| `shytzel` | `A` | `192.168.64.12` |
|
||||||
|
| `stream` | `A (LUA)` | `generated at query time` |
|
||||||
|
| `stream` | `AAAA` | `2a0e:97c0:4df:0:1::2` |
|
||||||
|
| `stream-core` | `A` | `192.168.64.2` |
|
||||||
|
| `stream-hi` | `A` | `192.168.68.2` |
|
||||||
|
| `stream-hi` | `AAAA` | `2a0e:97c0:4d0:1::2` |
|
||||||
|
| `stream-lo` | `A` | `192.168.72.2` |
|
||||||
|
| `stream-lo` | `AAAA` | `2a0e:97c0:4d0:2::2` |
|
||||||
|
| `stream-ut` | `A` | `192.168.80.2` |
|
||||||
|
| `stream-ut` | `AAAA` | `2a0e:97c0:4d0:3::2` |
|
||||||
|
| `unifi-ctr` | `A` | `192.168.68.100` |
|
||||||
|
| `unifi-ctr` | `AAAA` | `2a0e:97c0:4d0:1::5:1` |
|
||||||
|
| `unifi-ctr-core` | `A` | `192.168.64.21` |
|
||||||
|
| `ups` | `A` | `192.168.72.20` |
|
||||||
|
| `vibe` | `A` | `192.168.68.15` |
|
||||||
|
| `vibe` | `AAAA` | `2a0e:97c0:4d0:1::1:6` |
|
||||||
|
| `vibe-core` | `A` | `192.168.64.15` |
|
||||||
|
| `vibe-lo` | `A` | `192.168.72.15` |
|
||||||
|
| `vibe-lo` | `AAAA` | `2a0e:97c0:4d0:2::1:6` |
|
||||||
|
| `wave` | `A` | `192.168.72.14` |
|
||||||
|
| `wave` | `AAAA` | `2a0e:97c0:4d0:2::1:5` |
|
||||||
|
| `wave-core` | `A` | `192.168.64.14` |
|
||||||
|
<!-- dns-end -->
|
||||||
|
|
||||||
|
### IPv4 reverse zone: `168.192.in-addr.arpa`
|
||||||
|
|
||||||
|
<!-- dns: 168.192.in-addr.arpa -->
|
||||||
|
<!-- dns-start -->
|
||||||
|
| Address | Name |
|
||||||
|
|---|---|
|
||||||
|
| `192.168.64.1` | `river-core.h.nul.ie.` |
|
||||||
|
| `192.168.64.2` | `stream-core.h.nul.ie.` |
|
||||||
|
| `192.168.64.20` | `palace-core.h.nul.ie.` |
|
||||||
|
| `192.168.64.21` | `unifi-ctr-core.h.nul.ie.` |
|
||||||
|
| `192.168.68.1` | `river-hi.h.nul.ie.` |
|
||||||
|
| `192.168.68.2` | `stream-hi.h.nul.ie.` |
|
||||||
|
| `192.168.68.22` | `palace.h.nul.ie.` |
|
||||||
|
| `192.168.68.40` | `castle.h.nul.ie.` |
|
||||||
|
| `192.168.68.80` | `cellar.h.nul.ie.` |
|
||||||
|
| `192.168.68.81` | `sfh.h.nul.ie.` |
|
||||||
|
| `192.168.68.100` | `unifi-ctr.h.nul.ie.` |
|
||||||
|
| `192.168.68.103` | `hass-ctr.h.nul.ie.` |
|
||||||
|
| `192.168.71.254` | `router-hi.h.nul.ie.` |
|
||||||
|
| `192.168.72.1` | `river-lo.h.nul.ie.` |
|
||||||
|
| `192.168.72.2` | `stream-lo.h.nul.ie.` |
|
||||||
|
| `192.168.72.103` | `hass-ctr-lo.h.nul.ie.` |
|
||||||
|
| `192.168.79.254` | `router-lo.h.nul.ie.` |
|
||||||
|
| `192.168.80.1` | `river-ut.h.nul.ie.` |
|
||||||
|
| `192.168.80.2` | `stream-ut.h.nul.ie.` |
|
||||||
|
| `192.168.80.254` | `router-ut.h.nul.ie.` |
|
||||||
|
<!-- dns-end -->
|
||||||
|
|
||||||
|
### IPv6 reverse zone: `0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa`
|
||||||
|
|
||||||
|
<!-- dns: 0.d.4.0.0.c.7.9.e.0.a.2.ip6.arpa -->
|
||||||
|
<!-- dns-start -->
|
||||||
|
| Address | Name |
|
||||||
|
|---|---|
|
||||||
|
| `2a0e:97c0:4d0:1::1` | `river-hi.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:1::2` | `stream-hi.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:1::ffff` | `router-hi.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:1::2:1` | `palace.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:1::3:1` | `castle.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:1::4:1` | `cellar.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:1::4:2` | `sfh.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:1::5:1` | `unifi-ctr.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:1::5:3` | `hass-ctr.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:2::1` | `river-lo.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:2::2` | `stream-lo.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:2::ffff` | `router-lo.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:2::5:3` | `hass-ctr-lo.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:3::1` | `river-ut.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:3::2` | `stream-ut.h.nul.ie.` |
|
||||||
|
| `2a0e:97c0:4d0:3::ffff` | `router-ut.h.nul.ie.` |
|
||||||
|
<!-- dns-end -->
|
||||||
@@ -0,0 +1,286 @@
|
|||||||
|
# NixOS module options (`my.*`)
|
||||||
|
|
||||||
|
> **Generated** from the module option descriptions by `nix run .#update-docs-options`;
|
||||||
|
> CI keeps it current. The source of truth is the `mkOpt'` / `mkBoolOpt'` declarations in
|
||||||
|
> `nixos/modules/` — edit those, not this file. For what each module is *for*, see the
|
||||||
|
> [shared-modules overview](../architecture.md#shared-modules).
|
||||||
|
|
||||||
|
## `borgthin` — [`nixos/modules/borgthin.nix`](../../nixos/modules/borgthin.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.borgthin.enable` | boolean | `false` | Whether to enable borgthin jobs |
|
||||||
|
| `my.borgthin.jobs` | attribute set of (submodule) | `{ }` | borgthin jobs |
|
||||||
|
| `my.borgthin.jobs.<name>.archivePrefix` | string | `"${config.networking.hostName}-${name}-"` | Prefix to start new archives with |
|
||||||
|
| `my.borgthin.jobs.<name>.compression` | string | `"zstd,3"` | Compression options |
|
||||||
|
| `my.borgthin.jobs.<name>.dateFormat` | string | `"+%Y-%m-%dT%H:%M:%S"` | Format passed to the date command |
|
||||||
|
| `my.borgthin.jobs.<name>.environment` | attribute set of string | `{ }` | Extra environment variables to pass to borg |
|
||||||
|
| `my.borgthin.jobs.<name>.extraArgs` | list of string | `[ "--iec" ]` | Extra args to pass to all borg commands |
|
||||||
|
| `my.borgthin.jobs.<name>.extraCreateArgs` | list of string | `[ ]` | Extra args to pass to tcreate command |
|
||||||
|
| `my.borgthin.jobs.<name>.lvs` | list of string | `null` | Thin LVs to backup (vg/lv format) |
|
||||||
|
| `my.borgthin.jobs.<name>.passFile` | null or string | `null` | Path to file containing passphrase |
|
||||||
|
| `my.borgthin.jobs.<name>.prune.keep` | attribute set of (signed integer or string) | `{ }` | Borg pruning params |
|
||||||
|
| `my.borgthin.jobs.<name>.prune.pattern` | string | `"sh:${config.archivePrefix}*"` | Borg pattern to select archives for pruning |
|
||||||
|
| `my.borgthin.jobs.<name>.repo` | string | `null` | borg repository URL |
|
||||||
|
| `my.borgthin.jobs.<name>.timer.at` | string or list of string | `"5:00"` | systemd calendar time(s) to run backup at |
|
||||||
|
| `my.borgthin.jobs.<name>.timer.persistent` | boolean | `false` | Persistent systemd timer |
|
||||||
|
| `my.borgthin.lvmPackage` | package | `<derivation lvm2-2.03.39>` | Packge containing LVM tools |
|
||||||
|
| `my.borgthin.package` | package | `inputs.borgthin.packages.${system}.borgthin` | borgthin package |
|
||||||
|
| `my.borgthin.thinToolsPackage` | package | `<derivation thin-provisioning-tools-1.3.2>` | Package containing thin-provisioning-tools |
|
||||||
|
|
||||||
|
## `build` — [`nixos/modules/build.nix`](../../nixos/modules/build.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.build.allHardware` | boolean | `false` | Whether to enable a lot of firmware and kernel modules for a wide range of hardware.Only applies to some build targets. |
|
||||||
|
| `my.build.isDevVM` | boolean | `false` | Whether the system is a development VM. |
|
||||||
|
| `my.buildAs` | open submodule of lazy attribute set of unspecified value | `{ }` | Attribute set of derivations used to set up the system. |
|
||||||
|
|
||||||
|
## `common` — [`nixos/modules/common.nix`](../../nixos/modules/common.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.tmproot.persistence.config.users` | attribute set of (submodule) | `{ }` | A set of user submodules listing the files and directories to link to their respective user's home directories. Each attribute name should be the name of the user. For detailed usage, check the [documentation](https://github.com/nix-community/impermanence). |
|
||||||
|
|
||||||
|
## `containers` — [`nixos/modules/containers.nix`](../../nixos/modules/containers.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.containers.instances` | attribute set of (submodule) | `{ }` | Individual containers. |
|
||||||
|
| `my.containers.instances.<name>.autoStart` | boolean | `true` | Whether to start the container automatically at boot. |
|
||||||
|
| `my.containers.instances.<name>.bindMounts` | attribute set of (submodule) | `{ }` | An extra list of directories that is bound to the container. |
|
||||||
|
| `my.containers.instances.<name>.bindMounts.<name>.hostPath` | null or string | `"‹name›"` | Location of the host path to be mounted. |
|
||||||
|
| `my.containers.instances.<name>.bindMounts.<name>.mountPoint` | string | `"‹name›"` | Mount point on the container file system. |
|
||||||
|
| `my.containers.instances.<name>.bindMounts.<name>.readOnly` | boolean | `true` | Determine whether the mounted path will be accessed in read-only mode. |
|
||||||
|
| `my.containers.instances.<name>.containerSystem` | absolute path | `"/nix/var/nix/profiles/system"` | Path to NixOS system configuration from within container. |
|
||||||
|
| `my.containers.instances.<name>.hotReload` | boolean | `true` | Whether to apply new configuration by running `switch-to-configuration` instead of rebooting the container. |
|
||||||
|
| `my.containers.instances.<name>.networking.bridge` | null or string | `null` | Network bridge to connect to. |
|
||||||
|
| `my.containers.instances.<name>.networking.macVLAN` | null or string | `null` | Network interface to make MACVLAN interface from. |
|
||||||
|
| `my.containers.instances.<name>.system` | absolute path | `"/nix/var/nix/profiles/per-container/‹name›/system"` | Path to NixOS system configuration. |
|
||||||
|
| `my.containers.persistDir` | string | `"/persist/containers"` | Where to store container persistence data. |
|
||||||
|
|
||||||
|
## `deploy-rs` — [`nixos/modules/deploy-rs.nix`](../../nixos/modules/deploy-rs.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.deploy.authorizedKeys.keyFiles` | list of absolute path | `[ .keys/deploy.pub ]` | SSH public key files to add to the default deployment user. |
|
||||||
|
| `my.deploy.authorizedKeys.keys` | list of (optionally newline-terminated) single-line string | `[ ]` | SSH public keys to add to the default deployment user. |
|
||||||
|
| `my.deploy.enable` | boolean | `true` | Whether to expose deploy-rs configuration for this system. |
|
||||||
|
| `my.deploy.generate.containers.enable` | boolean | `true` | Whether to generate deploy-rs profiles for this system's containers. |
|
||||||
|
| `my.deploy.generate.containers.keepGenerations` | unsigned integer, meaning >=0 | `10` | Number of generations to keep when cleaning up old deployments (0 to disable deletion on deployment). |
|
||||||
|
| `my.deploy.generate.system.enable` | boolean | `true` | Whether to generate a deploy-rs profile for this system's config. |
|
||||||
|
| `my.deploy.generate.system.keepGenerations` | unsigned integer, meaning >=0 | `10` | Number of generations to keep when cleaning up old deployments (0 to disable deletion on deployment). |
|
||||||
|
| `my.deploy.generate.system.mode` | string | `"switch"` | switch-to-configuration mode. |
|
||||||
|
| `my.deploy.node` | submodule | `{ }` | deploy-rs node configuration. |
|
||||||
|
| `my.deploy.node.autoRollback` | null or boolean | `null` | Whether to roll back the profile if activation fails. |
|
||||||
|
| `my.deploy.node.confirmTimeout` | null or 16 bit unsigned integer; between 0 and 65535 (both inclusive) | `null` | Timeout for confirming activation succeeded. |
|
||||||
|
| `my.deploy.node.fastConnection` | null or boolean | `null` | Whether to copy the whole closure instead of using substitution. |
|
||||||
|
| `my.deploy.node.hostname` | string | `""` | Hostname deploy-rs will connect to. |
|
||||||
|
| `my.deploy.node.magicRollback` | null or boolean | `null` | Whether to roll back the profile if connectivity to the deployer is lost. |
|
||||||
|
| `my.deploy.node.profiles` | attribute set of (submodule) | `{ }` | Profiles to deploy. |
|
||||||
|
| `my.deploy.node.profiles.<name>.autoRollback` | null or boolean | `null` | Whether to roll back the profile if activation fails. |
|
||||||
|
| `my.deploy.node.profiles.<name>.confirmTimeout` | null or 16 bit unsigned integer; between 0 and 65535 (both inclusive) | `null` | Timeout for confirming activation succeeded. |
|
||||||
|
| `my.deploy.node.profiles.<name>.fastConnection` | null or boolean | `null` | Whether to copy the whole closure instead of using substitution. |
|
||||||
|
| `my.deploy.node.profiles.<name>.magicRollback` | null or boolean | `null` | Whether to roll back the profile if connectivity to the deployer is lost. |
|
||||||
|
| `my.deploy.node.profiles.<name>.path` | package | `""` | Derivation to build (should include activation script). |
|
||||||
|
| `my.deploy.node.profiles.<name>.profilePath` | null or string | `null` | Path to profile location |
|
||||||
|
| `my.deploy.node.profiles.<name>.sshOpts` | list of string | `[ ]` | Options deploy-rs will pass to ssh. Note: overriding at a lower level _merges_ options. |
|
||||||
|
| `my.deploy.node.profiles.<name>.sshUser` | null or string | `null` | Username deploy-rs will deploy with. |
|
||||||
|
| `my.deploy.node.profiles.<name>.sudo` | null or string | `null` | Command to elevate privileges with (used if the deployment user != profile user). |
|
||||||
|
| `my.deploy.node.profiles.<name>.tempPath` | null or string | `null` | Path that deploy-rs will use for temporary files. |
|
||||||
|
| `my.deploy.node.profiles.<name>.user` | null or string | `null` | Username deploy-rs will deploy with. |
|
||||||
|
| `my.deploy.node.profilesOrder` | null or (list of string) | `null` | Order to deploy profiles in (remainder will be deployed in arbitrary order). |
|
||||||
|
| `my.deploy.node.sshOpts` | list of string | `[ ]` | Options deploy-rs will pass to ssh. Note: overriding at a lower level _merges_ options. |
|
||||||
|
| `my.deploy.node.sshUser` | null or string | `null` | Username deploy-rs will deploy with. |
|
||||||
|
| `my.deploy.node.sudo` | null or string | `null` | Command to elevate privileges with (used if the deployment user != profile user). |
|
||||||
|
| `my.deploy.node.tempPath` | null or string | `null` | Path that deploy-rs will use for temporary files. |
|
||||||
|
| `my.deploy.node.user` | null or string | `null` | Username deploy-rs will deploy with. |
|
||||||
|
|
||||||
|
## `dynamic-motd` — [`nixos/modules/dynamic-motd.nix`](../../nixos/modules/dynamic-motd.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.dynamic-motd.enable` | boolean | `true` | Whether to enable the dynamic message of the day PAM module. |
|
||||||
|
| `my.dynamic-motd.script` | null or strings concatenated with "\n" | `null` | Script that generates message of the day. |
|
||||||
|
| `my.dynamic-motd.services` | list of string | `[ "login" "sshd" ]` | PAM services to enable the dynamic message of the day module for. |
|
||||||
|
|
||||||
|
## `firewall` — [`nixos/modules/firewall.nix`](../../nixos/modules/firewall.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.firewall.enable` | boolean | `true` | Whether to enable the nftables-based firewall. |
|
||||||
|
| `my.firewall.extraRules` | strings concatenated with "\n" | `""` | Arbitrary additional nftables rules. |
|
||||||
|
| `my.firewall.nat.enable` | boolean | `true` | Whether to enable IP forwarding and NAT. |
|
||||||
|
| `my.firewall.nat.externalInterface` | null or string | `null` | The name of the external network interface. |
|
||||||
|
| `my.firewall.nat.forwardPorts` | (list of (submodule)) or attribute set of list of (submodule) | `[ ]` | IPv4 port forwards |
|
||||||
|
| `my.firewall.tcp.allowed` | list of (16 bit unsigned integer; between 0 and 65535 (both inclusive) or string) | `[ ]` | TCP ports to open. |
|
||||||
|
| `my.firewall.trustedInterfaces` | list of string | `[ ]` | Traffic coming in from these interfaces will be accepted unconditionally. Traffic from the loopback (lo) interface will always be accepted. |
|
||||||
|
| `my.firewall.udp.allowTraceroute` | boolean | `true` | Whethor or not to add a rule to accept UDP traceroute packets. |
|
||||||
|
| `my.firewall.udp.allowed` | list of (16 bit unsigned integer; between 0 and 65535 (both inclusive) or string) | `[ ]` | UDP ports to open. |
|
||||||
|
|
||||||
|
## `gui` — [`nixos/modules/gui`](../../nixos/modules/gui)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.gui.enable` | boolean | `true` | Whether to enable GUI system options. |
|
||||||
|
|
||||||
|
## `l2mesh` — [`nixos/modules/l2mesh.nix`](../../nixos/modules/l2mesh.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.vpns.l2.pskFiles` | attribute set of string | `{ }` | PSK files for secured meshes. |
|
||||||
|
|
||||||
|
## `librespeed` — [`nixos/modules/librespeed`](../../nixos/modules/librespeed)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.librespeed.backend.enable` | boolean | `false` | Whether to enable librespeed backend. |
|
||||||
|
| `my.librespeed.backend.extraSettingsFile` | null or string | `null` | Extra settings file. |
|
||||||
|
| `my.librespeed.backend.settings` | attribute set of unspecified value | `{ }` | Backend settings. |
|
||||||
|
| `my.librespeed.frontend.servers` | list of attribute set of unspecified value | `{ }` | Server configs. |
|
||||||
|
| `my.librespeed.frontend.webroot` | package | — | Frontend webroot. |
|
||||||
|
|
||||||
|
## `netboot` — [`nixos/modules/netboot`](../../nixos/modules/netboot)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.netboot.client.configurationLimit` | unsigned integer, meaning >=0 | `10` | Max generations to show in boot menu. |
|
||||||
|
| `my.netboot.client.enable` | boolean | `false` | Whether network booting should be enabled. |
|
||||||
|
| `my.netboot.server.allowedPrefixes` | list of string | `null` | Prefixes clients should be allowed to connect from (NFS). |
|
||||||
|
| `my.netboot.server.enable` | boolean | `false` | Whether a netboot server should be enabled. |
|
||||||
|
| `my.netboot.server.host` | string | `config.networking.fqdn` | Hostname clients should connect to over HTTP / NFS. |
|
||||||
|
| `my.netboot.server.installer.storeSize` | string | `"16GiB"` | Total allowed writable size of store. |
|
||||||
|
| `my.netboot.server.instances` | list of string | `[ ]` | Systems to hold boot files for. |
|
||||||
|
| `my.netboot.server.ip` | string | `null` | IP clients should connect to via TFTP. |
|
||||||
|
|
||||||
|
## `nginx-sso` — [`nixos/modules/nginx-sso.nix`](../../nixos/modules/nginx-sso.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.nginx-sso.configuration` | attribute set of unspecified value | `{ }` | nginx-sso configuration. |
|
||||||
|
| `my.nginx-sso.enable` | boolean | `false` | Whether to enable custom nginx-sso. |
|
||||||
|
| `my.nginx-sso.extraConfigFile` | null or string | `null` | Path to configuration (e.g. for secrets). |
|
||||||
|
| `my.nginx-sso.includes.baseURL` | string | `null` | Base URL for redirects. |
|
||||||
|
| `my.nginx-sso.includes.endpoint` | string | `"http://localhost:8082"` | Upstream for proxied auth requests. |
|
||||||
|
| `my.nginx-sso.includes.instances` | attribute set of (submodule) | `{ }` | nginx includes instances. |
|
||||||
|
| `my.nginx-sso.includes.instances.<name>.auth.path` | string | `"/sso-auth"` | HTTP path for SSO auth. |
|
||||||
|
| `my.nginx-sso.includes.instances.<name>.auth.redirect` | string | `"$scheme://$http_host$request_uri"` | URL to redirect to upon successful login. |
|
||||||
|
| `my.nginx-sso.includes.instances.<name>.logout.path` | string | `"/sso-logout"` | HTTP path for SSO logout. |
|
||||||
|
| `my.nginx-sso.includes.instances.<name>.logout.redirect` | string | `"$scheme://$http_host/"` | URL to redirect to upon successful logout. |
|
||||||
|
| `my.nginx-sso.package` | package | `<derivation nginx-sso-0.27.7>` | nginx-sso package to use. |
|
||||||
|
|
||||||
|
## `nvme` — [`nixos/modules/nvme`](../../nixos/modules/nvme)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.nvme.boot.address` | string | `null` | Address of NVMe-oF target. |
|
||||||
|
| `my.nvme.boot.nqn` | null or string | `null` | NQN to connect to on boot |
|
||||||
|
| `my.nvme.uuid` | null or string | `null` | NVMe host ID |
|
||||||
|
|
||||||
|
## `pdns` — [`nixos/modules/pdns.nix`](../../nixos/modules/pdns.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.pdns.auth.bind.file-records.sshKey` | null or string | `null` | SSH public key for file record update user. |
|
||||||
|
| `my.pdns.auth.bind.options.also-notify` | list of string | `[ ]` | List of additional address to send DNS NOTIFY messages to. |
|
||||||
|
| `my.pdns.auth.bind.zones` | attribute set of (submodule) | `{ }` | BIND-style zones definitions. |
|
||||||
|
| `my.pdns.auth.bind.zones.<name>.also-notify` | list of string | `[ ]` | List of additional address to send DNS NOTIFY messages to. |
|
||||||
|
| `my.pdns.auth.bind.zones.<name>.masters` | list of string | `[ ]` | List of masters to retrieve data from (as slave). |
|
||||||
|
| `my.pdns.auth.bind.zones.<name>.path` | absolute path | `null` | Path to zone file. |
|
||||||
|
| `my.pdns.auth.bind.zones.<name>.template` | boolean | `true` | Whether to run the zone contents through a template for post-processing. |
|
||||||
|
| `my.pdns.auth.bind.zones.<name>.text` | null or strings concatenated with "\n" | `null` | Inline content of the zone file. |
|
||||||
|
| `my.pdns.auth.bind.zones.<name>.type` | one of "master", "slave", "native" | `"native"` | Zone type. |
|
||||||
|
| `my.pdns.auth.enable` | boolean | `false` | Whether to enable PowerDNS authoritative nameserver. |
|
||||||
|
| `my.pdns.auth.extraSettingsFile` | null or string | `null` | Path to extra settings (e.g. for secrets). |
|
||||||
|
| `my.pdns.auth.settings` | attribute set of (null or signed integer or string or boolean or absolute path or list of (signed integer or string or boolean or absolute path)) | `{ }` | Authoritative server settings. |
|
||||||
|
| `my.pdns.recursor.enable` | boolean | `false` | Whether to enable PowerDNS recursive nameserver. |
|
||||||
|
| `my.pdns.recursor.extraSettingsFile` | null or string | `null` | Path to extra settings (e.g. for secrets). |
|
||||||
|
|
||||||
|
## `secrets` — [`nixos/modules/secrets.nix`](../../nixos/modules/secrets.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.secrets.files` | attribute set of unspecified value | `{ }` | Secrets to decrypt with agenix. |
|
||||||
|
| `my.secrets.key` | null or string | `null` | Public key that secrets for this system should be encrypted for. |
|
||||||
|
| `my.secrets.vmKeyPath` | string | `"/tmp/xchg/dev.key"` | Path to dev key when in a dev VM. |
|
||||||
|
|
||||||
|
## `server` — [`nixos/modules/server.nix`](../../nixos/modules/server.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.server.enable` | boolean | `false` | Whether to enable common configuration for servers. |
|
||||||
|
|
||||||
|
## `spdk` — [`nixos/modules/spdk.nix`](../../nixos/modules/spdk.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.spdk.config.subsystems` | attribute set of list of (submodule) | `{ }` | Subsystem config / RPCs. |
|
||||||
|
| `my.spdk.config.subsystems.<name>.*.method` | string | `null` | RPC method name. |
|
||||||
|
| `my.spdk.config.subsystems.<name>.*.params` | attribute set of unspecified value | `{ }` | RPC params |
|
||||||
|
| `my.spdk.debugCommands` | strings concatenated with "\n" | `""` | Commands to run with the spdk-debug script. |
|
||||||
|
| `my.spdk.enable` | boolean | `false` | Whether to enable SPDK target. |
|
||||||
|
| `my.spdk.extraArgs` | string | `""` | Extra arguments to pass to spdk_tgt. |
|
||||||
|
|
||||||
|
## `tmproot` — [`nixos/modules/tmproot.nix`](../../nixos/modules/tmproot.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.tmproot.enable` | boolean | `true` | Whether to enable tmproot. |
|
||||||
|
| `my.tmproot.persistence.config` | submodule | `{ }` | Persistence configuration |
|
||||||
|
| `my.tmproot.persistence.dir` | null or string | `"/persist"` | Path where persisted files are stored. |
|
||||||
|
| `my.tmproot.size` | string | `"2G"` | Size of tmpfs root |
|
||||||
|
| `my.tmproot.unsaved.ignore` | list of string | `[ ]` | Path prefixes to ignore if unsaved. |
|
||||||
|
| `my.tmproot.unsaved.showMotd` | boolean | `true` | Whether to show unsaved files with `dynamic-motd`. |
|
||||||
|
|
||||||
|
## `user` — [`nixos/modules/user.nix`](../../nixos/modules/user.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.user.config` | submodule | `{ }` | User definition (as `users.users.*`). |
|
||||||
|
| `my.user.enable` | boolean | `true` | Whether to create a primary user. |
|
||||||
|
| `my.user.homeConfig` | Home Manager module | `{ }` | Home configuration (as `home-manager.users.*`) |
|
||||||
|
| `my.user.passwordSecret` | null or string | `"user-passwd.txt"` | Name of user password secret. |
|
||||||
|
| `my.user.tmphome` | boolean | `true` | Whether to persist home directory files under tmproot |
|
||||||
|
|
||||||
|
## `vms` — [`nixos/modules/vms.nix`](../../nixos/modules/vms.nix)
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `my.vms.instances` | attribute set of (submodule) | `{ }` | VM instances. |
|
||||||
|
| `my.vms.instances.<name>.autoStart` | boolean | `true` | Whether to start the VM automatically at boot. |
|
||||||
|
| `my.vms.instances.<name>.boot` | string or (attribute set of unspecified value) convertible to it | `{ menu = "on"; splash-time = 5000; }` | Boot options. |
|
||||||
|
| `my.vms.instances.<name>.cleanShutdown.enabled` | boolean | `true` | Whether to attempt to cleanly shut down the guest. |
|
||||||
|
| `my.vms.instances.<name>.cleanShutdown.timeout` | unsigned integer, meaning >=0 | `30` | Clean shutdown timeout (in seconds). |
|
||||||
|
| `my.vms.instances.<name>.cpu` | string | `"host"` | QEMU CPU model. |
|
||||||
|
| `my.vms.instances.<name>.drives` | list of (submodule) | `[ ]` | Drives to attach to VM. |
|
||||||
|
| `my.vms.instances.<name>.drives.*.backend` | string or (attribute set of unspecified value) convertible to it | `{ }` | Backend blockdev options. |
|
||||||
|
| `my.vms.instances.<name>.drives.*.format` | string or (attribute set of unspecified value) convertible to it | `{ }` | Format blockdev options. |
|
||||||
|
| `my.vms.instances.<name>.drives.*.formatBackendProp` | string | `"file"` | Property that references the backend blockdev. |
|
||||||
|
| `my.vms.instances.<name>.drives.*.frontend` | string | `"virtio-blk"` | Frontend device driver. |
|
||||||
|
| `my.vms.instances.<name>.drives.*.frontendOpts` | string or (attribute set of unspecified value) convertible to it | `{ }` | Frontend device options. |
|
||||||
|
| `my.vms.instances.<name>.drives.*.name` | string | `null` | Drive name. |
|
||||||
|
| `my.vms.instances.<name>.enableKVM` | boolean | `true` | Whether to enable KVM. |
|
||||||
|
| `my.vms.instances.<name>.enableUEFI` | boolean | `true` | Whether to enable UEFI. |
|
||||||
|
| `my.vms.instances.<name>.hostDevices` | attribute set of (submodule) | `{ }` | Host PCI devices to pass to the VM. |
|
||||||
|
| `my.vms.instances.<name>.hostDevices.<name>.bindVFIO` | boolean | `true` | Whether to automatically bind the device to vfio-pci. |
|
||||||
|
| `my.vms.instances.<name>.hostDevices.<name>.extraOptions` | string or (attribute set of unspecified value) convertible to it | `{ }` | Extra QEMU options for the vfio-pci QEMU device. |
|
||||||
|
| `my.vms.instances.<name>.hostDevices.<name>.hostBDF` | string | `null` | PCI BDF of host device. |
|
||||||
|
| `my.vms.instances.<name>.hostDevices.<name>.index` | unsigned integer, meaning >=0 | `null` | Index of device in guest (for root port chassis and slot). |
|
||||||
|
| `my.vms.instances.<name>.machine` | string | `"q35"` | QEMU machine type. |
|
||||||
|
| `my.vms.instances.<name>.memory` | unsigned integer, meaning >=0 | `1024` | Amount of RAM (mebibytes). |
|
||||||
|
| `my.vms.instances.<name>.networks` | attribute set of (submodule) | `{ }` | Networks to attach VM to. |
|
||||||
|
| `my.vms.instances.<name>.networks.<name>.bridge` | null or string | `"‹name›"` | Network bridge to connect to (null to not attach to bridge). |
|
||||||
|
| `my.vms.instances.<name>.networks.<name>.extraOptions` | string or (attribute set of unspecified value) convertible to it | `{ }` | Extra QEMU options to set for the NIC. |
|
||||||
|
| `my.vms.instances.<name>.networks.<name>.ifname` | string | `"vm-‹name›"` | TAP device to create / use. |
|
||||||
|
| `my.vms.instances.<name>.networks.<name>.mac` | string | `null` | Guest MAC address. |
|
||||||
|
| `my.vms.instances.<name>.networks.<name>.model` | string | `"virtio-net"` | Device type for network interface. |
|
||||||
|
| `my.vms.instances.<name>.networks.<name>.tapFD` | null or (unsigned integer, meaning >=0) | `null` | FD to use to pass existing TAP device. |
|
||||||
|
| `my.vms.instances.<name>.networks.<name>.waitOnline` | boolean or string | `true` | Whether to wait for networkd to consider the bridge / existing TAP device online. Pass a string to set the OPERSTATE will wait for. |
|
||||||
|
| `my.vms.instances.<name>.qemuBin` | absolute path | `"/nix/store/w4yhckm5wyvml3pqw8ai5fl174j14nrb-qemu-host-cpu-only-11.0.0/bin/qemu-kvm"` | Path to QEMU executable. |
|
||||||
|
| `my.vms.instances.<name>.qemuFlags` | list of string | `[ ]` | Additional flags to pass to QEMU. |
|
||||||
|
| `my.vms.instances.<name>.smp.cpus` | unsigned integer, meaning >=0 | `1` | Number of CPU cores. |
|
||||||
|
| `my.vms.instances.<name>.smp.threads` | unsigned integer, meaning >=0 | `1` | Number of threads per core. |
|
||||||
|
| `my.vms.instances.<name>.spice.enable` | boolean | `true` | Whether to enable SPICE. |
|
||||||
|
| `my.vms.instances.<name>.uuid` | string | `null` | QEMU machine UUID. |
|
||||||
|
| `my.vms.instances.<name>.vga` | string | `"virtio"` | VGA card type. |
|
||||||
|
| `my.vms.ovmfPackage` | package | `<derivation OVMF-202602>` | OVMF package. |
|
||||||
@@ -5,10 +5,8 @@ edge VPSes (`britway` in London, `britnet` in Birmingham) and the `kelder` site
|
|||||||
server at a remote location, linked back to colony over WireGuard and acting as a NixOS
|
server at a remote location, linked back to colony over WireGuard and acting as a NixOS
|
||||||
container host.
|
container host.
|
||||||
|
|
||||||
| Box | What it is | Docs |
|
| Box | What it is |
|
||||||
| --- | --- | --- |
|
| --- | --- |
|
||||||
| `britway` | Vultr VPS (London, `lon1`): Headscale control plane, Tailscale exit node, BGP edge, nginx | [britway.md](britway.md) |
|
| [`britway`](britway.md) | Vultr VPS (London, `lon1`): Headscale control plane, Tailscale exit node, BGP edge, nginx |
|
||||||
| `britnet` | VPS (Birmingham, `bhx1`): Tailscale exit node / WireGuard hub | [britnet.md](britnet.md) |
|
| [`britnet`](britnet.md) | VPS (Birmingham, `bhx1`): Tailscale exit node / WireGuard hub |
|
||||||
| `kelder` | Secondary home server (`hentai.engineer`): container host, Samba, DDNS | [kelder.md](kelder.md) |
|
| [`kelder`](kelder/README.md) | Secondary home server (`hentai.engineer`): container host, Samba, DDNS (containers on its page) |
|
||||||
| `kelder-acquisition` | Media stack container on `kelder` (Transmission over VPN, *arr, Jellyfin) | [kelder-acquisition.md](kelder-acquisition.md) |
|
|
||||||
| `kelder-spoder` | Web container on `kelder` (Nextcloud + nginx reverse proxy) | [kelder-spoder.md](kelder-spoder.md) |
|
|
||||||
|
|||||||
+14
-12
@@ -5,31 +5,33 @@ narrower gateway role than [`britway`](britway.md) (no control plane, no BGP).
|
|||||||
|
|
||||||
- **Source:** [`nixos/boxes/britnet.nix`](../../nixos/boxes/britnet.nix)
|
- **Source:** [`nixos/boxes/britnet.nix`](../../nixos/boxes/britnet.nix)
|
||||||
- **Host:** VPS (Birmingham, `bhx1`; provider uplink assignment `allhost`)
|
- **Host:** VPS (Birmingham, `bhx1`; provider uplink assignment `allhost`)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
|
## Platform
|
||||||
|
|
||||||
|
| Component | Allocation |
|
||||||
|
|---|---|
|
||||||
|
| Virtualisation | KVM/QEMU guest |
|
||||||
|
| Compute | 2 vCPUs and 2 GiB RAM |
|
||||||
|
| Storage | 32 GiB virtio disk with separate ext4 filesystems for `/boot`, `/nix` and `/persist`; root is tmpfs |
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **Tailscale exit node** — logs into the headscale on [`britway`](britway.md)
|
- **Tailscale exit node** — logs into the headscale on [`britway`](britway.md)
|
||||||
(`--login-server=https://hs.nul.ie`) with `--advertise-exit-node`.
|
(`--login-server=https://hs.nul.ie`) with `--advertise-exit-node`.
|
||||||
- **WireGuard hub** — `wg0` listens on UDP 51820 on the `vpn` network
|
- **WireGuard hub** — `wg0` listens on UDP 51820 on the `vpn` assignment, with a single static
|
||||||
(`10.200.0.0/24` / `fdfb:5ebf:6e84::/64`), with a single peer at `10.200.0.10` /
|
peer.
|
||||||
`fdfb:5ebf:6e84::10`.
|
|
||||||
- **NAT gateway** — traffic arriving on `tailscale0`/`wg0` is forwarded out `veth0` and SNATed
|
- **NAT gateway** — traffic arriving on `tailscale0`/`wg0` is forwarded out `veth0` and SNATed
|
||||||
to the `allhost` v4/v6 addresses.
|
to the `allhost` v4/v6 addresses.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: britnet -->
|
See the consolidated [network assignments](../networking.md#box-assignments) table (this box: `britnet`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| britnet | allhost | `77.74.199.67/24 gw 77.74.199.1` | `2a12:ab46:5344:99::a/64 gw 2a12:ab46:5344::1` | bhx1.int.nul.ie | |
|
|
||||||
| britnet | vpn | `10.200.0.1/24` | `fdfb:5ebf:6e84::1/64` | — | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
- The provider interface is renamed to `veth0` by MAC. The v6 default gateway
|
- The provider interface is renamed to `veth0` by MAC. Its IPv6 default gateway sits off-subnet, so
|
||||||
(`2a12:ab46:5344::1`) sits off-subnet, so a link-scope route is added to reach it.
|
a link-scope route is added to reach it.
|
||||||
- `wg0` is a networkd WireGuard netdev keyed from `britnet/wg.key`; RA is disabled on it.
|
- `wg0` is a networkd WireGuard netdev keyed from `britnet/wg.key`; RA is disabled on it.
|
||||||
- Upstream DNS is hardcoded to Cloudflare (`1.1.1.1` / `1.0.0.1`).
|
- Upstream DNS is hardcoded to Cloudflare (`1.1.1.1` / `1.0.0.1`).
|
||||||
- `iperf3` runs with an open port for bandwidth testing.
|
- `iperf3` runs with an open port for bandwidth testing.
|
||||||
|
|||||||
+14
-11
@@ -5,6 +5,15 @@ control plane, a tailnet exit node, and the BGP speaker for AS211024.
|
|||||||
|
|
||||||
- **Source:** [`nixos/boxes/britway/`](../../nixos/boxes/britway)
|
- **Source:** [`nixos/boxes/britway/`](../../nixos/boxes/britway)
|
||||||
- **Host:** VPS at Vultr (London, `lon1`)
|
- **Host:** VPS at Vultr (London, `lon1`)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
|
## Platform
|
||||||
|
|
||||||
|
| Component | Allocation |
|
||||||
|
|---|---|
|
||||||
|
| Virtualisation | Vultr VC2 virtual guest on a QEMU-compatible platform |
|
||||||
|
| Compute | 2 vCPUs and 2 GiB RAM |
|
||||||
|
| Storage | 65 GiB virtio disk with separate ext4 filesystems for `/boot`, `/nix` and `/persist`; root is tmpfs |
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
@@ -18,27 +27,21 @@ control plane, a tailnet exit node, and the BGP speaker for AS211024.
|
|||||||
- **BGP edge** — `bird2` speaks BGP as AS211024 to Vultr transit (AS64515, separate v4/v6
|
- **BGP edge** — `bird2` speaks BGP as AS211024 to Vultr transit (AS64515, separate v4/v6
|
||||||
sessions authenticated with a password from `britway/bgp-password-vultr.conf`) and exports
|
sessions authenticated with a password from `britway/bgp-password-vultr.conf`) and exports
|
||||||
everything to a `bgp.tools` monitoring session. It originates the internal, colony and home
|
everything to a `bgp.tools` monitoring session. It originates the internal, colony and home
|
||||||
v6 prefixes (`2a0e:97c0:4df::/48`, `2a0e:97c0:4d2::/48`, `2a0e:97c0:4d0::/48`).
|
IPv6 prefixes documented in [networking](../networking.md#domains).
|
||||||
- **nginx** — reverse proxy fronting headscale (`hs.nul.ie` → `localhost` headscale port),
|
- **nginx** — reverse proxy fronting headscale (`hs.nul.ie` → `localhost` headscale port),
|
||||||
with a wildcard ACME cert for `nul.ie` issued via Cloudflare DNS.
|
with a wildcard ACME cert for `nul.ie` issued via Cloudflare DNS.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: britway -->
|
See the consolidated [network assignments](../networking.md#box-assignments) table (this box: `britway`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| britway | as211024 | `10.100.50.5/24` | `2a0e:97c0:4df:0:2::1/64` | — | |
|
|
||||||
| britway | vultr | `45.76.141.188/23 gw 45.76.140.1` | `2001:19f0:7402:128b::1/64` | lon1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
- Two assignments: `vultr` on the provider interface `veth0` (renamed by MAC), and `as211024`
|
- Two assignments: `vultr` on the provider interface `veth0` (renamed by MAC), and `as211024`
|
||||||
on the l2mesh VXLAN interface (`my.vpns.l2`, a mesh shared with `estuary` and the home
|
on the `l2mesh` VXLAN interface (`my.vpns.l2`) — member of the shared mesh (see
|
||||||
routers `river`/`stream`).
|
[The AS211024 L2 mesh](../networking.md#the-as211024-l2-mesh)).
|
||||||
- Static routes steer colony/home v4 traffic over the `as211024` mesh. A separate `ts-extra`
|
- Static routes steer colony/home v4 traffic over the `as211024` mesh. A separate `ts-extra`
|
||||||
routing table (1337) with a policy rule on `tailscale0` ingress sends Tailscale-sourced v6
|
routing table with a policy rule on `tailscale0` ingress sends Tailscale-sourced v6
|
||||||
traffic for colony via `estuary`, while the box's own v6 uses WAN.
|
traffic for colony via `estuary`, while the box's own v6 uses WAN.
|
||||||
- The firewall trusts the `as211024` prefixes (`lib.my.c.as211024.nftTrust`) and
|
- The firewall trusts the `as211024` prefixes (`lib.my.c.as211024.nftTrust`) and
|
||||||
`tailscale0`; `iperf3` runs with an open port for bandwidth testing.
|
`tailscale0`; `iperf3` runs with an open port for bandwidth testing.
|
||||||
|
|||||||
@@ -1,48 +0,0 @@
|
|||||||
# kelder-acquisition
|
|
||||||
|
|
||||||
The media acquisition stack for the kelder site — Transmission (forced over VPN), the *arrs
|
|
||||||
and Jellyfin in one NixOS container.
|
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/kelder/containers/acquisition/`](../../nixos/boxes/kelder/containers/acquisition)
|
|
||||||
- **Host:** NixOS container on [`kelder`](kelder.md)
|
|
||||||
|
|
||||||
## Role
|
|
||||||
|
|
||||||
- **Transmission** (`transmission_4`) — BitTorrent client bound to the VPN
|
|
||||||
(`bindsTo systemd-networkd-wait-online@vpn.service`); peer port 26180 (forwarded in the
|
|
||||||
AirVPN config), 20 MiB/s down / 1 MiB/s up limits, ratio limit 2.0. Downloads land in
|
|
||||||
`/mnt/media/downloads/torrents` (`/mnt/media` is bind-mounted from the host's
|
|
||||||
`/mnt/storage/media`).
|
|
||||||
- **Jackett, Radarr, Sonarr** — indexer + media managers, in the shared `media` group with
|
|
||||||
`UMask 0002`.
|
|
||||||
- **Jellyfin** — streaming with Intel hardware transcoding: `/dev/dri` is bind-mounted from
|
|
||||||
the host, `intel-vaapi-driver`/`intel-ocl` are installed and the `jellyfin` user is in the
|
|
||||||
`render` group.
|
|
||||||
|
|
||||||
## Network assignments
|
|
||||||
|
|
||||||
<!-- assignments: kelder-acquisition -->
|
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| acquisition-ctr | internal | `172.16.64.2/24 gw 172.16.64.1` | — | hentai.engineer | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Networking
|
|
||||||
|
|
||||||
- `internal` assignment (name `acquisition-ctr`) on the host's `ctrs` bridge, MTU 1460 to
|
|
||||||
match the site WAN.
|
|
||||||
- All non-site traffic goes over an AirVPN WireGuard tunnel (`vpn` netdev, MTU 1320, AirVPN
|
|
||||||
IE endpoint) using fwmark 42 / route table 51820: policy rules keep traffic to and from the
|
|
||||||
kelder prefixes on the main table and push everything else via the VPN.
|
|
||||||
- An nftables input chain drops new TCP connections from the VPN interface except the
|
|
||||||
Transmission peer port; the web UI ports (9091 Transmission, 9117 Jackett, 7878 Radarr,
|
|
||||||
8989 Sonarr, 8096 Jellyfin) are accepted from the site. When built as a dev VM, those ports
|
|
||||||
are forwarded to the host.
|
|
||||||
- Sonarr still needs the EOL .NET 6 runtime, allowed via
|
|
||||||
`nixpkgs.config.permittedInsecurePackages`.
|
|
||||||
|
|
||||||
## Notable config files
|
|
||||||
|
|
||||||
- [`nixos/boxes/kelder/containers/acquisition/default.nix`](../../nixos/boxes/kelder/containers/acquisition/default.nix) — services and users.
|
|
||||||
- [`nixos/boxes/kelder/containers/acquisition/networking.nix`](../../nixos/boxes/kelder/containers/acquisition/networking.nix) — AirVPN tunnel + firewall.
|
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
# kelder
|
|
||||||
|
|
||||||
Secondary home server at a remote site, domain `hentai.engineer`. Linked back to colony over
|
|
||||||
WireGuard and acting as a NixOS container host (like `shill`/`sfh`).
|
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/kelder/`](../../nixos/boxes/kelder)
|
|
||||||
- **Host:** physical (Intel; LTS kernel, `kvm-intel`, IOMMU on)
|
|
||||||
|
|
||||||
## Role
|
|
||||||
|
|
||||||
- **Container host** — runs two NixOS containers on the `ctrs` bridge
|
|
||||||
(`my.containers.instances`): `kelder-acquisition` and `kelder-spoder` (below).
|
|
||||||
- **Public services via colony** — a WireGuard tunnel (`estuary` netdev) connects to colony's
|
|
||||||
`estuary` box, which DNATs public traffic to kelder's `/32`; policy routing (table 51820 +
|
|
||||||
connmark) sends replies back through the tunnel while ordinary traffic uses the LAN.
|
|
||||||
kelder's own NAT forwards `http`/`https` on to `kelder-spoder`.
|
|
||||||
- **Nextcloud host** — served from the `kelder-spoder` container.
|
|
||||||
- **Samba** — the `storage` share backed by `/mnt/storage`, with `nmbd` and `samba-wsdd` for
|
|
||||||
Windows discovery.
|
|
||||||
- **DDNS** — a `ddns-update` timer (every 10 min) runs `dns_update.py` to sync the
|
|
||||||
`hentai.engineer` and `kelder-local.hentai.engineer` Cloudflare records with the address on
|
|
||||||
`et1g0`.
|
|
||||||
|
|
||||||
## Network assignments
|
|
||||||
|
|
||||||
<!-- assignments: kelder -->
|
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| kelder-ctrs | ctrs | `172.16.64.1/24` | — | hentai.engineer | |
|
|
||||||
| kelder | estuary | `94.142.242.254/32` | — | — | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Containers
|
|
||||||
|
|
||||||
| Container | Role | Docs |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `kelder-acquisition` | Media stack (Transmission over AirVPN, Jackett/Radarr/Sonarr, Jellyfin) | [kelder-acquisition.md](kelder-acquisition.md) |
|
|
||||||
| `kelder-spoder` | Nextcloud + nginx reverse proxy | [kelder-spoder.md](kelder-spoder.md) |
|
|
||||||
|
|
||||||
The containers are not deploy targets (`my.deploy.enable = false`); they're managed through
|
|
||||||
the host.
|
|
||||||
|
|
||||||
## Networking
|
|
||||||
|
|
||||||
- LAN on `et1g0` (renamed by MAC) with DHCP and MTU 1460 (`lib.my.c.kelder.ipv4MTU`); the
|
|
||||||
kelder v4 prefixes are masqueraded out of it.
|
|
||||||
- The `estuary` WireGuard peer (`AllowedIPs = 0.0.0.0/0`, keepalive 25s) is combined with
|
|
||||||
rules that keep LAN traffic on the main table and only route tunnel-marked/owned traffic
|
|
||||||
via table 51820.
|
|
||||||
|
|
||||||
## Services
|
|
||||||
|
|
||||||
- `netdata` (proxied as `monitor.hentai.engineer` by `kelder-spoder`), `smartd`, `fstrim`,
|
|
||||||
LVM thin provisioning.
|
|
||||||
- `minecraft-server` is present but **disabled** (`enable = false`); the firewall still opens
|
|
||||||
25565 tcp/udp.
|
|
||||||
- Primary user `kontent` (in the `storage`/`media` groups).
|
|
||||||
- Sets `system.nixos.distroName = "KelderOS"`, a custom Plymouth theme and an `amogus-beep`
|
|
||||||
boot jingle ([`boot.nix`](../../nixos/boxes/kelder/boot.nix)).
|
|
||||||
|
|
||||||
## Notable config files
|
|
||||||
|
|
||||||
- [`nixos/boxes/kelder/default.nix`](../../nixos/boxes/kelder/default.nix) — system, assignments, tunnel, NAT, containers.
|
|
||||||
- [`nixos/boxes/kelder/boot.nix`](../../nixos/boxes/kelder/boot.nix) — Plymouth theme + boot beep.
|
|
||||||
- [`nixos/boxes/kelder/containers/`](../../nixos/boxes/kelder/containers) — the two container definitions.
|
|
||||||
- [`nixos/boxes/kelder/dns_update.py`](../../nixos/boxes/kelder/dns_update.py) — Cloudflare DDNS script.
|
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# kelder
|
||||||
|
|
||||||
|
Secondary home server at a remote site, domain `hentai.engineer`. Linked back to colony over
|
||||||
|
WireGuard and acting as a NixOS container host (like `shill`/`sfh`).
|
||||||
|
|
||||||
|
- **Source:** [`nixos/boxes/kelder/`](../../../nixos/boxes/kelder)
|
||||||
|
- **Host:** physical (Intel; LTS kernel, `kvm-intel`, IOMMU on)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
|
## Role
|
||||||
|
|
||||||
|
- **Container host** — runs two NixOS containers on the `ctrs` bridge
|
||||||
|
(`my.containers.instances`): `kelder-acquisition` and `kelder-spoder` (below).
|
||||||
|
- **Public services via colony** — a WireGuard tunnel (`estuary` netdev) connects to colony's
|
||||||
|
`estuary` box, which DNATs public traffic to kelder's tunneled assignment; connection-mark-based
|
||||||
|
policy routing sends replies back through the tunnel while ordinary traffic uses the LAN.
|
||||||
|
kelder's own NAT forwards `http`/`https` on to `kelder-spoder`.
|
||||||
|
- **Nextcloud host** — served from the `kelder-spoder` container.
|
||||||
|
- **Samba** — the `storage` share backed by `/mnt/storage`, with `nmbd` and `samba-wsdd` for
|
||||||
|
Windows discovery.
|
||||||
|
- **DDNS** — a `ddns-update` timer runs `dns_update.py` periodically to sync the
|
||||||
|
`hentai.engineer` and `kelder-local.hentai.engineer` Cloudflare records with the address on
|
||||||
|
`et1g0`.
|
||||||
|
|
||||||
|
## Network assignments
|
||||||
|
|
||||||
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `kelder`).
|
||||||
|
|
||||||
|
## Containers
|
||||||
|
|
||||||
|
| Container | Role |
|
||||||
|
| --- | --- |
|
||||||
|
| [`kelder-acquisition`](containers/kelder-acquisition.md) | Media stack (Transmission over AirVPN, Jackett/Radarr/Sonarr, Jellyfin) |
|
||||||
|
| [`kelder-spoder`](containers/kelder-spoder.md) | Nextcloud + nginx reverse proxy |
|
||||||
|
|
||||||
|
The containers are not deploy targets (`my.deploy.enable = false`); they're managed through
|
||||||
|
the host.
|
||||||
|
|
||||||
|
## Networking
|
||||||
|
|
||||||
|
- LAN on `et1g0` (renamed by MAC) with DHCP and MTU 1460 (`lib.my.c.kelder.ipv4MTU`); the
|
||||||
|
kelder v4 prefixes are masqueraded out of it.
|
||||||
|
- The `estuary` WireGuard peer is combined with rules that keep LAN traffic on the main table and
|
||||||
|
only route tunnel-marked or owned traffic through the tunnel's dedicated table.
|
||||||
|
|
||||||
|
## Services
|
||||||
|
|
||||||
|
- `netdata` (proxied as `monitor.hentai.engineer` by `kelder-spoder`), `smartd`, `fstrim`,
|
||||||
|
LVM thin provisioning.
|
||||||
|
- `minecraft-server` is present but **disabled** (`enable = false`); the firewall still opens
|
||||||
|
25565 tcp/udp.
|
||||||
|
- Primary user `kontent` (in the `storage`/`media` groups).
|
||||||
|
- Sets `system.nixos.distroName = "KelderOS"`, a custom Plymouth theme and an `amogus-beep`
|
||||||
|
boot jingle ([`boot.nix`](../../../nixos/boxes/kelder/boot.nix)).
|
||||||
|
|
||||||
|
## Notable config files
|
||||||
|
|
||||||
|
- [`nixos/boxes/kelder/default.nix`](../../../nixos/boxes/kelder/default.nix) — system, assignments, tunnel, NAT, containers.
|
||||||
|
- [`nixos/boxes/kelder/boot.nix`](../../../nixos/boxes/kelder/boot.nix) — Plymouth theme + boot beep.
|
||||||
|
- [`nixos/boxes/kelder/containers/`](../../../nixos/boxes/kelder/containers) — the two container definitions.
|
||||||
|
- [`nixos/boxes/kelder/dns_update.py`](../../../nixos/boxes/kelder/dns_update.py) — Cloudflare DDNS script.
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# kelder-acquisition
|
||||||
|
|
||||||
|
The media acquisition stack for the kelder site — Transmission (forced over VPN), the *arrs
|
||||||
|
and Jellyfin in one NixOS container.
|
||||||
|
|
||||||
|
- **Source:** [`nixos/boxes/kelder/containers/acquisition/`](../../../../nixos/boxes/kelder/containers/acquisition)
|
||||||
|
- **Host:** NixOS container on [`kelder`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
|
## Role
|
||||||
|
|
||||||
|
### Transmission
|
||||||
|
|
||||||
|
`transmission_4` is bound to the VPN wait-online unit and uses AirVPN's forwarded peer port. Upload,
|
||||||
|
download and seed-ratio limits are configured in the service source. Downloads use
|
||||||
|
`/mnt/media/downloads/torrents`, backed by the host's `/mnt/storage/media`.
|
||||||
|
|
||||||
|
### Media services
|
||||||
|
|
||||||
|
Jackett, Radarr and Sonarr share the `media` group with a group-writable umask. Jellyfin uses the
|
||||||
|
host's bind-mounted `/dev/dri` with `intel-vaapi-driver` / `intel-ocl`; its user belongs to
|
||||||
|
`render`.
|
||||||
|
|
||||||
|
## Network assignments
|
||||||
|
|
||||||
|
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `kelder-acquisition`).
|
||||||
|
|
||||||
|
## Networking
|
||||||
|
|
||||||
|
- `internal` assignment (name `acquisition-ctr`) on the host's `ctrs` bridge, MTU 1460 to
|
||||||
|
match the site WAN.
|
||||||
|
- All non-site traffic goes over an AirVPN WireGuard tunnel (`vpn` netdev, AirVPN IE endpoint).
|
||||||
|
Policy rules keep traffic to and from the kelder prefixes on the main table and push everything
|
||||||
|
else through the VPN's dedicated table.
|
||||||
|
- An nftables input chain drops new TCP connections from the VPN interface except the
|
||||||
|
Transmission peer port; the web UI ports (9091 Transmission, 9117 Jackett, 7878 Radarr,
|
||||||
|
8989 Sonarr, 8096 Jellyfin) are accepted from the site. When built as a dev VM, those ports
|
||||||
|
are forwarded to the host.
|
||||||
|
- Sonarr still needs the EOL .NET 6 runtime, allowed via
|
||||||
|
`nixpkgs.config.permittedInsecurePackages`.
|
||||||
|
|
||||||
|
## Notable config files
|
||||||
|
|
||||||
|
- [`nixos/boxes/kelder/containers/acquisition/default.nix`](../../../../nixos/boxes/kelder/containers/acquisition/default.nix) — services and users.
|
||||||
|
- [`nixos/boxes/kelder/containers/acquisition/networking.nix`](../../../../nixos/boxes/kelder/containers/acquisition/networking.nix) — AirVPN tunnel + firewall.
|
||||||
@@ -3,8 +3,9 @@
|
|||||||
The web container for the kelder site: Nextcloud plus an nginx (OpenResty) reverse proxy for
|
The web container for the kelder site: Nextcloud plus an nginx (OpenResty) reverse proxy for
|
||||||
the site's services.
|
the site's services.
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/kelder/containers/spoder/`](../../nixos/boxes/kelder/containers/spoder)
|
- **Source:** [`nixos/boxes/kelder/containers/spoder/`](../../../../nixos/boxes/kelder/containers/spoder)
|
||||||
- **Host:** NixOS container on [`kelder`](kelder.md)
|
- **Host:** NixOS container on [`kelder`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
@@ -18,12 +19,7 @@ the site's services.
|
|||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: kelder-spoder -->
|
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `kelder-spoder`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| spoder-ctr | internal | `172.16.64.3/24 gw 172.16.64.1` | — | hentai.engineer | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Proxy vhosts
|
## Proxy vhosts
|
||||||
|
|
||||||
@@ -39,8 +35,8 @@ All under `hentai.engineer`, each with a `*-local` alias:
|
|||||||
| `sonarr` | Sonarr on `kelder-acquisition` (:8989) | `htpasswd` |
|
| `sonarr` | Sonarr on `kelder-acquisition` (:8989) | `htpasswd` |
|
||||||
| `cloud` | Nextcloud (local) | — |
|
| `cloud` | Nextcloud (local) | — |
|
||||||
|
|
||||||
An `init_worker_by_lua` timer polls `v4.ident.me` every 5 minutes to track the site's public
|
An `init_worker_by_lua` timer polls `v4.ident.me` periodically to track the site's public IP; it
|
||||||
IP; it feeds `localRedirect` rewrites (bounce public-IP clients to the `*-local` name) that
|
feeds `localRedirect` rewrites (bounce public-IP clients to the `*-local` name) that
|
||||||
are currently **disabled** (commented out — Virgin Media filters DNS answers containing local
|
are currently **disabled** (commented out — Virgin Media filters DNS answers containing local
|
||||||
IPs, so the split doesn't work as intended).
|
IPs, so the split doesn't work as intended).
|
||||||
|
|
||||||
@@ -50,5 +46,5 @@ IPs, so the split doesn't work as intended).
|
|||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
- [`nixos/boxes/kelder/containers/spoder/default.nix`](../../nixos/boxes/kelder/containers/spoder/default.nix) — Nextcloud + ACME.
|
- [`nixos/boxes/kelder/containers/spoder/default.nix`](../../../../nixos/boxes/kelder/containers/spoder/default.nix) — Nextcloud + ACME.
|
||||||
- [`nixos/boxes/kelder/containers/spoder/nginx.nix`](../../nixos/boxes/kelder/containers/spoder/nginx.nix) — reverse proxy vhosts.
|
- [`nixos/boxes/kelder/containers/spoder/nginx.nix`](../../../../nixos/boxes/kelder/containers/spoder/nginx.nix) — reverse proxy vhosts.
|
||||||
+18
-76
@@ -7,84 +7,26 @@ the boxes: almost everything reachable from the internet lives here.
|
|||||||
- **Public domain:** `nul.ie` — public services are published as `*.nul.ie`
|
- **Public domain:** `nul.ie` — public services are published as `*.nul.ie`
|
||||||
- **Source:** [`nixos/boxes/colony/`](../../../nixos/boxes/colony)
|
- **Source:** [`nixos/boxes/colony/`](../../../nixos/boxes/colony)
|
||||||
|
|
||||||
## Shape
|
## Networking
|
||||||
|
|
||||||
`colony` is the physical VM host. It runs the VMs below; `shill` is itself a
|
`colony` separates the host, VMs, `shill` containers and `whale2` OCI workloads onto dedicated
|
||||||
NixOS container host where most applications run.
|
networks behind [`estuary`](estuary.md), which terminates the public addressing. The canonical
|
||||||
|
prefixes and routing overview are in the [`colony` section of networking.md](../../networking.md#colony).
|
||||||
|
|
||||||
```
|
## Boxes
|
||||||
colony (physical VM host, ams1)
|
|
||||||
├── estuary ── edge router: WAN, firewall/NAT, DNS, BGP (AS211024), WireGuard
|
|
||||||
├── shill ──── NixOS container host ──┬── middleman (reverse proxy, ACME, nginx-sso, librespeed)
|
|
||||||
│ ├── vaultwarden (password manager)
|
|
||||||
│ ├── colony-psql (shared PostgreSQL)
|
|
||||||
│ ├── chatterbox (Matrix Synapse + bridges)
|
|
||||||
│ ├── jackflix (media stack)
|
|
||||||
│ ├── object (MinIO, Harmonia Nix cache, Sharry, HedgeDoc, wastebin)
|
|
||||||
│ ├── toot (Bluesky PDS; Mastodon disabled)
|
|
||||||
│ ├── waffletail (Tailscale subnet router / exit node)
|
|
||||||
│ ├── qclk (WireGuard management appliance)
|
|
||||||
│ ├── gam (Terraria server)
|
|
||||||
│ └── jam (raw nspawn customer container)
|
|
||||||
├── whale2 ─── podman/OCI host for game servers
|
|
||||||
├── git ────── Gitea + Gitea Actions runner
|
|
||||||
├── mail ───── Debian VM running mailcow (not NixOS)
|
|
||||||
└── darts ──── third-party/customer VM (opaque, not NixOS)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Networks
|
| Box | Role |
|
||||||
|
|---|---|
|
||||||
|
| [`colony`](colony.md) | Physical VM host (AMD, KVM, LVM-thin, `borgthin` backups → rsync.net) |
|
||||||
|
| [`estuary`](estuary.md) | Edge router: WAN, firewall/NAT, DNS, BGP (AS211024), WireGuard |
|
||||||
|
| [`shill`](shill/README.md) | NixOS container host (most applications; per-container pages under `shill/`) |
|
||||||
|
| [`whale2`](whale2.md) | podman/OCI game-server host |
|
||||||
|
| [`git`](git.md) | Gitea + Gitea Actions runner |
|
||||||
|
| [`mail`](mail.md) | Debian VM running mailcow (not NixOS) |
|
||||||
|
| [`darts`](darts.md) | Third-party/customer VM (not NixOS) |
|
||||||
|
|
||||||
All internal space is carved out of `10.100.0.0/16` and `2a0e:97c0:4d2:10::/60`
|
The applications running on `shill` are listed on its own page — see
|
||||||
(`lib.my.c.colony.prefixes`):
|
[shill/README.md](shill/README.md#containers).
|
||||||
|
|
||||||
| Network | IPv4 CIDR | IPv6 CIDR | Purpose |
|
`mail` and `darts` are host-defined VMs whose guest operating systems are managed out of band; their
|
||||||
|---|---|---|---|
|
pages document only what this repository controls.
|
||||||
| `base` | `10.100.0.0/24` | `2a0e:97c0:4d2:10::/64` | Base LAN shared by `colony` and `estuary` |
|
|
||||||
| `vms` | `10.100.1.0/24` | `2a0e:97c0:4d2:11::/64` | VM network |
|
|
||||||
| `ctrs` | `10.100.2.0/24` | `2a0e:97c0:4d2:12::/64` | `shill` container network |
|
|
||||||
| `oci` | `10.100.3.0/24` | `2a0e:97c0:4d2:13::/64` | `whale2` podman network |
|
|
||||||
|
|
||||||
Public addressing — the WAN /24 (`94.142.240.44`), the `vip*` ranges shared by
|
|
||||||
the VMs, and the customer /32s for `mail` / `darts` — terminates on `estuary`;
|
|
||||||
see [estuary.md](estuary.md).
|
|
||||||
|
|
||||||
## Machines
|
|
||||||
|
|
||||||
| Machine | Role | Page |
|
|
||||||
|---|---|---|
|
|
||||||
| `colony` | Physical VM host (AMD, KVM, LVM-thin, `borgthin` backups → rsync.net) | [colony.md](colony.md) |
|
|
||||||
| `estuary` | Edge router: WAN, firewall/NAT, DNS, BGP (AS211024), WireGuard | [estuary.md](estuary.md) |
|
|
||||||
| `shill` | NixOS container host (most applications) | [shill.md](shill.md) |
|
|
||||||
| `whale2` | podman/OCI game-server host | [whale2.md](whale2.md) |
|
|
||||||
| `git` | Gitea + Gitea Actions runner | [git.md](git.md) |
|
|
||||||
| `mail` | Debian VM running mailcow (not NixOS) | [mail.md](mail.md) |
|
|
||||||
| `darts` | Third-party/customer VM (not NixOS) | [darts.md](darts.md) |
|
|
||||||
|
|
||||||
### `shill` containers
|
|
||||||
|
|
||||||
Each has its own page under `shill/containers/`:
|
|
||||||
|
|
||||||
| Container | Role | Page |
|
|
||||||
|---|---|---|
|
|
||||||
| `middleman` | Front-end nginx reverse proxy, ACME, nginx-sso, librespeed | [middleman](shill/containers/middleman.md) |
|
|
||||||
| `vaultwarden` | Vaultwarden password manager | [vaultwarden](shill/containers/vaultwarden.md) |
|
|
||||||
| `colony-psql` | Shared PostgreSQL for colony services | [colony-psql](shill/containers/colony-psql.md) |
|
|
||||||
| `chatterbox` | Matrix homeserver + bridges | [chatterbox](shill/containers/chatterbox.md) |
|
|
||||||
| `jackflix` | Media stack (Jellyfin, *arr, Transmission, PhotoPrism, copyparty) | [jackflix](shill/containers/jackflix.md) |
|
|
||||||
| `object` | MinIO (S3), Harmonia Nix cache, Sharry, HedgeDoc, wastebin | [object](shill/containers/object.md) |
|
|
||||||
| `toot` | Bluesky PDS (Mastodon disabled) | [toot](shill/containers/toot.md) |
|
|
||||||
| `waffletail` | Tailscale subnet router / exit node | [waffletail](shill/containers/waffletail.md) |
|
|
||||||
| `qclk` | WireGuard management appliance | [qclk](shill/containers/qclk.md) |
|
|
||||||
| `gam` | Terraria server | [gam](shill/containers/gam.md) |
|
|
||||||
|
|
||||||
## Non-NixOS VMs
|
|
||||||
|
|
||||||
Two VMs are declared in `colony`'s `my.vms.instances` — so `colony` runs them
|
|
||||||
and routes/firewalls their traffic — but they are **not** managed as NixOS
|
|
||||||
systems by this repo:
|
|
||||||
|
|
||||||
- **`mail`** — a Debian VM running [mailcow](https://mailcow.email/)
|
|
||||||
(`mail.nul.ie`). ACME certificates are pushed to it from `middleman`.
|
|
||||||
See [mail.md](mail.md).
|
|
||||||
- **`darts`** — an opaque third-party/customer VM, given a routed public /32
|
|
||||||
and IPv6 /64 and otherwise left alone. See [darts.md](darts.md).
|
|
||||||
|
|||||||
+61
-37
@@ -8,39 +8,64 @@ everything at the colony site.
|
|||||||
- **Host:** bare metal (this *is* the physical box)
|
- **Host:** bare metal (this *is* the physical box)
|
||||||
- **nixpkgs:** `mine-stable`
|
- **nixpkgs:** `mine-stable`
|
||||||
|
|
||||||
|
## Hardware
|
||||||
|
|
||||||
|
| Component | Inventory |
|
||||||
|
|---|---|
|
||||||
|
| Platform | ASRock Rack X570D4U server board |
|
||||||
|
| CPU | AMD Ryzen 9 5950X (16 cores / 32 threads) |
|
||||||
|
| Memory | 128 GiB |
|
||||||
|
| NVMe storage | Three 2 TB Samsung SSD 980 PRO devices providing the NVMe-backed LVM thin pool and data LVs |
|
||||||
|
| Bulk storage | Three 12 TB WD120EDBZ disks and one 18 TB WD180EDGZ disk for the bulk LVM volumes |
|
||||||
|
| Boot | SanDisk USB device holding the EFI system partition |
|
||||||
|
| Network / management | Two Intel I210 Gigabit Ethernet controllers, one passed through to `estuary`; ASPEED BMC graphics and console |
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
Bare-metal AMD host. It does little application work itself — its job is to run
|
Bare-metal AMD host. It does little application work itself — its job is to run
|
||||||
the VMs and provide them with storage, networking and backups.
|
the VMs and provide them with storage, networking and backups.
|
||||||
|
|
||||||
- **Virtualisation:** QEMU/KVM (`kvm-amd`, IOMMU on) driven by the `my.vms`
|
### Virtualisation
|
||||||
module: each entry in `my.vms.instances` becomes a `vm@<name>` systemd
|
|
||||||
service running `qemu-kvm` with UEFI, a QMP/monitor socket under
|
The `my.vms` module drives QEMU/KVM with `kvm-amd` and IOMMU enabled. Each `my.vms.instances` entry
|
||||||
`/run/vms/<name>/`, TAP networking and optional PCI passthrough
|
becomes a `vm@<name>` systemd service with UEFI, QMP/monitor sockets under `/run/vms/<name>/`, TAP
|
||||||
(`hostDevices`, bound to `vfio-pci`). `estuary` gets the WAN NIC this way.
|
networking and optional `hostDevices` passthrough through `vfio-pci`. `estuary` receives the WAN NIC
|
||||||
- **Storage:** LVM-thin (`services.lvm.boot.thin`) in the `main` VG; VM disks
|
this way.
|
||||||
are logical volumes (`vm-<name>-<disk>`, see the `lib.my.vm.disk` /
|
|
||||||
`lvmDisk` helpers). `/persist` holds host state, `/mnt/backup` the local
|
### Storage
|
||||||
borg repo. Only the boot-critical LVs are activated in the initrd; the rest
|
|
||||||
come up via `lvm-activate-main.service`.
|
All three NVMe SSDs and all four SATA HDDs are physical volumes in one `main` VG. Separate pools
|
||||||
- **Backups:** `my.borgthin` job `main` snapshots the persist/data LVs of the
|
and standalone RAID LVs keep workloads on the appropriate media:
|
||||||
host and its VMs into `/mnt/backup/main`; `borgthin-rsync.service` then
|
|
||||||
rsyncs the repo to rsync.net and `rsync-lvm-meta.service` ships the LVM
|
| Layer | Physical layout | Main consumers |
|
||||||
metadata alongside (both run idle-priority, after the borg job).
|
|---|---|---|
|
||||||
- **Monitoring/health:** netdata (freeipmi, ignoring the VCCM sensor), `smartd`
|
| `nvme-tpool` | Thin-pool data is RAID 0 across the three NVMe SSDs; thin metadata is RAID 1 on two of them. The data itself has no redundancy. | Host `/nix` and `/persist`; VM system and persistence disks; fast data LVs such as `minio`, `oci`, `git`, `gitea-actions-cache`, `nix-cache` and `jam` |
|
||||||
(logging to `/var/log/smartd`), `rasdaemon`, `fstrim` at 04:45 (before the
|
| `hdds-tpool` | Thin-pool data is RAID 5 across the three 12 TB HDDs; thin metadata is RAID 1 on two NVMe SSDs. | `media`, passed to `shill`, and `backup`, mounted by the host at `/mnt/backup` |
|
||||||
05:00 backup).
|
| `darts-media` | Standalone RAID 5 LV spanning all four HDDs. | Bulk storage passed to `darts` |
|
||||||
|
| `darts-ext` | Standalone linear LV using the remaining capacity of the 18 TB HDD. | Expansion storage passed to `darts` |
|
||||||
|
|
||||||
|
`media` and `backup` are thin-provisioned, so their virtual capacities are not additive physical
|
||||||
|
capacity and overcommit `hdds-tpool`. VM disks built with `lib.my.vm.disk` are named
|
||||||
|
`vm-<name>-<disk>` in `main`; `lib.my.vm.lvmDisk` attaches the named data LVs.
|
||||||
|
|
||||||
|
The initrd activates only `colony-nix` and `colony-persist`. This avoids checking and activating all
|
||||||
|
of the storage before switching root; `lvm-activate-main.service` activates the remaining LVs before
|
||||||
|
local filesystems and VMs need them.
|
||||||
|
|
||||||
|
### Backups
|
||||||
|
|
||||||
|
`my.borgthin` job `main` snapshots host and VM persistence/data LVs into `/mnt/backup/main`.
|
||||||
|
`borgthin-rsync.service` copies the repository to rsync.net and `rsync-lvm-meta.service` sends the
|
||||||
|
LVM metadata; both run at idle priority after the Borg job.
|
||||||
|
|
||||||
|
### Monitoring
|
||||||
|
|
||||||
|
Netdata uses FreeIPMI while ignoring the VCCM sensor. The box also runs `smartd` with logs under
|
||||||
|
`/var/log/smartd`, `rasdaemon`, and `fstrim` before the backup job.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: colony -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `colony`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| colony (vm) | internal | `94.142.241.224/32` | `2a0e:97c0:4d2:10::2/64` | ams1.int.nul.ie | |
|
|
||||||
| colony-routing | routing | `10.100.0.2/24 gw 10.100.0.1` | — | ams1.int.nul.ie | |
|
|
||||||
| colony-vms | vms | `10.100.1.1/24` | `2a0e:97c0:4d2:11::1/64` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
@@ -50,9 +75,8 @@ the VMs and provide them with storage, networking and backups.
|
|||||||
- `colony` sends RAs on `vms` (DNS = `estuary`'s base address) and carries
|
- `colony` sends RAs on `vms` (DNS = `estuary`'s base address) and carries
|
||||||
static routes for the downstream prefixes: `ctrs` via `shill`, `oci` via
|
static routes for the downstream prefixes: `ctrs` via `shill`, `oci` via
|
||||||
`whale2`, plus the Tailscale, `qclk` and `jam` prefixes via `shill`.
|
`whale2`, plus the Tailscale, `qclk` and `jam` prefixes via `shill`.
|
||||||
- `estuary` is the default gateway (via the `base` bridge); `colony`'s own
|
- `estuary` is the default gateway (via the `base` bridge); `colony`'s own public-facing address is
|
||||||
public-facing address is its `internal` assignment (a `vip1` /32, alt name
|
its `internal` assignment (alt name `vm`).
|
||||||
`vm`).
|
|
||||||
- The customer VMs attach to dedicated TAP devices (`vm-mail`, `vm-darts`)
|
- The customer VMs attach to dedicated TAP devices (`vm-mail`, `vm-darts`)
|
||||||
which are **not** bridged: networkd puts the point-to-point /32
|
which are **not** bridged: networkd puts the point-to-point /32
|
||||||
(`lib.my.c.colony.custRouting`) and the customer's IPv6 /64 on each, sends
|
(`lib.my.c.colony.custRouting`) and the customer's IPv6 /64 on each, sends
|
||||||
@@ -66,18 +90,18 @@ the VMs and provide them with storage, networking and backups.
|
|||||||
|
|
||||||
Declared in `my.vms.instances` (`cpus`/`threads` are QEMU `smp` values):
|
Declared in `my.vms.instances` (`cpus`/`threads` are QEMU `smp` values):
|
||||||
|
|
||||||
| VM | Cores | Threads | Memory | MAC | Disks |
|
| VM | Cores | Threads | Memory | Network | Disks |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| `estuary` | 2 | 2 | 3 GiB | `52:54:00:15:1a:53` (`base`) | `esp` / `nix` / `persist` LVs + WAN NIC passthrough |
|
| `estuary` | 2 | 2 | 3 GiB | `base` | `esp` / `nix` / `persist` LVs + WAN NIC passthrough |
|
||||||
| `shill` | 12 | 2 | 40 GiB | `52:54:00:27:3d:5c` (`vms`) | `esp` / `nix` / `persist` + `media` / `minio` / `nix-cache` / `jam` LVs |
|
| `shill` | 12 | 2 | 40 GiB | `vms` | `esp` / `nix` / `persist` + `media` / `minio` / `nix-cache` / `jam` LVs |
|
||||||
| `whale2` | 8 | 2 | 16 GiB | `52:54:00:d5:d9:c6` (`vms`) | `esp` / `nix` / `persist` + `oci` LV |
|
| `whale2` | 8 | 2 | 16 GiB | `vms` | `esp` / `nix` / `persist` + `oci` LV |
|
||||||
| `git` | 12 | 2 | 40 GiB | `52:54:00:75:78:a8` (`vms`) | `esp` / `nix` / `persist` / `oci` + `git` / `gitea-actions-cache` LVs |
|
| `git` | 12 | 2 | 40 GiB | `vms` | `esp` / `nix` / `persist` / `oci` + `git` / `gitea-actions-cache` LVs |
|
||||||
| `mail` | 3 | 2 | 6 GiB | `52:54:00:a8:d1:03` (`vm-mail` tap) | `root` / `data` LVs |
|
| `mail` | 3 | 2 | 6 GiB | `vm-mail` tap | `root` / `data` LVs |
|
||||||
| `darts` | 4 | 2 | 16 GiB | `52:54:00:a8:29:cd` (`vm-darts` tap) | `root` LV + `darts-media` / `darts-ext` LVs |
|
| `darts` | 4 | 2 | 16 GiB | `vm-darts` tap | `root` LV + `darts-media` / `darts-ext` LVs |
|
||||||
|
|
||||||
`estuary`, `shill`, `whale2` and `git` are NixOS systems with their own pages
|
`estuary`, `shill`, `whale2` and `git` are NixOS systems with their own pages
|
||||||
(see the [README](README.md#machines)); `mail` and `darts` are not (see
|
(see the [README](README.md#boxes)); [`mail`](mail.md) and [`darts`](darts.md) have host-side
|
||||||
[Non-NixOS VMs](README.md#non-nixos-vms)).
|
definitions here, but their guest operating systems are managed out of band.
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
@@ -9,19 +9,29 @@ and doesn't manage it.
|
|||||||
and the `90-vm-darts` network in
|
and the `90-vm-darts` network in
|
||||||
[`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
|
[`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
|
||||||
- **Host:** VM on `colony`
|
- **Host:** VM on `colony`
|
||||||
|
- **nixpkgs:** not applicable (unmanaged guest; host-side definition uses `colony`'s `mine-stable`)
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- Customer/dedicated VM, left alone beyond hosting and connectivity.
|
- Customer/dedicated VM, left alone beyond hosting and connectivity.
|
||||||
- 4 cores, 16 GiB RAM; a `root` LV plus `darts-media` and `darts-ext` LVs from
|
- A thin-provisioned `root` LV on the NVMe pool, plus `darts-media` (RAID 5 across the four HDDs)
|
||||||
the `media`/`ext` volume groups.
|
and `darts-ext` (linear storage on the 18 TB HDD) in the `main` VG. See
|
||||||
|
[`colony`'s storage layout](colony.md#storage).
|
||||||
|
|
||||||
## Networking
|
## Network assignments
|
||||||
|
|
||||||
|
This guest is not a NixOS system, so its host-routed addresses are not rows in the generated
|
||||||
|
[network assignments](../../networking.md#box-assignments) table.
|
||||||
|
|
||||||
- Same customer-VM pattern as [`mail`](mail.md): dedicated unbridged TAP
|
- Same customer-VM pattern as [`mail`](mail.md): dedicated unbridged TAP
|
||||||
(`vm-darts`, MAC `52:54:00:a8:29:cd`), point-to-point address
|
(`vm-darts`), point-to-point address
|
||||||
(`custRouting.darts-vm`) on the host side, link-routed public /32
|
(`custRouting.darts-vm`) on the host side, link-routed public /32
|
||||||
`94.142.242.255`, and the IPv6 /64 `2a0e:97c0:4d2:2001::/64` with RAs.
|
`94.142.242.255`, and the IPv6 /64 `2a0e:97c0:4d2:2001::/64` with RAs.
|
||||||
- DNS: `darts-cust.ams1.int.nul.ie`. Like the other customer prefixes, its
|
- DNS: `darts-cust.ams1.int.nul.ie`. Like the other customer prefixes, its
|
||||||
inbound traffic is accepted by `estuary` without per-port filtering and
|
inbound traffic is accepted by `estuary` without per-port filtering and
|
||||||
forwarded on by `colony`.
|
forwarded on by `colony`.
|
||||||
|
|
||||||
|
## Notable config files
|
||||||
|
|
||||||
|
- [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix) — VM definition.
|
||||||
|
- [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix) — host-side network and routing.
|
||||||
|
|||||||
@@ -10,30 +10,18 @@ and connects everything else at the site to the internet.
|
|||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **Edge routing / firewall / NAT:** owns the colony public IPv4/IPv6
|
- **Edge routing / firewall / NAT** — owns the colony public IPv4/IPv6 assignments, NATs outbound
|
||||||
(`94.142.240.44/24`, `2a02:898:0:20::329:1/64`), NATs outbound traffic, and
|
traffic, and port-forwards inbound services (see [Firewall and NAT](#firewall-and-nat)).
|
||||||
port-forwards inbound services (`my.firewall.nat.forwardPorts` driven by the
|
- **DNS** — PowerDNS authoritative server *and* recursor (see [DNS](#dns)).
|
||||||
shared `lib.my.c.colony.firewallForwards` list): HTTP/S and Matrix
|
- **BGP** — BIRD2 speaking AS211024 with upstreams, IXP route servers and
|
||||||
federation to `middleman`, git to `git`, game ports to the OCI servers on
|
direct peers (see [BGP](#bgp)).
|
||||||
`whale2` and to `gam`, Tailscale to `waffletail`, and the `qclk` WireGuard
|
- **VPNs** — the `as211024` L2 VXLAN mesh plus three point-to-point WireGuard
|
||||||
port.
|
tunnels (see [VPNs](#vpns)).
|
||||||
- **DNS:** PowerDNS authoritative server *and* recursor (see below).
|
- **Misc** — `iperf3` server, netdata.
|
||||||
- **BGP:** BIRD2 speaking AS211024 with upstreams, IXP route servers and
|
|
||||||
direct peers (see below).
|
|
||||||
- **VPNs:** member of the `as211024` L2 VXLAN mesh and endpoint for three
|
|
||||||
point-to-point WireGuard tunnels (see below).
|
|
||||||
- **Misc:** `iperf3` server, netdata.
|
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: estuary -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `estuary`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| estuary | as211024 | `10.100.50.1/24` | `2a0e:97c0:4df::1/64` | — | |
|
|
||||||
| estuary-vm-base | base | `10.100.0.1/24` | `2a0e:97c0:4d2:10::1/64` | ams1.int.nul.ie | |
|
|
||||||
| estuary-vm (fw) | internal | `94.142.240.44/24 gw 94.142.240.254` | `2a02:898:0:20::329:1/64 gw 2a02:898:0:20::1` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## WAN and IXP VLANs
|
## WAN and IXP VLANs
|
||||||
|
|
||||||
@@ -60,7 +48,18 @@ chain rejects non-IP/ARP ethertypes in both directions.
|
|||||||
|
|
||||||
## Firewall and NAT
|
## Firewall and NAT
|
||||||
|
|
||||||
`my.firewall` (nftables). Besides the port forwards, `extraRules` defines:
|
`my.firewall` (nftables). Inbound port forwards (`my.firewall.nat.forwardPorts`,
|
||||||
|
driven by the shared `lib.my.c.colony.firewallForwards` list):
|
||||||
|
|
||||||
|
| Service | Forwarded to |
|
||||||
|
|---|---|
|
||||||
|
| HTTP/S, Matrix federation | `middleman` |
|
||||||
|
| Git | `git` |
|
||||||
|
| Game servers | OCI servers on `whale2`, `gam` |
|
||||||
|
| Tailscale | `waffletail` |
|
||||||
|
| `qclk` WireGuard | `qclk` |
|
||||||
|
|
||||||
|
Besides the forwards, `extraRules` defines:
|
||||||
|
|
||||||
- `routing-tcp` / `routing-udp` chains — the inbound allow-list for new
|
- `routing-tcp` / `routing-udp` chains — the inbound allow-list for new
|
||||||
connections from `wan`/`as211024`/IXPs towards internal services (SSH
|
connections from `wan`/`as211024`/IXPs towards internal services (SSH
|
||||||
@@ -76,66 +75,88 @@ chain rejects non-IP/ARP ethertypes in both directions.
|
|||||||
|
|
||||||
## DNS
|
## DNS
|
||||||
|
|
||||||
Both halves are PowerDNS ([`dns.nix`](../../../nixos/boxes/colony/vms/estuary/dns.nix)):
|
Both halves are PowerDNS ([`dns.nix`](../../../nixos/boxes/colony/vms/estuary/dns.nix)). The live
|
||||||
|
forward and reverse records are listed in the generated [DNS reference](../../reference/dns.md).
|
||||||
|
|
||||||
- **Authoritative** (`my.pdns.auth`) listens on `0.0.0.0:5353` / `[::]:5353` as
|
### Authoritative
|
||||||
primary for `ams1.int.nul.ie`, `100.10.in-addr.arpa` and the
|
|
||||||
`2a0e:97c0:4d2::/48` reverse zone. Zone contents are largely generated from
|
`my.pdns.auth`, listening on `0.0.0.0:5353` / `[::]:5353`. Primary for
|
||||||
`allAssignments` (`lib.my.dns.fwdRecords` / `ptrRecords` / `ptr6Records`);
|
`ams1.int.nul.ie`, `100.10.in-addr.arpa` and the `2a0e:97c0:4d2::/48` reverse
|
||||||
`ALIAS` records (with `expand-alias`) point the zone apex at estuary itself.
|
zone.
|
||||||
AXFR is allowed to HE.net's secondary (`216.218.133.2` / `2001:470:600::2`),
|
|
||||||
and `_acme-challenge` is a LUA `TXT` record answered from a file (used for
|
- Zone contents are largely generated from `allAssignments`
|
||||||
DNS-01 issuance). Public DNS reaches it via the NAT redirect of port 53 to
|
(`lib.my.dns.fwdRecords` / `ptrRecords` / `ptr6Records`); `ALIAS` records
|
||||||
5353; the `base` side also accepts DNS directly.
|
(with `expand-alias`) point the zone apex at estuary itself.
|
||||||
- **Recursor** (`my.pdns.recursor`, `pdns-recursor`) listens on localhost and
|
- AXFR is allowed to HE.net's secondary and the trusted internal/site-egress sources used by the
|
||||||
the `base` addresses, serving `prefixes.all` and the Tailscale prefixes. The
|
generated DNS reference.
|
||||||
authoritative zones are forwarded back to `127.0.0.1:5353` (with NOTIFY
|
- `_acme-challenge` is a LUA `TXT` record answered from a file (DNS-01 issuance).
|
||||||
support so changes show up immediately), and a small Lua `preresolve` hook
|
- Reached publicly via the NAT redirect of port 53 → 5353; the `base` side also
|
||||||
rewrites `nix-cache.nul.ie` to `http.ams1.int.nul.ie` so cache traffic stays
|
accepts DNS directly.
|
||||||
on-site.
|
|
||||||
|
### Recursor
|
||||||
|
|
||||||
|
`my.pdns.recursor` (`pdns-recursor`), listening on localhost and the `base`
|
||||||
|
addresses, serving `prefixes.all` and the Tailscale prefixes.
|
||||||
|
|
||||||
|
- Authoritative zones are forwarded back to `127.0.0.1:5353` (with NOTIFY
|
||||||
|
support, so changes show up immediately).
|
||||||
|
- A small Lua `preresolve` hook rewrites `nix-cache.nul.ie` →
|
||||||
|
`http.ams1.int.nul.ie` so cache traffic stays on-site.
|
||||||
|
|
||||||
## BGP
|
## BGP
|
||||||
|
|
||||||
BIRD2 ([`bgp.nix`](../../../nixos/boxes/colony/vms/estuary/bgp.nix)) speaking **AS211024**:
|
BIRD2 ([`bgp.nix`](../../../nixos/boxes/colony/vms/estuary/bgp.nix)) speaking **AS211024**:
|
||||||
|
|
||||||
- **Upstreams:** ColoClue (AS8283, `euNetworks` 2/3, v4+v6); iFog IPv6 transit
|
| Peer | ASN | Role | Where / notes |
|
||||||
(AS34927); Hurricane Electric IPv6 over Frys-IX (AS6939).
|
|---|---|---|---|
|
||||||
- **IXP route servers:** Frys-IX (AS56393), NL-ix (AS34307, depref'd by 1),
|
| ColoClue | AS8283 | Upstream | `euNetworks` 2/3, v4+v6 |
|
||||||
FogIXP (AS47498).
|
| iFog | AS34927 | Upstream | IPv6 transit |
|
||||||
- **Direct peers:** LUJE.net (AS212855, on ColoClue/Frys-IX/FogIXP + multihop
|
| Hurricane Electric | AS6939 | Upstream | IPv6 over Frys-IX |
|
||||||
labs), jurrian (AS212635), Meta (AS32934, Frys-IX/NL-ix), Cloudflare
|
| Frys-IX | AS56393 | IXP route server | |
|
||||||
(AS13335, Frys-IX), Apple (AS714, NL-ix), HE (AS6939, Frys-IX v4).
|
| NL-ix | AS34307 | IXP route server | lower preference |
|
||||||
- **Monitoring:** a multihop session to the bgp.tools collector (AS212232)
|
| FogIXP | AS47498 | IXP route server | |
|
||||||
exporting everything with `add paths`.
|
| LUJE.net | AS212855 | Direct peer | ColoClue/Frys-IX/FogIXP + multihop labs |
|
||||||
- Statics originate the site prefixes: the `vip*` ranges via `base`, the
|
| jurrian | AS212635 | Direct peer | |
|
||||||
ams1/internal v6 prefixes via `base`/`as211024`, and the home /48 towards
|
| Meta | AS32934 | Direct peer | Frys-IX/NL-ix |
|
||||||
the home routers' `as211024` VIP.
|
| Cloudflare | AS13335 | Direct peer | Frys-IX |
|
||||||
|
| Apple | AS714 | Direct peer | NL-ix |
|
||||||
|
| HE | AS6939 | Direct peer | Frys-IX (v4) |
|
||||||
|
| bgp.tools | AS212232 | Monitoring | multihop collector, `add paths` |
|
||||||
|
|
||||||
|
Statics originate the site prefixes: the `vip*` ranges via `base`, the
|
||||||
|
ams1/internal v6 prefixes via `base`/`as211024`, and the home /48 towards the
|
||||||
|
home routers' `as211024` VIP.
|
||||||
|
|
||||||
Currently disabled (commented out): efero transit over FogIXP ("not working so
|
Currently disabled (commented out): efero transit over FogIXP ("not working so
|
||||||
well lately") and the NL-ix Cloudflare sessions.
|
well lately") and the NL-ix Cloudflare sessions.
|
||||||
|
|
||||||
## VPNs
|
## VPNs
|
||||||
|
|
||||||
- **`as211024` L2 mesh** (`my.vpns.l2`, the `l2mesh` module): VXLAN (VNI
|
### `as211024` L2 mesh
|
||||||
211024, UDP-encapsulated) secured with libreswan IPsec, meshing estuary with
|
|
||||||
the home routers `river`/`stream` and `britway`. This carries the AS211024
|
Member alongside `river`/`stream`/`britway` (`my.vpns.l2`, the `l2mesh` module).
|
||||||
anycast-ish internal address space between sites.
|
The mesh transport, crypto and addressing are shared fabric — see
|
||||||
- **WireGuard endpoints** (networkd `wireguard` netdevs, keys from agenix):
|
[The AS211024 L2 mesh](../../networking.md#the-as211024-l2-mesh) in networking.md.
|
||||||
- `kelder` — tunnel to the remote `kelder` site, port `51820`.
|
|
||||||
- `hillcrest` — port `51822`, point-to-point /32 pair out of
|
### WireGuard tunnels
|
||||||
`prefixes.hillcrest.v4`.
|
|
||||||
- `john-valorant` — port `51823`, same pattern out of
|
Point-to-point tunnels terminated here as networkd `wireguard` netdevs (private
|
||||||
`prefixes.john-valorant.v4`.
|
keys from agenix); each SNATs out its own interface address:
|
||||||
|
|
||||||
|
| Tunnel | Port | Prefix | Notes |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `kelder` | 51820 | — | to the remote `kelder` site; kelder's public `estuary` assignment is routed over the tunnel |
|
||||||
|
| `hillcrest` | 51822 | `prefixes.hillcrest.v4` (`10.100.5.0/30`) | /32 pair, estuary `.1` ↔ remote `.2` |
|
||||||
|
| `john-valorant` | 51823 | `prefixes.john-valorant.v4` (`10.100.5.4/30`) | /32 pair, estuary `.1` ↔ remote `.2` |
|
||||||
|
|
||||||
## Bandwidth management
|
## Bandwidth management
|
||||||
|
|
||||||
[`bandwidth.nix`](../../../nixos/boxes/colony/vms/estuary/bandwidth.nix)
|
[`bandwidth.nix`](../../../nixos/boxes/colony/vms/estuary/bandwidth.nix) implements a WAN shaper: a
|
||||||
implements a ~95% WAN shaper: a 245 Mbit token-bucket filter on `wan`
|
token-bucket filter on `wan` (outbound) and on an `ifb-wan` IFB device that ingress traffic is
|
||||||
(outbound) and on an `ifb-wan` IFB device that ingress traffic is mirrored
|
mirrored into (inbound), with
|
||||||
into (inbound), with [`bandwidth.py`](../../../nixos/boxes/colony/vms/estuary/bandwidth.py)
|
[`bandwidth.py`](../../../nixos/boxes/colony/vms/estuary/bandwidth.py) as a `bandwidth-limiter`
|
||||||
as a `bandwidth-limiter` service that watches/utilises the link and can adjust
|
service that watches utilisation and can adjust the configured rate. **Currently disabled** — the
|
||||||
the rate. **Currently disabled** — the file is not in estuary's `imports`
|
file is not in estuary's `imports` (only `dns.nix` and `bgp.nix` are), so no shaping is applied.
|
||||||
(only `dns.nix` and `bgp.nix` are), so no shaping is applied.
|
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
+32
-28
@@ -9,38 +9,42 @@ The Gitea VM — source hosting and CI for the boxes (`git.nul.ie`).
|
|||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **Gitea** ([`gitea.nix`](../../../nixos/boxes/colony/vms/git/gitea.nix)) — the
|
### Gitea
|
||||||
Git forge at `git.nul.ie` (self-registration disabled). Backed by PostgreSQL
|
|
||||||
on `colony-psql` (waiting on it via `lib.my.systemdAwaitPostgres`), LFS
|
The Git forge at `git.nul.ie` (self-registration disabled), configured in
|
||||||
enabled, with all object storage (incl. LFS and packages) on MinIO at
|
[`gitea.nix`](../../../nixos/boxes/colony/vms/git/gitea.nix).
|
||||||
`s3.nul.ie` (bucket `gitea`, on `object`; the secret is spliced into
|
|
||||||
`app.ini` at startup). Mail goes out via `mail.nul.ie`, including the
|
- Backed by PostgreSQL on [`colony-psql`](shill/containers/colony-psql.md) (waited on via
|
||||||
issue-reply incoming-mail poller.
|
`lib.my.systemdAwaitPostgres`).
|
||||||
- **Gitea Actions runner**
|
- LFS enabled; all object storage (incl. LFS and packages) is on MinIO at `s3.nul.ie` (bucket
|
||||||
([`gitea-actions.nix`](../../../nixos/boxes/colony/vms/git/gitea-actions.nix)) —
|
`gitea`, on [`object`](shill/containers/object.md)) — the secret is spliced into `app.ini` at
|
||||||
one Docker-mode instance (`main-docker`) on podman (privileged, `podman`
|
startup.
|
||||||
network), with labels for `node:24-trixie` and the self-built
|
- Mail goes out via `mail.nul.ie`, including the issue-reply incoming-mail poller.
|
||||||
`git.nul.ie/dev/actions-ubuntu:26.04` images. Runs as a fixed
|
|
||||||
`gitea-runner` user (not `DynamicUser`) so it can read its token; 8 h job
|
### Gitea Actions runner
|
||||||
timeout; the action cache lives on a dedicated disk
|
|
||||||
(`/var/cache/gitea-runner`). This runner executes the repo's own
|
One Docker-mode instance (`main-docker`) on podman (privileged, `podman` network), configured in
|
||||||
`.gitea/workflows/ci.yaml`.
|
[`gitea-actions.nix`](../../../nixos/boxes/colony/vms/git/gitea-actions.nix).
|
||||||
- **nginx** — terminates TLS for `git.nul.ie` (and a default vhost) and
|
|
||||||
proxies to Gitea on `:3000`. ACME (Let's Encrypt, production) issues
|
- Labels for `node:24-trixie` and the self-built `git.nul.ie/dev/actions-ubuntu:26.04` images.
|
||||||
`nul.ie` + `*.nul.ie` via the Cloudflare DNS-01 challenge.
|
- Runs as a fixed `gitea-runner` user (not `DynamicUser`) so it can read its token; jobs have a
|
||||||
- **podman** — local container backend for the runner;
|
configured timeout.
|
||||||
`/var/lib/containers` is an XFS data disk, and the default
|
- The action cache lives on a dedicated disk (`/var/cache/gitea-runner`).
|
||||||
|
- Executes the repo's own `.gitea/workflows/ci.yaml`.
|
||||||
|
|
||||||
|
### nginx
|
||||||
|
|
||||||
|
Terminates TLS for `git.nul.ie` (and a default vhost) and proxies to Gitea on `:3000`. ACME
|
||||||
|
(Let's Encrypt, production) issues `nul.ie` + `*.nul.ie` via the Cloudflare DNS-01 challenge.
|
||||||
|
|
||||||
|
### podman
|
||||||
|
|
||||||
|
Local container backend for the runner; `/var/lib/containers` is an XFS data disk, and the default
|
||||||
`10.88.0.0/16` podman subnet is allowed to forward.
|
`10.88.0.0/16` podman subnet is allowed to forward.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: git -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `git`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| git-vm | internal | `94.142.241.117/32` | `2a0e:97c0:4d2:11::4/64` | ams1.int.nul.ie | |
|
|
||||||
| git-vm-routing | routing | `10.100.1.4/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Storage
|
## Storage
|
||||||
|
|
||||||
|
|||||||
@@ -9,22 +9,26 @@ a NixOS system**: everything inside the VM is configured out of band.
|
|||||||
and the `90-vm-mail` network in
|
and the `90-vm-mail` network in
|
||||||
[`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
|
[`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix)
|
||||||
- **Host:** VM on `colony`
|
- **Host:** VM on `colony`
|
||||||
|
- **nixpkgs:** not applicable (Debian guest; host-side definition uses `colony`'s `mine-stable`)
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- Runs the full mailcow stack (Postfix/Dovecot/SOGo/Rspamd) for `nul.ie`.
|
- Runs the full mailcow stack (Postfix/Dovecot/SOGo/Rspamd) for `nul.ie`.
|
||||||
Other colony services send through it as `mail.nul.ie` (e.g. Gitea, and the
|
Other colony services send through it as `mail.nul.ie` (e.g. Gitea, and the
|
||||||
disabled Mastodon config).
|
disabled Mastodon config).
|
||||||
- 3 cores, 6 GiB RAM; `root` and `data` LVM disks (the `vm-mail-data` LV is
|
- `root` and `data` LVM disks; the `vm-mail-data` LV is included in `colony`'s `borgthin` backups.
|
||||||
included in `colony`'s `borgthin` backups).
|
|
||||||
|
|
||||||
## Networking
|
## Network assignments
|
||||||
|
|
||||||
|
This guest is not a NixOS system, so its host-routed addresses are not rows in the generated
|
||||||
|
[network assignments](../../networking.md#box-assignments) table.
|
||||||
|
|
||||||
|
### Link and addressing
|
||||||
|
|
||||||
|
The VM attaches to a dedicated, unbridged TAP (`vm-mail`). `colony` puts the point-to-point
|
||||||
|
`custRouting.mail-vm` address on the host side, link-routes public
|
||||||
|
`94.142.241.227/32` down the TAP, and advertises `2a0e:97c0:4d2:2000::/64` with RAs.
|
||||||
|
|
||||||
- Attaches to a dedicated, unbridged TAP (`vm-mail`, MAC
|
|
||||||
`52:54:00:a8:d1:03`). `colony` puts the point-to-point address
|
|
||||||
(`custRouting.mail-vm`) on the host side, link-routes the public /32
|
|
||||||
`94.142.241.227` down the tap, and serves the IPv6 /64
|
|
||||||
`2a0e:97c0:4d2:2000::/64` with RAs.
|
|
||||||
- DNS: `mail-vm.ams1.int.nul.ie` (and `mail.nul.ie` publicly, incl. the PTR in
|
- DNS: `mail-vm.ams1.int.nul.ie` (and `mail.nul.ie` publicly, incl. the PTR in
|
||||||
estuary's reverse zone). `estuary` accepts traffic to the customer prefixes
|
estuary's reverse zone). `estuary` accepts traffic to the customer prefixes
|
||||||
without per-port filtering; `colony` forwards it on ("trust for now").
|
without per-port filtering; `colony` forwards it on ("trust for now").
|
||||||
@@ -34,3 +38,8 @@ a NixOS system**: everything inside the VM is configured out of band.
|
|||||||
- ACME certificates are issued on `middleman` and pushed to the VM over SSH
|
- ACME certificates are issued on `middleman` and pushed to the VM over SSH
|
||||||
(`acme@mail.nul.ie mailcow-ssl-reload`, key `middleman/mailcow-ssh.key`);
|
(`acme@mail.nul.ie mailcow-ssl-reload`, key `middleman/mailcow-ssh.key`);
|
||||||
the VM's SSH host key is pinned at `.keys/mail-vm-host.pub`.
|
the VM's SSH host key is pinned at `.keys/mail-vm-host.pub`.
|
||||||
|
|
||||||
|
## Notable config files
|
||||||
|
|
||||||
|
- [`nixos/boxes/colony/vms/default.nix`](../../../nixos/boxes/colony/vms/default.nix) — VM definition.
|
||||||
|
- [`nixos/boxes/colony/default.nix`](../../../nixos/boxes/colony/default.nix) — host-side network and routing.
|
||||||
|
|||||||
@@ -1,83 +0,0 @@
|
|||||||
# shill
|
|
||||||
|
|
||||||
The colony NixOS container host — most colony applications run as
|
|
||||||
`systemd-nspawn` containers on this VM.
|
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/colony/vms/shill/`](../../../nixos/boxes/colony/vms/shill)
|
|
||||||
(`default.nix`, `containers-ext.nix`, `containers/`)
|
|
||||||
- **Host:** VM on `colony` (large: 12 cores, 40 GiB RAM)
|
|
||||||
- **nixpkgs:** `mine`
|
|
||||||
|
|
||||||
## Role
|
|
||||||
|
|
||||||
- Runs the colony containers via `my.containers.instances`, each attached to
|
|
||||||
the `ctrs` bridge with its own address. The containers are full NixOS
|
|
||||||
systems rendered via `my.asContainer` and deployed as container profiles on
|
|
||||||
`shill` (`my.deploy.enable = false` — they are not standalone deploy
|
|
||||||
targets); the `my.containers` module wires up the nspawn units, `/persist`
|
|
||||||
bind mounts and store binds.
|
|
||||||
- Provides shared data volumes to containers via bind mounts from LVM-backed
|
|
||||||
disks: `/mnt/media` (→ `middleman` read-only, `jackflix` read-write),
|
|
||||||
`/mnt/minio` and `/mnt/nix-cache` (→ `object`, both read-write).
|
|
||||||
- Routes between the `vms` network and the `ctrs` container network: sends RAs
|
|
||||||
on `ctrs` (DNS = `estuary`'s base address) and routes the Tailscale prefixes
|
|
||||||
via `waffletail` and the `qclk` prefix via `qclk`. Applies the shared
|
|
||||||
`firewallForwards` DNAT for traffic addressed to `estuary`'s public IP, with
|
|
||||||
an nftables `ct mark 0x1337` SNAT hack so forwarded return traffic stays
|
|
||||||
symmetric.
|
|
||||||
- Tuned for high connection counts (larger conntrack table, wider ephemeral
|
|
||||||
port range); netdata on 19999.
|
|
||||||
|
|
||||||
## Network assignments
|
|
||||||
|
|
||||||
<!-- assignments: shill -->
|
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| shill-vm-ctrs | ctrs | `10.100.2.1/24` | `2a0e:97c0:4d2:12::1/64` | ams1.int.nul.ie | |
|
|
||||||
| shill-vm (ctr) | internal | `94.142.241.225/32` | `2a0e:97c0:4d2:11::2/64` | ams1.int.nul.ie | |
|
|
||||||
| shill-vm-routing | routing | `10.100.1.2/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Containers
|
|
||||||
|
|
||||||
Defined under
|
|
||||||
[`shill/containers/`](../../../nixos/boxes/colony/vms/shill/containers) and
|
|
||||||
wired up in `shill`'s `my.containers.instances`. Each has its own page:
|
|
||||||
|
|
||||||
| Container | IPv4 | IPv6 | Role | Page |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| `middleman` | `10.100.2.2` | `2a0e:97c0:4d2:12::2` | Reverse proxy, ACME, nginx-sso, librespeed | [middleman](shill/containers/middleman.md) |
|
|
||||||
| `vaultwarden` | `10.100.2.3` | `2a0e:97c0:4d2:12::3` | Password manager | [vaultwarden](shill/containers/vaultwarden.md) |
|
|
||||||
| `colony-psql` | `10.100.2.4` | `2a0e:97c0:4d2:12::4` | Shared PostgreSQL (14) | [colony-psql](shill/containers/colony-psql.md) |
|
|
||||||
| `chatterbox` | `10.100.2.5` | `2a0e:97c0:4d2:12::5` | Matrix Synapse + bridges | [chatterbox](shill/containers/chatterbox.md) |
|
|
||||||
| `jackflix` | `10.100.2.6` | `2a0e:97c0:4d2:12::6` | Media stack | [jackflix](shill/containers/jackflix.md) |
|
|
||||||
| `object` | `10.100.2.7` | `2a0e:97c0:4d2:12::7` | MinIO, Harmonia Nix cache, Sharry, HedgeDoc, wastebin | [object](shill/containers/object.md) |
|
|
||||||
| `toot` | `10.100.2.8` | `2a0e:97c0:4d2:12::8` | Bluesky PDS (Mastodon disabled) | [toot](shill/containers/toot.md) |
|
|
||||||
| `waffletail` | `10.100.2.9` | `2a0e:97c0:4d2:12::9` | Tailscale subnet router / exit node | [waffletail](shill/containers/waffletail.md) |
|
|
||||||
| `qclk` | `10.100.2.10` | `2a0e:97c0:4d2:12::a` | WireGuard management appliance | [qclk](shill/containers/qclk.md) |
|
|
||||||
| `gam` | `10.100.2.11` | `2a0e:97c0:4d2:12::b` | Terraria server | [gam](shill/containers/gam.md) |
|
|
||||||
|
|
||||||
### `jam`
|
|
||||||
|
|
||||||
A one-off: [`containers-ext.nix`](../../../nixos/boxes/colony/vms/shill/containers-ext.nix)
|
|
||||||
runs a raw `systemd-nspawn` container (not a `my.containers` instance, not
|
|
||||||
NixOS) with its root on the `jam` LV, private user namespaces and a `ve-jam`
|
|
||||||
veth. It gets the `jam` customer prefix (`prefixes.jam`, `jam-cust` in DNS)
|
|
||||||
and SSH is forwarded to it from `shill`'s public IP port 60022.
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- `nix.settings.substituters` is forced to just `https://cache.nixos.org` —
|
|
||||||
`shill` sits next to the S3 cache on `object`, so it doesn't use it.
|
|
||||||
- [`hercules.nix`](../../../nixos/boxes/colony/vms/shill/hercules.nix)
|
|
||||||
(Hercules CI agent + the `nix-cache-gc` timer for the S3 binary cache)
|
|
||||||
exists but is **currently disabled**: the file is not imported by
|
|
||||||
`shill/default.nix`.
|
|
||||||
|
|
||||||
## Notable config files
|
|
||||||
|
|
||||||
- [`nixos/boxes/colony/vms/shill/default.nix`](../../../nixos/boxes/colony/vms/shill/default.nix) — VM config, networkd, firewall, `my.containers.instances`.
|
|
||||||
- [`nixos/boxes/colony/vms/shill/containers/default.nix`](../../../nixos/boxes/colony/vms/shill/containers/default.nix) — container imports.
|
|
||||||
- [`nixos/boxes/colony/vms/shill/containers-ext.nix`](../../../nixos/boxes/colony/vms/shill/containers-ext.nix) — the `jam` nspawn container.
|
|
||||||
- [`nixos/modules/containers.nix`](../../../nixos/modules/containers.nix) — the `my.containers` module.
|
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
# shill
|
||||||
|
|
||||||
|
The colony NixOS container host — most colony applications run as
|
||||||
|
`systemd-nspawn` containers on this VM.
|
||||||
|
|
||||||
|
- **Source:** [`nixos/boxes/colony/vms/shill/`](../../../../nixos/boxes/colony/vms/shill)
|
||||||
|
(`default.nix`, `containers-ext.nix`, `containers/`)
|
||||||
|
- **Host:** VM on `colony`
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
|
## Role
|
||||||
|
|
||||||
|
### Container hosting
|
||||||
|
|
||||||
|
`my.containers.instances` runs the colony containers on the `ctrs` bridge. Each is a full NixOS
|
||||||
|
system rendered through `my.asContainer` and deployed as a profile on `shill`; containers are not
|
||||||
|
standalone deploy targets. The shared container module supplies the nspawn units, `/persist` and
|
||||||
|
store binds.
|
||||||
|
|
||||||
|
### Shared storage
|
||||||
|
|
||||||
|
LVM-backed host volumes are bind-mounted into the consumers: `/mnt/media` is read-only in
|
||||||
|
`middleman` and read-write in `jackflix`; `/mnt/minio` and `/mnt/nix-cache` are read-write in
|
||||||
|
`object`.
|
||||||
|
|
||||||
|
### Routing
|
||||||
|
|
||||||
|
`shill` routes between `vms` and `ctrs`, advertises `estuary` as DNS on `ctrs`, and routes Tailscale
|
||||||
|
through `waffletail` and the `qclk` prefix through `qclk`. It applies the shared `firewallForwards`
|
||||||
|
DNAT for `estuary`'s public IP; a connection-mark-based SNAT rule keeps replies symmetric.
|
||||||
|
|
||||||
|
### Host tuning
|
||||||
|
|
||||||
|
The box has a larger conntrack table and ephemeral-port range for high connection counts. Netdata
|
||||||
|
listens on port 19999.
|
||||||
|
|
||||||
|
## Network assignments
|
||||||
|
|
||||||
|
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `shill`).
|
||||||
|
|
||||||
|
## Containers
|
||||||
|
|
||||||
|
Defined under
|
||||||
|
[`shill/containers/`](../../../../nixos/boxes/colony/vms/shill/containers) and
|
||||||
|
wired up in `shill`'s `my.containers.instances`. The generated
|
||||||
|
[network assignments](../../../networking.md#box-assignments) table is the source of truth for
|
||||||
|
their current addresses. Each container has its own page:
|
||||||
|
|
||||||
|
| Container | Role |
|
||||||
|
|---|---|
|
||||||
|
| [`middleman`](containers/middleman.md) | Reverse proxy, ACME, nginx-sso, librespeed |
|
||||||
|
| [`vaultwarden`](containers/vaultwarden.md) | Password manager |
|
||||||
|
| [`colony-psql`](containers/colony-psql.md) | Shared PostgreSQL (14) |
|
||||||
|
| [`chatterbox`](containers/chatterbox.md) | Matrix Synapse + bridges |
|
||||||
|
| [`jackflix`](containers/jackflix.md) | Media stack |
|
||||||
|
| [`object`](containers/object.md) | MinIO, Harmonia Nix cache, Sharry, HedgeDoc, wastebin |
|
||||||
|
| [`toot`](containers/toot.md) | Bluesky PDS (Mastodon disabled) |
|
||||||
|
| [`waffletail`](containers/waffletail.md) | Tailscale subnet router / exit node |
|
||||||
|
| [`qclk`](containers/qclk.md) | WireGuard management appliance |
|
||||||
|
| [`gam`](containers/gam.md) | Terraria server |
|
||||||
|
|
||||||
|
### `jam`
|
||||||
|
|
||||||
|
A one-off: [`containers-ext.nix`](../../../../nixos/boxes/colony/vms/shill/containers-ext.nix)
|
||||||
|
runs a raw `systemd-nspawn` container (not a `my.containers` instance, not
|
||||||
|
NixOS) with its root on the `jam` LV, private user namespaces and a `ve-jam`
|
||||||
|
veth. It gets the `jam` customer prefix (`prefixes.jam`, `jam-cust` in DNS)
|
||||||
|
and SSH is forwarded to it from `shill`'s public IP port 60022.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- `nix.settings.substituters` is forced to just `https://cache.nixos.org` —
|
||||||
|
`shill` sits next to the S3 cache on `object`, so it doesn't use it.
|
||||||
|
- [`hercules.nix`](../../../../nixos/boxes/colony/vms/shill/hercules.nix)
|
||||||
|
(Hercules CI agent + the `nix-cache-gc` timer for the S3 binary cache)
|
||||||
|
exists but is **currently disabled**: the file is not imported by
|
||||||
|
`shill/default.nix`.
|
||||||
|
|
||||||
|
## Notable config files
|
||||||
|
|
||||||
|
- [`nixos/boxes/colony/vms/shill/default.nix`](../../../../nixos/boxes/colony/vms/shill/default.nix) — VM config, networkd, firewall, `my.containers.instances`.
|
||||||
|
- [`nixos/boxes/colony/vms/shill/containers/default.nix`](../../../../nixos/boxes/colony/vms/shill/containers/default.nix) — container imports.
|
||||||
|
- [`nixos/boxes/colony/vms/shill/containers-ext.nix`](../../../../nixos/boxes/colony/vms/shill/containers-ext.nix) — the `jam` nspawn container.
|
||||||
|
- [`nixos/modules/containers.nix`](../../../../nixos/modules/containers.nix) — the `my.containers` module.
|
||||||
@@ -5,32 +5,28 @@ The Matrix homeserver for `nul.ie` (Synapse) and its bridges to other chat netwo
|
|||||||
federation.
|
federation.
|
||||||
|
|
||||||
- **Source:** [`shill/containers/chatterbox.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/chatterbox.nix)
|
- **Source:** [`shill/containers/chatterbox.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/chatterbox.nix)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md)
|
- **Host:** NixOS container on [`shill`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **matrix-synapse** — `server_name = "nul.ie"`, `public_baseurl = https://matrix.nul.ie`,
|
### Synapse
|
||||||
Element at `element.nul.ie` as the web client. Listens on `[::]:8008` (client + federation
|
|
||||||
resources, `x_forwarded`) with a localhost manhole on `:9000`. Registration and guest access
|
`matrix-synapse` serves `nul.ie` at `https://matrix.nul.ie`, with Element at `element.nul.ie`.
|
||||||
are disabled; uploads up to 1024M with dynamic thumbnails and URL previews enabled (previews
|
Port 8008 carries client and federation traffic with forwarded headers, while a localhost manhole
|
||||||
are limited to [middleman](middleman.md)'s addresses as the fetch proxy).
|
uses port 9000. Registration and guest access are disabled; uploads, dynamic thumbnails and URL
|
||||||
- **heisenbridge** — IRC bridge, owner `@dev:nul.ie`, exclusive `@irc_*` user namespace.
|
previews are enabled, with preview fetching restricted to [middleman](middleman.md).
|
||||||
- **mautrix-whatsapp** — WhatsApp bridge (appservice `whatsapp2`, `!wa` commands).
|
|
||||||
- **mautrix-meta** — two instances, `messenger` (`fbm2_*`, `!fbm`) and `instagram` (`ig_*`,
|
### Bridges
|
||||||
`!ig`), both with backfill enabled.
|
|
||||||
- All three mautrix bridges use Postgres on [colony-psql](colony-psql.md) (URIs in their secret
|
The box runs heisenbridge for IRC, `mautrix-whatsapp`, and two `mautrix-meta` instances for
|
||||||
env files), require end-to-end encryption by default, and double-puppet onto `nul.ie` via the
|
Messenger and Instagram. The mautrix bridges use [colony-psql](colony-psql.md), default to
|
||||||
shared `doublepuppet.yaml` appservice registration (an age secret).
|
end-to-end encryption, and share the secret `doublepuppet.yaml` registration for double puppeting
|
||||||
- The firewall allows `8008` (Synapse) and `8009` besides netdata.
|
onto `nul.ie`. The firewall admits ports 8008 and 8009 in addition to netdata.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: chatterbox -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `chatterbox`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| chatterbox-ctr | internal | `10.100.2.5/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::5/64` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
|
|||||||
@@ -4,12 +4,13 @@ The shared PostgreSQL instance for colony services. Rather than each service run
|
|||||||
database, the containers (and the `git` VM) connect here over the `ctrs` network.
|
database, the containers (and the `git` VM) connect here over the `ctrs` network.
|
||||||
|
|
||||||
- **Source:** [`shill/containers/colony-psql.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/colony-psql.nix)
|
- **Source:** [`shill/containers/colony-psql.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/colony-psql.nix)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md)
|
- **Host:** NixOS container on [`shill`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **PostgreSQL 14** with TCP/IP enabled, reachable from the whole colony (`10.100.0.0/16` and
|
- **PostgreSQL 14** with TCP/IP enabled, reachable from the whole colony address space using `md5`
|
||||||
`2a0e:97c0:4d2:10::/60`, `md5` auth). The firewall allows `5432`.
|
authentication. The firewall allows `5432`.
|
||||||
- Local `peer` auth maps `postgres`, `root`, `netdata` and `dev` to the `postgres` superuser via
|
- Local `peer` auth maps `postgres`, `root`, `netdata` and `dev` to the `postgres` superuser via
|
||||||
the ident map.
|
the ident map.
|
||||||
- **netdata** with the Python PostgreSQL collector.
|
- **netdata** with the Python PostgreSQL collector.
|
||||||
@@ -19,12 +20,7 @@ database, the containers (and the `git` VM) connect here over the `ctrs` network
|
|||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: colony-psql -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `colony-psql`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| colony-psql-ctr (colony-psql) | internal | `10.100.2.4/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::4/64` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
The assignment also has the alt name `colony-psql` (no `-ctr` suffix), which is what consumers
|
The assignment also has the alt name `colony-psql` (no `-ctr` suffix), which is what consumers
|
||||||
use as the database hostname.
|
use as the database hostname.
|
||||||
|
|||||||
@@ -1,26 +1,28 @@
|
|||||||
# gam
|
# gam
|
||||||
|
|
||||||
A game-server container — the lightweight counterpart to the OCI game servers on `whale2`,
|
A game-server container currently dedicated to Terraria.
|
||||||
running servers directly as NixOS services. Currently it runs a single Terraria server.
|
|
||||||
|
|
||||||
- **Source:** [`shill/containers/gam.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/gam.nix)
|
- **Source:** [`shill/containers/gam.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/gam.nix)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md)
|
- **Host:** NixOS container on [`shill`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **terraria** — dedicated server (`services.terraria`): world at
|
Runs lightweight game servers directly as NixOS services, rather than as OCI containers on
|
||||||
`/var/lib/terraria/NotWorld.wld`, auto-created large worlds, MOTD "sup gamers", UPnP off.
|
[`whale2`](../../whale2.md).
|
||||||
Extra settings (e.g. password) come from the `gam/terraria.conf` age secret used as the config
|
|
||||||
file. `openFirewall` is on, and `estuary` port-forwards TCP and UDP `7777` to this container.
|
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: gam -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `gam`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
## Terraria server
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| gam-ctr | internal | `10.100.2.11/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::b/64` | ams1.int.nul.ie | |
|
`services.terraria` runs the dedicated server with its world at
|
||||||
<!-- assignments-end -->
|
`/var/lib/terraria/NotWorld.wld`. It creates large worlds automatically, uses the MOTD
|
||||||
|
"sup gamers", and disables UPnP. Additional settings such as the password come from the
|
||||||
|
`gam/terraria.conf` age secret used as the service configuration file.
|
||||||
|
|
||||||
|
`openFirewall` is enabled, and `estuary` forwards TCP and UDP port `7777` to the container.
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
@@ -5,7 +5,8 @@ an AirVPN WireGuard tunnel so downloads only flow while the VPN is up.
|
|||||||
|
|
||||||
- **Source:** [`shill/containers/jackflix/`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix)
|
- **Source:** [`shill/containers/jackflix/`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix)
|
||||||
(`default.nix`, `networking.nix`)
|
(`default.nix`, `networking.nix`)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md) (bind-mounts `/mnt/media` read-write)
|
- **Host:** NixOS container on [`shill`](../README.md) (bind-mounts `/mnt/media` read-write)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
@@ -21,40 +22,33 @@ an AirVPN WireGuard tunnel so downloads only flow while the VPN is up.
|
|||||||
| PhotoPrism | `2342` | photos, `photos.nul.ie`; password auth, sqlite DB, originals/import under `/mnt/media/photoprism` |
|
| PhotoPrism | `2342` | photos, `photos.nul.ie`; password auth, sqlite DB, originals/import under `/mnt/media/photoprism` |
|
||||||
| copyparty | `3923` | file sharing, `stuff.nul.ie`; serves `/mnt/media/public` (read-only to everyone) and `/priv` → `/mnt/media/stuff` (admin for `dev`), share creation, indexing (`e2dsa`/`e2t`), file-magic checks |
|
| copyparty | `3923` | file sharing, `stuff.nul.ie`; serves `/mnt/media/public` (read-only to everyone) and `/priv` → `/mnt/media/stuff` (admin for `dev`), share creation, indexing (`e2dsa`/`e2t`), file-magic checks |
|
||||||
|
|
||||||
All published through [middleman](middleman.md) as shown. A shared `media` group (gid 2000) plus
|
All published through [middleman](middleman.md) as shown. A shared `media` group plus a group-writable
|
||||||
`UMask=0002` on Radarr/Sonarr gives the apps coordinated access to the media volume.
|
umask on Radarr/Sonarr gives the apps coordinated access to the media volume.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: jackflix -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `jackflix`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| jackflix-ctr | internal | `10.100.2.6/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::6/64` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## VPN download path
|
## VPN download path
|
||||||
|
|
||||||
[`networking.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix/networking.nix)
|
[`networking.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/jackflix/networking.nix)
|
||||||
defines a `vpn` WireGuard netdev to **AirVPN NL** (key + PSK from age secrets, MTU 1320 per
|
defines a `vpn` WireGuard netdev to **AirVPN NL** using a key and PSK from age secrets:
|
||||||
AirVPN, fwmark `42`, route table `51820`):
|
|
||||||
|
|
||||||
- Policy routing keeps colony traffic on the main table (from/to `prefixes.all` v4/v6 rules at
|
- Policy routing keeps colony traffic on the main table while everything else falls through to the
|
||||||
priority 100, plus a default-route suppression) while everything else falls through to the VPN
|
VPN's dedicated table, so the services stay reachable on the `ctrs` network while outbound
|
||||||
table — so the services stay reachable on the `ctrs` network while outbound torrent traffic
|
torrent traffic exits via AirVPN. `DNSDefaultRoute` is disabled on `host0`; the VPN provides DNS.
|
||||||
exits via AirVPN. `DNSDefaultRoute` is disabled on `host0`; the VPN provides DNS.
|
|
||||||
- `transmission` and `jackett` `bindsTo` `systemd-networkd-wait-online@vpn.service` — they only
|
- `transmission` and `jackett` `bindsTo` `systemd-networkd-wait-online@vpn.service` — they only
|
||||||
run while the tunnel is up.
|
run while the tunnel is up.
|
||||||
- AirVPN forwards peer port `28457` to Transmission (`peer-port`); the firewall accepts it and
|
- AirVPN forwards a peer port to Transmission (`peer-port`); the firewall accepts it and drops
|
||||||
drops other new inbound TCP from `vpn`, while non-VPN input is limited to the service ports
|
other new inbound TCP from `vpn`, while non-VPN input is limited to the service ports
|
||||||
(netdata, Transmission, Jackett, Radarr, Sonarr, Jellyfin, PhotoPrism) plus copyparty's `3923`
|
(netdata, Transmission, Jackett, Radarr, Sonarr, Jellyfin, PhotoPrism) plus copyparty's `3923`
|
||||||
from the base config and Jellyseerr's `5055`.
|
from the base config and Jellyseerr's `5055`.
|
||||||
|
|
||||||
## Storage
|
## Storage
|
||||||
|
|
||||||
Media lives on the shared `/mnt/media` volume (bind-mounted read-write from `shill`); Transmission
|
Media lives on the shared `/mnt/media` volume (bind-mounted read-write from `shill`); Transmission
|
||||||
downloads into `/mnt/media/downloads/torrents` with a `.incomplete` dir, 28 MB/s up/down limits
|
downloads into `/mnt/media/downloads/torrents` with a `.incomplete` directory and configured
|
||||||
and a seed ratio limit of 2.0.
|
bandwidth and seed-ratio limits.
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
@@ -7,42 +7,44 @@ backend.
|
|||||||
|
|
||||||
- **Source:** [`shill/containers/middleman/`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman)
|
- **Source:** [`shill/containers/middleman/`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman)
|
||||||
(`default.nix`, `vhosts.nix`)
|
(`default.nix`, `vhosts.nix`)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md) (`my.containers` ephemeral nspawn on the
|
- **Host:** NixOS container on [`shill`](../README.md) (`my.containers` ephemeral nspawn on the
|
||||||
`ctrs` bridge; bind-mounts `/mnt/media` read-only for the static file vhosts)
|
`ctrs` bridge; bind-mounts `/mnt/media` read-only for the static file vhosts)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **nginx** reverse proxy with the `vts` and `fancyindex` modules, Brotli, kTLS, a proxy cache,
|
### nginx
|
||||||
and a dynamic resolver pointed at `estuary` (`proxyResolveWhileRunning`) so upstreams re-resolve
|
|
||||||
at runtime. All vhosts are declared in
|
The reverse proxy enables `vts`, `fancyindex`, Brotli, kTLS and a proxy cache. Its dynamic resolver
|
||||||
[`vhosts.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix);
|
points at `estuary`, allowing upstreams named under `ams1.int.nul.ie` to resolve again at runtime.
|
||||||
upstreams are addressed by their internal container/VM hostnames under `ams1.int.nul.ie`.
|
All vhosts live in
|
||||||
nginx is made to wait for `colony-psql` (`systemdAwaitPostgres`) before starting — a hack for
|
[`vhosts.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/middleman/vhosts.nix). nginx
|
||||||
it getting stuck on DNS early in boot.
|
also waits for `colony-psql` through `systemdAwaitPostgres`, avoiding an early-boot DNS stall.
|
||||||
- **ACME** — issues the wildcard certificates its own vhosts are served with (it is not a shared
|
|
||||||
CA for the other boxes):
|
### ACME
|
||||||
- `ams1.int.nul.ie` + `*.ams1.int.nul.ie` via a lego `exec` DNS challenge that SSHes to
|
|
||||||
`pdns-file-records@estuary-vm` to add/remove records in the internal PowerDNS zone. This is
|
`middleman` issues certificates for its own vhosts; it is not a shared CA for other boxes.
|
||||||
the default cert (`useACMEHost`) for vhosts on the internal domain.
|
|
||||||
- `nul.ie` + `*.nul.ie` + `*.s3.nul.ie` via the Cloudflare DNS provider. A `postRun` hook
|
- `ams1.int.nul.ie` and its wildcard use a lego `exec` challenge that SSHes to
|
||||||
`scp`s the renewed key/cert to the `mail` (Mailcow) VM and runs `mailcow-ssl-reload` there.
|
`pdns-file-records@estuary-vm`. This is the default `useACMEHost` certificate internally.
|
||||||
- Certs reload `nginx` on renewal; the `acme` group owns the secret files and `nginx` is a
|
- `nul.ie`, `*.nul.ie` and `*.s3.nul.ie` use Cloudflare DNS. A `postRun` hook copies renewed
|
||||||
member.
|
material to the `mail` VM and runs `mailcow-ssl-reload` there.
|
||||||
- **nginx-sso** — single sign-on at `sso.nul.ie` (one instance, `generic`), with Google OAuth as
|
- Renewal reloads nginx; the `acme` group owns the secret files and includes the nginx user.
|
||||||
the default method and a simple username/password provider. Cookie domain is `.nul.ie`.
|
|
||||||
SSO-gated vhosts below include `server-generic.conf` / `location-generic.conf` from
|
### nginx-sso
|
||||||
`/etc/nginx/includes/sso/`.
|
|
||||||
- **librespeed** — speed test, backend + frontend, published as `speed.nul.ie` and
|
The `generic` SSO instance at `sso.nul.ie` uses Google OAuth by default and also offers a simple
|
||||||
`librespeed.ams1.int.nul.ie` (both proxy to `localhost:8989`).
|
username/password provider. Its cookie domain is `.nul.ie`; gated vhosts include the generated
|
||||||
|
`server-generic.conf` / `location-generic.conf` snippets from `/etc/nginx/includes/sso/`.
|
||||||
|
|
||||||
|
### librespeed
|
||||||
|
|
||||||
|
The frontend and backend are published as `speed.nul.ie` and `librespeed.ams1.int.nul.ie`, both
|
||||||
|
proxied to `localhost:8989`.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: middleman -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `middleman`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| middleman-ctr | internal | `10.100.2.2/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::2/64` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
The firewall allows `http`, `https` and `8448` (Matrix federation). A small nftables SNAT rule
|
The firewall allows `http`, `https` and `8448` (Matrix federation). A small nftables SNAT rule
|
||||||
rewrites outbound IPv6 to the container's own address on `host0`.
|
rewrites outbound IPv6 to the container's own address on `host0`.
|
||||||
@@ -72,16 +74,16 @@ all vhosts are `onlySSL`, kTLS and HTTP/2. "SSO" = gated behind nginx-sso (`gene
|
|||||||
| `share.nul.ie` | `object-ctr:9090` | Sharry ([object](object.md)); websockets |
|
| `share.nul.ie` | `object-ctr:9090` | Sharry ([object](object.md)); websockets |
|
||||||
| `stuff.nul.ie` | `jackflix-ctr:3923` | copyparty |
|
| `stuff.nul.ie` | `jackflix-ctr:3923` | copyparty |
|
||||||
| `public.nul.ie` (+ alias `p.nul.ie`) | static `/mnt/media/public` | fancyindex file listing; `addSSL` so plain HTTP also works |
|
| `public.nul.ie` (+ alias `p.nul.ie`) | static `/mnt/media/public` | fancyindex file listing; `addSSL` so plain HTTP also works |
|
||||||
| `mc-map.nul.ie` | `simpcraft-oci:8100` | Minecraft map (OCI container on `whale2`) |
|
| `mc-map.nul.ie` | `simpcraft-oci:8100` | Minecraft map (OCI container on [`whale2`](../../whale2.md#game-servers)) |
|
||||||
| `mc-rail.nul.ie` | `simpcraft-oci:3876` | Minecraft railway map |
|
| `mc-rail.nul.ie` | `simpcraft-oci:3876` | Minecraft railway map ([`whale2`](../../whale2.md#game-servers)) |
|
||||||
| `mc-map-kink.nul.ie` | `kinkcraft-oci:8100` | Minecraft map |
|
| `mc-map-kink.nul.ie` | `kinkcraft-oci:8100` | Minecraft map ([`whale2`](../../whale2.md#game-servers)) |
|
||||||
| `speed.nul.ie` | `localhost:8989` | librespeed |
|
| `speed.nul.ie` | `localhost:8989` | librespeed |
|
||||||
| `librespeed.ams1.int.nul.ie` | `localhost:8989` | librespeed on the internal domain (internal wildcard cert) |
|
| `librespeed.ams1.int.nul.ie` | `localhost:8989` | librespeed on the internal domain (internal wildcard cert) |
|
||||||
| `md.nul.ie` | `object-ctr:3000` | HedgeDoc; websockets |
|
| `md.nul.ie` | `object-ctr:3000` | HedgeDoc; websockets |
|
||||||
| `pb.nul.ie` | `object-ctr:8088` | wastebin |
|
| `pb.nul.ie` | `object-ctr:8088` | wastebin |
|
||||||
| `photos.nul.ie` | `jackflix-ctr:2342` | PhotoPrism; websockets |
|
| `photos.nul.ie` | `jackflix-ctr:2342` | PhotoPrism; websockets |
|
||||||
| `pront.nul.ie` | `stream-hi.h.nul.ie:5000` | OctoPrint on the home network; `/webcam/` → `:5050`; **SSO** |
|
| `pront.nul.ie` | `stream-hi.h.nul.ie:5000` | OctoPrint on the home network ([`stream`](../../../home/stream.md)); `/webcam/` → `:5050`; **SSO** |
|
||||||
| `hass.nul.ie` | `hass-ctr.h.nul.ie:8123` | Home Assistant (home network); websockets |
|
| `hass.nul.ie` | `hass-ctr.h.nul.ie:8123` | [Home Assistant](../../../home/sfh/containers/hass.md) (home network); websockets |
|
||||||
| `hass-john.nul.ie` | `john-valorant-tun.ams1.int.nul.ie:8123` | remote Home Assistant over the point-to-point tunnel; websockets |
|
| `hass-john.nul.ie` | `john-valorant-tun.ams1.int.nul.ie:8123` | remote Home Assistant over the point-to-point tunnel; websockets |
|
||||||
| `minio.nul.ie` | `object-ctr:9001` | MinIO console; `/ws` websockets |
|
| `minio.nul.ie` | `object-ctr:9001` | MinIO console; `/ws` websockets |
|
||||||
| `s3.nul.ie` (+ `*.s3.nul.ie`) | `object-ctr:9000` | MinIO S3 API (virtual-host style via the `*.s3` wildcard cert); `/gitea/packages/` has a hack forcing the correct `Content-Type` for Docker image manifests |
|
| `s3.nul.ie` (+ `*.s3.nul.ie`) | `object-ctr:9000` | MinIO S3 API (virtual-host style via the `*.s3` wildcard cert); `/gitea/packages/` has a hack forcing the correct `Content-Type` for Docker image manifests |
|
||||||
|
|||||||
@@ -4,16 +4,17 @@ Object storage and the Nix binary cache, plus a few small self-hosted web apps (
|
|||||||
HedgeDoc, wastebin).
|
HedgeDoc, wastebin).
|
||||||
|
|
||||||
- **Source:** [`shill/containers/object.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/object.nix)
|
- **Source:** [`shill/containers/object.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/object.nix)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md) (bind-mounts `/mnt/minio` and
|
- **Host:** NixOS container on [`shill`](../README.md) (bind-mounts `/mnt/minio` and
|
||||||
`/mnt/nix-cache` read-write)
|
`/mnt/nix-cache` read-write)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
| Service | Port | Purpose |
|
| Service | Port | Purpose |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| MinIO | `9000` (S3) / `9001` (console) | S3-compatible object storage, `s3.nul.ie` + `*.s3.nul.ie` (virtual-host style via `MINIO_DOMAIN`), console at `minio.nul.ie`; region `eu-central-1`; data on the `/mnt/minio` XFS volume |
|
| MinIO | `9000` (S3) / `9001` (console) | S3-compatible object storage, `s3.nul.ie` + `*.s3.nul.ie` (virtual-host style via `MINIO_DOMAIN`), console at `minio.nul.ie`; region `eu-central-1`; data on the `/mnt/minio` XFS volume |
|
||||||
| Harmonia | `5000` | Nix binary cache at `nix-cache.nul.ie` — `harmonia-dev` cache serves `shill`'s `/nix/store` out of a dedicated store view rooted at `/var/lib/harmonia` (bind-mounted from `/mnt/nix-cache`), signed with the `nix-cache.key` secret; a `harmonia` user with authorized keys exists for cache pushes; priority 30 |
|
| Harmonia | `5000` | Nix binary cache at `nix-cache.nul.ie` — `harmonia-dev` cache serves `shill`'s `/nix/store` out of a dedicated store view rooted at `/var/lib/harmonia` (bind-mounted from `/mnt/nix-cache`), signed with the `nix-cache.key` secret; a `harmonia` user with authorized keys exists for cache pushes |
|
||||||
| Sharry | `9090` | file sharing at `share.nul.ie`; Postgres on [colony-psql](colony-psql.md), files stored in the `share` MinIO bucket; fixed `dev` account + invite signup; mail via `mail.nul.ie`; max share size 128G |
|
| Sharry | `9090` | file sharing at `share.nul.ie`; Postgres on [colony-psql](colony-psql.md), files stored in the `share` MinIO bucket; fixed `dev` account + invite signup; mail via `mail.nul.ie`; configured share-size limit |
|
||||||
| HedgeDoc | `3000` | collaborative markdown notes at `md.nul.ie`; Postgres on [colony-psql](colony-psql.md); anonymous edits but no anonymous notes, email login, no open email registration |
|
| HedgeDoc | `3000` | collaborative markdown notes at `md.nul.ie`; Postgres on [colony-psql](colony-psql.md); anonymous edits but no anonymous notes, email login, no open email registration |
|
||||||
| wastebin | `8088` | pastebin at `pb.nul.ie` |
|
| wastebin | `8088` | pastebin at `pb.nul.ie` |
|
||||||
| atticd | `8069` | **currently disabled** (`services.atticd.enable = false`) — an alternative Nix cache that would store locally and sit behind `nix-cache.nul.ie`; config (including the `object/atticd.env` secret) is kept around |
|
| atticd | `8069` | **currently disabled** (`services.atticd.enable = false`) — an alternative Nix cache that would store locally and sit behind `nix-cache.nul.ie`; config (including the `object/atticd.env` secret) is kept around |
|
||||||
@@ -24,12 +25,7 @@ Everything public is fronted by [middleman](middleman.md) (see its vhost table).
|
|||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: object -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `object`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| object-ctr | internal | `10.100.2.7/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::7/64` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Backing services
|
## Backing services
|
||||||
|
|
||||||
|
|||||||
@@ -6,16 +6,17 @@ the container provides the network plumbing and opens the API port.
|
|||||||
|
|
||||||
- **Source:** [`shill/containers/qclk/`](../../../../../nixos/boxes/colony/vms/shill/containers/qclk)
|
- **Source:** [`shill/containers/qclk/`](../../../../../nixos/boxes/colony/vms/shill/containers/qclk)
|
||||||
(`default.nix`)
|
(`default.nix`)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md)
|
- **Host:** NixOS container on [`shill`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **WireGuard `management` interface** — listens on UDP `51821` (`lib.my.c.colony.qclk.wgPort`,
|
- **WireGuard `management` interface** — listens on UDP `51821` (`lib.my.c.colony.qclk.wgPort`,
|
||||||
allowed through the firewall; `estuary` port-forwards it here) with the private key from the
|
allowed through the firewall; `estuary` port-forwards it here) with the private key from the
|
||||||
`qclk/wg.key` age secret. Managed devices are static peers, each pinned to its own address in
|
`qclk/wg.key` age secret. Managed devices are static peers, each pinned to its own address in the
|
||||||
the `qclk` prefix (`10.100.4.0/24`); the peer list currently has a single entry (host 2).
|
`qclk` prefix; the peer list currently has a single entry.
|
||||||
- **Routing/NAT** — the container itself is host 1 of the `qclk` prefix. `shill` routes
|
- **Routing/NAT** — `shill` routes the `qclk` prefix to this container, and outbound traffic from
|
||||||
`10.100.4.0/24` to this container, and outbound traffic from `host0` into `management` is
|
`host0` into `management` is
|
||||||
SNATed to the container's `qclk` address. Forwarding into `management` is accepted from the
|
SNATed to the container's `qclk` address. Forwarding into `management` is accepted from the
|
||||||
AS211024 trusted IPv4 ranges (`lib.my.c.as211024.trusted.v4`).
|
AS211024 trusted IPv4 ranges (`lib.my.c.as211024.trusted.v4`).
|
||||||
- **API port** — TCP `8080` is accepted on the `management` interface (`apiPort`), but note
|
- **API port** — TCP `8080` is accepted on the `management` interface (`apiPort`), but note
|
||||||
@@ -23,16 +24,10 @@ the container provides the network plumbing and opens the API port.
|
|||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: qclk -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `qclk`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| qclk-ctr | internal | `10.100.2.10/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::a/64` | ams1.int.nul.ie | |
|
|
||||||
| qclk | qclk | `10.100.4.1/24` | — | — | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
Two assignments: `internal` on the `ctrs` network like the other containers, and `qclk` — host 1
|
Two assignments: `internal` on the `ctrs` network like the other containers, and `qclk` on the
|
||||||
of the `qclk` prefix on the `management` WireGuard interface (IPv4 only, no DNS name).
|
`management` WireGuard interface (IPv4 only, no DNS name).
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
@@ -4,34 +4,33 @@ Federated-social container. Despite the name, the only service actually running
|
|||||||
PDS** — the Mastodon instance ("toots") is **currently disabled**.
|
PDS** — the Mastodon instance ("toots") is **currently disabled**.
|
||||||
|
|
||||||
- **Source:** [`shill/containers/toot.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/toot.nix)
|
- **Source:** [`shill/containers/toot.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/toot.nix)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md)
|
- **Host:** NixOS container on [`shill`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **bluesky-pds** — the active service. `pds.nul.ie` on port `3000`, fronted by
|
### Bluesky PDS
|
||||||
[middleman](middleman.md) (which also redirects `/.well-known/atproto-did` here). Invites
|
|
||||||
required; blob store is the `pds` bucket on [object](object.md)'s MinIO (`s3.nul.ie`,
|
The active service listens on port 3000 as `pds.nul.ie`, fronted by [middleman](middleman.md),
|
||||||
`eu-central-1`), upload limit 50 MiB; federation settings point at the stock Bluesky
|
which also redirects `/.well-known/atproto-did` here. It requires invites, applies an upload-size
|
||||||
infrastructure (`plc.directory`, `api.bsky.app`, `mod.bsky.app`, `bsky.network` crawlers).
|
limit, and stores blobs in [object](object.md)'s `pds` MinIO bucket. Federation uses the standard
|
||||||
Secrets (including the S3 credentials) come from the `toot/pds.env` age secret. Email from
|
Bluesky services and crawlers. `toot/pds.env` supplies secrets, including S3 credentials, and mail
|
||||||
`pds@nul.ie`.
|
comes from `pds@nul.ie`.
|
||||||
- **mastodon** — **disabled** (`services.mastodon.enable = false`). The full config is still
|
|
||||||
present: `LOCAL_DOMAIN = nul.ie` with `WEB_DOMAIN = toot.nul.ie`, Postgres on
|
### Mastodon
|
||||||
[colony-psql](colony-psql.md), local Redis, SMTP via `mail.nul.ie`, media in the `mastodon`
|
|
||||||
MinIO bucket (`S3_ALIAS_HOST = mastodon.s3.nul.ie`), 4 streaming processes, and media
|
Mastodon is disabled with `services.mastodon.enable = false`, but its configuration remains:
|
||||||
auto-cleanup after 30 days. [middleman](middleman.md) still proxies `toot.nul.ie` →
|
|
||||||
`toot-ctr:80` and redirects the `webfinger`/`nodeinfo`/`host-meta` well-knowns there, but with
|
- `nul.ie` is the local domain and `toot.nul.ie` the web domain.
|
||||||
the service off those endpoints are dead. The config notes the removed `otpSecretFile` option
|
- PostgreSQL runs on [colony-psql](colony-psql.md), with local Redis and SMTP through `mail.nul.ie`.
|
||||||
would need addressing before Mastodon can come back.
|
- Media uses the `mastodon` MinIO bucket, a configured streaming-process pool, and periodic cleanup.
|
||||||
|
- [middleman](middleman.md) still proxies the dead vhost and `.well-known` endpoints.
|
||||||
|
|
||||||
|
The removed `otpSecretFile` option must be addressed before the service can return.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: toot -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `toot`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| toot-ctr | internal | `10.100.2.8/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::8/64` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
The firewall allows `http` (the Mastodon nginx vhost) and the PDS port `3000` besides netdata.
|
The firewall allows `http` (the Mastodon nginx vhost) and the PDS port `3000` besides netdata.
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,8 @@
|
|||||||
manager, published as `pass.nul.ie` through [middleman](middleman.md).
|
manager, published as `pass.nul.ie` through [middleman](middleman.md).
|
||||||
|
|
||||||
- **Source:** [`shill/containers/vaultwarden.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/vaultwarden.nix)
|
- **Source:** [`shill/containers/vaultwarden.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/vaultwarden.nix)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md)
|
- **Host:** NixOS container on [`shill`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
@@ -14,17 +15,12 @@ manager, published as `pass.nul.ie` through [middleman](middleman.md).
|
|||||||
- **SMTP** via `mail.nul.ie:587` (STARTTLS) as `pass@nul.ie`; credentials and other sensitive
|
- **SMTP** via `mail.nul.ie:587` (STARTTLS) as `pass@nul.ie`; credentials and other sensitive
|
||||||
settings come from the `vaultwarden/config.env` age secret.
|
settings come from the `vaultwarden/config.env` age secret.
|
||||||
- **Backups** — a `borgbackup` job pushes `/var/lib/vaultwarden` to rsync.net
|
- **Backups** — a `borgbackup` job pushes `/var/lib/vaultwarden` to rsync.net
|
||||||
(`zh2855@zh2855.rsync.net:borg/vaultwarden2`), repokey-encrypted (passphrase and SSH key from
|
using a repokey-encrypted repository; its passphrase and SSH key come from secrets, and the job
|
||||||
secrets), `zstd,10` compression, keeping 7 daily / 4 weekly / all monthly archives.
|
keeps daily, weekly and monthly archives according to its configured retention policy.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: vaultwarden -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `vaultwarden`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| vaultwarden-ctr | internal | `10.100.2.3/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::3/64` | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Persistence
|
## Persistence
|
||||||
|
|
||||||
|
|||||||
@@ -4,36 +4,30 @@ The colony Tailscale node: a subnet router and exit node that advertises the col
|
|||||||
into the tailnet.
|
into the tailnet.
|
||||||
|
|
||||||
- **Source:** [`shill/containers/waffletail.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/waffletail.nix)
|
- **Source:** [`shill/containers/waffletail.nix`](../../../../../nixos/boxes/colony/vms/shill/containers/waffletail.nix)
|
||||||
- **Host:** NixOS container on [`shill`](../../shill.md)
|
- **Host:** NixOS container on [`shill`](../README.md)
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **tailscale** — joins via an auth key from secrets (`tailscale-auth.key`) against the
|
### Tailscale
|
||||||
self-hosted Headscale control plane at `hs.nul.ie`. Runs with `--netfilter-mode=off` (firewall
|
|
||||||
is managed by the repo's own nftables rules), `--advertise-exit-node`, and
|
The node authenticates to `hs.nul.ie` with the secret `tailscale-auth.key`. It disables Tailscale's
|
||||||
`--advertise-routes` covering the whole colony — `10.100.0.0/16` and `2a0e:97c0:4d2:10::/60`.
|
netfilter management, advertises itself as an exit node and advertises the colony IPv4/IPv6 ranges,
|
||||||
Does **not** accept routes itself. Listens on UDP `41641` (`openFirewall`), which `estuary`
|
but does not accept routes. UDP port 41641 is open and forwarded here by `estuary`.
|
||||||
port-forwards to this container.
|
|
||||||
- `shill` routes the Tailscale prefixes (`100.64.0.0/10`, `fd7a:115c:a1e0::/48`) to this
|
### Routing and firewall
|
||||||
container, so colony hosts can reach tailnet clients and vice versa.
|
|
||||||
- nftables: `tailscale0` is a trusted interface; forwarding from `host0` into Tailscale is
|
`shill` routes the Tailscale prefixes to this container.
|
||||||
allowed for the colony source ranges, and tailnet-sourced traffic leaving via `host0` is
|
The repository's nftables rules trust `tailscale0`, permit colony-sourced forwarding into the
|
||||||
SNATed to the container's colony addresses (except when destined to the colony ranges
|
tailnet, and SNAT tailnet traffic leaving through `host0` unless its destination is already within
|
||||||
themselves).
|
colony.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: waffletail -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `waffletail`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| waffletail-ctr | internal | `10.100.2.9/24 gw 10.100.2.1` | `2a0e:97c0:4d2:12::9/64` | ams1.int.nul.ie | |
|
|
||||||
| waffletail | tailscale | `100.64.0.5/32` | `fd7a:115c:a1e0::5/128` | — | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
Two assignments: `internal` on the `ctrs` network like the other containers, and `tailscale` —
|
Two assignments: `internal` on the `ctrs` network like the other containers, and `tailscale` for
|
||||||
its addresses on the tailnet itself (host 5 of `100.64.0.0/10` and `fd7a:115c:a1e0::/48`; no
|
its addresses on the tailnet itself (no DNS name).
|
||||||
DNS name).
|
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
+36
-40
@@ -10,54 +10,52 @@ container churn and resource use stay isolated).
|
|||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- Runs OCI containers via podman (`virtualisation.oci-containers`, netavark
|
### Container runtime
|
||||||
backend, `firewall_driver = "none"` so podman doesn't fight `my.firewall`).
|
|
||||||
- Each game server gets its own routable address on the `colony` netavark
|
OCI containers run under podman (`virtualisation.oci-containers`) with the netavark backend and
|
||||||
network (defined in `/etc/containers/networks/colony.json`), which is backed
|
`firewall_driver = "none"`, leaving firewall management to `my.firewall`.
|
||||||
by the `oci` interface and the `prefixes.oci` v4/v6 ranges; per-game
|
|
||||||
addresses come from `extraAssignments` (`valheim-oci`, `simpcraft-oci`, …)
|
### Routable game servers
|
||||||
and are passed to podman with `--network=colony:ip=…` (`lib.my.dockerNetAssignment`).
|
|
||||||
- `estuary` forwards the game ports in (see `firewallForwards`), so the
|
Each server gets an address from `extraAssignments` (`valheim-oci`, `simpcraft-oci`, …) on the
|
||||||
servers are reachable on the public IP as well as directly over IPv6.
|
`colony` netavark network. That network is backed by the `oci` interface and `prefixes.oci` v4/v6
|
||||||
- `/var/lib/containers` is a dedicated XFS disk (project quotas).
|
ranges; `lib.my.dockerNetAssignment` supplies the address through `--network=colony:ip=…`.
|
||||||
|
`estuary` forwards the public game ports, while IPv6 reaches the containers directly.
|
||||||
|
|
||||||
|
### Storage
|
||||||
|
|
||||||
|
`/var/lib/containers` is a dedicated XFS disk with project quotas.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: whale2 -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `whale2`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| whale-vm (oci) | internal | `94.142.241.226/32` | `2a0e:97c0:4d2:11::3/64` | ams1.int.nul.ie | |
|
|
||||||
| whale-vm-oci | oci | `10.100.3.1/24` | `2a0e:97c0:4d2:13::1/64` | ams1.int.nul.ie | |
|
|
||||||
| whale-vm-routing | routing | `10.100.1.3/24 gw 10.100.1.1` | — | ams1.int.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Game servers
|
## Game servers
|
||||||
|
|
||||||
The OCI containers are documented here (they have no pages of their own).
|
The OCI containers are documented here (they have no pages of their own).
|
||||||
Addresses are the per-container `extraAssignments` on the `oci` network;
|
Their per-container `extraAssignments` on the `oci` network are listed in the generated
|
||||||
ports are the public ones forwarded by `estuary`.
|
[network assignments](../../networking.md#box-assignments) table. Ports below are the public ones
|
||||||
|
forwarded by `estuary`.
|
||||||
|
|
||||||
| Container | Address (v4 / v6 host) | Ports | Status |
|
| Container | Ports | Status |
|
||||||
|---|---|---|---|
|
|---|---|---|
|
||||||
| `valheim` | `10.100.3.2` / `2a0e:97c0:4d2:13::2` | `2456-2457`/udp | running |
|
| `valheim` | `2456-2457`/udp | running |
|
||||||
| `simpcraft` | `10.100.3.3` / `2a0e:97c0:4d2:13::3` | `25565` tcp+udp | running |
|
| `simpcraft` | `25565` tcp+udp | running |
|
||||||
| `simpcraft-staging` | `10.100.3.4` / `2a0e:97c0:4d2:13::4` | `25566` tcp | **disabled** (commented out) |
|
| `simpcraft-staging` | `25566` tcp | **disabled** (commented out) |
|
||||||
| `enshrouded` | `10.100.3.5` / `2a0e:97c0:4d2:13::5` | `15636-15637`/udp | **disabled** (`enshrouded.nix` not imported) |
|
| `enshrouded` | `15636-15637`/udp | **disabled** (`enshrouded.nix` not imported) |
|
||||||
| `kevcraft` | `10.100.3.6` / `2a0e:97c0:4d2:13::6` | `25567` tcp+udp | running |
|
| `kevcraft` | `25567` tcp+udp | running |
|
||||||
| `kinkcraft` | `10.100.3.7` / `2a0e:97c0:4d2:13::7` | `25568` tcp+udp | running |
|
| `kinkcraft` | `25568` tcp+udp | running |
|
||||||
| `graeme` | `10.100.3.8` / `2a0e:97c0:4d2:13::8` | `25569` tcp+udp | running |
|
| `graeme` | `25569` tcp+udp | running |
|
||||||
|
|
||||||
- **valheim** ([`valheim.nix`](../../../nixos/boxes/colony/vms/whale2/valheim.nix)) —
|
- **valheim** ([`valheim.nix`](../../../nixos/boxes/colony/vms/whale2/valheim.nix)) —
|
||||||
`lloesche/valheim-server`, public server "amogus sus", world `simpland2`,
|
`lloesche/valheim-server`, public server "amogus sus", world `simpland2`,
|
||||||
allow-listed Steam IDs, password from agenix.
|
allow-listed Steam IDs, password from agenix.
|
||||||
- **simpcraft** ([`minecraft/`](../../../nixos/boxes/colony/vms/whale2/minecraft)) —
|
- **simpcraft** ([`minecraft/`](../../../nixos/boxes/colony/vms/whale2/minecraft)) —
|
||||||
`itzg/minecraft-server` (self-built `git.nul.ie/dev/craftblock` image),
|
`itzg/minecraft-server` (self-built `git.nul.ie/dev/craftblock` image),
|
||||||
Modrinth "Simpcraft" modpack, whitelist + ops, 8 GiB heap.
|
Modrinth "Simpcraft" modpack, whitelist + ops.
|
||||||
**simpcraft-staging** is the same setup pinned to an older pack version,
|
- **simpcraft-staging** — the same setup pinned to an older pack version, currently commented out.
|
||||||
currently commented out.
|
- **kevcraft** — vanilla Minecraft 1.20.1, extra op.
|
||||||
- **kevcraft** — vanilla Minecraft 1.20.1, 4 GiB heap, extra op.
|
- **kinkcraft** — same Simpcraft modpack as `simpcraft`.
|
||||||
- **kinkcraft** — same Simpcraft modpack as `simpcraft`, 6 GiB heap.
|
|
||||||
- **graeme** — vanilla Minecraft on hard difficulty with its own whitelist.
|
- **graeme** — vanilla Minecraft on hard difficulty with its own whitelist.
|
||||||
- **enshrouded** ([`enshrouded.nix`](../../../nixos/boxes/colony/vms/whale2/enshrouded.nix)) —
|
- **enshrouded** ([`enshrouded.nix`](../../../nixos/boxes/colony/vms/whale2/enshrouded.nix)) —
|
||||||
`sknnr/enshrouded-dedicated-server` ("UWUshrouded"); the file exists but is
|
`sknnr/enshrouded-dedicated-server` ("UWUshrouded"); the file exists but is
|
||||||
@@ -69,12 +67,10 @@ The Minecraft containers share one whitelist/ops list and agenix env file
|
|||||||
|
|
||||||
## Backups
|
## Backups
|
||||||
|
|
||||||
A local borg job (`services.borgbackup.jobs.simpcraft`) archives the
|
A local borg job (`services.borgbackup.jobs.simpcraft`) archives the `simpcraft` world frequently,
|
||||||
`simpcraft` world every ~15 minutes (offset from the usual 5-minute autosave
|
offset from its autosave timer, into `/var/lib/containers/backup/simpcraft`. It uses `mcrcon` to
|
||||||
ticks) into `/var/lib/containers/backup/simpcraft`, using `mcrcon` to
|
`save-off`/`save-on` around each run and keeps a short history for quick world rollback rather than
|
||||||
`save-off`/`save-on` around each run. Retention is short (12 h + 48 hourly) —
|
disaster recovery; the `oci` LV itself is covered by `colony`'s `borgthin`.
|
||||||
this is for quick world rollback, not disaster recovery (the `oci` LV itself
|
|
||||||
is covered by `colony`'s `borgthin`).
|
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
+21
-57
@@ -8,70 +8,28 @@ pair, and everything clients touch (gateway, DNS) is a floating VIP that follows
|
|||||||
|
|
||||||
- **Source:** [`nixos/boxes/home/`](../../../nixos/boxes/home)
|
- **Source:** [`nixos/boxes/home/`](../../../nixos/boxes/home)
|
||||||
|
|
||||||
## Hierarchy
|
## Boxes
|
||||||
|
|
||||||
```
|
| Box | Role | Host |
|
||||||
h.nul.ie
|
|---|---|---|
|
||||||
├── palace (physical VM host — AMD, 100G, SR-IOV)
|
| [`palace`](palace.md) | VM host | physical |
|
||||||
│ ├── river ── primary router VM (PPPoE / Digiweb WAN)
|
| [`river`](river.md) | Primary router (VRRP pair with `stream`) | VM on `palace` |
|
||||||
│ ├── cellar ─ NVMe-oF / SPDK storage target VM
|
| [`stream`](stream.md) | Secondary router (VRRP pair with `river`) | physical |
|
||||||
│ └── sfh ──── container host VM ("services for home")
|
| [`cellar`](cellar.md) | NVMe-oF / SPDK storage target | VM on `palace` |
|
||||||
│ ├── hass ── Home Assistant + Frigate + MQTT (container)
|
| [`sfh`](sfh/README.md) | NixOS container host (containers on its page) | VM on `palace` |
|
||||||
│ └── unifi ─ UniFi controller (container)
|
| [`castle`](castle.md) | Workstation / gaming desktop | physical |
|
||||||
├── stream (physical secondary router — Virgin Media WAN)
|
|
||||||
└── castle (workstation / gaming desktop — netboot, NVMe-oF root)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Machines
|
|
||||||
|
|
||||||
| Box | Role | Host | Page |
|
|
||||||
|---|---|---|---|
|
|
||||||
| `palace` | VM host | physical | [palace.md](palace.md) |
|
|
||||||
| `river` | Primary router (VRRP pair with `stream`) | VM on `palace` | [river.md](river.md) |
|
|
||||||
| `stream` | Secondary router (VRRP pair with `river`) | physical | [stream.md](stream.md) |
|
|
||||||
| `cellar` | NVMe-oF / SPDK storage target | VM on `palace` | [cellar.md](cellar.md) |
|
|
||||||
| `sfh` | NixOS container host | VM on `palace` | [sfh.md](sfh.md) |
|
|
||||||
| `castle` | Workstation / gaming desktop | physical | [castle.md](castle.md) |
|
|
||||||
| `hass` | Home Assistant + Frigate + MQTT | container on `sfh` | [sfh/containers/hass.md](sfh/containers/hass.md) |
|
|
||||||
| `unifi` | UniFi controller | container on `sfh` | [sfh/containers/unifi.md](sfh/containers/unifi.md) |
|
|
||||||
|
|
||||||
## Router VIPs
|
## Router VIPs
|
||||||
|
|
||||||
Clients never use a router's real address: `kea` (DHCP) and `radvd` (RAs) hand out the per-VLAN
|
Clients use per-VLAN floating VIPs as their gateway and DNS server; `keepalived` moves them between
|
||||||
floating VIPs as both gateway and DNS server, and `keepalived` moves them between `river` and
|
`river` and `stream`. The addresses, DHCP/RA behavior and failover mechanics are documented once in
|
||||||
`stream`. They are declared as pseudo-systems (`router-hi` / `router-lo` / `router-ut`) in
|
[Router VIPs](../../networking.md#router-vips) and [Router HA](../../networking.md#router-ha).
|
||||||
`routing-common`'s `extraAssignments`, with addresses from `lib.my.c.home.vips`
|
|
||||||
([`lib/constants.nix`](../../../lib/constants.nix)); `pdns-recursor` binds them on both routers
|
|
||||||
(`ip_nonlocal_bind`) so DNS follows the master instead of relying on client resolver timeouts.
|
|
||||||
|
|
||||||
| Pseudo-system | Network | IPv4 | IPv6 |
|
|
||||||
|---|---|---|---|
|
|
||||||
| `router-hi` | `hi` (VLAN 100) | `192.168.71.254/22` | `2a0e:97c0:4d0:1::ffff/64` |
|
|
||||||
| `router-lo` | `lo` (VLAN 110) | `192.168.79.254/21` | `2a0e:97c0:4d0:2::ffff/64` |
|
|
||||||
| `router-ut` | `untrusted` (VLAN 120) | `192.168.80.254/24` | `2a0e:97c0:4d0:3::ffff/64` |
|
|
||||||
|
|
||||||
Notes:
|
|
||||||
|
|
||||||
- The IPv6 gateway clients learn from RAs is the link-local `fe80::1` on each VLAN (a
|
|
||||||
`virtual_ipaddress_excluded` VIP; `radvd` advertises from it), not the global VIP above.
|
|
||||||
- `keepalived` also floats a VIP on the `as211024` mesh interface (`10.100.50.4`,
|
|
||||||
`2a0e:97c0:4df:0:1::ffff`) — it has no pseudo-system because nothing client-facing uses it.
|
|
||||||
- The `untrusted` VLAN is an exception to "DNS follows the master": its DHCP/RA options hand out
|
|
||||||
Cloudflare resolvers (`1.1.1.1` / `2606:4700:4700::1111`), not the VIP.
|
|
||||||
|
|
||||||
## Networks
|
## Networks
|
||||||
|
|
||||||
| Network | VLAN | IPv4 | IPv6 | MTU |
|
The site separates core management, high-MTU trusted traffic, general trusted traffic, untrusted
|
||||||
|---|---|---|---|---|
|
clients and the two WAN paths. VLAN IDs, prefixes, MTUs and router addressing live in the canonical
|
||||||
| `core` | native | `192.168.64.0/24` | — | 1500 |
|
[`home` section of networking.md](../../networking.md#home).
|
||||||
| `hi` | 100 | `192.168.68.0/22` | `2a0e:97c0:4d0:1::/64` | 9000 |
|
|
||||||
| `lo` | 110 | `192.168.72.0/21` | `2a0e:97c0:4d0:2::/64` | 1500 |
|
|
||||||
| `untrusted` | 120 | `192.168.80.0/24` | `2a0e:97c0:4d0:3::/64` | 1500 |
|
|
||||||
| `wan` (stream) | 130 | DHCP public lease + modem mgmt `192.168.0.0/24` | — | 1500 |
|
|
||||||
| `pon-isp` (river) | 10 | PPPoE transport (no L3) | — | 1508 |
|
|
||||||
| `wan-pon-ont` (river) | 140 | ONT mgmt `192.168.100.0/24` | — | 1500 |
|
|
||||||
|
|
||||||
The logical network map lives in [networking.md](../../networking.md).
|
|
||||||
|
|
||||||
## Switch fabric
|
## Switch fabric
|
||||||
|
|
||||||
@@ -79,3 +37,9 @@ The boxes hang off three hand-configured switches — `jim` and `dave` (MikroTik
|
|||||||
`brian` (Ubiquiti, UniFi) — which are **not** managed by this flake. The physical topology, VLAN
|
`brian` (Ubiquiti, UniFi) — which are **not** managed by this flake. The physical topology, VLAN
|
||||||
map, the Digiweb WAN path (trunked VLAN 10 + PVID 140 at the ONT edge), and the multi-ONT plan are
|
map, the Digiweb WAN path (trunked VLAN 10 + PVID 140 at the ONT edge), and the multi-ONT plan are
|
||||||
documented in [switches.md](switches.md).
|
documented in [switches.md](switches.md).
|
||||||
|
|
||||||
|
## Wireless APs
|
||||||
|
|
||||||
|
The Wi-Fi APs — `vibe` (MikroTik cAP ax) and `wave` (Cudy AX3000 on OpenWrt) — are dumb APs, also
|
||||||
|
**not** managed by this flake. The shared VLAN-trunk design, SSIDs, per-AP management addressing,
|
||||||
|
and the OpenWrt flash/config for `wave` are in [aps.md](aps.md).
|
||||||
|
|||||||
@@ -0,0 +1,157 @@
|
|||||||
|
# Home wireless APs
|
||||||
|
|
||||||
|
Reference for the home Wi-Fi access points. Like the switches ([switches.md](switches.md)), these
|
||||||
|
are **not** managed by this flake — they are configured on-device (RouterOS on the MikroTik,
|
||||||
|
OpenWrt/UCI on the Cudy). This file documents the shared VLAN/trunk design and each AP.
|
||||||
|
|
||||||
|
Only the DNS records live in the flake
|
||||||
|
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix),
|
||||||
|
`h.nul.ie` zone). Everything else here is applied by hand on the device.
|
||||||
|
|
||||||
|
## The APs
|
||||||
|
|
||||||
|
| | vibe | wave |
|
||||||
|
|---|---|---|
|
||||||
|
| Model | MikroTik **cAP ax** (`cAPGi-5HaxD2HaxD`) | Cudy **AX3000** (OpenWrt id `cudy,ap3000-v1`) |
|
||||||
|
| OS | RouterOS 7.x | OpenWrt 25.12.x (MT7981B / Filogic 820) |
|
||||||
|
| Radio | 2×2 both bands | 2×2 both bands (2 spatial streams) |
|
||||||
|
| Uplink | trunk (multi-port; `ether2` is a wired LAN port) | single **2.5 GbE** trunk |
|
||||||
|
| Management | core/hi/lo `.15` | core/lo `.14` (**not** `hi`; see below) |
|
||||||
|
|
||||||
|
Both mirror the same two SSIDs. `wave` replaced an older AP of the same name; the new one is the
|
||||||
|
Cudy running OpenWrt.
|
||||||
|
|
||||||
|
> **Note on "AX3000":** MT7981B is 2×2 (2 spatial streams). The "3000" is *aggregate* Mbps —
|
||||||
|
> 574 (2.4 GHz 2ss) + 2402 (5 GHz 2ss @ **160 MHz**) — not three streams. `iw` confirms 2×2
|
||||||
|
> (`Available Antennas TX/RX 0x3`; "3 streams: not supported"). 160 MHz is what earns the "3000".
|
||||||
|
|
||||||
|
## Shared design (dumb AP)
|
||||||
|
|
||||||
|
Every AP is a **dumb AP**: it bridges wireless clients onto the right VLAN and does **no** routing,
|
||||||
|
DHCP, RA or firewalling. The home routers (`river`/`stream`) own DHCP/RA/gateway (per-VLAN VRRP VIPs)
|
||||||
|
and firewalling. The uplink is a **tagged trunk**:
|
||||||
|
|
||||||
|
| VLAN | `lib.my.c.home.vlans` | Role | AP use |
|
||||||
|
|---|---|---|---|
|
||||||
|
| — (native) | `core` | switch/fabric management (1500) | backup management (untagged) |
|
||||||
|
| 100 | `hi` | trusted LAN, **high-MTU** (jumbo 9000) | `vibe` management (it's jumbo-capable) |
|
||||||
|
| 110 | `lo` | trusted LAN (1500) | main SSID `wlan0`; `wave` management |
|
||||||
|
| 120 | `untrusted` | guest network | guest SSID `wlan1` |
|
||||||
|
|
||||||
|
`hi` and `lo` are both **trusted** client VLANs — the only difference is MTU (`hi` carries jumbo
|
||||||
|
9000, `lo` is standard 1500). An AP puts its own management on whichever it can do: `vibe` (jumbo)
|
||||||
|
sits on `hi`, `wave` (eth0 capped at 2026) sits on `lo`. The main SSID lands on `lo` because Wi-Fi
|
||||||
|
clients are 1500 regardless.
|
||||||
|
|
||||||
|
### SSIDs
|
||||||
|
|
||||||
|
| SSID | Bands | Security | VLAN |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `wlan0` (main) | 5 GHz + 2.4 GHz | WPA2/WPA3-PSK (`sae-mixed`) | 110 (`lo`) |
|
||||||
|
| `wlan1` (guest) | 2.4 GHz | WPA2-PSK (`psk2`) | 120 (`untrusted`) |
|
||||||
|
|
||||||
|
**Passphrases are never stored in this repo.** `vibe` is the source of truth; read them out-of-band
|
||||||
|
with `ssh admin@vibe '/interface wifi export show-sensitive'` (`.passphrase=` prints **unquoted**).
|
||||||
|
|
||||||
|
## vibe (MikroTik cAP ax)
|
||||||
|
|
||||||
|
RouterOS, one hardware-offloaded bridge `main` with `vlan-filtering=yes`. Access: `ssh admin@vibe`
|
||||||
|
— key auth for `admin` is installed (`~/.ssh/id_rsa`), with `admin`/`admin` as a fallback.
|
||||||
|
|
||||||
|
- **Uplink `ether1`** — trunk, tagged VLANs 100/110/120; native/untagged is the default VLAN 1
|
||||||
|
(PVID, no IP). `ether2` is a wired **access port** on VLAN 110 (`lo`). `l2mtu 9214`.
|
||||||
|
- **Radios** — `wifi1` (5 GHz, 20/40/80) + `wifi2` (2.4 GHz, 20/40) both broadcast `wlan0`
|
||||||
|
(WPA2/WPA3-PSK), untagged onto VLAN 110. `wifi3` is a virtual AP on `wifi2` broadcasting `wlan1`
|
||||||
|
(WPA2-PSK), untagged onto VLAN 120. `country=Ireland`.
|
||||||
|
- **Bridge VLANs** — 100 tagged `main,ether1`; 110 tagged `main,ether1` + untagged
|
||||||
|
`ether2,wifi1,wifi2`; 120 tagged `main,ether1` + untagged `wifi3`.
|
||||||
|
- **Roaming** — 802.11k/v via a `/interface wifi steering` profile (`rrm=yes wnm=yes`,
|
||||||
|
`neighbor-group=home-aps`) assigned to `wifi1`/`wifi2`/`wifi3`.
|
||||||
|
- **Resolver** — the hi VIP `192.168.71.254` / `2a0e:97c0:4d0:1::ffff`.
|
||||||
|
|
||||||
|
### Management
|
||||||
|
|
||||||
|
Management uses host `.15`: `192.168.64.15` on native/core as a backup,
|
||||||
|
`192.168.68.15/22` + `2a0e:97c0:4d0:1::1:6` on `hi` VLAN 100, and
|
||||||
|
`192.168.72.15/21` + `2a0e:97c0:4d0:2::1:6` on `lo` VLAN 110. The `hi` address holds the default
|
||||||
|
route through its VIP; `untrusted` has no address. With `l2mtu 9214`, `vibe` can use the jumbo
|
||||||
|
`hi` network unlike `wave`.
|
||||||
|
|
||||||
|
Like the switches, `vibe` is **pure L2 and must not route** — `ip-forward=no`, IPv6 `forward=no`,
|
||||||
|
`accept-router-advertisements=no`, `ra-lifetime=0`. RouterOS ships these forwarding-on and then
|
||||||
|
advertises itself as an IPv6 router; re-check after any upgrade. See
|
||||||
|
[switches.md#switches-must-not-route](switches.md#switches-must-not-route) for the rationale and
|
||||||
|
commands.
|
||||||
|
|
||||||
|
## wave (Cudy AX3000, OpenWrt)
|
||||||
|
|
||||||
|
Single-port AP, so the port is a VLAN **trunk** carrying management + both SSIDs.
|
||||||
|
|
||||||
|
### Management addressing
|
||||||
|
|
||||||
|
`wave` takes host `.14`, on **`lo` (primary) and `core` (backup)** — deliberately **not** `hi`,
|
||||||
|
unlike the switches and `vibe`. `hi` is the jumbo (9000) VLAN, but `wave`'s eth0 caps at 2026 (see
|
||||||
|
MTU note), so there's no reason to put it there; `lo` is 1500 with a proper VRRP VIP for the default
|
||||||
|
route + resolver, and `core` has no VIP/v6 so it can only be a backup. No IP on `untrusted`. Records
|
||||||
|
in `dns.nix`:
|
||||||
|
|
||||||
|
| Name | VLAN | Address |
|
||||||
|
|---|---|---|
|
||||||
|
| `wave-core` | core (native/untagged) | `192.168.64.14/24` — backup, like the switches (no VIP → backup only) |
|
||||||
|
| `wave` | lo 110 | `192.168.72.14/21`, `2a0e:97c0:4d0:2::1:5` — primary; holds the default route + resolver (lo VIP `192.168.79.254` / `2a0e:97c0:4d0:2::ffff`) |
|
||||||
|
|
||||||
|
**Firewall:** management (SSH/LuCI) reachable from `core`/`lo` only; `untrusted` is a separate
|
||||||
|
zone with `input REJECT` (and `wave` has no IP there) — **no management via the guest VLAN**.
|
||||||
|
|
||||||
|
### brian switch port
|
||||||
|
|
||||||
|
`wave` hangs off **brian** (UniFi). Its port is a **trunk**: tagged VLAN **110/120** (`lo` + guest),
|
||||||
|
and **native/untagged = core** (the fabric's management VLAN, carrying `wave-core`). VLAN 100 (`hi`)
|
||||||
|
is **not** needed here — `wave` isn't on `hi` (see Management addressing). Configure via the UniFi
|
||||||
|
controller (brian has no CLI); see [switches.md](switches.md).
|
||||||
|
|
||||||
|
### Flashing OpenWrt (Cudy AX3000 / `cudy_ap3000-v1`)
|
||||||
|
|
||||||
|
Hardware: MT7981B, 512 MB RAM, 256 MB SPI-NAND, 1× 2.5 GbE (RTL8221B), 2×2 WiFi 6.
|
||||||
|
|
||||||
|
> ⚠️ **Serial caveat:** units with a serial starting `2543…` (post ~Nov 2025) use a different flash
|
||||||
|
> chip and can brick with older firmware. Match firmware to the unit.
|
||||||
|
|
||||||
|
OpenWrt can't be flashed directly over stock. Two-stage, via a Cudy **transition** firmware (Cudy
|
||||||
|
OpenWrt download page / `support@cudy.com`; `warnning.txt` in that bundle has the steps):
|
||||||
|
|
||||||
|
1. Stock Cudy UI: update to **≥ 2.4.7** (adds TFTP `recovery.bin` recovery), then flash the Cudy
|
||||||
|
**intermediate** firmware (`cudy_ap3000-v1-sysupgrade_*.bin`), "keep settings" **unchecked**.
|
||||||
|
It reboots into an OpenWrt-based build at `192.168.1.1` (SSH `root`, empty password).
|
||||||
|
2. From there, `sysupgrade -n` to vanilla OpenWrt (`…-cudy_ap3000-v1-squashfs-sysupgrade.bin` from
|
||||||
|
`downloads.openwrt.org`; this release ships **no** factory image — sysupgrade only).
|
||||||
|
|
||||||
|
Stock default (out of box) is a DHCP client falling back to **`192.168.10.254`**; the stock UI is a
|
||||||
|
customised LuCI (only 80/443, no SSH) with a first-boot "create admin password" wizard — so the
|
||||||
|
stock-side flashing is done from a browser, not headless.
|
||||||
|
|
||||||
|
### On-device config notes
|
||||||
|
|
||||||
|
#### Wireless packages
|
||||||
|
|
||||||
|
The package manager is `apk`, not `opkg`. Wi-Fi uses the full `wpad-mbedtls` package so all SSIDs
|
||||||
|
can enable 802.11k/v (`ieee80211k` + `bss_transition`). Replacing `wpad-basic-mbedtls` live leaves
|
||||||
|
the mac80211 interfaces in a start/teardown loop that reloads cannot recover; reboot after
|
||||||
|
`apk add wpad-mbedtls`.
|
||||||
|
|
||||||
|
- Radios: `radio0` = 2.4 GHz, `radio1` = 5 GHz (keyed by `band`, don't assume). 5 GHz is pinned to
|
||||||
|
**channel 36 / HE160** (any 160 MHz block in IE is DFS; ch36 has the shortest ~60 s CAC).
|
||||||
|
- Bridge: `br-lan` with `vlan_filtering`, single port `eth0` — tagged `110/120`, untagged/PVID
|
||||||
|
VLAN 1 (= native/core). SSIDs attach via `network` = `lo`/`untrusted` (= `br-lan.110`/`.120`).
|
||||||
|
- Dumb-AP: no DHCP pools, `odhcpd.maindhcp=0`, `delegate=0` on the L3 interfaces.
|
||||||
|
- **MTU:** all interfaces are **1500**. The `mtk_eth_soc` 2.5 GbE (`eth0`) caps at **2026 bytes**
|
||||||
|
(`ip link set eth0 mtu 9000` → `SIOCSIFMTU: Invalid argument`), so `wave` can't join `hi`'s jumbo
|
||||||
|
(9000) fabric like `vibe` does — which is precisely **why `wave` is managed on `lo`, not `hi`**
|
||||||
|
(see Management addressing). Nothing on `wave` needs > 1500.
|
||||||
|
- **LuCI:** enabled, login `root` / `admin`. **SSH:** key-only (`PasswordAuth`/`RootPasswordAuth off`).
|
||||||
|
- `iperf3` installed for throughput testing.
|
||||||
|
|
||||||
|
### Access
|
||||||
|
|
||||||
|
- SSH: `ssh root@wave` (key-only; `wave`/`wave-core` resolve once `dns.nix` is deployed).
|
||||||
|
- LuCI: `http://192.168.72.14/` (or `http://wave/`), `root` / `admin`.
|
||||||
+41
-27
@@ -5,42 +5,56 @@ root storage on NVMe-oF volumes from `cellar`.
|
|||||||
|
|
||||||
- **Source:** [`nixos/boxes/home/castle/`](../../../nixos/boxes/home/castle) (`default.nix`)
|
- **Source:** [`nixos/boxes/home/castle/`](../../../nixos/boxes/home/castle) (`default.nix`)
|
||||||
- **Host:** physical
|
- **Host:** physical
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
|
## Hardware
|
||||||
|
|
||||||
|
| Component | Inventory |
|
||||||
|
|---|---|
|
||||||
|
| Platform | ASUS ProArt X670E-CREATOR WIFI |
|
||||||
|
| CPU | AMD Ryzen 9 7950X (16 cores / 32 threads) |
|
||||||
|
| Memory | 64 GiB |
|
||||||
|
| Graphics | Integrated AMD Radeon graphics |
|
||||||
|
| Network | Mellanox ConnectX-4 100G, Aquantia AQC113CS 10G, Intel I225-V 2.5G and MediaTek MT7922 Wi-Fi 6E controllers |
|
||||||
|
| System storage | No local root disk; the box netboots and uses the SPDK NVMe-oF namespace exported by `cellar` |
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- AMD desktop running the GUI stack (`my.gui.enable`, Sway/Wayland via home-manager), PipeWire
|
### Desktop
|
||||||
(low-latency `quantum 128`, EasyEffects, jacktrip), Bluetooth, Thunderbolt (`bolt`).
|
|
||||||
- **Netboot client** (`my.netboot.client.enable`): the firmware iPXE-boots off the 2.5G NIC —
|
The AMD desktop runs the GUI stack (`my.gui.enable`, Sway/Wayland via home-manager), low-latency
|
||||||
kea's client-class for `castle` matches `et2.5g`'s MAC (`c8:7f:54:6e:17:0f`) and points at
|
PipeWire, Bluetooth and Thunderbolt. Local `libvirtd`/`virt-manager` and the IOMMU are enabled, but
|
||||||
`boot.h.nul.ie` on [`river`](river.md).
|
the box has no VFIO or GPU-passthrough configuration.
|
||||||
- **Root on NVMe-oF**: `/nix`, `/persist` and `/home` are `/dev/nvmeof/*` LVs on the
|
|
||||||
`nqn.2016-06.io.spdk:castle` namespace from [`cellar`](cellar.md) (`my.nvme.boot`,
|
### Netboot
|
||||||
`192.168.68.80`, RDMA). The initrd brings up `et100g`/`lan-hi` plus `roceBootModules` to reach
|
|
||||||
it, and the running system keeps `KeepConfiguration=static` on `lan-hi` so networkd never drops
|
With `my.netboot.client.enable`, the firmware iPXE-boots from the 2.5G NIC. Kea matches
|
||||||
the NVMe-oF address. The root itself is a 24 GiB tmpfs (`my.tmproot`).
|
`et2.5g`'s MAC and directs it to `boot.h.nul.ie` on [`river`](river.md).
|
||||||
- Local virtualisation: `libvirtd` + `virt-manager` are enabled and the IOMMU is on
|
|
||||||
(`amd_iommu=on`), but no VFIO/GPU-passthrough is configured in the box config today.
|
### NVMe-oF root
|
||||||
- Both firewalls are off (`networking.firewall.enable` and `my.firewall.enable` — it's a trusted
|
|
||||||
desktop on `hi`).
|
`/nix`, `/persist` and `/home` are `/dev/nvmeof/*` LVs in `cellar`'s
|
||||||
- Misc: `binfmt` emulation for `aarch64-linux`/`armv7l-linux`, `recursive-nix`, Wireshark,
|
`nqn.2016-06.io.spdk:castle` namespace (`my.nvme.boot`, RDMA). The initrd brings
|
||||||
`rdma-core`/`qperf` for the RoCE link. A `drm-amd-display` flicker patch sits commented out in
|
up `et100g`/`lan-hi` with `roceBootModules`; the running network keeps
|
||||||
`kernelPatches`.
|
`KeepConfiguration=static` so networkd does not drop the storage address. The root filesystem is a
|
||||||
|
size-limited tmpfs (`my.tmproot`).
|
||||||
|
|
||||||
|
### Other configuration
|
||||||
|
|
||||||
|
Both firewalls are disabled on this trusted `hi` desktop. Other settings include `binfmt`
|
||||||
|
emulation for `aarch64-linux`/`armv7l-linux`, `recursive-nix`, Wireshark and `rdma-core`/`qperf`;
|
||||||
|
a `drm-amd-display` flicker patch remains commented out.
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: castle -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `castle`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| castle | hi | `192.168.68.40/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::3:1/64` | h.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
- `et100g` (100G, MTU 9000) carries `lan-hi` (the `hi` assignment, statically `.40` — also pinned
|
- `et100g` (100G, MTU 9000) carries `lan-hi` (the `hi` assignment, also pinned by a kea
|
||||||
by a kea reservation on its MAC) and `lan-lo`.
|
reservation on its MAC) and `lan-lo`.
|
||||||
- `lan-lo` is a secondary leg: DHCPv4 with `UseGateway`/`UseDNS` off and RAs accepted with
|
- `lan-lo` is a secondary network attachment: DHCPv4 with `UseGateway`/`UseDNS` off and RAs
|
||||||
gateway/DNS use off — present for reaching `lo` devices, never a default route.
|
accepted with gateway/DNS use off — present for reaching `lo` devices, never a default route.
|
||||||
- `et2.5g` (netboot) and `et10g` are renamed but carry no network config.
|
- `et2.5g` (netboot) and `et10g` are renamed but carry no network config.
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|||||||
@@ -7,35 +7,30 @@ it.
|
|||||||
- **Source:** [`nixos/boxes/home/palace/vms/cellar/`](../../../nixos/boxes/home/palace/vms/cellar)
|
- **Source:** [`nixos/boxes/home/palace/vms/cellar/`](../../../nixos/boxes/home/palace/vms/cellar)
|
||||||
(`default.nix`, `spdk.nix`)
|
(`default.nix`, `spdk.nix`)
|
||||||
- **Host:** VM on `palace`
|
- **Host:** VM on `palace`
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- Runs an **SPDK userspace target** (`my.spdk`, [`spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix)):
|
- Runs an **SPDK userspace target** (`my.spdk`, [`spdk.nix`](../../../nixos/boxes/home/palace/vms/cellar/spdk.nix)):
|
||||||
the kernel `nvme` driver is blacklisted so SPDK can claim the three NVMe controllers directly
|
the kernel `nvme` driver is blacklisted so SPDK can claim the three NVMe controllers directly
|
||||||
(host BDFs `41:00.0`–`43:00.0`, attached in the guest as `02:00.0`–`04:00.0`).
|
(host BDFs `41:00.0`–`43:00.0`, attached in the guest as `02:00.0`–`04:00.0`).
|
||||||
- Builds a **RAID-0** (`NVMeRaid`, 64 KiB strip) across the three drives and exports one
|
- Builds a **RAID-0** (`NVMeRaid`) across the three drives and exports one
|
||||||
partition per consumer as an **NVMe-oF subsystem over RDMA** (port 4420) on the `hi` network:
|
partition per consumer as an **NVMe-oF subsystem over RDMA** (port 4420) on the `hi` network:
|
||||||
|
|
||||||
| Bdev | NQN | Consumer |
|
| Bdev | NQN | Consumer |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `NVMeRaidp1` | `nqn.2016-06.io.spdk:river` | [`river`](river.md) |
|
| `NVMeRaidp1` | `nqn.2016-06.io.spdk:river` | [`river`](river.md) |
|
||||||
| `NVMeRaidp2` | `nqn.2016-06.io.spdk:castle` | [`castle`](castle.md) |
|
| `NVMeRaidp2` | `nqn.2016-06.io.spdk:castle` | [`castle`](castle.md) |
|
||||||
| `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh.md) |
|
| `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh/README.md) |
|
||||||
|
|
||||||
Each subsystem is pinned to its consumer's `hostnqn` (the `my.nvme.uuid` on the client side).
|
Each subsystem is pinned to its consumer's `hostnqn` (the `my.nvme.uuid` on the client side).
|
||||||
- `spdk-tgt` is ordered after `lan-hi` is online; the RDMA listener binds the `hi` address
|
- `spdk-tgt` is ordered after `lan-hi` is online; the RDMA listener binds the `hi` assignment on
|
||||||
(`192.168.68.80:4420`). The VM itself is pinned to NUMA node 1 on `palace` and gets SR-IOV VF 0.
|
port 4420. The VM itself is pinned to NUMA node 1 on `palace` and gets SR-IOV VF 0.
|
||||||
- `netdata` (port 19999 allowed in the firewall) and `fstrim`.
|
- `netdata` (port 19999 allowed in the firewall) and `fstrim`.
|
||||||
- Deploy address `192.168.68.80` (`my.deploy.node.hostname`).
|
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: cellar -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `cellar`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| cellar | hi | `192.168.68.80/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:1/64` | h.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
|
|||||||
+25
-14
@@ -6,6 +6,19 @@ SR-IOV VFs, PCI NVMe drives and LVM disks.
|
|||||||
- **Source:** [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix)
|
- **Source:** [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix)
|
||||||
(VM definitions in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
|
(VM definitions in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
|
||||||
- **Host:** physical
|
- **Host:** physical
|
||||||
|
- **nixpkgs:** `mine-stable`
|
||||||
|
|
||||||
|
## Hardware
|
||||||
|
|
||||||
|
| Component | Inventory |
|
||||||
|
|---|---|
|
||||||
|
| Platform | Gigabyte X399 DESIGNARE EX |
|
||||||
|
| CPU | AMD Ryzen Threadripper 1950X (16 cores / 32 threads) |
|
||||||
|
| Memory | 128 GiB |
|
||||||
|
| Host storage | 500 GB Samsung SSD 860 EVO containing the EFI partition and the `main` LVM thin pool |
|
||||||
|
| Bulk storage | Three 8 TB Seagate IronWolf disks in the `hdds` VG, providing the RAID-backed `hdd-storage` and `frigate` LVs |
|
||||||
|
| NVMe storage | Three 2 TB Samsung NVMe devices passed through to `cellar`; SPDK combines them as the `NVMeRaid` RAID 0 device |
|
||||||
|
| Network / graphics | Mellanox ConnectX-4 100G adapter with four SR-IOV VFs, two Intel I211 Gigabit Ethernet controllers, and an AMD Radeon RX 550/560-family GPU |
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
@@ -14,26 +27,24 @@ SR-IOV VFs, PCI NVMe drives and LVM disks.
|
|||||||
the `main` thin pool (`services.lvm.boot.thin.enable`).
|
the `main` thin pool (`services.lvm.boot.thin.enable`).
|
||||||
- AMD box (`kvm-amd`, `amd_iommu=on`, microcode updates); the kernel is built with
|
- AMD box (`kvm-amd`, `amd_iommu=on`, microcode updates); the kernel is built with
|
||||||
`ACPI_APEI_PCIEAER`/`PCIEAER` for the PCIe passthrough work below.
|
`ACPI_APEI_PCIEAER`/`PCIEAER` for the PCIe passthrough work below.
|
||||||
- Deploy address `192.168.68.22` (`my.deploy.node.hostname`).
|
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: palace -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `palace`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| palace-core | core | `192.168.64.20/24` | — | h.nul.ie | |
|
|
||||||
| palace | hi | `192.168.68.22/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::2:1/64` | h.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
- 100G `et100g` (mlx5, MTU 9000) uplinks to the `dave` switch and carries `lan-hi` (VLAN 100,
|
100G `et100g` (mlx5, MTU 9000) uplinks to the `dave` switch and carries `lan-hi` (VLAN 100, the
|
||||||
the `hi` assignment). A udev rule creates **4 SR-IOV VFs** on the PF, distributed in
|
`hi` assignment). A udev rule creates four SR-IOV VFs on the PF:
|
||||||
`50-et100g`'s `[SR-IOV]` sections: VF 0 → `cellar` (VLAN hi), VF 1 → `river` (no VLAN — river
|
|
||||||
tags all of its own VLANs, including both WAN VLANs, on this trunk), VF 2 → `sfh` (VLAN hi),
|
| VF | Consumer | VLAN handling |
|
||||||
VF 3 → `sfh`'s container MACVLAN parent (VLAN hi).
|
|---|---|---|
|
||||||
- `lan-core` is a bridge with the `core` assignment (`192.168.64.20`, no gateway); the 1G
|
| 0 | `cellar` | `hi` |
|
||||||
|
| 1 | `river` | untagged VF; `river` tags its LAN and WAN VLANs |
|
||||||
|
| 2 | `sfh` | `hi` |
|
||||||
|
| 3 | `sfh` container MACVLAN parent | `hi` |
|
||||||
|
|
||||||
|
- `lan-core` is a bridge with the `core` assignment (no gateway); the 1G
|
||||||
`lan-core-phy` and the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used
|
`lan-core-phy` and the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used
|
||||||
for VM netboot and `lo` clients.
|
for VM netboot and `lo` clients.
|
||||||
- The 1G `et1g0` (igb) is exported to `river` as a passthru-mode macvtap (`vm-et1g0`) — river sees
|
- The 1G `et1g0` (igb) is exported to `river` as a passthru-mode macvtap (`vm-et1g0`) — river sees
|
||||||
|
|||||||
+55
-57
@@ -1,75 +1,73 @@
|
|||||||
# river
|
# river
|
||||||
|
|
||||||
The primary home router. A VM on `palace`, built from `routing-common` at index 0; it runs PPPoE
|
A home router VM on `palace` with a Digiweb PPPoE WAN on the ISP's VLAN 10. It forms the redundant
|
||||||
to Digiweb directly on the ISP's VLAN 10 and normally holds the master side of the VRRP pair with
|
router pair with [`stream`](stream.md).
|
||||||
[`stream`](stream.md).
|
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix)
|
- **Source:** [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix)
|
||||||
(shared router config: [`routing-common`](../../../nixos/boxes/home/routing-common), index 0)
|
(shared router config: [`routing-common`](../../../nixos/boxes/home/routing-common), index 0)
|
||||||
- **Host:** VM on `palace`
|
- **Host:** VM on `palace`
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- Primary of the router pair: `routing-common` index 0 → keepalived starts `MASTER` (priority
|
At `routing-common` index 0, `river` normally holds the primary position in the router pair.
|
||||||
255), kea serves the lower-half DHCP pools, the zone's SOA/`ns1` point here.
|
Pair-wide addressing, VIP, DHCP/DNS and failover behavior is documented in the
|
||||||
- Everything from `routing-common`: keepalived/VRRP ([`keepalived.nix`](../../../nixos/boxes/home/routing-common/keepalived.nix)),
|
[`home` networking overview](../../networking.md#home) and [Router HA](../../networking.md#router-ha).
|
||||||
PowerDNS recursor + authoritative with a blocklist ([`dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix)),
|
This page covers `river`'s Digiweb WAN, VM platform, storage and netboot duties.
|
||||||
kea DHCP + DDNS ([`kea.nix`](../../../nixos/boxes/home/routing-common/kea.nix)), radvd
|
|
||||||
([`radvd.nix`](../../../nixos/boxes/home/routing-common/radvd.nix)), NAT/firewall, the
|
|
||||||
`as211024` L2 mesh link back to colony, `iperf3`, `nginx`. See
|
|
||||||
[networking.md](../../networking.md) for the logical view and [README.md](README.md#router-vips)
|
|
||||||
for the floating VIPs.
|
|
||||||
- **Netboot server** for `sfh` and `castle` (`my.netboot.server`): iPXE/TFTP at
|
|
||||||
`boot.h.nul.ie` (a CNAME to `river-hi`), served from the `lo` address `192.168.72.1` and
|
|
||||||
restricted to the hi/lo prefixes.
|
|
||||||
- **NVMe-oF client of `cellar`**: the VM's only local disk is an ESP (an installer ISO is also
|
|
||||||
still attached); `/nix` and `/persist` are LVs on the `nqn.2016-06.io.spdk:river` namespace
|
|
||||||
exported by `cellar` (`192.168.68.80`, RDMA).
|
|
||||||
The initrd brings up `lan-hi` with the RoCE modules (`roceBootModules`), and
|
|
||||||
`KeepConfiguration=static` on `lan-hi` stops networkd from dropping the NVMe-oF address on
|
|
||||||
reconfigure.
|
|
||||||
- **SR-IOV VF passthrough**: the 100G `lan` NIC is VF 1 of palace's `et100g` (MAC
|
|
||||||
`52:54:00:8a:8a:f2`, MTU 9000). All router VLANs — hi/lo/untrusted plus both WAN VLANs — are
|
|
||||||
tagged on top of it (`55-lan`).
|
|
||||||
- Also carries a macvtap passthrough of palace's 1G `et1g0`, renamed `wan-old` — the pre-100G WAN
|
|
||||||
path, kept around with no L3 config today.
|
|
||||||
- Deploy address `192.168.68.1` (`my.deploy.node.hostname`).
|
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: river -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `river`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| river | as211024 | `10.100.50.2/24` | `2a0e:97c0:4df:0:1::1/64 gw 2a0e:97c0:4df:0:2::1` | — | |
|
|
||||||
| river-core | core | `192.168.64.1/24` | — | h.nul.ie | |
|
|
||||||
| river-hi | hi | `192.168.68.1/22` | `2a0e:97c0:4d0:1::1/64` | h.nul.ie | |
|
|
||||||
| river-lo | lo | `192.168.72.1/21` | `2a0e:97c0:4d0:2::1/64` | h.nul.ie | |
|
|
||||||
| river-ut | untrusted | `192.168.80.1/24` | `2a0e:97c0:4d0:3::1/64` | h.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## WAN (Digiweb PPPoE)
|
## WAN (Digiweb PPPoE)
|
||||||
|
|
||||||
- `services.pppd` peer `digiweb` attaches PPPoE directly to `wan-pon-isp`
|
### Link and addressing
|
||||||
(`plugin pppoe.so wan-pon-isp`) — the **raw ISP VLAN 10** (`vlans.pon-isp`), trunked
|
|
||||||
untranslated from the ONT through `brian` and `dave` (the fabric side is in
|
`services.pppd` peer `digiweb` attaches directly to `wan-pon-isp`, the raw ISP VLAN 10
|
||||||
[switches.md](switches.md)). The netdev is up with no L3; `MTUBytes=1508` (baby jumbo) absorbs
|
(`vlans.pon-isp`) trunked untranslated from the ONT through `brian` and `dave`; see
|
||||||
PPPoE's 8-byte overhead so the `wan` ppp interface gets a clean `mtu`/`mru 1500`.
|
[switches.md](switches.md). The netdev has no L3 configuration and uses `MTUBytes=1508`, leaving a
|
||||||
- The Digiweb static IP (`84.203.124.128`, `elemAt routersPubV4 0`) is requested as the local
|
clean `mtu`/`mru 1500` after PPPoE overhead.
|
||||||
address in IPCP. Auth is the across-all-customers shared `digiweb@nga.digiweb.ie` / `digiweb`
|
|
||||||
(deliberately not a secret); `persist`, `maxfail 0`, 1s LCP echoes. `usepeerdns` is absent on
|
The static `84.203.124.128` address is requested through IPCP. The provider-wide credentials are
|
||||||
purpose — Digiweb's resolvers are ignored in favour of the local recursor.
|
deliberately not secret; the peer persists indefinitely with LCP echo monitoring and ignores
|
||||||
- **ONT management**: `wan-pon-ont` (VLAN 140, PVID'd at the `brian` edge) holds
|
Digiweb DNS in favor of the local recursor.
|
||||||
`192.168.100.100/24` so the ONT web UI at `192.168.100.1` is reachable — the `.100` mirrors
|
|
||||||
stream's modem-management convention.
|
### Management subnet
|
||||||
- **`wan-online.target`** is the shared "public WAN is up" gate declared by `routing-common`;
|
|
||||||
here it is driven by the pppd hooks (`DefaultDependencies=false`, so nothing pulls it in
|
`wan-pon-ont` (VLAN 140, PVID'd at `brian`) holds `192.168.100.100/24`, reaching the ONT UI at
|
||||||
early): `ip-up` installs `default dev wan scope link metric 100` and starts the target,
|
`192.168.100.1`. The `.100` address follows `stream`'s modem-management convention.
|
||||||
`ip-down` stops it and deletes the route. Consumers (`ipsec`, `ipv6-clear-default-route`)
|
|
||||||
attach with `wantedBy` + `partOf`, so they re-load on every WAN flap.
|
### WAN readiness
|
||||||
- The `wan-ifb` ingress-shaping pieces from `routing-common` are inert on this box: the CAKE
|
|
||||||
config itself is stream's, and `networkd-dispatcher` (which installs the `tc` mirror) is
|
The `pppd` hooks drive this shared gate with `DefaultDependencies=false`: `ip-up` installs the
|
||||||
`mkForce false` here pending scheduling testing.
|
link-scoped default route and starts the target, while `ip-down` stops it and removes the route.
|
||||||
|
`ipsec` and `ipv6-clear-default-route` attach through `wantedBy` + `partOf`, so they reload after
|
||||||
|
every WAN flap.
|
||||||
|
|
||||||
|
### Traffic shaping
|
||||||
|
|
||||||
|
The shared `wan-ifb` ingress-shaping pieces are inert here. CAKE is specific to `stream`, and
|
||||||
|
`networkd-dispatcher` is `mkForce false` pending scheduling tests.
|
||||||
|
|
||||||
|
## Platform
|
||||||
|
|
||||||
|
### Virtual machine and network attachment
|
||||||
|
|
||||||
|
The 100G `lan` NIC is VF 1 of `palace`'s `et100g` (MTU 9000), with every router VLAN tagged on top
|
||||||
|
as `55-lan`. A macvtap of `palace`'s 1G `et1g0` remains as the old `wan-old` path without L3
|
||||||
|
configuration. Deployments use the `hi` assignment.
|
||||||
|
|
||||||
|
### Storage
|
||||||
|
|
||||||
|
The VM's local disk holds only an ESP; `/nix` and `/persist` are LVs on `cellar`'s
|
||||||
|
`nqn.2016-06.io.spdk:river` namespace over RDMA. The initrd brings up `lan-hi`
|
||||||
|
with `roceBootModules`, and `KeepConfiguration=static` prevents networkd from dropping the address
|
||||||
|
during reconfiguration. An installer ISO remains attached.
|
||||||
|
|
||||||
|
## Netboot
|
||||||
|
|
||||||
|
`my.netboot.server` serves iPXE/TFTP for `sfh` and `castle` at `boot.h.nul.ie` from the `lo`
|
||||||
|
assignment, restricted to the `hi` and `lo` prefixes.
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
@@ -1,21 +1,22 @@
|
|||||||
# sfh
|
# sfh
|
||||||
|
|
||||||
"Services for home" — the NixOS container host for the home site. A VM on `palace` that netboots
|
"Shill from home" — the NixOS container host for the home site. A VM on `palace` that netboots
|
||||||
from `river` and runs its root off NVMe-oF from `cellar`.
|
from `river` and runs its root off NVMe-oF from `cellar`.
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/home/palace/vms/sfh/`](../../../nixos/boxes/home/palace/vms/sfh)
|
- **Source:** [`nixos/boxes/home/palace/vms/sfh/`](../../../../nixos/boxes/home/palace/vms/sfh)
|
||||||
(`default.nix`, `containers/`)
|
(`default.nix`, `containers/`)
|
||||||
- **Host:** VM on `palace`
|
- **Host:** VM on `palace`
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- Runs the home NixOS containers via `my.containers.instances` (systemd-nspawn); each container is
|
- Runs the home NixOS containers via `my.containers.instances` (systemd-nspawn); each container is
|
||||||
its own `nixos.systems.*` entry rendered through `my.asContainer`.
|
its own `nixos.systems.*` entry rendered through `my.asContainer`.
|
||||||
- **Netboot client** (`my.netboot.client.enable`): the VM has no boot disk — its `netboot` NIC
|
- **Netboot client** (`my.netboot.client.enable`): the VM has no boot disk — its `netboot` NIC
|
||||||
(MAC `52:54:00:a5:7e:93`, on palace's `lan-lo` bridge, `bootindex=1`) is matched by a kea
|
on palace's `lan-lo` bridge is matched by a kea client-class on [`river`](../river.md) and
|
||||||
client-class on [`river`](river.md) and iPXE-boots from `boot.h.nul.ie`.
|
iPXE-boots from `boot.h.nul.ie`.
|
||||||
- **Root on NVMe-oF**: `my.nvme.boot` connects to `nqn.2016-06.io.spdk:sfh` at `192.168.68.80`
|
- **Root on NVMe-oF**: `my.nvme.boot` connects to `nqn.2016-06.io.spdk:sfh`
|
||||||
([`cellar`](cellar.md), RDMA) from the initrd (`lan-hi` up + `roceBootModules`); `/nix` and
|
([`cellar`](../cellar.md), RDMA) from the initrd (`lan-hi` up + `roceBootModules`); `/nix` and
|
||||||
`/persist` are LVs on that volume. `KeepConfiguration=static` on `lan-hi` protects the
|
`/persist` are LVs on that volume. `KeepConfiguration=static` on `lan-hi` protects the
|
||||||
NVMe-oF address from networkd reconfigures.
|
NVMe-oF address from networkd reconfigures.
|
||||||
- **Frigate footage disk**: palace passes the `hdds/frigate` LVM LV through as a virtio disk;
|
- **Frigate footage disk**: palace passes the `hdds/frigate` LVM LV through as a virtio disk;
|
||||||
@@ -26,38 +27,33 @@ from `river` and runs its root off NVMe-oF from `cellar`.
|
|||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: sfh -->
|
See the consolidated [network assignments](../../../networking.md#box-assignments) table (this box: `sfh`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| sfh | hi | `192.168.68.81/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::4:2/64` | h.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
Four NICs, all MTU 9000 where jumbo-capable:
|
Four NICs, all MTU 9000 where jumbo-capable:
|
||||||
|
|
||||||
- `lan-hi` — SR-IOV VF 2, the box's own `hi` assignment (`192.168.68.81`).
|
- `lan-hi` — SR-IOV VF 2, the box's own `hi` assignment.
|
||||||
- `lan-hi-ctrs` — SR-IOV VF 3, no L3: the MACVLAN parent for the containers' `hi` legs
|
- `lan-hi-ctrs` — SR-IOV VF 3, no L3: the MACVLAN parent for the containers' `hi` interfaces
|
||||||
(`host0` inside each container).
|
(`host0` inside each container).
|
||||||
- `lan-core-ctrs` / `lan-lo-ctrs` — virtio NICs (bridged to palace's `lan-core` / `lan-lo`), no
|
- `lan-core-ctrs` / `lan-lo-ctrs` — virtio NICs (bridged to palace's `lan-core` / `lan-lo`), no
|
||||||
L3: MACVLAN parents for containers that need a `core` or `lo` leg.
|
L3: MACVLAN parents for containers that need a `core` or `lo` interface.
|
||||||
|
|
||||||
The per-container MACVLAN wiring lives in `systemd.nspawn.*.networkConfig` in
|
The per-container MACVLAN wiring lives in `systemd.nspawn.*.networkConfig` in
|
||||||
[`sfh/default.nix`](../../../nixos/boxes/home/palace/vms/sfh/default.nix).
|
[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix).
|
||||||
|
|
||||||
## Containers
|
## Containers
|
||||||
|
|
||||||
| Container | Role | Page |
|
| Container | Role |
|
||||||
|---|---|---|
|
|---|---|
|
||||||
| `hass` | Home Assistant + Frigate + MQTT | [sfh/containers/hass.md](sfh/containers/hass.md) |
|
| [`hass`](containers/hass.md) | Home Assistant + Frigate + MQTT |
|
||||||
| `unifi` | UniFi controller | [sfh/containers/unifi.md](sfh/containers/unifi.md) |
|
| [`unifi`](containers/unifi.md) | UniFi controller |
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
- [`nixos/boxes/home/palace/vms/sfh/default.nix`](../../../nixos/boxes/home/palace/vms/sfh/default.nix) —
|
- [`nixos/boxes/home/palace/vms/sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix) —
|
||||||
box config: netboot/NVMe-oF boot, container instances, MACVLAN plumbing, Frigate disk.
|
box config: netboot/NVMe-oF boot, container instances, MACVLAN plumbing, Frigate disk.
|
||||||
- [`nixos/boxes/home/palace/vms/sfh/containers/`](../../../nixos/boxes/home/palace/vms/sfh/containers) —
|
- [`nixos/boxes/home/palace/vms/sfh/containers/`](../../../../nixos/boxes/home/palace/vms/sfh/containers) —
|
||||||
the container system definitions.
|
the container system definitions.
|
||||||
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
|
- [`nixos/boxes/home/palace/vms/default.nix`](../../../../nixos/boxes/home/palace/vms/default.nix) —
|
||||||
the VM definition on `palace` (VFs, USB passthrough, netboot NIC, `hdds/frigate` disk).
|
the VM definition on `palace` (VFs, USB passthrough, netboot NIC, `hdds/frigate` disk).
|
||||||
@@ -1,23 +1,27 @@
|
|||||||
# hass
|
# hass
|
||||||
|
|
||||||
Home automation container: Home Assistant plus its supporting services (MQTT, camera restreaming,
|
Home automation container: Home Assistant plus its supporting services (MQTT, camera restreaming,
|
||||||
Frigate NVR), running on [`sfh`](../../sfh.md).
|
Frigate NVR), running on [`sfh`](../README.md).
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/hass.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/hass.nix)
|
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/hass.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/hass.nix)
|
||||||
- **Host:** NixOS container on `sfh`
|
- **Host:** NixOS container on `sfh`
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- **Home Assistant** (`services.home-assistant`) — declarative config (`configWritable = false`),
|
### Home Assistant
|
||||||
components `esphome`, `zha`, `denonavr`, `webostv`, `androidtv_remote`, `heos`, `mqtt`, `wled`,
|
|
||||||
`met`, `google_translate`; custom components `alarmo`, `frigate`, `west_wood_club`; a custom
|
`services.home-assistant` uses declarative configuration (`configWritable = false`). It enables
|
||||||
Irish Rail sensor (Glenageary ↔ Dublin Connolly). A `hass-cli` wrapper is on the box, wired to
|
the `esphome`, `zha`, `denonavr`, `webostv`, `androidtv_remote`, `heos`, `mqtt`, `wled`, `met` and
|
||||||
the local server with a token from `my.secrets`.
|
`google_translate` components, plus custom `alarmo`, `frigate`, `west_wood_club` and Irish Rail
|
||||||
|
integrations. A `hass-cli` wrapper uses a token from `my.secrets` to reach the local server.
|
||||||
|
|
||||||
- **mosquitto** — MQTT broker (anonymous local listener; port 1883 allowed, alongside HTTP).
|
- **mosquitto** — MQTT broker (anonymous local listener; port 1883 allowed, alongside HTTP).
|
||||||
- **go2rtc** — restreams the Reolink living-room camera (RTSP from `reolink-living-room`, on the
|
- **go2rtc** — restreams the Reolink living-room camera (RTSP from `reolink-living-room`, on the
|
||||||
`lo` leg) and the office USB webcam (`/dev/video0` via ffmpeg).
|
`lo` network) and the office USB webcam (`/dev/video0` via ffmpeg).
|
||||||
- **Frigate** (`services.frigate`, `frigate.h.nul.ie` — the `frigate` alt name on the `hi`
|
- **Frigate** (`services.frigate`, `frigate.h.nul.ie` — the `frigate` alt name on the `hi`
|
||||||
assignment) — records both restreamed cameras with 1-day retention; detection is disabled.
|
assignment) — records both restreamed cameras with a short retention policy; detection is
|
||||||
|
disabled.
|
||||||
- External access is via `https://hass.nul.ie` through the `middleman` reverse proxy
|
- External access is via `https://hass.nul.ie` through the `middleman` reverse proxy
|
||||||
(`trusted_proxies`); internally it's `hass-ctr.h.nul.ie`.
|
(`trusted_proxies`); internally it's `hass-ctr.h.nul.ie`.
|
||||||
- Not a deploy-rs target (`my.deploy.enable = false`) — it's rendered via `my.asContainer` and
|
- Not a deploy-rs target (`my.deploy.enable = false`) — it's rendered via `my.asContainer` and
|
||||||
@@ -25,13 +29,7 @@ Frigate NVR), running on [`sfh`](../../sfh.md).
|
|||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: hass -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `hass`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| hass-ctr (frigate) | hi | `192.168.68.103/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:3/64` | h.nul.ie | |
|
|
||||||
| hass-ctr-lo | lo | `192.168.72.103/21` | `2a0e:97c0:4d0:2::5:3/64` | h.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## Storage
|
## Storage
|
||||||
|
|
||||||
@@ -45,13 +43,13 @@ Passed through from `sfh` (USB host ports on `palace`):
|
|||||||
|
|
||||||
- Nabu Casa Connect ZBT-1 Zigbee coordinator → `/dev/ttyUSB0` (used by `zha`).
|
- Nabu Casa Connect ZBT-1 Zigbee coordinator → `/dev/ttyUSB0` (used by `zha`).
|
||||||
- USB webcam → `/dev/video0` (go2rtc's `webcam_office` stream).
|
- USB webcam → `/dev/video0` (go2rtc's `webcam_office` stream).
|
||||||
- `/dev/bus/usb/001/002` (raw USB device node).
|
- The matching raw USB device node allowed through from `sfh`.
|
||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
MACVLAN legs created from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment,
|
MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment,
|
||||||
alt name `frigate`, default gateway via the VIP) and `lan-lo` on `lan-lo-ctrs` (the `lo`
|
alt name `frigate`, default gateway via the VIP) and `lan-lo` on `lan-lo-ctrs` (the `lo`
|
||||||
assignment, no gateway) — the `lo` leg reaches the IoT devices (the Reolink camera lives there).
|
assignment, no gateway) — the `lo` interface reaches the IoT devices (the Reolink camera lives there).
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
@@ -1,19 +1,11 @@
|
|||||||
# unifi
|
# unifi
|
||||||
|
|
||||||
The UniFi network controller, running as a container on [`sfh`](../../sfh.md). It manages the
|
The UniFi network controller, running as a container on [`sfh`](../README.md). It manages the
|
||||||
home UniFi switch `brian` (see [switches.md](../../switches.md)).
|
home UniFi switch `brian` (see [switches.md](../../switches.md)).
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/unifi.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/unifi.nix)
|
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/unifi.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/unifi.nix)
|
||||||
- **Host:** NixOS container on `sfh`
|
- **Host:** NixOS container on `sfh`
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
## Status
|
|
||||||
|
|
||||||
**Currently enabled.** The container spent a while disabled — its import was commented out of
|
|
||||||
[`containers/default.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/default.nix)
|
|
||||||
while there was no UniFi gear to manage — and was re-enabled when the UniFi switch `brian` was
|
|
||||||
added, gaining a `core` leg (`unifi-ctr-core`) at the same time so it can reach the switch on its
|
|
||||||
management network. It is imported, listed in `sfh`'s `my.containers.instances`, and
|
|
||||||
`services.unifi.enable = true`.
|
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
@@ -24,20 +16,23 @@ management network. It is imported, listed in `sfh`'s `my.containers.instances`,
|
|||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: unifi -->
|
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `unifi`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
## Status
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| unifi-ctr-core | core | `192.168.64.21/24` | — | h.nul.ie | |
|
**Currently enabled.** The container spent a while disabled — its import was commented out of
|
||||||
| unifi-ctr | hi | `192.168.68.100/22 gw 192.168.71.254` | `2a0e:97c0:4d0:1::5:1/64` | h.nul.ie | |
|
[`containers/default.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/default.nix)
|
||||||
<!-- assignments-end -->
|
while there was no UniFi gear to manage — and was re-enabled when the UniFi switch `brian` was
|
||||||
|
added, gaining a `core` interface (`unifi-ctr-core`) at the same time so it can reach the switch on its
|
||||||
|
management network. It is imported, listed in `sfh`'s `my.containers.instances`, and
|
||||||
|
`services.unifi.enable = true`.
|
||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
Two MACVLAN legs from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment —
|
Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` carries the `hi`
|
||||||
`unifi-ctr`, `192.168.68.100`, default gateway via the VIP) and `lan-core` on `lan-core-ctrs`
|
assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `lan-core-ctrs` carries
|
||||||
(the `core` assignment — `unifi-ctr-core`, `192.168.64.21/24`, no gateway). The `core` leg is
|
the gatewayless `core` assignment (`unifi-ctr-core`). The `core` interface is how the controller
|
||||||
how the controller talks to the switches: `brian` lives at `192.168.64.13` on `core`.
|
talks to `brian` and the other switches on their management network.
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
+58
-42
@@ -1,77 +1,93 @@
|
|||||||
# stream
|
# stream
|
||||||
|
|
||||||
The secondary home router. A physical Intel box, built from `routing-common` at index 1; its WAN
|
A physical Intel home router with a DHCP WAN from the Virgin Media cable modem. It forms the
|
||||||
is a DHCP lease from the Virgin Media cable modem, and it is dual-homed to both switches.
|
redundant router pair with [`river`](river.md) and is dual-homed to both switches.
|
||||||
|
|
||||||
- **Source:** [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) (shared router
|
- **Source:** [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) (shared router
|
||||||
config: [`routing-common`](../../../nixos/boxes/home/routing-common), index 1)
|
config: [`routing-common`](../../../nixos/boxes/home/routing-common), index 1)
|
||||||
- **Host:** physical
|
- **Host:** physical
|
||||||
|
- **nixpkgs:** `mine`
|
||||||
|
|
||||||
|
## Hardware
|
||||||
|
|
||||||
|
| Component | Inventory |
|
||||||
|
|---|---|
|
||||||
|
| Platform | BROUNION R86S |
|
||||||
|
| CPU | Intel Celeron N5105 (4 cores / 4 threads) |
|
||||||
|
| Memory | 16 GiB |
|
||||||
|
| Storage | 512 GB Samsung SSD 970 PRO NVMe containing `/boot`, `/nix` and `/persist`; integrated 128 GB eMMC is present but is not used by the declared filesystems |
|
||||||
|
| Network | Three Intel `igc` interfaces and a dual-port Mellanox `mlx4_en` adapter; `wan`, `lan-jim` and `lan-dave` use three of these ports |
|
||||||
|
|
||||||
|
The platform configuration enables `kvm-intel`, `intel_iommu=on` and Intel microcode updates.
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
- Backup of the router pair: `routing-common` index 1 → keepalived starts `BACKUP` (priority
|
At `routing-common` index 1, `stream` normally holds the secondary position in the router pair.
|
||||||
254), kea serves the upper-half DHCP pools, the zone's `ns2` points here. Takes over all VIPs
|
Pair-wide addressing, VIP, DHCP/DNS and failover behavior is documented in the
|
||||||
when [`river`](river.md) is down — see [README.md](README.md#router-vips).
|
[`home` networking overview](../../networking.md#home) and [Router HA](../../networking.md#router-ha).
|
||||||
- Runs the same `routing-common` services as river: keepalived/VRRP, PowerDNS recursor +
|
This page covers `stream`'s Virgin Media WAN, physical platform and redundant switch attachment.
|
||||||
authoritative, kea DHCP + DDNS, radvd, NAT/firewall, the `as211024` L2 mesh, `iperf3`, `nginx`.
|
|
||||||
- Intel box (`kvm-intel`, `intel_iommu=on`, microcode updates).
|
|
||||||
- `octoprint` and `mjpg-streamer` (3D-printer services) are defined but **disabled**
|
|
||||||
(`enable = false`).
|
|
||||||
- `my.deploy.node.hostname` is currently commented out (it was `192.168.68.2`).
|
|
||||||
|
|
||||||
## Network assignments
|
## Network assignments
|
||||||
|
|
||||||
<!-- assignments: stream -->
|
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box: `stream`).
|
||||||
<!-- assignments-start -->
|
|
||||||
| Name | Assignment | IPv4 | IPv6 | Domain | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| stream | as211024 | `10.100.50.3/24` | `2a0e:97c0:4df:0:1::2/64 gw 2a0e:97c0:4df:0:2::1` | — | |
|
|
||||||
| stream-core | core | `192.168.64.2/24` | — | h.nul.ie | |
|
|
||||||
| stream-hi | hi | `192.168.68.2/22` | `2a0e:97c0:4d0:1::2/64` | h.nul.ie | |
|
|
||||||
| stream-lo | lo | `192.168.72.2/21` | `2a0e:97c0:4d0:2::2/64` | h.nul.ie | |
|
|
||||||
| stream-ut | untrusted | `192.168.80.2/24` | `2a0e:97c0:4d0:3::2/64` | h.nul.ie | |
|
|
||||||
<!-- assignments-end -->
|
|
||||||
|
|
||||||
## WAN (Virgin Media DHCP)
|
## WAN (Virgin Media DHCP)
|
||||||
|
|
||||||
- `wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) towards the cable modem. The modem segment is
|
### Link and addressing
|
||||||
switch **VLAN 130** — tagging is handled on the switch (`jim`), so the box interface itself is
|
|
||||||
untagged. The fabric side is in [switches.md](switches.md).
|
|
||||||
- `DHCP=ipv4` pulls the public lease; `dhcpV4Config.UseDNS=false` and the interface DNS points at
|
|
||||||
the local recursor. `IPv6AcceptRA=false` — this is an IPv4-only WAN (public IPv6 arrives over
|
|
||||||
the tunnel, not this link).
|
|
||||||
- A **static modem-management address** (`192.168.0.100/24`, host `.100` of `prefixes.modem.v4`)
|
|
||||||
sits on `wan` alongside the DHCP lease so the modem's web UI stays reachable; it has no
|
|
||||||
gateway.
|
|
||||||
- **`wan-online.target`** wiring: `wan-wait-online.service` (a oneshot) polls until the DHCP
|
|
||||||
default route exists, and the target `requires`/`after`s it (`wantedBy multi-user.target`).
|
|
||||||
The route — not networkd's wait-online — is the gate because the permanent static modem address
|
|
||||||
would otherwise report "online" before the public lease arrives, letting `ipsec` start
|
|
||||||
unoriented (`left=` is the public IP) and never connect.
|
|
||||||
- **CAKE QoS**: egress is shaped at the `wan` root qdisc (`Bandwidth=48M`); ingress is redirected
|
|
||||||
by `tc` (`mirred`, installed by the `networkd-dispatcher` rule in `routing-common`) into the
|
|
||||||
`wan-ifb` IFB at `Bandwidth=490M` with the DOCSIS overhead preset.
|
|
||||||
|
|
||||||
### Modem specifics (de-shared from `routing-common`)
|
`wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) towards the cable modem. The modem segment is
|
||||||
|
switch VLAN 130; `jim` handles the tag, so the box interface is untagged. See
|
||||||
|
[switches.md](switches.md) for the fabric side.
|
||||||
|
|
||||||
|
`DHCP=ipv4` pulls the public lease. `dhcpV4Config.UseDNS=false` points resolution at the local
|
||||||
|
recursor, and `IPv6AcceptRA=false` because public IPv6 arrives over the tunnel rather than this WAN.
|
||||||
|
|
||||||
|
### Management subnet
|
||||||
|
|
||||||
|
The static `192.168.0.100/24` address (host `.100` of `prefixes.modem.v4`) sits on `wan` without a
|
||||||
|
gateway, keeping the modem UI reachable alongside the DHCP lease.
|
||||||
|
|
||||||
|
### WAN readiness
|
||||||
|
|
||||||
|
`wan-wait-online.service` polls until the DHCP default route exists, then satisfies
|
||||||
|
`wan-online.target`. The route is the gate because the permanent modem address would make
|
||||||
|
networkd's wait-online report success before the public lease arrives, allowing `ipsec` to start
|
||||||
|
without its public `left=` address.
|
||||||
|
|
||||||
|
### Traffic shaping
|
||||||
|
|
||||||
|
Egress is shaped at the `wan` root qdisc. A `routing-common` `networkd-dispatcher` rule redirects
|
||||||
|
ingress through `tc`/`mirred` into `wan-ifb`; each direction has its own configured bandwidth and
|
||||||
|
uses the DOCSIS overhead preset.
|
||||||
|
|
||||||
|
### Per-box `routing-common` options
|
||||||
|
|
||||||
The modem's management subnet shares the `wan` interface, which `routing-common` itself knows
|
The modem's management subnet shares the `wan` interface, which `routing-common` itself knows
|
||||||
nothing about — it declares two per-box options
|
nothing about — it declares two per-box options
|
||||||
([`routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix)) that this
|
([`routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix)) that this
|
||||||
box sets:
|
box sets:
|
||||||
|
|
||||||
- `my.homeRouter.dns.wanSkipBroadcasts = [ 192.168.0.255 ]` — skip the modem subnet when
|
- `my.homeRouter.dns.wanSkipBroadcasts` — skip the modem subnet's broadcast address when
|
||||||
auto-selecting the router's own `wan` A record for the zone's LUA record.
|
auto-selecting the router's own `wan` A record for the zone's LUA record.
|
||||||
- `my.homeRouter.firewall.untrustedRejectV4 = [ 192.168.0.0/24 ]` — reject untrusted clients from
|
- `my.homeRouter.firewall.untrustedRejectV4` — reject untrusted clients from
|
||||||
reaching the modem subnet (needed only because it shares `wan`; WAN egress is otherwise
|
reaching the modem subnet (needed only because it shares `wan`; WAN egress is otherwise
|
||||||
accepted).
|
accepted).
|
||||||
|
|
||||||
## Switching (STP)
|
## Switching (RSTP)
|
||||||
|
|
||||||
`stream` is dual-homed to both switches: `lan-jim` (igc) and `lan-dave` (mlx4_en), both MTU 9000,
|
`stream` is dual-homed to both switches: `lan-jim` (igc) and `lan-dave` (mlx4_en), both MTU 9000,
|
||||||
are enslaved to the `lan` bridge with `STP=true`. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
|
are enslaved to the `lan` bridge with `STP=true`. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
|
||||||
runs a patched `mstpd` and forces RSTP on `lan` once it's routable, so exactly one uplink carries
|
runs a patched `mstpd` and forces RSTP on `lan` once it's routable, so exactly one uplink carries
|
||||||
traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfigured.)
|
traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfigured.)
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
`my.deploy.node.hostname` is currently commented out.
|
||||||
|
|
||||||
|
## Disabled printer services
|
||||||
|
|
||||||
|
`octoprint` and `mjpg-streamer` are defined but disabled (`enable = false`).
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
- [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) — box config: DHCP WAN,
|
- [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) — box config: DHCP WAN,
|
||||||
|
|||||||
+74
-42
@@ -10,17 +10,18 @@ and the ONT's untagged management is PVID'd onto VLAN 140 at brian, its edge swi
|
|||||||
carried untranslated because a single ONT makes it unique on the fabric — see
|
carried untranslated because a single ONT makes it unique on the fabric — see
|
||||||
[the WAN path](#the-digiweb-wan-path-trunked-vlan-10--pvid-140) and
|
[the WAN path](#the-digiweb-wan-path-trunked-vlan-10--pvid-140) and
|
||||||
[why not translation](#why-not-translation-for-one-ont). The router side lives in
|
[why not translation](#why-not-translation-for-one-ont). The router side lives in
|
||||||
[river.md](river.md); the logical network map in [networking.md](../../networking.md).
|
[river.md](river.md); the logical network map in [networking.md](../../networking.md). The Wi-Fi
|
||||||
|
APs that hang off these switches are in [aps.md](aps.md).
|
||||||
|
|
||||||
## The switches
|
## The switches
|
||||||
|
|
||||||
| | jim | dave | brian |
|
| | jim | dave | brian |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| Identity | `jim-sw` | `dave-sw` | (UniFi) |
|
| Identity | `jim-sw` | `dave-sw` | (UniFi) |
|
||||||
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti 10GBASE-T |
|
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti Switch Pro XG 8 PoE |
|
||||||
| Switch chip | Marvell 98DX3236 | Marvell 98DX4310 (+ Atheros 8227 for the 1G mgmt port) | — |
|
| Switch chip | Marvell 98DX3236 | Marvell 98DX4310 (+ Atheros 8227 for the 1G mgmt port) | — |
|
||||||
| OS | RouterOS 7.18 | RouterOS 7.18 | UniFi |
|
| OS | RouterOS | RouterOS | UniFi |
|
||||||
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 (100G, breakout-capable) + 1G mgmt | 10GBASE-T |
|
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 (100G, breakout-capable) + 1G mgmt | 8×10GBASE-T PoE + 2×10G SFP+ |
|
||||||
| Bridge | `main`, `vlan-filtering=yes` | `main`, `vlan-filtering=yes` | UniFi VLAN profiles |
|
| Bridge | `main`, `vlan-filtering=yes` | `main`, `vlan-filtering=yes` | UniFi VLAN profiles |
|
||||||
|
|
||||||
jim and dave run a single hardware-offloaded bridge (`main`) with VLAN filtering. Access to the
|
jim and dave run a single hardware-offloaded bridge (`main`) with VLAN filtering. Access to the
|
||||||
@@ -30,47 +31,45 @@ chips); brian cannot rewrite tags, only trunk/PVID them.
|
|||||||
|
|
||||||
## Physical topology
|
## Physical topology
|
||||||
|
|
||||||
The ONT terminates on brian; jim's `wan-pon-in` (`sfp-sfpplus2`) is a spare SFP+ port.
|
The two WAN sources enter at the top: the Virgin Media modem lands on **jim** (VLAN 130), and the
|
||||||
|
Digiweb **ONT** lands on **brian**. Both `jim` and `brian` are edge switches that uplink down into
|
||||||
|
the **dave** core; the home boxes hang off dave's 100G ports, with backup links up to jim. jim's
|
||||||
|
`wan-pon-in` (`sfp-sfpplus2`) is a spare SFP+ port, unused today.
|
||||||
|
|
||||||
```
|
```
|
||||||
Virgin Media cable modem
|
Virgin Media cable modem Digiweb ONT
|
||||||
│
|
stream WAN, VLAN 130 river WAN, management + VLAN 10
|
||||||
│ (VLAN 130)
|
| |
|
||||||
│ wan1/wan2/wan-in
|
jim brian
|
||||||
┌──────────────────────────────────────────────────┴──┐
|
| 10G trunk 802.3ad LAG |
|
||||||
│ jim CRS326-24G-2S+ (Marvell 98DX3236) │
|
+--------------------+ +---------------+
|
||||||
│ 1G edge ports: castle, fort, pronter, laptop-dock, │
|
| |
|
||||||
│ palace-kvm, ups, ether15-20, wan1/wan2/wan-in │
|
+---+----------+---+
|
||||||
│ wan-pon-in (= sfp-sfpplus2) ← spare SFP+ │
|
| dave |
|
||||||
└───────┬───────────────────────────────────┬─────────┘
|
+--------+---------+
|
||||||
dave-uplink ┤ (= sfp-sfpplus1) │ stream, palace (1G secondaries)
|
|
|
||||||
10G trunk│ └── stream (box, dual-homed)
|
+---------------------+---------------------+
|
||||||
┌─────────┴───────────────────────────────────────────────┐
|
| | |
|
||||||
│ dave CRS504-4XQ (Marvell 98DX4310) │
|
palace (100G) castle (100G) stream
|
||||||
│ jim-downlink = qsfp28-3-1 │
|
river VM NVMe-oF root second router
|
||||||
└──┬───────────┬──────────┬───────────────────┬────────────┘
|
|
||||||
palace(100G) │ castle │ stream │ brian-downlink
|
Backup links to jim (normally idle):
|
||||||
= qsfp28-1-1 │ │ │ (802.3ad LAG: brian1+brian2)
|
* palace: 1G
|
||||||
┌────────────────┴─┐ … │
|
* stream: 1G; STP selects the active link
|
||||||
│ palace host │ Digiweb (PPPoE) via ONT
|
* castle: 2.5G, normally down; no live failover
|
||||||
│ └ river (VM) │ │
|
|
||||||
└──────────────────┘ ┌─────┴─────┐
|
|
||||||
↑ river's WAN + LAN ride the 100G link │ ONT │ untagged mgmt 192.168.100.1
|
|
||||||
└─────┬─────┘ + tagged VLAN 10 (ISP)
|
|
||||||
│
|
|
||||||
┌────────┴────────┐
|
|
||||||
│ brian (UniFi) │ PVID 140 on the ONT port,
|
|
||||||
│ 10GBASE-T │ tagged VLAN 10 allowed through
|
|
||||||
└────────┬────────┘
|
|
||||||
└── brian-downlink LAG up to dave
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Notes:
|
Notes:
|
||||||
- **river** runs as a VM on the **palace** host; its uplink is dave's 100G `palace` port. jim also
|
- **river** runs as a VM on the **palace** host; its uplink is dave's 100G `palace` port. jim also
|
||||||
has 1G `palace`/`stream` ports, but those are secondary links and do **not** carry the WAN.
|
has 1G `palace`/`stream` ports, but those are secondary links and do **not** carry the WAN.
|
||||||
- **stream** (the second router box) is dual-homed to both jim and dave (STP picks the active path).
|
- **stream** (the second router box) is dual-homed to both jim and dave (STP picks the active path).
|
||||||
- **brian** is a Ubiquiti 10GBASE-T switch, downlinked from dave over an **802.3ad LAG**
|
|
||||||
(`brian-downlink` = `brian1` + `brian2`, layer-2 hash). It hosts the ONT.
|
### Castle storage dependency
|
||||||
|
|
||||||
|
`castle` is dual-homed without STP: its primary link is dave's 100G `castle` port (`et100g`), while
|
||||||
|
the 2.5G link to jim (`et2.5g`) is normally down and provides no live failover. Its root disk is
|
||||||
|
NVMe-oF over `et100g` and dave, so interrupting either freezes `castle` mid-I/O. Do dave maintenance
|
||||||
|
from a box that does not depend on it, or power `castle` off cleanly first.
|
||||||
|
|
||||||
## VLANs
|
## VLANs
|
||||||
|
|
||||||
@@ -87,7 +86,9 @@ Notes:
|
|||||||
|
|
||||||
Switch L3 presence (`/interface vlan` on `main`) exists **only** for VLANs the switch is managed
|
Switch L3 presence (`/interface vlan` on `main`) exists **only** for VLANs the switch is managed
|
||||||
from — `hi` (100) and `lo` (110), plus native core. WAN and guest VLANs deliberately have no switch
|
from — `hi` (100) and `lo` (110), plus native core. WAN and guest VLANs deliberately have no switch
|
||||||
L3 interface.
|
L3 interface. jim and dave carry a static IPv4 and **global IPv6** address on `hi`/`lo` (plus a
|
||||||
|
static default route on each stack) purely for management — they are **pure L2, never routers**. See
|
||||||
|
[Switches must not route](#switches-must-not-route).
|
||||||
|
|
||||||
## The Digiweb WAN path (trunked VLAN 10 + PVID 140)
|
## The Digiweb WAN path (trunked VLAN 10 + PVID 140)
|
||||||
|
|
||||||
@@ -115,9 +116,9 @@ ONT.** See [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/p
|
|||||||
for the river side.
|
for the river side.
|
||||||
|
|
||||||
```
|
```
|
||||||
ONT ──(untagged + VLAN10)── brian ──(VLAN140 + VLAN10)── dave ──(VLAN140 + VLAN10)── river
|
ONT -- untagged + VLAN 10 -- brian -- VLAN 140 + VLAN 10 -- dave -- palace -- river
|
||||||
ONT port │ PVID140 + tagged 10 │ plain bridging
|
|
|
||||||
└─ brian-downlink LAG ── dave ┘
|
+-- ONT port PVID 140; VLAN 10 remains tagged
|
||||||
```
|
```
|
||||||
|
|
||||||
### Why not translation (for one ONT)?
|
### Why not translation (for one ONT)?
|
||||||
@@ -148,6 +149,33 @@ this is plain tagged bridging.
|
|||||||
10/140/141 rows. `wan-pon-in` (`sfp-sfpplus2`) sits at `pvid=1` as a spare port. jim only handles
|
10/140/141 rows. `wan-pon-in` (`sfp-sfpplus2`) sits at `pvid=1` as a spare port. jim only handles
|
||||||
stream's VLAN-130 WAN and the LAN VLANs.
|
stream's VLAN-130 WAN and the LAN VLANs.
|
||||||
|
|
||||||
|
## Switches must not route
|
||||||
|
|
||||||
|
jim and dave (and the `vibe` AP) are **pure L2** — river/stream do all routing. Their per-stack
|
||||||
|
management addresses and static default routes exist only so the boxes themselves can be reached and
|
||||||
|
reach out; they must **never** forward traffic or advertise themselves as routers. RouterOS defaults
|
||||||
|
work against this: `ip-forward` and IPv6 `forward` ship **on**, and with IPv6 forwarding on RouterOS
|
||||||
|
also emits Router Advertisements (`ra-lifetime=30m`) on every L3 interface — so a switch silently
|
||||||
|
becomes a competing IPv6 default router. This surfaced after the 7.18 → 7.23 upgrade, when clients
|
||||||
|
picked up dave/jim as default routers alongside river.
|
||||||
|
|
||||||
|
The required config on each RouterOS box:
|
||||||
|
```
|
||||||
|
/ip settings set ip-forward=no
|
||||||
|
/ipv6 settings set forward=no accept-router-advertisements=no
|
||||||
|
/ipv6 nd set [find] ra-lifetime=0
|
||||||
|
```
|
||||||
|
- `ip-forward=no` / `forward=no` — no L3 forwarding on either stack; IPv6 `forward=no` also stops RA
|
||||||
|
emission at the source.
|
||||||
|
- `accept-router-advertisements=no` — with forwarding off RouterOS would otherwise start *accepting*
|
||||||
|
RAs; this keeps the box on its deterministic **static** default route.
|
||||||
|
- `ra-lifetime=0` — belt-and-suspenders: even if forwarding is ever re-enabled the box advertises
|
||||||
|
router-lifetime 0 (i.e. "not a default router"). Setting it also emits a withdrawal RA that
|
||||||
|
actively clears the rogue default from clients (they otherwise cache it for up to ~30 min).
|
||||||
|
|
||||||
|
**After any RouterOS upgrade, re-check `/ip settings` and `/ipv6 settings`** — an upgrade can reset
|
||||||
|
these to the forwarding-on defaults. brian (UniFi) is not a RouterOS box and was not affected.
|
||||||
|
|
||||||
## Future: multiple ONTs (per-port VLAN translation)
|
## Future: multiple ONTs (per-port VLAN translation)
|
||||||
|
|
||||||
If a second ONT arrives (e.g. a Digiweb line for stream, or a second river), trunking breaks: both
|
If a second ONT arrives (e.g. a Digiweb line for stream, or a second river), trunking breaks: both
|
||||||
@@ -183,7 +211,11 @@ The switches resolve by **short hostname** on the home network — the home rout
|
|||||||
records in the home zone
|
records in the home zone
|
||||||
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix):
|
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix):
|
||||||
`jim` → hi `.10`, `dave` → hi `.11`, `brian` → core `.13`). From a box on the home network just
|
`jim` → hi `.10`, `dave` → hi `.11`, `brian` → core `.13`). From a box on the home network just
|
||||||
`ssh admin@jim` / `admin@dave`. Non-interactive pattern (password auth, avoids the ssh-agent hang):
|
`ssh admin@jim` / `admin@dave`.
|
||||||
|
|
||||||
|
**Key auth** for `admin` is installed on jim/dave (and the `vibe` AP) — `ssh -i ~/.ssh/id_rsa
|
||||||
|
admin@jim` works keyless (imported via `/user ssh-keys import`). Password `admin`/`admin` remains as
|
||||||
|
a fallback. Non-interactive password pattern (avoids the ssh-agent hang) if the key isn't available:
|
||||||
|
|
||||||
```
|
```
|
||||||
sshpass -p admin ssh -o IdentityAgent=none -o PubkeyAuthentication=no \
|
sshpass -p admin ssh -o IdentityAgent=none -o PubkeyAuthentication=no \
|
||||||
|
|||||||
Generated
+4
-130
@@ -8,7 +8,7 @@
|
|||||||
"ragenix",
|
"ragenix",
|
||||||
"nixpkgs"
|
"nixpkgs"
|
||||||
],
|
],
|
||||||
"systems": "systems_8"
|
"systems": "systems_7"
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1761656077,
|
"lastModified": 1761656077,
|
||||||
@@ -67,34 +67,6 @@
|
|||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"bun2nix": {
|
|
||||||
"inputs": {
|
|
||||||
"flake-parts": "flake-parts",
|
|
||||||
"nixpkgs": [
|
|
||||||
"pi-agent",
|
|
||||||
"nixpkgs"
|
|
||||||
],
|
|
||||||
"systems": [
|
|
||||||
"pi-agent",
|
|
||||||
"systems"
|
|
||||||
],
|
|
||||||
"treefmt-nix": "treefmt-nix_2"
|
|
||||||
},
|
|
||||||
"locked": {
|
|
||||||
"lastModified": 1778446047,
|
|
||||||
"narHash": "sha256-oQvcadh2BCkrog+SGrG6YffKJrveYpjj3TdQJWaKhaM=",
|
|
||||||
"owner": "nix-community",
|
|
||||||
"repo": "bun2nix",
|
|
||||||
"rev": "f2bc12af1a6369648aac41041ceeaa0b866599c6",
|
|
||||||
"type": "github"
|
|
||||||
},
|
|
||||||
"original": {
|
|
||||||
"owner": "nix-community",
|
|
||||||
"ref": "2.1.0",
|
|
||||||
"repo": "bun2nix",
|
|
||||||
"type": "github"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"copyparty": {
|
"copyparty": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"flake-utils": "flake-utils_5",
|
"flake-utils": "flake-utils_5",
|
||||||
@@ -284,28 +256,6 @@
|
|||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"flake-parts": {
|
|
||||||
"inputs": {
|
|
||||||
"nixpkgs-lib": [
|
|
||||||
"pi-agent",
|
|
||||||
"bun2nix",
|
|
||||||
"nixpkgs"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"locked": {
|
|
||||||
"lastModified": 1777988971,
|
|
||||||
"narHash": "sha256-qIoWPDs+0/8JecyYgE3gpKQxW/4bLW/gp45vow9ioCQ=",
|
|
||||||
"owner": "hercules-ci",
|
|
||||||
"repo": "flake-parts",
|
|
||||||
"rev": "0678d8986be1661af6bb555f3489f2fdfc31f6ff",
|
|
||||||
"type": "github"
|
|
||||||
},
|
|
||||||
"original": {
|
|
||||||
"owner": "hercules-ci",
|
|
||||||
"repo": "flake-parts",
|
|
||||||
"type": "github"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"flake-utils": {
|
"flake-utils": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"systems": "systems"
|
"systems": "systems"
|
||||||
@@ -326,7 +276,7 @@
|
|||||||
},
|
},
|
||||||
"flake-utils_10": {
|
"flake-utils_10": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"systems": "systems_10"
|
"systems": "systems_9"
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1709126324,
|
"lastModified": 1709126324,
|
||||||
@@ -344,7 +294,7 @@
|
|||||||
},
|
},
|
||||||
"flake-utils_11": {
|
"flake-utils_11": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"systems": "systems_11"
|
"systems": "systems_10"
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1705309234,
|
"lastModified": 1705309234,
|
||||||
@@ -494,7 +444,7 @@
|
|||||||
},
|
},
|
||||||
"flake-utils_9": {
|
"flake-utils_9": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"systems": "systems_9"
|
"systems": "systems_8"
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1731533236,
|
"lastModified": 1731533236,
|
||||||
@@ -636,21 +586,6 @@
|
|||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"jail-nix": {
|
|
||||||
"locked": {
|
|
||||||
"lastModified": 1776230864,
|
|
||||||
"narHash": "sha256-YsEjjdOsGEzTeD+iT7ONh071BqWAOQWpzYVei3okAXE=",
|
|
||||||
"owner": "~alexdavid",
|
|
||||||
"repo": "jail.nix",
|
|
||||||
"rev": "404e7da9da5ab9aa643666682b2ba1312fa5fbe8",
|
|
||||||
"type": "sourcehut"
|
|
||||||
},
|
|
||||||
"original": {
|
|
||||||
"owner": "~alexdavid",
|
|
||||||
"repo": "jail.nix",
|
|
||||||
"type": "sourcehut"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"libnetRepo": {
|
"libnetRepo": {
|
||||||
"flake": false,
|
"flake": false,
|
||||||
"locked": {
|
"locked": {
|
||||||
@@ -860,29 +795,6 @@
|
|||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"pi-agent": {
|
|
||||||
"inputs": {
|
|
||||||
"bun2nix": "bun2nix",
|
|
||||||
"jail-nix": "jail-nix",
|
|
||||||
"nixpkgs": [
|
|
||||||
"nixpkgs-unstable"
|
|
||||||
],
|
|
||||||
"systems": "systems_7"
|
|
||||||
},
|
|
||||||
"locked": {
|
|
||||||
"lastModified": 1784984457,
|
|
||||||
"narHash": "sha256-y7jv+RTP0TkcJaDm7/D2xIFF9JDWKc6g5L3X3EUu3Eo=",
|
|
||||||
"owner": "lukasl-dev",
|
|
||||||
"repo": "pi.nix",
|
|
||||||
"rev": "fd2c62853f66a5803dae621769570f1165e59b80",
|
|
||||||
"type": "github"
|
|
||||||
},
|
|
||||||
"original": {
|
|
||||||
"owner": "lukasl-dev",
|
|
||||||
"repo": "pi.nix",
|
|
||||||
"type": "github"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"pyproject-nix": {
|
"pyproject-nix": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"nixpkgs": [
|
"nixpkgs": [
|
||||||
@@ -948,7 +860,6 @@
|
|||||||
"nixpkgs-mine-stable": "nixpkgs-mine-stable",
|
"nixpkgs-mine-stable": "nixpkgs-mine-stable",
|
||||||
"nixpkgs-stable": "nixpkgs-stable",
|
"nixpkgs-stable": "nixpkgs-stable",
|
||||||
"nixpkgs-unstable": "nixpkgs-unstable",
|
"nixpkgs-unstable": "nixpkgs-unstable",
|
||||||
"pi-agent": "pi-agent",
|
|
||||||
"ragenix": "ragenix",
|
"ragenix": "ragenix",
|
||||||
"sharry": "sharry"
|
"sharry": "sharry"
|
||||||
}
|
}
|
||||||
@@ -1046,21 +957,6 @@
|
|||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"systems_11": {
|
|
||||||
"locked": {
|
|
||||||
"lastModified": 1681028828,
|
|
||||||
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
|
|
||||||
"owner": "nix-systems",
|
|
||||||
"repo": "default",
|
|
||||||
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
|
|
||||||
"type": "github"
|
|
||||||
},
|
|
||||||
"original": {
|
|
||||||
"owner": "nix-systems",
|
|
||||||
"repo": "default",
|
|
||||||
"type": "github"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"systems_2": {
|
"systems_2": {
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1681028828,
|
"lastModified": 1681028828,
|
||||||
@@ -1202,28 +1098,6 @@
|
|||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"treefmt-nix_2": {
|
|
||||||
"inputs": {
|
|
||||||
"nixpkgs": [
|
|
||||||
"pi-agent",
|
|
||||||
"bun2nix",
|
|
||||||
"nixpkgs"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"locked": {
|
|
||||||
"lastModified": 1775636079,
|
|
||||||
"narHash": "sha256-pc20NRoMdiar8oPQceQT47UUZMBTiMdUuWrYu2obUP0=",
|
|
||||||
"owner": "numtide",
|
|
||||||
"repo": "treefmt-nix",
|
|
||||||
"rev": "790751ff7fd3801feeaf96d7dc416a8d581265ba",
|
|
||||||
"type": "github"
|
|
||||||
},
|
|
||||||
"original": {
|
|
||||||
"owner": "numtide",
|
|
||||||
"repo": "treefmt-nix",
|
|
||||||
"type": "github"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"utils": {
|
"utils": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"systems": "systems_3"
|
"systems": "systems_3"
|
||||||
|
|||||||
@@ -49,8 +49,13 @@
|
|||||||
copyparty.inputs.nixpkgs.follows = "nixpkgs-unstable";
|
copyparty.inputs.nixpkgs.follows = "nixpkgs-unstable";
|
||||||
hass-west-wood.url = "github:devplayer0/hass-west-wood";
|
hass-west-wood.url = "github:devplayer0/hass-west-wood";
|
||||||
hass-west-wood.inputs.nixpkgs.follows = "nixpkgs-unstable";
|
hass-west-wood.inputs.nixpkgs.follows = "nixpkgs-unstable";
|
||||||
pi-agent.url = "github:lukasl-dev/pi.nix";
|
# Disabled: `pi-coding-agent-bun` breaks `nix flake check` in CI. Its bun2nix `fetchBunDeps`
|
||||||
pi-agent.inputs.nixpkgs.follows = "nixpkgs-unstable";
|
# calls `builtins.filterSource` on subpaths of the pi.nix flake source (coding-agent/bun.nix),
|
||||||
|
# which requires that source derivation to be realised in the local store. On a fresh CI runner
|
||||||
|
# it isn't, so eval aborts with `path '…-source.drv' is not valid`. Works locally only because
|
||||||
|
# the source is already realised there. Re-enable (and revisit upstream) if we start using pi.
|
||||||
|
# pi-agent.url = "github:lukasl-dev/pi.nix";
|
||||||
|
# pi-agent.inputs.nixpkgs.follows = "nixpkgs-unstable";
|
||||||
};
|
};
|
||||||
|
|
||||||
outputs =
|
outputs =
|
||||||
|
|||||||
@@ -8,7 +8,8 @@ let
|
|||||||
inherit (lib.my) mkOpt' dummyOption;
|
inherit (lib.my) mkOpt' dummyOption;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
imports = [ inputs.pi-agent.homeModules.default ];
|
# Disabled: breaks `nix flake check` in CI (see flake.nix pi-agent note).
|
||||||
|
# imports = [ inputs.pi-agent.homeModules.default ];
|
||||||
|
|
||||||
options = with lib.types; {
|
options = with lib.types; {
|
||||||
my = {
|
my = {
|
||||||
@@ -216,6 +217,7 @@ in
|
|||||||
jq
|
jq
|
||||||
yq-go
|
yq-go
|
||||||
nix-tree
|
nix-tree
|
||||||
|
treemd
|
||||||
];
|
];
|
||||||
|
|
||||||
sessionVariables = {
|
sessionVariables = {
|
||||||
@@ -232,7 +234,7 @@ in
|
|||||||
inputs.deploy-rs.overlays.default
|
inputs.deploy-rs.overlays.default
|
||||||
inputs.boardie.overlays.default
|
inputs.boardie.overlays.default
|
||||||
inputs.nixGL.overlays.default
|
inputs.nixGL.overlays.default
|
||||||
inputs.pi-agent.overlays.default
|
# inputs.pi-agent.overlays.default
|
||||||
];
|
];
|
||||||
config = {
|
config = {
|
||||||
allowUnfree = true;
|
allowUnfree = true;
|
||||||
|
|||||||
@@ -88,16 +88,17 @@ in
|
|||||||
ffmpeg-full
|
ffmpeg-full
|
||||||
xournalpp
|
xournalpp
|
||||||
|
|
||||||
(pkgs.symlinkJoin {
|
# Disabled: breaks `nix flake check` in CI (see flake.nix pi-agent note).
|
||||||
name = "pi-coding-agent";
|
# (pkgs.symlinkJoin {
|
||||||
buildInputs = [ pkgs.makeWrapper ];
|
# name = "pi-coding-agent";
|
||||||
paths = [ pkgs.pi-coding-agent-bun ];
|
# buildInputs = [ pkgs.makeWrapper ];
|
||||||
postBuild = ''
|
# paths = [ pkgs.pi-coding-agent-bun ];
|
||||||
wrapProgram $out/bin/pi \
|
# postBuild = ''
|
||||||
--set NPM_CONFIG_PREFIX ${config.home.homeDirectory}/.pi/npm/ \
|
# wrapProgram $out/bin/pi \
|
||||||
--prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.nodejs_latest ]}
|
# --set NPM_CONFIG_PREFIX ${config.home.homeDirectory}/.pi/npm/ \
|
||||||
'';
|
# --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.nodejs_latest ]}
|
||||||
})
|
# '';
|
||||||
|
# })
|
||||||
];
|
];
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -307,6 +307,8 @@ rec {
|
|||||||
|
|
||||||
home = rec {
|
home = rec {
|
||||||
domain = "h.${pubDomain}";
|
domain = "h.${pubDomain}";
|
||||||
|
# Search domains advertised to clients (radvd DNSSL / kea domain-search)
|
||||||
|
searchDomains = [ domain "dyn.${domain}" colony.domain britway.domain ];
|
||||||
vlans = {
|
vlans = {
|
||||||
hi = 100;
|
hi = 100;
|
||||||
lo = 110;
|
lo = 110;
|
||||||
@@ -383,6 +385,16 @@ rec {
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
|
# networkConfig fragment anchoring a VLAN client's DNS on the router pair's
|
||||||
|
# VRRP VIPs rather than the RA RDNSS. v6 addresses here are RA/token-derived,
|
||||||
|
# so when RA is absent (e.g. v6 disabled) there is no v6 and no RDNSS at all;
|
||||||
|
# the always-present static v4 VIP keeps name resolution working, with the v6
|
||||||
|
# VIP as a bonus when v6 is up. Merge into the VLAN network's networkConfig.
|
||||||
|
vlanDns = vlan: {
|
||||||
|
DNS = [ vips.${vlan}.v4 vips.${vlan}.v6 ];
|
||||||
|
Domains = searchDomains;
|
||||||
|
};
|
||||||
|
|
||||||
roceBootModules = [ "ib_core" "ib_uverbs" "mlx5_core" "mlx5_ib" ];
|
roceBootModules = [ "ib_core" "ib_uverbs" "mlx5_core" "mlx5_ib" ];
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
+5
-1
@@ -149,6 +149,10 @@ rec {
|
|||||||
(optional (a.ipv6.gateway != null) a.ipv6.gateway);
|
(optional (a.ipv6.gateway != null) a.ipv6.gateway);
|
||||||
networkConfig = {
|
networkConfig = {
|
||||||
IPv6AcceptRA = a.ipv6.gateway == null || a.ipv6.iid != null;
|
IPv6AcceptRA = a.ipv6.gateway == null || a.ipv6.iid != null;
|
||||||
|
# These are servers: temporary (privacy) addresses only rotate our stable source
|
||||||
|
# address out from under long-lived connected sockets (e.g. nginx's resolver, which
|
||||||
|
# wedges permanently when the address it bound to expires).
|
||||||
|
IPv6PrivacyExtensions = "no";
|
||||||
# NOTE: LLDP emission / reception is ignored on bridge interfaces
|
# NOTE: LLDP emission / reception is ignored on bridge interfaces
|
||||||
LLDP = true;
|
LLDP = true;
|
||||||
EmitLLDP = "customer-bridge";
|
EmitLLDP = "customer-bridge";
|
||||||
@@ -191,7 +195,7 @@ rec {
|
|||||||
};
|
};
|
||||||
lvmDisk' = vg: lv: lvmDisk'' lv vg lv;
|
lvmDisk' = vg: lv: lvmDisk'' lv vg lv;
|
||||||
lvmDisk = lvmDisk' "main";
|
lvmDisk = lvmDisk' "main";
|
||||||
disk = vm: lv: lvmDisk' lv "vm-${vm}-${lv}";
|
disk = vm: lv: lvmDisk'' lv "main" "vm-${vm}-${lv}";
|
||||||
};
|
};
|
||||||
|
|
||||||
deploy-rs =
|
deploy-rs =
|
||||||
|
|||||||
@@ -212,8 +212,8 @@
|
|||||||
cleanShutdown.timeout = 120;
|
cleanShutdown.timeout = 120;
|
||||||
drives = [
|
drives = [
|
||||||
(mkMerge [ (vm.disk "darts" "root") { frontendOpts.bootindex = 0; } ])
|
(mkMerge [ (vm.disk "darts" "root") { frontendOpts.bootindex = 0; } ])
|
||||||
(vm.lvmDisk' "media" "darts-media")
|
(vm.lvmDisk'' "media" "main" "darts-media")
|
||||||
(vm.lvmDisk' "ext" "darts-ext")
|
(vm.lvmDisk'' "ext" "main" "darts-ext")
|
||||||
];
|
];
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -114,8 +114,12 @@ in
|
|||||||
];
|
];
|
||||||
also-notify = [ "127.0.0.1" ];
|
also-notify = [ "127.0.0.1" ];
|
||||||
allow-axfr-ips = [
|
allow-axfr-ips = [
|
||||||
|
"127.0.0.0/8" "::1/128"
|
||||||
"216.218.133.2" "2001:470:600::2"
|
"216.218.133.2" "2001:470:600::2"
|
||||||
];
|
]
|
||||||
|
++ lib.my.c.home.routersPubV4
|
||||||
|
++ lib.my.c.as211024.trusted.v4
|
||||||
|
++ lib.my.c.as211024.trusted.v6;
|
||||||
enable-lua-records = true;
|
enable-lua-records = true;
|
||||||
#loglevel = 7;
|
#loglevel = 7;
|
||||||
#log-dns-queries = true;
|
#log-dns-queries = true;
|
||||||
|
|||||||
@@ -48,8 +48,9 @@ in
|
|||||||
"s3-secret-key.txt"
|
"s3-secret-key.txt"
|
||||||
])
|
])
|
||||||
(_: with config.services.mastodon; {
|
(_: with config.services.mastodon; {
|
||||||
owner = user;
|
# user doesn't exist any more, so this breaks on deploy
|
||||||
inherit group;
|
# owner = user;
|
||||||
|
# inherit group;
|
||||||
})) // {
|
})) // {
|
||||||
"toot/pds.env" = {
|
"toot/pds.env" = {
|
||||||
owner = "pds";
|
owner = "pds";
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
let
|
let
|
||||||
inherit (lib.my) net;
|
inherit (lib.my) net;
|
||||||
inherit (lib.my.c) networkd;
|
inherit (lib.my.c) networkd;
|
||||||
inherit (lib.my.c.home) domain vlans prefixes vips roceBootModules;
|
inherit (lib.my.c.home) domain vlans prefixes vips vlanDns roceBootModules;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
nixos.systems.castle = {
|
nixos.systems.castle = {
|
||||||
@@ -189,8 +189,12 @@ in
|
|||||||
};
|
};
|
||||||
"40-lan-hi" = mkMerge [
|
"40-lan-hi" = mkMerge [
|
||||||
(networkdAssignment "lan-hi" assignments.hi)
|
(networkdAssignment "lan-hi" assignments.hi)
|
||||||
|
{
|
||||||
|
networkConfig = vlanDns "hi" // {
|
||||||
# So we don't drop the IP we use to connect to NVMe-oF!
|
# So we don't drop the IP we use to connect to NVMe-oF!
|
||||||
{ networkConfig.KeepConfiguration = "static"; }
|
KeepConfiguration = "static";
|
||||||
|
};
|
||||||
|
}
|
||||||
];
|
];
|
||||||
"45-lan-lo" = {
|
"45-lan-lo" = {
|
||||||
matchConfig.Name = "lan-lo";
|
matchConfig.Name = "lan-lo";
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
let
|
let
|
||||||
inherit (lib.my) net mkVLAN;
|
inherit (lib.my) net mkVLAN;
|
||||||
inherit (lib.my.c) pubDomain;
|
inherit (lib.my.c) pubDomain;
|
||||||
inherit (lib.my.c.home) domain vlans prefixes vips hiMTU;
|
inherit (lib.my.c.home) domain vlans prefixes vips vlanDns hiMTU;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
imports = [ ./vms ];
|
imports = [ ./vms ];
|
||||||
@@ -203,7 +203,10 @@ in
|
|||||||
MACAddress=52:54:00:90:34:95
|
MACAddress=52:54:00:90:34:95
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
"60-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
|
"60-lan-hi" = mkMerge [
|
||||||
|
(networkdAssignment "lan-hi" assignments.hi)
|
||||||
|
{ networkConfig = vlanDns "hi"; }
|
||||||
|
];
|
||||||
|
|
||||||
"50-lan-core-phy" = {
|
"50-lan-core-phy" = {
|
||||||
matchConfig.Name = "lan-core-phy";
|
matchConfig.Name = "lan-core-phy";
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
let
|
let
|
||||||
inherit (lib.my) net;
|
inherit (lib.my) net;
|
||||||
inherit (lib.my.c) pubDomain;
|
inherit (lib.my.c) pubDomain;
|
||||||
inherit (lib.my.c.home) domain prefixes vips hiMTU;
|
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
nixos.systems.cellar = {
|
nixos.systems.cellar = {
|
||||||
@@ -79,7 +79,10 @@ in
|
|||||||
};
|
};
|
||||||
|
|
||||||
networks = {
|
networks = {
|
||||||
"80-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
|
"80-lan-hi" = mkMerge [
|
||||||
|
(networkdAssignment "lan-hi" assignments.hi)
|
||||||
|
{ networkConfig = vlanDns "hi"; }
|
||||||
|
];
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
let
|
let
|
||||||
inherit (lib.my) net;
|
inherit (lib.my) net;
|
||||||
inherit (lib.my.c) pubDomain;
|
inherit (lib.my.c) pubDomain;
|
||||||
inherit (lib.my.c.home) domain prefixes vips hiMTU;
|
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
nixos.systems.hass = { config, ... }: {
|
nixos.systems.hass = { config, ... }: {
|
||||||
@@ -82,7 +82,10 @@ in
|
|||||||
|
|
||||||
systemd = {
|
systemd = {
|
||||||
network.networks = {
|
network.networks = {
|
||||||
"80-container-host0" = networkdAssignment "host0" assignments.hi;
|
"80-container-host0" = mkMerge [
|
||||||
|
(networkdAssignment "host0" assignments.hi)
|
||||||
|
{ networkConfig = vlanDns "hi"; }
|
||||||
|
];
|
||||||
"80-container-lan-lo" = networkdAssignment "lan-lo" assignments.lo;
|
"80-container-lan-lo" = networkdAssignment "lan-lo" assignments.lo;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{ lib, ... }:
|
{ lib, ... }:
|
||||||
let
|
let
|
||||||
inherit (lib.my) net;
|
inherit (lib.my) net;
|
||||||
inherit (lib.my.c.home) domain prefixes vips hiMTU;
|
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
nixos.systems.unifi = { config, ... }: {
|
nixos.systems.unifi = { config, ... }: {
|
||||||
@@ -58,7 +58,10 @@ in
|
|||||||
|
|
||||||
systemd = {
|
systemd = {
|
||||||
network.networks = {
|
network.networks = {
|
||||||
"80-container-host0" = networkdAssignment "host0" assignments.hi;
|
"80-container-host0" = mkMerge [
|
||||||
|
(networkdAssignment "host0" assignments.hi)
|
||||||
|
{ networkConfig = vlanDns "hi"; }
|
||||||
|
];
|
||||||
"80-lan-core" = networkdAssignment "lan-core" assignments.core;
|
"80-lan-core" = networkdAssignment "lan-core" assignments.core;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{ lib, ... }:
|
{ lib, ... }:
|
||||||
let
|
let
|
||||||
inherit (lib.my) net;
|
inherit (lib.my) net;
|
||||||
inherit (lib.my.c.home) domain prefixes vips hiMTU roceBootModules;
|
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU roceBootModules;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
imports = [ ./containers ];
|
imports = [ ./containers ];
|
||||||
@@ -134,8 +134,10 @@ in
|
|||||||
networks = {
|
networks = {
|
||||||
"30-lan-hi" = mkMerge [
|
"30-lan-hi" = mkMerge [
|
||||||
(networkdAssignment "lan-hi" assignments.hi)
|
(networkdAssignment "lan-hi" assignments.hi)
|
||||||
|
{
|
||||||
# So we don't drop the IP we use to connect to NVMe-oF!
|
# So we don't drop the IP we use to connect to NVMe-oF!
|
||||||
{ networkConfig.KeepConfiguration = "static"; }
|
networkConfig = vlanDns "hi" // { KeepConfiguration = "static"; };
|
||||||
|
}
|
||||||
];
|
];
|
||||||
"30-lan-hi-ctrs" = {
|
"30-lan-hi-ctrs" = {
|
||||||
matchConfig.Name = "lan-hi-ctrs";
|
matchConfig.Name = "lan-hi-ctrs";
|
||||||
|
|||||||
@@ -170,6 +170,12 @@ in
|
|||||||
"0.0.0.0:5353" "[::]:5353"
|
"0.0.0.0:5353" "[::]:5353"
|
||||||
];
|
];
|
||||||
also-notify = [ "127.0.0.1" ];
|
also-notify = [ "127.0.0.1" ];
|
||||||
|
allow-axfr-ips = [
|
||||||
|
"127.0.0.0/8" "::1/128"
|
||||||
|
allAssignments.estuary.internal.ipv4.address
|
||||||
|
]
|
||||||
|
++ lib.my.c.as211024.trusted.v4
|
||||||
|
++ lib.my.c.as211024.trusted.v6;
|
||||||
enable-lua-records = true;
|
enable-lua-records = true;
|
||||||
# loglevel = 7;
|
# loglevel = 7;
|
||||||
# log-dns-queries = true;
|
# log-dns-queries = true;
|
||||||
@@ -242,10 +248,14 @@ in
|
|||||||
shytzel IN A ${net.cidr.host 12 prefixes.core.v4}
|
shytzel IN A ${net.cidr.host 12 prefixes.core.v4}
|
||||||
brian IN A ${net.cidr.host 13 prefixes.core.v4}
|
brian IN A ${net.cidr.host 13 prefixes.core.v4}
|
||||||
|
|
||||||
wave IN A ${net.cidr.host 12 prefixes.hi.v4}
|
vibe-core IN A ${net.cidr.host 15 prefixes.core.v4}
|
||||||
; wave IN AAAA ${net.cidr.host (65536+3) prefixes.hi.v6}
|
vibe IN A ${net.cidr.host 15 prefixes.hi.v4}
|
||||||
vibe IN A ${net.cidr.host 13 prefixes.hi.v4}
|
vibe IN AAAA ${net.cidr.host (65536+6) prefixes.hi.v6}
|
||||||
vibe IN AAAA ${net.cidr.host (65536+4) prefixes.hi.v6}
|
vibe-lo IN A ${net.cidr.host 15 prefixes.lo.v4}
|
||||||
|
vibe-lo IN AAAA ${net.cidr.host (65536+6) prefixes.lo.v6}
|
||||||
|
wave-core IN A ${net.cidr.host 14 prefixes.core.v4}
|
||||||
|
wave IN A ${net.cidr.host 14 prefixes.lo.v4}
|
||||||
|
wave IN AAAA ${net.cidr.host (65536+5) prefixes.lo.v6}
|
||||||
|
|
||||||
ups IN A ${net.cidr.host 20 prefixes.lo.v4}
|
ups IN A ${net.cidr.host 20 prefixes.lo.v4}
|
||||||
palace-kvm IN A ${net.cidr.host 21 prefixes.lo.v4}
|
palace-kvm IN A ${net.cidr.host 21 prefixes.lo.v4}
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
index: { lib, pkgs, config, assignments, allAssignments, ... }:
|
index: { lib, pkgs, config, assignments, allAssignments, ... }:
|
||||||
let
|
let
|
||||||
inherit (lib) mkForce;
|
inherit (lib) mkForce concatStringsSep;
|
||||||
inherit (lib.my) net netbootKeaClientClasses;
|
inherit (lib.my) net netbootKeaClientClasses;
|
||||||
inherit (lib.my.c.home) domain prefixes vips hiMTU;
|
inherit (lib.my.c.home) domain searchDomains prefixes vips hiMTU;
|
||||||
|
|
||||||
dns-servers = [
|
dns-servers = [
|
||||||
{
|
{
|
||||||
@@ -59,7 +59,7 @@ in
|
|||||||
}
|
}
|
||||||
{
|
{
|
||||||
name = "domain-search";
|
name = "domain-search";
|
||||||
data = "${domain}, dyn.${domain}, ${lib.my.c.colony.domain}, ${lib.my.c.britway.domain}";
|
data = concatStringsSep ", " searchDomains;
|
||||||
always-send = true;
|
always-send = true;
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
index: { lib, pkgs, ... }:
|
index: { lib, pkgs, ... }:
|
||||||
let
|
let
|
||||||
inherit (lib) mkForce concatMapStringsSep;
|
inherit (lib) mkForce concatMapStringsSep concatStringsSep;
|
||||||
inherit (lib.my) net;
|
inherit (lib.my) net;
|
||||||
inherit (lib.my.c.home) domain prefixes vips;
|
inherit (lib.my.c.home) domain searchDomains prefixes vips;
|
||||||
|
|
||||||
# untrusted uses external (Cloudflare) resolvers, matching the v4 kea config;
|
# untrusted uses external (Cloudflare) resolvers, matching the v4 kea config;
|
||||||
# trusted VLANs use the internal recursor via its floating VRRP VIP
|
# trusted VLANs use the internal recursor via its floating VRRP VIP
|
||||||
@@ -18,7 +18,7 @@ let
|
|||||||
AdvLinkMTU ${toString prefixes."${name}".mtu};
|
AdvLinkMTU ${toString prefixes."${name}".mtu};
|
||||||
prefix ${prefixes."${name}".v6} {};
|
prefix ${prefixes."${name}".v6} {};
|
||||||
RDNSS ${rdnss name} {};
|
RDNSS ${rdnss name} {};
|
||||||
DNSSL ${domain} dyn.${domain} ${lib.my.c.colony.domain} ${lib.my.c.britway.domain} {};
|
DNSSL ${concatStringsSep " " searchDomains} {};
|
||||||
};
|
};
|
||||||
'';
|
'';
|
||||||
in
|
in
|
||||||
|
|||||||
+45
-3
@@ -100,6 +100,40 @@ let
|
|||||||
] ++ defs;
|
] ++ defs;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
# JSON dump of the custom `my.*` options, rendered to docs/ by `update-docs-options`.
|
||||||
|
# Built from a minimal synthetic system (the shared modules are applied to every system,
|
||||||
|
# so any eval exposes the same option declarations) rather than a real box, so defaults
|
||||||
|
# don't pick up a specific host's values.
|
||||||
|
optionsDoc =
|
||||||
|
let
|
||||||
|
eval = mkSystem {
|
||||||
|
name = "options-doc";
|
||||||
|
config' = {
|
||||||
|
system = "x86_64-linux";
|
||||||
|
nixpkgs = "mine";
|
||||||
|
home-manager = "mine";
|
||||||
|
hmNixpkgs = "mine";
|
||||||
|
docCustom = false;
|
||||||
|
assignments = { };
|
||||||
|
};
|
||||||
|
defs = [ ];
|
||||||
|
};
|
||||||
|
pkgs = pkgs'.mine."x86_64-linux";
|
||||||
|
in
|
||||||
|
(pkgs.nixosOptionsDoc {
|
||||||
|
options = { my = eval.options.my; };
|
||||||
|
warningsAreErrors = false;
|
||||||
|
# Rewrite store-path declarations to repo-relative so the reference can link module
|
||||||
|
# files. The internal `asX` build-target options are dropped by nixosOptionsDoc itself.
|
||||||
|
transformOptions = opt: opt // {
|
||||||
|
declarations = map
|
||||||
|
(decl:
|
||||||
|
let m = builtins.match ".*-source/(.*)" (toString decl); in
|
||||||
|
if m != null then builtins.head m else decl)
|
||||||
|
opt.declarations;
|
||||||
|
};
|
||||||
|
}).optionsJSON;
|
||||||
|
|
||||||
assignmentOpts = with lib.types; { name, config, ... }: {
|
assignmentOpts = with lib.types; { name, config, ... }: {
|
||||||
options = {
|
options = {
|
||||||
name = mkOpt' str name "Name of assignment.";
|
name = mkOpt' str name "Name of assignment.";
|
||||||
@@ -154,8 +188,11 @@ let
|
|||||||
options = {
|
options = {
|
||||||
inherit (commonOpts) system nixpkgs home-manager;
|
inherit (commonOpts) system nixpkgs home-manager;
|
||||||
hmNixpkgs = commonOpts.nixpkgs;
|
hmNixpkgs = commonOpts.nixpkgs;
|
||||||
# This causes a (very slow) docs rebuild on every change to a module's options it seems
|
# Routes the custom modules into `baseModules` so the NixOS manual documents them. The old
|
||||||
# TODO: Currently broken with infinite recursion...
|
# infinite-recursion is gone, but enabling this makes every system build regenerate the
|
||||||
|
# manual, and it documents everything the modules transitively import — including third-party
|
||||||
|
# modules that aren't doc-clean (e.g. `services.sharry`). Prefer the generated
|
||||||
|
# `nixos.optionsDoc` reference (`docs/reference/nixos-options.md`) instead.
|
||||||
docCustom = mkBoolOpt' false "Whether to document nixfiles' custom NixOS modules.";
|
docCustom = mkBoolOpt' false "Whether to document nixfiles' custom NixOS modules.";
|
||||||
|
|
||||||
assignments = mkOpt' (attrsOf (submoduleWith {
|
assignments = mkOpt' (attrsOf (submoduleWith {
|
||||||
@@ -197,6 +234,11 @@ in
|
|||||||
description = "All network assignments.";
|
description = "All network assignments.";
|
||||||
readOnly = true;
|
readOnly = true;
|
||||||
};
|
};
|
||||||
|
optionsDoc = mkOption {
|
||||||
|
type = package;
|
||||||
|
description = "nixosOptionsDoc JSON dump of the custom `my.*` module options.";
|
||||||
|
readOnly = true;
|
||||||
|
};
|
||||||
vpns = {
|
vpns = {
|
||||||
l2 = mkOpt' (attrsOf (submodule l2MeshOpts)) { } "Layer 2 meshes.";
|
l2 = mkOpt' (attrsOf (submodule l2MeshOpts)) { } "Layer 2 meshes.";
|
||||||
};
|
};
|
||||||
@@ -224,7 +266,7 @@ in
|
|||||||
];
|
];
|
||||||
|
|
||||||
nixos = {
|
nixos = {
|
||||||
inherit allAssignments;
|
inherit allAssignments optionsDoc;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ let
|
|||||||
inherit (builtins) substring match;
|
inherit (builtins) substring match;
|
||||||
inherit (lib)
|
inherit (lib)
|
||||||
nameValuePair optional optionalString optionalAttrs mapAttrs' mapAttrsToList concatStringsSep
|
nameValuePair optional optionalString optionalAttrs mapAttrs' mapAttrsToList concatStringsSep
|
||||||
concatMapStringsSep mkIf;
|
concatMapStringsSep mkIf mkOption literalExpression;
|
||||||
inherit (lib.my) mkOpt' mkBoolOpt';
|
inherit (lib.my) mkOpt' mkBoolOpt';
|
||||||
|
|
||||||
jobType = with lib.types; submodule ({ name, ... }@args:
|
jobType = with lib.types; submodule ({ name, ... }@args:
|
||||||
@@ -15,12 +15,22 @@ let
|
|||||||
repo = mkOpt' str null "borg repository URL";
|
repo = mkOpt' str null "borg repository URL";
|
||||||
passFile = mkOpt' (nullOr str) null "Path to file containing passphrase";
|
passFile = mkOpt' (nullOr str) null "Path to file containing passphrase";
|
||||||
|
|
||||||
archivePrefix = mkOpt' str "${config.networking.hostName}-${name}-" "Prefix to start new archives with";
|
archivePrefix = mkOption {
|
||||||
|
type = str;
|
||||||
|
default = "${config.networking.hostName}-${name}-";
|
||||||
|
defaultText = literalExpression ''"''${config.networking.hostName}-''${name}-"'';
|
||||||
|
description = "Prefix to start new archives with";
|
||||||
|
};
|
||||||
dateFormat = mkOpt' str "+%Y-%m-%dT%H:%M:%S" "Format passed to the date command";
|
dateFormat = mkOpt' str "+%Y-%m-%dT%H:%M:%S" "Format passed to the date command";
|
||||||
compression = mkOpt' str "zstd,3" "Compression options";
|
compression = mkOpt' str "zstd,3" "Compression options";
|
||||||
lvs = mkOpt' (listOf str) null "Thin LVs to backup (vg/lv format)";
|
lvs = mkOpt' (listOf str) null "Thin LVs to backup (vg/lv format)";
|
||||||
prune = {
|
prune = {
|
||||||
pattern = mkOpt' str "sh:${cfg.archivePrefix}*" "Borg pattern to select archives for pruning";
|
pattern = mkOption {
|
||||||
|
type = str;
|
||||||
|
default = "sh:${cfg.archivePrefix}*";
|
||||||
|
defaultText = literalExpression ''"sh:''${config.archivePrefix}*"'';
|
||||||
|
description = "Borg pattern to select archives for pruning";
|
||||||
|
};
|
||||||
keep = mkOpt' (attrsOf (either int str)) { } "Borg pruning params";
|
keep = mkOpt' (attrsOf (either int str)) { } "Borg pruning params";
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -129,7 +139,12 @@ in
|
|||||||
thinToolsPackage = mkOpt' package pkgs.thin-provisioning-tools "Package containing thin-provisioning-tools";
|
thinToolsPackage = mkOpt' package pkgs.thin-provisioning-tools "Package containing thin-provisioning-tools";
|
||||||
# Really we should use the version from the overlay, but the package is quite far behind...
|
# Really we should use the version from the overlay, but the package is quite far behind...
|
||||||
# Not bothering to update until Borg 2.0 releases
|
# Not bothering to update until Borg 2.0 releases
|
||||||
package = mkOpt' package inputs.borgthin.packages.${config.nixpkgs.system}.borgthin "borgthin package";
|
package = mkOption {
|
||||||
|
type = package;
|
||||||
|
default = inputs.borgthin.packages.${config.nixpkgs.system}.borgthin;
|
||||||
|
defaultText = literalExpression "inputs.borgthin.packages.\${system}.borgthin";
|
||||||
|
description = "borgthin package";
|
||||||
|
};
|
||||||
jobs = mkOpt' (attrsOf jobType) { } "borgthin jobs";
|
jobs = mkOpt' (attrsOf jobType) { } "borgthin jobs";
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -190,6 +190,9 @@ let
|
|||||||
type = unspecified;
|
type = unspecified;
|
||||||
default = ext;
|
default = ext;
|
||||||
visible = "shallow";
|
visible = "shallow";
|
||||||
|
# Build-target plumbing: the default evaluates a whole alternate system, so keep
|
||||||
|
# it out of the generated option docs (see `optionsDoc` in flake.nix).
|
||||||
|
internal = true;
|
||||||
description = "Configuration as ${desc}.";
|
description = "Configuration as ${desc}.";
|
||||||
};
|
};
|
||||||
in
|
in
|
||||||
|
|||||||
@@ -83,7 +83,7 @@ in
|
|||||||
inputs.boardie.overlays.default
|
inputs.boardie.overlays.default
|
||||||
inputs.copyparty.overlays.default
|
inputs.copyparty.overlays.default
|
||||||
inputs.hass-west-wood.overlays.default
|
inputs.hass-west-wood.overlays.default
|
||||||
inputs.pi-agent.overlays.default
|
# inputs.pi-agent.overlays.default
|
||||||
];
|
];
|
||||||
config = {
|
config = {
|
||||||
allowUnfree = true;
|
allowUnfree = true;
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{ lib, pkgs, config, ... }:
|
{ lib, pkgs, config, ... }:
|
||||||
let
|
let
|
||||||
inherit (lib) mkMerge mkIf mkForce genAttrs concatMapStringsSep;
|
inherit (lib) mkMerge mkIf mkForce genAttrs concatMapStringsSep mkOption literalExpression;
|
||||||
inherit (lib.my) mkOpt' mkBoolOpt';
|
inherit (lib.my) mkOpt' mkBoolOpt';
|
||||||
|
|
||||||
cfg = config.my.netboot;
|
cfg = config.my.netboot;
|
||||||
@@ -47,7 +47,12 @@ in
|
|||||||
server = {
|
server = {
|
||||||
enable = mkBoolOpt' false "Whether a netboot server should be enabled.";
|
enable = mkBoolOpt' false "Whether a netboot server should be enabled.";
|
||||||
ip = mkOpt' str null "IP clients should connect to via TFTP.";
|
ip = mkOpt' str null "IP clients should connect to via TFTP.";
|
||||||
host = mkOpt' str config.networking.fqdn "Hostname clients should connect to over HTTP / NFS.";
|
host = mkOption {
|
||||||
|
type = str;
|
||||||
|
default = config.networking.fqdn;
|
||||||
|
defaultText = literalExpression "config.networking.fqdn";
|
||||||
|
description = "Hostname clients should connect to over HTTP / NFS.";
|
||||||
|
};
|
||||||
allowedPrefixes = mkOpt' (listOf str) null "Prefixes clients should be allowed to connect from (NFS).";
|
allowedPrefixes = mkOpt' (listOf str) null "Prefixes clients should be allowed to connect from (NFS).";
|
||||||
installer = {
|
installer = {
|
||||||
storeSize = mkOpt' str "16GiB" "Total allowed writable size of store.";
|
storeSize = mkOpt' str "16GiB" "Total allowed writable size of store.";
|
||||||
|
|||||||
@@ -15,4 +15,16 @@ in
|
|||||||
update-docs-assignments = pkgs.writeShellScriptBin "update-docs-assignments" ''
|
update-docs-assignments = pkgs.writeShellScriptBin "update-docs-assignments" ''
|
||||||
exec ${pkgs.python3}/bin/python3 ${../ci/update-docs-assignments.py} "$@"
|
exec ${pkgs.python3}/bin/python3 ${../ci/update-docs-assignments.py} "$@"
|
||||||
'';
|
'';
|
||||||
|
|
||||||
|
update-docs-options = pkgs.writeShellScriptBin "update-docs-options" ''
|
||||||
|
exec ${pkgs.python3}/bin/python3 ${../ci/update-docs-options.py} "$@"
|
||||||
|
'';
|
||||||
|
|
||||||
|
update-docs-dns =
|
||||||
|
let
|
||||||
|
python = pkgs.python3.withPackages (ps: [ ps.dnspython ]);
|
||||||
|
in
|
||||||
|
pkgs.writeShellScriptBin "update-docs-dns" ''
|
||||||
|
exec ${python}/bin/python3 ${../ci/update-docs-dns.py} "$@"
|
||||||
|
'';
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user