8 Commits

Author SHA1 Message Date
jackos1998 135d52d3de nix: Skip readLinkAt unit test on XFS builder
Update docs / update (push) Successful in 1m6s
CI / Check, build and cache nixfiles (push) Successful in 1h7m38s
CI builds Determinate Nix from source, running its unit-test suite. The
`nix-util` `readLinkAt.works` test creates symlinks with PATH_MAX-length
targets, but our CI runner's build filesystem is XFS, which hard-caps
symlink targets at 1024 bytes (XFS_SYMLINK_MAXLEN). Creation fails with
ENAMETOOLONG, so the test — and the whole determinate-nix build — fails
on the runner while passing on non-XFS filesystems.

Filter out just that test via gtest's GTEST_FILTER on the
`nix-util-tests-run` check input, leaving the rest of the unit and
functional tests gating the build (they still matter, since we build
against several nixpkgs channels).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-04 00:26:37 +01:00
jackos1998 f8a89b9c87 docs: Add guided nixpkgs upgrade procedure
CI / Check, build and cache nixfiles (push) Has been cancelled
Update docs / update (push) Successful in 1m10s
Capture the periodic upgrade of the four nixpkgs channels and
home-manager as a repo doc: check for a NixOS stable bump first, rebase
the devplayer0 fork against upstream (re-verifying the patch stack
against freshly fetched refs), run the update commands, sweep
version-gated TODOs, and review the remaining flake inputs.

Keep the canonical, agent-agnostic procedure in docs/ and point both
AGENTS.md and a thin Claude Code skill at it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-03 23:31:26 +01:00
jackos1998 5e3b196ee0 nixos/git: Fix Actions runner cache timeout
Job containers using actions/cache hung and timed out reaching the
runner's built-in artifact cache server. With cache.host unset,
act_runner announced the box's autodetected outbound address, which
containers on podman0 can't route back to; even to the right address
the host input chain (policy drop) dropped the connection, as only the
forward chain was opened for the podman subnet.

Pin cache.host to the podman bridge gateway and cache.port to a fixed
value, and open that one port on podman0 in the input chain. Declare
the podman subnet once in the box file and derive the gateway, the
default_subnet and both firewall rules from it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-03 22:56:30 +01:00
jackos1998 b8f31e23f8 nixos/home: Prefer dave for stream RSTP
CI / Check, build and cache nixfiles (push) Failing after 24m39s
Update docs / update (push) Successful in 1m9s
Set explicit port costs so `stream` prefers `dave` over `jim`.

Trigger `mstpd` when the bridge is configured, before it is routable.
2026-08-03 22:31:43 +01:00
jackos1998 7fe3c8186c nix: Adopt Determinate Nix as the common Nix
Use Determinate Nix as `nix.package` for systems, homes and the
devshell, for its parallel evaluation and lazy trees. We only take the
package, not `determinate-nixd`: the daemon and `nix.conf` model are
unchanged and the Determinate NixOS module is not imported.

- Add the `determinate-nix` (`nix-src`) input, following our
  `nixpkgs-unstable`. FlakeHub's cache needs auth, so we build it
  ourselves and let it flow through Harmonia like everything else.
- `determinateOverlay` exposes it as `pkgs'.mine.determinate-nix`;
  `lib.my.c.nix.determinateSettings` (`lazy-trees`, `eval-cores = 0`)
  is merged into `nix.settings` and the devshell `nix.conf`.
- Switch CI to `DeterminateSystems/determinate-nix-action` so the
  runner itself evaluates with Determinate.
- Advertise the Harmonia cache via the flake's `nixConfig`, trusted
  without a prompt via `accept-flake-config` in the devshell, `.envrc`
  and CI only (boxes already trust it through `nix.settings`).
- Re-attach `pkgs`/`lib` to container and installer
  `nixosConfigurations` so Determinate's flake schemas can evaluate
  them (`nix flake check` otherwise fails with `attribute 'pkgs'
  missing`).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-03 22:31:43 +01:00
jackos1998 1ca978baf4 home: Raise stateVersion and drop dead nix.package guards
Every managed home already had its stateVersion force-set per
home-manager branch (22.11 for stable/mine-stable, 23.05 otherwise);
pin them all to 23.05.

With the floor at 23.05 the `versionAtLeast config.home.stateVersion`
guards on `nix.package` are always taken, so drop them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-03 22:31:43 +01:00
jackos1998 b9bcb1eaeb nixos/home: Anchor static hi clients' DNS on VIPs
Update docs / update (push) Successful in 1m10s
CI / Check, build and cache nixfiles (push) Has been cancelled
Statically-addressed home servers on hi run no DHCP, so they learned a
resolver only from the v6 RA RDNSS and lost DNS whenever v6 (and thus
the RA) was absent. Factor the fix castle/palace applied inline into a
shared lib.my.c.home.vlanDns helper that points resolved at the VLAN's
VRRP VIPs (always-present static v4, plus v6 when up) and sets the
advertised search domains, then apply it to every statically-addressed
hi client: castle, palace, cellar, sfh and the sfh hass/unifi
containers. Document it under the router client DNS section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-03 22:27:47 +01:00
jackos1998 f9f61e19ad docs/home: Document switches/AP must not route
CI / Check, build and cache nixfiles (push) Successful in 47m37s
Update docs / update (push) Successful in 1m7s
jim, dave and the vibe AP are pure L2, but RouterOS ships ip-forward and
IPv6 forward on, and with IPv6 forwarding enabled it also advertises
itself as a default router. After the 7.18 -> 7.23 upgrade clients began
picking up the switches as IPv6 default routers alongside river.

Replace the earlier advertise-dns framing (which only strips RA options,
not the router lifetime) with the actual requirement: ip-forward=no,
IPv6 forward=no, accept-router-advertisements=no, ra-lifetime=0, and a
re-check after every RouterOS upgrade.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-03 14:15:40 +01:00
29 changed files with 487 additions and 47 deletions
+25
View File
@@ -0,0 +1,25 @@
---
name: upgrade-nixpkgs
description: >-
Upgrade all four nixpkgs channels (unstable, stable, mine, mine-stable) and home-manager for this
flake: check for a NixOS stable bump, rebase the devplayer0 nixpkgs fork against upstream, run the
update commands, sweep version-gated TODOs, and review flake inputs. Use when the user wants to
update/bump nixpkgs, refresh the pins, or do the periodic nixpkgs/home-manager upgrade.
---
# Upgrade nixpkgs
The canonical, agent-agnostic procedure lives in the repo at
[`docs/nixpkgs-upgrade.md`](../../../docs/nixpkgs-upgrade.md). Read it and follow the phases in
order.
Key reminders (see the doc for the full steps):
- It is **guided, not automated** — do the mechanical/investigative work but stop at the ⏸ points:
pushing the fork, resolving rebase conflicts, editing the `flake.nix` stable pins, and deleting
version guards. Report and let the user decide.
- **Check the current NixOS stable first** (Phase 1) — the fork's `devplayer0-stable` rebase target
and the `flake.nix` stable pins must agree on one release.
- **Re-verify the patch stack against freshly fetched upstream**, not stale refs — enumerate it with
`git log`, don't assume a remembered list (stale `upstream/*` refs make already-upstreamed commits
masquerade as fork-only patches).
+3 -1
View File
@@ -1,2 +1,4 @@
watch_file devshell/{default,commands,install,vm-tasks}.nix
use flake
# --accept-flake-config trusts the flake's nixConfig (our Harmonia cache) non-interactively, so
# direnv doesn't stall on the trust prompt.
use flake . --accept-flake-config
+10 -5
View File
@@ -10,17 +10,22 @@ jobs:
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v6
- uses: cachix/install-nix-action@v31
- uses: DeterminateSystems/determinate-nix-action@v3
with:
# Gitea will supply a token in GITHUB_TOKEN, which this action will
# try to pass to Nix when downloading from GitHub
github_access_token: ${{ secrets.GH_PULL_TOKEN }}
extra_nix_config: |
# Gitea will supply a token in GITHUB_TOKEN, which this action passes to
# Nix (as access-tokens) when downloading from GitHub
github-token: ${{ secrets.GH_PULL_TOKEN }}
extra-conf: |
# Make sure we're using sandbox
sandbox-fallback = false
# Big C++ projects fill up memory...
cores = 6
# Determinate performance features
lazy-trees = true
eval-cores = 0
accept-flake-config = true
extra-substituters = https://nix-cache.nul.ie
extra-trusted-public-keys = nix-cache.nul.ie-1:BzH5yMfF4HbzY1C977XzOxoPhEc9Zbu39ftPkUbH+m4=
+3 -1
View File
@@ -61,7 +61,9 @@ Common ones:
`SSH_AUTH_SOCK= ssh-machine …` (or add `-o IdentityAgent=none` to a raw `ssh`).
- `ragenix` — edit age secrets using `.keys/dev.key` as identity (see Secrets).
- `repl``nix repl .#`.
- `update-nixpkgs` / `update-home-manager` — bump pinned inputs.
- `update-nixpkgs` / `update-home-manager` — bump pinned inputs. For the full periodic upgrade
(rebasing the `devplayer0` nixpkgs fork, stable-release bumps, version-gate sweep, input review)
follow the guided procedure in [`docs/nixpkgs-upgrade.md`](docs/nixpkgs-upgrade.md).
Use the narrowest relevant evaluation while iterating: `check-system <host>` for a box config,
`nix eval .#nixfiles.config.nixos.allAssignments --json` for assignment generation, or
+4 -1
View File
@@ -12,6 +12,9 @@ in
NIX_USER_CONF_FILES = toString (pkgs.writeText "nix.conf"
''
experimental-features = nix-command flakes ca-derivations
lazy-trees = true
eval-cores = 0
accept-flake-config = true
connect-timeout = 5
fallback = true
${lib.my.c.nix.cache.conf}
@@ -22,7 +25,7 @@ in
packages = with pkgs; [
coreutils
nixVersions.stable
determinate-nix
rage
wireguard-tools
(pkgs.writeShellScriptBin "deploy" ''
+1 -1
View File
@@ -29,7 +29,7 @@ let
coreutils
gnugrep
openssh
nixVersions.stable
determinate-nix
jq
];
text =
+2
View File
@@ -25,6 +25,8 @@ Not every box fits this pattern, but **colony** and **home** are organised this
- [`networking.md`](networking.md) — network assignments, domains, site topologies, router HA,
the AS211024 L2 mesh, BGP, WireGuard, Tailscale.
- [`deployment.md`](deployment.md) — deploy-rs, devshell commands, secrets workflow, CI.
- [`nixpkgs-upgrade.md`](nixpkgs-upgrade.md) — guided procedure for the periodic upgrade of the four
nixpkgs channels and home-manager (fork rebase, stable bumps, input review).
- [`reference/dns.md`](reference/dns.md) — generated forward and reverse DNS record reference.
- [`reference/nixos-options.md`](reference/nixos-options.md) — generated per-option reference for
the custom `my.*` NixOS modules.
+29 -2
View File
@@ -110,6 +110,31 @@ VM's unix sockets from `/run/vms/<vm>/` on `<host>` over SSH):
| `vm-monitor <host> <vm>` | QEMU monitor socket in `minicom`. |
| `vm-viewer <host> <vm>` | SPICE display in `virt-viewer` (not on Darwin). |
## Nix implementation
Every context uses **Determinate Nix** as its `nix.package`, for its performance features
(parallel evaluation and lazy trees) — not `determinate-nixd`; the daemon and `nix.conf` model
are unchanged, and the Determinate NixOS module is deliberately not imported.
- **Input and package.** The [`determinate-nix`](../flake.nix) input is the `nix-src` flake
(`flakehub.com/f/DeterminateSystems/nix-src`), with `nixpkgs.follows = "nixpkgs-unstable"`. We
build it ourselves against our pinned nixpkgs — FlakeHub's own cache needs authentication, so
there is nothing to gain from leaving it unpinned — and it then flows through the Harmonia cache
like everything else. `determinateOverlay` exposes it under the stable attr `determinate-nix`,
added to both the devshell `pkgs'` and the config `configPkgs'` overlay lists, so systems, homes
and the devshell all resolve the same package (`pkgs'.mine.determinate-nix`).
- **Settings.** `lib.my.c.nix.determinateSettings` (`lazy-trees`, `eval-cores = 0`) is merged into
`nix.settings` for systems and homes and into the devshell's `nix.conf`. These keys are only
understood by the Determinate binary.
- **Consumers follow automatically.** Everything that shells out to Nix references
`config.nix.package` (deploy-rs, containers, `build`, netboot, Harmonia), so they inherit
Determinate without further change.
- **`accept-flake-config`.** Set true only in the devshell `nix.conf`, `.envrc` (as
`--accept-flake-config`, for direnv) and CI — the contexts that build this flake — so its
`nixConfig` (the Harmonia cache) is trusted without an interactive prompt. It is deliberately not
set system-wide: boxes already trust that cache via `nix.settings`, so a global setting would only
blanket-trust every flake's `nixConfig` for no gain.
## Secrets
Secrets are age-encrypted files in [`secrets/`](../secrets), managed with **ragenix** (a fork
@@ -152,8 +177,10 @@ GitHub/Gitea Actions workflows live in [`.gitea/workflows/`](../.gitea/workflows
### `ci.yaml`
On pushes to `master`, this runs `nix flake check --no-build`, then builds every attribute of
`.#ci.x86_64-linux`: systems as `system-<name>`, homes as `home-<name>` (with `@` changed to
On pushes to `master`, this installs Determinate Nix on the runner (via
`DeterminateSystems/determinate-nix-action`, configured with the same performance settings and
Harmonia substituter as the boxes), runs `nix flake check --no-build`, then builds every attribute
of `.#ci.x86_64-linux`: systems as `system-<name>`, homes as `home-<name>` (with `@` changed to
`-at-`), packages as `package-<name>`, and the development `shell`. Each result is pushed to the
Harmonia cache with [`ci/push-to-cache.sh`](../ci/push-to-cache.sh).
+6
View File
@@ -321,6 +321,12 @@ family move together.
serving disjoint pool halves. `radvd` advertises the v6 VIP as RDNSS (`untrusted` gets Cloudflare)
and keepalived's `notify_master`/`notify_backup` hooks ensure that only the master sends RAs.
Statically-addressed boxes (the servers on `hi`) don't run DHCP, so they'd otherwise learn a
resolver only from the v6 RA RDNSS — which vanishes when v6 is disabled, taking DNS with it. They
instead anchor DNS on the VIPs via the shared `lib.my.c.home.vlanDns "<vlan>"` fragment, which sets
`DNS` to `vips.<vlan>.{v4,v6}` and `Domains` to the advertised search list; the always-present
static v4 VIP keeps resolution working even with v6 down.
#### DNS binding
`pdns-recursor` binds the VIPs directly; see
+125
View File
@@ -0,0 +1,125 @@
# Upgrading nixpkgs
Procedure for the periodic upgrade of all four nixpkgs channels (`unstable`, `stable`, `mine`,
`mine-stable`) and home-manager. Written to be followed by a person or any coding agent; a
Claude Code entry point exists at `.claude/skills/upgrade-nixpkgs/` but the steps below are the
canonical source.
The upgrade is **guided, not automated**: do the mechanical and investigative steps, but stop at
the judgment points (marked ⏸) — pushing the fork, resolving rebase conflicts, editing the
`flake.nix` stable pins, and deleting version guards. Report findings and let the maintainer
decide. Keep a running summary and present it before any push or commit.
Work the phases in order; skip one only if explicitly scoped to a subset.
## Setup facts
- **Fork checkout:** `~/documents/projects/nixpkgs` — remotes `origin` (`devplayer0/nixpkgs`) and
`upstream` (`NixOS/nixpkgs`). Confirm the path exists; if not, ask.
- **Fork branches:** `devplayer0` (tracks `nixos-unstable`) and `devplayer0-stable` (tracks the
current NixOS stable). Each is a small stack of local patches rebased onto upstream.
- **Flake pins** in `flake.nix` (`inputs`):
- `nixpkgs-unstable.url = "nixpkgs/nixos-unstable"`
- `nixpkgs-stable.url = "nixpkgs/nixos-<STABLE>"` (e.g. `nixos-26.05`)
- `nixpkgs-mine.url = "github:devplayer0/nixpkgs/devplayer0"`
- `nixpkgs-mine-stable.url = "github:devplayer0/nixpkgs/devplayer0-stable"`
- `home-manager-unstable.url = "home-manager"`
- `home-manager-stable.url = "home-manager/release-<STABLE>"`
- **Devshell commands:** `update-nixpkgs` = `nix flake update nixpkgs-{unstable,stable,mine,mine-stable}`;
`update-home-manager` = `nix flake update home-manager-{unstable,stable}`.
- **Validation:** prefer `check-system <host>` and `nix flake check --no-build` over full builds.
## Phase 1 — Determine the current NixOS stable
Do this first: everything downstream (the fork's `devplayer0-stable` rebase target, the `flake.nix`
stable pins) has to agree on one NixOS stable release, so establish it up front.
1. Find the latest NixOS stable release branch — check `git branch -r` on `upstream` for the newest
`release-YY.NN`, or the NixOS release schedule.
2. Compare it to `<STABLE>` in the `flake.nix` `nixpkgs-stable` / `home-manager-stable` URLs.
3. **If they already match** (no new stable): note "stable is current" and carry `<STABLE>` into the
later phases.
4.**If a newer stable has cut:** stop and report the coordinated change set before proceeding —
the pieces must all move to the same release together:
- Rebase `devplayer0-stable` onto the new `upstream/release-YY.NN` (Phase 2 uses this target).
- Edit `flake.nix`: `nixpkgs-stable.url` and `home-manager-stable.url` → the new release.
- Bump each system's `stateVersion` / `home.stateVersion` only if the maintainer explicitly
wants to — that is a separate, deliberate decision; never auto-bump.
Don't edit `flake.nix` here without confirmation.
## Phase 2 — Rebase the nixpkgs fork
For **both** branches — `devplayer0` onto `upstream/nixos-unstable`, and `devplayer0-stable` onto
`upstream/release-<STABLE>` (the release established in Phase 1):
1. In `~/documents/projects/nixpkgs`, confirm a clean working tree (`git status`). If dirty, stop
and report — don't stash silently.
2. `git fetch upstream --prune` and `git fetch origin --prune`. If the checkout has been idle a
long time this fetch can be large and slow; let it finish.
3. Enumerate the patch stack before rebasing:
`git log --oneline upstream/nixos-unstable..origin/devplayer0` (and the stable equivalent
against `upstream/release-<STABLE>`). For each commit, check whether it has landed upstream or
been superseded — e.g. `git log --oneline upstream/nixos-unstable -- <path>` or grep the
upstream tree for the package/option. Note any patch that now looks redundant.
4. Rebase: `git switch devplayer0 && git rebase upstream/nixos-unstable` (and the stable branch
onto `upstream/release-<STABLE>`).
-**Conflicts:** stop. Report which patch conflicts and against what upstream change; let the
maintainer resolve, or drop the patch if it has been upstreamed.
- Clean rebase: continue.
5. Summarize: which patches still apply, which are now redundant (candidate to drop), which
conflicted.
6.**Push:** only after confirmation. `git push --force-with-lease origin devplayer0
devplayer0-stable` (force needed — rebase rewrites history).
## Phase 3 — Update the pinned inputs
Run together (they move as a set):
```
update-nixpkgs
update-home-manager
```
Then show the `flake.lock` diff for the nixpkgs/home-manager entries so the old→new revisions are
visible.
## Phase 4 — Sweep version-gated behavior
The repo carries branch-conditional logic and TODOs keyed to specific nixpkgs versions; some become
removable after an upgrade, especially after a stable bump. Surface them:
```
grep -rn "versionAtLeast\|versionOlder\|when 2[0-9]\.[0-9][0-9]\|TODO.*2[0-9]\.[0-9][0-9]" \
--include=*.nix nixos home-manager lib pkgs flake.nix
```
Known example: `nixos/modules/common.nix` carries a `# TODO: Remove if-else when 26.11 releases`
guard. For each hit, evaluate whether the now-current versions make the guard removable and list
candidates. ⏸ Don't delete guards without confirmation — some protect the still-supported stable.
## Phase 5 — Review remaining flake inputs
Don't blanket-update. Walk the other inputs deliberately:
1. List inputs and locked revisions from `flake.lock` (or `nix flake metadata`).
2. For each meaningful input (`libnetRepo`, `devshell`, `determinate-nix`, `ragenix`, `deploy-rs`,
`impermanence`, and the packaged apps like `boardie`, `harmonia`, `copyparty`, `sharry`, …),
compare the locked revision to upstream and summarize notable changes (breaking changes,
relevant fixes). Many inputs `follows` `nixpkgs-unstable` and already moved in Phase 3.
3. Propose a per-input update list with reasons; update the approved ones with targeted
`nix flake update <input>`, not a global update.
## Phase 6 — Validate
1. `nix flake check --no-build` (broad eval; reproduces CI's cheap checks).
2. `check-system <host>` on a representative box, and one exercising the stable channel if the
boxes mix channels.
3. Report eval/build results honestly. On failure, surface the error and stop rather than papering
over it.
## Wrap-up
Present a final summary: fork rebase outcome (patches kept/dropped/conflicted), whether a stable
bump is pending or was applied, the lock diff, version-gate cleanup candidates, inputs updated, and
validation results. Leave committing to the maintainer unless asked; if committing, follow the
repo's `area/scope: Capitalized summary` convention.
+6
View File
@@ -77,6 +77,12 @@ Management uses host `.15`: `192.168.64.15` on native/core as a backup,
route through its VIP; `untrusted` has no address. With `l2mtu 9214`, `vibe` can use the jumbo
`hi` network unlike `wave`.
Like the switches, `vibe` is **pure L2 and must not route**`ip-forward=no`, IPv6 `forward=no`,
`accept-router-advertisements=no`, `ra-lifetime=0`. RouterOS ships these forwarding-on and then
advertises itself as an IPv6 router; re-check after any upgrade. See
[switches.md#switches-must-not-route](switches.md#switches-must-not-route) for the rationale and
commands.
## wave (Cudy AX3000, OpenWrt)
Single-port AP, so the port is a VLAN **trunk** carrying management + both SSIDs.
+3 -2
View File
@@ -76,8 +76,9 @@ box sets:
## Switching (RSTP)
`stream` is dual-homed to both switches: `lan-jim` (igc) and `lan-dave` (mlx4_en), both MTU 9000,
are enslaved to the `lan` bridge with `STP=true`. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
runs a patched `mstpd` and forces RSTP on `lan` once it's routable, so exactly one uplink carries
are enslaved to the `lan` bridge with `STP=true`. The explicit bridge-port costs prefer
`lan-dave` at 10 over `lan-jim` at 100. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
runs a patched `mstpd` and forces RSTP on `lan` once it is configured, so exactly one uplink carries
traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfigured.)
## Deployment
+30 -1
View File
@@ -86,7 +86,9 @@ from a box that does not depend on it, or power `castle` off cleanly first.
Switch L3 presence (`/interface vlan` on `main`) exists **only** for VLANs the switch is managed
from — `hi` (100) and `lo` (110), plus native core. WAN and guest VLANs deliberately have no switch
L3 interface.
L3 interface. jim and dave carry a static IPv4 and **global IPv6** address on `hi`/`lo` (plus a
static default route on each stack) purely for management — they are **pure L2, never routers**. See
[Switches must not route](#switches-must-not-route).
## The Digiweb WAN path (trunked VLAN 10 + PVID 140)
@@ -147,6 +149,33 @@ this is plain tagged bridging.
10/140/141 rows. `wan-pon-in` (`sfp-sfpplus2`) sits at `pvid=1` as a spare port. jim only handles
stream's VLAN-130 WAN and the LAN VLANs.
## Switches must not route
jim and dave (and the `vibe` AP) are **pure L2** — river/stream do all routing. Their per-stack
management addresses and static default routes exist only so the boxes themselves can be reached and
reach out; they must **never** forward traffic or advertise themselves as routers. RouterOS defaults
work against this: `ip-forward` and IPv6 `forward` ship **on**, and with IPv6 forwarding on RouterOS
also emits Router Advertisements (`ra-lifetime=30m`) on every L3 interface — so a switch silently
becomes a competing IPv6 default router. This surfaced after the 7.18 → 7.23 upgrade, when clients
picked up dave/jim as default routers alongside river.
The required config on each RouterOS box:
```
/ip settings set ip-forward=no
/ipv6 settings set forward=no accept-router-advertisements=no
/ipv6 nd set [find] ra-lifetime=0
```
- `ip-forward=no` / `forward=no` — no L3 forwarding on either stack; IPv6 `forward=no` also stops RA
emission at the source.
- `accept-router-advertisements=no` — with forwarding off RouterOS would otherwise start *accepting*
RAs; this keeps the box on its deterministic **static** default route.
- `ra-lifetime=0` — belt-and-suspenders: even if forwarding is ever re-enabled the box advertises
router-lifetime 0 (i.e. "not a default router"). Setting it also emits a withdrawal RA that
actively clears the rogue default from clients (they otherwise cache it for up to ~30 min).
**After any RouterOS upgrade, re-check `/ip settings` and `/ipv6 settings`** — an upgrade can reset
these to the forwarding-on defaults. brian (UniFi) is not a RouterOS box and was not affected.
## Future: multiple ONTs (per-port VLAN translation)
If a second ONT arrives (e.g. a Digiweb line for stream, or a second river), trunking breaks: both
Generated
+116
View File
@@ -163,6 +163,29 @@
"type": "github"
}
},
"determinate-nix": {
"inputs": {
"flake-parts": "flake-parts",
"git-hooks-nix": "git-hooks-nix",
"nixpkgs": [
"nixpkgs-unstable"
],
"nixpkgs-23-11": "nixpkgs-23-11",
"nixpkgs-regression": "nixpkgs-regression"
},
"locked": {
"lastModified": 1785428605,
"narHash": "sha256-wfaiSRLM1wDb4MV+NEzbyheK9Y03/oe56NR2I84UF7E=",
"rev": "0ff46631f69584c9f76792cae595ea253bd482c3",
"revCount": 26288,
"type": "tarball",
"url": "https://api.flakehub.com/f/pinned/DeterminateSystems/nix-src/3.21.9/019fb409-4d6e-7243-8a88-23ceee2520e9/source.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://flakehub.com/f/DeterminateSystems/nix-src/%2A"
}
},
"devshell": {
"inputs": {
"flake-utils": "flake-utils",
@@ -256,6 +279,42 @@
"type": "github"
}
},
"flake-compat_2": {
"flake": false,
"locked": {
"lastModified": 1696426674,
"narHash": "sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "0f9255e01c2351cc7d116c072cb317785dd33b33",
"type": "github"
},
"original": {
"owner": "edolstra",
"repo": "flake-compat",
"type": "github"
}
},
"flake-parts": {
"inputs": {
"nixpkgs-lib": [
"determinate-nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1748821116,
"narHash": "sha256-F82+gS044J1APL0n4hH50GYdPRv/5JWm34oCJYmVKdE=",
"rev": "49f0870db23e8c1ca0b5259734a02cd9e1e371a1",
"revCount": 377,
"type": "tarball",
"url": "https://api.flakehub.com/f/pinned/hercules-ci/flake-parts/0.1.377%2Brev-49f0870db23e8c1ca0b5259734a02cd9e1e371a1/01972f28-554a-73f8-91f4-d488cc502f08/source.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://flakehub.com/f/hercules-ci/flake-parts/0.1"
}
},
"flake-utils": {
"inputs": {
"systems": "systems"
@@ -460,6 +519,30 @@
"type": "github"
}
},
"git-hooks-nix": {
"inputs": {
"flake-compat": "flake-compat_2",
"gitignore": [
"determinate-nix"
],
"nixpkgs": [
"determinate-nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1747372754,
"narHash": "sha256-2Y53NGIX2vxfie1rOW0Qb86vjRZ7ngizoo+bnXU9D9k=",
"rev": "80479b6ec16fefd9c1db3ea13aeb038c60530f46",
"revCount": 1026,
"type": "tarball",
"url": "https://api.flakehub.com/f/pinned/cachix/git-hooks.nix/0.1.1026%2Brev-80479b6ec16fefd9c1db3ea13aeb038c60530f46/0196d79a-1b35-7b8e-a021-c894fb62163d/source.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://flakehub.com/f/cachix/git-hooks.nix/0.1.941"
}
},
"harmonia": {
"inputs": {
"crane": "crane",
@@ -655,6 +738,22 @@
"type": "github"
}
},
"nixpkgs-23-11": {
"locked": {
"lastModified": 1717159533,
"narHash": "sha256-oamiKNfr2MS6yH64rUn99mIZjc45nGJlj9eGth/3Xuw=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a62e6edd6d5e1fa0329b8653c801147986f8d446",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a62e6edd6d5e1fa0329b8653c801147986f8d446",
"type": "github"
}
},
"nixpkgs-mine": {
"locked": {
"lastModified": 1781356656,
@@ -687,6 +786,22 @@
"type": "github"
}
},
"nixpkgs-regression": {
"locked": {
"lastModified": 1643052045,
"narHash": "sha256-uGJ0VXIhWKGXxkeNnq4TvV3CIOkUJ3PAoLZ3HMzNVMw=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
}
},
"nixpkgs-stable": {
"locked": {
"lastModified": 1780902259,
@@ -847,6 +962,7 @@
"borgthin": "borgthin",
"copyparty": "copyparty",
"deploy-rs": "deploy-rs",
"determinate-nix": "determinate-nix",
"devshell": "devshell_3",
"flake-utils": "flake-utils_6",
"harmonia": "harmonia",
+44 -1
View File
@@ -1,6 +1,19 @@
{
description = "System configs";
# Offer our Harmonia cache when building the flake itself, so `nix develop` / `nix build` don't
# rebuild from source. Nix reads `nixConfig` before the flake evaluates and rejects any computed
# value (imports/thunks), so these must stay literal — keep them in sync with `lib.my.c.nix.cache`.
# Consumers must trust these (accept-flake-config / a trusted user) for them to take effect.
nixConfig = {
extra-substituters = [
"https://nix-cache.nul.ie"
];
extra-trusted-public-keys = [
"nix-cache.nul.ie-1:BzH5yMfF4HbzY1C977XzOxoPhEc9Zbu39ftPkUbH+m4="
];
};
inputs = {
flake-utils.url = "github:numtide/flake-utils";
# libnet.url = "github:reo101/nix-lib-net";
@@ -21,6 +34,12 @@
home-manager-stable.url = "home-manager/release-26.05";
home-manager-stable.inputs.nixpkgs.follows = "nixpkgs-stable";
# Determinate Nix, used as the common Nix implementation across systems, homes, the devshell and
# CI (see lib.my.c.nix). We build it ourselves against our pinned nixpkgs (FlakeHub's cache needs
# auth), so it flows through our own Harmonia cache like everything else.
determinate-nix.url = "https://flakehub.com/f/DeterminateSystems/nix-src/*";
determinate-nix.inputs.nixpkgs.follows = "nixpkgs-unstable";
# Stuff used by the flake for build / deployment
# ragenix.url = "github:yaxitech/ragenix";
ragenix.url = "github:devplayer0/ragenix/add-rekey-one-flag";
@@ -83,6 +102,22 @@
};
pkgsLibOverlay = final: prev: { lib = prev.lib.extend libOverlay; };
myPkgsOverlay = final: prev: import ./pkgs { lib = final.lib; pkgs = prev; };
# Exposes Determinate Nix under a stable attr name so systems, homes and the devshell all
# resolve the exact same package (referenced as `pkgs'.mine.determinate-nix` in configs).
# `nix-util`'s `readLinkAt.works` unit test creates PATH_MAX-length symlinks, which our CI
# runner's XFS-backed build filesystem rejects (XFS hard-caps symlink targets at 1024 bytes).
# Skip just that test via gtest's GTEST_FILTER so the rest of the suite still gates the build.
determinateOverlay = final: prev: {
determinate-nix =
(inputs.determinate-nix.packages.${prev.stdenv.hostPlatform.system}.default).overrideAttrs (o: {
checkInputs = map
(drv:
if (drv.name or "") == "nix-util-tests-run"
then drv.overrideAttrs (_: { GTEST_FILTER = "-readLinkAt.*"; })
else drv)
o.checkInputs;
});
};
# Override the flake-level lib since we're going to use it for non-config specific stuff
pkgsFlakes = mapAttrs (_: pkgsFlake: pkgsFlake // { lib = pkgsFlake.lib.extend libOverlay; }) {
@@ -111,6 +146,7 @@
pkgsLibOverlay
myPkgsOverlay
determinateOverlay
inputs.devshell.overlays.default
inputs.ragenix.overlays.default
inputs.deploy-rs.overlays.default
@@ -126,6 +162,7 @@
pkgsLibOverlay
myPkgsOverlay
determinateOverlay
];
config = {
@@ -187,7 +224,13 @@
nixosModules = nixfiles.config.nixos.modules;
homeModules = nixfiles.config.home-manager.modules;
nixosConfigurations = mapAttrs (_: s: s.rendered) nixfiles.config.nixos.systems;
# Containers and the installer override `rendered` with a bare `extendModules` config
# (`my.asContainer` / `my.asISO`) that lacks the `pkgs`/`lib` attrs `eval-config` exposes on a
# normal system. Determinate Nix's flake schemas read `machine.pkgs.stdenv.system` for every
# `nixosConfigurations` entry, so re-attach them from the full system eval (`configuration`).
nixosConfigurations = mapAttrs
(_: s: s.rendered // { inherit (s.configuration) pkgs lib; })
nixfiles.config.nixos.systems;
homeConfigurations = mapAttrs (_: s: s.configuration) nixfiles.config.home-manager.homes;
deploy = nixfiles.config.deploy-rs.rendered;
+4 -5
View File
@@ -2,7 +2,7 @@
let
inherit (builtins) listToAttrs mapAttrs readFile;
inherit (lib)
optionalString nameValuePair concatMapStrings concatStringsSep optionalAttrs versionAtLeast
optionalString nameValuePair concatMapStrings concatStringsSep optionalAttrs
mapAttrsToList mkMerge mkIf mkDefault mkOption;
inherit (lib.hm) dag;
inherit (lib.my) mkOpt' dummyOption;
@@ -50,8 +50,7 @@ in
};
nix = {
package = mkIf (!(versionAtLeast config.home.stateVersion "22.11")) pkgs.nix;
settings = with lib.my.c.nix; {
settings = with lib.my.c.nix; determinateSettings // {
experimental-features = [ "nix-command" "flakes" "ca-derivations" ];
max-jobs = mkDefault "auto";
@@ -257,13 +256,13 @@ in
ssh.authKeys.files = [ lib.my.c.sshKeyFiles.me ];
};
nix.package = mkIf (versionAtLeast config.home.stateVersion "22.05") pkgs.nix;
nix.package = pkgs'.mine.determinate-nix;
fonts.fontconfig.enable = true;
home = {
packages = with pkgs; [
pkgs'.mine.nix
pkgs'.mine.determinate-nix
];
# Without this, we are at the mercy of whatever version of nix is in $PATH...
+17
View File
@@ -111,6 +111,13 @@ rec {
extra-trusted-public-keys = ${concatStringsSep " " keys}
'';
};
# Determinate-specific settings enabling its performance features. Only understood by the
# Determinate Nix binary, so they must not be emitted for a base-Nix package.
determinateSettings = {
lazy-trees = true;
eval-cores = 0;
};
};
pubDomain = "nul.ie";
@@ -385,6 +392,16 @@ rec {
};
};
# networkConfig fragment anchoring a VLAN client's DNS on the router pair's
# VRRP VIPs rather than the RA RDNSS. v6 addresses here are RA/token-derived,
# so when RA is absent (e.g. v6 disabled) there is no v6 and no RDNSS at all;
# the always-present static v4 VIP keeps name resolution working, with the v6
# VIP as a bonus when v6 is up. Merge into the VLAN network's networkConfig.
vlanDns = vlan: {
DNS = [ vips.${vlan}.v4 vips.${vlan}.v6 ];
Domains = searchDomains;
};
roceBootModules = [ "ib_core" "ib_uverbs" "mlx5_core" "mlx5_ib" ];
};
+1 -1
View File
@@ -107,7 +107,7 @@ rec {
then throw "\nFailed assertions:\n${concatStringsSep "\n" (map (x: "- ${x}") failedAssertions)}"
else showWarnings config.warnings res;
homeStateVersion' = hmBranch: (if (hmBranch == "stable" || hmBranch == "mine-stable") then "22.11" else "23.05");
homeStateVersion' = hmBranch: "23.05";
homeStateVersion = hmBranch: {
# The flake passes a default setting, but we don't care about that
home.stateVersion = mkForce (homeStateVersion' hmBranch);
+4 -2
View File
@@ -38,6 +38,8 @@ in
let
inherit (lib) mkMerge;
inherit (lib.my) networkdAssignment;
podmanSubnet = "10.88.0.0/16";
in
{
imports = [
@@ -158,7 +160,7 @@ in
oci-containers = {
backend = "podman";
};
containers.containersConf.settings.network.default_subnet = "10.88.0.0/16";
containers.containersConf.settings.network.default_subnet = podmanSubnet;
};
systemd.network = {
@@ -195,7 +197,7 @@ in
extraRules = ''
table inet filter {
chain forward {
ip saddr 10.88.0.0/16 accept
ip saddr ${podmanSubnet} accept
}
}
'';
@@ -1,7 +1,14 @@
{ lib, pkgs, config, ... }:
let
inherit (lib) mkForce;
inherit (lib.my) net;
inherit (lib.my.c) pubDomain;
# The podman bridge gateway (first host of the default subnet); job
# containers reach the runner's artifact cache server here, through a single
# fixed port opened in the firewall below.
podmanGateway = net.cidr.host 1 config.virtualisation.containers.containersConf.settings.network.default_subnet;
cachePort = 34567;
in
{
config = {
@@ -34,6 +41,11 @@ in
cache = {
enabled = true;
dir = "/var/cache/gitea-runner";
# Announce the podman bridge gateway rather than let act_runner
# autodetect the box's outbound address, which containers can't
# route back to.
host = podmanGateway;
port = cachePort;
};
};
};
@@ -73,6 +85,15 @@ in
group = "gitea-runner";
};
};
# Let job containers reach the runner's artifact cache server on the host.
firewall.extraRules = ''
table inet filter {
chain input {
iifname "podman0" tcp dport ${toString cachePort} accept
}
}
'';
};
};
}
+2 -6
View File
@@ -2,7 +2,7 @@
let
inherit (lib.my) net;
inherit (lib.my.c) networkd;
inherit (lib.my.c.home) domain searchDomains vlans prefixes vips roceBootModules;
inherit (lib.my.c.home) domain vlans prefixes vips vlanDns roceBootModules;
in
{
nixos.systems.castle = {
@@ -190,11 +190,7 @@ in
"40-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
{
networkConfig = {
# v6 is RA/SLAAC-derived, so when RA is absent we have no v6 at all;
# anchor DNS on the always-present static v4 (the VRRP VIP follows the master)
DNS = [ vips.hi.v4 vips.hi.v6 ];
Domains = searchDomains;
networkConfig = vlanDns "hi" // {
# So we don't drop the IP we use to connect to NVMe-oF!
KeepConfiguration = "static";
};
+5 -2
View File
@@ -2,7 +2,7 @@
let
inherit (lib.my) net mkVLAN;
inherit (lib.my.c) pubDomain;
inherit (lib.my.c.home) domain vlans prefixes vips hiMTU;
inherit (lib.my.c.home) domain vlans prefixes vips vlanDns hiMTU;
in
{
imports = [ ./vms ];
@@ -203,7 +203,10 @@ in
MACAddress=52:54:00:90:34:95
'';
};
"60-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
"60-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
{ networkConfig = vlanDns "hi"; }
];
"50-lan-core-phy" = {
matchConfig.Name = "lan-core-phy";
@@ -2,7 +2,7 @@
let
inherit (lib.my) net;
inherit (lib.my.c) pubDomain;
inherit (lib.my.c.home) domain prefixes vips hiMTU;
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
in
{
nixos.systems.cellar = {
@@ -79,7 +79,10 @@ in
};
networks = {
"80-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
"80-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
{ networkConfig = vlanDns "hi"; }
];
};
};
@@ -2,7 +2,7 @@
let
inherit (lib.my) net;
inherit (lib.my.c) pubDomain;
inherit (lib.my.c.home) domain prefixes vips hiMTU;
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
in
{
nixos.systems.hass = { config, ... }: {
@@ -82,7 +82,10 @@ in
systemd = {
network.networks = {
"80-container-host0" = networkdAssignment "host0" assignments.hi;
"80-container-host0" = mkMerge [
(networkdAssignment "host0" assignments.hi)
{ networkConfig = vlanDns "hi"; }
];
"80-container-lan-lo" = networkdAssignment "lan-lo" assignments.lo;
};
};
@@ -1,7 +1,7 @@
{ lib, ... }:
let
inherit (lib.my) net;
inherit (lib.my.c.home) domain prefixes vips hiMTU;
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU;
in
{
nixos.systems.unifi = { config, ... }: {
@@ -58,7 +58,10 @@ in
systemd = {
network.networks = {
"80-container-host0" = networkdAssignment "host0" assignments.hi;
"80-container-host0" = mkMerge [
(networkdAssignment "host0" assignments.hi)
{ networkConfig = vlanDns "hi"; }
];
"80-lan-core" = networkdAssignment "lan-core" assignments.core;
};
};
+5 -3
View File
@@ -1,7 +1,7 @@
{ lib, ... }:
let
inherit (lib.my) net;
inherit (lib.my.c.home) domain prefixes vips hiMTU roceBootModules;
inherit (lib.my.c.home) domain prefixes vips vlanDns hiMTU roceBootModules;
in
{
imports = [ ./containers ];
@@ -134,8 +134,10 @@ in
networks = {
"30-lan-hi" = mkMerge [
(networkdAssignment "lan-hi" assignments.hi)
# So we don't drop the IP we use to connect to NVMe-oF!
{ networkConfig.KeepConfiguration = "static"; }
{
# So we don't drop the IP we use to connect to NVMe-oF!
networkConfig = vlanDns "hi" // { KeepConfiguration = "static"; };
}
];
"30-lan-hi-ctrs" = {
matchConfig.Name = "lan-hi-ctrs";
+1 -1
View File
@@ -24,7 +24,7 @@ in
services = {
networkd-dispatcher.rules = {
configure-mstpd = {
onState = [ "routable" ];
onState = [ "configured" ];
script = ''
#!${pkgs.runtimeShell}
if [ "$IFACE" = "lan" ]; then
+3 -4
View File
@@ -115,10 +115,7 @@
Name = "lan";
Kind = "bridge";
};
extraConfig = ''
[Bridge]
STP=true
'';
bridgeConfig.STP = true;
};
};
links = {
@@ -175,10 +172,12 @@
"50-lan-jim" = {
matchConfig.Name = "lan-jim";
networkConfig.Bridge = "lan";
bridgeConfig.Cost = 100;
};
"50-lan-dave" = {
matchConfig.Name = "lan-dave";
networkConfig.Bridge = "lan";
bridgeConfig.Cost = 10;
};
"50-wan-ifb" = {
+2 -2
View File
@@ -50,9 +50,9 @@ in
};
nix = {
package = pkgs'.mine.nix;
package = pkgs'.mine.determinate-nix;
channel.enable = false;
settings = with lib.my.c.nix; {
settings = with lib.my.c.nix; determinateSettings // {
trusted-users = [ "@wheel" ];
experimental-features = [ "nix-command" "flakes" "ca-derivations" ];
extra-substituters = cache.substituters;