Files

T

jackos1998 87cdfdbd97 nixos/portcullis: Bring up 10G on the home hi VLAN

portcullis is wired over 10G to fergal, which uplinks to jim's spare
SFP+ port. That uplink is untagged VLAN 1, so hi is carried tagged on a
lan-hi VLAN interface: a static assignment at 192.168.68.41 / ::6:1,
resolving through the router VIPs like any other hi client. Its gateway
route outranks the DHCP default, making 10G the preferred path while the
2.5G bootstrap stays as a fallback. Deploy now targets that address.

The hi MTU goes on the .network rather than the .link, since a .link is
only applied at udev device-add -- with it there, et10g-0 stays at 1500
across a switch and lan-hi cannot take 9000.

jim's sfp-spare was tagged into hi and lo out of band to match.

fergal turns out to belong with portcullis rather than to the home
fabric -- it goes to Nikhef when the box does -- so its documentation
moves to the colony site, leaving home/switches.md a short section on
what it borrows from that fabric.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-22 01:30:50 +01:00

3.6 KiB

Raw Blame History

portcullis

A bare-metal box destined for Nikhef, intended to take over most of the colony edge routing currently done by the estuary VM.

Source: nixos/boxes/colony/portcullis/
Host: bare metal
nixpkgs: mine-stable

Hardware

Component	Inventory
Platform	Mini PC (no vendor DMI strings)
CPU	Intel N150 (4 cores / 4 threads)
Memory	8 GiB
Storage	One 128 GB NVMe SSD (`nvme0n1`), partitioned as a 2 GiB ESP plus an LVM PV holding the `nix` and `persist` volumes
Network	Four Intel I226-V 2.5 GbE ports (`et2g5-0`…`et2g5-3`) and one dual-port Intel 82599ES 10 GbE SFP+ card (`et10g-0`, `et10g-1`)
Management	JetKVM (HDMI/USB KVM with virtual media)

Role

Not yet in service. The eventual job is to be the physical edge for the colony site at Nikhef, taking over most of what estuary does today — WAN termination, firewalling and NAT, BGP for AS211024 and DNS. Some of that functionality stays on estuary, and the surrounding network topology will change with the move, so the split is not settled yet. Until it is, the config in this repository covers only what is needed to boot and reach the box.

Network assignments

portcullis has no colony assignments yet — those land alongside the routing config once the topology is decided. While it is staged at home it holds a single home hi assignment, listed in networking.md#box-assignments.

Networking

The four I226-V ports are named et2g5-0…et2g5-3 and the 82599ES SFP+ ports et10g-0 / et10g-1, pinned by permanent MAC address in .link files.
Bootstrap: a single .network matches every et2g5-* port and takes DHCP on the home lo VLAN, so whichever port happens to be patched in brings the box up. wait-online.anyInterface keeps boot from blocking on the unpatched ports.
kea registers the DHCP hostname, so while staged the box also answers to portcullis.dyn.h.nul.ie.
my.deploy.node.hostname is the hi address, taken from the assignment rather than written out, since there is no colony FQDN for the box yet.

10G to the home `hi` VLAN

et10g-0 runs over fibre to fergal, which uplinks to jim's sfp-spare port. That uplink is untagged VLAN 1, so hi is carried tagged on a lan-hi VLAN interface rather than on the port itself; the physical link takes the hi jumbo MTU so the whole path is consistent with the rest of the VLAN. lan-hi carries the static assignment, resolves through the router VIPs like every other hi client, and its gateway route outranks the DHCP default, so the 10G path is preferred while the 2.5G one stays as a fallback.

Both jim and fergal tag hi and lo along that path. It exists only while the box is staged at home — fergal goes to Nikhef with it.

The other SFP+ port, et10g-1, is unused.

Storage

A single NVMe SSD, following the usual tmpfs-root layout: a 2 GiB ESP at /boot, then one LVM PV in volume group main carrying portcullis-nix (48 GiB, /nix) and portcullis-persist (the remainder, /persist).

Secrets

my.secrets.key is the SSH host key adopted from the installer session at install time (seeded onto the persist volume before first boot), so secrets could be encrypted for the box without waiting for it to come up. The box declares nothing of its own yet — only the default user-passwd.txt that my.user brings in.

Notable config files

nixos/boxes/colony/portcullis/default.nix — hardware, filesystems and bootstrap networking.

3.6 KiB Raw Blame History