dev/nixfiles
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

nixos/portcullis: Bring up 10G on the home hi VLAN

Browse Source 
portcullis is wired over 10G to fergal, which uplinks to jim's spare
SFP+ port. That uplink is untagged VLAN 1, so hi is carried tagged on a
lan-hi VLAN interface: a static assignment at 192.168.68.41 / ::6:1,
resolving through the router VIPs like any other hi client. Its gateway
route outranks the DHCP default, making 10G the preferred path while the
2.5G bootstrap stays as a fallback. Deploy now targets that address.

The hi MTU goes on the .network rather than the .link, since a .link is
only applied at udev device-add -- with it there, et10g-0 stays at 1500
across a switch and lan-hi cannot take 9000.

jim's sfp-spare was tagged into hi and lo out of band to match.

fergal turns out to belong with portcullis rather than to the home
fabric -- it goes to Nikhef when the box does -- so its documentation
moves to the colony site, leaving home/switches.md a short section on
what it borrows from that fabric.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Jack O'Sullivan
2026-08-22 01:30:50 +01:00
parent 7bebac194c
commit 87cdfdbd97
9 changed files with 210 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/sites/colony/portcullis.md
+23 -8
View File
@@ -28,19 +28,34 @@ this repository covers only what is needed to boot and reach the box.
## Network assignments
`portcullis` has no static assignments yet. It is being staged at home before it is racked, so it
takes DHCP on the home `lo` VLAN; the colony assignments land alongside the routing config once the
topology is decided.
`portcullis` has no colony assignments yet — those land alongside the routing config once the
topology is decided. While it is staged at home it holds a single home `hi` assignment, listed in
[`networking.md#box-assignments`](../../networking.md#box-assignments).
## Networking
- The four I226-V ports are named `et2g5-0`…`et2g5-3` and the 82599ES SFP+ ports `et10g-0` /
`et10g-1`, pinned by permanent MAC address in `.link` files.
- Bootstrap only: a single `.network` matches every `et2g5-*` port and takes DHCP, so whichever
port happens to be patched in brings the box up. `wait-online.anyInterface` keeps boot from
blocking on the unpatched ports.
- kea registers the DHCP hostname, so while staged the box answers to `portcullis.dyn.h.nul.ie` —
which is also what `my.deploy.node.hostname` points at, since there is no colony FQDN for it yet.
- Bootstrap: a single `.network` matches every `et2g5-*` port and takes DHCP on the home `lo` VLAN,
so whichever port happens to be patched in brings the box up. `wait-online.anyInterface` keeps
boot from blocking on the unpatched ports.
- kea registers the DHCP hostname, so while staged the box also answers to `portcullis.dyn.h.nul.ie`.
- `my.deploy.node.hostname` is the `hi` address, taken from the assignment rather than written out,
since there is no colony FQDN for the box yet.
### 10G to the home `hi` VLAN
`et10g-0` runs over fibre to [`fergal`](fergal.md), which uplinks to jim's `sfp-spare` port. That
uplink is untagged VLAN 1, so `hi` is carried tagged on a `lan-hi` VLAN interface rather than on the
port itself; the physical link takes the `hi` jumbo MTU so the whole path is consistent with the
rest of the VLAN. `lan-hi` carries the static assignment, resolves through the router VIPs like
every other `hi` client, and its gateway route outranks the DHCP default, so the 10G path is
preferred while the 2.5G one stays as a fallback.
Both jim and `fergal` tag `hi` and `lo` along that path. It exists only while the box is staged at
home — `fergal` goes to Nikhef with it.
The other SFP+ port, `et10g-1`, is unused.
## Storage