jackos1998
87cdfdbd97
portcullis is wired over 10G to fergal, which uplinks to jim's spare SFP+ port. That uplink is untagged VLAN 1, so hi is carried tagged on a lan-hi VLAN interface: a static assignment at 192.168.68.41 / ::6:1, resolving through the router VIPs like any other hi client. Its gateway route outranks the DHCP default, making 10G the preferred path while the 2.5G bootstrap stays as a fallback. Deploy now targets that address. The hi MTU goes on the .network rather than the .link, since a .link is only applied at udev device-add -- with it there, et10g-0 stays at 1500 across a switch and lan-hi cannot take 9000. jim's sfp-spare was tagged into hi and lo out of band to match. fergal turns out to belong with portcullis rather than to the home fabric -- it goes to Nikhef when the box does -- so its documentation moves to the colony site, leaving home/switches.md a short section on what it borrows from that fabric. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
76 lines
3.6 KiB
Markdown
76 lines
3.6 KiB
Markdown
# portcullis
|
|
|
|
A bare-metal box destined for Nikhef, intended to take over most of the colony edge
|
|
routing currently done by the [`estuary`](estuary.md) VM.
|
|
|
|
- **Source:** [`nixos/boxes/colony/portcullis/`](../../../nixos/boxes/colony/portcullis)
|
|
- **Host:** bare metal
|
|
- **nixpkgs:** `mine-stable`
|
|
|
|
## Hardware
|
|
|
|
| Component | Inventory |
|
|
|---|---|
|
|
| Platform | Mini PC (no vendor DMI strings) |
|
|
| CPU | Intel N150 (4 cores / 4 threads) |
|
|
| Memory | 8 GiB |
|
|
| Storage | One 128 GB NVMe SSD (`nvme0n1`), partitioned as a 2 GiB ESP plus an LVM PV holding the `nix` and `persist` volumes |
|
|
| Network | Four Intel I226-V 2.5 GbE ports (`et2g5-0`…`et2g5-3`) and one dual-port Intel 82599ES 10 GbE SFP+ card (`et10g-0`, `et10g-1`) |
|
|
| Management | JetKVM (HDMI/USB KVM with virtual media) |
|
|
|
|
## Role
|
|
|
|
Not yet in service. The eventual job is to be the physical edge for the colony site at Nikhef,
|
|
taking over most of what `estuary` does today — WAN termination, firewalling and NAT, BGP for
|
|
AS211024 and DNS. Some of that functionality stays on `estuary`, and the surrounding network
|
|
topology will change with the move, so the split is not settled yet. Until it is, the config in
|
|
this repository covers only what is needed to boot and reach the box.
|
|
|
|
## Network assignments
|
|
|
|
`portcullis` has no colony assignments yet — those land alongside the routing config once the
|
|
topology is decided. While it is staged at home it holds a single home `hi` assignment, listed in
|
|
[`networking.md#box-assignments`](../../networking.md#box-assignments).
|
|
|
|
## Networking
|
|
|
|
- The four I226-V ports are named `et2g5-0`…`et2g5-3` and the 82599ES SFP+ ports `et10g-0` /
|
|
`et10g-1`, pinned by permanent MAC address in `.link` files.
|
|
- Bootstrap: a single `.network` matches every `et2g5-*` port and takes DHCP on the home `lo` VLAN,
|
|
so whichever port happens to be patched in brings the box up. `wait-online.anyInterface` keeps
|
|
boot from blocking on the unpatched ports.
|
|
- kea registers the DHCP hostname, so while staged the box also answers to `portcullis.dyn.h.nul.ie`.
|
|
- `my.deploy.node.hostname` is the `hi` address, taken from the assignment rather than written out,
|
|
since there is no colony FQDN for the box yet.
|
|
|
|
### 10G to the home `hi` VLAN
|
|
|
|
`et10g-0` runs over fibre to [`fergal`](fergal.md), which uplinks to jim's `sfp-spare` port. That
|
|
uplink is untagged VLAN 1, so `hi` is carried tagged on a `lan-hi` VLAN interface rather than on the
|
|
port itself; the physical link takes the `hi` jumbo MTU so the whole path is consistent with the
|
|
rest of the VLAN. `lan-hi` carries the static assignment, resolves through the router VIPs like
|
|
every other `hi` client, and its gateway route outranks the DHCP default, so the 10G path is
|
|
preferred while the 2.5G one stays as a fallback.
|
|
|
|
Both jim and `fergal` tag `hi` and `lo` along that path. It exists only while the box is staged at
|
|
home — `fergal` goes to Nikhef with it.
|
|
|
|
The other SFP+ port, `et10g-1`, is unused.
|
|
|
|
## Storage
|
|
|
|
A single NVMe SSD, following the usual tmpfs-root layout: a 2 GiB ESP at `/boot`, then one LVM PV
|
|
in volume group `main` carrying `portcullis-nix` (48 GiB, `/nix`) and `portcullis-persist` (the
|
|
remainder, `/persist`).
|
|
|
|
## Secrets
|
|
|
|
`my.secrets.key` is the SSH host key adopted from the installer session at install time (seeded onto
|
|
the persist volume before first boot), so secrets could be encrypted for the box without waiting for
|
|
it to come up. The box declares nothing of its own yet — only the default `user-passwd.txt` that
|
|
`my.user` brings in.
|
|
|
|
## Notable config files
|
|
|
|
- [`nixos/boxes/colony/portcullis/default.nix`](../../../nixos/boxes/colony/portcullis/default.nix) — hardware, filesystems and bootstrap networking.
|