Attach `clue-ix` VLAN 121 directly to `wan` with the assigned IPv6 address. Add sessions to both AS218511 route servers using `rs_client` roles and mandatory GTSM, following the existing IXP policy. Include the interface in the direct protocol and IXP firewall set, and document the VLAN and peers. Omit IPv4 peering because no address was assigned.
8.2 KiB
estuary
The colony edge router and firewall — the box that holds colony's public IPs and connects everything else at the site to the internet.
- Source:
nixos/boxes/colony/vms/estuary/(default.nix,bgp.nix,dns.nix,bandwidth.nix) - Host: VM on
colony(gets the WAN NIC by PCI passthrough) - nixpkgs:
mine
Role
- Edge routing / firewall / NAT — owns the colony public IPv4/IPv6 assignments, NATs outbound traffic, and port-forwards inbound services (see Firewall and NAT).
- DNS — PowerDNS authoritative server and recursor (see DNS).
- BGP — BIRD2 speaking AS211024 with upstreams, IXP route servers and direct peers (see BGP).
- VPNs — the
as211024L2 VXLAN mesh plus three point-to-point WireGuard tunnels (see VPNs). - Misc —
iperf3server, netdata.
Network assignments
See the consolidated network assignments table (this box: estuary).
WAN and IXP VLANs
wan— the passed-throughigbNIC (9000 MTU, enlarged rings). It carries the plain upstream uplink (static v4/v6 with gateways from theinternalassignment) plus the taggedifogandclue-ixVLANs.ifog(VLAN 409) is an iFog QinQ transport that carries the IXP VLANs as nested tags.clue-ixsits directly onwan:
| Interface | Parent | VLAN | IPv4 | IPv6 | Purpose |
|---|---|---|---|---|---|
clue-ix |
wan |
121 | — | 2001:7f8:154::21:1024:1/64 |
Clue-IX peering LAN |
frys-ix |
ifog |
701 | 185.1.160.196/23 |
2001:7f8:10f::3:3850:196/64 |
Frys-IX peering LAN |
nl-ix |
ifog |
1845 | 193.239.116.145/22 |
2001:7f8:13::a521:1024:1/64 |
NL-ix peering LAN |
fogixp |
ifog |
1147 | 185.1.147.159/24 |
2001:7f8:ca:1::159/64 |
FogIXP peering LAN |
ifog-transit |
ifog |
702 | — | 2a0c:9a40:100f:370::2/64 |
iFog IPv6 transit |
The IXP interfaces run at 1500 MTU with DHCP/RA/LLDP off; an nftables ixp
chain rejects non-IP/ARP ethertypes in both directions.
base— colony base network; sends RAs and serves DNS to the site, and routes thevms/ctrs/oci, Tailscale,qclk,vip*and customer prefixes back viacolony.as211024— the L2 mesh interface (see VPNs).
Firewall and NAT
my.firewall (nftables). Inbound port forwards (my.firewall.nat.forwardPorts,
driven by the shared lib.my.c.colony.firewallForwards list):
| Service | Forwarded to |
|---|---|
| HTTP/S, Matrix federation | middleman |
| Git | git |
| Game servers | OCI servers on whale2, gam |
| Tailscale | waffletail |
qclk WireGuard |
qclk |
Besides the forwards, extraRules defines:
routing-tcp/routing-udpchains — the inbound allow-list for new connections fromwan/as211024/IXPs towards internal services (SSH anywhere, otherwise per-service v4/v6 rules mirroringfirewallForwards).filter-routing— applied towan/as211024/IXPs →baseforwards; customer prefixes (mail/dartsv4,cust.v6) are accepted wholesale, the rest goes through therouting-*chains.- SNAT: everything from
prefixes.all.v4leaving non-as211024interfaces is NATed to the public IP; the WireGuard tunnel prefixes get their own SNAT addresses. - DNS redirect: DNS traffic arriving at estuary's own public addresses is redirected to port 5353 (the authoritative server) — see below.
DNS
Both halves are PowerDNS (dns.nix). The live
forward and reverse records are listed in the generated DNS reference.
Authoritative
my.pdns.auth, listening on 0.0.0.0:5353 / [::]:5353. Primary for
ams1.int.nul.ie, 100.10.in-addr.arpa and the 2a0e:97c0:4d2::/48 reverse
zone.
- Zone contents are largely generated from
allAssignments(lib.my.dns.fwdRecords/ptrRecords/ptr6Records);ALIASrecords (withexpand-alias) point the zone apex at estuary itself. - AXFR is allowed to HE.net's secondary and the trusted internal/site-egress sources used by the generated DNS reference.
_acme-challengeis a LUATXTrecord answered from a file (DNS-01 issuance).- Reached publicly via the NAT redirect of port 53 → 5353; the
baseside also accepts DNS directly.
Recursor
my.pdns.recursor (pdns-recursor), listening on localhost and the base
addresses, serving prefixes.all and the Tailscale prefixes.
- Authoritative zones are forwarded back to
127.0.0.1:5353(with NOTIFY support, so changes show up immediately). - A small Lua
preresolvehook rewritesnix-cache.nul.ie→http.ams1.int.nul.ieso cache traffic stays on-site.
BGP
BIRD2 (bgp.nix) speaking AS211024:
| Peer | ASN | Role | Where / notes |
|---|---|---|---|
| ColoClue | AS8283 | Upstream | euNetworks 2/3, v4+v6 |
| iFog | AS34927 | Upstream | IPv6 transit |
| Hurricane Electric | AS6939 | Upstream | IPv6 over Frys-IX |
| Clue-IX | AS218511 | IXP route server | rs1/rs2, IPv6 only; rs_client role and GTSM |
| Frys-IX | AS56393 | IXP route server | |
| NL-ix | AS34307 | IXP route server | lower preference |
| FogIXP | AS47498 | IXP route server | |
| LUJE.net | AS212855 | Direct peer | ColoClue/Frys-IX/FogIXP + multihop labs |
| jurrian | AS212635 | Direct peer | |
| Meta | AS32934 | Direct peer | Frys-IX/NL-ix |
| Cloudflare | AS13335 | Direct peer | Frys-IX |
| Apple | AS714 | Direct peer | NL-ix |
| HE | AS6939 | Direct peer | Frys-IX (v4) |
| bgp.tools | AS212232 | Monitoring | multihop collector, add paths |
Statics originate the site prefixes: the vip* ranges via base, the
ams1/internal v6 prefixes via base/as211024, and the home /48 towards the
home routers' as211024 VIP.
Currently disabled (commented out): efero transit over FogIXP ("not working so well lately") and the NL-ix Cloudflare sessions.
Clue-IX uses IPv6-only RIPE policy entries under the existing IXP remarks:
remarks: \/\/ Clue-IX route servers \/\/
mp-import: afi ipv6.unicast from AS218511 accept AS218511:AS-CLUE-IX
mp-export: afi ipv6.unicast to AS218511 announce AS211024:AS-DEVPLAYER0
One pair covers both route servers. See RIPE routing policy and peering records for the shared attribute convention and update procedure.
VPNs
as211024 L2 mesh
Member alongside river/stream/britway (my.vpns.l2, the l2mesh module).
The mesh transport, crypto and addressing are shared fabric — see
The AS211024 L2 mesh in networking.md.
WireGuard tunnels
Point-to-point tunnels terminated here as networkd wireguard netdevs (private
keys from agenix); each SNATs out its own interface address:
| Tunnel | Port | Prefix | Notes |
|---|---|---|---|
kelder |
51820 | — | to the remote kelder site; kelder's public estuary assignment is routed over the tunnel |
hillcrest |
51822 | prefixes.hillcrest.v4 (10.100.5.0/30) |
/32 pair, estuary .1 ↔ remote .2 |
john-valorant |
51823 | prefixes.john-valorant.v4 (10.100.5.4/30) |
/32 pair, estuary .1 ↔ remote .2 |
Bandwidth management
bandwidth.nix implements a WAN shaper: a
token-bucket filter on wan (outbound) and on an ifb-wan IFB device that ingress traffic is
mirrored into (inbound), with
bandwidth.py as a bandwidth-limiter
service that watches utilisation and can adjust the configured rate. Currently disabled — the
file is not in estuary's imports (only dns.nix and bgp.nix are), so no shaping is applied.
Notable config files
nixos/boxes/colony/vms/estuary/default.nix— system, networkd, firewall, WireGuard, mesh membership.nixos/boxes/colony/vms/estuary/bgp.nix— BIRD2 config.nixos/boxes/colony/vms/estuary/dns.nix— PowerDNS auth + recursor.nixos/boxes/colony/vms/estuary/bandwidth.nix— WAN shaper (disabled, not imported).