nixos/home: Separate router storage and uplinks
Give `river` a dedicated RDMA VF and two RSTP trunk attachments. Keep client services on the router VIPs and carry Digiweb VLANs over both switch paths. Set VF trust, multicast reception, bridge MAC and ARP behavior for local switching. Wait for Kea interface readiness and apply RSTP costs through `mstpd` despite port-registration and rename races. Load `palace`'s RAID module in the initrd. Record RouterOS and RouterBOOT upgrades and validate port, carrier and switch outages. Document the deployed architecture, interface handoffs and recovery procedures. Add physical socket maps, NIC capacities and cable connections alongside the logical topology and failover checks.
This commit is contained in:
+4
-4
@@ -65,10 +65,10 @@ portcullis (bare-metal edge box for Nikhef — staged, not yet in service)
|
|||||||
## Site: home
|
## Site: home
|
||||||
|
|
||||||
Redundant routers, VM host, storage, IoT containers and the workstation — see
|
Redundant routers, VM host, storage, IoT containers and the workstation — see
|
||||||
[`sites/home/README.md`](sites/home/README.md). The hand-configured switch fabric (jim/dave/brian)
|
[`sites/home/README.md`](sites/home/README.md). The hand-configured switch fabric
|
||||||
and the Digiweb WAN path are documented in [`sites/home/switches.md`](sites/home/switches.md); the
|
(`jim`/`dave`/`brian`/`shytzel`) and WAN paths are documented in
|
||||||
5G modem being evaluated as a replacement for `stream`'s WAN is in
|
[`sites/home/switches.md`](sites/home/switches.md); the 5G modem being evaluated as a replacement
|
||||||
[`sites/home/wwan.md`](sites/home/wwan.md).
|
for `stream`'s WAN is in [`sites/home/wwan.md`](sites/home/wwan.md).
|
||||||
|
|
||||||
```
|
```
|
||||||
h.nul.ie
|
h.nul.ie
|
||||||
|
|||||||
+33
-5
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
This page describes how addressing works across the boxes: the assignment mechanism, the
|
This page describes how addressing works across the boxes: the assignment mechanism, the
|
||||||
per-site domains and prefixes, the home router HA pair, and the overlays/tunnels that tie the
|
per-site domains and prefixes, the home router HA pair, and the overlays/tunnels that tie the
|
||||||
sites together. Switch-level home topology (jim/dave/brian, the ONT path) lives in
|
sites together. Switch-level home topology (`jim`/`dave`/`brian`/`shytzel`, the WAN paths) lives in
|
||||||
[sites/home/switches.md](sites/home/switches.md).
|
[sites/home/switches.md](sites/home/switches.md).
|
||||||
|
|
||||||
## Assignments
|
## Assignments
|
||||||
@@ -314,6 +314,29 @@ which the other sites use as their next-hop into the home prefixes.
|
|||||||
|
|
||||||
### Router HA
|
### Router HA
|
||||||
|
|
||||||
|
#### Switch attachments
|
||||||
|
|
||||||
|
Both routers use `lan-dave` and `lan-jim` under an RSTP bridge named `lan`. The
|
||||||
|
[failover validation](sites/home/router-failover.md) covers router, attachment and switch outages.
|
||||||
|
[`routing-common`](../nixos/boxes/home/routing-common/default.nix) owns the bridge, `mstpd` setup and
|
||||||
|
explicit port costs: `dave` is preferred at cost 10 and `jim` is redundant at cost 100. Native
|
||||||
|
`core`, the client VLANs and `river`'s Digiweb VLANs sit above this bridge. The `jim` path can reach
|
||||||
|
the ONT through the RSTP-protected connection between `jim` and `brian` without traversing `dave`.
|
||||||
|
|
||||||
|
RSTP follows the fabric topology, including failures beyond a local cable that retains carrier;
|
||||||
|
an active-backup bond using only MII monitoring would miss those failures. The guest bridge has an
|
||||||
|
explicit jumbo MTU. Root priorities and port roles across the complete fabric must agree with the
|
||||||
|
intended preference; local port costs alone do not establish a working redundant path.
|
||||||
|
On networkd configuration events, the dispatcher inspects the current bridge ports and applies
|
||||||
|
the costs through `mstpctl`, retrying until the daemon has registered each port. The daemon's
|
||||||
|
configuration hook reapplies them on restart or reload. The shared DHCPv4 service
|
||||||
|
waits for IPv4 readiness on the client interfaces and retries delayed startup without exhausting
|
||||||
|
systemd's start limit.
|
||||||
|
|
||||||
|
`river` keeps its NVMe/RDMA traffic on a separate VF, outside the bridge; see
|
||||||
|
[river](sites/home/river.md#storage). The physical fabric is documented in
|
||||||
|
[switches.md](sites/home/switches.md).
|
||||||
|
|
||||||
#### VRRP
|
#### VRRP
|
||||||
|
|
||||||
[`routing-common/keepalived.nix`](../nixos/boxes/home/routing-common/keepalived.nix) defines separate
|
[`routing-common/keepalived.nix`](../nixos/boxes/home/routing-common/keepalived.nix) defines separate
|
||||||
@@ -351,15 +374,20 @@ route on `wan`, while `river`'s `pppd` hooks start and stop it. Consumers such a
|
|||||||
`wantedBy` + `partOf` + `after`, never `requires`/`wants`, so they cannot pull the target in early
|
`wantedBy` + `partOf` + `after`, never `requires`/`wants`, so they cannot pull the target in early
|
||||||
and they reload on WAN flap.
|
and they reload on WAN flap.
|
||||||
|
|
||||||
|
DDNS starts after `network.target` rather than this WAN gate and can fail before the WAN is
|
||||||
|
ready. Its timer retries periodically; `doas systemctl start ddns-update.service` retries
|
||||||
|
immediately once connectivity is available.
|
||||||
|
|
||||||
### WAN paths (summary)
|
### WAN paths (summary)
|
||||||
|
|
||||||
#### `river`
|
#### `river`
|
||||||
|
|
||||||
The VM on `palace` runs Digiweb PPPoE directly on VLAN 10 (`vlans.pon-isp`), trunked untranslated
|
The VM on `palace` runs Digiweb PPPoE directly on VLAN 10 (`vlans.pon-isp`), trunked untranslated
|
||||||
through the switches. The carrying interface is named `wan-pon-isp`; VLAN ID 141 with that name is
|
over its RSTP `lan` bridge. `dave` is the normal carrying path; `jim` provides the redundant path.
|
||||||
reserved for a future translation scheme. MTU 1508 preserves a 1500-byte PPP session, IPCP requests
|
The VLAN interface is named `wan-pon-isp`; VLAN ID 141 with that name is reserved for a future
|
||||||
the static address, and the `pppd` hooks own `wan-online.target`. `wan-pon-ont` (VLAN 140) reaches
|
translation scheme. MTU 1508 preserves a 1500-byte PPP session, IPCP
|
||||||
the ONT management subnet at `192.168.100.0/24`, where `river` takes `.100`.
|
requests the static address, and the `pppd` hooks own `wan-online.target`. `wan-pon-ont` (VLAN 140)
|
||||||
|
reaches the ONT management subnet at `192.168.100.0/24`, where `river` takes `.100`.
|
||||||
|
|
||||||
#### `stream`
|
#### `stream`
|
||||||
|
|
||||||
|
|||||||
@@ -33,10 +33,12 @@ clients and the two WAN paths. VLAN IDs, prefixes, MTUs and router addressing li
|
|||||||
|
|
||||||
## Switch fabric
|
## Switch fabric
|
||||||
|
|
||||||
The boxes hang off three hand-configured switches — `jim` and `dave` (MikroTik, RouterOS) and
|
The boxes hang off four hand-configured switches: `jim` and `dave` (MikroTik, RouterOS), `brian`
|
||||||
`brian` (Ubiquiti, UniFi) — which are **not** managed by this flake. The physical topology, VLAN
|
(Ubiquiti, UniFi), and the L2-managed Zyxel switch `shytzel`. They are **not** managed by this
|
||||||
map, the Digiweb WAN path (trunked VLAN 10 + PVID 140 at the ONT edge), and the multi-ONT plan are
|
flake. The physical topology, VLAN map, the Digiweb WAN path (trunked VLAN 10 + PVID 140 at the ONT
|
||||||
documented in [switches.md](switches.md).
|
edge), and the multi-ONT plan are documented in [switches.md](switches.md). Router attachments
|
||||||
|
are described in [Router HA](../../networking.md#router-ha); tested failure scenarios and checks
|
||||||
|
for maintenance are documented in [router-failover.md](router-failover.md).
|
||||||
|
|
||||||
## Wireless APs
|
## Wireless APs
|
||||||
|
|
||||||
|
|||||||
@@ -13,7 +13,8 @@ Only the DNS records live in the flake
|
|||||||
| | vibe | wave |
|
| | vibe | wave |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Model | MikroTik **cAP ax** (`cAPGi-5HaxD2HaxD`) | Cudy **AX3000** (OpenWrt id `cudy,ap3000-v1`) |
|
| Model | MikroTik **cAP ax** (`cAPGi-5HaxD2HaxD`) | Cudy **AX3000** (OpenWrt id `cudy,ap3000-v1`) |
|
||||||
| OS | RouterOS 7.x | OpenWrt 25.12.x (MT7981B / Filogic 820) |
|
| OS | RouterOS 7.25.1 | OpenWrt 25.12.x (MT7981B / Filogic 820) |
|
||||||
|
| RouterBOOT | 7.25.1 | — |
|
||||||
| Radio | 2×2 both bands | 2×2 both bands (2 spatial streams) |
|
| Radio | 2×2 both bands | 2×2 both bands (2 spatial streams) |
|
||||||
| Uplink | trunk (multi-port; `ether2` is a wired LAN port) | single **2.5 GbE** trunk |
|
| Uplink | trunk (multi-port; `ether2` is a wired LAN port) | single **2.5 GbE** trunk |
|
||||||
| Management | core/hi/lo `.15` | core/lo `.14` (**not** `hi`; see below) |
|
| Management | core/hi/lo `.15` | core/lo `.14` (**not** `hi`; see below) |
|
||||||
@@ -58,6 +59,10 @@ with `ssh admin@vibe '/interface wifi export show-sensitive'` (`.passphrase=` pr
|
|||||||
RouterOS, one hardware-offloaded bridge `main` with `vlan-filtering=yes`. Access: `ssh admin@vibe`
|
RouterOS, one hardware-offloaded bridge `main` with `vlan-filtering=yes`. Access: `ssh admin@vibe`
|
||||||
— key auth for `admin` is installed (`~/.ssh/id_rsa`), with `admin`/`admin` as a fallback.
|
— key auth for `admin` is installed (`~/.ssh/id_rsa`), with `admin`/`admin` as a fallback.
|
||||||
|
|
||||||
|
`wifi-qcom` matches the RouterOS version. Follow the shared
|
||||||
|
[upgrade procedure](switches.md#upgrades) for backups and version checks. A DFS channel
|
||||||
|
availability check can delay the 5 GHz radio after reboot while 2.4 GHz is already running.
|
||||||
|
|
||||||
- **Uplink `ether1`** — trunk, tagged VLANs 100/110/120; native/untagged is the default VLAN 1
|
- **Uplink `ether1`** — trunk, tagged VLANs 100/110/120; native/untagged is the default VLAN 1
|
||||||
(PVID, no IP). `ether2` is a wired **access port** on VLAN 110 (`lo`). `l2mtu 9214`.
|
(PVID, no IP). `ether2` is a wired **access port** on VLAN 110 (`lo`). `l2mtu 9214`.
|
||||||
- **Radios** — `wifi1` (5 GHz, 20/40/80) + `wifi2` (2.4 GHz, 20/40) both broadcast `wlan0`
|
- **Radios** — `wifi1` (5 GHz, 20/40/80) + `wifi2` (2.4 GHz, 20/40) both broadcast `wlan0`
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
# Home network diagrams
|
||||||
|
|
||||||
|
`switch-ports.json` holds the socket inventory, NIC capacities and cable endpoints. Regenerate
|
||||||
|
`switch-panels.svg` and the cable table in `../switches.md` from the repository root:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 docs/sites/home/diagrams/render-switch-panels.py
|
||||||
|
```
|
||||||
|
|
||||||
|
The renderer uses only the Python standard library. Socket geometry and schematic cable routing
|
||||||
|
live in the renderer. Keep the inventory and routing together when adding a connection.
|
||||||
|
|
||||||
|
`home-fabric.mmd` is the Mermaid source for the logical fabric diagram. Render it with Mermaid
|
||||||
|
CLI (the checked-in SVG was rendered with Mermaid 11.12.0):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mmdc -i docs/sites/home/diagrams/home-fabric.mmd -o docs/sites/home/diagrams/home-fabric.svg
|
||||||
|
```
|
||||||
|
|
||||||
|
Check rendered SVGs visually after editing. Both diagrams are checked in so Markdown viewers
|
||||||
|
can show them without Mermaid support.
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
flowchart LR
|
||||||
|
virgin["Virgin Media modem"] --- shytzel["shytzel"]
|
||||||
|
ont["Digiweb ONT"] --- brian["brian"]
|
||||||
|
shytzel ---|"VLAN 130 trunk"| jim["jim"]
|
||||||
|
brian ---|"2 × 10G LAG"| dave["dave · RSTP root"]
|
||||||
|
brian -. "1G backup" .- jim
|
||||||
|
jim ---|"10G fabric trunk"| dave
|
||||||
|
dave ---|"100G"| palace["palace · river / cellar / sfh"]
|
||||||
|
jim ---|"1G host + separate 1G river trunk"| palace
|
||||||
|
dave ---|"100G"| castle["castle · NVMe-oF root"]
|
||||||
|
jim -. "1G · normally down" .- castle
|
||||||
|
dave ---|"10G LAN"| stream["stream"]
|
||||||
|
jim -. "1G LAN backup" .- stream
|
||||||
|
jim ---|"Separate VLAN 130 WAN handoff"| stream
|
||||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 24 KiB |
@@ -0,0 +1,207 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Render the rack panel map and cable table from switch-ports.json."""
|
||||||
|
import html
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import textwrap
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parent
|
||||||
|
DATA = json.loads((ROOT / "switch-ports.json").read_text())
|
||||||
|
COLORS = ["#2563eb", "#7c3aed", "#d97706", "#0891b2", "#64748b", "#16a34a", "#db2777", "#dc2626", "#92400e", "#0f766e", "#4f46e5"]
|
||||||
|
TAGS = {}
|
||||||
|
POSITIONS = {}
|
||||||
|
foreground = []
|
||||||
|
for link, color in zip(DATA["links"], COLORS):
|
||||||
|
for end in ("from", "to"):
|
||||||
|
TAGS[tuple(link[end])] = (link["id"], color)
|
||||||
|
|
||||||
|
svg = ['<svg xmlns="http://www.w3.org/2000/svg" width="1440" height="1580" viewBox="0 0 1440 1580" role="img" aria-labelledby="title desc">',
|
||||||
|
'<title id="title">Home rack switch front panels and cable endpoints</title>',
|
||||||
|
'<desc id="desc">Stream is drawn above Brian but physically sits to its left. Brian above Jim above Dave, with palace several rack units below. Socket order follows the hardware; matching letter badges identify cable ends.</desc>',
|
||||||
|
'<style>text{font-family:system-ui,sans-serif;fill:#172033}.heading{font-weight:650;font-size:22px}.sub{fill:#526075;font-size:14px}.port{font-size:12px;font-weight:600}.small{font-size:11px}.mono{font-family:monospace}</style>']
|
||||||
|
|
||||||
|
|
||||||
|
def rect(x, y, w, h, fill, stroke="none", radius=5):
|
||||||
|
svg.append(f'<rect x="{x}" y="{y}" width="{w}" height="{h}" rx="{radius}" fill="{fill}" stroke="{stroke}"/>')
|
||||||
|
|
||||||
|
|
||||||
|
def text(x, y, value, cls="", anchor="start", color=None):
|
||||||
|
style = f' style="fill:{color}"' if color else ""
|
||||||
|
svg.append(f'<text x="{x}" y="{y}" class="{cls}" text-anchor="{anchor}"{style}>{html.escape(value)}</text>')
|
||||||
|
|
||||||
|
|
||||||
|
def badge(x, y, tag, color):
|
||||||
|
rect(x - 10, y - 12, 20, 20, color, radius=4)
|
||||||
|
text(x, y + 2, tag, "port", "middle", "white")
|
||||||
|
|
||||||
|
|
||||||
|
def chassis(y, name, height, note):
|
||||||
|
rect(115, y, 1210, height, "#eef1f5", "#a8b2c1", 8)
|
||||||
|
text(140, y + 28, name, "heading")
|
||||||
|
text(1300, y + 27, note, "sub", "end")
|
||||||
|
|
||||||
|
|
||||||
|
def port(host, socket, x, y, width=46, height=27, label=True):
|
||||||
|
start = len(svg)
|
||||||
|
POSITIONS[(host, socket)] = (x, y, width, height)
|
||||||
|
p = next(p for p in DATA[host]["ports"] if p["socket"] == socket)
|
||||||
|
svg.append('<g><title>' + html.escape(f'{host} {socket}: {p["interface"]} — {p["role"]}') + '</title>')
|
||||||
|
tag, color = TAGS.get((host, socket), (None, "#64748b"))
|
||||||
|
rect(x - width / 2, y, width, height, "#fff", color, 3)
|
||||||
|
rect(x - width / 2 + 6, y + 5, width - 12, height - 10, "#28364a", radius=1)
|
||||||
|
text(x, y - 9, socket, "port mono", "middle")
|
||||||
|
if tag:
|
||||||
|
badge(x + width / 2 - 2, y - 9, tag, color)
|
||||||
|
if label:
|
||||||
|
for row, line in enumerate(textwrap.wrap(p["label"], width=8 if host == "jim" else 11)[:2]):
|
||||||
|
text(x, y + height + 15 + row * 12, line, "small", "middle")
|
||||||
|
svg.append('</g>')
|
||||||
|
foreground.extend(svg[start:])
|
||||||
|
del svg[start:]
|
||||||
|
|
||||||
|
|
||||||
|
rect(0, 0, 1440, 1580, "white", radius=0)
|
||||||
|
text(95, 38, "Home rack · physical socket map", "heading")
|
||||||
|
text(95, 64, "Front views, top to bottom. Socket positions follow the hardware; spacing and chassis widths are schematic.", "sub")
|
||||||
|
svg.append('<g transform="translate(0,330)">')
|
||||||
|
rect(330, -235, 780, 305, "#eef1f5", "#a8b2c1", 8)
|
||||||
|
text(350, -205, "stream · BROUNION R86S", "heading")
|
||||||
|
text(350, -180, "Physically left of brian; drawn above for cable routing", "sub")
|
||||||
|
text(350, -150, "DC / USB", "small")
|
||||||
|
for n, x in enumerate((600, 780, 960)):
|
||||||
|
port("stream", f"ETH{n}", x, -135, width=60)
|
||||||
|
text(x, -65, "Intel I225-V · 2.5G", "small", "middle")
|
||||||
|
text(x, -48, "1G connection" if n < 2 else "Unused", "small", "middle")
|
||||||
|
for socket, x in (("SFP+ L", 650), ("SFP+ R", 890)):
|
||||||
|
port("stream", socket, x, -12, width=70)
|
||||||
|
text(x, 53, "ConnectX-3 · 10G SFP+", "small", "middle")
|
||||||
|
# SFP captions sit below the narrow chassis to preserve the two-row socket layout.
|
||||||
|
for x in (93, 1347):
|
||||||
|
rect(x, 90, 9, 1080, "#dce2eb", radius=2)
|
||||||
|
for y in range(106, 1170, 24):
|
||||||
|
rect(x + 2, y, 5, 8, "white", radius=1)
|
||||||
|
|
||||||
|
chassis(100, "brian · USW-Pro-XG-8-PoE", 155, "8 × RJ45, then 2 × SFP+")
|
||||||
|
rect(205, 180, 10, 10, "#60a5fa", radius=5)
|
||||||
|
text(210, 218, "LED", "small", "middle")
|
||||||
|
for n in range(1, 9):
|
||||||
|
port("brian", str(n), 320 + (n - 1) * 75, 174, width=48)
|
||||||
|
for n in (9, 10):
|
||||||
|
port("brian", str(n), 1060 + (n - 9) * 120, 174, width=65)
|
||||||
|
|
||||||
|
chassis(340, "jim · CRS326-24G-2S+RM", 205, "3 banks of 8 RJ45 · even above odd · SFP+ at right")
|
||||||
|
rect(175, 425, 54, 29, "#28364a", "#94a3b8")
|
||||||
|
text(202, 410, "CONSOLE", "small", "middle")
|
||||||
|
for n in range(1, 25):
|
||||||
|
col = (n - 1) // 2
|
||||||
|
x = 285 + col * 59 + (col // 4) * 17
|
||||||
|
port("jim", str(n), x, 476 if n % 2 else 402, width=43, height=24)
|
||||||
|
port("jim", "S1", 1134, 445, width=54)
|
||||||
|
port("jim", "S2", 1235, 445, width=54)
|
||||||
|
|
||||||
|
chassis(630, "dave · CRS504-4XQ-IN", 165, "4 QSFP28 cages · console above management at right")
|
||||||
|
for x, label in ((175, "PSU 1"), (290, "PSU 2")):
|
||||||
|
rect(x, 680, 90, 78, "#d1d8e2", "#94a3b8")
|
||||||
|
text(x + 45, 725, label, "sub", "middle")
|
||||||
|
for n in range(1, 5):
|
||||||
|
x = 490 + (n - 1) * 140
|
||||||
|
if n == 3:
|
||||||
|
rect(x - 46, 695, 92, 29, "#fff", "#0891b2")
|
||||||
|
rect(x - 38, 701, 76, 17, "#28364a", radius=1)
|
||||||
|
text(x, 686, "Q3", "port mono", "middle")
|
||||||
|
text(x, 743, "4 × 10G breakout", "small", "middle")
|
||||||
|
svg.append(f'<path d="M{x} 757 V820" stroke="#0891b2" stroke-width="2" fill="none"/>')
|
||||||
|
else:
|
||||||
|
port("dave", f"Q{n}", x, 695, width=92)
|
||||||
|
rect(1095, 690, 54, 27, "#28364a", "#94a3b8")
|
||||||
|
text(1122, 681, "CONSOLE", "small", "middle")
|
||||||
|
port("dave", "MGMT", 1122, 741, width=54, height=24, label=False)
|
||||||
|
text(1235, 725, "DC / reset", "sub", "middle")
|
||||||
|
|
||||||
|
rect(430, 820, 820, 84, "#f4f8fc", "#b9cadb")
|
||||||
|
text(445, 841, "Q3 breakout lanes (one physical cage)", "sub")
|
||||||
|
for n in range(1, 5):
|
||||||
|
start = len(svg)
|
||||||
|
p = next(p for p in DATA["dave"]["ports"] if p["socket"] == f"Q3.{n}")
|
||||||
|
x = 485 + (n - 1) * 200
|
||||||
|
tag, color = TAGS.get(("dave", p["socket"]), (None, "#64748b"))
|
||||||
|
text(x, 870, p["socket"], "port mono")
|
||||||
|
if tag:
|
||||||
|
badge(x + 61, 866, tag, color)
|
||||||
|
text(x, 889, p["label"], "small")
|
||||||
|
rect(x + 83, 854, 34, 20, "white", color, radius=3)
|
||||||
|
rect(x + 89, 860, 22, 8, color, radius=1)
|
||||||
|
POSITIONS[("dave", p["socket"])] = (x + 100, 854, 34, 20)
|
||||||
|
foreground.extend(svg[start:])
|
||||||
|
del svg[start:]
|
||||||
|
|
||||||
|
text(720, 963, "⋮ several rack units ⋮", "sub", "middle")
|
||||||
|
chassis(1030, "palace", 160, "NIC endpoints only; rear-panel positions are not depicted")
|
||||||
|
for x, name in ((350, "uplink-dave"), (730, "host-jim"), (1110, "river-jim-phy")):
|
||||||
|
start = len(svg)
|
||||||
|
tag, color = TAGS[("palace", name)]
|
||||||
|
rect(x - 145, 1090, 290, 76, "white", color)
|
||||||
|
text(x - 118, 1120, name, "port mono")
|
||||||
|
badge(x + 119, 1114, tag, color)
|
||||||
|
spec = next(p for p in DATA["palace"]["ports"] if p["socket"] == name)
|
||||||
|
text(x, 1146, spec["nic"] + " · " + spec["capacity"], "small", "middle")
|
||||||
|
POSITIONS[("palace", name)] = (x, 1090, 290, 76)
|
||||||
|
foreground.extend(svg[start:])
|
||||||
|
del svg[start:]
|
||||||
|
|
||||||
|
|
||||||
|
def anchor(host, socket, side="bottom"):
|
||||||
|
x, y, w, h = POSITIONS[(host, socket)]
|
||||||
|
return {"top": (x, y), "bottom": (x, y + h),
|
||||||
|
"left": (x - w / 2, y + h / 2), "right": (x + w / 2, y + h / 2)}[side]
|
||||||
|
|
||||||
|
|
||||||
|
def cable(tag, points):
|
||||||
|
color = COLORS[ord(tag) - ord("A")]
|
||||||
|
points = " ".join(f"{x},{y}" for x, y in points)
|
||||||
|
svg.append(f'<polyline points="{points}" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>')
|
||||||
|
svg.append(f'<polyline points="{points}" fill="none" stroke="{color}" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>')
|
||||||
|
|
||||||
|
|
||||||
|
cable("A", [anchor("brian", "9"), (1060, 280), (38, 280), (38, 925),
|
||||||
|
(785, 925), anchor("dave", "Q3.2")])
|
||||||
|
cable("B", [anchor("brian", "10"), (1180, 300), (58, 300), (58, 950),
|
||||||
|
(1185, 950), anchor("dave", "Q3.4")])
|
||||||
|
cable("C", [anchor("brian", "5"), (620, 315), (492, 315), (492, 458),
|
||||||
|
(462, 458), anchor("jim", "7", "top")])
|
||||||
|
cable("D", [anchor("jim", "S1"), (1134, 575), (1394, 575), (1394, 975),
|
||||||
|
(585, 975), anchor("dave", "Q3.1")])
|
||||||
|
cable("E", [anchor("jim", "24", "right"), (1010, 414), (1010, 600),
|
||||||
|
(1380, 600), (1380, 753), anchor("dave", "MGMT", "right")])
|
||||||
|
cable("F", [anchor("jim", "5"), (403, 568), (1368, 568), (1368, 1010),
|
||||||
|
(730, 1010), anchor("palace", "host-jim", "top")])
|
||||||
|
cable("G", [anchor("jim", "22", "right"), (940, 414), (940, 590),
|
||||||
|
(1416, 590), (1416, 1025), (1110, 1025), anchor("palace", "river-jim-phy", "top")])
|
||||||
|
cable("H", [anchor("dave", "Q1"), (490, 790), (410, 790), (410, 1000),
|
||||||
|
(350, 1000), anchor("palace", "uplink-dave", "top")])
|
||||||
|
|
||||||
|
cable("I", [anchor("stream", "ETH0", "top"), (600, -160), (18, -160),
|
||||||
|
(18, 555), (898, 555), anchor("jim", "21")])
|
||||||
|
cable("J", [anchor("stream", "ETH1", "top"), (780, -170), (70, -170),
|
||||||
|
(70, 322), (403, 322), anchor("jim", "6", "top")])
|
||||||
|
cable("K", [anchor("stream", "SFP+ R"), (890, 80), (1428, 80),
|
||||||
|
(1428, 920), (985, 920), anchor("dave", "Q3.3")])
|
||||||
|
|
||||||
|
svg.extend(foreground)
|
||||||
|
|
||||||
|
text(95, 1220, "Matching letter badges identify cable ends. Port labels describe configured use; they do not indicate current link state.", "sub")
|
||||||
|
svg.append('</g></svg>')
|
||||||
|
(ROOT / "switch-panels.svg").write_text('\n'.join(svg) + '\n')
|
||||||
|
|
||||||
|
rows = ["| Cable | Endpoint | Endpoint | Connection |", "|---|---|---|---|"]
|
||||||
|
for link in DATA["links"]:
|
||||||
|
ends = [f'`{link[end][0]}` `{link[end][1]}`' for end in ("from", "to")]
|
||||||
|
rows.append(f'| **{link["id"]}** | {ends[0]} | {ends[1]} | {link["description"]} |')
|
||||||
|
page = ROOT.parent / "switches.md"
|
||||||
|
content = page.read_text()
|
||||||
|
start, end = "<!-- switch-cables -->", "<!-- /switch-cables -->"
|
||||||
|
if start in content:
|
||||||
|
content = re.sub(re.escape(start) + r'.*?' + re.escape(end),
|
||||||
|
start + '\n' + '\n'.join(rows) + '\n' + end, content, flags=re.S)
|
||||||
|
page.write_text(content)
|
||||||
@@ -0,0 +1,517 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" width="1440" height="1580" viewBox="0 0 1440 1580" role="img" aria-labelledby="title desc">
|
||||||
|
<title id="title">Home rack switch front panels and cable endpoints</title>
|
||||||
|
<desc id="desc">Stream is drawn above Brian but physically sits to its left. Brian above Jim above Dave, with palace several rack units below. Socket order follows the hardware; matching letter badges identify cable ends.</desc>
|
||||||
|
<style>text{font-family:system-ui,sans-serif;fill:#172033}.heading{font-weight:650;font-size:22px}.sub{fill:#526075;font-size:14px}.port{font-size:12px;font-weight:600}.small{font-size:11px}.mono{font-family:monospace}</style>
|
||||||
|
<rect x="0" y="0" width="1440" height="1580" rx="0" fill="white" stroke="none"/>
|
||||||
|
<text x="95" y="38" class="heading" text-anchor="start">Home rack · physical socket map</text>
|
||||||
|
<text x="95" y="64" class="sub" text-anchor="start">Front views, top to bottom. Socket positions follow the hardware; spacing and chassis widths are schematic.</text>
|
||||||
|
<g transform="translate(0,330)">
|
||||||
|
<rect x="330" y="-235" width="780" height="305" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||||
|
<text x="350" y="-205" class="heading" text-anchor="start">stream · BROUNION R86S</text>
|
||||||
|
<text x="350" y="-180" class="sub" text-anchor="start">Physically left of brian; drawn above for cable routing</text>
|
||||||
|
<text x="350" y="-150" class="small" text-anchor="start">DC / USB</text>
|
||||||
|
<text x="600" y="-65" class="small" text-anchor="middle">Intel I225-V · 2.5G</text>
|
||||||
|
<text x="600" y="-48" class="small" text-anchor="middle">1G connection</text>
|
||||||
|
<text x="780" y="-65" class="small" text-anchor="middle">Intel I225-V · 2.5G</text>
|
||||||
|
<text x="780" y="-48" class="small" text-anchor="middle">1G connection</text>
|
||||||
|
<text x="960" y="-65" class="small" text-anchor="middle">Intel I225-V · 2.5G</text>
|
||||||
|
<text x="960" y="-48" class="small" text-anchor="middle">Unused</text>
|
||||||
|
<text x="650" y="53" class="small" text-anchor="middle">ConnectX-3 · 10G SFP+</text>
|
||||||
|
<text x="890" y="53" class="small" text-anchor="middle">ConnectX-3 · 10G SFP+</text>
|
||||||
|
<rect x="93" y="90" width="9" height="1080" rx="2" fill="#dce2eb" stroke="none"/>
|
||||||
|
<rect x="95" y="106" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="130" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="154" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="178" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="202" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="226" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="250" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="274" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="298" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="322" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="346" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="370" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="394" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="418" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="442" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="466" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="490" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="514" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="538" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="562" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="586" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="610" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="634" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="658" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="682" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="706" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="730" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="754" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="778" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="802" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="826" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="850" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="874" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="898" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="922" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="946" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="970" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="994" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="1018" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="1042" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="1066" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="1090" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="1114" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="1138" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="95" y="1162" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1347" y="90" width="9" height="1080" rx="2" fill="#dce2eb" stroke="none"/>
|
||||||
|
<rect x="1349" y="106" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="130" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="154" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="178" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="202" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="226" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="250" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="274" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="298" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="322" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="346" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="370" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="394" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="418" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="442" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="466" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="490" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="514" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="538" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="562" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="586" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="610" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="634" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="658" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="682" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="706" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="730" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="754" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="778" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="802" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="826" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="850" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="874" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="898" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="922" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="946" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="970" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="994" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="1018" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="1042" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="1066" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="1090" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="1114" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="1138" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="1349" y="1162" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||||
|
<rect x="115" y="100" width="1210" height="155" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||||
|
<text x="140" y="128" class="heading" text-anchor="start">brian · USW-Pro-XG-8-PoE</text>
|
||||||
|
<text x="1300" y="127" class="sub" text-anchor="end">8 × RJ45, then 2 × SFP+</text>
|
||||||
|
<rect x="205" y="180" width="10" height="10" rx="5" fill="#60a5fa" stroke="none"/>
|
||||||
|
<text x="210" y="218" class="small" text-anchor="middle">LED</text>
|
||||||
|
<rect x="115" y="340" width="1210" height="205" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||||
|
<text x="140" y="368" class="heading" text-anchor="start">jim · CRS326-24G-2S+RM</text>
|
||||||
|
<text x="1300" y="367" class="sub" text-anchor="end">3 banks of 8 RJ45 · even above odd · SFP+ at right</text>
|
||||||
|
<rect x="175" y="425" width="54" height="29" rx="5" fill="#28364a" stroke="#94a3b8"/>
|
||||||
|
<text x="202" y="410" class="small" text-anchor="middle">CONSOLE</text>
|
||||||
|
<rect x="115" y="630" width="1210" height="165" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||||
|
<text x="140" y="658" class="heading" text-anchor="start">dave · CRS504-4XQ-IN</text>
|
||||||
|
<text x="1300" y="657" class="sub" text-anchor="end">4 QSFP28 cages · console above management at right</text>
|
||||||
|
<rect x="175" y="680" width="90" height="78" rx="5" fill="#d1d8e2" stroke="#94a3b8"/>
|
||||||
|
<text x="220" y="725" class="sub" text-anchor="middle">PSU 1</text>
|
||||||
|
<rect x="290" y="680" width="90" height="78" rx="5" fill="#d1d8e2" stroke="#94a3b8"/>
|
||||||
|
<text x="335" y="725" class="sub" text-anchor="middle">PSU 2</text>
|
||||||
|
<rect x="724" y="695" width="92" height="29" rx="5" fill="#fff" stroke="#0891b2"/>
|
||||||
|
<rect x="732" y="701" width="76" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="770" y="686" class="port mono" text-anchor="middle">Q3</text>
|
||||||
|
<text x="770" y="743" class="small" text-anchor="middle">4 × 10G breakout</text>
|
||||||
|
<path d="M770 757 V820" stroke="#0891b2" stroke-width="2" fill="none"/>
|
||||||
|
<rect x="1095" y="690" width="54" height="27" rx="5" fill="#28364a" stroke="#94a3b8"/>
|
||||||
|
<text x="1122" y="681" class="small" text-anchor="middle">CONSOLE</text>
|
||||||
|
<text x="1235" y="725" class="sub" text-anchor="middle">DC / reset</text>
|
||||||
|
<rect x="430" y="820" width="820" height="84" rx="5" fill="#f4f8fc" stroke="#b9cadb"/>
|
||||||
|
<text x="445" y="841" class="sub" text-anchor="start">Q3 breakout lanes (one physical cage)</text>
|
||||||
|
<text x="720" y="963" class="sub" text-anchor="middle">⋮ several rack units ⋮</text>
|
||||||
|
<rect x="115" y="1030" width="1210" height="160" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||||
|
<text x="140" y="1058" class="heading" text-anchor="start">palace</text>
|
||||||
|
<text x="1300" y="1057" class="sub" text-anchor="end">NIC endpoints only; rear-panel positions are not depicted</text>
|
||||||
|
<polyline points="1060,201 1060,280 38,280 38,925 785,925 785,874" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="1060,201 1060,280 38,280 38,925 785,925 785,874" fill="none" stroke="#2563eb" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="1180,201 1180,300 58,300 58,950 1185,950 1185,874" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="1180,201 1180,300 58,300 58,950 1185,950 1185,874" fill="none" stroke="#7c3aed" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="620,201 620,315 492,315 492,458 462,458 462,476" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="620,201 620,315 492,315 492,458 462,458 462,476" fill="none" stroke="#d97706" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="1134,472 1134,575 1394,575 1394,975 585,975 585,874" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="1134,472 1134,575 1394,575 1394,975 585,975 585,874" fill="none" stroke="#0891b2" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="989.5,414.0 1010,414 1010,600 1380,600 1380,753 1149.0,753.0" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="989.5,414.0 1010,414 1010,600 1380,600 1380,753 1149.0,753.0" fill="none" stroke="#64748b" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="403,500 403,568 1368,568 1368,1010 730,1010 730,1090" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="403,500 403,568 1368,568 1368,1010 730,1010 730,1090" fill="none" stroke="#16a34a" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="930.5,414.0 940,414 940,590 1416,590 1416,1025 1110,1025 1110,1090" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="930.5,414.0 940,414 940,590 1416,590 1416,1025 1110,1025 1110,1090" fill="none" stroke="#db2777" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="490,722 490,790 410,790 410,1000 350,1000 350,1090" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="490,722 490,790 410,790 410,1000 350,1000 350,1090" fill="none" stroke="#dc2626" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="600,-135 600,-160 18,-160 18,555 898,555 909,500" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="600,-135 600,-160 18,-160 18,555 898,555 909,500" fill="none" stroke="#92400e" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="780,-135 780,-170 70,-170 70,322 403,322 403,402" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="780,-135 780,-170 70,-170 70,322 403,322 403,402" fill="none" stroke="#0f766e" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="890,15 890,80 1428,80 1428,920 985,920 985,874" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<polyline points="890,15 890,80 1428,80 1428,920 985,920 985,874" fill="none" stroke="#4f46e5" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||||
|
<g><title>stream ETH0: wan — 1G connection to jim 21</title>
|
||||||
|
<rect x="570.0" y="-135" width="60" height="27" rx="3" fill="#fff" stroke="#92400e"/>
|
||||||
|
<rect x="576.0" y="-130" width="48" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="600" y="-144" class="port mono" text-anchor="middle">ETH0</text>
|
||||||
|
<rect x="618.0" y="-156" width="20" height="20" rx="4" fill="#92400e" stroke="none"/>
|
||||||
|
<text x="628.0" y="-142" class="port" text-anchor="middle" style="fill:white">I</text>
|
||||||
|
<text x="600" y="-93" class="small" text-anchor="middle">wan</text>
|
||||||
|
</g>
|
||||||
|
<g><title>stream ETH1: lan-jim — 1G connection to jim 6</title>
|
||||||
|
<rect x="750.0" y="-135" width="60" height="27" rx="3" fill="#fff" stroke="#0f766e"/>
|
||||||
|
<rect x="756.0" y="-130" width="48" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="780" y="-144" class="port mono" text-anchor="middle">ETH1</text>
|
||||||
|
<rect x="798.0" y="-156" width="20" height="20" rx="4" fill="#0f766e" stroke="none"/>
|
||||||
|
<text x="808.0" y="-142" class="port" text-anchor="middle" style="fill:white">J</text>
|
||||||
|
<text x="780" y="-93" class="small" text-anchor="middle">lan-jim</text>
|
||||||
|
</g>
|
||||||
|
<g><title>stream ETH2: et2 — Unused</title>
|
||||||
|
<rect x="930.0" y="-135" width="60" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="936.0" y="-130" width="48" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="960" y="-144" class="port mono" text-anchor="middle">ETH2</text>
|
||||||
|
<text x="960" y="-93" class="small" text-anchor="middle">et2</text>
|
||||||
|
</g>
|
||||||
|
<g><title>stream SFP+ L: et5 — Unused</title>
|
||||||
|
<rect x="615.0" y="-12" width="70" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="621.0" y="-7" width="58" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="650" y="-21" class="port mono" text-anchor="middle">SFP+ L</text>
|
||||||
|
<text x="650" y="30" class="small" text-anchor="middle">et5</text>
|
||||||
|
</g>
|
||||||
|
<g><title>stream SFP+ R: lan-dave — 10G connection to dave Q3.3</title>
|
||||||
|
<rect x="855.0" y="-12" width="70" height="27" rx="3" fill="#fff" stroke="#4f46e5"/>
|
||||||
|
<rect x="861.0" y="-7" width="58" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="890" y="-21" class="port mono" text-anchor="middle">SFP+ R</text>
|
||||||
|
<rect x="913.0" y="-33" width="20" height="20" rx="4" fill="#4f46e5" stroke="none"/>
|
||||||
|
<text x="923.0" y="-19" class="port" text-anchor="middle" style="fill:white">K</text>
|
||||||
|
<text x="890" y="30" class="small" text-anchor="middle">lan-dave</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 1: Port 1 — Configured port label</title>
|
||||||
|
<rect x="296.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="302.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="320" y="165" class="port mono" text-anchor="middle">1</text>
|
||||||
|
<text x="320" y="216" class="small" text-anchor="middle">fort</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 2: Port 2 — Configured port label</title>
|
||||||
|
<rect x="371.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="377.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="395" y="165" class="port mono" text-anchor="middle">2</text>
|
||||||
|
<text x="395" y="216" class="small" text-anchor="middle">castle</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 3: Port 3 — portcullis test connection</title>
|
||||||
|
<rect x="446.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="452.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="470" y="165" class="port mono" text-anchor="middle">3</text>
|
||||||
|
<text x="470" y="216" class="small" text-anchor="middle">portcullis</text>
|
||||||
|
<text x="470" y="228" class="small" text-anchor="middle">test</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 4: Port 4 — Second JetKVM</title>
|
||||||
|
<rect x="521.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="527.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="545" y="165" class="port mono" text-anchor="middle">4</text>
|
||||||
|
<text x="545" y="216" class="small" text-anchor="middle">portcullis</text>
|
||||||
|
<text x="545" y="228" class="small" text-anchor="middle">KVM</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 5: Port 5 — Backup fabric trunk</title>
|
||||||
|
<rect x="596.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#d97706"/>
|
||||||
|
<rect x="602.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="620" y="165" class="port mono" text-anchor="middle">5</text>
|
||||||
|
<rect x="632.0" y="153" width="20" height="20" rx="4" fill="#d97706" stroke="none"/>
|
||||||
|
<text x="642.0" y="167" class="port" text-anchor="middle" style="fill:white">C</text>
|
||||||
|
<text x="620" y="216" class="small" text-anchor="middle">jim P7</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 6: Port 6 — Native VLAN 140; tagged VLAN 10</title>
|
||||||
|
<rect x="671.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="677.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="695" y="165" class="port mono" text-anchor="middle">6</text>
|
||||||
|
<text x="695" y="216" class="small" text-anchor="middle">Digiweb ONT</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 7: Port 7 — AP trunk</title>
|
||||||
|
<rect x="746.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="752.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="770" y="165" class="port mono" text-anchor="middle">7</text>
|
||||||
|
<text x="770" y="216" class="small" text-anchor="middle">wave</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 8: Port 8 — AP trunk</title>
|
||||||
|
<rect x="821.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="827.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="845" y="165" class="port mono" text-anchor="middle">8</text>
|
||||||
|
<text x="845" y="216" class="small" text-anchor="middle">vibe</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 9: SFP+ 1 — LAG member 1</title>
|
||||||
|
<rect x="1027.5" y="174" width="65" height="27" rx="3" fill="#fff" stroke="#2563eb"/>
|
||||||
|
<rect x="1033.5" y="179" width="53" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="1060" y="165" class="port mono" text-anchor="middle">9</text>
|
||||||
|
<rect x="1080.5" y="153" width="20" height="20" rx="4" fill="#2563eb" stroke="none"/>
|
||||||
|
<text x="1090.5" y="167" class="port" text-anchor="middle" style="fill:white">A</text>
|
||||||
|
<text x="1060" y="216" class="small" text-anchor="middle">dave Q3.2</text>
|
||||||
|
</g>
|
||||||
|
<g><title>brian 10: SFP+ 2 — LAG member 2</title>
|
||||||
|
<rect x="1147.5" y="174" width="65" height="27" rx="3" fill="#fff" stroke="#7c3aed"/>
|
||||||
|
<rect x="1153.5" y="179" width="53" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="1180" y="165" class="port mono" text-anchor="middle">10</text>
|
||||||
|
<rect x="1200.5" y="153" width="20" height="20" rx="4" fill="#7c3aed" stroke="none"/>
|
||||||
|
<text x="1210.5" y="167" class="port" text-anchor="middle" style="fill:white">B</text>
|
||||||
|
<text x="1180" y="216" class="small" text-anchor="middle">dave Q3.4</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 1: ether1 — Configured member of shytzel-downlink</title>
|
||||||
|
<rect x="263.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="269.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="285" y="467" class="port mono" text-anchor="middle">1</text>
|
||||||
|
<text x="285" y="515" class="small" text-anchor="middle">shytzel</text>
|
||||||
|
<text x="285" y="527" class="small" text-anchor="middle">LAG</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 2: ether2 — Configured member of shytzel-downlink</title>
|
||||||
|
<rect x="263.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="269.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="285" y="393" class="port mono" text-anchor="middle">2</text>
|
||||||
|
<text x="285" y="441" class="small" text-anchor="middle">shytzel</text>
|
||||||
|
<text x="285" y="453" class="small" text-anchor="middle">LAG</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 3: ether3 — Configured member of shytzel-downlink</title>
|
||||||
|
<rect x="322.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="328.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="344" y="467" class="port mono" text-anchor="middle">3</text>
|
||||||
|
<text x="344" y="515" class="small" text-anchor="middle">shytzel</text>
|
||||||
|
<text x="344" y="527" class="small" text-anchor="middle">LAG</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 4: ether4 — Configured member of shytzel-downlink</title>
|
||||||
|
<rect x="322.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="328.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="344" y="393" class="port mono" text-anchor="middle">4</text>
|
||||||
|
<text x="344" y="441" class="small" text-anchor="middle">shytzel</text>
|
||||||
|
<text x="344" y="453" class="small" text-anchor="middle">LAG</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 5: palace — Core / netboot trunk</title>
|
||||||
|
<rect x="381.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#16a34a"/>
|
||||||
|
<rect x="387.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="403" y="467" class="port mono" text-anchor="middle">5</text>
|
||||||
|
<rect x="412.5" y="455" width="20" height="20" rx="4" fill="#16a34a" stroke="none"/>
|
||||||
|
<text x="422.5" y="469" class="port" text-anchor="middle" style="fill:white">F</text>
|
||||||
|
<text x="403" y="515" class="small" text-anchor="middle">palace</text>
|
||||||
|
<text x="403" y="527" class="small" text-anchor="middle">host</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 6: stream — Backup RSTP trunk</title>
|
||||||
|
<rect x="381.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#0f766e"/>
|
||||||
|
<rect x="387.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="403" y="393" class="port mono" text-anchor="middle">6</text>
|
||||||
|
<rect x="412.5" y="381" width="20" height="20" rx="4" fill="#0f766e" stroke="none"/>
|
||||||
|
<text x="422.5" y="395" class="port" text-anchor="middle" style="fill:white">J</text>
|
||||||
|
<text x="403" y="441" class="small" text-anchor="middle">stream</text>
|
||||||
|
<text x="403" y="453" class="small" text-anchor="middle">LAN</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 7: brian-downlink — Backup fabric trunk</title>
|
||||||
|
<rect x="440.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#d97706"/>
|
||||||
|
<rect x="446.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="462" y="467" class="port mono" text-anchor="middle">7</text>
|
||||||
|
<rect x="471.5" y="455" width="20" height="20" rx="4" fill="#d97706" stroke="none"/>
|
||||||
|
<text x="481.5" y="469" class="port" text-anchor="middle" style="fill:white">C</text>
|
||||||
|
<text x="462" y="515" class="small" text-anchor="middle">brian P5</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 8: palace-kvm — lo access</title>
|
||||||
|
<rect x="440.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="446.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="462" y="393" class="port mono" text-anchor="middle">8</text>
|
||||||
|
<text x="462" y="441" class="small" text-anchor="middle">JetKVM</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 9: ether9 — Unbridged spare</title>
|
||||||
|
<rect x="516.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="522.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="538" y="467" class="port mono" text-anchor="middle">9</text>
|
||||||
|
<text x="538" y="515" class="small" text-anchor="middle">spare</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 10: castle — Backup attachment, normally down</title>
|
||||||
|
<rect x="516.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="522.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="538" y="393" class="port mono" text-anchor="middle">10</text>
|
||||||
|
<text x="538" y="441" class="small" text-anchor="middle">castle</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 11: fort — lo access (configured label)</title>
|
||||||
|
<rect x="575.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="581.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="597" y="467" class="port mono" text-anchor="middle">11</text>
|
||||||
|
<text x="597" y="515" class="small" text-anchor="middle">fort</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 12: laptop-dock — lo access (configured label)</title>
|
||||||
|
<rect x="575.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="581.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="597" y="393" class="port mono" text-anchor="middle">12</text>
|
||||||
|
<text x="597" y="441" class="small" text-anchor="middle">dock</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 13: pronter — lo access (configured label)</title>
|
||||||
|
<rect x="634.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="640.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="656" y="467" class="port mono" text-anchor="middle">13</text>
|
||||||
|
<text x="656" y="515" class="small" text-anchor="middle">pronter</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 14: untrusted-sta1 — Untrusted access</title>
|
||||||
|
<rect x="634.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="640.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="656" y="393" class="port mono" text-anchor="middle">14</text>
|
||||||
|
<text x="656" y="441" class="small" text-anchor="middle">guest</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 15: ether15 — lo access; peer unconfirmed</title>
|
||||||
|
<rect x="693.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="699.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="715" y="467" class="port mono" text-anchor="middle">15</text>
|
||||||
|
<text x="715" y="515" class="small" text-anchor="middle">unmapped</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 16: ether16 — lo access; peer unconfirmed</title>
|
||||||
|
<rect x="693.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="699.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="715" y="393" class="port mono" text-anchor="middle">16</text>
|
||||||
|
<text x="715" y="441" class="small" text-anchor="middle">unmapped</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 17: ether17 — lo access; peer unconfirmed</title>
|
||||||
|
<rect x="769.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="775.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="791" y="467" class="port mono" text-anchor="middle">17</text>
|
||||||
|
<text x="791" y="515" class="small" text-anchor="middle">unmapped</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 18: ether18 — lo access; peer unconfirmed</title>
|
||||||
|
<rect x="769.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="775.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="791" y="393" class="port mono" text-anchor="middle">18</text>
|
||||||
|
<text x="791" y="441" class="small" text-anchor="middle">unmapped</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 19: ether19 — lo access; peer unconfirmed</title>
|
||||||
|
<rect x="828.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="834.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="850" y="467" class="port mono" text-anchor="middle">19</text>
|
||||||
|
<text x="850" y="515" class="small" text-anchor="middle">unmapped</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 20: ether20 — lo access; peer unconfirmed</title>
|
||||||
|
<rect x="828.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="834.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="850" y="393" class="port mono" text-anchor="middle">20</text>
|
||||||
|
<text x="850" y="441" class="small" text-anchor="middle">unmapped</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 21: stream-wan — Untagged VLAN 130 handoff</title>
|
||||||
|
<rect x="887.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#92400e"/>
|
||||||
|
<rect x="893.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="909" y="467" class="port mono" text-anchor="middle">21</text>
|
||||||
|
<rect x="918.5" y="455" width="20" height="20" rx="4" fill="#92400e" stroke="none"/>
|
||||||
|
<text x="928.5" y="469" class="port" text-anchor="middle" style="fill:white">I</text>
|
||||||
|
<text x="909" y="515" class="small" text-anchor="middle">stream</text>
|
||||||
|
<text x="909" y="527" class="small" text-anchor="middle">WAN</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 22: river — Dedicated router trunk</title>
|
||||||
|
<rect x="887.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#db2777"/>
|
||||||
|
<rect x="893.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="909" y="393" class="port mono" text-anchor="middle">22</text>
|
||||||
|
<rect x="918.5" y="381" width="20" height="20" rx="4" fill="#db2777" stroke="none"/>
|
||||||
|
<text x="928.5" y="395" class="port" text-anchor="middle" style="fill:white">G</text>
|
||||||
|
<text x="909" y="441" class="small" text-anchor="middle">river</text>
|
||||||
|
<text x="909" y="453" class="small" text-anchor="middle">backup</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 23: ups — lo access</title>
|
||||||
|
<rect x="946.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="952.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="968" y="467" class="port mono" text-anchor="middle">23</text>
|
||||||
|
<text x="968" y="515" class="small" text-anchor="middle">UPS</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim 24: dave-management — Core management</title>
|
||||||
|
<rect x="946.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="952.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="968" y="393" class="port mono" text-anchor="middle">24</text>
|
||||||
|
<rect x="977.5" y="381" width="20" height="20" rx="4" fill="#64748b" stroke="none"/>
|
||||||
|
<text x="987.5" y="395" class="port" text-anchor="middle" style="fill:white">E</text>
|
||||||
|
<text x="968" y="441" class="small" text-anchor="middle">dave</text>
|
||||||
|
<text x="968" y="453" class="small" text-anchor="middle">mgmt</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim S1: dave-uplink — 10G fabric trunk</title>
|
||||||
|
<rect x="1107.0" y="445" width="54" height="27" rx="3" fill="#fff" stroke="#0891b2"/>
|
||||||
|
<rect x="1113.0" y="450" width="42" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="1134" y="436" class="port mono" text-anchor="middle">S1</text>
|
||||||
|
<rect x="1149.0" y="424" width="20" height="20" rx="4" fill="#0891b2" stroke="none"/>
|
||||||
|
<text x="1159.0" y="438" class="port" text-anchor="middle" style="fill:white">D</text>
|
||||||
|
<text x="1134" y="487" class="small" text-anchor="middle">dave</text>
|
||||||
|
<text x="1134" y="499" class="small" text-anchor="middle">Q3.1</text>
|
||||||
|
</g>
|
||||||
|
<g><title>jim S2: sfp-spare — 10G trunk; colony staging</title>
|
||||||
|
<rect x="1208.0" y="445" width="54" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="1214.0" y="450" width="42" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="1235" y="436" class="port mono" text-anchor="middle">S2</text>
|
||||||
|
<text x="1235" y="487" class="small" text-anchor="middle">fergal</text>
|
||||||
|
</g>
|
||||||
|
<g><title>dave Q1: palace — 100G; whole cage</title>
|
||||||
|
<rect x="444.0" y="695" width="92" height="27" rx="3" fill="#fff" stroke="#dc2626"/>
|
||||||
|
<rect x="450.0" y="700" width="80" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="490" y="686" class="port mono" text-anchor="middle">Q1</text>
|
||||||
|
<rect x="524.0" y="674" width="20" height="20" rx="4" fill="#dc2626" stroke="none"/>
|
||||||
|
<text x="534.0" y="688" class="port" text-anchor="middle" style="fill:white">H</text>
|
||||||
|
<text x="490" y="737" class="small" text-anchor="middle">palace</text>
|
||||||
|
</g>
|
||||||
|
<g><title>dave Q2: castle — 100G; whole cage</title>
|
||||||
|
<rect x="584.0" y="695" width="92" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="590.0" y="700" width="80" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="630" y="686" class="port mono" text-anchor="middle">Q2</text>
|
||||||
|
<text x="630" y="737" class="small" text-anchor="middle">castle</text>
|
||||||
|
</g>
|
||||||
|
<g><title>dave Q4: qsfp28-4-1 — No identified peer</title>
|
||||||
|
<rect x="864.0" y="695" width="92" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="870.0" y="700" width="80" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="910" y="686" class="port mono" text-anchor="middle">Q4</text>
|
||||||
|
<text x="910" y="737" class="small" text-anchor="middle">unmapped</text>
|
||||||
|
</g>
|
||||||
|
<g><title>dave MGMT: management — 100M management RJ45</title>
|
||||||
|
<rect x="1095.0" y="741" width="54" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||||
|
<rect x="1101.0" y="746" width="42" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||||
|
<text x="1122" y="732" class="port mono" text-anchor="middle">MGMT</text>
|
||||||
|
<rect x="1137.0" y="720" width="20" height="20" rx="4" fill="#64748b" stroke="none"/>
|
||||||
|
<text x="1147.0" y="734" class="port" text-anchor="middle" style="fill:white">E</text>
|
||||||
|
</g>
|
||||||
|
<text x="485" y="870" class="port mono" text-anchor="start">Q3.1</text>
|
||||||
|
<rect x="536" y="854" width="20" height="20" rx="4" fill="#0891b2" stroke="none"/>
|
||||||
|
<text x="546" y="868" class="port" text-anchor="middle" style="fill:white">D</text>
|
||||||
|
<text x="485" y="889" class="small" text-anchor="start">jim SFP+1</text>
|
||||||
|
<rect x="568" y="854" width="34" height="20" rx="3" fill="white" stroke="#0891b2"/>
|
||||||
|
<rect x="574" y="860" width="22" height="8" rx="1" fill="#0891b2" stroke="none"/>
|
||||||
|
<text x="685" y="870" class="port mono" text-anchor="start">Q3.2</text>
|
||||||
|
<rect x="736" y="854" width="20" height="20" rx="4" fill="#2563eb" stroke="none"/>
|
||||||
|
<text x="746" y="868" class="port" text-anchor="middle" style="fill:white">A</text>
|
||||||
|
<text x="685" y="889" class="small" text-anchor="start">brian SFP+1</text>
|
||||||
|
<rect x="768" y="854" width="34" height="20" rx="3" fill="white" stroke="#2563eb"/>
|
||||||
|
<rect x="774" y="860" width="22" height="8" rx="1" fill="#2563eb" stroke="none"/>
|
||||||
|
<text x="885" y="870" class="port mono" text-anchor="start">Q3.3</text>
|
||||||
|
<rect x="936" y="854" width="20" height="20" rx="4" fill="#4f46e5" stroke="none"/>
|
||||||
|
<text x="946" y="868" class="port" text-anchor="middle" style="fill:white">K</text>
|
||||||
|
<text x="885" y="889" class="small" text-anchor="start">stream LAN</text>
|
||||||
|
<rect x="968" y="854" width="34" height="20" rx="3" fill="white" stroke="#4f46e5"/>
|
||||||
|
<rect x="974" y="860" width="22" height="8" rx="1" fill="#4f46e5" stroke="none"/>
|
||||||
|
<text x="1085" y="870" class="port mono" text-anchor="start">Q3.4</text>
|
||||||
|
<rect x="1136" y="854" width="20" height="20" rx="4" fill="#7c3aed" stroke="none"/>
|
||||||
|
<text x="1146" y="868" class="port" text-anchor="middle" style="fill:white">B</text>
|
||||||
|
<text x="1085" y="889" class="small" text-anchor="start">brian SFP+2</text>
|
||||||
|
<rect x="1168" y="854" width="34" height="20" rx="3" fill="white" stroke="#7c3aed"/>
|
||||||
|
<rect x="1174" y="860" width="22" height="8" rx="1" fill="#7c3aed" stroke="none"/>
|
||||||
|
<rect x="205" y="1090" width="290" height="76" rx="5" fill="white" stroke="#dc2626"/>
|
||||||
|
<text x="232" y="1120" class="port mono" text-anchor="start">uplink-dave</text>
|
||||||
|
<rect x="459" y="1102" width="20" height="20" rx="4" fill="#dc2626" stroke="none"/>
|
||||||
|
<text x="469" y="1116" class="port" text-anchor="middle" style="fill:white">H</text>
|
||||||
|
<text x="350" y="1146" class="small" text-anchor="middle">Mellanox ConnectX-4 · 100G QSFP28</text>
|
||||||
|
<rect x="585" y="1090" width="290" height="76" rx="5" fill="white" stroke="#16a34a"/>
|
||||||
|
<text x="612" y="1120" class="port mono" text-anchor="start">host-jim</text>
|
||||||
|
<rect x="839" y="1102" width="20" height="20" rx="4" fill="#16a34a" stroke="none"/>
|
||||||
|
<text x="849" y="1116" class="port" text-anchor="middle" style="fill:white">F</text>
|
||||||
|
<text x="730" y="1146" class="small" text-anchor="middle">Intel I211 · 1G RJ45</text>
|
||||||
|
<rect x="965" y="1090" width="290" height="76" rx="5" fill="white" stroke="#db2777"/>
|
||||||
|
<text x="992" y="1120" class="port mono" text-anchor="start">river-jim-phy</text>
|
||||||
|
<rect x="1219" y="1102" width="20" height="20" rx="4" fill="#db2777" stroke="none"/>
|
||||||
|
<text x="1229" y="1116" class="port" text-anchor="middle" style="fill:white">G</text>
|
||||||
|
<text x="1110" y="1146" class="small" text-anchor="middle">Intel I211 · 1G RJ45</text>
|
||||||
|
<text x="95" y="1220" class="sub" text-anchor="start">Matching letter badges identify cable ends. Port labels describe configured use; they do not indicate current link state.</text>
|
||||||
|
</g></svg>
|
||||||
|
After Width: | Height: | Size: 38 KiB |
@@ -0,0 +1,608 @@
|
|||||||
|
{
|
||||||
|
"view": "Front view; rack order is brian, jim, dave, then palace several U below. Socket order is physical; spacing and chassis widths are schematic. Stream sits physically left of brian and is drawn above it.",
|
||||||
|
"sources": {
|
||||||
|
"brian": "https://dl.ui.com/qig/usw-pro-xg-8-poe/",
|
||||||
|
"jim": "https://mikrotik.com/product/CRS326-24G-2SplusRM",
|
||||||
|
"dave": "https://mikrotik.com/product/crs504_4xq_in",
|
||||||
|
"stream": "https://www.gowinfanless.com/news/blogs/the-r86s-revolution-low-power-2-5gbe-and-10gbe-networking"
|
||||||
|
},
|
||||||
|
"brian": {
|
||||||
|
"model": "USW-Pro-XG-8-PoE",
|
||||||
|
"ports": [
|
||||||
|
{
|
||||||
|
"socket": "1",
|
||||||
|
"interface": "Port 1",
|
||||||
|
"label": "fort",
|
||||||
|
"role": "Configured port label",
|
||||||
|
"peer": "fort",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "2",
|
||||||
|
"interface": "Port 2",
|
||||||
|
"label": "castle",
|
||||||
|
"role": "Configured port label",
|
||||||
|
"peer": "castle",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "3",
|
||||||
|
"interface": "Port 3",
|
||||||
|
"label": "portcullis test",
|
||||||
|
"role": "portcullis test connection",
|
||||||
|
"peer": "portcullis",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "4",
|
||||||
|
"interface": "Port 4",
|
||||||
|
"label": "portcullis KVM",
|
||||||
|
"role": "Second JetKVM",
|
||||||
|
"peer": "portcullis-kvm",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "5",
|
||||||
|
"interface": "Port 5",
|
||||||
|
"label": "jim P7",
|
||||||
|
"role": "Backup fabric trunk",
|
||||||
|
"peer": "jim",
|
||||||
|
"peer_port": "7"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "6",
|
||||||
|
"interface": "Port 6",
|
||||||
|
"label": "Digiweb ONT",
|
||||||
|
"role": "Native VLAN 140; tagged VLAN 10",
|
||||||
|
"peer": "Digiweb ONT",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "7",
|
||||||
|
"interface": "Port 7",
|
||||||
|
"label": "wave",
|
||||||
|
"role": "AP trunk",
|
||||||
|
"peer": "wave",
|
||||||
|
"peer_port": "eth0"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "8",
|
||||||
|
"interface": "Port 8",
|
||||||
|
"label": "vibe",
|
||||||
|
"role": "AP trunk",
|
||||||
|
"peer": "vibe",
|
||||||
|
"peer_port": "ether1"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "9",
|
||||||
|
"interface": "SFP+ 1",
|
||||||
|
"label": "dave Q3.2",
|
||||||
|
"role": "LAG member 1",
|
||||||
|
"peer": "dave",
|
||||||
|
"peer_port": "Q3.2"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "10",
|
||||||
|
"interface": "SFP+ 2",
|
||||||
|
"label": "dave Q3.4",
|
||||||
|
"role": "LAG member 2",
|
||||||
|
"peer": "dave",
|
||||||
|
"peer_port": "Q3.4"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"jim": {
|
||||||
|
"model": "CRS326-24G-2S+RM",
|
||||||
|
"ports": [
|
||||||
|
{
|
||||||
|
"socket": "1",
|
||||||
|
"interface": "ether1",
|
||||||
|
"default_name": "ether1",
|
||||||
|
"label": "shytzel LAG",
|
||||||
|
"role": "Configured member of shytzel-downlink",
|
||||||
|
"peer": "shytzel",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "2",
|
||||||
|
"interface": "ether2",
|
||||||
|
"default_name": "ether2",
|
||||||
|
"label": "shytzel LAG",
|
||||||
|
"role": "Configured member of shytzel-downlink",
|
||||||
|
"peer": "shytzel",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "3",
|
||||||
|
"interface": "ether3",
|
||||||
|
"default_name": "ether3",
|
||||||
|
"label": "shytzel LAG",
|
||||||
|
"role": "Configured member of shytzel-downlink",
|
||||||
|
"peer": "shytzel",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "4",
|
||||||
|
"interface": "ether4",
|
||||||
|
"default_name": "ether4",
|
||||||
|
"label": "shytzel LAG",
|
||||||
|
"role": "Configured member of shytzel-downlink",
|
||||||
|
"peer": "shytzel",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "5",
|
||||||
|
"interface": "palace",
|
||||||
|
"default_name": "ether5",
|
||||||
|
"label": "palace host",
|
||||||
|
"role": "Core / netboot trunk",
|
||||||
|
"peer": "palace",
|
||||||
|
"peer_port": "host-jim"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "6",
|
||||||
|
"interface": "stream",
|
||||||
|
"default_name": "ether6",
|
||||||
|
"label": "stream LAN",
|
||||||
|
"role": "Backup RSTP trunk",
|
||||||
|
"peer": "stream",
|
||||||
|
"peer_port": "lan-jim"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "7",
|
||||||
|
"interface": "brian-downlink",
|
||||||
|
"default_name": "ether7",
|
||||||
|
"label": "brian P5",
|
||||||
|
"role": "Backup fabric trunk",
|
||||||
|
"peer": "brian",
|
||||||
|
"peer_port": "RJ45 5"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "8",
|
||||||
|
"interface": "palace-kvm",
|
||||||
|
"default_name": "ether8",
|
||||||
|
"label": "JetKVM",
|
||||||
|
"role": "lo access",
|
||||||
|
"peer": "palace-kvm",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "9",
|
||||||
|
"interface": "ether9",
|
||||||
|
"default_name": "ether9",
|
||||||
|
"label": "spare",
|
||||||
|
"role": "Unbridged spare",
|
||||||
|
"peer": null,
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "10",
|
||||||
|
"interface": "castle",
|
||||||
|
"default_name": "ether10",
|
||||||
|
"label": "castle",
|
||||||
|
"role": "Backup attachment, normally down",
|
||||||
|
"peer": "castle",
|
||||||
|
"peer_port": "et2.5g"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "11",
|
||||||
|
"interface": "fort",
|
||||||
|
"default_name": "ether11",
|
||||||
|
"label": "fort",
|
||||||
|
"role": "lo access (configured label)",
|
||||||
|
"peer": "fort",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "12",
|
||||||
|
"interface": "laptop-dock",
|
||||||
|
"default_name": "ether12",
|
||||||
|
"label": "dock",
|
||||||
|
"role": "lo access (configured label)",
|
||||||
|
"peer": "laptop-dock",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "13",
|
||||||
|
"interface": "pronter",
|
||||||
|
"default_name": "ether13",
|
||||||
|
"label": "pronter",
|
||||||
|
"role": "lo access (configured label)",
|
||||||
|
"peer": "pronter",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "14",
|
||||||
|
"interface": "untrusted-sta1",
|
||||||
|
"default_name": "ether14",
|
||||||
|
"label": "guest",
|
||||||
|
"role": "Untrusted access",
|
||||||
|
"peer": "untrusted-sta1",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "15",
|
||||||
|
"interface": "ether15",
|
||||||
|
"default_name": "ether15",
|
||||||
|
"label": "unmapped",
|
||||||
|
"role": "lo access; peer unconfirmed",
|
||||||
|
"peer": null,
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "16",
|
||||||
|
"interface": "ether16",
|
||||||
|
"default_name": "ether16",
|
||||||
|
"label": "unmapped",
|
||||||
|
"role": "lo access; peer unconfirmed",
|
||||||
|
"peer": null,
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "17",
|
||||||
|
"interface": "ether17",
|
||||||
|
"default_name": "ether17",
|
||||||
|
"label": "unmapped",
|
||||||
|
"role": "lo access; peer unconfirmed",
|
||||||
|
"peer": null,
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "18",
|
||||||
|
"interface": "ether18",
|
||||||
|
"default_name": "ether18",
|
||||||
|
"label": "unmapped",
|
||||||
|
"role": "lo access; peer unconfirmed",
|
||||||
|
"peer": null,
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "19",
|
||||||
|
"interface": "ether19",
|
||||||
|
"default_name": "ether19",
|
||||||
|
"label": "unmapped",
|
||||||
|
"role": "lo access; peer unconfirmed",
|
||||||
|
"peer": null,
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "20",
|
||||||
|
"interface": "ether20",
|
||||||
|
"default_name": "ether20",
|
||||||
|
"label": "unmapped",
|
||||||
|
"role": "lo access; peer unconfirmed",
|
||||||
|
"peer": null,
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "21",
|
||||||
|
"interface": "stream-wan",
|
||||||
|
"default_name": "ether21",
|
||||||
|
"label": "stream WAN",
|
||||||
|
"role": "Untagged VLAN 130 handoff",
|
||||||
|
"peer": "stream",
|
||||||
|
"peer_port": "wan"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "22",
|
||||||
|
"interface": "river",
|
||||||
|
"default_name": "ether22",
|
||||||
|
"label": "river backup",
|
||||||
|
"role": "Dedicated router trunk",
|
||||||
|
"peer": "palace",
|
||||||
|
"peer_port": "river-jim-phy"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "23",
|
||||||
|
"interface": "ups",
|
||||||
|
"default_name": "ether23",
|
||||||
|
"label": "UPS",
|
||||||
|
"role": "lo access",
|
||||||
|
"peer": "ups",
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "24",
|
||||||
|
"interface": "dave-management",
|
||||||
|
"default_name": "ether24",
|
||||||
|
"label": "dave mgmt",
|
||||||
|
"role": "Core management",
|
||||||
|
"peer": "dave",
|
||||||
|
"peer_port": "MGMT"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "S1",
|
||||||
|
"interface": "dave-uplink",
|
||||||
|
"default_name": "sfp-sfpplus1",
|
||||||
|
"label": "dave Q3.1",
|
||||||
|
"role": "10G fabric trunk",
|
||||||
|
"peer": "dave",
|
||||||
|
"peer_port": "Q3.1"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "S2",
|
||||||
|
"interface": "sfp-spare",
|
||||||
|
"default_name": "sfp-sfpplus2",
|
||||||
|
"label": "fergal",
|
||||||
|
"role": "10G trunk; colony staging",
|
||||||
|
"peer": "fergal",
|
||||||
|
"peer_port": "lan1"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"dave": {
|
||||||
|
"model": "CRS504-4XQ-IN",
|
||||||
|
"ports": [
|
||||||
|
{
|
||||||
|
"socket": "Q1",
|
||||||
|
"interface": "palace",
|
||||||
|
"default_name": "qsfp28-1-1",
|
||||||
|
"label": "palace",
|
||||||
|
"role": "100G; whole cage",
|
||||||
|
"peer": "palace",
|
||||||
|
"peer_port": "uplink-dave"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "Q2",
|
||||||
|
"interface": "castle",
|
||||||
|
"default_name": "qsfp28-2-1",
|
||||||
|
"label": "castle",
|
||||||
|
"role": "100G; whole cage",
|
||||||
|
"peer": "castle",
|
||||||
|
"peer_port": "et100g"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "Q3.1",
|
||||||
|
"interface": "jim-downlink",
|
||||||
|
"default_name": "qsfp28-3-1",
|
||||||
|
"label": "jim SFP+1",
|
||||||
|
"role": "10G breakout lane 1",
|
||||||
|
"peer": "jim",
|
||||||
|
"peer_port": "S1"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "Q3.2",
|
||||||
|
"interface": "brian1",
|
||||||
|
"default_name": "qsfp28-3-2",
|
||||||
|
"label": "brian SFP+1",
|
||||||
|
"role": "10G breakout lane 2; LAG",
|
||||||
|
"peer": "brian",
|
||||||
|
"peer_port": "9"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "Q3.3",
|
||||||
|
"interface": "stream",
|
||||||
|
"default_name": "qsfp28-3-3",
|
||||||
|
"label": "stream LAN",
|
||||||
|
"role": "10G breakout lane 3",
|
||||||
|
"peer": "stream",
|
||||||
|
"peer_port": "lan-dave"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "Q3.4",
|
||||||
|
"interface": "brian2",
|
||||||
|
"default_name": "qsfp28-3-4",
|
||||||
|
"label": "brian SFP+2",
|
||||||
|
"role": "10G breakout lane 4; LAG",
|
||||||
|
"peer": "brian",
|
||||||
|
"peer_port": "10"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "Q4",
|
||||||
|
"interface": "qsfp28-4-1",
|
||||||
|
"default_name": "qsfp28-4-1",
|
||||||
|
"label": "unmapped",
|
||||||
|
"role": "No identified peer",
|
||||||
|
"peer": null,
|
||||||
|
"peer_port": null
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "MGMT",
|
||||||
|
"interface": "management",
|
||||||
|
"default_name": "ether1",
|
||||||
|
"label": "jim P24",
|
||||||
|
"role": "100M management RJ45",
|
||||||
|
"peer": "jim",
|
||||||
|
"peer_port": "24"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"links": [
|
||||||
|
{
|
||||||
|
"id": "A",
|
||||||
|
"from": [
|
||||||
|
"brian",
|
||||||
|
"9"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"dave",
|
||||||
|
"Q3.2"
|
||||||
|
],
|
||||||
|
"description": "10G; first LAG member"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "B",
|
||||||
|
"from": [
|
||||||
|
"brian",
|
||||||
|
"10"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"dave",
|
||||||
|
"Q3.4"
|
||||||
|
],
|
||||||
|
"description": "10G; second LAG member"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "C",
|
||||||
|
"from": [
|
||||||
|
"brian",
|
||||||
|
"5"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"jim",
|
||||||
|
"7"
|
||||||
|
],
|
||||||
|
"description": "1G; redundant fabric trunk"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "D",
|
||||||
|
"from": [
|
||||||
|
"jim",
|
||||||
|
"S1"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"dave",
|
||||||
|
"Q3.1"
|
||||||
|
],
|
||||||
|
"description": "10G; direct fabric trunk"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "E",
|
||||||
|
"from": [
|
||||||
|
"jim",
|
||||||
|
"24"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"dave",
|
||||||
|
"MGMT"
|
||||||
|
],
|
||||||
|
"description": "100M; dave management"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "F",
|
||||||
|
"from": [
|
||||||
|
"jim",
|
||||||
|
"5"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"palace",
|
||||||
|
"host-jim"
|
||||||
|
],
|
||||||
|
"description": "1G; host core / netboot"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "G",
|
||||||
|
"from": [
|
||||||
|
"jim",
|
||||||
|
"22"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"palace",
|
||||||
|
"river-jim-phy"
|
||||||
|
],
|
||||||
|
"description": "1G; river backup trunk"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "H",
|
||||||
|
"from": [
|
||||||
|
"dave",
|
||||||
|
"Q1"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"palace",
|
||||||
|
"uplink-dave"
|
||||||
|
],
|
||||||
|
"description": "100G; host PF and guest VFs"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "I",
|
||||||
|
"from": [
|
||||||
|
"stream",
|
||||||
|
"ETH0"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"jim",
|
||||||
|
"21"
|
||||||
|
],
|
||||||
|
"description": "1G WAN; 2.5G socket on stream"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "J",
|
||||||
|
"from": [
|
||||||
|
"stream",
|
||||||
|
"ETH1"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"jim",
|
||||||
|
"6"
|
||||||
|
],
|
||||||
|
"description": "1G backup LAN; 2.5G socket on stream"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "K",
|
||||||
|
"from": [
|
||||||
|
"stream",
|
||||||
|
"SFP+ R"
|
||||||
|
],
|
||||||
|
"to": [
|
||||||
|
"dave",
|
||||||
|
"Q3.3"
|
||||||
|
],
|
||||||
|
"description": "10G preferred LAN"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"stream": {
|
||||||
|
"model": "BROUNION R86S",
|
||||||
|
"ports": [
|
||||||
|
{
|
||||||
|
"socket": "ETH0",
|
||||||
|
"interface": "wan",
|
||||||
|
"label": "wan",
|
||||||
|
"role": "1G connection to jim 21",
|
||||||
|
"nic": "Intel I225-V",
|
||||||
|
"capacity": "2.5G"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "ETH1",
|
||||||
|
"interface": "lan-jim",
|
||||||
|
"label": "lan-jim",
|
||||||
|
"role": "1G connection to jim 6",
|
||||||
|
"nic": "Intel I225-V",
|
||||||
|
"capacity": "2.5G"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "ETH2",
|
||||||
|
"interface": "et2",
|
||||||
|
"label": "et2",
|
||||||
|
"role": "Unused",
|
||||||
|
"nic": "Intel I225-V",
|
||||||
|
"capacity": "2.5G"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "SFP+ R",
|
||||||
|
"interface": "lan-dave",
|
||||||
|
"label": "lan-dave",
|
||||||
|
"role": "10G connection to dave Q3.3",
|
||||||
|
"nic": "Mellanox ConnectX-3",
|
||||||
|
"capacity": "10G"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "SFP+ L",
|
||||||
|
"interface": "et5",
|
||||||
|
"label": "et5",
|
||||||
|
"role": "Unused",
|
||||||
|
"nic": "Mellanox ConnectX-3",
|
||||||
|
"capacity": "10G"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"palace": {
|
||||||
|
"ports": [
|
||||||
|
{
|
||||||
|
"socket": "uplink-dave",
|
||||||
|
"nic": "Mellanox ConnectX-4",
|
||||||
|
"capacity": "100G QSFP28"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "host-jim",
|
||||||
|
"nic": "Intel I211",
|
||||||
|
"capacity": "1G RJ45"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"socket": "river-jim-phy",
|
||||||
|
"nic": "Intel I211",
|
||||||
|
"capacity": "1G RJ45"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
+102
-19
@@ -18,7 +18,7 @@ SR-IOV VFs, PCI NVMe drives and LVM disks.
|
|||||||
| Host storage | 500 GB Samsung SSD 860 EVO containing the EFI partition and the `main` LVM thin pool |
|
| Host storage | 500 GB Samsung SSD 860 EVO containing the EFI partition and the `main` LVM thin pool |
|
||||||
| Bulk storage | Three 8 TB Seagate IronWolf disks in the `hdds` VG, providing the RAID-backed `hdd-storage` and `frigate` LVs |
|
| Bulk storage | Three 8 TB Seagate IronWolf disks in the `hdds` VG, providing the RAID-backed `hdd-storage` and `frigate` LVs |
|
||||||
| NVMe storage | Three 2 TB Samsung NVMe devices passed through to `cellar`; SPDK combines them as the `NVMeRaid` RAID 0 device |
|
| NVMe storage | Three 2 TB Samsung NVMe devices passed through to `cellar`; SPDK combines them as the `NVMeRaid` RAID 0 device |
|
||||||
| Network / graphics | Mellanox ConnectX-4 100G adapter with four SR-IOV VFs, two Intel I211 Gigabit Ethernet controllers, and an AMD Radeon RX 550/560-family GPU |
|
| Network / graphics | Mellanox ConnectX-4 100G adapter using SR-IOV, two Intel I211 Gigabit Ethernet controllers (the only BASE-T NICs), and an AMD Radeon RX 550/560-family GPU |
|
||||||
|
|
||||||
## Role
|
## Role
|
||||||
|
|
||||||
@@ -34,33 +34,116 @@ See the consolidated [network assignments](../../networking.md#box-assignments)
|
|||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
100G `et100g` (mlx5, MTU 9000) uplinks to the `dave` switch and carries `lan-hi` (VLAN 100, the
|
100G `uplink-dave` (mlx5, MTU 9000) connects to the `dave` switch and carries `lan-hi` (VLAN 100,
|
||||||
`hi` assignment). A udev rule creates four SR-IOV VFs on the PF:
|
the `hi` assignment). A udev rule creates five SR-IOV VFs on the PF. Two physically separate Intel
|
||||||
|
connections lead to `jim`: `host-jim` serves `palace` and `sfh`, while `river-jim-phy` is exported
|
||||||
|
exclusively to `river`.
|
||||||
|
|
||||||
| VF | Consumer | VLAN handling |
|
- `lan-core` is a bridge with the `core` assignment (no gateway); the 1G `host-jim` connection and
|
||||||
|---|---|---|
|
the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used for VM netboot and
|
||||||
| 0 | `cellar` | `hi` |
|
`lo` clients.
|
||||||
| 1 | `river` | untagged VF; `river` tags its LAN and WAN VLANs |
|
- The separate 1G `river-jim-phy` connection is exported exclusively to `river` as the
|
||||||
| 2 | `sfh` | `hi` |
|
passthru-mode macvtap `river-jim-tap`; the guest names it `lan-jim`. Direct PCI passthrough is
|
||||||
| 3 | `sfh` container MACVLAN parent | `hi` |
|
blocked by its shared X399 IOMMU group; see [PCI isolation](#pci-isolation).
|
||||||
|
|
||||||
- `lan-core` is a bridge with the `core` assignment (no gateway); the 1G
|
### PCI isolation
|
||||||
`lan-core-phy` and the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used
|
|
||||||
for VM netboot and `lo` clients.
|
The Intel I211 controllers at `04:00.0` (`river-jim-phy`) and `06:00.0` (`host-jim`) share IOMMU
|
||||||
- The 1G `et1g0` (igb) is exported to `river` as a passthru-mode macvtap (`vm-et1g0`) — river sees
|
group 13 with chipset USB/SATA controllers, PCIe bridges and the wireless controller. Passing
|
||||||
it as `wan-old`.
|
only the router NIC through VFIO would leave the group nonviable; detaching the whole group
|
||||||
|
would remove essential host devices. The dedicated passthru-mode macvtap keeps the NIC host-owned.
|
||||||
|
Direct passthrough would need a NIC or firmware layout that isolates the controller.
|
||||||
|
|
||||||
|
Keep both Intel connections: `host-jim` provides independent host management and VM netboot,
|
||||||
|
while `river-jim-phy` carries the router's redundant trunk. The host physical router interface and
|
||||||
|
its macvtap both use `hiMTU`; the macvtap enables all-multicast reception so guest RSTP can receive
|
||||||
|
BPDUs without a guest-to-host receive-filter synchronization service.
|
||||||
|
|
||||||
|
### End-to-end interface topology
|
||||||
|
|
||||||
|
```text
|
||||||
|
dave
|
||||||
|
└─ palace uplink-dave (ConnectX-4 PF)
|
||||||
|
├─ VLAN 100 ───────────────────────────────> palace lan-hi
|
||||||
|
├─ VF 0 cellar-hi ────────────────────────> cellar lan-hi
|
||||||
|
├─ VF 1 river-rdma ───────────────────────> river rdma-hi ──> cellar NVMe/RDMA
|
||||||
|
├─ VF 2 sfh-hi ───────────────────────────> sfh lan-hi ─────> cellar NVMe/RDMA
|
||||||
|
├─ VF 3 sfh-ctrs ─────────────────────────> sfh ctrs-hi
|
||||||
|
│ ├─> hass host0
|
||||||
|
│ └─> unifi host0
|
||||||
|
└─ VF 4 river-dave ───────────────────────> river lan-dave ──┐
|
||||||
|
├─> river lan (RSTP)
|
||||||
|
jim │
|
||||||
|
├─ palace river-jim-phy ── river-jim-tap ─────> river lan-jim ───┘
|
||||||
|
└─ palace host-jim ──> palace lan-core
|
||||||
|
├─ sfh-core-tap ───────────────────────────> sfh ctrs-core ──> unifi lan-core
|
||||||
|
└─ VLAN 110 lan-lo-phy ──> palace lan-lo
|
||||||
|
└─ sfh-lo-tap ──────────────────────────> sfh ctrs-lo
|
||||||
|
├─> sfh PXE
|
||||||
|
└─> hass lan-lo
|
||||||
|
```
|
||||||
|
|
||||||
|
Inside `river`, `lan-dave` and `lan-jim` terminate on the shared `lan` RSTP bridge. `lan-core` is a
|
||||||
|
MACVLAN above it; `lan-hi`, `lan-lo` and `lan-untrusted` are VLAN uppers. `river` also adds its
|
||||||
|
Digiweb VLANs 10 and 140 to this bridge. `rdma-hi` is deliberately outside it.
|
||||||
|
|
||||||
|
VF 4 explicitly enables trust and disables MAC spoof checking for bridge traffic, which can use
|
||||||
|
MACs other than the VF's assigned address. Its automatic link state follows the physical port;
|
||||||
|
the storage VFs retain forced link state so local VF-to-VF storage can continue without carrier.
|
||||||
|
The firmware must expose enough VFs before the udev allocation can succeed.
|
||||||
|
|
||||||
|
### Interface map
|
||||||
|
|
||||||
|
The physical connections terminate on `palace`; PCI VFs and virtual interfaces hand traffic
|
||||||
|
from the host to its guests. Interface names in the destination columns belong to that box.
|
||||||
|
|
||||||
|
#### Physical connections
|
||||||
|
|
||||||
|
| Switch port | Host interface | Host networks | Guest handoff |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `dave` `palace` | `uplink-dave` | VLAN 100 → `lan-hi` | Five PCI VFs, listed below |
|
||||||
|
| `jim` `palace` | `host-jim` | Native `core` → `lan-core`; VLAN 110 → `lan-lo-phy` → `lan-lo` | `sfh-core-tap` and `sfh-lo-tap` |
|
||||||
|
| `jim` `river` | `river-jim-phy` | No host L3 configuration | `river-jim-tap` macvtap → `river` `lan-jim` |
|
||||||
|
|
||||||
|
#### ConnectX-4 VFs
|
||||||
|
|
||||||
|
All five VFs belong to `uplink-dave`. VLAN filtering and carrier policy are applied on the PF.
|
||||||
|
|
||||||
|
| VF / PCI BDF | QEMU device | Guest | Guest interface | VLAN / carrier | Role |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| 0 / `44:00.1` | `cellar-hi` | [cellar](cellar.md) | `lan-hi` | 100 / forced up | Storage target and management |
|
||||||
|
| 1 / `44:00.2` | `river-rdma` | [river](river.md#storage) | `rdma-hi` | 100 / forced up | Dedicated NVMe/RDMA root path |
|
||||||
|
| 2 / `44:00.3` | `sfh-hi` | [sfh](sfh/README.md) | `lan-hi` | 100 / forced up | NVMe/RDMA root and management |
|
||||||
|
| 3 / `44:00.4` | `sfh-ctrs` | [sfh](sfh/README.md#networking) | `ctrs-hi` | 100 / forced up | Container `hi` network |
|
||||||
|
| 4 / `44:00.5` | `river-dave` | [river](river.md#virtual-machine-and-network-attachment) | `lan-dave` | Unfiltered trunk / automatic | Preferred RSTP attachment |
|
||||||
|
|
||||||
|
#### Virtual Ethernet handoffs
|
||||||
|
|
||||||
|
| Host interface | Kind / parent | Guest interface | Downstream use |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `river-jim-tap` | Passthru macvtap on `river-jim-phy` | `river` `lan-jim` | Backup RSTP attachment |
|
||||||
|
| `sfh-core-tap` | TAP on `lan-core` | `sfh` `ctrs-core` | `unifi` `lan-core` |
|
||||||
|
| `sfh-lo-tap` | TAP on `lan-lo` | `sfh` `ctrs-lo` | PXE boot; `hass` `lan-lo` |
|
||||||
|
|
||||||
|
`sfh` keeps no L3 configuration on the three `ctrs-*` parents. Its nspawn configuration creates the
|
||||||
|
container-side names shown above; `host0` remains the default `hi` interface in both containers.
|
||||||
|
See [the `sfh` page](sfh/README.md#networking) for its container-oriented view and
|
||||||
|
[failover validation](router-failover.md) for tested failure scenarios and maintenance checks.
|
||||||
|
|
||||||
## VMs
|
## VMs
|
||||||
|
|
||||||
|
The RAID-backed `hdds` LVs need `dm-raid` loaded in the initrd: LVM's initial autoactivation can
|
||||||
|
run before stage-two module loading. `sfh` requires the `frigate` LV before QEMU can start.
|
||||||
|
|
||||||
| VM | vCPUs | RAM | Passthrough | Notes |
|
| VM | vCPUs | RAM | Passthrough | Notes |
|
||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| `cellar` | 8c × 2t | 16 GiB | VF 0 (`44:00.1`); NVMe `41:00.0`–`43:00.0` | pinned to NUMA node 1; split IRQ chip + vIOMMU |
|
| `cellar` | 8c × 2t | 16 GiB | `cellar-hi` VF 0 (`44:00.1`); NVMe `41:00.0`–`43:00.0` | pinned to NUMA node 1; split IRQ chip + vIOMMU |
|
||||||
| `river` | 3c × 2t | 4 GiB | VF 1 (`44:00.2`); macvtap `vm-et1g0` | only an ESP local disk (plus an installer ISO) — root is NVMe-oF from `cellar` |
|
| `river` | 3c × 2t | 4 GiB | `river-rdma` VF 1 (`44:00.2`), `river-dave` VF 4 (`44:00.5`), `river-jim-tap` | only an ESP local disk (plus an installer ISO) — root is NVMe-oF from `cellar` |
|
||||||
| `sfh` | 8c × 2t | 32 GiB | VF 2 (`44:00.3`), VF 3 (`44:00.4`); two USB host ports | no boot disk — netboots; gets the `hdds/frigate` LV |
|
| `sfh` | 8c × 2t | 32 GiB | `sfh-hi` VF 2 (`44:00.3`), `sfh-ctrs` VF 3 (`44:00.4`); two USB host ports | no boot disk — netboots; gets the `hdds/frigate` LV |
|
||||||
|
|
||||||
Boot ordering is enforced with systemd dependencies: `vm@river` waits for `cellar`'s SSH port (and
|
Boot ordering is enforced with systemd dependencies: `vm@river` waits for `cellar`'s SSH port (and
|
||||||
for the `vm-et1g0` device), and `vm@sfh` waits for `river` — storage first, then the router, then
|
for the `river-jim-tap` device), and `vm@sfh` waits for `river` — storage first, then the router,
|
||||||
everything that boots off both.
|
then everything that boots off both.
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
|
|||||||
+35
-12
@@ -24,9 +24,10 @@ See the consolidated [network assignments](../../networking.md#box-assignments)
|
|||||||
### Link and addressing
|
### Link and addressing
|
||||||
|
|
||||||
`services.pppd` peer `digiweb` attaches directly to `wan-pon-isp`, the raw ISP VLAN 10
|
`services.pppd` peer `digiweb` attaches directly to `wan-pon-isp`, the raw ISP VLAN 10
|
||||||
(`vlans.pon-isp`) trunked untranslated from the ONT through `brian` and `dave`; see
|
(`vlans.pon-isp`) trunked untranslated from the ONT. It normally reaches `river` through `brian`
|
||||||
[switches.md](switches.md). The netdev has no L3 configuration and uses `MTUBytes=1508`, leaving a
|
and `dave`; the configured RSTP path through `jim` is intended to provide redundancy after
|
||||||
clean `mtu`/`mru 1500` after PPPoE overhead.
|
successful deployment and validation. See [switches.md](switches.md). The netdev has no L3
|
||||||
|
configuration and uses `MTUBytes=1508`, leaving a clean `mtu`/`mru 1500` after PPPoE overhead.
|
||||||
|
|
||||||
The static `84.203.124.128` address is requested through IPCP. The provider-wide credentials are
|
The static `84.203.124.128` address is requested through IPCP. The provider-wide credentials are
|
||||||
deliberately not secret; the peer persists indefinitely with LCP echo monitoring and ignores
|
deliberately not secret; the peer persists indefinitely with LCP echo monitoring and ignores
|
||||||
@@ -46,23 +47,45 @@ every WAN flap.
|
|||||||
|
|
||||||
### Traffic shaping
|
### Traffic shaping
|
||||||
|
|
||||||
The shared `wan-ifb` ingress-shaping pieces are inert here. CAKE is specific to `stream`, and
|
CAKE and the `wan-ifb` network configuration remain specific to `stream`. The shared IFB netdev
|
||||||
`networkd-dispatcher` is `mkForce false` pending scheduling tests.
|
exists here but has no matching `.network` file to trigger the ingress-mirroring rule. The source
|
||||||
|
enables the shared `networkd-dispatcher` for RSTP setup.
|
||||||
|
|
||||||
## Platform
|
## Platform
|
||||||
|
|
||||||
### Virtual machine and network attachment
|
### Virtual machine and network attachment
|
||||||
|
|
||||||
The 100G `lan` NIC is VF 1 of `palace`'s `et100g` (MTU 9000), with every router VLAN tagged on top
|
The router's ordinary networks sit above the `lan` RSTP bridge. Its preferred port, `lan-dave`, is
|
||||||
as `55-lan`. A macvtap of `palace`'s 1G `et1g0` remains as the old `wan-old` path without L3
|
an unfiltered ConnectX-4 VF trunk; `lan-jim` is the 1G `river-jim-tap` macvtap exported by
|
||||||
configuration. Deployments use the `hi` assignment.
|
`palace`. Native `core`, the client VLANs and Digiweb VLANs 10/140 use `dave` normally and fail
|
||||||
|
over to `jim`. The shared policy is documented in [Router HA](../../networking.md#router-ha);
|
||||||
|
[failover validation](router-failover.md) covers attachment and switch outages.
|
||||||
|
|
||||||
|
The bridge uses the trunk VF's assigned MAC so the ConnectX-4's local switching can deliver
|
||||||
|
traffic from `palace` and the other VFs to the router. On `lan-hi` and `rdma-hi`, `arp_ignore=1`
|
||||||
|
and `arp_announce=2` keep router and storage addresses on their respective interfaces; otherwise
|
||||||
|
the storage VF can answer ARP for the router VIP and attract traffic to the wrong interface.
|
||||||
|
|
||||||
|
The separate ConnectX-4 VF `rdma-hi` is not a bridge port. It has a PF-applied VLAN 100 and exists
|
||||||
|
only for storage traffic.
|
||||||
|
|
||||||
|
Ordinary Ethernet traverses the guest software bridge; this does not require mlx5
|
||||||
|
switchdev or host representors. The bridge disables LRO on its ports; RDMA stays outside this
|
||||||
|
path.
|
||||||
|
|
||||||
### Storage
|
### Storage
|
||||||
|
|
||||||
The VM's local disk holds only an ESP; `/nix` and `/persist` are LVs on `cellar`'s
|
The VM's local disk holds only an ESP; `/nix` and `/persist` are LVs on `cellar`'s
|
||||||
`nqn.2016-06.io.spdk:river` namespace over RDMA. The initrd brings up `lan-hi`
|
`nqn.2016-06.io.spdk:river` namespace over RDMA. The initrd brings up `rdma-hi` with the dedicated
|
||||||
with `roceBootModules`, and `KeepConfiguration=static` prevents networkd from dropping the address
|
`rdma` assignment and a link-scoped host route to `cellar`, alongside `roceBootModules`.
|
||||||
during reconfiguration. An installer ISO remains attached.
|
|
||||||
|
NixOS copies the main system's `.link` units into the systemd initrd, so the `rdma-hi` rename is
|
||||||
|
shared, but its `.network` and `.netdev` units are separate. `river` therefore declares the storage
|
||||||
|
assignment and route once under `boot.initrd.systemd.network` and again under the main
|
||||||
|
`systemd.network`. The systemd initrd leaves its kernel network state in place during the stage-two
|
||||||
|
handover; `KeepConfiguration=static` on the main `.network` prevents networkd from dropping and
|
||||||
|
recreating the static address and route underneath the mounted NVMe-oF root. An installer ISO
|
||||||
|
remains attached.
|
||||||
|
|
||||||
## Netboot
|
## Netboot
|
||||||
|
|
||||||
@@ -76,4 +99,4 @@ assignment, restricted to the `hi` and `lo` prefixes.
|
|||||||
- [`nixos/boxes/home/routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix) —
|
- [`nixos/boxes/home/routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix) —
|
||||||
shared router definition (assignments, firewall/NAT, `as211024`).
|
shared router definition (assignments, firewall/NAT, `as211024`).
|
||||||
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
|
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
|
||||||
the VM definition on `palace` (VF 1, macvtap, ESP disk).
|
the VM definition on `palace` (dedicated RDMA VF, trunk VF, macvtap and ESP disk).
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
# Router failover validation
|
||||||
|
|
||||||
|
Failure-scenario checks for the deployed home router pair. The design lives in
|
||||||
|
[Router HA](../../networking.md#router-ha), the physical paths in [switches.md](switches.md),
|
||||||
|
and the host/VF layout in [palace.md](palace.md) and [river.md](river.md).
|
||||||
|
|
||||||
|
## Validated scenarios
|
||||||
|
|
||||||
|
These checks were performed on 2026-11-15. WAN and ONT probes check Ethernet path availability;
|
||||||
|
reads from `river`'s root LV and probes over `rdma-hi` check its separate storage path.
|
||||||
|
|
||||||
|
| Scenario | Test | Validated behavior |
|
||||||
|
|---|---|---|
|
||||||
|
| Router startup | Boot both routers | Kea starts without retries; the configured bridge MAC, ARP policy and administrative RSTP costs are applied |
|
||||||
|
| Primary router offline | Reboot its host | `stream` takes over both IPv4 and IPv6 VIPs, then returns to standby when `river` recovers |
|
||||||
|
| Preferred guest port down | Disable `river`'s `lan-dave`, then restore it | `lan-jim` becomes root/forwarding; WAN, ONT and 9000-byte RDMA pings pass while direct root-LV reads continue for 20 seconds |
|
||||||
|
| Backup guest port down | Disable `river`'s `lan-jim`, then restore it | `lan-dave` stays root/forwarding; WAN, ONT, jumbo RDMA and root-LV read checks pass |
|
||||||
|
| Physical `palace` attachment lost | Disable `dave`'s `palace` port for 30 seconds with automatic restoration | `lan-dave` loses carrier and `lan-jim` becomes root/forwarding; forced-link storage VFs retain local RDMA and root-LV reads succeed |
|
||||||
|
| Whole `dave` outage | Reboot the switch | `jim` becomes root and forwards on `brian-downlink`; Digiweb WAN and ONT access use `jim`–`brian` while local RDMA reads continue |
|
||||||
|
| Whole `jim` outage | Reboot the switch | `river` continues using `dave`; WAN, ONT and root-storage read probes succeed |
|
||||||
|
| Restoration | Restore each failed attachment or switch | The normal RSTP preferences return and `palace` is reachable on `hi` again |
|
||||||
|
|
||||||
|
Brief WAN/ONT probe losses occur during the tested `dave` tree transitions; subsequent probes
|
||||||
|
succeed on the backup tree and after restoration. The monitors recorded no storage-read
|
||||||
|
failures or matched kernel NVMe errors. The whole-switch test demonstrates the backup WAN path
|
||||||
|
without relying on the direct `jim`–`dave` trunk.
|
||||||
|
|
||||||
|
## Test limits
|
||||||
|
|
||||||
|
The storage checks exercise reads, not write integrity or long-duration endurance. They validate
|
||||||
|
local VF-to-VF RDMA during physical carrier loss, not failure of the ConnectX-4 itself. Storage
|
||||||
|
still depends on the adapter, `palace` and `cellar`; see [river's storage](river.md#storage).
|
||||||
|
|
||||||
|
`brian`'s live RSTP port state was not captured during the outage tests. Its backup
|
||||||
|
connection is validated by successful WAN and ONT traffic while `dave` is offline.
|
||||||
|
|
||||||
|
Disabling only the guest trunk VF while the host PF remains healthy is an artificial fault:
|
||||||
|
`palace`'s `hi` return traffic can still target that local VF's MAC and become unreachable through
|
||||||
|
`river`. During physical carrier loss, external `hi` access to the host and its guests is also
|
||||||
|
unavailable. `palace` remains accessible on `core` through a jump connection from `stream`;
|
||||||
|
direct routed management access depends on a working return route.
|
||||||
|
|
||||||
|
## Repeating the checks
|
||||||
|
|
||||||
|
Perform maintenance from a box independent of `palace`, such as `tower`. Before interrupting
|
||||||
|
`palace` or `dave`, power `castle` off cleanly: its NVMe-oF root depends on `cellar` and `dave`.
|
||||||
|
Keep a known-working boot entry and working JetKVM video and keyboard access.
|
||||||
|
|
||||||
|
1. Check boot defaults, system profiles, switch VLAN membership and spanning-tree state. When
|
||||||
|
changing host and guest networking together, stage matching configurations with `deploy --boot`.
|
||||||
|
2. Verify VF isolation, VLAN/trust/carrier policies, MTUs, the RoCE GID and the mounted NVMe
|
||||||
|
subsystem before injecting a fault.
|
||||||
|
3. Arrange independent automatic restoration before disabling an interface or switch port.
|
||||||
|
Run probes and storage reads in a service that survives loss of the SSH connection; bound
|
||||||
|
remote commands so a path change cannot leave a check waiting indefinitely.
|
||||||
|
4. Inject one fault at a time. Monitor root/alternate port roles, native `core`, client VLANs,
|
||||||
|
ONT access, PPPoE, DNS through the floating VIPs and storage I/O. Capture BPDUs when changing
|
||||||
|
the topology and watch for returned own-source packets or MAC flapping.
|
||||||
|
5. Restore the path and verify the normal RSTP roles, VIP ownership, services and management
|
||||||
|
access before testing another fault.
|
||||||
|
|
||||||
|
Switch changes require confirmation; see [switches.md](switches.md#accessing-the-switches).
|
||||||
@@ -31,16 +31,21 @@ See the consolidated [network assignments](../../../networking.md#box-assignment
|
|||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
Four NICs, all MTU 9000 where jumbo-capable:
|
| Interface | `palace` attachment | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| `lan-hi` | `sfh-hi` SR-IOV VF 2 | Box's `hi` assignment and NVMe/RDMA root path; MTU 9000 |
|
||||||
|
| `ctrs-hi` | `sfh-ctrs` SR-IOV VF 3 | L3-less MACVLAN parent for each container's `host0`; MTU 9000 |
|
||||||
|
| `ctrs-core` | Virtio via `sfh-core-tap`, `lan-core` and `host-jim` | L3-less MACVLAN parent for UniFi's `lan-core` management attachment |
|
||||||
|
| `ctrs-lo` | Virtio via `sfh-lo-tap`, `lan-lo` and `host-jim` | L3-less MACVLAN parent for `hass`'s `lan-lo` and the VM's PXE path |
|
||||||
|
|
||||||
- `lan-hi` — SR-IOV VF 2, the box's own `hi` assignment.
|
The two VFs separate the box's RDMA root traffic from container `hi` traffic; they do not provide
|
||||||
- `lan-hi-ctrs` — SR-IOV VF 3, no L3: the MACVLAN parent for the containers' `hi` interfaces
|
redundant switch paths. The Virtio attachments reach the `jim`-facing `palace` host bridges. Container
|
||||||
(`host0` inside each container).
|
`host0` names are intentionally unchanged.
|
||||||
- `lan-core-ctrs` / `lan-lo-ctrs` — virtio NICs (bridged to palace's `lan-core` / `lan-lo`), no
|
|
||||||
L3: MACVLAN parents for containers that need a `core` or `lo` interface.
|
|
||||||
|
|
||||||
The per-container MACVLAN wiring lives in `systemd.nspawn.*.networkConfig` in
|
The per-container MACVLAN wiring lives in `systemd.nspawn.*.networkConfig` in
|
||||||
[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix).
|
[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix). The
|
||||||
|
[`palace` interface topology](../palace.md#end-to-end-interface-topology) traces these interfaces
|
||||||
|
back through the VM handoffs to the physical switch connections.
|
||||||
|
|
||||||
## Containers
|
## Containers
|
||||||
|
|
||||||
|
|||||||
@@ -47,8 +47,8 @@ Passed through from `sfh` (USB host ports on `palace`):
|
|||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment,
|
MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `ctrs-hi` (the `hi` assignment,
|
||||||
alt name `frigate`, default gateway via the VIP) and `lan-lo` on `lan-lo-ctrs` (the `lo`
|
alt name `frigate`, default gateway via the VIP) and `lan-lo` on `ctrs-lo` (the `lo`
|
||||||
assignment, no gateway) — the `lo` interface reaches the IoT devices (the Reolink camera lives there).
|
assignment, no gateway) — the `lo` interface reaches the IoT devices (the Reolink camera lives there).
|
||||||
|
|
||||||
## Notable config files
|
## Notable config files
|
||||||
|
|||||||
@@ -29,8 +29,8 @@ management network. It is imported, listed in `sfh`'s `my.containers.instances`,
|
|||||||
|
|
||||||
## Networking
|
## Networking
|
||||||
|
|
||||||
Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` carries the `hi`
|
Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `ctrs-hi` carries the `hi`
|
||||||
assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `lan-core-ctrs` carries
|
assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `ctrs-core` carries
|
||||||
the gatewayless `core` assignment (`unifi-ctr-core`). The `core` interface is how the controller
|
the gatewayless `core` assignment (`unifi-ctr-core`). The `core` interface is how the controller
|
||||||
talks to `brian` and the other switches on their management network.
|
talks to `brian` and the other switches on their management network.
|
||||||
|
|
||||||
|
|||||||
+15
-10
@@ -16,7 +16,7 @@ redundant router pair with [`river`](river.md) and is dual-homed to both switche
|
|||||||
| CPU | Intel Celeron N5105 (4 cores / 4 threads) |
|
| CPU | Intel Celeron N5105 (4 cores / 4 threads) |
|
||||||
| Memory | 16 GiB |
|
| Memory | 16 GiB |
|
||||||
| Storage | 512 GB Samsung SSD 970 PRO NVMe containing `/boot`, `/nix` and `/persist`; integrated 128 GB eMMC is present but is not used by the declared filesystems |
|
| Storage | 512 GB Samsung SSD 970 PRO NVMe containing `/boot`, `/nix` and `/persist`; integrated 128 GB eMMC is present but is not used by the declared filesystems |
|
||||||
| Network | Three Intel `igc` interfaces and a dual-port Mellanox `mlx4_en` adapter; `wan`, `lan-jim` and `lan-dave` use three of these ports |
|
| Network | Three Intel I225-V 2.5G interfaces (`igc`) and a dual-port Mellanox ConnectX-3 10G SFP+ adapter (`mlx4_en`); `wan`, `lan-jim` and `lan-dave` use three of these ports |
|
||||||
|
|
||||||
The platform configuration enables `kvm-intel`, `intel_iommu=on` and Intel microcode updates.
|
The platform configuration enables `kvm-intel`, `intel_iommu=on` and Intel microcode updates.
|
||||||
|
|
||||||
@@ -39,8 +39,9 @@ lease this section assumes — see [wwan.md](wwan.md).
|
|||||||
|
|
||||||
### Link and addressing
|
### Link and addressing
|
||||||
|
|
||||||
`wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) towards the cable modem. The modem segment is
|
`wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) on a physically separate connection to `jim`'s
|
||||||
switch VLAN 130; `jim` handles the tag, so the box interface is untagged. See
|
`stream-wan` port. The Virgin Media modem connects to `shytzel`, which trunks
|
||||||
|
VLAN 130 to `jim`; `stream-wan` delivers it untagged to the box. See
|
||||||
[switches.md](switches.md) for the fabric side.
|
[switches.md](switches.md) for the fabric side.
|
||||||
|
|
||||||
`DHCP=ipv4` pulls the public lease. `dhcpV4Config.UseDNS=false` points resolution at the local
|
`DHCP=ipv4` pulls the public lease. `dhcpV4Config.UseDNS=false` points resolution at the local
|
||||||
@@ -79,11 +80,15 @@ box sets:
|
|||||||
|
|
||||||
## Switching (RSTP)
|
## Switching (RSTP)
|
||||||
|
|
||||||
`stream` is dual-homed to both switches: `lan-jim` (igc) and `lan-dave` (mlx4_en), both MTU 9000,
|
`stream` is dual-homed to both switches: `lan-dave` (mlx4_en) and `lan-jim` (igc), both MTU 9000,
|
||||||
are enslaved to the `lan` bridge with `STP=true`. The explicit bridge-port costs prefer
|
are ports of the shared `lan` bridge. `routing-common` owns the bridge, its STP setting and the
|
||||||
`lan-dave` at 10 over `lan-jim` at 100. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
|
explicit `dave`/`jim` costs of 10/100. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
|
||||||
runs a patched `mstpd` and forces RSTP on `lan` once it is configured, so exactly one uplink carries
|
runs a patched `mstpd` and forces RSTP once `lan` is configured, so exactly one uplink carries
|
||||||
traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfigured.)
|
traffic at a time and `dave` is preferred. (The remaining NICs are renamed `et2`/`et5` and left
|
||||||
|
unconfigured.)
|
||||||
|
|
||||||
|
The separate `wan` connection to `jim` is not a bridge port. Retaining it provides an isolated
|
||||||
|
untagged DHCP handoff for Virgin Media and a simple path for a future replacement direct-IP WAN.
|
||||||
|
|
||||||
## Deployment
|
## Deployment
|
||||||
|
|
||||||
@@ -96,8 +101,8 @@ traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfig
|
|||||||
## Notable config files
|
## Notable config files
|
||||||
|
|
||||||
- [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) — box config: DHCP WAN,
|
- [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) — box config: DHCP WAN,
|
||||||
modem management, CAKE, `wan-online.target` gate, STP bridge.
|
modem management, CAKE, `wan-online.target` gate and hardware link matching.
|
||||||
- [`nixos/boxes/home/routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix) —
|
- [`nixos/boxes/home/routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix) —
|
||||||
shared router definition (index 1).
|
shared router definition, including the `lan` bridge and port preferences (index 1).
|
||||||
- [`nixos/boxes/home/routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix) —
|
- [`nixos/boxes/home/routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix) —
|
||||||
RSTP on the `lan` bridge.
|
RSTP on the `lan` bridge.
|
||||||
|
|||||||
+165
-72
@@ -1,12 +1,11 @@
|
|||||||
# Home switches
|
# Home switches
|
||||||
|
|
||||||
Reference for the two MikroTik switches on the home network — **jim** and **dave** — plus the
|
Reference for the home switches: MikroTik boxes `jim` and `dave`, Ubiquiti switch `brian`, and
|
||||||
Ubiquiti switch **brian**, and how the home boxes and the Digiweb WAN hang off them. These switches
|
Zyxel L2 switch `shytzel`. These switches are **not** managed by this flake; they are configured by
|
||||||
are **not** managed by this flake; they are configured by hand (RouterOS on jim/dave, UniFi on
|
hand. This page covers the physical topology, VLAN map, and WAN paths.
|
||||||
brian). It covers the physical topology, the VLAN map, and how the Digiweb WAN reaches river.
|
|
||||||
|
|
||||||
In short: the Digiweb ISP VLAN (10) is trunked straight through to river (which runs PPPoE on it),
|
In short: the Digiweb ISP VLAN (10) is trunked straight through to `river` (which runs PPPoE on it),
|
||||||
and the ONT's untagged management is PVID'd onto VLAN 140 at brian, its edge switch. VLAN 10 is
|
and the ONT's untagged management is PVID'd onto VLAN 140 at `brian`, its edge switch. VLAN 10 is
|
||||||
carried untranslated because a single ONT makes it unique on the fabric — see
|
carried untranslated because a single ONT makes it unique on the fabric — see
|
||||||
[the WAN path](#the-digiweb-wan-path-trunked-vlan-10--pvid-140) and
|
[the WAN path](#the-digiweb-wan-path-trunked-vlan-10--pvid-140) and
|
||||||
[why not translation](#why-not-translation-for-one-ont). The router side lives in
|
[why not translation](#why-not-translation-for-one-ont). The router side lives in
|
||||||
@@ -16,72 +15,116 @@ but belongs to the colony site — see [fergal](#fergal-portculliss-switch).
|
|||||||
|
|
||||||
## The switches
|
## The switches
|
||||||
|
|
||||||
| | jim | dave | brian |
|
| | `jim` | `dave` | `brian` | `shytzel` |
|
||||||
|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| Identity | `jim-sw` | `dave-sw` | (UniFi) |
|
| Identity | `jim-sw` | `dave-sw` | UniFi | `shytzel` |
|
||||||
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti Switch Pro XG 8 PoE |
|
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti Switch Pro XG 8 PoE | Zyxel GS1920 |
|
||||||
| Switch chip | Marvell 98DX3236 | Marvell 98DX4310 (+ Atheros 8227 for the 1G mgmt port) | — |
|
| Switch chip | Marvell 98DX3236 | Marvell 98DX4310 (+ Atheros 8227 for the 100M management port) | — | — |
|
||||||
| OS | RouterOS | RouterOS | UniFi |
|
| OS | RouterOS 7.25.1 | RouterOS 7.25.1 | UniFi | Zyxel firmware |
|
||||||
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 (100G, breakout-capable) + 1G mgmt | 8×10GBASE-T PoE + 2×10G SFP+ |
|
| RouterBOOT | 7.25.1 | 7.25.1 | — | — |
|
||||||
| Bridge | `main`, `vlan-filtering=yes` | `main`, `vlan-filtering=yes` | UniFi VLAN profiles |
|
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 + 100M management | 8×10GBASE-T PoE + 2×10G SFP+ | Not recorded |
|
||||||
|
| VLAN configuration | `main`, `vlan-filtering=yes` | `main`, `vlan-filtering=yes` | UniFi VLAN profiles | 802.1Q L2; management only on `core` |
|
||||||
|
|
||||||
jim and dave run a single hardware-offloaded bridge (`main`) with VLAN filtering. Access to the
|
`jim` and `dave` run a single hardware-offloaded bridge (`main`) with VLAN filtering. Access to the
|
||||||
MikroTiks is SSH as `admin` / `admin` by short hostname (see [Accessing the switches](#accessing-the-switches)).
|
MikroTiks is SSH as `admin` / `admin` by short hostname (see
|
||||||
Only jim and dave can do hardware VLAN translation (`/interface ethernet switch rule` on the Marvell
|
[Accessing the switches](#accessing-the-switches)). Only `jim` and `dave` can do hardware VLAN
|
||||||
chips); brian cannot rewrite tags, only trunk/PVID them.
|
translation (`/interface ethernet switch rule` on the Marvell chips); `brian` and `shytzel` provide
|
||||||
|
ordinary L2 VLAN switching.
|
||||||
|
|
||||||
## Physical topology
|
## Physical topology
|
||||||
|
|
||||||
The two WAN sources enter at the top: the Virgin Media modem lands on **jim** (VLAN 130), and the
|
The Virgin Media modem lands on `shytzel`, which trunks VLAN 130 to `jim`; the Digiweb ONT lands on
|
||||||
Digiweb **ONT** lands on **brian**. Both `jim` and `brian` are edge switches that uplink down into
|
`brian`. `brian` has its primary LAG to the `dave` core and an RSTP-protected secondary connection
|
||||||
the **dave** core; the home boxes hang off dave's 100G ports, with backup links up to jim. jim's
|
to `jim`; `jim` and `dave` are also joined directly. The home boxes hang off `dave`'s high-speed
|
||||||
second SFP+ port (`sfp-spare`, `sfp-sfpplus2`) feeds [fergal](#fergal-portculliss-switch), which
|
ports, with backup links to `jim`. `jim`'s second SFP+ port (`sfp-spare`, `sfp-sfpplus2`) feeds
|
||||||
[`portcullis`](../colony/portcullis.md) hangs off while it is staged at home.
|
[fergal](#fergal-portculliss-switch), which [`portcullis`](../colony/portcullis.md) hangs off while
|
||||||
|
it is staged at home.
|
||||||
|
|
||||||
```
|
[](diagrams/home-fabric.svg)
|
||||||
Virgin Media cable modem Digiweb ONT
|
|
||||||
stream WAN, VLAN 130 river WAN, management + VLAN 10
|
The [Mermaid source](diagrams/home-fabric.mmd) defines the logical topology; the rendered SVG
|
||||||
| |
|
also works in Markdown viewers without Mermaid support.
|
||||||
jim ---- 10G ---- fergal ---- portcullis brian
|
|
||||||
| 10G trunk 802.3ad LAG |
|
Solid and dotted lines distinguish normal carrying paths from backup attachments; RSTP selects
|
||||||
+--------------------+ +---------------+
|
ports for the whole fabric. The separate WAN connection to `stream` is outside its LAN bridge.
|
||||||
| |
|
The 10G trunk from `jim` to [fergal](#fergal-portculliss-switch) serves the staged colony boxes.
|
||||||
+---+----------+---+
|
|
||||||
| dave |
|
### Front-panel port maps
|
||||||
+--------+---------+
|
|
||||||
|
|
In the rack, viewed from the front, the order is **`brian`, `jim`, `dave`**, with `palace` several
|
||||||
+---------------------+---------------------+
|
units below. `stream` sits to the left of `brian`; it is drawn above it to make the connections
|
||||||
| | |
|
easier to follow. The map follows each model's physical socket order: `brian` has a single RJ45 row,
|
||||||
palace (100G) castle (100G) stream
|
`jim` has three banks with even ports above odd ports, and `dave` has four QSFP28 cages. Chassis
|
||||||
river VM NVMe-oF root second router
|
widths, spacing and the gap above `palace` are schematic; exact rack-unit positions are not known.
|
||||||
|
`palace`'s NICs are rear connections, shown only as endpoints rather than an invented front layout.
|
||||||
|
|
||||||
|
[](diagrams/switch-panels.svg)
|
||||||
|
|
||||||
|
Drawn lines and matching letter badges identify the two ends of a cable. Cable routes are
|
||||||
|
schematic, not a record of how the cables are dressed in the rack. NIC labels give socket
|
||||||
|
capacity; `stream`'s 2.5G copper ports connect to `jim` at 1G. Q3.1–Q3.4 are breakout lanes within
|
||||||
|
**one physical cage** on `dave`, not four separate front-panel sockets. Its management RJ45
|
||||||
|
is separate from the console above it; `jim`'s console is also separate from Ethernet port 1.
|
||||||
|
|
||||||
|
<!-- switch-cables -->
|
||||||
|
| Cable | Endpoint | Endpoint | Connection |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **A** | `brian` `9` | `dave` `Q3.2` | 10G; first LAG member |
|
||||||
|
| **B** | `brian` `10` | `dave` `Q3.4` | 10G; second LAG member |
|
||||||
|
| **C** | `brian` `5` | `jim` `7` | 1G; redundant fabric trunk |
|
||||||
|
| **D** | `jim` `S1` | `dave` `Q3.1` | 10G; direct fabric trunk |
|
||||||
|
| **E** | `jim` `24` | `dave` `MGMT` | 100M; dave management |
|
||||||
|
| **F** | `jim` `5` | `palace` `host-jim` | 1G; host core / netboot |
|
||||||
|
| **G** | `jim` `22` | `palace` `river-jim-phy` | 1G; river backup trunk |
|
||||||
|
| **H** | `dave` `Q1` | `palace` `uplink-dave` | 100G; host PF and guest VFs |
|
||||||
|
| **I** | `stream` `ETH0` | `jim` `21` | 1G WAN; 2.5G socket on stream |
|
||||||
|
| **J** | `stream` `ETH1` | `jim` `6` | 1G backup LAN; 2.5G socket on stream |
|
||||||
|
| **K** | `stream` `SFP+ R` | `dave` `Q3.3` | 10G preferred LAN |
|
||||||
|
<!-- /switch-cables -->
|
||||||
|
|
||||||
|
Port labels describe configured use, not current link state. `jim` ports 1–4 are configured
|
||||||
|
members of `shytzel-downlink`; only port 1 had an active link at inspection. `brian` port 3 is
|
||||||
|
reserved for testing `portcullis`, and port 4 serves its second JetKVM.
|
||||||
|
|
||||||
|
The panel layouts reference the manufacturers' [Pro XG 8 PoE guide](https://dl.ui.com/qig/usw-pro-xg-8-poe/),
|
||||||
|
[CRS326-24G-2S+RM panel](https://mikrotik.com/product/CRS326-24G-2SplusRM),
|
||||||
|
[CRS504-4XQ-IN panel](https://mikrotik.com/product/crs504_4xq_in), and
|
||||||
|
[R86S panel photograph](https://www.gowinfanless.com/news/blogs/the-r86s-revolution-low-power-2-5gbe-and-10gbe-networking).
|
||||||
|
`stream`'s connected SFP+ socket is the right-hand one, confirmed physically; the left is unused.
|
||||||
|
Its copper socket mapping is inferred from PCI order and sequential permanent MACs. The SFP+
|
||||||
|
sockets are labelled left/right in the drawing because both carry the same printed label.
|
||||||
|
Port use comes from RouterOS interface/bonding configuration, LLDP, the UniFi controller and
|
||||||
|
physical connection confirmation.
|
||||||
|
The editable inventory is [switch-ports.json](diagrams/switch-ports.json); run
|
||||||
|
`python3 docs/sites/home/diagrams/render-switch-panels.py` to regenerate the SVG and cable table.
|
||||||
|
|
||||||
Backup links to jim (normally idle):
|
|
||||||
* palace: 1G
|
|
||||||
* stream: 1G; STP selects the active link
|
|
||||||
* castle: 2.5G, normally down; no live failover
|
|
||||||
```
|
|
||||||
|
|
||||||
Notes:
|
Notes:
|
||||||
- **river** runs as a VM on the **palace** host; its uplink is dave's 100G `palace` port. jim also
|
- **river** uses `palace`'s 100G connection to `dave` and the dedicated 1G `river-jim-phy`
|
||||||
has 1G `palace`/`stream` ports, but those are secondary links and do **not** carry the WAN.
|
connection to `jim`. The separate `host-jim` connection serves host management and VM netboot.
|
||||||
|
See [palace's networking](palace.md#networking) for the handoffs and
|
||||||
|
[failover validation](router-failover.md) for tested failure scenarios.
|
||||||
|
- `jim`'s `palace-kvm` port is a JetKVM connection, not either of `palace`'s network attachments.
|
||||||
- **stream** (the second router box) is dual-homed to both jim and dave (STP picks the active path).
|
- **stream** (the second router box) is dual-homed to both jim and dave (STP picks the active path).
|
||||||
|
- The `brian`–`jim` secondary connection provides the redundant Digiweb path. VLANs 10 and 140 are
|
||||||
|
tagged on `brian-downlink` and `river`, keeping the ONT reachable without `dave`.
|
||||||
|
|
||||||
### Castle storage dependency
|
### `castle` storage dependency
|
||||||
|
|
||||||
`castle` is dual-homed without STP: its primary link is dave's 100G `castle` port (`et100g`), while
|
`castle` is dual-homed without STP: its primary link is dave's 100G `castle` port (`et100g`), while
|
||||||
the 2.5G link to jim (`et2.5g`) is normally down and provides no live failover. Its root disk is
|
the backup NIC (`et2.5g`) connects at 1G to jim and is normally down, providing no live failover.
|
||||||
NVMe-oF over `et100g` and dave, so interrupting either freezes `castle` mid-I/O. Do dave maintenance
|
Its root disk is NVMe-oF over `et100g` and dave, so interrupting either freezes `castle` mid-I/O.
|
||||||
from a box that does not depend on it, or power `castle` off cleanly first.
|
Do dave maintenance from a box that does not depend on it, or power `castle` off cleanly first.
|
||||||
|
|
||||||
## VLANs
|
## VLANs
|
||||||
|
|
||||||
| VLAN | Name | Purpose |
|
| VLAN | Name | Purpose |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| — (native) | core | Switch management, `192.168.64.0/24` (jim `.10`, dave `.11`, brian `.13`) |
|
| — (native) | core | Switch management, `192.168.64.0/24` (`jim` `.10`, `dave` `.11`, `shytzel` `.12`, `brian` `.13`) |
|
||||||
| 100 | hi | High-performance / jumbo network (MTU 9000) |
|
| 100 | hi | High-performance / jumbo network (MTU 9000) |
|
||||||
| 110 | lo | Standard LAN |
|
| 110 | lo | Standard LAN |
|
||||||
| 120 | untrusted | Guest / untrusted network |
|
| 120 | untrusted | Guest / untrusted network |
|
||||||
| 130 | wan | **stream's WAN** — Virgin Media cable modem (untagged on jim's `wan1`/`wan2`/`wan-in`) |
|
| 130 | wan | `stream`'s WAN — tagged from `shytzel`, untagged on `jim`'s `stream-wan` |
|
||||||
| 140 | wan-pon-ont | ONT management, `192.168.100.0/24` (PVID'd at the ONT edge) |
|
| 140 | wan-pon-ont | ONT management, `192.168.100.0/24` (PVID'd at the ONT edge) |
|
||||||
| 10 | pon-isp | Digiweb ISP transport — **trunked straight through** to river, PPPoE runs on it |
|
| 10 | pon-isp | Digiweb ISP transport — **trunked straight through** to river, PPPoE runs on it |
|
||||||
| 141 | wan-pon-isp | **Reserved** — the translated ISP VLAN for the future multi-ONT design |
|
| 141 | wan-pon-isp | **Reserved** — the translated ISP VLAN for the future multi-ONT design |
|
||||||
@@ -107,11 +150,12 @@ we just carry it end to end and let river run PPPoE directly on it:
|
|||||||
reach the ONT web UI at `192.168.100.1`. Doing the PVID at the ONT-facing edge keeps it clean —
|
reach the ONT web UI at `192.168.100.1`. Doing the PVID at the ONT-facing edge keeps it clean —
|
||||||
the untagged frames never share a domain with anything else.
|
the untagged frames never share a domain with anything else.
|
||||||
|
|
||||||
2. **VLAN 10 (ISP) trunked straight through, untranslated.** brian → dave → palace carry tagged
|
2. **VLAN 10 (ISP) trunked straight through, untranslated.** The primary path is
|
||||||
VLAN 10 by ordinary bridge-VLAN membership. No `/interface ethernet switch rule`, no pinning, no
|
brian → dave → palace; the RSTP-protected secondary path reaches jim before returning to the
|
||||||
asymmetric-learning issues — it's just a normal tagged VLAN. river attaches PPPoE to VLAN 10
|
router attachment. Both carry tagged VLAN 10 by ordinary bridge-VLAN membership. No
|
||||||
directly (`wan-pon-isp` netdev = VLAN `pon-isp` = 10; baby-jumbo MTU 1508 so PPP nets a clean
|
`/interface ethernet switch rule`, no pinning, no asymmetric-learning issues — it's just a
|
||||||
1500).
|
normal tagged VLAN. river attaches PPPoE to VLAN 10 directly (`wan-pon-isp` netdev = VLAN
|
||||||
|
`pon-isp` = 10; baby-jumbo MTU 1508 so PPP nets a clean 1500).
|
||||||
|
|
||||||
Net result: **river runs PPPoE single-tagged on VLAN 10 and holds a VLAN 140 address to reach the
|
Net result: **river runs PPPoE single-tagged on VLAN 10 and holds a VLAN 140 address to reach the
|
||||||
ONT.** See [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix)
|
ONT.** See [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix)
|
||||||
@@ -139,18 +183,41 @@ How each switch is set up for the Digiweb WAN path. **Confirm any change on the
|
|||||||
- The ONT port has **native/untagged network = VLAN 140** (PVID) and is a **tagged member of VLAN 10**,
|
- The ONT port has **native/untagged network = VLAN 140** (PVID) and is a **tagged member of VLAN 10**,
|
||||||
so the ONT's untagged management lands on 140 and its tagged ISP frames pass through.
|
so the ONT's untagged management lands on 140 and its tagged ISP frames pass through.
|
||||||
- The `brian-downlink` LAG up to dave trunks **tagged 140 + tagged 10** (alongside the LAN VLANs).
|
- The `brian-downlink` LAG up to dave trunks **tagged 140 + tagged 10** (alongside the LAN VLANs).
|
||||||
|
- Its secondary connection to jim carries the same relevant VLANs and participates in RSTP, so it
|
||||||
|
normally remains blocked while the primary path is healthy.
|
||||||
|
|
||||||
**dave (RouterOS)** — trunks both WAN-pon VLANs to `brian-downlink` and `palace`. The ISP VLAN 10 row:
|
**dave (RouterOS)** — trunks both WAN-pon VLANs to `brian-downlink` and `palace`. The ISP VLAN 10 row:
|
||||||
```
|
```
|
||||||
/interface bridge vlan add bridge=main vlan-ids=10 tagged=brian-downlink,palace
|
/interface bridge vlan add bridge=main vlan-ids=10 tagged=brian-downlink,palace,jim-downlink
|
||||||
```
|
```
|
||||||
VLAN 140 also spans `brian-downlink,palace` (it carries a few other members too). No switch rules —
|
VLAN 140 also spans `brian-downlink,palace,jim-downlink` (it carries a few other members too).
|
||||||
this is plain tagged bridging.
|
The direct connection to `jim` must carry both VLANs: if only `palace`'s `dave` connection fails,
|
||||||
|
the surviving RSTP path from `river` through `jim` can still traverse `dave` to reach `brian`.
|
||||||
|
These are ordinary tagged VLANs with no switch rules.
|
||||||
|
|
||||||
|
**jim (RouterOS)** — the physical `brian-downlink` is the RSTP-protected secondary connection from
|
||||||
|
`brian`. VLANs 10 and 140 have `brian-downlink`, `river` and `dave-uplink` as tagged members; no
|
||||||
|
translation rules are involved. RSTP selects one tree for the bridge, so VLAN membership must
|
||||||
|
include the surviving paths through `dave` as well as the direct backup connection to `brian`.
|
||||||
|
The `river` port is native `core` with VLANs 10, 100, 110, 120 and 140 tagged. It
|
||||||
|
faces `palace`'s `river-jim-phy`; `palace-kvm` is the separate JetKVM connection and must remain
|
||||||
|
unchanged. See [failover validation](router-failover.md) for attachment and switch-outage checks.
|
||||||
|
|
||||||
|
Other connections on `jim`:
|
||||||
|
|
||||||
|
| Interface | Connection |
|
||||||
|
|---|---|
|
||||||
|
| `ether9` | Disconnected, unbridged spare |
|
||||||
|
| `stream-wan` | Dedicated untagged VLAN 130 handoff to `stream` `wan` |
|
||||||
|
| `river` | `river-jim-phy`, redundant native-core trunk |
|
||||||
|
|
||||||
|
The modem connects to `shytzel`. `stream` has two distinct connections to `jim`: `stream-wan`
|
||||||
|
for its isolated WAN handoff and `stream` for the `lan-jim` RSTP trunk. VLAN 130 has
|
||||||
|
`shytzel-downlink` tagged and only `stream-wan` untagged.
|
||||||
|
|
||||||
|
`sfp-spare` (`sfp-sfpplus2`) stays at `pvid=1`—the switch feeding `portcullis` is reached over
|
||||||
|
VLAN 1 untagged—and is a tagged member of `hi` (100) and `lo` (110):
|
||||||
|
|
||||||
**jim (RouterOS)** — carries **none** of the Digiweb WAN path: no translation rules, and no VLAN
|
|
||||||
10/140/141 rows. jim only handles stream's VLAN-130 WAN and the LAN VLANs. `sfp-spare`
|
|
||||||
(`sfp-sfpplus2`) stays at `pvid=1` — the switch feeding `portcullis` is reached over VLAN 1
|
|
||||||
untagged — and is a **tagged** member of `hi` (100) and `lo` (110) so those reach `portcullis`:
|
|
||||||
```
|
```
|
||||||
/interface bridge vlan set [find bridge=main vlan-ids=100] tagged=...,sfp-spare
|
/interface bridge vlan set [find bridge=main vlan-ids=100] tagged=...,sfp-spare
|
||||||
/interface bridge vlan set [find bridge=main vlan-ids=110] tagged=...,sfp-spare
|
/interface bridge vlan set [find bridge=main vlan-ids=110] tagged=...,sfp-spare
|
||||||
@@ -163,8 +230,7 @@ management addresses and static default routes exist only so the boxes themselve
|
|||||||
reach out; they must **never** forward traffic or advertise themselves as routers. RouterOS defaults
|
reach out; they must **never** forward traffic or advertise themselves as routers. RouterOS defaults
|
||||||
work against this: `ip-forward` and IPv6 `forward` ship **on**, and with IPv6 forwarding on RouterOS
|
work against this: `ip-forward` and IPv6 `forward` ship **on**, and with IPv6 forwarding on RouterOS
|
||||||
also emits Router Advertisements (`ra-lifetime=30m`) on every L3 interface — so a switch silently
|
also emits Router Advertisements (`ra-lifetime=30m`) on every L3 interface — so a switch silently
|
||||||
becomes a competing IPv6 default router. This surfaced after the 7.18 → 7.23 upgrade, when clients
|
becomes a competing IPv6 default router.
|
||||||
picked up dave/jim as default routers alongside river.
|
|
||||||
|
|
||||||
The required config on each RouterOS box:
|
The required config on each RouterOS box:
|
||||||
```
|
```
|
||||||
@@ -181,7 +247,7 @@ The required config on each RouterOS box:
|
|||||||
actively clears the rogue default from clients (they otherwise cache it for up to ~30 min).
|
actively clears the rogue default from clients (they otherwise cache it for up to ~30 min).
|
||||||
|
|
||||||
**After any RouterOS upgrade, re-check `/ip settings` and `/ipv6 settings`** — an upgrade can reset
|
**After any RouterOS upgrade, re-check `/ip settings` and `/ipv6 settings`** — an upgrade can reset
|
||||||
these to the forwarding-on defaults. brian (UniFi) is not a RouterOS box and was not affected.
|
these to the forwarding-on defaults. `brian` uses UniFi and does not have these RouterOS settings.
|
||||||
|
|
||||||
## Future: multiple ONTs (per-port VLAN translation)
|
## Future: multiple ONTs (per-port VLAN translation)
|
||||||
|
|
||||||
@@ -225,6 +291,31 @@ documented in [sites/colony/fergal.md](../colony/fergal.md).
|
|||||||
|
|
||||||
## Accessing the switches
|
## Accessing the switches
|
||||||
|
|
||||||
|
### Upgrades
|
||||||
|
|
||||||
|
Save a text export and an encrypted binary backup, and download both before rebooting. Local
|
||||||
|
upgrade backups are kept outside Git under `.cache/mikrotik/<date>/`; the backup password is
|
||||||
|
stored alongside them with restricted permissions. Binary backups are intended for restoration
|
||||||
|
on the same box and RouterOS version.
|
||||||
|
|
||||||
|
Upgrade one switch at a time. Upgrade `dave` while `jim` carries the backup path, verify its
|
||||||
|
return, then upgrade `jim`. Check RouterOS packages first, then upgrade RouterBOOT and reboot
|
||||||
|
again if its running version differs. After each upgrade, verify bridge VLAN membership, RSTP
|
||||||
|
roles, hardware offload and the [forwarding/RA settings](#switches-must-not-route).
|
||||||
|
|
||||||
|
During failover, use bounded SSH calls and an explicit IPv4 management address where useful.
|
||||||
|
RouterOS parses newlines in SSH command strings separately: send a multi-statement block as one
|
||||||
|
line with semicolons so local variables and braces retain their scope.
|
||||||
|
|
||||||
|
Whole-switch outage behavior is covered by the
|
||||||
|
[validated scenarios](router-failover.md#validated-scenarios).
|
||||||
|
|
||||||
|
RouterOS uses Ed25519 host keys by default. Upgrading an older installation can change the
|
||||||
|
host-key type from RSA; verify the box's identity independently before updating the client's
|
||||||
|
`known_hosts` entry.
|
||||||
|
|
||||||
|
### SSH
|
||||||
|
|
||||||
The switches resolve by **short hostname** on the home network — the home routers serve their
|
The switches resolve by **short hostname** on the home network — the home routers serve their
|
||||||
records in the home zone
|
records in the home zone
|
||||||
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix):
|
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix):
|
||||||
@@ -241,13 +332,15 @@ sshpass -p admin ssh -o IdentityAgent=none -o PubkeyAuthentication=no \
|
|||||||
-o UserKnownHostsFile=/tmp/sw_known_hosts admin@jim
|
-o UserKnownHostsFile=/tmp/sw_known_hosts admin@jim
|
||||||
```
|
```
|
||||||
|
|
||||||
**Always confirm config changes on the switch** (print the affected menu, apply, re-verify). brian
|
**Always confirm config changes on the switch** (print the affected menu, apply, re-verify).
|
||||||
is UniFi — configured through its controller, not RouterOS CLI.
|
`brian` is configured through its UniFi controller. `shytzel` is an L2 managed switch whose
|
||||||
|
management interface is reachable only through `core`.
|
||||||
|
|
||||||
## Management IPs
|
## Management IPs
|
||||||
|
|
||||||
| | core (`192.168.64.0/24`) | hi (`192.168.68.0/22`) | lo (`192.168.72.0/21`) |
|
| | core (`192.168.64.0/24`) | hi (`192.168.68.0/22`) | lo (`192.168.72.0/21`) |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| jim | `.10` (on `main`) | `.10` | `.10` |
|
| `jim` | `.10` (on `main`) | `.10` | `.10` |
|
||||||
| dave | `.11` (on `management`, the 1G Atheros port) | `.11` | `.11` |
|
| `dave` | `.11` (on `management`, the 1G Atheros port) | `.11` | `.11` |
|
||||||
| brian | `.13` (core) | — | — |
|
| `brian` | `.13` (`core`) | — | — |
|
||||||
|
| `shytzel` | `.12` | — | — |
|
||||||
|
|||||||
@@ -57,6 +57,8 @@ in
|
|||||||
kernelParams = [ "amd_iommu=on" ];
|
kernelParams = [ "amd_iommu=on" ];
|
||||||
initrd = {
|
initrd = {
|
||||||
availableKernelModules = [ "xhci_pci" "ahci" "usb_storage" "usbhid" "sd_mod" "sr_mod" ];
|
availableKernelModules = [ "xhci_pci" "ahci" "usb_storage" "usbhid" "sd_mod" "sr_mod" ];
|
||||||
|
# LVM autoactivation can discover the RAID LVs before stage-two module loading.
|
||||||
|
kernelModules = [ "dm-raid" ];
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -94,7 +96,7 @@ in
|
|||||||
extraOptions = [ "-A /var/log/smartd/" "--interval=600" ];
|
extraOptions = [ "-A /var/log/smartd/" "--interval=600" ];
|
||||||
};
|
};
|
||||||
udev.extraRules = ''
|
udev.extraRules = ''
|
||||||
ACTION=="add", SUBSYSTEM=="net", ENV{ID_NET_DRIVER}=="mlx5_core", ENV{ID_PATH}=="pci-0000:44:00.0", ATTR{device/sriov_numvfs}="4"
|
ACTION=="add", SUBSYSTEM=="net", ENV{ID_NET_DRIVER}=="mlx5_core", ENV{ID_PATH}=="pci-0000:44:00.0", ATTR{device/sriov_numvfs}="5"
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -119,24 +121,24 @@ in
|
|||||||
|
|
||||||
network = {
|
network = {
|
||||||
links = {
|
links = {
|
||||||
"10-et1g0" = {
|
"10-river-jim-phy" = {
|
||||||
matchConfig = {
|
matchConfig = {
|
||||||
PermanentMACAddress = "e0:d5:5e:68:0c:6e";
|
PermanentMACAddress = "e0:d5:5e:68:0c:6e";
|
||||||
Driver = "igb";
|
Driver = "igb";
|
||||||
};
|
};
|
||||||
linkConfig.Name = "et1g0";
|
linkConfig.Name = "river-jim-phy";
|
||||||
};
|
};
|
||||||
"10-lan-core-phy" = {
|
"10-host-jim" = {
|
||||||
matchConfig.PermanentMACAddress = "e0:d5:5e:68:0c:70";
|
matchConfig.PermanentMACAddress = "e0:d5:5e:68:0c:70";
|
||||||
linkConfig.Name = "lan-core-phy";
|
linkConfig.Name = "host-jim";
|
||||||
};
|
};
|
||||||
"10-et100g" = {
|
"10-uplink-dave" = {
|
||||||
matchConfig = {
|
matchConfig = {
|
||||||
PermanentMACAddress = "24:8a:07:ac:59:c0";
|
PermanentMACAddress = "24:8a:07:ac:59:c0";
|
||||||
Driver = "mlx5_core";
|
Driver = "mlx5_core";
|
||||||
};
|
};
|
||||||
linkConfig = {
|
linkConfig = {
|
||||||
Name = "et100g";
|
Name = "uplink-dave";
|
||||||
MTUBytes = toString hiMTU;
|
MTUBytes = toString hiMTU;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
@@ -169,8 +171,8 @@ in
|
|||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
"50-et100g" = {
|
"50-uplink-dave" = {
|
||||||
matchConfig.Name = "et100g";
|
matchConfig.Name = "uplink-dave";
|
||||||
vlan = [ "lan-hi" ];
|
vlan = [ "lan-hi" ];
|
||||||
networkConfig = networkd.noL3;
|
networkConfig = networkd.noL3;
|
||||||
linkConfig.RequiredForOnline = "no";
|
linkConfig.RequiredForOnline = "no";
|
||||||
@@ -182,9 +184,10 @@ in
|
|||||||
LinkState=yes
|
LinkState=yes
|
||||||
MACAddress=52:54:00:cc:3e:70
|
MACAddress=52:54:00:cc:3e:70
|
||||||
|
|
||||||
# river
|
# river RDMA
|
||||||
[SR-IOV]
|
[SR-IOV]
|
||||||
VirtualFunction=1
|
VirtualFunction=1
|
||||||
|
VLANId=${toString vlans.hi}
|
||||||
LinkState=yes
|
LinkState=yes
|
||||||
MACAddress=52:54:00:8a:8a:f2
|
MACAddress=52:54:00:8a:8a:f2
|
||||||
|
|
||||||
@@ -195,12 +198,20 @@ in
|
|||||||
LinkState=yes
|
LinkState=yes
|
||||||
MACAddress=52:54:00:ac:15:a9
|
MACAddress=52:54:00:ac:15:a9
|
||||||
|
|
||||||
# sfh bridge
|
# sfh containers
|
||||||
[SR-IOV]
|
[SR-IOV]
|
||||||
VirtualFunction=3
|
VirtualFunction=3
|
||||||
VLANId=${toString vlans.hi}
|
VLANId=${toString vlans.hi}
|
||||||
LinkState=yes
|
LinkState=yes
|
||||||
MACAddress=52:54:00:90:34:95
|
MACAddress=52:54:00:90:34:95
|
||||||
|
|
||||||
|
# river LAN trunk
|
||||||
|
[SR-IOV]
|
||||||
|
VirtualFunction=4
|
||||||
|
LinkState=auto
|
||||||
|
MACAddress=52:54:00:8a:8a:f4
|
||||||
|
Trust=yes
|
||||||
|
MACSpoofCheck=no
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
"60-lan-hi" = mkMerge [
|
"60-lan-hi" = mkMerge [
|
||||||
@@ -208,8 +219,8 @@ in
|
|||||||
{ networkConfig = vlanDns "hi"; }
|
{ networkConfig = vlanDns "hi"; }
|
||||||
];
|
];
|
||||||
|
|
||||||
"50-lan-core-phy" = {
|
"50-host-jim" = {
|
||||||
matchConfig.Name = "lan-core-phy";
|
matchConfig.Name = "host-jim";
|
||||||
networkConfig = {
|
networkConfig = {
|
||||||
Bridge = "lan-core";
|
Bridge = "lan-core";
|
||||||
} // networkd.noL3;
|
} // networkd.noL3;
|
||||||
|
|||||||
@@ -10,6 +10,7 @@
|
|||||||
inherit (lib) mkMerge;
|
inherit (lib) mkMerge;
|
||||||
inherit (lib.my) vm;
|
inherit (lib.my) vm;
|
||||||
inherit (lib.my.c) networkd;
|
inherit (lib.my.c) networkd;
|
||||||
|
inherit (lib.my.c.home) hiMTU;
|
||||||
|
|
||||||
installerDisk = {
|
installerDisk = {
|
||||||
name = "installer";
|
name = "installer";
|
||||||
@@ -28,10 +29,11 @@
|
|||||||
{
|
{
|
||||||
systemd.network = {
|
systemd.network = {
|
||||||
netdevs = {
|
netdevs = {
|
||||||
"25-vm-et1g0" = {
|
"25-river-jim-tap" = {
|
||||||
netdevConfig = {
|
netdevConfig = {
|
||||||
Name = "vm-et1g0";
|
Name = "river-jim-tap";
|
||||||
Kind = "macvtap";
|
Kind = "macvtap";
|
||||||
|
MTUBytes = toString hiMTU;
|
||||||
};
|
};
|
||||||
# TODO: Upstream this missing section
|
# TODO: Upstream this missing section
|
||||||
extraConfig = ''
|
extraConfig = ''
|
||||||
@@ -41,16 +43,24 @@
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
networks = {
|
networks = {
|
||||||
"75-et1g0" = {
|
"75-river-jim-phy" = {
|
||||||
matchConfig.Name = "et1g0";
|
matchConfig.Name = "river-jim-phy";
|
||||||
linkConfig.RequiredForOnline = "no";
|
linkConfig = {
|
||||||
|
RequiredForOnline = "no";
|
||||||
|
MTUBytes = toString hiMTU;
|
||||||
|
};
|
||||||
networkConfig = {
|
networkConfig = {
|
||||||
MACVTAP = "vm-et1g0";
|
MACVTAP = "river-jim-tap";
|
||||||
} // networkd.noL3;
|
} // networkd.noL3;
|
||||||
};
|
};
|
||||||
"75-vm-et1g0" = {
|
"75-river-jim-tap" = {
|
||||||
matchConfig.Name = "vm-et1g0";
|
matchConfig.Name = "river-jim-tap";
|
||||||
linkConfig.RequiredForOnline = "no";
|
linkConfig = {
|
||||||
|
RequiredForOnline = "no";
|
||||||
|
MTUBytes = toString hiMTU;
|
||||||
|
# Guest receive-filter changes are not synchronized to the host macvtap.
|
||||||
|
AllMulticast = true;
|
||||||
|
};
|
||||||
networkConfig = networkd.noL3;
|
networkConfig = networkd.noL3;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
@@ -79,7 +89,7 @@
|
|||||||
|
|
||||||
"vm@river" =
|
"vm@river" =
|
||||||
let
|
let
|
||||||
vtapUnit = "sys-subsystem-net-devices-vm\\x2det1g0.device";
|
vtapUnit = "sys-subsystem-net-devices-river\\x2djim\\x2dtap.device";
|
||||||
in
|
in
|
||||||
mkMerge [
|
mkMerge [
|
||||||
(awaitVM "cellar")
|
(awaitVM "cellar")
|
||||||
@@ -109,7 +119,7 @@
|
|||||||
(vm.disk "cellar" "persist")
|
(vm.disk "cellar" "persist")
|
||||||
];
|
];
|
||||||
hostDevices = {
|
hostDevices = {
|
||||||
et100g0vf0 = {
|
cellar-hi = {
|
||||||
index = 0;
|
index = 0;
|
||||||
hostBDF = "44:00.1";
|
hostBDF = "44:00.1";
|
||||||
};
|
};
|
||||||
@@ -142,8 +152,8 @@
|
|||||||
memory = 4096;
|
memory = 4096;
|
||||||
cleanShutdown.timeout = 60;
|
cleanShutdown.timeout = 60;
|
||||||
networks = {
|
networks = {
|
||||||
et1g0 = {
|
lan-jim = {
|
||||||
ifname = "vm-et1g0";
|
ifname = "river-jim-tap";
|
||||||
bridge = null;
|
bridge = null;
|
||||||
tapFD = 100;
|
tapFD = 100;
|
||||||
# Real hardware MAC
|
# Real hardware MAC
|
||||||
@@ -156,10 +166,14 @@
|
|||||||
(mkMerge [ (vm.disk "river" "esp") { frontendOpts.bootindex = 0; } ])
|
(mkMerge [ (vm.disk "river" "esp") { frontendOpts.bootindex = 0; } ])
|
||||||
];
|
];
|
||||||
hostDevices = {
|
hostDevices = {
|
||||||
et100g0vf1 = {
|
river-rdma = {
|
||||||
index = 0;
|
index = 0;
|
||||||
hostBDF = "44:00.2";
|
hostBDF = "44:00.2";
|
||||||
};
|
};
|
||||||
|
river-dave = {
|
||||||
|
index = 1;
|
||||||
|
hostBDF = "44:00.5";
|
||||||
|
};
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -175,13 +189,14 @@
|
|||||||
networks = {
|
networks = {
|
||||||
netboot = {
|
netboot = {
|
||||||
bridge = "lan-lo";
|
bridge = "lan-lo";
|
||||||
|
ifname = "sfh-lo-tap";
|
||||||
waitOnline = "carrier";
|
waitOnline = "carrier";
|
||||||
mac = "52:54:00:a5:7e:93";
|
mac = "52:54:00:a5:7e:93";
|
||||||
extraOptions.bootindex = 1;
|
extraOptions.bootindex = 1;
|
||||||
};
|
};
|
||||||
core = {
|
core = {
|
||||||
bridge = "lan-core";
|
bridge = "lan-core";
|
||||||
ifname = "vm-sfh-core";
|
ifname = "sfh-core-tap";
|
||||||
waitOnline = "carrier";
|
waitOnline = "carrier";
|
||||||
mac = "52:54:00:72:67:51";
|
mac = "52:54:00:72:67:51";
|
||||||
};
|
};
|
||||||
@@ -190,11 +205,11 @@
|
|||||||
(vm.lvmDisk' "hdds" "frigate")
|
(vm.lvmDisk' "hdds" "frigate")
|
||||||
];
|
];
|
||||||
hostDevices = {
|
hostDevices = {
|
||||||
et100g0vf2 = {
|
sfh-hi = {
|
||||||
index = 0;
|
index = 0;
|
||||||
hostBDF = "44:00.3";
|
hostBDF = "44:00.3";
|
||||||
};
|
};
|
||||||
et100g0vf3 = {
|
sfh-ctrs = {
|
||||||
index = 1;
|
index = 1;
|
||||||
hostBDF = "44:00.4";
|
hostBDF = "44:00.4";
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1,3 +1,8 @@
|
|||||||
|
{ lib, ... }:
|
||||||
|
let
|
||||||
|
inherit (lib.my) net;
|
||||||
|
inherit (lib.my.c.home) domain prefixes hiMTU;
|
||||||
|
in
|
||||||
{
|
{
|
||||||
imports = [ (import ../../routing-common 0) ];
|
imports = [ (import ../../routing-common 0) ];
|
||||||
|
|
||||||
@@ -6,6 +11,17 @@
|
|||||||
nixpkgs = "mine";
|
nixpkgs = "mine";
|
||||||
home-manager = "mine";
|
home-manager = "mine";
|
||||||
|
|
||||||
|
assignments.rdma = {
|
||||||
|
name = "river-rdma";
|
||||||
|
inherit domain;
|
||||||
|
mtu = hiMTU;
|
||||||
|
ipv4 = {
|
||||||
|
address = net.cidr.host 3 prefixes.hi.v4;
|
||||||
|
mask = 32;
|
||||||
|
gateway = null;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
configuration = { lib, modulesPath, pkgs, config, assignments, allAssignments, ... }:
|
configuration = { lib, modulesPath, pkgs, config, assignments, allAssignments, ... }:
|
||||||
let
|
let
|
||||||
inherit (builtins) elemAt;
|
inherit (builtins) elemAt;
|
||||||
@@ -14,11 +30,11 @@
|
|||||||
inherit (lib.my.c) networkd;
|
inherit (lib.my.c) networkd;
|
||||||
inherit (lib.my.c.home) vlans domain prefixes roceBootModules routersPubV4;
|
inherit (lib.my.c.home) vlans domain prefixes roceBootModules routersPubV4;
|
||||||
|
|
||||||
# river reaches the ONT over its 100G `lan` uplink to the dave switch (which downlinks to
|
# river reaches the ONT through its RSTP `lan` bridge, normally over `lan-dave`. Digiweb
|
||||||
# brian, where the ONT lands). Digiweb delivers the ISP VLAN (pon-isp, 10) single-tagged at
|
# delivers the ISP VLAN (pon-isp, 10) single-tagged at the ONT alongside the ONT's untagged
|
||||||
# the ONT alongside the ONT's untagged management traffic. With a single ONT there's no VLAN
|
# management traffic. With a single ONT there's no VLAN collision, so the switches simply
|
||||||
# collision, so the switches simply trunk the ISP's VLAN 10 straight through to river (PPPoE
|
# trunk the ISP's VLAN 10 straight through to river (PPPoE runs directly on it) and PVID the
|
||||||
# runs directly on it) and PVID the ONT's untagged management port onto wan-pon-ont (140).
|
# ONT's untagged management port onto wan-pon-ont (140).
|
||||||
# river takes .100 in the ONT's /24 (matching stream's modem-mgmt .100) to reach its web
|
# river takes .100 in the ONT's /24 (matching stream's modem-mgmt .100) to reach its web
|
||||||
# UI at 192.168.100.1. (See docs/sites/home/switches.md for the switch side and the multi-ONT plan.)
|
# UI at 192.168.100.1. (See docs/sites/home/switches.md for the switch side and the multi-ONT plan.)
|
||||||
ontV4 = net.cidr.host 100 prefixes.ont.v4;
|
ontV4 = net.cidr.host 100 prefixes.ont.v4;
|
||||||
@@ -35,24 +51,36 @@
|
|||||||
boot = {
|
boot = {
|
||||||
kernelModules = [ "kvm-amd" ];
|
kernelModules = [ "kvm-amd" ];
|
||||||
kernelParams = [ "console=ttyS0,115200n8" ];
|
kernelParams = [ "console=ttyS0,115200n8" ];
|
||||||
|
kernel.sysctl = {
|
||||||
|
# Keep storage and router addresses on their own interfaces on the shared L2 network.
|
||||||
|
"net.ipv4.conf.rdma-hi.arp_ignore" = 1;
|
||||||
|
"net.ipv4.conf.lan-hi.arp_ignore" = 1;
|
||||||
|
"net.ipv4.conf.rdma-hi.arp_announce" = 2;
|
||||||
|
"net.ipv4.conf.lan-hi.arp_announce" = 2;
|
||||||
|
};
|
||||||
initrd = {
|
initrd = {
|
||||||
availableKernelModules = [
|
availableKernelModules = [
|
||||||
"virtio_pci" "ahci" "sr_mod" "virtio_blk"
|
"virtio_pci" "ahci" "sr_mod" "virtio_blk"
|
||||||
"8021q"
|
|
||||||
] ++ roceBootModules;
|
] ++ roceBootModules;
|
||||||
kernelModules = [ "dm-snapshot" ];
|
kernelModules = [ "dm-snapshot" ];
|
||||||
systemd = {
|
systemd = {
|
||||||
network = {
|
network = {
|
||||||
# Don't need to put the link config here, they're copied from main config
|
# NixOS copies the main system's .link units into the systemd initrd, so the
|
||||||
netdevs = mkVLAN "lan-hi" vlans.hi;
|
# rdma-hi rename is already available here. It does not copy .network units.
|
||||||
networks = {
|
networks = {
|
||||||
"20-lan" = {
|
"20-rdma-hi" = mkMerge [
|
||||||
matchConfig.Name = "lan";
|
(networkdAssignment "rdma-hi" assignments.rdma)
|
||||||
vlan = [ "lan-hi" ];
|
{
|
||||||
linkConfig.RequiredForOnline = "no";
|
networkConfig.IPv6AcceptRA = mkForce false;
|
||||||
networkConfig = networkd.noL3;
|
routes = [
|
||||||
};
|
{
|
||||||
"30-lan-hi" = networkdAssignment "lan-hi" assignments.hi;
|
Destination = "${allAssignments.cellar.hi.ipv4.address}/32";
|
||||||
|
PreferredSource = assignments.rdma.ipv4.address;
|
||||||
|
Scope = "link";
|
||||||
|
}
|
||||||
|
];
|
||||||
|
}
|
||||||
|
];
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
@@ -86,9 +114,6 @@
|
|||||||
};
|
};
|
||||||
fstrim.enable = true;
|
fstrim.enable = true;
|
||||||
|
|
||||||
# TODO: re-enable once scheduling is tested
|
|
||||||
networkd-dispatcher.enable = mkForce false;
|
|
||||||
|
|
||||||
pppd = {
|
pppd = {
|
||||||
enable = true;
|
enable = true;
|
||||||
peers.digiweb = {
|
peers.digiweb = {
|
||||||
@@ -118,8 +143,8 @@
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
# PPPoE WAN (Digiweb): pppd owns the `wan` interface on top of wan-pon-isp (the switch's
|
# PPPoE WAN (Digiweb): pppd owns the `wan` interface on top of wan-pon-isp (the ISP's
|
||||||
# swap of the ISP's VLAN 10), and its ip-up/ip-down hooks toggle the shared
|
# VLAN 10, trunked untranslated), and its ip-up/ip-down hooks toggle the shared
|
||||||
# wan-online.target. Nothing else Wants the target, so it stays inactive until the link
|
# wan-online.target. Nothing else Wants the target, so it stays inactive until the link
|
||||||
# is actually up.
|
# is actually up.
|
||||||
systemd.targets.wan-online.unitConfig.DefaultDependencies = false;
|
systemd.targets.wan-online.unitConfig.DefaultDependencies = false;
|
||||||
@@ -159,10 +184,18 @@
|
|||||||
(mkVLAN "wan-pon-ont" vlans.wan-pon-ont)
|
(mkVLAN "wan-pon-ont" vlans.wan-pon-ont)
|
||||||
# The ISP VLAN is trunked through untranslated, so this is the raw pon-isp (10)
|
# The ISP VLAN is trunked through untranslated, so this is the raw pon-isp (10)
|
||||||
(mkVLAN "wan-pon-isp" vlans.pon-isp)
|
(mkVLAN "wan-pon-isp" vlans.pon-isp)
|
||||||
|
{
|
||||||
|
# The adapter's local switching must recognize the router's destination MAC.
|
||||||
|
"25-lan".netdevConfig = {
|
||||||
|
Name = "lan";
|
||||||
|
Kind = "bridge";
|
||||||
|
MACAddress = "52:54:00:8a:8a:f4";
|
||||||
|
};
|
||||||
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
links = {
|
links = {
|
||||||
"10-wan-old" = {
|
"10-lan-jim" = {
|
||||||
matchConfig = {
|
matchConfig = {
|
||||||
# Matching against MAC address seems to break VLAN interfaces
|
# Matching against MAC address seems to break VLAN interfaces
|
||||||
# (since they share the same MAC address)
|
# (since they share the same MAC address)
|
||||||
@@ -170,34 +203,61 @@
|
|||||||
PermanentMACAddress = "e0:d5:5e:68:0c:6e";
|
PermanentMACAddress = "e0:d5:5e:68:0c:6e";
|
||||||
};
|
};
|
||||||
linkConfig = {
|
linkConfig = {
|
||||||
Name = "wan-old";
|
Name = "lan-jim";
|
||||||
RxBufferSize = 4096;
|
RxBufferSize = 4096;
|
||||||
TxBufferSize = 4096;
|
TxBufferSize = 4096;
|
||||||
|
MTUBytes = toString lib.my.c.home.hiMTU;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
"10-lan" = {
|
"10-rdma-hi" = {
|
||||||
matchConfig = {
|
matchConfig = {
|
||||||
Driver = "mlx5_core";
|
Driver = "mlx5_core";
|
||||||
PermanentMACAddress = "52:54:00:8a:8a:f2";
|
PermanentMACAddress = "52:54:00:8a:8a:f2";
|
||||||
};
|
};
|
||||||
linkConfig = {
|
linkConfig = {
|
||||||
Name = "lan";
|
Name = "rdma-hi";
|
||||||
|
MTUBytes = toString lib.my.c.home.hiMTU;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
"10-lan-dave" = {
|
||||||
|
matchConfig = {
|
||||||
|
Driver = "mlx5_core";
|
||||||
|
PermanentMACAddress = "52:54:00:8a:8a:f4";
|
||||||
|
};
|
||||||
|
linkConfig = {
|
||||||
|
Name = "lan-dave";
|
||||||
MTUBytes = toString lib.my.c.home.hiMTU;
|
MTUBytes = toString lib.my.c.home.hiMTU;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
networks = {
|
networks = {
|
||||||
|
"30-rdma-hi" = mkMerge [
|
||||||
|
(networkdAssignment "rdma-hi" assignments.rdma)
|
||||||
|
{
|
||||||
|
networkConfig = {
|
||||||
|
IPv6AcceptRA = mkForce false;
|
||||||
|
# Keep the address and route established in the initrd: dropping either while
|
||||||
|
# stage 2 networkd takes over would interrupt the mounted NVMe-oF root.
|
||||||
|
KeepConfiguration = "static";
|
||||||
|
};
|
||||||
|
routes = [
|
||||||
|
{
|
||||||
|
Destination = "${allAssignments.cellar.hi.ipv4.address}/32";
|
||||||
|
PreferredSource = assignments.rdma.ipv4.address;
|
||||||
|
Scope = "link";
|
||||||
|
}
|
||||||
|
];
|
||||||
|
}
|
||||||
|
];
|
||||||
"55-lan" = {
|
"55-lan" = {
|
||||||
# both WAN VLANs arrive single-tagged on the 100G uplink to dave: wan-pon-ont (140,
|
# Both WAN VLANs arrive single-tagged on the RSTP bridge: wan-pon-ont (140, the ONT's
|
||||||
# the ONT's management, PVID-tagged at the brian edge) and the ISP's VLAN 10, trunked
|
# management, PVID-tagged at the brian edge) and the ISP's VLAN 10, trunked straight
|
||||||
# straight through
|
# through.
|
||||||
vlan = [ "wan-pon-ont" "wan-pon-isp" ];
|
vlan = [ "wan-pon-ont" "wan-pon-isp" ];
|
||||||
};
|
};
|
||||||
# So we don't drop the IP we use to connect to NVMe-oF!
|
|
||||||
"60-lan-hi".networkConfig.KeepConfiguration = "static";
|
|
||||||
|
|
||||||
# ONT management: the brian edge PVIDs the ONT's untagged port onto wan-pon-ont, so give
|
# ONT management: the brian edge PVIDs the ONT's untagged port onto wan-pon-ont, so give
|
||||||
# ourselves an address in its /24 to reach the ONT's web UI at 192.168.100.1.
|
# ourselves an address in its /24 to reach the ONT's web UI at 192.168.100.1.
|
||||||
"70-wan-pon-ont" = {
|
"70-wan-pon-ont" = {
|
||||||
|
|||||||
@@ -105,29 +105,29 @@ in
|
|||||||
MTUBytes = toString lib.my.c.home.hiMTU;
|
MTUBytes = toString lib.my.c.home.hiMTU;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
"10-lan-hi-ctrs" = {
|
"10-ctrs-hi" = {
|
||||||
matchConfig = {
|
matchConfig = {
|
||||||
Driver = "mlx5_core";
|
Driver = "mlx5_core";
|
||||||
PermanentMACAddress = "52:54:00:90:34:95";
|
PermanentMACAddress = "52:54:00:90:34:95";
|
||||||
};
|
};
|
||||||
linkConfig = {
|
linkConfig = {
|
||||||
Name = "lan-hi-ctrs";
|
Name = "ctrs-hi";
|
||||||
MTUBytes = toString lib.my.c.home.hiMTU;
|
MTUBytes = toString lib.my.c.home.hiMTU;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
"10-lan-core-ctrs" = {
|
"10-ctrs-core" = {
|
||||||
matchConfig = {
|
matchConfig = {
|
||||||
Driver = "virtio_net";
|
Driver = "virtio_net";
|
||||||
PermanentMACAddress = "52:54:00:72:67:51";
|
PermanentMACAddress = "52:54:00:72:67:51";
|
||||||
};
|
};
|
||||||
linkConfig.Name = "lan-core-ctrs";
|
linkConfig.Name = "ctrs-core";
|
||||||
};
|
};
|
||||||
"10-lan-lo-ctrs" = {
|
"10-ctrs-lo" = {
|
||||||
matchConfig = {
|
matchConfig = {
|
||||||
Driver = "virtio_net";
|
Driver = "virtio_net";
|
||||||
PermanentMACAddress = "52:54:00:a5:7e:93";
|
PermanentMACAddress = "52:54:00:a5:7e:93";
|
||||||
};
|
};
|
||||||
linkConfig.Name = "lan-lo-ctrs";
|
linkConfig.Name = "ctrs-lo";
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -139,18 +139,18 @@ in
|
|||||||
networkConfig = vlanDns "hi" // { KeepConfiguration = "static"; };
|
networkConfig = vlanDns "hi" // { KeepConfiguration = "static"; };
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
"30-lan-hi-ctrs" = {
|
"30-ctrs-hi" = {
|
||||||
matchConfig.Name = "lan-hi-ctrs";
|
matchConfig.Name = "ctrs-hi";
|
||||||
linkConfig.RequiredForOnline = "no";
|
linkConfig.RequiredForOnline = "no";
|
||||||
networkConfig = networkd.noL3;
|
networkConfig = networkd.noL3;
|
||||||
};
|
};
|
||||||
"30-lan-core-ctrs" = {
|
"30-ctrs-core" = {
|
||||||
matchConfig.Name = "lan-core-ctrs";
|
matchConfig.Name = "ctrs-core";
|
||||||
linkConfig.RequiredForOnline = "no";
|
linkConfig.RequiredForOnline = "no";
|
||||||
networkConfig = networkd.noL3;
|
networkConfig = networkd.noL3;
|
||||||
};
|
};
|
||||||
"30-lan-lo-ctrs" = {
|
"30-ctrs-lo" = {
|
||||||
matchConfig.Name = "lan-lo-ctrs";
|
matchConfig.Name = "ctrs-lo";
|
||||||
linkConfig.RequiredForOnline = "no";
|
linkConfig.RequiredForOnline = "no";
|
||||||
networkConfig = networkd.noL3;
|
networkConfig = networkd.noL3;
|
||||||
};
|
};
|
||||||
@@ -160,12 +160,12 @@ in
|
|||||||
systemd.nspawn = {
|
systemd.nspawn = {
|
||||||
hass = {
|
hass = {
|
||||||
networkConfig = {
|
networkConfig = {
|
||||||
MACVLAN = mkForce "lan-hi-ctrs:host0 lan-lo-ctrs:lan-lo";
|
MACVLAN = mkForce "ctrs-hi:host0 ctrs-lo:lan-lo";
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
unifi = {
|
unifi = {
|
||||||
networkConfig = {
|
networkConfig = {
|
||||||
MACVLAN = mkForce "lan-hi-ctrs:host0 lan-core-ctrs:lan-core";
|
MACVLAN = mkForce "ctrs-hi:host0 ctrs-core:lan-core";
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
@@ -217,7 +217,7 @@ in
|
|||||||
mkMerge [
|
mkMerge [
|
||||||
instances
|
instances
|
||||||
(mapAttrs (n: i: {
|
(mapAttrs (n: i: {
|
||||||
networking.macVLAN = "lan-hi-ctrs";
|
networking.macVLAN = "ctrs-hi";
|
||||||
}) instances)
|
}) instances)
|
||||||
];
|
];
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ let
|
|||||||
inherit (builtins) elemAt;
|
inherit (builtins) elemAt;
|
||||||
inherit (lib.my) net mkVLAN;
|
inherit (lib.my) net mkVLAN;
|
||||||
inherit (lib.my.c) pubDomain;
|
inherit (lib.my.c) pubDomain;
|
||||||
inherit (lib.my.c.home) domain vlans prefixes vips routers routersPubV4;
|
inherit (lib.my.c.home) domain vlans prefixes vips routers routersPubV4 hiMTU;
|
||||||
|
|
||||||
name = elemAt routers index;
|
name = elemAt routers index;
|
||||||
otherIndex = 1 - index;
|
otherIndex = 1 - index;
|
||||||
@@ -106,7 +106,7 @@ in
|
|||||||
./dns.nix
|
./dns.nix
|
||||||
./radvd.nix
|
./radvd.nix
|
||||||
./kea.nix
|
./kea.nix
|
||||||
];
|
] ++ [ ./mstpd.nix ];
|
||||||
|
|
||||||
# Per-box WAN-management specifics: the Virgin Media modem on stream lives on the `wan`
|
# Per-box WAN-management specifics: the Virgin Media modem on stream lives on the `wan`
|
||||||
# interface itself, whereas river's ONT sits on its own interface. Declared as options the
|
# interface itself, whereas river's ONT sits on its own interface. Declared as options the
|
||||||
@@ -203,6 +203,14 @@ in
|
|||||||
|
|
||||||
netdevs = mkMerge [
|
netdevs = mkMerge [
|
||||||
{
|
{
|
||||||
|
"25-lan" = {
|
||||||
|
netdevConfig = {
|
||||||
|
Name = "lan";
|
||||||
|
Kind = "bridge";
|
||||||
|
MTUBytes = toString hiMTU;
|
||||||
|
};
|
||||||
|
bridgeConfig.STP = true;
|
||||||
|
};
|
||||||
"25-wan-ifb".netdevConfig = {
|
"25-wan-ifb".netdevConfig = {
|
||||||
Name = "wan-ifb";
|
Name = "wan-ifb";
|
||||||
Kind = "ifb";
|
Kind = "ifb";
|
||||||
@@ -238,6 +246,16 @@ in
|
|||||||
in
|
in
|
||||||
mkMerge [
|
mkMerge [
|
||||||
{
|
{
|
||||||
|
"50-lan-dave" = {
|
||||||
|
matchConfig.Name = "lan-dave";
|
||||||
|
networkConfig.Bridge = "lan";
|
||||||
|
bridgeConfig.Cost = 10;
|
||||||
|
};
|
||||||
|
"50-lan-jim" = {
|
||||||
|
matchConfig.Name = "lan-jim";
|
||||||
|
networkConfig.Bridge = "lan";
|
||||||
|
bridgeConfig.Cost = 100;
|
||||||
|
};
|
||||||
"55-lan" = {
|
"55-lan" = {
|
||||||
matchConfig.Name = "lan";
|
matchConfig.Name = "lan";
|
||||||
vlan = [ "lan-hi" "lan-lo" "lan-untrusted" "wan-tunnel" ];
|
vlan = [ "lan-hi" "lan-lo" "lan-untrusted" "wan-tunnel" ];
|
||||||
|
|||||||
@@ -26,11 +26,19 @@ in
|
|||||||
};
|
};
|
||||||
|
|
||||||
systemd.services = {
|
systemd.services = {
|
||||||
kea-dhcp4-server.serviceConfig = {
|
kea-dhcp4-server = {
|
||||||
# Sometimes interfaces might not be ready in time and Kea doesn't like that
|
unitConfig.StartLimitIntervalSec = 0;
|
||||||
|
preStart = ''
|
||||||
|
${config.systemd.package}/lib/systemd/systemd-networkd-wait-online \
|
||||||
|
--interface=lan-hi:routable --interface=lan-lo:routable \
|
||||||
|
--interface=lan-untrusted:routable --ipv4 --timeout=60
|
||||||
|
'';
|
||||||
|
serviceConfig = {
|
||||||
Restart = "on-failure";
|
Restart = "on-failure";
|
||||||
|
RestartSec = 5;
|
||||||
DynamicUser = mkForce false;
|
DynamicUser = mkForce false;
|
||||||
};
|
};
|
||||||
|
};
|
||||||
kea-dhcp-ddns-server.serviceConfig.DynamicUser = mkForce false;
|
kea-dhcp-ddns-server.serviceConfig.DynamicUser = mkForce false;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -6,6 +6,34 @@ let
|
|||||||
# Delete postInstall since it nukes the bridge-stp script we need
|
# Delete postInstall since it nukes the bridge-stp script we need
|
||||||
postInstall = "";
|
postInstall = "";
|
||||||
};
|
};
|
||||||
|
configureRSTP = pkgs.writeShellScript "configure-router-rstp" ''
|
||||||
|
set -eu
|
||||||
|
[ "$1" = lan ] || exit 0
|
||||||
|
event_port="''${2:-}"
|
||||||
|
${mstpd}/bin/mstpctl addbridge lan
|
||||||
|
${mstpd}/bin/mstpctl setforcevers lan rstp
|
||||||
|
for port in lan-dave lan-jim; do
|
||||||
|
if [ "$event_port" = "$port" ] || [ -e "/sys/class/net/lan/brif/$port" ]; then
|
||||||
|
case "$port" in
|
||||||
|
lan-dave) cost=10 ;;
|
||||||
|
lan-jim) cost=100 ;;
|
||||||
|
esac
|
||||||
|
# networkd can report configuration before mstpd has registered the port.
|
||||||
|
configured=false
|
||||||
|
for attempt in {1..50}; do
|
||||||
|
if ${mstpd}/bin/mstpctl setportpathcost lan "$port" "$cost"; then
|
||||||
|
configured=true
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
${pkgs.coreutils}/bin/sleep 0.2
|
||||||
|
done
|
||||||
|
if [ "$configured" = false ]; then
|
||||||
|
echo "Failed to configure RSTP cost for $port" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
'';
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
environment = {
|
environment = {
|
||||||
@@ -17,6 +45,7 @@ in
|
|||||||
"bridge-stp.conf".text = ''
|
"bridge-stp.conf".text = ''
|
||||||
MANAGE_MSTPD=n
|
MANAGE_MSTPD=n
|
||||||
MSTP_BRIDGES=lan
|
MSTP_BRIDGES=lan
|
||||||
|
config_cmd=${configureRSTP}
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
@@ -27,8 +56,9 @@ in
|
|||||||
onState = [ "configured" ];
|
onState = [ "configured" ];
|
||||||
script = ''
|
script = ''
|
||||||
#!${pkgs.runtimeShell}
|
#!${pkgs.runtimeShell}
|
||||||
if [ "$IFACE" = "lan" ]; then
|
# A queued event can retain the interface's name from before a rename.
|
||||||
${mstpd}/sbin/mstpctl setforcevers "$IFACE" rstp
|
if [ -d /sys/class/net/lan/bridge ]; then
|
||||||
|
${configureRSTP} lan "$IFACE"
|
||||||
fi
|
fi
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -18,8 +18,6 @@
|
|||||||
modemV4 = net.cidr.host 100 prefixes.modem.v4;
|
modemV4 = net.cidr.host 100 prefixes.modem.v4;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
imports = [ ./routing-common/mstpd.nix ];
|
|
||||||
|
|
||||||
config = {
|
config = {
|
||||||
boot = {
|
boot = {
|
||||||
kernelModules = [ "kvm-intel" ];
|
kernelModules = [ "kvm-intel" ];
|
||||||
@@ -109,15 +107,6 @@
|
|||||||
};
|
};
|
||||||
|
|
||||||
systemd.network = {
|
systemd.network = {
|
||||||
netdevs = {
|
|
||||||
"25-lan" = {
|
|
||||||
netdevConfig = {
|
|
||||||
Name = "lan";
|
|
||||||
Kind = "bridge";
|
|
||||||
};
|
|
||||||
bridgeConfig.STP = true;
|
|
||||||
};
|
|
||||||
};
|
|
||||||
links = {
|
links = {
|
||||||
"10-wan" = {
|
"10-wan" = {
|
||||||
matchConfig = {
|
matchConfig = {
|
||||||
@@ -169,17 +158,6 @@
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
networks = {
|
networks = {
|
||||||
"50-lan-jim" = {
|
|
||||||
matchConfig.Name = "lan-jim";
|
|
||||||
networkConfig.Bridge = "lan";
|
|
||||||
bridgeConfig.Cost = 100;
|
|
||||||
};
|
|
||||||
"50-lan-dave" = {
|
|
||||||
matchConfig.Name = "lan-dave";
|
|
||||||
networkConfig.Bridge = "lan";
|
|
||||||
bridgeConfig.Cost = 10;
|
|
||||||
};
|
|
||||||
|
|
||||||
"50-wan-ifb" = {
|
"50-wan-ifb" = {
|
||||||
matchConfig.Name = "wan-ifb";
|
matchConfig.Name = "wan-ifb";
|
||||||
networkConfig = networkd.noL3;
|
networkConfig = networkd.noL3;
|
||||||
|
|||||||
Reference in New Issue
Block a user