nixos/home: Separate router storage and uplinks
Give `river` a dedicated RDMA VF and two RSTP trunk attachments. Keep client services on the router VIPs and carry Digiweb VLANs over both switch paths. Set VF trust, multicast reception, bridge MAC and ARP behavior for local switching. Wait for Kea interface readiness and apply RSTP costs through `mstpd` despite port-registration and rename races. Load `palace`'s RAID module in the initrd. Record RouterOS and RouterBOOT upgrades and validate port, carrier and switch outages. Document the deployed architecture, interface handoffs and recovery procedures. Add physical socket maps, NIC capacities and cable connections alongside the logical topology and failover checks.
This commit is contained in:
@@ -33,10 +33,12 @@ clients and the two WAN paths. VLAN IDs, prefixes, MTUs and router addressing li
|
||||
|
||||
## Switch fabric
|
||||
|
||||
The boxes hang off three hand-configured switches — `jim` and `dave` (MikroTik, RouterOS) and
|
||||
`brian` (Ubiquiti, UniFi) — which are **not** managed by this flake. The physical topology, VLAN
|
||||
map, the Digiweb WAN path (trunked VLAN 10 + PVID 140 at the ONT edge), and the multi-ONT plan are
|
||||
documented in [switches.md](switches.md).
|
||||
The boxes hang off four hand-configured switches: `jim` and `dave` (MikroTik, RouterOS), `brian`
|
||||
(Ubiquiti, UniFi), and the L2-managed Zyxel switch `shytzel`. They are **not** managed by this
|
||||
flake. The physical topology, VLAN map, the Digiweb WAN path (trunked VLAN 10 + PVID 140 at the ONT
|
||||
edge), and the multi-ONT plan are documented in [switches.md](switches.md). Router attachments
|
||||
are described in [Router HA](../../networking.md#router-ha); tested failure scenarios and checks
|
||||
for maintenance are documented in [router-failover.md](router-failover.md).
|
||||
|
||||
## Wireless APs
|
||||
|
||||
|
||||
@@ -13,7 +13,8 @@ Only the DNS records live in the flake
|
||||
| | vibe | wave |
|
||||
|---|---|---|
|
||||
| Model | MikroTik **cAP ax** (`cAPGi-5HaxD2HaxD`) | Cudy **AX3000** (OpenWrt id `cudy,ap3000-v1`) |
|
||||
| OS | RouterOS 7.x | OpenWrt 25.12.x (MT7981B / Filogic 820) |
|
||||
| OS | RouterOS 7.25.1 | OpenWrt 25.12.x (MT7981B / Filogic 820) |
|
||||
| RouterBOOT | 7.25.1 | — |
|
||||
| Radio | 2×2 both bands | 2×2 both bands (2 spatial streams) |
|
||||
| Uplink | trunk (multi-port; `ether2` is a wired LAN port) | single **2.5 GbE** trunk |
|
||||
| Management | core/hi/lo `.15` | core/lo `.14` (**not** `hi`; see below) |
|
||||
@@ -58,6 +59,10 @@ with `ssh admin@vibe '/interface wifi export show-sensitive'` (`.passphrase=` pr
|
||||
RouterOS, one hardware-offloaded bridge `main` with `vlan-filtering=yes`. Access: `ssh admin@vibe`
|
||||
— key auth for `admin` is installed (`~/.ssh/id_rsa`), with `admin`/`admin` as a fallback.
|
||||
|
||||
`wifi-qcom` matches the RouterOS version. Follow the shared
|
||||
[upgrade procedure](switches.md#upgrades) for backups and version checks. A DFS channel
|
||||
availability check can delay the 5 GHz radio after reboot while 2.4 GHz is already running.
|
||||
|
||||
- **Uplink `ether1`** — trunk, tagged VLANs 100/110/120; native/untagged is the default VLAN 1
|
||||
(PVID, no IP). `ether2` is a wired **access port** on VLAN 110 (`lo`). `l2mtu 9214`.
|
||||
- **Radios** — `wifi1` (5 GHz, 20/40/80) + `wifi2` (2.4 GHz, 20/40) both broadcast `wlan0`
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# Home network diagrams
|
||||
|
||||
`switch-ports.json` holds the socket inventory, NIC capacities and cable endpoints. Regenerate
|
||||
`switch-panels.svg` and the cable table in `../switches.md` from the repository root:
|
||||
|
||||
```sh
|
||||
python3 docs/sites/home/diagrams/render-switch-panels.py
|
||||
```
|
||||
|
||||
The renderer uses only the Python standard library. Socket geometry and schematic cable routing
|
||||
live in the renderer. Keep the inventory and routing together when adding a connection.
|
||||
|
||||
`home-fabric.mmd` is the Mermaid source for the logical fabric diagram. Render it with Mermaid
|
||||
CLI (the checked-in SVG was rendered with Mermaid 11.12.0):
|
||||
|
||||
```sh
|
||||
mmdc -i docs/sites/home/diagrams/home-fabric.mmd -o docs/sites/home/diagrams/home-fabric.svg
|
||||
```
|
||||
|
||||
Check rendered SVGs visually after editing. Both diagrams are checked in so Markdown viewers
|
||||
can show them without Mermaid support.
|
||||
@@ -0,0 +1,14 @@
|
||||
flowchart LR
|
||||
virgin["Virgin Media modem"] --- shytzel["shytzel"]
|
||||
ont["Digiweb ONT"] --- brian["brian"]
|
||||
shytzel ---|"VLAN 130 trunk"| jim["jim"]
|
||||
brian ---|"2 × 10G LAG"| dave["dave · RSTP root"]
|
||||
brian -. "1G backup" .- jim
|
||||
jim ---|"10G fabric trunk"| dave
|
||||
dave ---|"100G"| palace["palace · river / cellar / sfh"]
|
||||
jim ---|"1G host + separate 1G river trunk"| palace
|
||||
dave ---|"100G"| castle["castle · NVMe-oF root"]
|
||||
jim -. "1G · normally down" .- castle
|
||||
dave ---|"10G LAN"| stream["stream"]
|
||||
jim -. "1G LAN backup" .- stream
|
||||
jim ---|"Separate VLAN 130 WAN handoff"| stream
|
||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 24 KiB |
@@ -0,0 +1,207 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Render the rack panel map and cable table from switch-ports.json."""
|
||||
import html
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import textwrap
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
DATA = json.loads((ROOT / "switch-ports.json").read_text())
|
||||
COLORS = ["#2563eb", "#7c3aed", "#d97706", "#0891b2", "#64748b", "#16a34a", "#db2777", "#dc2626", "#92400e", "#0f766e", "#4f46e5"]
|
||||
TAGS = {}
|
||||
POSITIONS = {}
|
||||
foreground = []
|
||||
for link, color in zip(DATA["links"], COLORS):
|
||||
for end in ("from", "to"):
|
||||
TAGS[tuple(link[end])] = (link["id"], color)
|
||||
|
||||
svg = ['<svg xmlns="http://www.w3.org/2000/svg" width="1440" height="1580" viewBox="0 0 1440 1580" role="img" aria-labelledby="title desc">',
|
||||
'<title id="title">Home rack switch front panels and cable endpoints</title>',
|
||||
'<desc id="desc">Stream is drawn above Brian but physically sits to its left. Brian above Jim above Dave, with palace several rack units below. Socket order follows the hardware; matching letter badges identify cable ends.</desc>',
|
||||
'<style>text{font-family:system-ui,sans-serif;fill:#172033}.heading{font-weight:650;font-size:22px}.sub{fill:#526075;font-size:14px}.port{font-size:12px;font-weight:600}.small{font-size:11px}.mono{font-family:monospace}</style>']
|
||||
|
||||
|
||||
def rect(x, y, w, h, fill, stroke="none", radius=5):
|
||||
svg.append(f'<rect x="{x}" y="{y}" width="{w}" height="{h}" rx="{radius}" fill="{fill}" stroke="{stroke}"/>')
|
||||
|
||||
|
||||
def text(x, y, value, cls="", anchor="start", color=None):
|
||||
style = f' style="fill:{color}"' if color else ""
|
||||
svg.append(f'<text x="{x}" y="{y}" class="{cls}" text-anchor="{anchor}"{style}>{html.escape(value)}</text>')
|
||||
|
||||
|
||||
def badge(x, y, tag, color):
|
||||
rect(x - 10, y - 12, 20, 20, color, radius=4)
|
||||
text(x, y + 2, tag, "port", "middle", "white")
|
||||
|
||||
|
||||
def chassis(y, name, height, note):
|
||||
rect(115, y, 1210, height, "#eef1f5", "#a8b2c1", 8)
|
||||
text(140, y + 28, name, "heading")
|
||||
text(1300, y + 27, note, "sub", "end")
|
||||
|
||||
|
||||
def port(host, socket, x, y, width=46, height=27, label=True):
|
||||
start = len(svg)
|
||||
POSITIONS[(host, socket)] = (x, y, width, height)
|
||||
p = next(p for p in DATA[host]["ports"] if p["socket"] == socket)
|
||||
svg.append('<g><title>' + html.escape(f'{host} {socket}: {p["interface"]} — {p["role"]}') + '</title>')
|
||||
tag, color = TAGS.get((host, socket), (None, "#64748b"))
|
||||
rect(x - width / 2, y, width, height, "#fff", color, 3)
|
||||
rect(x - width / 2 + 6, y + 5, width - 12, height - 10, "#28364a", radius=1)
|
||||
text(x, y - 9, socket, "port mono", "middle")
|
||||
if tag:
|
||||
badge(x + width / 2 - 2, y - 9, tag, color)
|
||||
if label:
|
||||
for row, line in enumerate(textwrap.wrap(p["label"], width=8 if host == "jim" else 11)[:2]):
|
||||
text(x, y + height + 15 + row * 12, line, "small", "middle")
|
||||
svg.append('</g>')
|
||||
foreground.extend(svg[start:])
|
||||
del svg[start:]
|
||||
|
||||
|
||||
rect(0, 0, 1440, 1580, "white", radius=0)
|
||||
text(95, 38, "Home rack · physical socket map", "heading")
|
||||
text(95, 64, "Front views, top to bottom. Socket positions follow the hardware; spacing and chassis widths are schematic.", "sub")
|
||||
svg.append('<g transform="translate(0,330)">')
|
||||
rect(330, -235, 780, 305, "#eef1f5", "#a8b2c1", 8)
|
||||
text(350, -205, "stream · BROUNION R86S", "heading")
|
||||
text(350, -180, "Physically left of brian; drawn above for cable routing", "sub")
|
||||
text(350, -150, "DC / USB", "small")
|
||||
for n, x in enumerate((600, 780, 960)):
|
||||
port("stream", f"ETH{n}", x, -135, width=60)
|
||||
text(x, -65, "Intel I225-V · 2.5G", "small", "middle")
|
||||
text(x, -48, "1G connection" if n < 2 else "Unused", "small", "middle")
|
||||
for socket, x in (("SFP+ L", 650), ("SFP+ R", 890)):
|
||||
port("stream", socket, x, -12, width=70)
|
||||
text(x, 53, "ConnectX-3 · 10G SFP+", "small", "middle")
|
||||
# SFP captions sit below the narrow chassis to preserve the two-row socket layout.
|
||||
for x in (93, 1347):
|
||||
rect(x, 90, 9, 1080, "#dce2eb", radius=2)
|
||||
for y in range(106, 1170, 24):
|
||||
rect(x + 2, y, 5, 8, "white", radius=1)
|
||||
|
||||
chassis(100, "brian · USW-Pro-XG-8-PoE", 155, "8 × RJ45, then 2 × SFP+")
|
||||
rect(205, 180, 10, 10, "#60a5fa", radius=5)
|
||||
text(210, 218, "LED", "small", "middle")
|
||||
for n in range(1, 9):
|
||||
port("brian", str(n), 320 + (n - 1) * 75, 174, width=48)
|
||||
for n in (9, 10):
|
||||
port("brian", str(n), 1060 + (n - 9) * 120, 174, width=65)
|
||||
|
||||
chassis(340, "jim · CRS326-24G-2S+RM", 205, "3 banks of 8 RJ45 · even above odd · SFP+ at right")
|
||||
rect(175, 425, 54, 29, "#28364a", "#94a3b8")
|
||||
text(202, 410, "CONSOLE", "small", "middle")
|
||||
for n in range(1, 25):
|
||||
col = (n - 1) // 2
|
||||
x = 285 + col * 59 + (col // 4) * 17
|
||||
port("jim", str(n), x, 476 if n % 2 else 402, width=43, height=24)
|
||||
port("jim", "S1", 1134, 445, width=54)
|
||||
port("jim", "S2", 1235, 445, width=54)
|
||||
|
||||
chassis(630, "dave · CRS504-4XQ-IN", 165, "4 QSFP28 cages · console above management at right")
|
||||
for x, label in ((175, "PSU 1"), (290, "PSU 2")):
|
||||
rect(x, 680, 90, 78, "#d1d8e2", "#94a3b8")
|
||||
text(x + 45, 725, label, "sub", "middle")
|
||||
for n in range(1, 5):
|
||||
x = 490 + (n - 1) * 140
|
||||
if n == 3:
|
||||
rect(x - 46, 695, 92, 29, "#fff", "#0891b2")
|
||||
rect(x - 38, 701, 76, 17, "#28364a", radius=1)
|
||||
text(x, 686, "Q3", "port mono", "middle")
|
||||
text(x, 743, "4 × 10G breakout", "small", "middle")
|
||||
svg.append(f'<path d="M{x} 757 V820" stroke="#0891b2" stroke-width="2" fill="none"/>')
|
||||
else:
|
||||
port("dave", f"Q{n}", x, 695, width=92)
|
||||
rect(1095, 690, 54, 27, "#28364a", "#94a3b8")
|
||||
text(1122, 681, "CONSOLE", "small", "middle")
|
||||
port("dave", "MGMT", 1122, 741, width=54, height=24, label=False)
|
||||
text(1235, 725, "DC / reset", "sub", "middle")
|
||||
|
||||
rect(430, 820, 820, 84, "#f4f8fc", "#b9cadb")
|
||||
text(445, 841, "Q3 breakout lanes (one physical cage)", "sub")
|
||||
for n in range(1, 5):
|
||||
start = len(svg)
|
||||
p = next(p for p in DATA["dave"]["ports"] if p["socket"] == f"Q3.{n}")
|
||||
x = 485 + (n - 1) * 200
|
||||
tag, color = TAGS.get(("dave", p["socket"]), (None, "#64748b"))
|
||||
text(x, 870, p["socket"], "port mono")
|
||||
if tag:
|
||||
badge(x + 61, 866, tag, color)
|
||||
text(x, 889, p["label"], "small")
|
||||
rect(x + 83, 854, 34, 20, "white", color, radius=3)
|
||||
rect(x + 89, 860, 22, 8, color, radius=1)
|
||||
POSITIONS[("dave", p["socket"])] = (x + 100, 854, 34, 20)
|
||||
foreground.extend(svg[start:])
|
||||
del svg[start:]
|
||||
|
||||
text(720, 963, "⋮ several rack units ⋮", "sub", "middle")
|
||||
chassis(1030, "palace", 160, "NIC endpoints only; rear-panel positions are not depicted")
|
||||
for x, name in ((350, "uplink-dave"), (730, "host-jim"), (1110, "river-jim-phy")):
|
||||
start = len(svg)
|
||||
tag, color = TAGS[("palace", name)]
|
||||
rect(x - 145, 1090, 290, 76, "white", color)
|
||||
text(x - 118, 1120, name, "port mono")
|
||||
badge(x + 119, 1114, tag, color)
|
||||
spec = next(p for p in DATA["palace"]["ports"] if p["socket"] == name)
|
||||
text(x, 1146, spec["nic"] + " · " + spec["capacity"], "small", "middle")
|
||||
POSITIONS[("palace", name)] = (x, 1090, 290, 76)
|
||||
foreground.extend(svg[start:])
|
||||
del svg[start:]
|
||||
|
||||
|
||||
def anchor(host, socket, side="bottom"):
|
||||
x, y, w, h = POSITIONS[(host, socket)]
|
||||
return {"top": (x, y), "bottom": (x, y + h),
|
||||
"left": (x - w / 2, y + h / 2), "right": (x + w / 2, y + h / 2)}[side]
|
||||
|
||||
|
||||
def cable(tag, points):
|
||||
color = COLORS[ord(tag) - ord("A")]
|
||||
points = " ".join(f"{x},{y}" for x, y in points)
|
||||
svg.append(f'<polyline points="{points}" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>')
|
||||
svg.append(f'<polyline points="{points}" fill="none" stroke="{color}" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>')
|
||||
|
||||
|
||||
cable("A", [anchor("brian", "9"), (1060, 280), (38, 280), (38, 925),
|
||||
(785, 925), anchor("dave", "Q3.2")])
|
||||
cable("B", [anchor("brian", "10"), (1180, 300), (58, 300), (58, 950),
|
||||
(1185, 950), anchor("dave", "Q3.4")])
|
||||
cable("C", [anchor("brian", "5"), (620, 315), (492, 315), (492, 458),
|
||||
(462, 458), anchor("jim", "7", "top")])
|
||||
cable("D", [anchor("jim", "S1"), (1134, 575), (1394, 575), (1394, 975),
|
||||
(585, 975), anchor("dave", "Q3.1")])
|
||||
cable("E", [anchor("jim", "24", "right"), (1010, 414), (1010, 600),
|
||||
(1380, 600), (1380, 753), anchor("dave", "MGMT", "right")])
|
||||
cable("F", [anchor("jim", "5"), (403, 568), (1368, 568), (1368, 1010),
|
||||
(730, 1010), anchor("palace", "host-jim", "top")])
|
||||
cable("G", [anchor("jim", "22", "right"), (940, 414), (940, 590),
|
||||
(1416, 590), (1416, 1025), (1110, 1025), anchor("palace", "river-jim-phy", "top")])
|
||||
cable("H", [anchor("dave", "Q1"), (490, 790), (410, 790), (410, 1000),
|
||||
(350, 1000), anchor("palace", "uplink-dave", "top")])
|
||||
|
||||
cable("I", [anchor("stream", "ETH0", "top"), (600, -160), (18, -160),
|
||||
(18, 555), (898, 555), anchor("jim", "21")])
|
||||
cable("J", [anchor("stream", "ETH1", "top"), (780, -170), (70, -170),
|
||||
(70, 322), (403, 322), anchor("jim", "6", "top")])
|
||||
cable("K", [anchor("stream", "SFP+ R"), (890, 80), (1428, 80),
|
||||
(1428, 920), (985, 920), anchor("dave", "Q3.3")])
|
||||
|
||||
svg.extend(foreground)
|
||||
|
||||
text(95, 1220, "Matching letter badges identify cable ends. Port labels describe configured use; they do not indicate current link state.", "sub")
|
||||
svg.append('</g></svg>')
|
||||
(ROOT / "switch-panels.svg").write_text('\n'.join(svg) + '\n')
|
||||
|
||||
rows = ["| Cable | Endpoint | Endpoint | Connection |", "|---|---|---|---|"]
|
||||
for link in DATA["links"]:
|
||||
ends = [f'`{link[end][0]}` `{link[end][1]}`' for end in ("from", "to")]
|
||||
rows.append(f'| **{link["id"]}** | {ends[0]} | {ends[1]} | {link["description"]} |')
|
||||
page = ROOT.parent / "switches.md"
|
||||
content = page.read_text()
|
||||
start, end = "<!-- switch-cables -->", "<!-- /switch-cables -->"
|
||||
if start in content:
|
||||
content = re.sub(re.escape(start) + r'.*?' + re.escape(end),
|
||||
start + '\n' + '\n'.join(rows) + '\n' + end, content, flags=re.S)
|
||||
page.write_text(content)
|
||||
@@ -0,0 +1,517 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1440" height="1580" viewBox="0 0 1440 1580" role="img" aria-labelledby="title desc">
|
||||
<title id="title">Home rack switch front panels and cable endpoints</title>
|
||||
<desc id="desc">Stream is drawn above Brian but physically sits to its left. Brian above Jim above Dave, with palace several rack units below. Socket order follows the hardware; matching letter badges identify cable ends.</desc>
|
||||
<style>text{font-family:system-ui,sans-serif;fill:#172033}.heading{font-weight:650;font-size:22px}.sub{fill:#526075;font-size:14px}.port{font-size:12px;font-weight:600}.small{font-size:11px}.mono{font-family:monospace}</style>
|
||||
<rect x="0" y="0" width="1440" height="1580" rx="0" fill="white" stroke="none"/>
|
||||
<text x="95" y="38" class="heading" text-anchor="start">Home rack · physical socket map</text>
|
||||
<text x="95" y="64" class="sub" text-anchor="start">Front views, top to bottom. Socket positions follow the hardware; spacing and chassis widths are schematic.</text>
|
||||
<g transform="translate(0,330)">
|
||||
<rect x="330" y="-235" width="780" height="305" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||
<text x="350" y="-205" class="heading" text-anchor="start">stream · BROUNION R86S</text>
|
||||
<text x="350" y="-180" class="sub" text-anchor="start">Physically left of brian; drawn above for cable routing</text>
|
||||
<text x="350" y="-150" class="small" text-anchor="start">DC / USB</text>
|
||||
<text x="600" y="-65" class="small" text-anchor="middle">Intel I225-V · 2.5G</text>
|
||||
<text x="600" y="-48" class="small" text-anchor="middle">1G connection</text>
|
||||
<text x="780" y="-65" class="small" text-anchor="middle">Intel I225-V · 2.5G</text>
|
||||
<text x="780" y="-48" class="small" text-anchor="middle">1G connection</text>
|
||||
<text x="960" y="-65" class="small" text-anchor="middle">Intel I225-V · 2.5G</text>
|
||||
<text x="960" y="-48" class="small" text-anchor="middle">Unused</text>
|
||||
<text x="650" y="53" class="small" text-anchor="middle">ConnectX-3 · 10G SFP+</text>
|
||||
<text x="890" y="53" class="small" text-anchor="middle">ConnectX-3 · 10G SFP+</text>
|
||||
<rect x="93" y="90" width="9" height="1080" rx="2" fill="#dce2eb" stroke="none"/>
|
||||
<rect x="95" y="106" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="130" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="154" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="178" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="202" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="226" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="250" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="274" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="298" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="322" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="346" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="370" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="394" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="418" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="442" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="466" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="490" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="514" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="538" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="562" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="586" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="610" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="634" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="658" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="682" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="706" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="730" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="754" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="778" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="802" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="826" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="850" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="874" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="898" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="922" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="946" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="970" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="994" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="1018" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="1042" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="1066" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="1090" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="1114" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="1138" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="95" y="1162" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1347" y="90" width="9" height="1080" rx="2" fill="#dce2eb" stroke="none"/>
|
||||
<rect x="1349" y="106" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="130" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="154" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="178" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="202" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="226" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="250" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="274" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="298" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="322" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="346" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="370" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="394" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="418" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="442" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="466" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="490" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="514" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="538" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="562" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="586" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="610" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="634" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="658" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="682" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="706" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="730" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="754" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="778" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="802" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="826" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="850" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="874" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="898" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="922" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="946" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="970" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="994" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="1018" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="1042" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="1066" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="1090" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="1114" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="1138" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="1349" y="1162" width="5" height="8" rx="1" fill="white" stroke="none"/>
|
||||
<rect x="115" y="100" width="1210" height="155" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||
<text x="140" y="128" class="heading" text-anchor="start">brian · USW-Pro-XG-8-PoE</text>
|
||||
<text x="1300" y="127" class="sub" text-anchor="end">8 × RJ45, then 2 × SFP+</text>
|
||||
<rect x="205" y="180" width="10" height="10" rx="5" fill="#60a5fa" stroke="none"/>
|
||||
<text x="210" y="218" class="small" text-anchor="middle">LED</text>
|
||||
<rect x="115" y="340" width="1210" height="205" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||
<text x="140" y="368" class="heading" text-anchor="start">jim · CRS326-24G-2S+RM</text>
|
||||
<text x="1300" y="367" class="sub" text-anchor="end">3 banks of 8 RJ45 · even above odd · SFP+ at right</text>
|
||||
<rect x="175" y="425" width="54" height="29" rx="5" fill="#28364a" stroke="#94a3b8"/>
|
||||
<text x="202" y="410" class="small" text-anchor="middle">CONSOLE</text>
|
||||
<rect x="115" y="630" width="1210" height="165" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||
<text x="140" y="658" class="heading" text-anchor="start">dave · CRS504-4XQ-IN</text>
|
||||
<text x="1300" y="657" class="sub" text-anchor="end">4 QSFP28 cages · console above management at right</text>
|
||||
<rect x="175" y="680" width="90" height="78" rx="5" fill="#d1d8e2" stroke="#94a3b8"/>
|
||||
<text x="220" y="725" class="sub" text-anchor="middle">PSU 1</text>
|
||||
<rect x="290" y="680" width="90" height="78" rx="5" fill="#d1d8e2" stroke="#94a3b8"/>
|
||||
<text x="335" y="725" class="sub" text-anchor="middle">PSU 2</text>
|
||||
<rect x="724" y="695" width="92" height="29" rx="5" fill="#fff" stroke="#0891b2"/>
|
||||
<rect x="732" y="701" width="76" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="770" y="686" class="port mono" text-anchor="middle">Q3</text>
|
||||
<text x="770" y="743" class="small" text-anchor="middle">4 × 10G breakout</text>
|
||||
<path d="M770 757 V820" stroke="#0891b2" stroke-width="2" fill="none"/>
|
||||
<rect x="1095" y="690" width="54" height="27" rx="5" fill="#28364a" stroke="#94a3b8"/>
|
||||
<text x="1122" y="681" class="small" text-anchor="middle">CONSOLE</text>
|
||||
<text x="1235" y="725" class="sub" text-anchor="middle">DC / reset</text>
|
||||
<rect x="430" y="820" width="820" height="84" rx="5" fill="#f4f8fc" stroke="#b9cadb"/>
|
||||
<text x="445" y="841" class="sub" text-anchor="start">Q3 breakout lanes (one physical cage)</text>
|
||||
<text x="720" y="963" class="sub" text-anchor="middle">⋮ several rack units ⋮</text>
|
||||
<rect x="115" y="1030" width="1210" height="160" rx="8" fill="#eef1f5" stroke="#a8b2c1"/>
|
||||
<text x="140" y="1058" class="heading" text-anchor="start">palace</text>
|
||||
<text x="1300" y="1057" class="sub" text-anchor="end">NIC endpoints only; rear-panel positions are not depicted</text>
|
||||
<polyline points="1060,201 1060,280 38,280 38,925 785,925 785,874" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="1060,201 1060,280 38,280 38,925 785,925 785,874" fill="none" stroke="#2563eb" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="1180,201 1180,300 58,300 58,950 1185,950 1185,874" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="1180,201 1180,300 58,300 58,950 1185,950 1185,874" fill="none" stroke="#7c3aed" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="620,201 620,315 492,315 492,458 462,458 462,476" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="620,201 620,315 492,315 492,458 462,458 462,476" fill="none" stroke="#d97706" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="1134,472 1134,575 1394,575 1394,975 585,975 585,874" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="1134,472 1134,575 1394,575 1394,975 585,975 585,874" fill="none" stroke="#0891b2" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="989.5,414.0 1010,414 1010,600 1380,600 1380,753 1149.0,753.0" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="989.5,414.0 1010,414 1010,600 1380,600 1380,753 1149.0,753.0" fill="none" stroke="#64748b" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="403,500 403,568 1368,568 1368,1010 730,1010 730,1090" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="403,500 403,568 1368,568 1368,1010 730,1010 730,1090" fill="none" stroke="#16a34a" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="930.5,414.0 940,414 940,590 1416,590 1416,1025 1110,1025 1110,1090" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="930.5,414.0 940,414 940,590 1416,590 1416,1025 1110,1025 1110,1090" fill="none" stroke="#db2777" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="490,722 490,790 410,790 410,1000 350,1000 350,1090" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="490,722 490,790 410,790 410,1000 350,1000 350,1090" fill="none" stroke="#dc2626" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="600,-135 600,-160 18,-160 18,555 898,555 909,500" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="600,-135 600,-160 18,-160 18,555 898,555 909,500" fill="none" stroke="#92400e" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="780,-135 780,-170 70,-170 70,322 403,322 403,402" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="780,-135 780,-170 70,-170 70,322 403,322 403,402" fill="none" stroke="#0f766e" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="890,15 890,80 1428,80 1428,920 985,920 985,874" fill="none" stroke="white" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<polyline points="890,15 890,80 1428,80 1428,920 985,920 985,874" fill="none" stroke="#4f46e5" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round"/>
|
||||
<g><title>stream ETH0: wan — 1G connection to jim 21</title>
|
||||
<rect x="570.0" y="-135" width="60" height="27" rx="3" fill="#fff" stroke="#92400e"/>
|
||||
<rect x="576.0" y="-130" width="48" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="600" y="-144" class="port mono" text-anchor="middle">ETH0</text>
|
||||
<rect x="618.0" y="-156" width="20" height="20" rx="4" fill="#92400e" stroke="none"/>
|
||||
<text x="628.0" y="-142" class="port" text-anchor="middle" style="fill:white">I</text>
|
||||
<text x="600" y="-93" class="small" text-anchor="middle">wan</text>
|
||||
</g>
|
||||
<g><title>stream ETH1: lan-jim — 1G connection to jim 6</title>
|
||||
<rect x="750.0" y="-135" width="60" height="27" rx="3" fill="#fff" stroke="#0f766e"/>
|
||||
<rect x="756.0" y="-130" width="48" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="780" y="-144" class="port mono" text-anchor="middle">ETH1</text>
|
||||
<rect x="798.0" y="-156" width="20" height="20" rx="4" fill="#0f766e" stroke="none"/>
|
||||
<text x="808.0" y="-142" class="port" text-anchor="middle" style="fill:white">J</text>
|
||||
<text x="780" y="-93" class="small" text-anchor="middle">lan-jim</text>
|
||||
</g>
|
||||
<g><title>stream ETH2: et2 — Unused</title>
|
||||
<rect x="930.0" y="-135" width="60" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="936.0" y="-130" width="48" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="960" y="-144" class="port mono" text-anchor="middle">ETH2</text>
|
||||
<text x="960" y="-93" class="small" text-anchor="middle">et2</text>
|
||||
</g>
|
||||
<g><title>stream SFP+ L: et5 — Unused</title>
|
||||
<rect x="615.0" y="-12" width="70" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="621.0" y="-7" width="58" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="650" y="-21" class="port mono" text-anchor="middle">SFP+ L</text>
|
||||
<text x="650" y="30" class="small" text-anchor="middle">et5</text>
|
||||
</g>
|
||||
<g><title>stream SFP+ R: lan-dave — 10G connection to dave Q3.3</title>
|
||||
<rect x="855.0" y="-12" width="70" height="27" rx="3" fill="#fff" stroke="#4f46e5"/>
|
||||
<rect x="861.0" y="-7" width="58" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="890" y="-21" class="port mono" text-anchor="middle">SFP+ R</text>
|
||||
<rect x="913.0" y="-33" width="20" height="20" rx="4" fill="#4f46e5" stroke="none"/>
|
||||
<text x="923.0" y="-19" class="port" text-anchor="middle" style="fill:white">K</text>
|
||||
<text x="890" y="30" class="small" text-anchor="middle">lan-dave</text>
|
||||
</g>
|
||||
<g><title>brian 1: Port 1 — Configured port label</title>
|
||||
<rect x="296.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="302.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="320" y="165" class="port mono" text-anchor="middle">1</text>
|
||||
<text x="320" y="216" class="small" text-anchor="middle">fort</text>
|
||||
</g>
|
||||
<g><title>brian 2: Port 2 — Configured port label</title>
|
||||
<rect x="371.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="377.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="395" y="165" class="port mono" text-anchor="middle">2</text>
|
||||
<text x="395" y="216" class="small" text-anchor="middle">castle</text>
|
||||
</g>
|
||||
<g><title>brian 3: Port 3 — portcullis test connection</title>
|
||||
<rect x="446.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="452.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="470" y="165" class="port mono" text-anchor="middle">3</text>
|
||||
<text x="470" y="216" class="small" text-anchor="middle">portcullis</text>
|
||||
<text x="470" y="228" class="small" text-anchor="middle">test</text>
|
||||
</g>
|
||||
<g><title>brian 4: Port 4 — Second JetKVM</title>
|
||||
<rect x="521.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="527.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="545" y="165" class="port mono" text-anchor="middle">4</text>
|
||||
<text x="545" y="216" class="small" text-anchor="middle">portcullis</text>
|
||||
<text x="545" y="228" class="small" text-anchor="middle">KVM</text>
|
||||
</g>
|
||||
<g><title>brian 5: Port 5 — Backup fabric trunk</title>
|
||||
<rect x="596.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#d97706"/>
|
||||
<rect x="602.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="620" y="165" class="port mono" text-anchor="middle">5</text>
|
||||
<rect x="632.0" y="153" width="20" height="20" rx="4" fill="#d97706" stroke="none"/>
|
||||
<text x="642.0" y="167" class="port" text-anchor="middle" style="fill:white">C</text>
|
||||
<text x="620" y="216" class="small" text-anchor="middle">jim P7</text>
|
||||
</g>
|
||||
<g><title>brian 6: Port 6 — Native VLAN 140; tagged VLAN 10</title>
|
||||
<rect x="671.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="677.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="695" y="165" class="port mono" text-anchor="middle">6</text>
|
||||
<text x="695" y="216" class="small" text-anchor="middle">Digiweb ONT</text>
|
||||
</g>
|
||||
<g><title>brian 7: Port 7 — AP trunk</title>
|
||||
<rect x="746.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="752.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="770" y="165" class="port mono" text-anchor="middle">7</text>
|
||||
<text x="770" y="216" class="small" text-anchor="middle">wave</text>
|
||||
</g>
|
||||
<g><title>brian 8: Port 8 — AP trunk</title>
|
||||
<rect x="821.0" y="174" width="48" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="827.0" y="179" width="36" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="845" y="165" class="port mono" text-anchor="middle">8</text>
|
||||
<text x="845" y="216" class="small" text-anchor="middle">vibe</text>
|
||||
</g>
|
||||
<g><title>brian 9: SFP+ 1 — LAG member 1</title>
|
||||
<rect x="1027.5" y="174" width="65" height="27" rx="3" fill="#fff" stroke="#2563eb"/>
|
||||
<rect x="1033.5" y="179" width="53" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="1060" y="165" class="port mono" text-anchor="middle">9</text>
|
||||
<rect x="1080.5" y="153" width="20" height="20" rx="4" fill="#2563eb" stroke="none"/>
|
||||
<text x="1090.5" y="167" class="port" text-anchor="middle" style="fill:white">A</text>
|
||||
<text x="1060" y="216" class="small" text-anchor="middle">dave Q3.2</text>
|
||||
</g>
|
||||
<g><title>brian 10: SFP+ 2 — LAG member 2</title>
|
||||
<rect x="1147.5" y="174" width="65" height="27" rx="3" fill="#fff" stroke="#7c3aed"/>
|
||||
<rect x="1153.5" y="179" width="53" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="1180" y="165" class="port mono" text-anchor="middle">10</text>
|
||||
<rect x="1200.5" y="153" width="20" height="20" rx="4" fill="#7c3aed" stroke="none"/>
|
||||
<text x="1210.5" y="167" class="port" text-anchor="middle" style="fill:white">B</text>
|
||||
<text x="1180" y="216" class="small" text-anchor="middle">dave Q3.4</text>
|
||||
</g>
|
||||
<g><title>jim 1: ether1 — Configured member of shytzel-downlink</title>
|
||||
<rect x="263.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="269.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="285" y="467" class="port mono" text-anchor="middle">1</text>
|
||||
<text x="285" y="515" class="small" text-anchor="middle">shytzel</text>
|
||||
<text x="285" y="527" class="small" text-anchor="middle">LAG</text>
|
||||
</g>
|
||||
<g><title>jim 2: ether2 — Configured member of shytzel-downlink</title>
|
||||
<rect x="263.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="269.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="285" y="393" class="port mono" text-anchor="middle">2</text>
|
||||
<text x="285" y="441" class="small" text-anchor="middle">shytzel</text>
|
||||
<text x="285" y="453" class="small" text-anchor="middle">LAG</text>
|
||||
</g>
|
||||
<g><title>jim 3: ether3 — Configured member of shytzel-downlink</title>
|
||||
<rect x="322.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="328.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="344" y="467" class="port mono" text-anchor="middle">3</text>
|
||||
<text x="344" y="515" class="small" text-anchor="middle">shytzel</text>
|
||||
<text x="344" y="527" class="small" text-anchor="middle">LAG</text>
|
||||
</g>
|
||||
<g><title>jim 4: ether4 — Configured member of shytzel-downlink</title>
|
||||
<rect x="322.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="328.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="344" y="393" class="port mono" text-anchor="middle">4</text>
|
||||
<text x="344" y="441" class="small" text-anchor="middle">shytzel</text>
|
||||
<text x="344" y="453" class="small" text-anchor="middle">LAG</text>
|
||||
</g>
|
||||
<g><title>jim 5: palace — Core / netboot trunk</title>
|
||||
<rect x="381.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#16a34a"/>
|
||||
<rect x="387.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="403" y="467" class="port mono" text-anchor="middle">5</text>
|
||||
<rect x="412.5" y="455" width="20" height="20" rx="4" fill="#16a34a" stroke="none"/>
|
||||
<text x="422.5" y="469" class="port" text-anchor="middle" style="fill:white">F</text>
|
||||
<text x="403" y="515" class="small" text-anchor="middle">palace</text>
|
||||
<text x="403" y="527" class="small" text-anchor="middle">host</text>
|
||||
</g>
|
||||
<g><title>jim 6: stream — Backup RSTP trunk</title>
|
||||
<rect x="381.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#0f766e"/>
|
||||
<rect x="387.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="403" y="393" class="port mono" text-anchor="middle">6</text>
|
||||
<rect x="412.5" y="381" width="20" height="20" rx="4" fill="#0f766e" stroke="none"/>
|
||||
<text x="422.5" y="395" class="port" text-anchor="middle" style="fill:white">J</text>
|
||||
<text x="403" y="441" class="small" text-anchor="middle">stream</text>
|
||||
<text x="403" y="453" class="small" text-anchor="middle">LAN</text>
|
||||
</g>
|
||||
<g><title>jim 7: brian-downlink — Backup fabric trunk</title>
|
||||
<rect x="440.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#d97706"/>
|
||||
<rect x="446.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="462" y="467" class="port mono" text-anchor="middle">7</text>
|
||||
<rect x="471.5" y="455" width="20" height="20" rx="4" fill="#d97706" stroke="none"/>
|
||||
<text x="481.5" y="469" class="port" text-anchor="middle" style="fill:white">C</text>
|
||||
<text x="462" y="515" class="small" text-anchor="middle">brian P5</text>
|
||||
</g>
|
||||
<g><title>jim 8: palace-kvm — lo access</title>
|
||||
<rect x="440.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="446.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="462" y="393" class="port mono" text-anchor="middle">8</text>
|
||||
<text x="462" y="441" class="small" text-anchor="middle">JetKVM</text>
|
||||
</g>
|
||||
<g><title>jim 9: ether9 — Unbridged spare</title>
|
||||
<rect x="516.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="522.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="538" y="467" class="port mono" text-anchor="middle">9</text>
|
||||
<text x="538" y="515" class="small" text-anchor="middle">spare</text>
|
||||
</g>
|
||||
<g><title>jim 10: castle — Backup attachment, normally down</title>
|
||||
<rect x="516.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="522.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="538" y="393" class="port mono" text-anchor="middle">10</text>
|
||||
<text x="538" y="441" class="small" text-anchor="middle">castle</text>
|
||||
</g>
|
||||
<g><title>jim 11: fort — lo access (configured label)</title>
|
||||
<rect x="575.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="581.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="597" y="467" class="port mono" text-anchor="middle">11</text>
|
||||
<text x="597" y="515" class="small" text-anchor="middle">fort</text>
|
||||
</g>
|
||||
<g><title>jim 12: laptop-dock — lo access (configured label)</title>
|
||||
<rect x="575.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="581.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="597" y="393" class="port mono" text-anchor="middle">12</text>
|
||||
<text x="597" y="441" class="small" text-anchor="middle">dock</text>
|
||||
</g>
|
||||
<g><title>jim 13: pronter — lo access (configured label)</title>
|
||||
<rect x="634.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="640.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="656" y="467" class="port mono" text-anchor="middle">13</text>
|
||||
<text x="656" y="515" class="small" text-anchor="middle">pronter</text>
|
||||
</g>
|
||||
<g><title>jim 14: untrusted-sta1 — Untrusted access</title>
|
||||
<rect x="634.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="640.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="656" y="393" class="port mono" text-anchor="middle">14</text>
|
||||
<text x="656" y="441" class="small" text-anchor="middle">guest</text>
|
||||
</g>
|
||||
<g><title>jim 15: ether15 — lo access; peer unconfirmed</title>
|
||||
<rect x="693.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="699.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="715" y="467" class="port mono" text-anchor="middle">15</text>
|
||||
<text x="715" y="515" class="small" text-anchor="middle">unmapped</text>
|
||||
</g>
|
||||
<g><title>jim 16: ether16 — lo access; peer unconfirmed</title>
|
||||
<rect x="693.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="699.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="715" y="393" class="port mono" text-anchor="middle">16</text>
|
||||
<text x="715" y="441" class="small" text-anchor="middle">unmapped</text>
|
||||
</g>
|
||||
<g><title>jim 17: ether17 — lo access; peer unconfirmed</title>
|
||||
<rect x="769.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="775.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="791" y="467" class="port mono" text-anchor="middle">17</text>
|
||||
<text x="791" y="515" class="small" text-anchor="middle">unmapped</text>
|
||||
</g>
|
||||
<g><title>jim 18: ether18 — lo access; peer unconfirmed</title>
|
||||
<rect x="769.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="775.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="791" y="393" class="port mono" text-anchor="middle">18</text>
|
||||
<text x="791" y="441" class="small" text-anchor="middle">unmapped</text>
|
||||
</g>
|
||||
<g><title>jim 19: ether19 — lo access; peer unconfirmed</title>
|
||||
<rect x="828.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="834.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="850" y="467" class="port mono" text-anchor="middle">19</text>
|
||||
<text x="850" y="515" class="small" text-anchor="middle">unmapped</text>
|
||||
</g>
|
||||
<g><title>jim 20: ether20 — lo access; peer unconfirmed</title>
|
||||
<rect x="828.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="834.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="850" y="393" class="port mono" text-anchor="middle">20</text>
|
||||
<text x="850" y="441" class="small" text-anchor="middle">unmapped</text>
|
||||
</g>
|
||||
<g><title>jim 21: stream-wan — Untagged VLAN 130 handoff</title>
|
||||
<rect x="887.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#92400e"/>
|
||||
<rect x="893.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="909" y="467" class="port mono" text-anchor="middle">21</text>
|
||||
<rect x="918.5" y="455" width="20" height="20" rx="4" fill="#92400e" stroke="none"/>
|
||||
<text x="928.5" y="469" class="port" text-anchor="middle" style="fill:white">I</text>
|
||||
<text x="909" y="515" class="small" text-anchor="middle">stream</text>
|
||||
<text x="909" y="527" class="small" text-anchor="middle">WAN</text>
|
||||
</g>
|
||||
<g><title>jim 22: river — Dedicated router trunk</title>
|
||||
<rect x="887.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#db2777"/>
|
||||
<rect x="893.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="909" y="393" class="port mono" text-anchor="middle">22</text>
|
||||
<rect x="918.5" y="381" width="20" height="20" rx="4" fill="#db2777" stroke="none"/>
|
||||
<text x="928.5" y="395" class="port" text-anchor="middle" style="fill:white">G</text>
|
||||
<text x="909" y="441" class="small" text-anchor="middle">river</text>
|
||||
<text x="909" y="453" class="small" text-anchor="middle">backup</text>
|
||||
</g>
|
||||
<g><title>jim 23: ups — lo access</title>
|
||||
<rect x="946.5" y="476" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="952.5" y="481" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="968" y="467" class="port mono" text-anchor="middle">23</text>
|
||||
<text x="968" y="515" class="small" text-anchor="middle">UPS</text>
|
||||
</g>
|
||||
<g><title>jim 24: dave-management — Core management</title>
|
||||
<rect x="946.5" y="402" width="43" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="952.5" y="407" width="31" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="968" y="393" class="port mono" text-anchor="middle">24</text>
|
||||
<rect x="977.5" y="381" width="20" height="20" rx="4" fill="#64748b" stroke="none"/>
|
||||
<text x="987.5" y="395" class="port" text-anchor="middle" style="fill:white">E</text>
|
||||
<text x="968" y="441" class="small" text-anchor="middle">dave</text>
|
||||
<text x="968" y="453" class="small" text-anchor="middle">mgmt</text>
|
||||
</g>
|
||||
<g><title>jim S1: dave-uplink — 10G fabric trunk</title>
|
||||
<rect x="1107.0" y="445" width="54" height="27" rx="3" fill="#fff" stroke="#0891b2"/>
|
||||
<rect x="1113.0" y="450" width="42" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="1134" y="436" class="port mono" text-anchor="middle">S1</text>
|
||||
<rect x="1149.0" y="424" width="20" height="20" rx="4" fill="#0891b2" stroke="none"/>
|
||||
<text x="1159.0" y="438" class="port" text-anchor="middle" style="fill:white">D</text>
|
||||
<text x="1134" y="487" class="small" text-anchor="middle">dave</text>
|
||||
<text x="1134" y="499" class="small" text-anchor="middle">Q3.1</text>
|
||||
</g>
|
||||
<g><title>jim S2: sfp-spare — 10G trunk; colony staging</title>
|
||||
<rect x="1208.0" y="445" width="54" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="1214.0" y="450" width="42" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="1235" y="436" class="port mono" text-anchor="middle">S2</text>
|
||||
<text x="1235" y="487" class="small" text-anchor="middle">fergal</text>
|
||||
</g>
|
||||
<g><title>dave Q1: palace — 100G; whole cage</title>
|
||||
<rect x="444.0" y="695" width="92" height="27" rx="3" fill="#fff" stroke="#dc2626"/>
|
||||
<rect x="450.0" y="700" width="80" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="490" y="686" class="port mono" text-anchor="middle">Q1</text>
|
||||
<rect x="524.0" y="674" width="20" height="20" rx="4" fill="#dc2626" stroke="none"/>
|
||||
<text x="534.0" y="688" class="port" text-anchor="middle" style="fill:white">H</text>
|
||||
<text x="490" y="737" class="small" text-anchor="middle">palace</text>
|
||||
</g>
|
||||
<g><title>dave Q2: castle — 100G; whole cage</title>
|
||||
<rect x="584.0" y="695" width="92" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="590.0" y="700" width="80" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="630" y="686" class="port mono" text-anchor="middle">Q2</text>
|
||||
<text x="630" y="737" class="small" text-anchor="middle">castle</text>
|
||||
</g>
|
||||
<g><title>dave Q4: qsfp28-4-1 — No identified peer</title>
|
||||
<rect x="864.0" y="695" width="92" height="27" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="870.0" y="700" width="80" height="17" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="910" y="686" class="port mono" text-anchor="middle">Q4</text>
|
||||
<text x="910" y="737" class="small" text-anchor="middle">unmapped</text>
|
||||
</g>
|
||||
<g><title>dave MGMT: management — 100M management RJ45</title>
|
||||
<rect x="1095.0" y="741" width="54" height="24" rx="3" fill="#fff" stroke="#64748b"/>
|
||||
<rect x="1101.0" y="746" width="42" height="14" rx="1" fill="#28364a" stroke="none"/>
|
||||
<text x="1122" y="732" class="port mono" text-anchor="middle">MGMT</text>
|
||||
<rect x="1137.0" y="720" width="20" height="20" rx="4" fill="#64748b" stroke="none"/>
|
||||
<text x="1147.0" y="734" class="port" text-anchor="middle" style="fill:white">E</text>
|
||||
</g>
|
||||
<text x="485" y="870" class="port mono" text-anchor="start">Q3.1</text>
|
||||
<rect x="536" y="854" width="20" height="20" rx="4" fill="#0891b2" stroke="none"/>
|
||||
<text x="546" y="868" class="port" text-anchor="middle" style="fill:white">D</text>
|
||||
<text x="485" y="889" class="small" text-anchor="start">jim SFP+1</text>
|
||||
<rect x="568" y="854" width="34" height="20" rx="3" fill="white" stroke="#0891b2"/>
|
||||
<rect x="574" y="860" width="22" height="8" rx="1" fill="#0891b2" stroke="none"/>
|
||||
<text x="685" y="870" class="port mono" text-anchor="start">Q3.2</text>
|
||||
<rect x="736" y="854" width="20" height="20" rx="4" fill="#2563eb" stroke="none"/>
|
||||
<text x="746" y="868" class="port" text-anchor="middle" style="fill:white">A</text>
|
||||
<text x="685" y="889" class="small" text-anchor="start">brian SFP+1</text>
|
||||
<rect x="768" y="854" width="34" height="20" rx="3" fill="white" stroke="#2563eb"/>
|
||||
<rect x="774" y="860" width="22" height="8" rx="1" fill="#2563eb" stroke="none"/>
|
||||
<text x="885" y="870" class="port mono" text-anchor="start">Q3.3</text>
|
||||
<rect x="936" y="854" width="20" height="20" rx="4" fill="#4f46e5" stroke="none"/>
|
||||
<text x="946" y="868" class="port" text-anchor="middle" style="fill:white">K</text>
|
||||
<text x="885" y="889" class="small" text-anchor="start">stream LAN</text>
|
||||
<rect x="968" y="854" width="34" height="20" rx="3" fill="white" stroke="#4f46e5"/>
|
||||
<rect x="974" y="860" width="22" height="8" rx="1" fill="#4f46e5" stroke="none"/>
|
||||
<text x="1085" y="870" class="port mono" text-anchor="start">Q3.4</text>
|
||||
<rect x="1136" y="854" width="20" height="20" rx="4" fill="#7c3aed" stroke="none"/>
|
||||
<text x="1146" y="868" class="port" text-anchor="middle" style="fill:white">B</text>
|
||||
<text x="1085" y="889" class="small" text-anchor="start">brian SFP+2</text>
|
||||
<rect x="1168" y="854" width="34" height="20" rx="3" fill="white" stroke="#7c3aed"/>
|
||||
<rect x="1174" y="860" width="22" height="8" rx="1" fill="#7c3aed" stroke="none"/>
|
||||
<rect x="205" y="1090" width="290" height="76" rx="5" fill="white" stroke="#dc2626"/>
|
||||
<text x="232" y="1120" class="port mono" text-anchor="start">uplink-dave</text>
|
||||
<rect x="459" y="1102" width="20" height="20" rx="4" fill="#dc2626" stroke="none"/>
|
||||
<text x="469" y="1116" class="port" text-anchor="middle" style="fill:white">H</text>
|
||||
<text x="350" y="1146" class="small" text-anchor="middle">Mellanox ConnectX-4 · 100G QSFP28</text>
|
||||
<rect x="585" y="1090" width="290" height="76" rx="5" fill="white" stroke="#16a34a"/>
|
||||
<text x="612" y="1120" class="port mono" text-anchor="start">host-jim</text>
|
||||
<rect x="839" y="1102" width="20" height="20" rx="4" fill="#16a34a" stroke="none"/>
|
||||
<text x="849" y="1116" class="port" text-anchor="middle" style="fill:white">F</text>
|
||||
<text x="730" y="1146" class="small" text-anchor="middle">Intel I211 · 1G RJ45</text>
|
||||
<rect x="965" y="1090" width="290" height="76" rx="5" fill="white" stroke="#db2777"/>
|
||||
<text x="992" y="1120" class="port mono" text-anchor="start">river-jim-phy</text>
|
||||
<rect x="1219" y="1102" width="20" height="20" rx="4" fill="#db2777" stroke="none"/>
|
||||
<text x="1229" y="1116" class="port" text-anchor="middle" style="fill:white">G</text>
|
||||
<text x="1110" y="1146" class="small" text-anchor="middle">Intel I211 · 1G RJ45</text>
|
||||
<text x="95" y="1220" class="sub" text-anchor="start">Matching letter badges identify cable ends. Port labels describe configured use; they do not indicate current link state.</text>
|
||||
</g></svg>
|
||||
|
After Width: | Height: | Size: 38 KiB |
@@ -0,0 +1,608 @@
|
||||
{
|
||||
"view": "Front view; rack order is brian, jim, dave, then palace several U below. Socket order is physical; spacing and chassis widths are schematic. Stream sits physically left of brian and is drawn above it.",
|
||||
"sources": {
|
||||
"brian": "https://dl.ui.com/qig/usw-pro-xg-8-poe/",
|
||||
"jim": "https://mikrotik.com/product/CRS326-24G-2SplusRM",
|
||||
"dave": "https://mikrotik.com/product/crs504_4xq_in",
|
||||
"stream": "https://www.gowinfanless.com/news/blogs/the-r86s-revolution-low-power-2-5gbe-and-10gbe-networking"
|
||||
},
|
||||
"brian": {
|
||||
"model": "USW-Pro-XG-8-PoE",
|
||||
"ports": [
|
||||
{
|
||||
"socket": "1",
|
||||
"interface": "Port 1",
|
||||
"label": "fort",
|
||||
"role": "Configured port label",
|
||||
"peer": "fort",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "2",
|
||||
"interface": "Port 2",
|
||||
"label": "castle",
|
||||
"role": "Configured port label",
|
||||
"peer": "castle",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "3",
|
||||
"interface": "Port 3",
|
||||
"label": "portcullis test",
|
||||
"role": "portcullis test connection",
|
||||
"peer": "portcullis",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "4",
|
||||
"interface": "Port 4",
|
||||
"label": "portcullis KVM",
|
||||
"role": "Second JetKVM",
|
||||
"peer": "portcullis-kvm",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "5",
|
||||
"interface": "Port 5",
|
||||
"label": "jim P7",
|
||||
"role": "Backup fabric trunk",
|
||||
"peer": "jim",
|
||||
"peer_port": "7"
|
||||
},
|
||||
{
|
||||
"socket": "6",
|
||||
"interface": "Port 6",
|
||||
"label": "Digiweb ONT",
|
||||
"role": "Native VLAN 140; tagged VLAN 10",
|
||||
"peer": "Digiweb ONT",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "7",
|
||||
"interface": "Port 7",
|
||||
"label": "wave",
|
||||
"role": "AP trunk",
|
||||
"peer": "wave",
|
||||
"peer_port": "eth0"
|
||||
},
|
||||
{
|
||||
"socket": "8",
|
||||
"interface": "Port 8",
|
||||
"label": "vibe",
|
||||
"role": "AP trunk",
|
||||
"peer": "vibe",
|
||||
"peer_port": "ether1"
|
||||
},
|
||||
{
|
||||
"socket": "9",
|
||||
"interface": "SFP+ 1",
|
||||
"label": "dave Q3.2",
|
||||
"role": "LAG member 1",
|
||||
"peer": "dave",
|
||||
"peer_port": "Q3.2"
|
||||
},
|
||||
{
|
||||
"socket": "10",
|
||||
"interface": "SFP+ 2",
|
||||
"label": "dave Q3.4",
|
||||
"role": "LAG member 2",
|
||||
"peer": "dave",
|
||||
"peer_port": "Q3.4"
|
||||
}
|
||||
]
|
||||
},
|
||||
"jim": {
|
||||
"model": "CRS326-24G-2S+RM",
|
||||
"ports": [
|
||||
{
|
||||
"socket": "1",
|
||||
"interface": "ether1",
|
||||
"default_name": "ether1",
|
||||
"label": "shytzel LAG",
|
||||
"role": "Configured member of shytzel-downlink",
|
||||
"peer": "shytzel",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "2",
|
||||
"interface": "ether2",
|
||||
"default_name": "ether2",
|
||||
"label": "shytzel LAG",
|
||||
"role": "Configured member of shytzel-downlink",
|
||||
"peer": "shytzel",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "3",
|
||||
"interface": "ether3",
|
||||
"default_name": "ether3",
|
||||
"label": "shytzel LAG",
|
||||
"role": "Configured member of shytzel-downlink",
|
||||
"peer": "shytzel",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "4",
|
||||
"interface": "ether4",
|
||||
"default_name": "ether4",
|
||||
"label": "shytzel LAG",
|
||||
"role": "Configured member of shytzel-downlink",
|
||||
"peer": "shytzel",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "5",
|
||||
"interface": "palace",
|
||||
"default_name": "ether5",
|
||||
"label": "palace host",
|
||||
"role": "Core / netboot trunk",
|
||||
"peer": "palace",
|
||||
"peer_port": "host-jim"
|
||||
},
|
||||
{
|
||||
"socket": "6",
|
||||
"interface": "stream",
|
||||
"default_name": "ether6",
|
||||
"label": "stream LAN",
|
||||
"role": "Backup RSTP trunk",
|
||||
"peer": "stream",
|
||||
"peer_port": "lan-jim"
|
||||
},
|
||||
{
|
||||
"socket": "7",
|
||||
"interface": "brian-downlink",
|
||||
"default_name": "ether7",
|
||||
"label": "brian P5",
|
||||
"role": "Backup fabric trunk",
|
||||
"peer": "brian",
|
||||
"peer_port": "RJ45 5"
|
||||
},
|
||||
{
|
||||
"socket": "8",
|
||||
"interface": "palace-kvm",
|
||||
"default_name": "ether8",
|
||||
"label": "JetKVM",
|
||||
"role": "lo access",
|
||||
"peer": "palace-kvm",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "9",
|
||||
"interface": "ether9",
|
||||
"default_name": "ether9",
|
||||
"label": "spare",
|
||||
"role": "Unbridged spare",
|
||||
"peer": null,
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "10",
|
||||
"interface": "castle",
|
||||
"default_name": "ether10",
|
||||
"label": "castle",
|
||||
"role": "Backup attachment, normally down",
|
||||
"peer": "castle",
|
||||
"peer_port": "et2.5g"
|
||||
},
|
||||
{
|
||||
"socket": "11",
|
||||
"interface": "fort",
|
||||
"default_name": "ether11",
|
||||
"label": "fort",
|
||||
"role": "lo access (configured label)",
|
||||
"peer": "fort",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "12",
|
||||
"interface": "laptop-dock",
|
||||
"default_name": "ether12",
|
||||
"label": "dock",
|
||||
"role": "lo access (configured label)",
|
||||
"peer": "laptop-dock",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "13",
|
||||
"interface": "pronter",
|
||||
"default_name": "ether13",
|
||||
"label": "pronter",
|
||||
"role": "lo access (configured label)",
|
||||
"peer": "pronter",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "14",
|
||||
"interface": "untrusted-sta1",
|
||||
"default_name": "ether14",
|
||||
"label": "guest",
|
||||
"role": "Untrusted access",
|
||||
"peer": "untrusted-sta1",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "15",
|
||||
"interface": "ether15",
|
||||
"default_name": "ether15",
|
||||
"label": "unmapped",
|
||||
"role": "lo access; peer unconfirmed",
|
||||
"peer": null,
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "16",
|
||||
"interface": "ether16",
|
||||
"default_name": "ether16",
|
||||
"label": "unmapped",
|
||||
"role": "lo access; peer unconfirmed",
|
||||
"peer": null,
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "17",
|
||||
"interface": "ether17",
|
||||
"default_name": "ether17",
|
||||
"label": "unmapped",
|
||||
"role": "lo access; peer unconfirmed",
|
||||
"peer": null,
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "18",
|
||||
"interface": "ether18",
|
||||
"default_name": "ether18",
|
||||
"label": "unmapped",
|
||||
"role": "lo access; peer unconfirmed",
|
||||
"peer": null,
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "19",
|
||||
"interface": "ether19",
|
||||
"default_name": "ether19",
|
||||
"label": "unmapped",
|
||||
"role": "lo access; peer unconfirmed",
|
||||
"peer": null,
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "20",
|
||||
"interface": "ether20",
|
||||
"default_name": "ether20",
|
||||
"label": "unmapped",
|
||||
"role": "lo access; peer unconfirmed",
|
||||
"peer": null,
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "21",
|
||||
"interface": "stream-wan",
|
||||
"default_name": "ether21",
|
||||
"label": "stream WAN",
|
||||
"role": "Untagged VLAN 130 handoff",
|
||||
"peer": "stream",
|
||||
"peer_port": "wan"
|
||||
},
|
||||
{
|
||||
"socket": "22",
|
||||
"interface": "river",
|
||||
"default_name": "ether22",
|
||||
"label": "river backup",
|
||||
"role": "Dedicated router trunk",
|
||||
"peer": "palace",
|
||||
"peer_port": "river-jim-phy"
|
||||
},
|
||||
{
|
||||
"socket": "23",
|
||||
"interface": "ups",
|
||||
"default_name": "ether23",
|
||||
"label": "UPS",
|
||||
"role": "lo access",
|
||||
"peer": "ups",
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "24",
|
||||
"interface": "dave-management",
|
||||
"default_name": "ether24",
|
||||
"label": "dave mgmt",
|
||||
"role": "Core management",
|
||||
"peer": "dave",
|
||||
"peer_port": "MGMT"
|
||||
},
|
||||
{
|
||||
"socket": "S1",
|
||||
"interface": "dave-uplink",
|
||||
"default_name": "sfp-sfpplus1",
|
||||
"label": "dave Q3.1",
|
||||
"role": "10G fabric trunk",
|
||||
"peer": "dave",
|
||||
"peer_port": "Q3.1"
|
||||
},
|
||||
{
|
||||
"socket": "S2",
|
||||
"interface": "sfp-spare",
|
||||
"default_name": "sfp-sfpplus2",
|
||||
"label": "fergal",
|
||||
"role": "10G trunk; colony staging",
|
||||
"peer": "fergal",
|
||||
"peer_port": "lan1"
|
||||
}
|
||||
]
|
||||
},
|
||||
"dave": {
|
||||
"model": "CRS504-4XQ-IN",
|
||||
"ports": [
|
||||
{
|
||||
"socket": "Q1",
|
||||
"interface": "palace",
|
||||
"default_name": "qsfp28-1-1",
|
||||
"label": "palace",
|
||||
"role": "100G; whole cage",
|
||||
"peer": "palace",
|
||||
"peer_port": "uplink-dave"
|
||||
},
|
||||
{
|
||||
"socket": "Q2",
|
||||
"interface": "castle",
|
||||
"default_name": "qsfp28-2-1",
|
||||
"label": "castle",
|
||||
"role": "100G; whole cage",
|
||||
"peer": "castle",
|
||||
"peer_port": "et100g"
|
||||
},
|
||||
{
|
||||
"socket": "Q3.1",
|
||||
"interface": "jim-downlink",
|
||||
"default_name": "qsfp28-3-1",
|
||||
"label": "jim SFP+1",
|
||||
"role": "10G breakout lane 1",
|
||||
"peer": "jim",
|
||||
"peer_port": "S1"
|
||||
},
|
||||
{
|
||||
"socket": "Q3.2",
|
||||
"interface": "brian1",
|
||||
"default_name": "qsfp28-3-2",
|
||||
"label": "brian SFP+1",
|
||||
"role": "10G breakout lane 2; LAG",
|
||||
"peer": "brian",
|
||||
"peer_port": "9"
|
||||
},
|
||||
{
|
||||
"socket": "Q3.3",
|
||||
"interface": "stream",
|
||||
"default_name": "qsfp28-3-3",
|
||||
"label": "stream LAN",
|
||||
"role": "10G breakout lane 3",
|
||||
"peer": "stream",
|
||||
"peer_port": "lan-dave"
|
||||
},
|
||||
{
|
||||
"socket": "Q3.4",
|
||||
"interface": "brian2",
|
||||
"default_name": "qsfp28-3-4",
|
||||
"label": "brian SFP+2",
|
||||
"role": "10G breakout lane 4; LAG",
|
||||
"peer": "brian",
|
||||
"peer_port": "10"
|
||||
},
|
||||
{
|
||||
"socket": "Q4",
|
||||
"interface": "qsfp28-4-1",
|
||||
"default_name": "qsfp28-4-1",
|
||||
"label": "unmapped",
|
||||
"role": "No identified peer",
|
||||
"peer": null,
|
||||
"peer_port": null
|
||||
},
|
||||
{
|
||||
"socket": "MGMT",
|
||||
"interface": "management",
|
||||
"default_name": "ether1",
|
||||
"label": "jim P24",
|
||||
"role": "100M management RJ45",
|
||||
"peer": "jim",
|
||||
"peer_port": "24"
|
||||
}
|
||||
]
|
||||
},
|
||||
"links": [
|
||||
{
|
||||
"id": "A",
|
||||
"from": [
|
||||
"brian",
|
||||
"9"
|
||||
],
|
||||
"to": [
|
||||
"dave",
|
||||
"Q3.2"
|
||||
],
|
||||
"description": "10G; first LAG member"
|
||||
},
|
||||
{
|
||||
"id": "B",
|
||||
"from": [
|
||||
"brian",
|
||||
"10"
|
||||
],
|
||||
"to": [
|
||||
"dave",
|
||||
"Q3.4"
|
||||
],
|
||||
"description": "10G; second LAG member"
|
||||
},
|
||||
{
|
||||
"id": "C",
|
||||
"from": [
|
||||
"brian",
|
||||
"5"
|
||||
],
|
||||
"to": [
|
||||
"jim",
|
||||
"7"
|
||||
],
|
||||
"description": "1G; redundant fabric trunk"
|
||||
},
|
||||
{
|
||||
"id": "D",
|
||||
"from": [
|
||||
"jim",
|
||||
"S1"
|
||||
],
|
||||
"to": [
|
||||
"dave",
|
||||
"Q3.1"
|
||||
],
|
||||
"description": "10G; direct fabric trunk"
|
||||
},
|
||||
{
|
||||
"id": "E",
|
||||
"from": [
|
||||
"jim",
|
||||
"24"
|
||||
],
|
||||
"to": [
|
||||
"dave",
|
||||
"MGMT"
|
||||
],
|
||||
"description": "100M; dave management"
|
||||
},
|
||||
{
|
||||
"id": "F",
|
||||
"from": [
|
||||
"jim",
|
||||
"5"
|
||||
],
|
||||
"to": [
|
||||
"palace",
|
||||
"host-jim"
|
||||
],
|
||||
"description": "1G; host core / netboot"
|
||||
},
|
||||
{
|
||||
"id": "G",
|
||||
"from": [
|
||||
"jim",
|
||||
"22"
|
||||
],
|
||||
"to": [
|
||||
"palace",
|
||||
"river-jim-phy"
|
||||
],
|
||||
"description": "1G; river backup trunk"
|
||||
},
|
||||
{
|
||||
"id": "H",
|
||||
"from": [
|
||||
"dave",
|
||||
"Q1"
|
||||
],
|
||||
"to": [
|
||||
"palace",
|
||||
"uplink-dave"
|
||||
],
|
||||
"description": "100G; host PF and guest VFs"
|
||||
},
|
||||
{
|
||||
"id": "I",
|
||||
"from": [
|
||||
"stream",
|
||||
"ETH0"
|
||||
],
|
||||
"to": [
|
||||
"jim",
|
||||
"21"
|
||||
],
|
||||
"description": "1G WAN; 2.5G socket on stream"
|
||||
},
|
||||
{
|
||||
"id": "J",
|
||||
"from": [
|
||||
"stream",
|
||||
"ETH1"
|
||||
],
|
||||
"to": [
|
||||
"jim",
|
||||
"6"
|
||||
],
|
||||
"description": "1G backup LAN; 2.5G socket on stream"
|
||||
},
|
||||
{
|
||||
"id": "K",
|
||||
"from": [
|
||||
"stream",
|
||||
"SFP+ R"
|
||||
],
|
||||
"to": [
|
||||
"dave",
|
||||
"Q3.3"
|
||||
],
|
||||
"description": "10G preferred LAN"
|
||||
}
|
||||
],
|
||||
"stream": {
|
||||
"model": "BROUNION R86S",
|
||||
"ports": [
|
||||
{
|
||||
"socket": "ETH0",
|
||||
"interface": "wan",
|
||||
"label": "wan",
|
||||
"role": "1G connection to jim 21",
|
||||
"nic": "Intel I225-V",
|
||||
"capacity": "2.5G"
|
||||
},
|
||||
{
|
||||
"socket": "ETH1",
|
||||
"interface": "lan-jim",
|
||||
"label": "lan-jim",
|
||||
"role": "1G connection to jim 6",
|
||||
"nic": "Intel I225-V",
|
||||
"capacity": "2.5G"
|
||||
},
|
||||
{
|
||||
"socket": "ETH2",
|
||||
"interface": "et2",
|
||||
"label": "et2",
|
||||
"role": "Unused",
|
||||
"nic": "Intel I225-V",
|
||||
"capacity": "2.5G"
|
||||
},
|
||||
{
|
||||
"socket": "SFP+ R",
|
||||
"interface": "lan-dave",
|
||||
"label": "lan-dave",
|
||||
"role": "10G connection to dave Q3.3",
|
||||
"nic": "Mellanox ConnectX-3",
|
||||
"capacity": "10G"
|
||||
},
|
||||
{
|
||||
"socket": "SFP+ L",
|
||||
"interface": "et5",
|
||||
"label": "et5",
|
||||
"role": "Unused",
|
||||
"nic": "Mellanox ConnectX-3",
|
||||
"capacity": "10G"
|
||||
}
|
||||
]
|
||||
},
|
||||
"palace": {
|
||||
"ports": [
|
||||
{
|
||||
"socket": "uplink-dave",
|
||||
"nic": "Mellanox ConnectX-4",
|
||||
"capacity": "100G QSFP28"
|
||||
},
|
||||
{
|
||||
"socket": "host-jim",
|
||||
"nic": "Intel I211",
|
||||
"capacity": "1G RJ45"
|
||||
},
|
||||
{
|
||||
"socket": "river-jim-phy",
|
||||
"nic": "Intel I211",
|
||||
"capacity": "1G RJ45"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
+102
-19
@@ -18,7 +18,7 @@ SR-IOV VFs, PCI NVMe drives and LVM disks.
|
||||
| Host storage | 500 GB Samsung SSD 860 EVO containing the EFI partition and the `main` LVM thin pool |
|
||||
| Bulk storage | Three 8 TB Seagate IronWolf disks in the `hdds` VG, providing the RAID-backed `hdd-storage` and `frigate` LVs |
|
||||
| NVMe storage | Three 2 TB Samsung NVMe devices passed through to `cellar`; SPDK combines them as the `NVMeRaid` RAID 0 device |
|
||||
| Network / graphics | Mellanox ConnectX-4 100G adapter with four SR-IOV VFs, two Intel I211 Gigabit Ethernet controllers, and an AMD Radeon RX 550/560-family GPU |
|
||||
| Network / graphics | Mellanox ConnectX-4 100G adapter using SR-IOV, two Intel I211 Gigabit Ethernet controllers (the only BASE-T NICs), and an AMD Radeon RX 550/560-family GPU |
|
||||
|
||||
## Role
|
||||
|
||||
@@ -34,33 +34,116 @@ See the consolidated [network assignments](../../networking.md#box-assignments)
|
||||
|
||||
## Networking
|
||||
|
||||
100G `et100g` (mlx5, MTU 9000) uplinks to the `dave` switch and carries `lan-hi` (VLAN 100, the
|
||||
`hi` assignment). A udev rule creates four SR-IOV VFs on the PF:
|
||||
100G `uplink-dave` (mlx5, MTU 9000) connects to the `dave` switch and carries `lan-hi` (VLAN 100,
|
||||
the `hi` assignment). A udev rule creates five SR-IOV VFs on the PF. Two physically separate Intel
|
||||
connections lead to `jim`: `host-jim` serves `palace` and `sfh`, while `river-jim-phy` is exported
|
||||
exclusively to `river`.
|
||||
|
||||
| VF | Consumer | VLAN handling |
|
||||
|---|---|---|
|
||||
| 0 | `cellar` | `hi` |
|
||||
| 1 | `river` | untagged VF; `river` tags its LAN and WAN VLANs |
|
||||
| 2 | `sfh` | `hi` |
|
||||
| 3 | `sfh` container MACVLAN parent | `hi` |
|
||||
- `lan-core` is a bridge with the `core` assignment (no gateway); the 1G `host-jim` connection and
|
||||
the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used for VM netboot and
|
||||
`lo` clients.
|
||||
- The separate 1G `river-jim-phy` connection is exported exclusively to `river` as the
|
||||
passthru-mode macvtap `river-jim-tap`; the guest names it `lan-jim`. Direct PCI passthrough is
|
||||
blocked by its shared X399 IOMMU group; see [PCI isolation](#pci-isolation).
|
||||
|
||||
- `lan-core` is a bridge with the `core` assignment (no gateway); the 1G
|
||||
`lan-core-phy` and the `lan-lo-phy` VLAN ride on it. `lan-lo` is a second, L3-less bridge used
|
||||
for VM netboot and `lo` clients.
|
||||
- The 1G `et1g0` (igb) is exported to `river` as a passthru-mode macvtap (`vm-et1g0`) — river sees
|
||||
it as `wan-old`.
|
||||
### PCI isolation
|
||||
|
||||
The Intel I211 controllers at `04:00.0` (`river-jim-phy`) and `06:00.0` (`host-jim`) share IOMMU
|
||||
group 13 with chipset USB/SATA controllers, PCIe bridges and the wireless controller. Passing
|
||||
only the router NIC through VFIO would leave the group nonviable; detaching the whole group
|
||||
would remove essential host devices. The dedicated passthru-mode macvtap keeps the NIC host-owned.
|
||||
Direct passthrough would need a NIC or firmware layout that isolates the controller.
|
||||
|
||||
Keep both Intel connections: `host-jim` provides independent host management and VM netboot,
|
||||
while `river-jim-phy` carries the router's redundant trunk. The host physical router interface and
|
||||
its macvtap both use `hiMTU`; the macvtap enables all-multicast reception so guest RSTP can receive
|
||||
BPDUs without a guest-to-host receive-filter synchronization service.
|
||||
|
||||
### End-to-end interface topology
|
||||
|
||||
```text
|
||||
dave
|
||||
└─ palace uplink-dave (ConnectX-4 PF)
|
||||
├─ VLAN 100 ───────────────────────────────> palace lan-hi
|
||||
├─ VF 0 cellar-hi ────────────────────────> cellar lan-hi
|
||||
├─ VF 1 river-rdma ───────────────────────> river rdma-hi ──> cellar NVMe/RDMA
|
||||
├─ VF 2 sfh-hi ───────────────────────────> sfh lan-hi ─────> cellar NVMe/RDMA
|
||||
├─ VF 3 sfh-ctrs ─────────────────────────> sfh ctrs-hi
|
||||
│ ├─> hass host0
|
||||
│ └─> unifi host0
|
||||
└─ VF 4 river-dave ───────────────────────> river lan-dave ──┐
|
||||
├─> river lan (RSTP)
|
||||
jim │
|
||||
├─ palace river-jim-phy ── river-jim-tap ─────> river lan-jim ───┘
|
||||
└─ palace host-jim ──> palace lan-core
|
||||
├─ sfh-core-tap ───────────────────────────> sfh ctrs-core ──> unifi lan-core
|
||||
└─ VLAN 110 lan-lo-phy ──> palace lan-lo
|
||||
└─ sfh-lo-tap ──────────────────────────> sfh ctrs-lo
|
||||
├─> sfh PXE
|
||||
└─> hass lan-lo
|
||||
```
|
||||
|
||||
Inside `river`, `lan-dave` and `lan-jim` terminate on the shared `lan` RSTP bridge. `lan-core` is a
|
||||
MACVLAN above it; `lan-hi`, `lan-lo` and `lan-untrusted` are VLAN uppers. `river` also adds its
|
||||
Digiweb VLANs 10 and 140 to this bridge. `rdma-hi` is deliberately outside it.
|
||||
|
||||
VF 4 explicitly enables trust and disables MAC spoof checking for bridge traffic, which can use
|
||||
MACs other than the VF's assigned address. Its automatic link state follows the physical port;
|
||||
the storage VFs retain forced link state so local VF-to-VF storage can continue without carrier.
|
||||
The firmware must expose enough VFs before the udev allocation can succeed.
|
||||
|
||||
### Interface map
|
||||
|
||||
The physical connections terminate on `palace`; PCI VFs and virtual interfaces hand traffic
|
||||
from the host to its guests. Interface names in the destination columns belong to that box.
|
||||
|
||||
#### Physical connections
|
||||
|
||||
| Switch port | Host interface | Host networks | Guest handoff |
|
||||
|---|---|---|---|
|
||||
| `dave` `palace` | `uplink-dave` | VLAN 100 → `lan-hi` | Five PCI VFs, listed below |
|
||||
| `jim` `palace` | `host-jim` | Native `core` → `lan-core`; VLAN 110 → `lan-lo-phy` → `lan-lo` | `sfh-core-tap` and `sfh-lo-tap` |
|
||||
| `jim` `river` | `river-jim-phy` | No host L3 configuration | `river-jim-tap` macvtap → `river` `lan-jim` |
|
||||
|
||||
#### ConnectX-4 VFs
|
||||
|
||||
All five VFs belong to `uplink-dave`. VLAN filtering and carrier policy are applied on the PF.
|
||||
|
||||
| VF / PCI BDF | QEMU device | Guest | Guest interface | VLAN / carrier | Role |
|
||||
|---|---|---|---|---|---|
|
||||
| 0 / `44:00.1` | `cellar-hi` | [cellar](cellar.md) | `lan-hi` | 100 / forced up | Storage target and management |
|
||||
| 1 / `44:00.2` | `river-rdma` | [river](river.md#storage) | `rdma-hi` | 100 / forced up | Dedicated NVMe/RDMA root path |
|
||||
| 2 / `44:00.3` | `sfh-hi` | [sfh](sfh/README.md) | `lan-hi` | 100 / forced up | NVMe/RDMA root and management |
|
||||
| 3 / `44:00.4` | `sfh-ctrs` | [sfh](sfh/README.md#networking) | `ctrs-hi` | 100 / forced up | Container `hi` network |
|
||||
| 4 / `44:00.5` | `river-dave` | [river](river.md#virtual-machine-and-network-attachment) | `lan-dave` | Unfiltered trunk / automatic | Preferred RSTP attachment |
|
||||
|
||||
#### Virtual Ethernet handoffs
|
||||
|
||||
| Host interface | Kind / parent | Guest interface | Downstream use |
|
||||
|---|---|---|---|
|
||||
| `river-jim-tap` | Passthru macvtap on `river-jim-phy` | `river` `lan-jim` | Backup RSTP attachment |
|
||||
| `sfh-core-tap` | TAP on `lan-core` | `sfh` `ctrs-core` | `unifi` `lan-core` |
|
||||
| `sfh-lo-tap` | TAP on `lan-lo` | `sfh` `ctrs-lo` | PXE boot; `hass` `lan-lo` |
|
||||
|
||||
`sfh` keeps no L3 configuration on the three `ctrs-*` parents. Its nspawn configuration creates the
|
||||
container-side names shown above; `host0` remains the default `hi` interface in both containers.
|
||||
See [the `sfh` page](sfh/README.md#networking) for its container-oriented view and
|
||||
[failover validation](router-failover.md) for tested failure scenarios and maintenance checks.
|
||||
|
||||
## VMs
|
||||
|
||||
The RAID-backed `hdds` LVs need `dm-raid` loaded in the initrd: LVM's initial autoactivation can
|
||||
run before stage-two module loading. `sfh` requires the `frigate` LV before QEMU can start.
|
||||
|
||||
| VM | vCPUs | RAM | Passthrough | Notes |
|
||||
|---|---|---|---|---|
|
||||
| `cellar` | 8c × 2t | 16 GiB | VF 0 (`44:00.1`); NVMe `41:00.0`–`43:00.0` | pinned to NUMA node 1; split IRQ chip + vIOMMU |
|
||||
| `river` | 3c × 2t | 4 GiB | VF 1 (`44:00.2`); macvtap `vm-et1g0` | only an ESP local disk (plus an installer ISO) — root is NVMe-oF from `cellar` |
|
||||
| `sfh` | 8c × 2t | 32 GiB | VF 2 (`44:00.3`), VF 3 (`44:00.4`); two USB host ports | no boot disk — netboots; gets the `hdds/frigate` LV |
|
||||
| `cellar` | 8c × 2t | 16 GiB | `cellar-hi` VF 0 (`44:00.1`); NVMe `41:00.0`–`43:00.0` | pinned to NUMA node 1; split IRQ chip + vIOMMU |
|
||||
| `river` | 3c × 2t | 4 GiB | `river-rdma` VF 1 (`44:00.2`), `river-dave` VF 4 (`44:00.5`), `river-jim-tap` | only an ESP local disk (plus an installer ISO) — root is NVMe-oF from `cellar` |
|
||||
| `sfh` | 8c × 2t | 32 GiB | `sfh-hi` VF 2 (`44:00.3`), `sfh-ctrs` VF 3 (`44:00.4`); two USB host ports | no boot disk — netboots; gets the `hdds/frigate` LV |
|
||||
|
||||
Boot ordering is enforced with systemd dependencies: `vm@river` waits for `cellar`'s SSH port (and
|
||||
for the `vm-et1g0` device), and `vm@sfh` waits for `river` — storage first, then the router, then
|
||||
everything that boots off both.
|
||||
for the `river-jim-tap` device), and `vm@sfh` waits for `river` — storage first, then the router,
|
||||
then everything that boots off both.
|
||||
|
||||
## Notable config files
|
||||
|
||||
|
||||
+35
-12
@@ -24,9 +24,10 @@ See the consolidated [network assignments](../../networking.md#box-assignments)
|
||||
### Link and addressing
|
||||
|
||||
`services.pppd` peer `digiweb` attaches directly to `wan-pon-isp`, the raw ISP VLAN 10
|
||||
(`vlans.pon-isp`) trunked untranslated from the ONT through `brian` and `dave`; see
|
||||
[switches.md](switches.md). The netdev has no L3 configuration and uses `MTUBytes=1508`, leaving a
|
||||
clean `mtu`/`mru 1500` after PPPoE overhead.
|
||||
(`vlans.pon-isp`) trunked untranslated from the ONT. It normally reaches `river` through `brian`
|
||||
and `dave`; the configured RSTP path through `jim` is intended to provide redundancy after
|
||||
successful deployment and validation. See [switches.md](switches.md). The netdev has no L3
|
||||
configuration and uses `MTUBytes=1508`, leaving a clean `mtu`/`mru 1500` after PPPoE overhead.
|
||||
|
||||
The static `84.203.124.128` address is requested through IPCP. The provider-wide credentials are
|
||||
deliberately not secret; the peer persists indefinitely with LCP echo monitoring and ignores
|
||||
@@ -46,23 +47,45 @@ every WAN flap.
|
||||
|
||||
### Traffic shaping
|
||||
|
||||
The shared `wan-ifb` ingress-shaping pieces are inert here. CAKE is specific to `stream`, and
|
||||
`networkd-dispatcher` is `mkForce false` pending scheduling tests.
|
||||
CAKE and the `wan-ifb` network configuration remain specific to `stream`. The shared IFB netdev
|
||||
exists here but has no matching `.network` file to trigger the ingress-mirroring rule. The source
|
||||
enables the shared `networkd-dispatcher` for RSTP setup.
|
||||
|
||||
## Platform
|
||||
|
||||
### Virtual machine and network attachment
|
||||
|
||||
The 100G `lan` NIC is VF 1 of `palace`'s `et100g` (MTU 9000), with every router VLAN tagged on top
|
||||
as `55-lan`. A macvtap of `palace`'s 1G `et1g0` remains as the old `wan-old` path without L3
|
||||
configuration. Deployments use the `hi` assignment.
|
||||
The router's ordinary networks sit above the `lan` RSTP bridge. Its preferred port, `lan-dave`, is
|
||||
an unfiltered ConnectX-4 VF trunk; `lan-jim` is the 1G `river-jim-tap` macvtap exported by
|
||||
`palace`. Native `core`, the client VLANs and Digiweb VLANs 10/140 use `dave` normally and fail
|
||||
over to `jim`. The shared policy is documented in [Router HA](../../networking.md#router-ha);
|
||||
[failover validation](router-failover.md) covers attachment and switch outages.
|
||||
|
||||
The bridge uses the trunk VF's assigned MAC so the ConnectX-4's local switching can deliver
|
||||
traffic from `palace` and the other VFs to the router. On `lan-hi` and `rdma-hi`, `arp_ignore=1`
|
||||
and `arp_announce=2` keep router and storage addresses on their respective interfaces; otherwise
|
||||
the storage VF can answer ARP for the router VIP and attract traffic to the wrong interface.
|
||||
|
||||
The separate ConnectX-4 VF `rdma-hi` is not a bridge port. It has a PF-applied VLAN 100 and exists
|
||||
only for storage traffic.
|
||||
|
||||
Ordinary Ethernet traverses the guest software bridge; this does not require mlx5
|
||||
switchdev or host representors. The bridge disables LRO on its ports; RDMA stays outside this
|
||||
path.
|
||||
|
||||
### Storage
|
||||
|
||||
The VM's local disk holds only an ESP; `/nix` and `/persist` are LVs on `cellar`'s
|
||||
`nqn.2016-06.io.spdk:river` namespace over RDMA. The initrd brings up `lan-hi`
|
||||
with `roceBootModules`, and `KeepConfiguration=static` prevents networkd from dropping the address
|
||||
during reconfiguration. An installer ISO remains attached.
|
||||
`nqn.2016-06.io.spdk:river` namespace over RDMA. The initrd brings up `rdma-hi` with the dedicated
|
||||
`rdma` assignment and a link-scoped host route to `cellar`, alongside `roceBootModules`.
|
||||
|
||||
NixOS copies the main system's `.link` units into the systemd initrd, so the `rdma-hi` rename is
|
||||
shared, but its `.network` and `.netdev` units are separate. `river` therefore declares the storage
|
||||
assignment and route once under `boot.initrd.systemd.network` and again under the main
|
||||
`systemd.network`. The systemd initrd leaves its kernel network state in place during the stage-two
|
||||
handover; `KeepConfiguration=static` on the main `.network` prevents networkd from dropping and
|
||||
recreating the static address and route underneath the mounted NVMe-oF root. An installer ISO
|
||||
remains attached.
|
||||
|
||||
## Netboot
|
||||
|
||||
@@ -76,4 +99,4 @@ assignment, restricted to the `hi` and `lo` prefixes.
|
||||
- [`nixos/boxes/home/routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix) —
|
||||
shared router definition (assignments, firewall/NAT, `as211024`).
|
||||
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
|
||||
the VM definition on `palace` (VF 1, macvtap, ESP disk).
|
||||
the VM definition on `palace` (dedicated RDMA VF, trunk VF, macvtap and ESP disk).
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
# Router failover validation
|
||||
|
||||
Failure-scenario checks for the deployed home router pair. The design lives in
|
||||
[Router HA](../../networking.md#router-ha), the physical paths in [switches.md](switches.md),
|
||||
and the host/VF layout in [palace.md](palace.md) and [river.md](river.md).
|
||||
|
||||
## Validated scenarios
|
||||
|
||||
These checks were performed on 2026-11-15. WAN and ONT probes check Ethernet path availability;
|
||||
reads from `river`'s root LV and probes over `rdma-hi` check its separate storage path.
|
||||
|
||||
| Scenario | Test | Validated behavior |
|
||||
|---|---|---|
|
||||
| Router startup | Boot both routers | Kea starts without retries; the configured bridge MAC, ARP policy and administrative RSTP costs are applied |
|
||||
| Primary router offline | Reboot its host | `stream` takes over both IPv4 and IPv6 VIPs, then returns to standby when `river` recovers |
|
||||
| Preferred guest port down | Disable `river`'s `lan-dave`, then restore it | `lan-jim` becomes root/forwarding; WAN, ONT and 9000-byte RDMA pings pass while direct root-LV reads continue for 20 seconds |
|
||||
| Backup guest port down | Disable `river`'s `lan-jim`, then restore it | `lan-dave` stays root/forwarding; WAN, ONT, jumbo RDMA and root-LV read checks pass |
|
||||
| Physical `palace` attachment lost | Disable `dave`'s `palace` port for 30 seconds with automatic restoration | `lan-dave` loses carrier and `lan-jim` becomes root/forwarding; forced-link storage VFs retain local RDMA and root-LV reads succeed |
|
||||
| Whole `dave` outage | Reboot the switch | `jim` becomes root and forwards on `brian-downlink`; Digiweb WAN and ONT access use `jim`–`brian` while local RDMA reads continue |
|
||||
| Whole `jim` outage | Reboot the switch | `river` continues using `dave`; WAN, ONT and root-storage read probes succeed |
|
||||
| Restoration | Restore each failed attachment or switch | The normal RSTP preferences return and `palace` is reachable on `hi` again |
|
||||
|
||||
Brief WAN/ONT probe losses occur during the tested `dave` tree transitions; subsequent probes
|
||||
succeed on the backup tree and after restoration. The monitors recorded no storage-read
|
||||
failures or matched kernel NVMe errors. The whole-switch test demonstrates the backup WAN path
|
||||
without relying on the direct `jim`–`dave` trunk.
|
||||
|
||||
## Test limits
|
||||
|
||||
The storage checks exercise reads, not write integrity or long-duration endurance. They validate
|
||||
local VF-to-VF RDMA during physical carrier loss, not failure of the ConnectX-4 itself. Storage
|
||||
still depends on the adapter, `palace` and `cellar`; see [river's storage](river.md#storage).
|
||||
|
||||
`brian`'s live RSTP port state was not captured during the outage tests. Its backup
|
||||
connection is validated by successful WAN and ONT traffic while `dave` is offline.
|
||||
|
||||
Disabling only the guest trunk VF while the host PF remains healthy is an artificial fault:
|
||||
`palace`'s `hi` return traffic can still target that local VF's MAC and become unreachable through
|
||||
`river`. During physical carrier loss, external `hi` access to the host and its guests is also
|
||||
unavailable. `palace` remains accessible on `core` through a jump connection from `stream`;
|
||||
direct routed management access depends on a working return route.
|
||||
|
||||
## Repeating the checks
|
||||
|
||||
Perform maintenance from a box independent of `palace`, such as `tower`. Before interrupting
|
||||
`palace` or `dave`, power `castle` off cleanly: its NVMe-oF root depends on `cellar` and `dave`.
|
||||
Keep a known-working boot entry and working JetKVM video and keyboard access.
|
||||
|
||||
1. Check boot defaults, system profiles, switch VLAN membership and spanning-tree state. When
|
||||
changing host and guest networking together, stage matching configurations with `deploy --boot`.
|
||||
2. Verify VF isolation, VLAN/trust/carrier policies, MTUs, the RoCE GID and the mounted NVMe
|
||||
subsystem before injecting a fault.
|
||||
3. Arrange independent automatic restoration before disabling an interface or switch port.
|
||||
Run probes and storage reads in a service that survives loss of the SSH connection; bound
|
||||
remote commands so a path change cannot leave a check waiting indefinitely.
|
||||
4. Inject one fault at a time. Monitor root/alternate port roles, native `core`, client VLANs,
|
||||
ONT access, PPPoE, DNS through the floating VIPs and storage I/O. Capture BPDUs when changing
|
||||
the topology and watch for returned own-source packets or MAC flapping.
|
||||
5. Restore the path and verify the normal RSTP roles, VIP ownership, services and management
|
||||
access before testing another fault.
|
||||
|
||||
Switch changes require confirmation; see [switches.md](switches.md#accessing-the-switches).
|
||||
@@ -31,16 +31,21 @@ See the consolidated [network assignments](../../../networking.md#box-assignment
|
||||
|
||||
## Networking
|
||||
|
||||
Four NICs, all MTU 9000 where jumbo-capable:
|
||||
| Interface | `palace` attachment | Purpose |
|
||||
|---|---|---|
|
||||
| `lan-hi` | `sfh-hi` SR-IOV VF 2 | Box's `hi` assignment and NVMe/RDMA root path; MTU 9000 |
|
||||
| `ctrs-hi` | `sfh-ctrs` SR-IOV VF 3 | L3-less MACVLAN parent for each container's `host0`; MTU 9000 |
|
||||
| `ctrs-core` | Virtio via `sfh-core-tap`, `lan-core` and `host-jim` | L3-less MACVLAN parent for UniFi's `lan-core` management attachment |
|
||||
| `ctrs-lo` | Virtio via `sfh-lo-tap`, `lan-lo` and `host-jim` | L3-less MACVLAN parent for `hass`'s `lan-lo` and the VM's PXE path |
|
||||
|
||||
- `lan-hi` — SR-IOV VF 2, the box's own `hi` assignment.
|
||||
- `lan-hi-ctrs` — SR-IOV VF 3, no L3: the MACVLAN parent for the containers' `hi` interfaces
|
||||
(`host0` inside each container).
|
||||
- `lan-core-ctrs` / `lan-lo-ctrs` — virtio NICs (bridged to palace's `lan-core` / `lan-lo`), no
|
||||
L3: MACVLAN parents for containers that need a `core` or `lo` interface.
|
||||
The two VFs separate the box's RDMA root traffic from container `hi` traffic; they do not provide
|
||||
redundant switch paths. The Virtio attachments reach the `jim`-facing `palace` host bridges. Container
|
||||
`host0` names are intentionally unchanged.
|
||||
|
||||
The per-container MACVLAN wiring lives in `systemd.nspawn.*.networkConfig` in
|
||||
[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix).
|
||||
[`sfh/default.nix`](../../../../nixos/boxes/home/palace/vms/sfh/default.nix). The
|
||||
[`palace` interface topology](../palace.md#end-to-end-interface-topology) traces these interfaces
|
||||
back through the VM handoffs to the physical switch connections.
|
||||
|
||||
## Containers
|
||||
|
||||
|
||||
@@ -47,8 +47,8 @@ Passed through from `sfh` (USB host ports on `palace`):
|
||||
|
||||
## Networking
|
||||
|
||||
MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` (the `hi` assignment,
|
||||
alt name `frigate`, default gateway via the VIP) and `lan-lo` on `lan-lo-ctrs` (the `lo`
|
||||
MACVLAN interfaces created from `sfh`'s container NICs: `host0` on `ctrs-hi` (the `hi` assignment,
|
||||
alt name `frigate`, default gateway via the VIP) and `lan-lo` on `ctrs-lo` (the `lo`
|
||||
assignment, no gateway) — the `lo` interface reaches the IoT devices (the Reolink camera lives there).
|
||||
|
||||
## Notable config files
|
||||
|
||||
@@ -29,8 +29,8 @@ management network. It is imported, listed in `sfh`'s `my.containers.instances`,
|
||||
|
||||
## Networking
|
||||
|
||||
Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `lan-hi-ctrs` carries the `hi`
|
||||
assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `lan-core-ctrs` carries
|
||||
Two MACVLAN interfaces come from `sfh`'s container NICs: `host0` on `ctrs-hi` carries the `hi`
|
||||
assignment (`unifi-ctr`, default gateway via the VIP), while `lan-core` on `ctrs-core` carries
|
||||
the gatewayless `core` assignment (`unifi-ctr-core`). The `core` interface is how the controller
|
||||
talks to `brian` and the other switches on their management network.
|
||||
|
||||
|
||||
+15
-10
@@ -16,7 +16,7 @@ redundant router pair with [`river`](river.md) and is dual-homed to both switche
|
||||
| CPU | Intel Celeron N5105 (4 cores / 4 threads) |
|
||||
| Memory | 16 GiB |
|
||||
| Storage | 512 GB Samsung SSD 970 PRO NVMe containing `/boot`, `/nix` and `/persist`; integrated 128 GB eMMC is present but is not used by the declared filesystems |
|
||||
| Network | Three Intel `igc` interfaces and a dual-port Mellanox `mlx4_en` adapter; `wan`, `lan-jim` and `lan-dave` use three of these ports |
|
||||
| Network | Three Intel I225-V 2.5G interfaces (`igc`) and a dual-port Mellanox ConnectX-3 10G SFP+ adapter (`mlx4_en`); `wan`, `lan-jim` and `lan-dave` use three of these ports |
|
||||
|
||||
The platform configuration enables `kvm-intel`, `intel_iommu=on` and Intel microcode updates.
|
||||
|
||||
@@ -39,8 +39,9 @@ lease this section assumes — see [wwan.md](wwan.md).
|
||||
|
||||
### Link and addressing
|
||||
|
||||
`wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) towards the cable modem. The modem segment is
|
||||
switch VLAN 130; `jim` handles the tag, so the box interface is untagged. See
|
||||
`wan` is a renamed igc NIC (`00:f0:cb:ee:ca:dd`) on a physically separate connection to `jim`'s
|
||||
`stream-wan` port. The Virgin Media modem connects to `shytzel`, which trunks
|
||||
VLAN 130 to `jim`; `stream-wan` delivers it untagged to the box. See
|
||||
[switches.md](switches.md) for the fabric side.
|
||||
|
||||
`DHCP=ipv4` pulls the public lease. `dhcpV4Config.UseDNS=false` points resolution at the local
|
||||
@@ -79,11 +80,15 @@ box sets:
|
||||
|
||||
## Switching (RSTP)
|
||||
|
||||
`stream` is dual-homed to both switches: `lan-jim` (igc) and `lan-dave` (mlx4_en), both MTU 9000,
|
||||
are enslaved to the `lan` bridge with `STP=true`. The explicit bridge-port costs prefer
|
||||
`lan-dave` at 10 over `lan-jim` at 100. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
|
||||
runs a patched `mstpd` and forces RSTP on `lan` once it is configured, so exactly one uplink carries
|
||||
traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfigured.)
|
||||
`stream` is dual-homed to both switches: `lan-dave` (mlx4_en) and `lan-jim` (igc), both MTU 9000,
|
||||
are ports of the shared `lan` bridge. `routing-common` owns the bridge, its STP setting and the
|
||||
explicit `dave`/`jim` costs of 10/100. [`routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix)
|
||||
runs a patched `mstpd` and forces RSTP once `lan` is configured, so exactly one uplink carries
|
||||
traffic at a time and `dave` is preferred. (The remaining NICs are renamed `et2`/`et5` and left
|
||||
unconfigured.)
|
||||
|
||||
The separate `wan` connection to `jim` is not a bridge port. Retaining it provides an isolated
|
||||
untagged DHCP handoff for Virgin Media and a simple path for a future replacement direct-IP WAN.
|
||||
|
||||
## Deployment
|
||||
|
||||
@@ -96,8 +101,8 @@ traffic at a time. (The remaining NICs are renamed `et2`/`et5` and left unconfig
|
||||
## Notable config files
|
||||
|
||||
- [`nixos/boxes/home/stream.nix`](../../../nixos/boxes/home/stream.nix) — box config: DHCP WAN,
|
||||
modem management, CAKE, `wan-online.target` gate, STP bridge.
|
||||
modem management, CAKE, `wan-online.target` gate and hardware link matching.
|
||||
- [`nixos/boxes/home/routing-common/default.nix`](../../../nixos/boxes/home/routing-common/default.nix) —
|
||||
shared router definition (index 1).
|
||||
shared router definition, including the `lan` bridge and port preferences (index 1).
|
||||
- [`nixos/boxes/home/routing-common/mstpd.nix`](../../../nixos/boxes/home/routing-common/mstpd.nix) —
|
||||
RSTP on the `lan` bridge.
|
||||
|
||||
+165
-72
@@ -1,12 +1,11 @@
|
||||
# Home switches
|
||||
|
||||
Reference for the two MikroTik switches on the home network — **jim** and **dave** — plus the
|
||||
Ubiquiti switch **brian**, and how the home boxes and the Digiweb WAN hang off them. These switches
|
||||
are **not** managed by this flake; they are configured by hand (RouterOS on jim/dave, UniFi on
|
||||
brian). It covers the physical topology, the VLAN map, and how the Digiweb WAN reaches river.
|
||||
Reference for the home switches: MikroTik boxes `jim` and `dave`, Ubiquiti switch `brian`, and
|
||||
Zyxel L2 switch `shytzel`. These switches are **not** managed by this flake; they are configured by
|
||||
hand. This page covers the physical topology, VLAN map, and WAN paths.
|
||||
|
||||
In short: the Digiweb ISP VLAN (10) is trunked straight through to river (which runs PPPoE on it),
|
||||
and the ONT's untagged management is PVID'd onto VLAN 140 at brian, its edge switch. VLAN 10 is
|
||||
In short: the Digiweb ISP VLAN (10) is trunked straight through to `river` (which runs PPPoE on it),
|
||||
and the ONT's untagged management is PVID'd onto VLAN 140 at `brian`, its edge switch. VLAN 10 is
|
||||
carried untranslated because a single ONT makes it unique on the fabric — see
|
||||
[the WAN path](#the-digiweb-wan-path-trunked-vlan-10--pvid-140) and
|
||||
[why not translation](#why-not-translation-for-one-ont). The router side lives in
|
||||
@@ -16,72 +15,116 @@ but belongs to the colony site — see [fergal](#fergal-portculliss-switch).
|
||||
|
||||
## The switches
|
||||
|
||||
| | jim | dave | brian |
|
||||
|---|---|---|---|
|
||||
| Identity | `jim-sw` | `dave-sw` | (UniFi) |
|
||||
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti Switch Pro XG 8 PoE |
|
||||
| Switch chip | Marvell 98DX3236 | Marvell 98DX4310 (+ Atheros 8227 for the 1G mgmt port) | — |
|
||||
| OS | RouterOS | RouterOS | UniFi |
|
||||
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 (100G, breakout-capable) + 1G mgmt | 8×10GBASE-T PoE + 2×10G SFP+ |
|
||||
| Bridge | `main`, `vlan-filtering=yes` | `main`, `vlan-filtering=yes` | UniFi VLAN profiles |
|
||||
| | `jim` | `dave` | `brian` | `shytzel` |
|
||||
|---|---|---|---|---|
|
||||
| Identity | `jim-sw` | `dave-sw` | UniFi | `shytzel` |
|
||||
| Model | CRS326-24G-2S+ | CRS504-4XQ | Ubiquiti Switch Pro XG 8 PoE | Zyxel GS1920 |
|
||||
| Switch chip | Marvell 98DX3236 | Marvell 98DX4310 (+ Atheros 8227 for the 100M management port) | — | — |
|
||||
| OS | RouterOS 7.25.1 | RouterOS 7.25.1 | UniFi | Zyxel firmware |
|
||||
| RouterBOOT | 7.25.1 | 7.25.1 | — | — |
|
||||
| Ports | 24×1G + 2×SFP+ | 4×QSFP28 + 100M management | 8×10GBASE-T PoE + 2×10G SFP+ | Not recorded |
|
||||
| VLAN configuration | `main`, `vlan-filtering=yes` | `main`, `vlan-filtering=yes` | UniFi VLAN profiles | 802.1Q L2; management only on `core` |
|
||||
|
||||
jim and dave run a single hardware-offloaded bridge (`main`) with VLAN filtering. Access to the
|
||||
MikroTiks is SSH as `admin` / `admin` by short hostname (see [Accessing the switches](#accessing-the-switches)).
|
||||
Only jim and dave can do hardware VLAN translation (`/interface ethernet switch rule` on the Marvell
|
||||
chips); brian cannot rewrite tags, only trunk/PVID them.
|
||||
`jim` and `dave` run a single hardware-offloaded bridge (`main`) with VLAN filtering. Access to the
|
||||
MikroTiks is SSH as `admin` / `admin` by short hostname (see
|
||||
[Accessing the switches](#accessing-the-switches)). Only `jim` and `dave` can do hardware VLAN
|
||||
translation (`/interface ethernet switch rule` on the Marvell chips); `brian` and `shytzel` provide
|
||||
ordinary L2 VLAN switching.
|
||||
|
||||
## Physical topology
|
||||
|
||||
The two WAN sources enter at the top: the Virgin Media modem lands on **jim** (VLAN 130), and the
|
||||
Digiweb **ONT** lands on **brian**. Both `jim` and `brian` are edge switches that uplink down into
|
||||
the **dave** core; the home boxes hang off dave's 100G ports, with backup links up to jim. jim's
|
||||
second SFP+ port (`sfp-spare`, `sfp-sfpplus2`) feeds [fergal](#fergal-portculliss-switch), which
|
||||
[`portcullis`](../colony/portcullis.md) hangs off while it is staged at home.
|
||||
The Virgin Media modem lands on `shytzel`, which trunks VLAN 130 to `jim`; the Digiweb ONT lands on
|
||||
`brian`. `brian` has its primary LAG to the `dave` core and an RSTP-protected secondary connection
|
||||
to `jim`; `jim` and `dave` are also joined directly. The home boxes hang off `dave`'s high-speed
|
||||
ports, with backup links to `jim`. `jim`'s second SFP+ port (`sfp-spare`, `sfp-sfpplus2`) feeds
|
||||
[fergal](#fergal-portculliss-switch), which [`portcullis`](../colony/portcullis.md) hangs off while
|
||||
it is staged at home.
|
||||
|
||||
```
|
||||
Virgin Media cable modem Digiweb ONT
|
||||
stream WAN, VLAN 130 river WAN, management + VLAN 10
|
||||
| |
|
||||
jim ---- 10G ---- fergal ---- portcullis brian
|
||||
| 10G trunk 802.3ad LAG |
|
||||
+--------------------+ +---------------+
|
||||
| |
|
||||
+---+----------+---+
|
||||
| dave |
|
||||
+--------+---------+
|
||||
|
|
||||
+---------------------+---------------------+
|
||||
| | |
|
||||
palace (100G) castle (100G) stream
|
||||
river VM NVMe-oF root second router
|
||||
[](diagrams/home-fabric.svg)
|
||||
|
||||
The [Mermaid source](diagrams/home-fabric.mmd) defines the logical topology; the rendered SVG
|
||||
also works in Markdown viewers without Mermaid support.
|
||||
|
||||
Solid and dotted lines distinguish normal carrying paths from backup attachments; RSTP selects
|
||||
ports for the whole fabric. The separate WAN connection to `stream` is outside its LAN bridge.
|
||||
The 10G trunk from `jim` to [fergal](#fergal-portculliss-switch) serves the staged colony boxes.
|
||||
|
||||
### Front-panel port maps
|
||||
|
||||
In the rack, viewed from the front, the order is **`brian`, `jim`, `dave`**, with `palace` several
|
||||
units below. `stream` sits to the left of `brian`; it is drawn above it to make the connections
|
||||
easier to follow. The map follows each model's physical socket order: `brian` has a single RJ45 row,
|
||||
`jim` has three banks with even ports above odd ports, and `dave` has four QSFP28 cages. Chassis
|
||||
widths, spacing and the gap above `palace` are schematic; exact rack-unit positions are not known.
|
||||
`palace`'s NICs are rear connections, shown only as endpoints rather than an invented front layout.
|
||||
|
||||
[](diagrams/switch-panels.svg)
|
||||
|
||||
Drawn lines and matching letter badges identify the two ends of a cable. Cable routes are
|
||||
schematic, not a record of how the cables are dressed in the rack. NIC labels give socket
|
||||
capacity; `stream`'s 2.5G copper ports connect to `jim` at 1G. Q3.1–Q3.4 are breakout lanes within
|
||||
**one physical cage** on `dave`, not four separate front-panel sockets. Its management RJ45
|
||||
is separate from the console above it; `jim`'s console is also separate from Ethernet port 1.
|
||||
|
||||
<!-- switch-cables -->
|
||||
| Cable | Endpoint | Endpoint | Connection |
|
||||
|---|---|---|---|
|
||||
| **A** | `brian` `9` | `dave` `Q3.2` | 10G; first LAG member |
|
||||
| **B** | `brian` `10` | `dave` `Q3.4` | 10G; second LAG member |
|
||||
| **C** | `brian` `5` | `jim` `7` | 1G; redundant fabric trunk |
|
||||
| **D** | `jim` `S1` | `dave` `Q3.1` | 10G; direct fabric trunk |
|
||||
| **E** | `jim` `24` | `dave` `MGMT` | 100M; dave management |
|
||||
| **F** | `jim` `5` | `palace` `host-jim` | 1G; host core / netboot |
|
||||
| **G** | `jim` `22` | `palace` `river-jim-phy` | 1G; river backup trunk |
|
||||
| **H** | `dave` `Q1` | `palace` `uplink-dave` | 100G; host PF and guest VFs |
|
||||
| **I** | `stream` `ETH0` | `jim` `21` | 1G WAN; 2.5G socket on stream |
|
||||
| **J** | `stream` `ETH1` | `jim` `6` | 1G backup LAN; 2.5G socket on stream |
|
||||
| **K** | `stream` `SFP+ R` | `dave` `Q3.3` | 10G preferred LAN |
|
||||
<!-- /switch-cables -->
|
||||
|
||||
Port labels describe configured use, not current link state. `jim` ports 1–4 are configured
|
||||
members of `shytzel-downlink`; only port 1 had an active link at inspection. `brian` port 3 is
|
||||
reserved for testing `portcullis`, and port 4 serves its second JetKVM.
|
||||
|
||||
The panel layouts reference the manufacturers' [Pro XG 8 PoE guide](https://dl.ui.com/qig/usw-pro-xg-8-poe/),
|
||||
[CRS326-24G-2S+RM panel](https://mikrotik.com/product/CRS326-24G-2SplusRM),
|
||||
[CRS504-4XQ-IN panel](https://mikrotik.com/product/crs504_4xq_in), and
|
||||
[R86S panel photograph](https://www.gowinfanless.com/news/blogs/the-r86s-revolution-low-power-2-5gbe-and-10gbe-networking).
|
||||
`stream`'s connected SFP+ socket is the right-hand one, confirmed physically; the left is unused.
|
||||
Its copper socket mapping is inferred from PCI order and sequential permanent MACs. The SFP+
|
||||
sockets are labelled left/right in the drawing because both carry the same printed label.
|
||||
Port use comes from RouterOS interface/bonding configuration, LLDP, the UniFi controller and
|
||||
physical connection confirmation.
|
||||
The editable inventory is [switch-ports.json](diagrams/switch-ports.json); run
|
||||
`python3 docs/sites/home/diagrams/render-switch-panels.py` to regenerate the SVG and cable table.
|
||||
|
||||
Backup links to jim (normally idle):
|
||||
* palace: 1G
|
||||
* stream: 1G; STP selects the active link
|
||||
* castle: 2.5G, normally down; no live failover
|
||||
```
|
||||
|
||||
Notes:
|
||||
- **river** runs as a VM on the **palace** host; its uplink is dave's 100G `palace` port. jim also
|
||||
has 1G `palace`/`stream` ports, but those are secondary links and do **not** carry the WAN.
|
||||
- **river** uses `palace`'s 100G connection to `dave` and the dedicated 1G `river-jim-phy`
|
||||
connection to `jim`. The separate `host-jim` connection serves host management and VM netboot.
|
||||
See [palace's networking](palace.md#networking) for the handoffs and
|
||||
[failover validation](router-failover.md) for tested failure scenarios.
|
||||
- `jim`'s `palace-kvm` port is a JetKVM connection, not either of `palace`'s network attachments.
|
||||
- **stream** (the second router box) is dual-homed to both jim and dave (STP picks the active path).
|
||||
- The `brian`–`jim` secondary connection provides the redundant Digiweb path. VLANs 10 and 140 are
|
||||
tagged on `brian-downlink` and `river`, keeping the ONT reachable without `dave`.
|
||||
|
||||
### Castle storage dependency
|
||||
### `castle` storage dependency
|
||||
|
||||
`castle` is dual-homed without STP: its primary link is dave's 100G `castle` port (`et100g`), while
|
||||
the 2.5G link to jim (`et2.5g`) is normally down and provides no live failover. Its root disk is
|
||||
NVMe-oF over `et100g` and dave, so interrupting either freezes `castle` mid-I/O. Do dave maintenance
|
||||
from a box that does not depend on it, or power `castle` off cleanly first.
|
||||
the backup NIC (`et2.5g`) connects at 1G to jim and is normally down, providing no live failover.
|
||||
Its root disk is NVMe-oF over `et100g` and dave, so interrupting either freezes `castle` mid-I/O.
|
||||
Do dave maintenance from a box that does not depend on it, or power `castle` off cleanly first.
|
||||
|
||||
## VLANs
|
||||
|
||||
| VLAN | Name | Purpose |
|
||||
|---|---|---|
|
||||
| — (native) | core | Switch management, `192.168.64.0/24` (jim `.10`, dave `.11`, brian `.13`) |
|
||||
| — (native) | core | Switch management, `192.168.64.0/24` (`jim` `.10`, `dave` `.11`, `shytzel` `.12`, `brian` `.13`) |
|
||||
| 100 | hi | High-performance / jumbo network (MTU 9000) |
|
||||
| 110 | lo | Standard LAN |
|
||||
| 120 | untrusted | Guest / untrusted network |
|
||||
| 130 | wan | **stream's WAN** — Virgin Media cable modem (untagged on jim's `wan1`/`wan2`/`wan-in`) |
|
||||
| 130 | wan | `stream`'s WAN — tagged from `shytzel`, untagged on `jim`'s `stream-wan` |
|
||||
| 140 | wan-pon-ont | ONT management, `192.168.100.0/24` (PVID'd at the ONT edge) |
|
||||
| 10 | pon-isp | Digiweb ISP transport — **trunked straight through** to river, PPPoE runs on it |
|
||||
| 141 | wan-pon-isp | **Reserved** — the translated ISP VLAN for the future multi-ONT design |
|
||||
@@ -107,11 +150,12 @@ we just carry it end to end and let river run PPPoE directly on it:
|
||||
reach the ONT web UI at `192.168.100.1`. Doing the PVID at the ONT-facing edge keeps it clean —
|
||||
the untagged frames never share a domain with anything else.
|
||||
|
||||
2. **VLAN 10 (ISP) trunked straight through, untranslated.** brian → dave → palace carry tagged
|
||||
VLAN 10 by ordinary bridge-VLAN membership. No `/interface ethernet switch rule`, no pinning, no
|
||||
asymmetric-learning issues — it's just a normal tagged VLAN. river attaches PPPoE to VLAN 10
|
||||
directly (`wan-pon-isp` netdev = VLAN `pon-isp` = 10; baby-jumbo MTU 1508 so PPP nets a clean
|
||||
1500).
|
||||
2. **VLAN 10 (ISP) trunked straight through, untranslated.** The primary path is
|
||||
brian → dave → palace; the RSTP-protected secondary path reaches jim before returning to the
|
||||
router attachment. Both carry tagged VLAN 10 by ordinary bridge-VLAN membership. No
|
||||
`/interface ethernet switch rule`, no pinning, no asymmetric-learning issues — it's just a
|
||||
normal tagged VLAN. river attaches PPPoE to VLAN 10 directly (`wan-pon-isp` netdev = VLAN
|
||||
`pon-isp` = 10; baby-jumbo MTU 1508 so PPP nets a clean 1500).
|
||||
|
||||
Net result: **river runs PPPoE single-tagged on VLAN 10 and holds a VLAN 140 address to reach the
|
||||
ONT.** See [`nixos/boxes/home/palace/vms/river.nix`](../../../nixos/boxes/home/palace/vms/river.nix)
|
||||
@@ -139,18 +183,41 @@ How each switch is set up for the Digiweb WAN path. **Confirm any change on the
|
||||
- The ONT port has **native/untagged network = VLAN 140** (PVID) and is a **tagged member of VLAN 10**,
|
||||
so the ONT's untagged management lands on 140 and its tagged ISP frames pass through.
|
||||
- The `brian-downlink` LAG up to dave trunks **tagged 140 + tagged 10** (alongside the LAN VLANs).
|
||||
- Its secondary connection to jim carries the same relevant VLANs and participates in RSTP, so it
|
||||
normally remains blocked while the primary path is healthy.
|
||||
|
||||
**dave (RouterOS)** — trunks both WAN-pon VLANs to `brian-downlink` and `palace`. The ISP VLAN 10 row:
|
||||
```
|
||||
/interface bridge vlan add bridge=main vlan-ids=10 tagged=brian-downlink,palace
|
||||
/interface bridge vlan add bridge=main vlan-ids=10 tagged=brian-downlink,palace,jim-downlink
|
||||
```
|
||||
VLAN 140 also spans `brian-downlink,palace` (it carries a few other members too). No switch rules —
|
||||
this is plain tagged bridging.
|
||||
VLAN 140 also spans `brian-downlink,palace,jim-downlink` (it carries a few other members too).
|
||||
The direct connection to `jim` must carry both VLANs: if only `palace`'s `dave` connection fails,
|
||||
the surviving RSTP path from `river` through `jim` can still traverse `dave` to reach `brian`.
|
||||
These are ordinary tagged VLANs with no switch rules.
|
||||
|
||||
**jim (RouterOS)** — the physical `brian-downlink` is the RSTP-protected secondary connection from
|
||||
`brian`. VLANs 10 and 140 have `brian-downlink`, `river` and `dave-uplink` as tagged members; no
|
||||
translation rules are involved. RSTP selects one tree for the bridge, so VLAN membership must
|
||||
include the surviving paths through `dave` as well as the direct backup connection to `brian`.
|
||||
The `river` port is native `core` with VLANs 10, 100, 110, 120 and 140 tagged. It
|
||||
faces `palace`'s `river-jim-phy`; `palace-kvm` is the separate JetKVM connection and must remain
|
||||
unchanged. See [failover validation](router-failover.md) for attachment and switch-outage checks.
|
||||
|
||||
Other connections on `jim`:
|
||||
|
||||
| Interface | Connection |
|
||||
|---|---|
|
||||
| `ether9` | Disconnected, unbridged spare |
|
||||
| `stream-wan` | Dedicated untagged VLAN 130 handoff to `stream` `wan` |
|
||||
| `river` | `river-jim-phy`, redundant native-core trunk |
|
||||
|
||||
The modem connects to `shytzel`. `stream` has two distinct connections to `jim`: `stream-wan`
|
||||
for its isolated WAN handoff and `stream` for the `lan-jim` RSTP trunk. VLAN 130 has
|
||||
`shytzel-downlink` tagged and only `stream-wan` untagged.
|
||||
|
||||
`sfp-spare` (`sfp-sfpplus2`) stays at `pvid=1`—the switch feeding `portcullis` is reached over
|
||||
VLAN 1 untagged—and is a tagged member of `hi` (100) and `lo` (110):
|
||||
|
||||
**jim (RouterOS)** — carries **none** of the Digiweb WAN path: no translation rules, and no VLAN
|
||||
10/140/141 rows. jim only handles stream's VLAN-130 WAN and the LAN VLANs. `sfp-spare`
|
||||
(`sfp-sfpplus2`) stays at `pvid=1` — the switch feeding `portcullis` is reached over VLAN 1
|
||||
untagged — and is a **tagged** member of `hi` (100) and `lo` (110) so those reach `portcullis`:
|
||||
```
|
||||
/interface bridge vlan set [find bridge=main vlan-ids=100] tagged=...,sfp-spare
|
||||
/interface bridge vlan set [find bridge=main vlan-ids=110] tagged=...,sfp-spare
|
||||
@@ -163,8 +230,7 @@ management addresses and static default routes exist only so the boxes themselve
|
||||
reach out; they must **never** forward traffic or advertise themselves as routers. RouterOS defaults
|
||||
work against this: `ip-forward` and IPv6 `forward` ship **on**, and with IPv6 forwarding on RouterOS
|
||||
also emits Router Advertisements (`ra-lifetime=30m`) on every L3 interface — so a switch silently
|
||||
becomes a competing IPv6 default router. This surfaced after the 7.18 → 7.23 upgrade, when clients
|
||||
picked up dave/jim as default routers alongside river.
|
||||
becomes a competing IPv6 default router.
|
||||
|
||||
The required config on each RouterOS box:
|
||||
```
|
||||
@@ -181,7 +247,7 @@ The required config on each RouterOS box:
|
||||
actively clears the rogue default from clients (they otherwise cache it for up to ~30 min).
|
||||
|
||||
**After any RouterOS upgrade, re-check `/ip settings` and `/ipv6 settings`** — an upgrade can reset
|
||||
these to the forwarding-on defaults. brian (UniFi) is not a RouterOS box and was not affected.
|
||||
these to the forwarding-on defaults. `brian` uses UniFi and does not have these RouterOS settings.
|
||||
|
||||
## Future: multiple ONTs (per-port VLAN translation)
|
||||
|
||||
@@ -225,6 +291,31 @@ documented in [sites/colony/fergal.md](../colony/fergal.md).
|
||||
|
||||
## Accessing the switches
|
||||
|
||||
### Upgrades
|
||||
|
||||
Save a text export and an encrypted binary backup, and download both before rebooting. Local
|
||||
upgrade backups are kept outside Git under `.cache/mikrotik/<date>/`; the backup password is
|
||||
stored alongside them with restricted permissions. Binary backups are intended for restoration
|
||||
on the same box and RouterOS version.
|
||||
|
||||
Upgrade one switch at a time. Upgrade `dave` while `jim` carries the backup path, verify its
|
||||
return, then upgrade `jim`. Check RouterOS packages first, then upgrade RouterBOOT and reboot
|
||||
again if its running version differs. After each upgrade, verify bridge VLAN membership, RSTP
|
||||
roles, hardware offload and the [forwarding/RA settings](#switches-must-not-route).
|
||||
|
||||
During failover, use bounded SSH calls and an explicit IPv4 management address where useful.
|
||||
RouterOS parses newlines in SSH command strings separately: send a multi-statement block as one
|
||||
line with semicolons so local variables and braces retain their scope.
|
||||
|
||||
Whole-switch outage behavior is covered by the
|
||||
[validated scenarios](router-failover.md#validated-scenarios).
|
||||
|
||||
RouterOS uses Ed25519 host keys by default. Upgrading an older installation can change the
|
||||
host-key type from RSA; verify the box's identity independently before updating the client's
|
||||
`known_hosts` entry.
|
||||
|
||||
### SSH
|
||||
|
||||
The switches resolve by **short hostname** on the home network — the home routers serve their
|
||||
records in the home zone
|
||||
([`nixos/boxes/home/routing-common/dns.nix`](../../../nixos/boxes/home/routing-common/dns.nix):
|
||||
@@ -241,13 +332,15 @@ sshpass -p admin ssh -o IdentityAgent=none -o PubkeyAuthentication=no \
|
||||
-o UserKnownHostsFile=/tmp/sw_known_hosts admin@jim
|
||||
```
|
||||
|
||||
**Always confirm config changes on the switch** (print the affected menu, apply, re-verify). brian
|
||||
is UniFi — configured through its controller, not RouterOS CLI.
|
||||
**Always confirm config changes on the switch** (print the affected menu, apply, re-verify).
|
||||
`brian` is configured through its UniFi controller. `shytzel` is an L2 managed switch whose
|
||||
management interface is reachable only through `core`.
|
||||
|
||||
## Management IPs
|
||||
|
||||
| | core (`192.168.64.0/24`) | hi (`192.168.68.0/22`) | lo (`192.168.72.0/21`) |
|
||||
|---|---|---|---|
|
||||
| jim | `.10` (on `main`) | `.10` | `.10` |
|
||||
| dave | `.11` (on `management`, the 1G Atheros port) | `.11` | `.11` |
|
||||
| brian | `.13` (core) | — | — |
|
||||
| `jim` | `.10` (on `main`) | `.10` | `.10` |
|
||||
| `dave` | `.11` (on `management`, the 1G Atheros port) | `.11` | `.11` |
|
||||
| `brian` | `.13` (`core`) | — | — |
|
||||
| `shytzel` | `.12` | — | — |
|
||||
|
||||
Reference in New Issue
Block a user