nixos/slop-cannon: Move to a VM on palace
CI / Check, build and cache nixfiles (push) Successful in 45m39s
Update docs / update (push) Successful in 1m0s

Give agents their own kernel, Nix daemon and disk for workloads that
need root. Put the VM on the `untrusted` VLAN with MAC spoof
checking on VF 5.

Export a dedicated partition from `cellar` and connect it on
`palace`. Give the VM its own SSH host key and age secrets for Paseo
and Gitea. Keep the agent home and Paseo identity on persistent
storage and provide `slop-pair` for pairing clients.

Retire the container, restore `sfh` to 8 cores and 32 GiB, and rekey
the shared password secret. Document the VM and its operating
procedures on the relevant box pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-18 22:02:14 +00:00
co-authored by Claude Opus 5.5
parent 5058dec3f1
commit a6491c7535
20 changed files with 588 additions and 353 deletions
+1
View File
@@ -17,6 +17,7 @@ pair, and everything clients touch (gateway, DNS) is a floating VIP that follows
| [`stream`](stream.md) | Secondary router (VRRP pair with `river`) | physical |
| [`cellar`](cellar.md) | NVMe-oF / SPDK storage target | VM on `palace` |
| [`sfh`](sfh/README.md) | NixOS container host (containers on its page) | VM on `palace` |
| [`slop-cannon`](slop-cannon.md) | Coding agents, driven remotely through Paseo; on `untrusted` | VM on `palace` |
| [`castle`](castle.md) | Workstation / gaming desktop | physical |
## Router VIPs
+8 -6
View File
@@ -2,7 +2,7 @@
The home storage target. A VM on `palace` that drives three passed-through NVMe disks with SPDK
and exports them over NVMe-oF/RDMA — `river`, `sfh` and `castle` all run their root storage off
it.
it, and `slop-cannon`'s disk lives on it.
- **Source:** [`nixos/boxes/home/palace/vms/cellar/`](../../../nixos/boxes/home/palace/vms/cellar)
(`default.nix`, `spdk.nix`)
@@ -22,6 +22,7 @@ it.
| `NVMeRaidp1` | `nqn.2016-06.io.spdk:river` | [`river`](river.md) |
| `NVMeRaidp2` | `nqn.2016-06.io.spdk:castle` | [`castle`](castle.md) |
| `NVMeRaidp3` | `nqn.2016-06.io.spdk:sfh` | [`sfh`](sfh/README.md) |
| `NVMeRaidp4` | `nqn.2016-06.io.spdk:slop-cannon` | [`palace`](palace.md#slop-cannons-disk), for [`slop-cannon`](slop-cannon.md) |
Each subsystem is pinned to its consumer's `hostnqn` (the `my.nvme.uuid` on the client side).
- `spdk-tgt` doesn't restart on deploy (`restartIfChanged = false`): restarting it takes every
@@ -38,7 +39,7 @@ See the consolidated [network assignments](../../networking.md#box-assignments)
Partitions on `NVMeRaid` are ordinary GPT partitions. SPDK's `gpt` module finds them when it
examines the RAID (`bdev_examine` in `spdk.nix`; auto-examine is off) and exposes those whose type
GUID is SPDK's, as `NVMeRaidp<n>`. `p1`–`p3` use the current type GUID,
GUID is SPDK's, as `NVMeRaidp<n>`. `p1`–`p4` use the current type GUID,
`6527994e-2c5a-4eec-9613-8f5944074e8b`. Use it for new partitions and leave existing partition
types as they are. SPDK still supports its deprecated type, `7c5222bd-8f5d-4087-9c00-bf9843c7b58c`,
and exposes one fewer block for that type to preserve compatibility with existing on-disk metadata.
@@ -51,9 +52,10 @@ while the exports are live: every consumer has to be down. Start from a box that
when the exports are offline.
1. Shut down every consumer. Power off `castle` and run
`doas systemctl stop vm@sfh vm@river` on `palace`. `stream` takes over the router VIPs. On
`cellar`, verify that `doas spdk-rpc nvmf_subsystem_get_controllers <nqn>` returns `[]` for each
exported subsystem in the table above.
`doas systemctl stop vm@sfh vm@river slop-cannon-disk` on `palace`. `stream` takes over the
router VIPs. Stopping `slop-cannon-disk` shuts down `vm@slop-cannon` before disconnecting its
disk. On `cellar`, verify that `doas spdk-rpc nvmf_subsystem_get_controllers <nqn>` returns `[]`
for each exported subsystem in the table above.
2. On `cellar`, stop the target and start `spdk-debug` in its place. It runs `spdk_tgt` on a
config generated from the real one: the same controllers and RAID, but nothing examined (so
the `gpt` module never claims the RAID), nothing exported, and the RAID exposed through ublk
@@ -88,7 +90,7 @@ when the exports are offline.
serial), deploy `cellar` (the unit doesn't restart) and `doas systemctl start spdk-tgt`. Check
`doas spdk-rpc bdev_get_bdevs` lists the new partition and `doas spdk-rpc nvmf_get_subsystems`
the new subsystem.
7. Start the consumers again.
7. Start the consumers again. Starting `vm@slop-cannon` also starts `slop-cannon-disk`.
If `spdk-tgt` won't start in step 6, the usual cause is a config naming a bdev that doesn't exist.
Switch `cellar` back to its previous generation
+39 -13
View File
@@ -1,7 +1,7 @@
# palace
The physical VM host for the home site. Runs the `river`, `cellar` and `sfh` VMs and feeds them
SR-IOV VFs, PCI NVMe drives and LVM disks.
The physical VM host for the home site. Runs the `river`, `cellar`, `sfh` and `slop-cannon` VMs and
feeds them SR-IOV VFs, PCI NVMe drives, LVM disks and an NVMe-oF disk.
- **Source:** [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix)
(VM definitions in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
@@ -35,7 +35,7 @@ See the consolidated [network assignments](../../networking.md#box-assignments)
## Networking
100G `uplink-dave` (mlx5, MTU 9000) connects to the `dave` switch and carries `lan-hi` (VLAN 100,
the `hi` assignment). A udev rule creates five SR-IOV VFs on the PF. Two physically separate Intel
the `hi` assignment). A udev rule creates six SR-IOV VFs on the PF. Two physically separate Intel
connections lead to `jim`: `host-jim` serves `palace` and `sfh`, while `river-jim-phy` is exported
exclusively to `river`.
@@ -64,14 +64,15 @@ BPDUs without a guest-to-host receive-filter synchronization service.
```text
dave
└─ palace uplink-dave (ConnectX-4 PF)
├─ VLAN 100 ───────────────────────────────> palace lan-hi
├─ VLAN 100 ───────────────────────────────> palace lan-hi ───> cellar NVMe/RDMA (slop-cannon's disk)
├─ VF 0 cellar-hi ────────────────────────> cellar lan-hi
├─ VF 1 river-rdma ───────────────────────> river rdma-hi ──> cellar NVMe/RDMA
├─ VF 2 sfh-hi ───────────────────────────> sfh lan-hi ─────> cellar NVMe/RDMA
├─ VF 3 sfh-ctrs ─────────────────────────> sfh ctrs-hi
│ ├─> hass host0
│ └─> unifi host0
└─ VF 4 river-dave ───────────────────────> river lan-dave ──┐
├─ VF 4 river-dave ───────────────────────> river lan-dave ──┐
└─ VF 5 slop-cannon-ut ───> slop-cannon lan-untrusted │
├─> river lan (RSTP)
jim │
├─ palace river-jim-phy ── river-jim-tap ─────> river lan-jim ───┘
@@ -101,13 +102,13 @@ from the host to its guests. Interface names in the destination columns belong t
| Switch port | Host interface | Host networks | Guest handoff |
|---|---|---|---|
| `dave` `palace` | `uplink-dave` | VLAN 100 → `lan-hi` | Five PCI VFs, listed below |
| `dave` `palace` | `uplink-dave` | VLAN 100 → `lan-hi` | Six PCI VFs, listed below |
| `jim` `palace` | `host-jim` | Native `core` → `lan-core`; VLAN 110 → `lan-lo-phy` → `lan-lo` | `sfh-core-tap` and `sfh-lo-tap` |
| `jim` `river` | `river-jim-phy` | No host L3 configuration | `river-jim-tap` macvtap → `river` `lan-jim` |
#### ConnectX-4 VFs
All five VFs belong to `uplink-dave`. VLAN filtering and carrier policy are applied on the PF.
All six VFs belong to `uplink-dave`. VLAN filtering and carrier policy are applied on the PF.
| VF / PCI BDF | QEMU device | Guest | Guest interface | VLAN / carrier | Role |
|---|---|---|---|---|---|
@@ -116,6 +117,7 @@ All five VFs belong to `uplink-dave`. VLAN filtering and carrier policy are appl
| 2 / `44:00.3` | `sfh-hi` | [sfh](sfh/README.md) | `lan-hi` | 100 / forced up | NVMe/RDMA root and management |
| 3 / `44:00.4` | `sfh-ctrs` | [sfh](sfh/README.md#networking) | `ctrs-hi` | 100 / forced up | Container `hi` network |
| 4 / `44:00.5` | `river-dave` | [river](river.md#virtual-machine-and-network-attachment) | `lan-dave` | Unfiltered trunk / automatic | Preferred RSTP attachment |
| 5 / `44:00.6` | `slop-cannon-ut` | [slop-cannon](slop-cannon.md#network) | `lan-untrusted` | 120 / automatic | Agents' only network. MAC spoof checking is on. |
#### Virtual Ethernet handoffs
@@ -139,19 +141,43 @@ run before stage-two module loading. `sfh` requires the `frigate` LV before QEMU
|---|---|---|---|---|
| `cellar` | 8c × 2t | 16 GiB | `cellar-hi` VF 0 (`44:00.1`); NVMe `41:00.0`–`43:00.0` | pinned to NUMA node 1; split IRQ chip + vIOMMU |
| `river` | 3c × 2t | 4 GiB | `river-rdma` VF 1 (`44:00.2`), `river-dave` VF 4 (`44:00.5`), `river-jim-tap` | only an ESP local disk (plus an installer ISO) — root is NVMe-oF from `cellar` |
| `sfh` | 16c × 2t | 96 GiB | `sfh-hi` VF 2 (`44:00.3`), `sfh-ctrs` VF 3 (`44:00.4`); two USB host ports | no boot disk — netboots; gets the `hdds/frigate` LV |
| `sfh` | 8c × 2t | 32 GiB | `sfh-hi` VF 2 (`44:00.3`), `sfh-ctrs` VF 3 (`44:00.4`); two USB host ports | no boot disk — netboots; gets the `hdds/frigate` LV |
| `slop-cannon` | 16c × 2t | 48 GiB | `slop-cannon-ut` VF 5 (`44:00.6`) | disk is NVMe-oF from `cellar`, connected by `palace` (plus an installer ISO) |
`sfh`'s vCPUs are overcommitted (the VMs total more vCPUs than the host's 32 threads) so the coding
agents in [`slop-cannon`](sfh/containers/slop-cannon.md) can burst; that container's own CPU weight
and memory cap keep `hass` and `unifi` responsive.
`slop-cannon`'s vCPUs are overcommitted (the VMs total more vCPUs than the host's 32 threads) so
coding agents can burst; `vm@slop-cannon` has a low `CPUWeight`, so `cellar`, `river` and `sfh`
win under contention.
Boot ordering is enforced with systemd dependencies: `vm@river` waits for `cellar`'s SSH port (and
for the `river-jim-tap` device), and `vm@sfh` waits for `river` — storage first, then the router,
then everything that boots off both.
then everything that boots off both. `vm@slop-cannon` waits only for its disk (below); nothing
waits for it.
`vm@` units don't restart on deploy (`restartIfChanged = false`); a changed VM config applies the
next time the VM starts.
### `slop-cannon`'s disk
`palace` is an NVMe-oF initiator for one guest: `slop-cannon`'s disk is
`nqn.2016-06.io.spdk:slop-cannon` on [`cellar`](cellar.md), connected by `palace` (`my.nvme.uuid`
is its host NQN) and passed to QEMU as a virtio disk. The guest never speaks NVMe-oF itself:
SPDK only checks the host NQN, which root in the guest could forge.
`palace` hosts `cellar`, so the connection is kept out of `palace`'s own boot and shutdown:
- `slop-cannon-disk.service` waits for `cellar`'s SSH port, runs `nvme connect -t rdma` over
`lan-hi` and disconnects on stop. It is `BindsTo` / `After` `vm@cellar`, and only
`vm@slop-cannon` requires it. Nothing in the initrd, `fileSystems` or `multi-user.target` uses
the disk.
- Shutdown runs in reverse: `vm@slop-cannon`, then the disconnect, then `cellar`.
- `--ctrl-loss-tmo 60`: if `cellar` dies, the controller is removed after a minute and QEMU sees
I/O errors, instead of the kernel retrying forever.
- `devices/global_filter` rejects `/dev/nvme*` in `lvm.conf`, so `palace` never activates the
guest's volume group (also called `main`). `palace` has no NVMe of its own.
## Notable config files
- [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix) — host
hardware, networkd (links/bridges/SR-IOV), LVM.
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
VM instances and boot ordering.
VM instances, boot ordering and `slop-cannon`'s disk connection.
-3
View File
@@ -24,8 +24,6 @@ from `river` and runs its root off NVMe-oF from `cellar`.
`/var/lib/frigate`.
- USB: two host ports are passed to the VM (qemu flags) for the Zigbee coordinator and webcam used
by `hass`; the nspawn unit gets `DeviceAllow` for `char-ttyUSB` and `char-video4linux`.
- `slop-cannon`'s nspawn unit gets a low `CPUWeight` and a `MemoryMax`, so agent builds can't starve
`hass` and `unifi`, and a larger `/tmp` tmpfs than nspawn's default (10% of RAM).
## Network assignments
@@ -55,7 +53,6 @@ back through the VM handoffs to the physical switch connections.
|---|---|
| [`hass`](containers/hass.md) | Home Assistant + Frigate + MQTT |
| [`unifi`](containers/unifi.md) | UniFi controller |
| [`slop-cannon`](containers/slop-cannon.md) | Coding agents, driven remotely through Paseo |
## Notable config files
@@ -1,162 +0,0 @@
# slop-cannon
Coding agents (Claude Code and Codex), driven from the phone or a browser through
[Paseo](https://github.com/getpaseo/paseo), running as a container on [`sfh`](../README.md).
- **Source:** [`nixos/boxes/home/palace/vms/sfh/containers/slop-cannon.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/slop-cannon.nix)
- **Host:** NixOS container on `sfh`
- **nixpkgs:** `mine`
## Role
- Runs the Paseo daemon and the agents it starts, all as the unprivileged user `slop-cannon`.
- Not a deploy-rs target (`my.deploy.enable = false`): it's rendered via `my.asContainer` and
deployed with `sfh`.
- `sfh` gives the container a low `CPUWeight`, a `MemoryMax` and a larger `/tmp` tmpfs for agent
scratch files; `/tmp` counts against the memory cap.
## Network assignments
See the consolidated [network assignments](../../../../networking.md#box-assignments) table (this box: `slop-cannon`).
A single MACVLAN interface, `host0` on `sfh`'s `ctrs-hi`, carries the `hi` assignment. Nothing
listens on it apart from SSH.
## Users
| User | Purpose |
|---|---|
| `slop-cannon` | Runs Paseo and the agents; no `wheel`, locked password, Dev's SSH key; home persisted; UID/GID pinned in `lib.my.c.ids` |
| `dev` | Administration (`doas`); the usual primary user |
Both have Dev's SSH key; connect with the shared `ssh` aliases `slop-cannon` and `slop-cannon-dev`
(see [SSH and first login](#ssh-and-first-login)).
## Paseo
A daemon (`paseo-server`) that starts and manages agents and serves a WebSocket API to the Paseo
app, the web UI and the `paseo` CLI. Claude Code runs through the Claude Agent SDK, using the
installed `claude` and its claude.ai login; Codex runs through `codex app-server`.
### Service
- The upstream `services.paseo` module (flake input `paseo`, pinned to a release, imported in
`nixos/modules/common.nix`), with a package override for the web UI bug
[getpaseo/paseo#7673](https://github.com/getpaseo/paseo/issues/7673).
- Runs as `slop-cannon`, so agents use that user's logins and settings, with `~/.paseo` as
`PASEO_HOME`.
- The daemon password comes from the age secret `slop-cannon/paseo.env` (`PASEO_PASSWORD=…`). Read
it with `ragenix -e secrets/slop-cannon/paseo.env.age`.
- The service is not restarted on deploy, so running agents aren't interrupted. Restart it by hand
(`doas systemctl restart paseo`) to pick up a new config; stopping it stops its agents.
- Startup takes about a minute and a half while bundled plugins load; the daemon answers 503 until
it is ready.
### Settings
`services.paseo.settings` is rewritten into `~/.paseo/config.json` on every start, so changes made
from the app don't survive a restart. It:
- disables the tools Paseo injects into agents (`daemon.mcp.injectIntoAgents = false`), which would
let an agent change its own mode, answer its own permission prompts or open terminals;
- requires relay replay protection;
- turns off dictation and voice, which would otherwise download a large local speech model;
- hides providers other than Claude and Codex;
- adds a "Claude, accept edits" agent profile. Each provider's default mode is hardcoded upstream
(Claude `auto`, Codex `auto-review`), and profiles are the only way to preset one.
### Connectivity
- The daemon listens on `127.0.0.1:6767` only. The phone app and the browser reach it through our
relay on [`middleman`](../../../colony/shill/containers/middleman.md#paseo-relay)
(`paseo-relay.nul.ie`).
- Traffic through the relay is end-to-end encrypted between the app and the daemon; the relay sees
only addresses, timing and sizes. The pairing QR code or link is the trust anchor; treat it like a
password. Switching relays means re-pairing.
- Push notifications go from the daemon to Expo's push service (`exp.host`), one outbound request
per notification carrying its text.
- In a browser, open the link `paseo daemon pair` prints. `app.baseUrl` points it at our copy of
the web app on `paseo.nul.ie`, served by `middleman`, instead of `app.paseo.sh`. The daemon's own
web UI stays off, so nothing needs to reach the daemon directly.
- Opening a pairing link in the address bar adds the host without asking for the daemon password,
and the host page then shows "Password required" with no way to enter it. Instead, use the app's
option to paste a pairing link, which prompts for the password; remove a host added the other way
first.
### State
Paseo keeps only its own records in `~/.paseo`: `config.json`, the daemon keypair and local
credentials, one record per agent under `agents/`, registered projects and workspaces, and the git
worktrees it creates under `worktrees/`. Conversations stay in the providers' own stores: a Claude
agent is a normal session under `~/.claude/projects/`, a Codex agent a normal rollout under
`~/.codex/sessions/`.
## Repos, worktrees and jj
Repos live under `~/src` as colocated jj clones (see `slop-clone` below). Paseo only calls `git`:
- A `local` workspace runs the agent in the checkout itself; `jj` and Paseo's git views see the same
changes. Use it for work you want to steer with `jj`.
- A worktree workspace is a `git worktree` under `~/.paseo/worktrees` on its own branch. Use it for
unattended agents. Archiving it removes the worktree with `--force`, losing uncommitted work.
- Archiving a `local` workspace removes only Paseo's record; the folder is kept.
- Only one colocated working copy per repo is possible, so parallel jj-driven work needs separate
clones.
- Don't run `jj` in a folder while an agent is mid-turn there: jj snapshots its partial edits.
## Terminal access
- The `paseo` CLI talks to the running daemon: `paseo ls`, `paseo logs -f <id>`, `paseo send <id>
"…"`, `paseo permit ls|allow|deny`. It is command-line only, not a chat interface.
- A session can be taken over in the provider's own TUI by its ID: `claude --resume <session>` in
the agent's folder, or `codex resume <session>`. `paseo import` goes the other way. This is a
hand-off, not a shared view: stop the agent in Paseo first and don't type in both.
## Agent permissions
Paseo has the full rights of `slop-cannon`: anyone with the daemon password or a pairing can start
agents in any mode, answer prompts and open terminals. The boundaries are the container, the
unprivileged user and the approval broker in front of the deploy key.
- Claude Code and Codex come from the shared `my.agents` home-manager module.
- Claude's user settings deny raw `deploy`, `ssh-machine` and `.keys/`, and ask before
`slop-deploy` / `slop-ssh`. These match command text, so they steer agents rather than enforce a
boundary.
- Claude's managed settings (`/etc/claude-code/managed-settings.json`) repeat the `deny` rules and
disable bypass mode; agents can edit `~/.claude` but not `/etc`. The app still lists Bypass, but
choosing it fails.
- Codex stores its credentials in a file. A bubblewrap diagnostic compatibility override enables
its existing nspawn fallback; deployed and verified directly and through a fresh Paseo session.
See [Codex sandbox in nspawn](../../../../../agents-remote.md#codex-sandbox-in-nspawn).
- Deploys and `ssh-machine` access are meant to go through an approval broker, which is not built
yet; until then agents can't deploy.
## SSH and first login
`slop-cannon` is not a deploy node, so `ssh-machine` can't reach it; SSH to
`slop-cannon-ctr.h.nul.ie` directly. Never forward your SSH agent into the box: agents could use
it while you're logged in. As a last resort, `ssh-machine sfh` and
`doas machinectl shell root@slop-cannon`.
The box has one Git key, `~slop-cannon/.ssh/id_ed25519`, generated on first boot. Add its public key
in Gitea as a write-enabled deploy key on each repo agents may use. Commits are authored as Dev
through the shared Git config.
First login, once, over `ssh slop-cannon`:
1. `claude auth login` (a full login; `setup-token` tokens are not enough for every feature).
2. `codex login --device-auth`.
3. `paseo daemon pair`, then scan the QR code with the Paseo app. The pairing link carries the
relay endpoint.
4. `slop-clone <owner>/<repo>` for each repo. It clones a colocated jj repo into `~/src/<repo>`,
registers it with Paseo as a local workspace, and prints the box's public key if Gitea refuses
the clone.
## Notable config files
- [`nixos/boxes/home/palace/vms/sfh/containers/slop-cannon.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/containers/slop-cannon.nix) —
container system: users, Paseo, Claude settings, `slop-clone`.
- [`home-manager/modules/agents/default.nix`](../../../../../home-manager/modules/agents/default.nix) —
shared Claude Code and Codex configuration.
- [`nixos/boxes/home/palace/vms/sfh/default.nix`](../../../../../nixos/boxes/home/palace/vms/sfh/default.nix) —
the `sfh` side: container instance, MACVLAN, CPU, memory and `/tmp` limits.
+208
View File
@@ -0,0 +1,208 @@
# slop-cannon
Coding agents (Claude Code and Codex), driven from the phone or a browser through
[Paseo](https://github.com/getpaseo/paseo), running in a VM on [`palace`](palace.md) with root
inside it.
- **Source:** [`nixos/boxes/home/palace/vms/slop-cannon.nix`](../../../nixos/boxes/home/palace/vms/slop-cannon.nix)
(the VM itself in [`palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix))
- **Host:** VM on `palace`
- **nixpkgs:** `mine`
## VM resources
| vCPUs | RAM | Disk | Network | Other |
|---|---|---|---|---|
| 16c × 2t (overcommitted) | 48 GiB | 512 GiB virtio, NVMe-oF from `cellar` via `palace` | SR-IOV VF 5 (`44:00.6`) on VLAN 120 | installer ISO |
`vm@slop-cannon` has a low `CPUWeight` on `palace`, so `cellar`, `river` and `sfh` win under
contention. `palace` has `kvm_amd` `nested=1`, so the guest has `/dev/kvm`.
## Role
- Runs the Paseo daemon and the agents it starts, as the user `slop-cannon`. Agents have root in
the VM (`wheel`, passwordless `doas`): nested VMs, kernel modules, system services.
- A normal deploy node (`deploy .#slop-cannon`, `ssh-machine slop-cannon`).
- Everything inside the guest is treated as agent-controlled. The boundaries are outside it:
the `untrusted` filter on the routers, the VLAN tag on the NIC and the disk connection on
`palace`. Credentials for deploying or logging into other boxes stay outside the VM.
## Network assignments
See the consolidated [network assignments](../../networking.md#box-assignments) table (this box:
`slop-cannon`).
## Network
- The only interface, `lan-untrusted`, is SR-IOV VF 5 on `palace`. The NIC tags and filters VLAN
120 and enforces MAC spoof checking on the host side. The guest loads `mlx5_core` at boot.
See the [VF attachment table](palace.md#connectx-4-vfs).
- A static `untrusted` address (outside both routers' DHCP pools), MTU 1500. DNS is Cloudflare,
as `untrusted`'s DHCP and RAs hand out; the routers' resolvers aren't reachable from
`untrusted`.
- The routers keep `untrusted` to the Internet (see
[Untrusted clients](../../networking.md#untrusted-clients)). `git.nul.ie` and
`paseo-relay.nul.ie` resolve to colony IPv6 addresses, which are rejected, so Gitea SSH and the
Paseo relay fall back to IPv4 immediately.
- SSH from `hi` and `lo` works through the routers. The VM shares layer 2 with other clients
on the `untrusted` VLAN.
- Builds use the guest's own Nix daemon and store, behind the same filter.
## Storage
- One virtio disk: GPT with a 2 GiB ESP (partition label `esp`) and an LVM PV in VG `main`.
- `slop-cannon-nix`, `slop-cannon-persist` and `slop-cannon-tmp` are thin LVs in `main/tpool`,
overcommitted, so space goes to whichever needs it. The pool takes 90% of the VG; LVM grows it
into the rest at 80% full (`thin_pool_autoextend_threshold`). Watch `lvs` if it gets close.
- The root is `tmproot`; `/tmp` is its own LV, cleaned on boot and mounted with `discard`, so
agents' scratch files don't use RAM and freed space returns to the pool.
- The disk is `nqn.2016-06.io.spdk:slop-cannon` on [`cellar`](cellar.md), connected by `palace`
(see [`palace`'s disk notes](palace.md#slop-cannons-disk)). The guest never speaks NVMe-oF.
## Users and secrets
| User | Purpose |
|---|---|
| `slop-cannon` | Runs Paseo and the agents; `wheel` and `kvm`; locked password, Dev's SSH key; home persisted; UID/GID pinned in `lib.my.c.ids` |
| `dev` | Administration; locked password, Dev's SSH key |
- No `user-passwd.txt` (`my.user.passwordSecret = null`): root belongs to the agents, so the box
gets no shared secrets. Recovery is through the installer ISO.
- The VM's SSH host key is the age identity for `slop-cannon/paseo.env` and
`slop-cannon/gitea-key`. Agents can read the host key and decrypt anything encrypted to it.
## Paseo
A daemon (`paseo-server`) that starts and manages agents and serves a WebSocket API to the Paseo
app, the web UI and the `paseo` CLI. Claude Code runs through the Claude Agent SDK, using the
installed `claude` and its claude.ai login; Codex runs through `codex app-server`.
### Service
- The upstream `services.paseo` module (flake input `paseo`, pinned to a release, imported in
`nixos/modules/common.nix`), with a package override for the web UI bug
[getpaseo/paseo#7673](https://github.com/getpaseo/paseo/issues/7673).
- Runs as `slop-cannon`, so agents use that user's logins and settings, with `~/.paseo` as
`PASEO_HOME`.
- The daemon password comes from the age secret `slop-cannon/paseo.env`. To read it for a new
client, follow [Pair a Paseo client](#pair-a-paseo-client).
- The service is not restarted on deploy, so running agents aren't interrupted. Restart it by hand
(`doas systemctl restart paseo`) to pick up a new config; stopping it stops its agents.
- Startup takes about a minute and a half while bundled plugins load; the daemon answers 503 until
it is ready.
### Settings
`services.paseo.settings` is rewritten into `~/.paseo/config.json` on every start, so changes made
from the app don't survive a restart. It:
- disables the tools Paseo injects into agents (`daemon.mcp.injectIntoAgents = false`), which would
let an agent change its own mode, answer its own permission prompts or open terminals;
- requires relay replay protection;
- turns off dictation and voice, which would otherwise download a large local speech model;
- hides providers other than Claude and Codex;
- adds a "Claude, accept edits" agent profile. Each provider's default mode is hardcoded upstream
(Claude `auto`, Codex `auto-review`), and profiles are the only way to preset one.
### Connectivity
- The daemon listens on `127.0.0.1:6767` only. The phone app and the browser reach it through our
relay on [`middleman`](../colony/shill/containers/middleman.md#paseo-relay)
(`paseo-relay.nul.ie`).
- Traffic through the relay is end-to-end encrypted between the app and the daemon; the relay sees
only addresses, timing and sizes. The pairing QR code or link is the trust anchor; treat it like a
password. Switching relays means re-pairing.
- Push notifications go from the daemon to Expo's push service (`exp.host`), one outbound request
per notification carrying its text.
- `middleman` serves the static web app on `paseo.nul.ie`. The daemon's own web UI stays off.
### State
The agent home is backed by `/persist/home/slop-cannon`. It holds the provider logins and
sessions, repos, and `~/.paseo`, which contains the daemon keypair, agent records, projects,
workspaces and Paseo-managed Git worktrees. Paired clients trust that daemon keypair. Claude
sessions live under `~/.claude/projects/` and Codex sessions under `~/.codex/sessions/`.
## Repos, worktrees and jj
Repos live under `~/src` as colocated jj clones (see `slop-clone` below). Paseo only calls `git`:
- A `local` workspace runs the agent in the checkout itself; `jj` and Paseo's git views see the same
changes. Use it for work you want to steer with `jj`.
- A worktree workspace is a `git worktree` under `~/.paseo/worktrees` on its own branch. Use it for
unattended agents. Archiving it removes the worktree with `--force`, losing uncommitted work.
- Archiving a `local` workspace removes only Paseo's record; the folder is kept.
- Only one colocated working copy per repo is possible, so parallel jj-driven work needs separate
clones.
- Don't run `jj` in a folder while an agent is mid-turn there: jj snapshots its partial edits.
## Terminal access
- The `paseo` CLI talks to the running daemon: `paseo ls`, `paseo logs -f <id>`, `paseo send <id>
"…"`, `paseo permit ls|allow|deny`. It is command-line only, not a chat interface.
- A session can be taken over in the provider's own TUI by its ID: `claude --resume <session>` in
the agent's folder, or `codex resume <session>`. `paseo import` goes the other way. This is a
hand-off, not a shared view: stop the agent in Paseo first and don't type in both.
## Agent permissions
Paseo has the full rights of `slop-cannon`, and `slop-cannon` has root: anyone with the daemon
password or a pairing controls the VM.
- Claude Code and Codex come from the shared `my.agents` home-manager module.
- Claude's user settings deny raw `deploy`, `ssh-machine` and `.keys/`, and ask before
`slop-deploy` / `slop-ssh`. Claude's managed settings (`/etc/claude-code/managed-settings.json`)
repeat the `deny` rules and disable bypass mode. All of these match command text and live inside
the guest, so they steer agents rather than enforce anything.
- Codex stores its credentials in a file and uses its standard Linux sandbox.
- Deployments and administrative SSH to other boxes are handled by an operator outside the VM.
## SSH and first login
`ssh slop-cannon` (agent user) and `ssh slop-cannon-dev` (`dev`) are the shared aliases, both with
`ForwardAgent=no`. Never forward your SSH agent into the box: agents could use it while you're
logged in. For recovery, use the [VM serial console](../../deployment.md#vm-consoles) for
`slop-cannon` on `palace`.
The VM uses a Gitea deploy key from the age secret
`slop-cannon/gitea-key`. As `slop-cannon`, run
`ssh-keygen -y -f /run/secrets/slop-cannon/gitea-key` to get its public key. Add that public key
in Gitea as a write-enabled deploy key on each repo agents may use. Commits are authored as Dev
through the shared Git config.
To set up provider logins and repos on a fresh agent home, connect with `ssh slop-cannon`:
1. `claude auth login` (a full login; `setup-token` tokens are not enough for every feature).
2. `codex login --device-auth`.
3. [Pair a Paseo client](#pair-a-paseo-client).
4. `slop-clone <owner>/<repo>` for each repo. It clones a colocated jj repo into `~/src/<repo>`,
registers it with Paseo as a local workspace, and prints the box's public key if Gitea refuses
the clone.
## Pair a Paseo client
1. On the VM, run `slop-pair` as `slop-cannon`. It prints the QR code, pairing link and
`PASEO_PASSWORD`. Keep its output private.
2. If you only need the QR code and link, run `paseo daemon pair` as `slop-cannon`. To look up the
password separately, run `doas cat /run/secrets/slop-cannon/paseo.env` on the VM. You can also
decrypt it from a repo checkout with access to the dev age key:
```sh
rage --decrypt -i .keys/dev.key secrets/slop-cannon/paseo.env.age
```
3. On a phone, scan the QR code from the Paseo app. In a browser, open `https://paseo.nul.ie` and
use the app's paste-pairing-link option. Enter the value after `PASEO_PASSWORD=` when prompted
and open the paired host. If the browser shows “Password required” after opening the link in
the address bar, remove that host entry and pair again using the paste dialog.
## Notable config files
- [`nixos/boxes/home/palace/vms/slop-cannon.nix`](../../../nixos/boxes/home/palace/vms/slop-cannon.nix) —
the box: users, Paseo, Claude settings, `slop-clone`, `slop-pair`, disk layout, network.
- [`nixos/boxes/home/palace/vms/default.nix`](../../../nixos/boxes/home/palace/vms/default.nix) —
the VM on `palace` and `slop-cannon-disk.service`.
- [`nixos/boxes/home/palace/default.nix`](../../../nixos/boxes/home/palace/default.nix) — VF 5, the
NVMe host ID and the LVM filter.
- [`home-manager/modules/agents/default.nix`](../../../home-manager/modules/agents/default.nix) —
shared Claude Code and Codex configuration.