# NetBird Mesh + Local LLM Remote Access — Setup Guide Goal: reach the desktop's local LLM stack (llama-swap) securely from a laptop and phone, over a self-hosted-friendly VPN mesh — not the raw LAN, not the open internet. ## Architecture at a glance ``` Phone ─┐ ├─ NetBird mesh (encrypted, peer-to-peer) ─→ Desktop (Heimdall-home) Laptop ─┘ └─ llama-swap :8080 └─ UFW: only wt0 allowed in ``` - **NetBird Cloud** provides the control plane (who's allowed to talk to whom). Actual traffic between peers goes directly, peer-to-peer — it does not route through NetBird's servers. - **llama-swap** on the desktop is only reachable via the mesh interface (`wt0`), enforced by UFW — not from the raw home LAN, not from the internet. - **Open WebUI** runs locally on each client device (laptop) rather than centrally on the desktop, so the desktop doesn't need to expose an extra service. --- ## Prerequisites - A NetBird Cloud account (free): https://app.netbird.io - Arch-based Linux on desktop and laptop (commands below assume `yay`/`pacman`) - `sudo` access on both machines --- ## Step 1 — Create a NetBird Cloud account Go to https://app.netbird.io and sign up (Google/GitHub/Microsoft/email — doesn't matter, it's just for the dashboard). You'll land on an empty dashboard; peers will appear here as you add them below. --- ## Step 2 — Install and connect the NetBird client (any Arch-based device) ```bash yay -S netbird # or: paru -S netbird netbird version # sanity check (note: no "--" before version) sudo netbird service install sudo netbird service start sudo netbird up ``` `netbird up` prints a login URL. Open it in a browser, log into your NetBird account, and approve the device. The command then returns once approved. ### ⚠️ Known pitfall: "no such device" / TUN interface error If `netbird up` fails with something like: ``` failed creating tunnel interface wt0: [error creating tun device: no such device] ``` and `sudo modprobe tun` responds with: ``` modprobe: FATAL: Module tun not found in directory /lib/modules/ ``` — this is **not a NetBird problem**. It means a kernel package update replaced `/lib/modules//` on disk, but you're still *running* the old kernel in memory (no reboot since the update). Confirm with: ```bash uname -r # currently running kernel pacman -Q linux # installed kernel package ``` If these two versions differ, **reboot** — that's the fix. This is a general Arch/rolling-release gotcha, not specific to this tool. ### Verify the connection ```bash sudo netbird status ``` Look for `Management: Connected`, `Signal: Connected`, and a NetBird IP (usually `100.x.x.x`). Note this IP down for each device — you'll need it below. --- ## Step 3 — Confirm connectivity between two peers From one device, ping the other's NetBird IP: ```bash ping -c 4 ``` The **first** packet may be slow (100–200ms) — that's the mesh negotiating the actual path between peers (NAT traversal). Subsequent packets should drop to single-digit ms if a direct peer-to-peer path was found. If *all* packets stay slow, the connection likely fell back to a relay server instead of going direct — still functional, just not optimal. Also check: ```bash sudo netbird status ``` `Peers count: 1/1 Connected` confirms it. Note: NetBird uses **lazy connections** — it only establishes the tunnel once there's real traffic, so `0/1 Connected` immediately after joining is normal, not a failure. --- ## Step 4 — Lock down the LLM service with UFW (on the desktop) ```bash sudo pacman -S ufw sudo ufw default deny incoming sudo ufw default allow outgoing sudo ufw allow in on wt0 to any port 8080 proto tcp sudo ufw enable sudo ufw status verbose ``` `wt0` is NetBird's virtual network interface — this rule only allows port 8080 traffic that arrives *via the mesh*, not the raw LAN interface, not Docker's bridge, not the internet. > **If you rely on remote SSH access**, add an allow rule for port 22 > *before* enabling default-deny, or you may lock yourself out. This guide > assumes SSH access is either local-only or handled separately. ### Verify — from a *different* device, not the desktop itself ```bash curl http://:8080/ # should respond curl http://:8080/ # should hang / time out ``` ⚠️ **Run this from the laptop or another peer — not from the desktop itself.** A machine connecting to its own IP typically routes over loopback internally, which bypasses the interface-specific firewall rule entirely and gives a false pass. **Diagnostic signal worth knowing:** an *instant* "connection refused" means nothing is listening on that port. A *hang/timeout* means something is listening, but the firewall is silently dropping the packets. Different causes, different fixes. --- ## Step 5 — Run llama-swap as a systemd service Converts a manually-started process into something that auto-restarts on crash and survives reboots. ```bash sudo nano /etc/systemd/system/llama-swap.service ``` ```ini [Unit] Description=llama-swap LLM router After=network-online.target Wants=network-online.target [Service] Type=simple User=heimdall WorkingDirectory=/home/heimdall/apps/llama-swap ExecStart=/home/heimdall/apps/llama-swap/llama-swap -config config.yaml Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target ``` If you had it running manually beforehand, kill that process first (check `ps aux | grep llama-swap` for its PID) so it doesn't conflict over port 8080 with the new service. ```bash sudo systemctl daemon-reload sudo systemctl enable --now llama-swap.service sudo systemctl status llama-swap.service ``` `enable` makes it start on every boot; `--now` also starts it immediately. `Restart=on-failure` (not `always`) means it restarts on a crash, but not if you intentionally stop it for maintenance. --- ## Step 6 — Prevent the desktop from sleeping A sleeping desktop can't serve requests, mesh or not. Check both the desktop-environment layer and the underlying system layer — they can disagree even when one looks correctly configured. ```bash cat /etc/systemd/logind.conf | grep -i idleaction systemctl status sleep.target suspend.target ``` For a hard guarantee regardless of any GUI setting or future update: ```bash sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target ``` Verify: ```bash systemctl status sleep.target suspend.target hibernate.target hybrid-sleep.target ``` Each should now show `Loaded: masked`. --- ## Step 7 — Install Docker + Open WebUI on the laptop ```bash sudo pacman -S docker sudo systemctl enable --now docker.service sudo usermod -aG docker $USER ``` ⚠️ Group membership only applies after a fresh login — log out/in (or reboot) before expecting `docker` commands to work without `sudo`. ```bash docker run -d -p 3000:8080 \ -v open-webui:/app/backend/data \ --name open-webui \ --restart always \ ghcr.io/open-webui/open-webui:main ``` Open `http://localhost:3000` — the **first account you create automatically becomes admin**, no separate seed step needed. > ⚠️ **Pending hardening step (not yet done):** this binds to all > interfaces (`0.0.0.0:3000`), meaning it's reachable on *any* network the > laptop joins, not just the mesh. To restrict it to the mesh only: > ```bash > docker stop open-webui && docker rm open-webui > docker run -d -p :3000:8080 \ > -v open-webui:/app/backend/data \ > --name open-webui --restart always \ > ghcr.io/open-webui/open-webui:main > ``` > Trade-off: if the laptop's mesh IP ever changes, this binding breaks > silently until updated. --- ## Step 8 — Point Open WebUI at llama-swap In Open WebUI: **Admin Panel → Settings → Connections → OpenAI API** - **URL:** `http://:8080/v1` - **API Key:** any non-empty placeholder if llama-swap has no auth configured (most clients require *something* in the field even if it's not validated). Save, and the model list should populate automatically. Test with a real chat message, not just the model list loading — that confirms actual inference is working end to end, not just the metadata endpoint. --- ## Step 9 — Add more devices (e.g. phone) Install the official NetBird app, log in with the same account, approve the device. It'll appear as a new peer in the dashboard automatically. --- ## Troubleshooting: a Docker container can't reach a service on its own host Symptom: Open WebUI *on the desktop itself* (in Docker) can't reach llama-swap via `host.docker.internal:8080`, even though llama-swap is healthy and the same setup works fine from other peers over the mesh. **Diagnosis path:** ```bash # 1. Confirm the service is actually up sudo ss -tlnp | grep 8080 sudo systemctl status llama-swap.service # 2. Check if requests are even arriving sudo journalctl -u llama-swap -n 20 --no-pager # 3. Check the firewall's own block log # (Arch doesn't ship /var/log/ufw.log by default — check the kernel log instead) sudo journalctl -k -n 50 --no-pager | grep -i "block\|deny" ``` **Root cause:** `host.docker.internal` resolves to Docker's bridge gateway (commonly `172.17.0.1`), not to `localhost` or the mesh interface. Traffic from a container to the host arrives via the `docker0` interface. Since the UFW rule from Step 4 only allowed `wt0`, this traffic gets silently dropped under default-deny. This is a **different problem** from "Docker bypasses UFW for published ports" (that's about *incoming* traffic to a container from outside the machine, via Docker's own NAT rules bypassing UFW's chain). This case is a container reaching *out* to something on the host — which does go through UFW's normal `INPUT` chain like any other interface. **Fix:** ```bash sudo ufw allow in on docker0 to any port 8080 proto tcp ``` Re-test with a real chat message in the desktop's Open WebUI, and confirm with `journalctl -u llama-swap` that a fresh request appears from the container's Docker IP. --- ## Diagnostic cheatsheet (built up during this setup) | Symptom | Likely meaning | |---|---| | Instant "connection refused" | Nothing is listening on that port | | Hang / timeout | Something's listening, but a firewall is silently dropping packets | | `modprobe: module not found` after a kernel update | Running kernel ≠ installed kernel package; needs a reboot | | `netbird up` → `context deadline exceeded` | Vague — check `client.log` or journal for the *real* underlying error, don't assume it's the login step | | Curl test passes when run *on* the target machine, fails from elsewhere | You tested via loopback, not the actual interface — re-test from a different device | | Docker container can't reach a host service | Check `host.docker.internal` → `docker0` gateway → firewall rule for that interface specifically | --- ## Open items (see companion status document for full detail) - UFW rule for `docker0`→llama-swap: written above, not yet confirmed tested - Laptop's Open WebUI: still bound to all interfaces, not yet restricted to the mesh IP - SSH remote access to the desktop: deliberately deferred, currently unreachable remotely - Cline on the laptop: configured and confirmed working - llama-swap's "reachable by all hosts" startup warning: expected, mitigated at the firewall layer — not a bug