352 lines
11 KiB
Markdown
352 lines
11 KiB
Markdown
# NetBird Mesh + Local LLM Remote Access — Setup Guide
|
||
|
||
Goal: reach the desktop's local LLM stack (llama-swap) securely from a laptop
|
||
and phone, over a self-hosted-friendly VPN mesh — not the raw LAN, not the
|
||
open internet.
|
||
|
||
## Architecture at a glance
|
||
|
||
```
|
||
Phone ─┐
|
||
├─ NetBird mesh (encrypted, peer-to-peer) ─→ Desktop (Heimdall-home)
|
||
Laptop ─┘ └─ llama-swap :8080
|
||
└─ UFW: only wt0 allowed in
|
||
```
|
||
|
||
- **NetBird Cloud** provides the control plane (who's allowed to talk to
|
||
whom). Actual traffic between peers goes directly, peer-to-peer — it does
|
||
not route through NetBird's servers.
|
||
- **llama-swap** on the desktop is only reachable via the mesh interface
|
||
(`wt0`), enforced by UFW — not from the raw home LAN, not from the internet.
|
||
- **Open WebUI** runs locally on each client device (laptop) rather than
|
||
centrally on the desktop, so the desktop doesn't need to expose an extra
|
||
service.
|
||
|
||
---
|
||
|
||
## Prerequisites
|
||
|
||
- A NetBird Cloud account (free): https://app.netbird.io
|
||
- Arch-based Linux on desktop and laptop (commands below assume `yay`/`pacman`)
|
||
- `sudo` access on both machines
|
||
|
||
---
|
||
|
||
## Step 1 — Create a NetBird Cloud account
|
||
|
||
Go to https://app.netbird.io and sign up (Google/GitHub/Microsoft/email —
|
||
doesn't matter, it's just for the dashboard). You'll land on an empty
|
||
dashboard; peers will appear here as you add them below.
|
||
|
||
---
|
||
|
||
## Step 2 — Install and connect the NetBird client (any Arch-based device)
|
||
|
||
```bash
|
||
yay -S netbird # or: paru -S netbird
|
||
netbird version # sanity check (note: no "--" before version)
|
||
|
||
sudo netbird service install
|
||
sudo netbird service start
|
||
sudo netbird up
|
||
```
|
||
|
||
`netbird up` prints a login URL. Open it in a browser, log into your NetBird
|
||
account, and approve the device. The command then returns once approved.
|
||
|
||
### ⚠️ Known pitfall: "no such device" / TUN interface error
|
||
|
||
If `netbird up` fails with something like:
|
||
|
||
```
|
||
failed creating tunnel interface wt0: [error creating tun device: no such device]
|
||
```
|
||
|
||
and `sudo modprobe tun` responds with:
|
||
|
||
```
|
||
modprobe: FATAL: Module tun not found in directory /lib/modules/<version>
|
||
```
|
||
|
||
— this is **not a NetBird problem**. It means a kernel package update
|
||
replaced `/lib/modules/<new-version>/` on disk, but you're still *running*
|
||
the old kernel in memory (no reboot since the update). Confirm with:
|
||
|
||
```bash
|
||
uname -r # currently running kernel
|
||
pacman -Q linux # installed kernel package
|
||
```
|
||
|
||
If these two versions differ, **reboot** — that's the fix. This is a
|
||
general Arch/rolling-release gotcha, not specific to this tool.
|
||
|
||
### Verify the connection
|
||
|
||
```bash
|
||
sudo netbird status
|
||
```
|
||
|
||
Look for `Management: Connected`, `Signal: Connected`, and a NetBird IP
|
||
(usually `100.x.x.x`). Note this IP down for each device — you'll need it
|
||
below.
|
||
|
||
---
|
||
|
||
## Step 3 — Confirm connectivity between two peers
|
||
|
||
From one device, ping the other's NetBird IP:
|
||
|
||
```bash
|
||
ping -c 4 <other-peer-netbird-ip>
|
||
```
|
||
|
||
The **first** packet may be slow (100–200ms) — that's the mesh negotiating
|
||
the actual path between peers (NAT traversal). Subsequent packets should
|
||
drop to single-digit ms if a direct peer-to-peer path was found. If *all*
|
||
packets stay slow, the connection likely fell back to a relay server instead
|
||
of going direct — still functional, just not optimal.
|
||
|
||
Also check:
|
||
|
||
```bash
|
||
sudo netbird status
|
||
```
|
||
|
||
`Peers count: 1/1 Connected` confirms it. Note: NetBird uses **lazy
|
||
connections** — it only establishes the tunnel once there's real traffic, so
|
||
`0/1 Connected` immediately after joining is normal, not a failure.
|
||
|
||
---
|
||
|
||
## Step 4 — Lock down the LLM service with UFW (on the desktop)
|
||
|
||
```bash
|
||
sudo pacman -S ufw
|
||
sudo ufw default deny incoming
|
||
sudo ufw default allow outgoing
|
||
sudo ufw allow in on wt0 to any port 8080 proto tcp
|
||
sudo ufw enable
|
||
sudo ufw status verbose
|
||
```
|
||
|
||
`wt0` is NetBird's virtual network interface — this rule only allows port
|
||
8080 traffic that arrives *via the mesh*, not the raw LAN interface, not
|
||
Docker's bridge, not the internet.
|
||
|
||
> **If you rely on remote SSH access**, add an allow rule for port 22
|
||
> *before* enabling default-deny, or you may lock yourself out. This guide
|
||
> assumes SSH access is either local-only or handled separately.
|
||
|
||
### Verify — from a *different* device, not the desktop itself
|
||
|
||
```bash
|
||
curl http://<desktop-netbird-ip>:8080/ # should respond
|
||
curl http://<desktop-lan-ip>:8080/ # should hang / time out
|
||
```
|
||
|
||
⚠️ **Run this from the laptop or another peer — not from the desktop
|
||
itself.** A machine connecting to its own IP typically routes over loopback
|
||
internally, which bypasses the interface-specific firewall rule entirely and
|
||
gives a false pass.
|
||
|
||
**Diagnostic signal worth knowing:** an *instant* "connection refused" means
|
||
nothing is listening on that port. A *hang/timeout* means something is
|
||
listening, but the firewall is silently dropping the packets. Different
|
||
causes, different fixes.
|
||
|
||
---
|
||
|
||
## Step 5 — Run llama-swap as a systemd service
|
||
|
||
Converts a manually-started process into something that auto-restarts on
|
||
crash and survives reboots.
|
||
|
||
```bash
|
||
sudo nano /etc/systemd/system/llama-swap.service
|
||
```
|
||
|
||
```ini
|
||
[Unit]
|
||
Description=llama-swap LLM router
|
||
After=network-online.target
|
||
Wants=network-online.target
|
||
|
||
[Service]
|
||
Type=simple
|
||
User=heimdall
|
||
WorkingDirectory=/home/heimdall/apps/llama-swap
|
||
ExecStart=/home/heimdall/apps/llama-swap/llama-swap -config config.yaml
|
||
Restart=on-failure
|
||
RestartSec=5
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
```
|
||
|
||
If you had it running manually beforehand, kill that process first (check
|
||
`ps aux | grep llama-swap` for its PID) so it doesn't conflict over port
|
||
8080 with the new service.
|
||
|
||
```bash
|
||
sudo systemctl daemon-reload
|
||
sudo systemctl enable --now llama-swap.service
|
||
sudo systemctl status llama-swap.service
|
||
```
|
||
|
||
`enable` makes it start on every boot; `--now` also starts it immediately.
|
||
`Restart=on-failure` (not `always`) means it restarts on a crash, but not if
|
||
you intentionally stop it for maintenance.
|
||
|
||
---
|
||
|
||
## Step 6 — Prevent the desktop from sleeping
|
||
|
||
A sleeping desktop can't serve requests, mesh or not. Check both the
|
||
desktop-environment layer and the underlying system layer — they can
|
||
disagree even when one looks correctly configured.
|
||
|
||
```bash
|
||
cat /etc/systemd/logind.conf | grep -i idleaction
|
||
systemctl status sleep.target suspend.target
|
||
```
|
||
|
||
For a hard guarantee regardless of any GUI setting or future update:
|
||
|
||
```bash
|
||
sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target
|
||
```
|
||
|
||
Verify:
|
||
|
||
```bash
|
||
systemctl status sleep.target suspend.target hibernate.target hybrid-sleep.target
|
||
```
|
||
|
||
Each should now show `Loaded: masked`.
|
||
|
||
---
|
||
|
||
## Step 7 — Install Docker + Open WebUI on the laptop
|
||
|
||
```bash
|
||
sudo pacman -S docker
|
||
sudo systemctl enable --now docker.service
|
||
sudo usermod -aG docker $USER
|
||
```
|
||
|
||
⚠️ Group membership only applies after a fresh login — log out/in (or
|
||
reboot) before expecting `docker` commands to work without `sudo`.
|
||
|
||
```bash
|
||
docker run -d -p 3000:8080 \
|
||
-v open-webui:/app/backend/data \
|
||
--name open-webui \
|
||
--restart always \
|
||
ghcr.io/open-webui/open-webui:main
|
||
```
|
||
|
||
Open `http://localhost:3000` — the **first account you create automatically
|
||
becomes admin**, no separate seed step needed.
|
||
|
||
> ⚠️ **Pending hardening step (not yet done):** this binds to all
|
||
> interfaces (`0.0.0.0:3000`), meaning it's reachable on *any* network the
|
||
> laptop joins, not just the mesh. To restrict it to the mesh only:
|
||
> ```bash
|
||
> docker stop open-webui && docker rm open-webui
|
||
> docker run -d -p <laptop-netbird-ip>:3000:8080 \
|
||
> -v open-webui:/app/backend/data \
|
||
> --name open-webui --restart always \
|
||
> ghcr.io/open-webui/open-webui:main
|
||
> ```
|
||
> Trade-off: if the laptop's mesh IP ever changes, this binding breaks
|
||
> silently until updated.
|
||
|
||
---
|
||
|
||
## Step 8 — Point Open WebUI at llama-swap
|
||
|
||
In Open WebUI: **Admin Panel → Settings → Connections → OpenAI API**
|
||
|
||
- **URL:** `http://<desktop-netbird-ip>:8080/v1`
|
||
- **API Key:** any non-empty placeholder if llama-swap has no auth
|
||
configured (most clients require *something* in the field even if it's
|
||
not validated).
|
||
|
||
Save, and the model list should populate automatically. Test with a real
|
||
chat message, not just the model list loading — that confirms actual
|
||
inference is working end to end, not just the metadata endpoint.
|
||
|
||
---
|
||
|
||
## Step 9 — Add more devices (e.g. phone)
|
||
|
||
Install the official NetBird app, log in with the same account, approve the
|
||
device. It'll appear as a new peer in the dashboard automatically.
|
||
|
||
---
|
||
|
||
## Troubleshooting: a Docker container can't reach a service on its own host
|
||
|
||
Symptom: Open WebUI *on the desktop itself* (in Docker) can't reach
|
||
llama-swap via `host.docker.internal:8080`, even though llama-swap is
|
||
healthy and the same setup works fine from other peers over the mesh.
|
||
|
||
**Diagnosis path:**
|
||
|
||
```bash
|
||
# 1. Confirm the service is actually up
|
||
sudo ss -tlnp | grep 8080
|
||
sudo systemctl status llama-swap.service
|
||
|
||
# 2. Check if requests are even arriving
|
||
sudo journalctl -u llama-swap -n 20 --no-pager
|
||
|
||
# 3. Check the firewall's own block log
|
||
# (Arch doesn't ship /var/log/ufw.log by default — check the kernel log instead)
|
||
sudo journalctl -k -n 50 --no-pager | grep -i "block\|deny"
|
||
```
|
||
|
||
**Root cause:** `host.docker.internal` resolves to Docker's bridge gateway
|
||
(commonly `172.17.0.1`), not to `localhost` or the mesh interface. Traffic
|
||
from a container to the host arrives via the `docker0` interface. Since the
|
||
UFW rule from Step 4 only allowed `wt0`, this traffic gets silently dropped
|
||
under default-deny.
|
||
|
||
This is a **different problem** from "Docker bypasses UFW for published
|
||
ports" (that's about *incoming* traffic to a container from outside the
|
||
machine, via Docker's own NAT rules bypassing UFW's chain). This case is a
|
||
container reaching *out* to something on the host — which does go through
|
||
UFW's normal `INPUT` chain like any other interface.
|
||
|
||
**Fix:**
|
||
|
||
```bash
|
||
sudo ufw allow in on docker0 to any port 8080 proto tcp
|
||
```
|
||
|
||
Re-test with a real chat message in the desktop's Open WebUI, and confirm
|
||
with `journalctl -u llama-swap` that a fresh request appears from the
|
||
container's Docker IP.
|
||
|
||
---
|
||
|
||
## Diagnostic cheatsheet (built up during this setup)
|
||
|
||
| Symptom | Likely meaning |
|
||
|---|---|
|
||
| Instant "connection refused" | Nothing is listening on that port |
|
||
| Hang / timeout | Something's listening, but a firewall is silently dropping packets |
|
||
| `modprobe: module not found` after a kernel update | Running kernel ≠ installed kernel package; needs a reboot |
|
||
| `netbird up` → `context deadline exceeded` | Vague — check `client.log` or journal for the *real* underlying error, don't assume it's the login step |
|
||
| Curl test passes when run *on* the target machine, fails from elsewhere | You tested via loopback, not the actual interface — re-test from a different device |
|
||
| Docker container can't reach a host service | Check `host.docker.internal` → `docker0` gateway → firewall rule for that interface specifically |
|
||
|
||
---
|
||
|
||
## Open items (see companion status document for full detail)
|
||
|
||
- UFW rule for `docker0`→llama-swap: written above, not yet confirmed tested
|
||
- Laptop's Open WebUI: still bound to all interfaces, not yet restricted to the mesh IP
|
||
- SSH remote access to the desktop: deliberately deferred, currently unreachable remotely
|
||
- Cline on the laptop: configured and confirmed working
|
||
- llama-swap's "reachable by all hosts" startup warning: expected, mitigated at the firewall layer — not a bug
|