# NetBird Mesh + Local LLM Remote Access — Setup Guide

Goal: reach the desktop's local LLM stack (llama-swap) securely from a laptop
and phone, over a self-hosted-friendly VPN mesh — not the raw LAN, not the
open internet.

## Architecture at a glance

```
Phone  ─┐
         ├─ NetBird mesh (encrypted, peer-to-peer) ─→  Desktop (Heimdall-home)
Laptop ─┘                                                 └─ llama-swap :8080
                                                            └─ UFW: only wt0 allowed in
```

- **NetBird Cloud** provides the control plane (who's allowed to talk to
  whom). Actual traffic between peers goes directly, peer-to-peer — it does
  not route through NetBird's servers.
- **llama-swap** on the desktop is only reachable via the mesh interface
  (`wt0`), enforced by UFW — not from the raw home LAN, not from the internet.
- **Open WebUI** runs locally on each client device (laptop) rather than
  centrally on the desktop, so the desktop doesn't need to expose an extra
  service.

---

## Prerequisites

- A NetBird Cloud account (free): https://app.netbird.io
- Arch-based Linux on desktop and laptop (commands below assume `yay`/`pacman`)
- `sudo` access on both machines

---

## Step 1 — Create a NetBird Cloud account

Go to https://app.netbird.io and sign up (Google/GitHub/Microsoft/email —
doesn't matter, it's just for the dashboard). You'll land on an empty
dashboard; peers will appear here as you add them below.

---

## Step 2 — Install and connect the NetBird client (any Arch-based device)

```bash
yay -S netbird              # or: paru -S netbird
netbird version              # sanity check (note: no "--" before version)

sudo netbird service install
sudo netbird service start
sudo netbird up
```

`netbird up` prints a login URL. Open it in a browser, log into your NetBird
account, and approve the device. The command then returns once approved.

### ⚠️ Known pitfall: "no such device" / TUN interface error

If `netbird up` fails with something like:

```
failed creating tunnel interface wt0: [error creating tun device: no such device]
```

and `sudo modprobe tun` responds with:

```
modprobe: FATAL: Module tun not found in directory /lib/modules/<version>
```

— this is **not a NetBird problem**. It means a kernel package update
replaced `/lib/modules/<new-version>/` on disk, but you're still *running*
the old kernel in memory (no reboot since the update). Confirm with:

```bash
uname -r          # currently running kernel
pacman -Q linux   # installed kernel package
```

If these two versions differ, **reboot** — that's the fix. This is a
general Arch/rolling-release gotcha, not specific to this tool.

### Verify the connection

```bash
sudo netbird status
```

Look for `Management: Connected`, `Signal: Connected`, and a NetBird IP
(usually `100.x.x.x`). Note this IP down for each device — you'll need it
below.

---

## Step 3 — Confirm connectivity between two peers

From one device, ping the other's NetBird IP:

```bash
ping -c 4 <other-peer-netbird-ip>
```

The **first** packet may be slow (100–200ms) — that's the mesh negotiating
the actual path between peers (NAT traversal). Subsequent packets should
drop to single-digit ms if a direct peer-to-peer path was found. If *all*
packets stay slow, the connection likely fell back to a relay server instead
of going direct — still functional, just not optimal.

Also check:

```bash
sudo netbird status
```

`Peers count: 1/1 Connected` confirms it. Note: NetBird uses **lazy
connections** — it only establishes the tunnel once there's real traffic, so
`0/1 Connected` immediately after joining is normal, not a failure.

---

## Step 4 — Lock down the LLM service with UFW (on the desktop)

```bash
sudo pacman -S ufw
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow in on wt0 to any port 8080 proto tcp
sudo ufw enable
sudo ufw status verbose
```

`wt0` is NetBird's virtual network interface — this rule only allows port
8080 traffic that arrives *via the mesh*, not the raw LAN interface, not
Docker's bridge, not the internet.

> **If you rely on remote SSH access**, add an allow rule for port 22
> *before* enabling default-deny, or you may lock yourself out. This guide
> assumes SSH access is either local-only or handled separately.

### Verify — from a *different* device, not the desktop itself

```bash
curl http://<desktop-netbird-ip>:8080/     # should respond
curl http://<desktop-lan-ip>:8080/         # should hang / time out
```

⚠️ **Run this from the laptop or another peer — not from the desktop
itself.** A machine connecting to its own IP typically routes over loopback
internally, which bypasses the interface-specific firewall rule entirely and
gives a false pass.

**Diagnostic signal worth knowing:** an *instant* "connection refused" means
nothing is listening on that port. A *hang/timeout* means something is
listening, but the firewall is silently dropping the packets. Different
causes, different fixes.

---

## Step 5 — Run llama-swap as a systemd service

Converts a manually-started process into something that auto-restarts on
crash and survives reboots.

```bash
sudo nano /etc/systemd/system/llama-swap.service
```

```ini
[Unit]
Description=llama-swap LLM router
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=heimdall
WorkingDirectory=/home/heimdall/apps/llama-swap
ExecStart=/home/heimdall/apps/llama-swap/llama-swap -config config.yaml
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
```

If you had it running manually beforehand, kill that process first (check
`ps aux | grep llama-swap` for its PID) so it doesn't conflict over port
8080 with the new service.

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now llama-swap.service
sudo systemctl status llama-swap.service
```

`enable` makes it start on every boot; `--now` also starts it immediately.
`Restart=on-failure` (not `always`) means it restarts on a crash, but not if
you intentionally stop it for maintenance.

---

## Step 6 — Prevent the desktop from sleeping

A sleeping desktop can't serve requests, mesh or not. Check both the
desktop-environment layer and the underlying system layer — they can
disagree even when one looks correctly configured.

```bash
cat /etc/systemd/logind.conf | grep -i idleaction
systemctl status sleep.target suspend.target
```

For a hard guarantee regardless of any GUI setting or future update:

```bash
sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target
```

Verify:

```bash
systemctl status sleep.target suspend.target hibernate.target hybrid-sleep.target
```

Each should now show `Loaded: masked`.

---

## Step 7 — Install Docker + Open WebUI on the laptop

```bash
sudo pacman -S docker
sudo systemctl enable --now docker.service
sudo usermod -aG docker $USER
```

⚠️ Group membership only applies after a fresh login — log out/in (or
reboot) before expecting `docker` commands to work without `sudo`.

```bash
docker run -d -p 3000:8080 \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main
```

Open `http://localhost:3000` — the **first account you create automatically
becomes admin**, no separate seed step needed.

> ⚠️ **Pending hardening step (not yet done):** this binds to all
> interfaces (`0.0.0.0:3000`), meaning it's reachable on *any* network the
> laptop joins, not just the mesh. To restrict it to the mesh only:
> ```bash
> docker stop open-webui && docker rm open-webui
> docker run -d -p <laptop-netbird-ip>:3000:8080 \
>   -v open-webui:/app/backend/data \
>   --name open-webui --restart always \
>   ghcr.io/open-webui/open-webui:main
> ```
> Trade-off: if the laptop's mesh IP ever changes, this binding breaks
> silently until updated.

---

## Step 8 — Point Open WebUI at llama-swap

In Open WebUI: **Admin Panel → Settings → Connections → OpenAI API**

- **URL:** `http://<desktop-netbird-ip>:8080/v1`
- **API Key:** any non-empty placeholder if llama-swap has no auth
  configured (most clients require *something* in the field even if it's
  not validated).

Save, and the model list should populate automatically. Test with a real
chat message, not just the model list loading — that confirms actual
inference is working end to end, not just the metadata endpoint.

---

## Step 9 — Add more devices (e.g. phone)

Install the official NetBird app, log in with the same account, approve the
device. It'll appear as a new peer in the dashboard automatically.

---

## Troubleshooting: a Docker container can't reach a service on its own host

Symptom: Open WebUI *on the desktop itself* (in Docker) can't reach
llama-swap via `host.docker.internal:8080`, even though llama-swap is
healthy and the same setup works fine from other peers over the mesh.

**Diagnosis path:**

```bash
# 1. Confirm the service is actually up
sudo ss -tlnp | grep 8080
sudo systemctl status llama-swap.service

# 2. Check if requests are even arriving
sudo journalctl -u llama-swap -n 20 --no-pager

# 3. Check the firewall's own block log
#    (Arch doesn't ship /var/log/ufw.log by default — check the kernel log instead)
sudo journalctl -k -n 50 --no-pager | grep -i "block\|deny"
```

**Root cause:** `host.docker.internal` resolves to Docker's bridge gateway
(commonly `172.17.0.1`), not to `localhost` or the mesh interface. Traffic
from a container to the host arrives via the `docker0` interface. Since the
UFW rule from Step 4 only allowed `wt0`, this traffic gets silently dropped
under default-deny.

This is a **different problem** from "Docker bypasses UFW for published
ports" (that's about *incoming* traffic to a container from outside the
machine, via Docker's own NAT rules bypassing UFW's chain). This case is a
container reaching *out* to something on the host — which does go through
UFW's normal `INPUT` chain like any other interface.

**Fix:**

```bash
sudo ufw allow in on docker0 to any port 8080 proto tcp
```

Re-test with a real chat message in the desktop's Open WebUI, and confirm
with `journalctl -u llama-swap` that a fresh request appears from the
container's Docker IP.

---

## Diagnostic cheatsheet (built up during this setup)

| Symptom | Likely meaning |
|---|---|
| Instant "connection refused" | Nothing is listening on that port |
| Hang / timeout | Something's listening, but a firewall is silently dropping packets |
| `modprobe: module not found` after a kernel update | Running kernel ≠ installed kernel package; needs a reboot |
| `netbird up` → `context deadline exceeded` | Vague — check `client.log` or journal for the *real* underlying error, don't assume it's the login step |
| Curl test passes when run *on* the target machine, fails from elsewhere | You tested via loopback, not the actual interface — re-test from a different device |
| Docker container can't reach a host service | Check `host.docker.internal` → `docker0` gateway → firewall rule for that interface specifically |

---

## Open items (see companion status document for full detail)

- UFW rule for `docker0`→llama-swap: written above, not yet confirmed tested
- Laptop's Open WebUI: still bound to all interfaces, not yet restricted to the mesh IP
- SSH remote access to the desktop: deliberately deferred, currently unreachable remotely
- Cline on the laptop: configured and confirmed working
- llama-swap's "reachable by all hosts" startup warning: expected, mitigated at the firewall layer — not a bug
