patsy aa5666a614 update

2026-08-07 16:01:24 +02:00

91 KiB

Raw Permalink Blame History

VPS & Infrastructure Setup — Running Guide

How to use this document: Upload this file at the start of every new chat session covering this project. At the end of each session, ask Claude to update it with a new section and download the result. This document is the single source of truth for what has been done, why, and what comes next.

For Claude reading this: Read the full document carefully. Do NOT automatically continue where the session log left off. Wait for Patsy to tell you what they want to work on today. When Patsy says "update the guide", generate the updated file and present it for download — do not ask follow-up questions or propose next steps.

Machines Reference

Machine	Hostname	User	OS	Role
Desktop	`Heimdall-home`	`heimdall`	EndeavourOS (Arch)	Anchor machine, local LLM host
VPS	IONOS VPS 6-8-240	`patsy`	Debian 13 (trixie)	Multi-app server
Laptop	`heymdall-750xed`	`heymdall`	EndeavourOS	Secondary access point

VPS specs: 6 vCore, 8 GB RAM, 240 GB NVMe SSD Desktop specs: AMD Ryzen 7 7700X (8-core), AMD Radeon RX 9070 XT (RDNA4, gfx1201, dedicated GPU on PCI bus 03:00.0, 16 GB VRAM — 16304 MiB confirmed via rocminfo) + Raphael iGPU (bus 12:00.0, both bound to amdgpu), Linux kernel 7.0.x. Single-partition scheme — / and /home share one ext4 filesystem (nvme0n1p3, 192G, 113G free as of Chat 10). Separate dedicated storage: /mnt/data — 1.6T ext4 partition (nvme0n1p2), permanently mounted via /etc/fstab, holds all GGUF model files under /mnt/data/models/ plus general storage (added Chat 10, replacing an old untouched Btrfs install found on the same partition — see Chat 10 log). Domain: bujour.de IONOS web console: always confirm this is reachable before any hardening step — it is the recovery path if SSH breaks.

NetBird mesh (added Chat 11): Desktop 100.105.210.21 · Laptop 100.105.247.153 · Phone joined as a peer, not yet pointed at any service. Home LAN subnet 192.168.178.0/24. Desktop's Docker bridge 172.17.0.0/16, gateway 172.17.0.1. The VPS is not yet a mesh member — see Offsite Backup under Open Items.

Architecture Decisions

Shell

Interactive: Zsh (Fish-like ergonomics via plugins, fully Bash-compatible)
Scripts: Bash/POSIX — always, no exceptions
Why: The VPS has no Zsh or Fish. Any script that touches the server must be portable. Zsh interactively gives autosuggestions and syntax highlighting without sacrificing compatibility.

SSH Keys — One Keypair Per Machine (Option A)

Desktop: id_ed25519 (heimdall@Desktop) — fingerprint SHA256:86v6Z+lTQR6xpZQ9zGZ2jY2i7nLrc7flG3Nbz1RIYE0
Laptop: separate id_ed25519 (patsy@laptop)
Why Option A over "one key per service": For a single-person setup, the machine is the real security boundary. If a machine is lost or compromised, remove that machine's public key from all services. One key per service adds complexity without meaningful security gain here.
Private keys never leave the device. Public keys go to the VPS and to each service (e.g. Forgejo).

VPS Multi-App Structure

Isolation: Docker Compose — each app lives in its own stack directory
Reverse proxy: Caddy — handles HTTP/HTTPS only; does not touch SSH traffic
Stacks directory: ~/stacks/ on VPS; each app has its own subdirectory with a compose.yaml
Shared Docker network: external network named proxy — Caddy and app containers connect via this

LLM Access

Routes over VPN mesh (NetBird), not via SSH tunnel through the VPS
Desktop is the local LLM host
Mesh status (confirmed Chat 11): NetBird Cloud mesh live across desktop, laptop, and phone — 3 peers, peer-to-peer connectivity verified as direct (not relayed). The VPS is not yet part of this mesh.
Access model: llama-swap on the desktop is firewalled via UFW to the wt0 (NetBird) interface only — reachable over the mesh, blocked on the raw home LAN.

Open WebUI Topology — Desktop Is the Phone's Target (Chat 12 decision)

Open WebUI now runs in two places: on the desktop (Docker, host port 3000, deployed Chat 10) and on the laptop (Docker, deployed during the NetBird mesh work — Chat 11), both pointed at the desktop's llama-swap.
Decision (Chat 12): the phone will reach Open WebUI through the desktop's instance, not the laptop's.
Why: simplifies the access model to one path (phone → desktop) instead of maintaining two live Open WebUI instances as parallel phone-access candidates.
Consequence: this promotes the desktop's own docker0/UFW bug (see Open Items) from a side issue to a direct blocker of the phone-access goal, and de-prioritizes (without cancelling) the laptop Open WebUI rebind that was queued from Chat 11.

Office / Document Suite — Nextcloud (self-hosted)

Decision: Long-term commitment to self-host Nextcloud on the VPS as the Google Docs / Microsoft Office replacement.
Why: Matches the self-hosting/EU-sovereignty philosophy already applied to the rest of the stack. Keeps document data under Patsy's own control, and reuses the existing Docker Compose + Caddy + proxy network pattern — becomes just another stack in ~/stacks/.
Rejected/parked alternatives and why:
- Proton Docs — already paid for, zero setup, genuinely end-to-end encrypted. Good for casual/personal docs today, but Proton is Swiss, not EU — doesn't satisfy the "own EU-hosted server" bar for the long term.
- OnlyOffice (self-hosted) — best MS Office format fidelity of the self-hosted options, but under active governance dispute: a March 2026 consortium (including IONOS, Nextcloud, and Proton) forked it into "Euro-Office" over concerns that OnlyOffice's dev team is Russia-based despite Latvian registration. Not rejected outright — parked as the pending editor-engine decision below.
- CryptPad — zero-knowledge, EU (French company, XWiki), lightweight to self-host. Kept as a secondary option for one-off sensitive shares, not the primary suite (own-editors reduce Office format fidelity).
- WPS Office — rejected. Closed-source, not end-to-end encrypted by default, documented history of content-scanning/access-blocking on cloud-stored files in China, a 2023 privacy-policy walkback on using uploaded docs for AI training, and was targeted by a 2020 US national-security executive order. Even in offline-only mode it offers no advantage over LibreOffice, since LibreOffice gives the same "nothing leaves the device" model but is auditable (open-source).
- LibreOffice — kept as the standing offline/local default regardless of what gets self-hosted; zero network exposure by design.
Still open: which editor engine to pair with Nextcloud — Collabora (LibreOffice-based, no governance baggage) vs. self-hosted OnlyOffice (better format fidelity, disputed governance) vs. waiting for Euro-Office (targeted stable release summer 2026, not production-ready yet). Needs its own options-first discussion before implementation, and hasn't been sequenced against the LLM/mesh/backup roadmap yet.

Local LLM Inference Engine — llama.cpp (not Ollama)

Decision: Build and run llama.cpp directly on the desktop for local LLM serving, rather than Ollama.
Why: Priority is the most efficient possible setup for coding and agent work. llama.cpp gives full control over the build itself — exact backend, quantization, and compile flags — rather than sitting behind Ollama's managed abstraction layer.
Serving happens via llama-server, which exposes an OpenAI-compatible API — Continue.dev and other agent tooling can point at it directly. Current llama.cpp builds also bundle a browser-based chat UI, so there's a casual-use frontend available with no separate install.

GPU Backend — Vulkan First, Benchmark Against HIP/ROCm (Option C)

Decision: Build llama.cpp with both the Vulkan and HIP/ROCm backends and benchmark on the actual hardware (RX 9070 XT, RDNA4, gfx1201) rather than defaulting to ROCm on the assumption that "the official AMD compute stack" is automatically best.
Why: gfx1201 only received official ROCm support in ROCm 7.2 (March 2026) — the kernels for this specific architecture are new and not yet fully mature. Current community benchmarks show the Vulkan backend can outperform HIP/ROCm on this exact GPU generation (the reverse of the usual pattern on older AMD cards), and there's a known bug where the HIP backend keeps RDNA4 GPUs permanently out of idle power state once initialized — a real concern given this desktop is meant to run 24/7.
Status — decided: Benchmark complete (Chat 9). ROCm 7.2.4-1 installed via pacman; gfx1201 detected cleanly in rocminfo with no known "2 ISAs" rejection bug. HIP backend built in a separate build-hip/ dir, confirmed genuinely GPU-active at runtime (not silent CPU fallback), then benchmarked head-to-head against Vulkan with llama-bench. Result: Vulkan beat HIP/ROCm by ~53–54% on both prompt processing (17,596 vs. 11,512 t/s) and token generation (332.8 vs. 215.6 t/s), with non-overlapping confidence intervals — a real result, not noise. Vulkan is the active inference backend. ROCm/HIP packages and the build-hip/ build are kept installed as a fallback (e.g. future training workloads), not used for inference.

Local LLM Multi-Model Roster & Orchestration

Decision: serve local LLM needs through five task-tiers, each loaded on-demand / sequentially rather than running everything concurrently.
Why: on a single 16GB VRAM card, "split roles," "meaningful headroom," and "a genuinely high-quality reasoning model" can't all be true at the same time. Sequential on-demand loading resolves the tension — headroom then applies per active model, not across the whole roster at once.
Orchestration: llama-swap — a single-binary Go proxy that sits in front of llama-server, exposes one OpenAI-compatible endpoint, and auto-loads/unloads models by requested name (TTL-based idle unload; groups for models that can't coexist in VRAM). Continue.dev and Open WebUI will both point at the llama-swap endpoint, not directly at llama-server.
Interface: Open WebUI (browser-based) chosen over a CLI client — keeps conversation history, which matters for day-to-day ADHD/task-support continuity; a CLI client can be added later pointed at the same endpoint if wanted.

Roster:

Tier	Role	Model	Approx. VRAM	Trigger
D	Daily/ADHD companion (`daily`)	Qwen3-8B, Q4_K_M — thinking mode disabled	~5 GB	On-demand
B	Coding help (`coder-fim`) — original FIM-autocomplete role now open, see note below	Qwen2.5-Coder-7B-Instruct, Q6_K	~6.25 GB	On-demand
A1	Architecture/design discussion (`architect`)	gpt-oss-20b, Q6_K	~12 GB	On-demand
A2	Debugging, top tier (`debug`)	Qwen3.6-35B-A3B, Q3_K_L (upgraded from planned Q3_K_M — see Chat 10)	85% of 16GB VRAM at tuned settings (see Key Configs)	On-demand
A2-review	Code review (`review`)	same weights as A2, exposed via llama-swap `aliases:` — zero extra VRAM, zero reload switching between `debug`/`review`	0 GB extra	On-demand
—	Documentation/vault/commits	no dedicated model — reuses whatever's already loaded	—	—

Why code review isn't its own model: debugging and review both need the same underlying skill (deeply understanding real code, not creativity) — the difference is the lens, not the capability. A dedicated review model can be added later if the generalist's reviews feel shallow in practice.
B's role note (Chat 10): originally designed around Continue.dev's dedicated FIM ghost-text autocomplete feature. The editor tool actually deployed is Cline instead (see new Architecture Decision below), which has no equivalent feature — B is still installed and usable as a manually-selected model for lightweight coding asks, but its original purpose no longer applies. Redefinition of this tier is an open item.
Status: fully deployed and confirmed working end-to-end (Chat 10). llama-swap v240 installed at ~/apps/llama-swap/. All 5 roster model files downloaded to /mnt/data/models/. Full working config — including A2's tuned MoE-offload/context settings — in Key Configs below. Open WebUI running in Docker, connected via http://host.docker.internal:8080/v1. Editor integration is Cline, not Continue.dev (plan changed — see next section).

Code Editor / Agent Tool — Cline (supersedes planned Continue.dev)

Original plan (Chat 9): wire Continue.dev to the llama-swap endpoint.
Finding that changed the plan (Chat 10): Continue.dev was acquired by Cursor in June 2026. Final release v2.0.0-vscode shipped, GitHub repo is now read-only. The extension still installs and runs today, and local/BYOK model support is reportedly intact in that final release — but there will be no further updates from the original team, ever. Verified across multiple independent sources before acting on it.
Alternatives compared, fairly, before deciding:
- Cline — agentic chat panel (reads files, proposes diffs, runs commands), Plan/Act dual-model modes, most actively maintained community option (62K+ stars, 5M+ installs, Apache 2.0). No dedicated FIM/ghost-text autocomplete.
- Tabby — self-hosted, purpose-built only for low-latency autocomplete. Would restore a true FIM role, but runs its own separate inference server rather than routing through llama-swap — real added complexity for a "nice to have," not the core workflow.
- Aider — terminal-first, git-aware pair-programmer. Actively maintained, but a genuinely different workflow shape (CLI, not editor-panel) than what was being replaced.
- Roo Code — a Cline fork with a near-identical feature set; smaller community, same trade-offs as Cline with less long-term certainty.
- VS Code's own native BYOK + Agent Mode — real and current (added by Microsoft through 2026): can point VS Code's built-in Chat/Agent features at llama-swap directly, no extension, no GitHub account or Copilot subscription needed for this path. Same autocomplete gap as Cline (BYOK explicitly excludes inline completions). Rejected specifically for vendor-entanglement reasons — tied to GitHub Copilot's evolving product/policy decisions, versus Cline being independent and forkable if maintenance ever stalls.
Decision: Cline. Actively maintained, independent of any single vendor's roadmap, and its Plan/Act modes map cleanly onto tiers already built:
- Plan Mode → architect
- Act Mode → debug
Trade-off accepted: no dedicated FIM/ghost-text autocomplete in the new setup — same limitation as nearly every alternative considered except Tabby. coder-fim (B)'s original role is now open (see Open Items).

Current Stack on VPS

Service	URL	Ports	Stack location
Caddy (reverse proxy)	—	80, 443	`~/stacks/caddy/`
Forgejo (git server)	`git.bujour.de`	443 (web, via Caddy), 2222 (SSH)	`~/stacks/forgejo/`
CheckPoint Ehrenamt	TBD	via Caddy	`~/stacks/checkpoint/`

What Is Confirmed Working

✅ Key-based SSH to VPS from both desktop and laptop
✅ VPS hardening: PasswordAuthentication no, UFW firewall, Fail2ban
✅ IONOS web console confirmed as recovery path
✅ Multi-app Docker Compose structure in place
✅ Caddy running as reverse proxy
✅ CheckPoint Ehrenamt deployed and running
✅ CheckPoint daily backup configured (currently on-VPS only — offsite pending)
✅ Forgejo running at git.bujour.de
✅ Forgejo SSH port mapping fixed (host 2222 → container 22)
✅ UFW rule added for port 2222
✅ IONOS firewall opened for port 2222
✅ Forgejo SSH confirmed working — vault synced across desktop and laptop
✅ Bluetooth autostart fixed on laptop (EndeavourOS)
✅ Knowledge vault created on laptop at ~/Schreibtisch/05_vault/
✅ Locale fixed on laptop — en_GB language, de_DE formats via KDE System Settings
✅ VS Code crash fixed on laptop — replaced code (pacman) with visual-studio-code-bin (AUR)
✅ VS Code workspace set up — 05_vault + 02_DEV open simultaneously
✅ Vault Git repo synced across desktop and laptop via Forgejo (git pull origin main / git push origin main)
✅ Pandoc installed (v3.6) on laptop — confirmed working (pandoc --version)
✅ Pan Am CSS (styling.css) downloaded — confirmed valid CSS, ready for use
✅ fzf + bash-completion activated on laptop (~/.bashrc) — fuzzy history, file, dir search
✅ fzf + bash-completion installed and activated on VPS (~/.bashrc)
✅ SSH agent auto-start configured in laptop ~/.bashrc
✅ forgejo_laptop private key permissions fixed (chmod 600)
✅ VPS server structure prompt written (04_VPS_Serverstruktur_Prompt.md) — reusable for future deployment chats
✅ Vault file bash_fzf_setup.md created and pushed to Forgejo
✅ Desktop readiness-audit process established before installing/building anything (kernel driver, Vulkan stack, build tools, disk space, kernel version, group membership checked up front)
✅ llama.cpp built from source on desktop with Vulkan backend (GGML_VULKAN=ON, Ninja, Release) — GPU offload confirmed via smoke test (gemma-3-1b-it, ~150–260 t/s generation)
✅ heimdall video/render group membership confirmed active (groups output shows both — relogin/reboot happened between Chat 8 and Chat 9)
✅ ROCm 7.2.4-1 installed via pacman; gfx1201 confirmed cleanly detected in rocminfo (no known RDNA4 "2 ISAs" rejection bug hit)
✅ llama.cpp HIP/ROCm backend built (build-hip/), confirmed linked against real ROCm libraries via ldd, and confirmed genuinely GPU-active at runtime (not silent CPU fallback) via CUDA Graph ... reused log lines
✅ Vulkan vs. HIP/ROCm benchmarked head-to-head with llama-bench — Vulkan confirmed faster by ~53–54% on both prompt processing and token generation; GPU backend decision finalized (Vulkan active, ROCm kept as fallback)
✅ 5-tier local LLM roster designed and agreed (daily/ADHD companion, FIM autocomplete, architecture discussion, debugging, code review) — architecture: on-demand sequential swapping via llama-swap, Open WebUI as the browser interface (full detail under Architecture Decisions)
✅ Vault files build_deps_headers_vs_runtime.md, llama_cpp_vulkan_rx9070xt.md (Chat 8), vulkan_vs_hip_benchmark_gfx1201.md, local_llm_multi_model_roster.md (Chat 9) written — none yet confirmed pushed to Forgejo, verify next session
✅ /mnt/data — dedicated 1.6T ext4 partition, permanently mounted via /etc/fstab (UUID-based), holding all GGUF model files at /mnt/data/models/
✅ llama-swap v240 installed at ~/apps/llama-swap/, confirmed working via one-model smoke test before the full roster was added
✅ All 5 roster models downloaded and confirmed working through llama-swap: daily, coder-fim, architect, debug/review (shared backend via aliases)
✅ Qwen3-8B (daily) thinking-mode disabled and confirmed off (--chat-template-kwargs '{"enable_thinking":false}') — verified via absence of reasoning_content and a 3-token vs. 129-token response for the same prompt
✅ A2 (debug/review) VRAM and context tuned and confirmed stable via radeontop -b 03:00.0: --n-cpu-moe 16 --ctx-size 65536 --parallel 1 → 85% VRAM
✅ debug/review reload gap fixed via llama-swap aliases: — confirmed via server logs (single health-check event, not two) and response timing, not inferred from output alone
✅ Docker installed on desktop for the first time, verified working (hello-world test container)
✅ Open WebUI running in Docker (host port 3000), connected to llama-swap at http://host.docker.internal:8080/v1, all 5 models visible and usable
✅ Cline installed and configured (OpenAI-Compatible provider → http://localhost:8080/v1), Plan Mode → architect, Act Mode → debug, confirmed working end-to-end
⚠️ Continue.dev not wired up — superseded by the Cline decision (see Architecture Decisions); extension not installed
✅ NetBird Cloud mesh live across desktop, laptop, and phone (3 peers) — peer-to-peer connectivity confirmed direct, not relayed
✅ llama-swap converted to a proper systemd service (llama-swap.service, Restart=on-failure, enabled for boot) — replaces the previously manually-run foreground process
✅ UFW on desktop: default-deny incoming / default-allow outgoing; port 8080 (llama-swap) restricted to the wt0 (NetBird) interface only — verified from the laptop (mesh access works; a raw LAN attempt hangs/times out rather than being refused instantly)
✅ Desktop sleep prevention — KDE PowerDevil and systemd-logind confirmed already defaulting to no idle action, plus sleep.target/suspend.target/hibernate.target/hybrid-sleep.target hard-masked at the systemd level as a belt-and-suspenders guarantee (desktop is meant to run 24/7 as LLM host and future backup destination)
✅ Open WebUI installed on the laptop (Docker), connected to the desktop's llama-swap via http://100.105.210.21:8080/v1 — all 5 roster models tested with real inference over the mesh, not just model-list loading
✅ Cline on the laptop repointed at the desktop's llama-swap over the mesh — confirmed working
✅ Phone joined the NetBird mesh, shows as a connected peer — not yet pointed at any service
⚠️ Desktop's own Open WebUI container still cannot reach the desktop's own llama-swap (docker0 UFW gap) — root cause confirmed, fix identified, not yet applied (see Key Configs and Open Items)
✅ (Side quest, Chat 13) architect model system-prompt bugs debugged via test-driven iteration in Open WebUI: confirmed fixes for (a) generic templated questions that ignored prior answers, and (b) a silent refusal to produce output after the user had explicitly and calmly granted permission earlier in the same conversation
✅ (Side quest, Chat 13) Scope-discipline rule (no full-file regeneration on a small edit request) confirmed working — tested on a trivial one-line change only
⚠️ (Side quest, Chat 13) Frustration-triggers-dropped-content-constraint hypothesis tested once, not reproduced — low confidence, not conclusively ruled out either way
⚠️ (Side quest, Chat 13) Final 5-rule architect system prompt used successfully via direct API calls (test harness) — not yet confirmed saved in Open WebUI's persistent model settings

Key Configs

UFW Status (VPS)

OpenSSH        ALLOW
80/tcp         ALLOW
443/tcp        ALLOW
443/udp        ALLOW
2222           ALLOW

`~/stacks/forgejo/compose.yaml`

services:
  forgejo:
    image: codeberg.org/forgejo/forgejo:10
    container_name: forgejo
    restart: unless-stopped
    environment:
      - USER_UID=1000
      - USER_GID=1000
    volumes:
      - forgejo_data:/data
    networks:
      - proxy
    expose:
      - "3000"
    ports:
      - "2222:22"
volumes:
  forgejo_data:
networks:
  proxy:
    external: true

SSH Agent Auto-Start (`~/.bashrc` on laptop)

# SSH agent
if [ -z "$SSH_AUTH_SOCK" ]; then
    eval "$(ssh-agent -s)"
    ssh-add ~/.ssh/id_ed25519
    ssh-add ~/.ssh/forgejo_laptop
fi

fzf + bash-completion (`~/.bashrc` — both laptop and VPS)

# bash-completion
[[ -r /usr/share/bash-completion/bash_completion ]] && . /usr/share/bash-completion/bash_completion

# fzf
eval "$(fzf --bash)"

llama.cpp HIP/ROCm Build Command (desktop, `~/apps/llama.cpp`)

HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
    cmake -S . -B build-hip -G Ninja -DGGML_HIP=ON -DGPU_TARGETS=gfx1201 -DCMAKE_BUILD_TYPE=Release
cmake --build build-hip -j 16

Kept as a separate build directory from the active Vulkan build/. Not the active inference backend (Vulkan won the benchmark — see Architecture Decisions) but kept buildable/installed as a fallback.

`/mnt/data` fstab entry (desktop, `/etc/fstab`)

UUID=bfa4f976-abad-4912-be85-b579c857f63a  /mnt/data  ext4  defaults  0  2

After editing /etc/fstab, run sudo systemctl daemon-reload before sudo mount -a — systemd caches fstab and won't pick up edits otherwise (its own mount output explicitly says so, it's not an error).

`~/apps/llama-swap/config.yaml` (desktop, full working roster)

models:
  gemma-smoke-test:
    cmd: /home/heimdall/apps/llama.cpp/build/bin/llama-server --port ${PORT} --model /mnt/data/models/gemma-3-1b-it-Q4_K_M.gguf

  daily:
    cmd: /home/heimdall/apps/llama.cpp/build/bin/llama-server --port ${PORT} --model /mnt/data/models/Qwen_Qwen3-8B-Q4_K_M.gguf --chat-template-kwargs '{"enable_thinking":false}'

  coder-fim:
    cmd: /home/heimdall/apps/llama.cpp/build/bin/llama-server --port ${PORT} --model /mnt/data/models/Qwen2.5-Coder-7B-Instruct-Q6_K.gguf

  architect:
    cmd: /home/heimdall/apps/llama.cpp/build/bin/llama-server --port ${PORT} --model /mnt/data/models/gpt-oss-20b-Q6_K.gguf

  debug:
    cmd: /home/heimdall/apps/llama.cpp/build/bin/llama-server --port ${PORT} --model /mnt/data/models/Qwen_Qwen3.6-35B-A3B-Q3_K_L.gguf -ngl 999 --n-cpu-moe 16 --ctx-size 65536 --parallel 1
    aliases:
      - review

Not yet configured (deliberately deferred): TTL/auto-unload tuning, and -listen localhost:8080 to restrict llama-swap to loopback only (currently reachable on all network interfaces — flagged as an open security item, see Open Items).

Open WebUI (desktop, Docker)

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Mapped to host port 3000, not --network=host, specifically to avoid colliding with llama-swap already on host port 8080. --add-host=host.docker.internal:host-gateway is required on Linux for the container to reach the host — automatic on Docker Desktop (Mac/Windows) but not here. Connection configured in Open WebUI's Settings → Connections pointing at http://host.docker.internal:8080/v1.

Desktop UFW — `docker0` Fix (identified, NOT yet applied)

sudo ufw allow in on docker0 to any port 8080 proto tcp

Needed because the desktop's own Open WebUI container is configured with host.docker.internal:8080, which resolves to the Docker bridge gateway (172.17.0.1) — so the request arrives on the docker0 interface, which was never given an allow rule (only wt0 was opened for llama-swap). Root cause confirmed via sudo journalctl -k | grep -i block. Status: not yet run — pending Patsy's go-ahead (Chat 12). After running it, verify with sudo journalctl -u llama-swap -n 5 --no-pager — a fresh request should appear from the container's Docker IP (172.17.0.2).

Desktop SSH Config (`~/.ssh/config`)

Host git.bujour.de
    HostName git.bujour.de
    User git
    IdentityFile ~/.ssh/id_ed25519

`architect` Model System Prompt — v2, Test-Confirmed (Side Quest, Chat 13)

You are a design-questions assistant. Follow this process:

1. Ask exactly one question at a time. Wait for the user's answer before asking the next.

2. Before writing each new question, use the user's previous answers explicitly.
Never ask something the user would have to guess from nothing (e.g. "what hex/hue for
each role?" when they've said they don't know color theory). Instead, propose 2-3
concrete options that already reflect their earlier answers, and ask them to pick
or adjust.

3. Two kinds of signals exist here: PROCESS signals (pacing, frustration, "hurry up",
"this is slow") and CONTENT signals (explicit statements about what the final output
may or may not contain, e.g. "no code", or later "an HTML page is fine now").
- PROCESS signals never change a content constraint. Respond by adjusting pace only —
skip ahead, group questions, move to a full answer sooner.
- An explicit, calm CONTENT signal changes the constraint going forward. If the user
states outright that something is now fine, honor it — do not keep enforcing a rule
they've already superseded.
- Never refuse a request with no explanation. If declining something, say in one
sentence why, referencing the specific constraint you believe is still active.
- If unsure whether a message changed a content constraint, ask one direct question
rather than silently guessing either way.

4. Only produce a final deliverable when the user explicitly asks for it, or when
all questions are answered and you've confirmed that with them.

5. When producing any deliverable (code, file, or other artifact), scope the output
strictly to what was asked in that turn:
- If the user asks to change, fix, or adapt one part of something already produced,
output only that changed part (or a minimal diff/patch), not the entire file
regenerated from scratch.
- Never add anything not requested: no extra elements, no unrequested features, no
"while I'm at it" additions.
- Match the size of your answer to the size of the request. A request to fix one
color is a few lines, not a full-file rewrite.
- If it's genuinely unclear how much of the existing work needs to change, ask in
one sentence before regenerating anything large.

Test-confirmed piece by piece (see Session Log, Chat 13): Rules 1–2 fixed generic templated questions that ignored prior answers; Rule 3 fixed a silent no-explanation refusal after an explicit permission change (confirmed via a scripted re-run); Rule 5 fixed full-file regeneration on a trivial one-line edit. Rule 4 unchanged from the first draft, never independently stress-tested. Not yet confirmed: this exact text saved persistently in Open WebUI (Workspace → Models → Edit → architect → System Prompt) — testing so far has sent it per-request via direct API calls from the script below, not through the UI's saved setting.

`test3_scripted.py` — Reusable API Test Harness for the `architect` Model (Side Quest, Chat 13)

#!/usr/bin/env python3
"""
Sends a fixed conversation to the architect model via its OpenAI-compatible API
(llama-swap) — used to re-test prompt-behavior fixes without manually retyping
the same conversation and risking wording drift between runs.

Fill in BASE_URL and MODEL for your setup, then run:
    python3 test3_scripted.py
"""

import requests

# ---- fill these in for your setup ----
BASE_URL = "http://localhost:8080/v1"   # llama-swap OpenAI-compatible endpoint
MODEL = "architect"                      # model alias as configured in llama-swap
# ----------------------------------------

SYSTEM_PROMPT = """<see 'architect Model System Prompt — v2' above>"""

# Edit this list to change what's being tested
USER_TURNS = [
    "I am building an app and in that app I need calming adhd friendly colours. "
    "What do I need to look out for when creating a design scheme? I do not wish "
    "to have code. Asking me questions. I will answer them one by one until all "
    "are answered. Please be patient, we will get through all of it.",
    "pastel colours",
    "balanced pastels, and I need a calm dark mode too",
    "For this it is fine to create an HTML page.",
    "Can you now give me the HTML template?",
]

def main():
    messages = [{"role": "system", "content": SYSTEM_PROMPT}]
    transcript_lines = []
    for turn in USER_TURNS:
        messages.append({"role": "user", "content": turn})
        resp = requests.post(
            f"{BASE_URL}/chat/completions",
            json={"model": MODEL, "messages": messages, "temperature": 0.3},
            timeout=300,
        )
        resp.raise_for_status()
        assistant_msg = resp.json()["choices"][0]["message"]["content"]
        messages.append({"role": "assistant", "content": assistant_msg})
        transcript_lines.append(f"### USER\n{turn}\n")
        transcript_lines.append(f"### ASSISTANT\n{assistant_msg}\n")
    with open("test3_transcript.md", "w") as f:
        f.write("\n".join(transcript_lines))
    print("Saved full transcript to test3_transcript.md")

if __name__ == "__main__":
    main()

Delivered as a downloadable file in Chat 13. Not yet pushed to Forgejo.

Hard-Won Lessons

Never harden before testing

Before any SSH/firewall hardening — always verify key access works first AND confirm the hoster's web console is reachable as a recovery path. Locking yourself out with no recovery path means a full reinstall.

Three firewall layers for Docker

Traffic from the internet to a Docker container passes through three gates in order:

Hoster network firewall (IONOS) — blocks before anything reaches the VPS
UFW — blocks before anything reaches Docker
Docker port mapping — forwards traffic into the container

All three must be open. Missing any one of them silently breaks connectivity.

`expose` ≠ `ports` in Docker Compose

expose — makes a port visible to other containers on the same Docker network only. No external access.
ports — maps a host port to a container port. Required for anything reachable from outside.

Port conflicts with system SSH

The VPS system SSH daemon owns port 22 on the host. Any service needing SSH must use a different host port. Convention: port 2222 for git services (Forgejo, Gitea, GitLab).

UFW requires `sudo` on Debian

ufw lives in /usr/sbin/, which is not in the default user PATH on Debian — by design. Always sudo ufw, never just ufw.

DB seed + first admin user from the start

CheckPoint and any database-backed app: always seed the DB and create the first admin user at deployment time. This was missed in a previous attempt and caused extra work.

SSH diagnosis shortcuts

Permission denied = connection works, authentication fails (key/user/permission issue)
Connection refused = nothing listening on that port, or firewall blocking
ssh -v shows exactly how far the connection gets — separates network problems from auth problems

systemd enable vs start are separate things

systemctl start — starts a service right now, this session only
systemctl enable — tells systemd to start it automatically on every boot
Installing a package does NOT enable it automatically — always check with systemctl is-enabled <service>
Symptom: service works after restart but not after reboot → it was never enabled

KDE overrides `/etc/locale.conf`

On KDE Plasma, locale settings must be changed in System Settings → Region & Language, not in /etc/locale.conf. KDE writes its own values at login that override the system file. Changes to /etc/locale.conf alone will have no effect on a KDE system.

Locales must be generated before use

On Arch/EndeavourOS, locales are not installed via pacman. They are generated from /etc/locale.gen:

Uncomment the desired locale in /etc/locale.gen
Run sudo locale-gen
Then set it in KDE System Settings (or /etc/locale.conf on non-KDE systems)

VS Code on Arch — always use AUR binary

code from pacman = open-source rebuild — known stability issues (crashes, rendering bugs)
visual-studio-code-bin from AUR = official Microsoft binary — stable
Install: yay -S visual-studio-code-bin
Uninstall pacman version first: sudo pacman -R code

git fetch vs git pull

git fetch — downloads new commits from the remote but does NOT touch your working files
git pull — fetch + applies the changes to your local files (what you want for day-to-day sync)
Day-to-day rhythm: git pull origin main before starting work; git add . && git commit -m "…" && git push origin main when done

curl -L is required for redirected downloads

curl -O <url> without -L will silently save a redirect error page (HTML 301) instead of the actual file
Always use curl -LO <url> — -L follows redirects, -O saves with the original filename
Symptom: downloaded file starts with <html> instead of the expected content

Pandoc CSS must be embedded for portable output

pandoc --css=styling.css links the CSS as a relative path — breaks if files are separated
pandoc --css=styling.css -s --self-contained embeds everything into one portable HTML file
A self-contained file with embedded fonts/CSS will be significantly larger (50KB+) than a bare HTML file

A backup on the same machine is not a backup

A backup that lives on the same server as the data it protects only guards against accidental deletion — not against hardware failure, provider incidents, or accidental wipes. The backup destination must be a physically separate machine. Until data leaves the server, it is a single copy.

SSH agent must be started before ssh-add

ssh-add fails with "Could not open a connection to your authentication agent" if no agent is running
Fix: eval "$(ssh-agent -s)" then ssh-add
To avoid this every session, add an agent auto-start block to ~/.bashrc (see Key Configs below)

Private key permissions must be 600

SSH refuses to load a private key with permissions wider than 600
Symptom: "WARNING: UNPROTECTED PRIVATE KEY FILE! ... This private key will be ignored."
Fix: chmod 600 ~/.ssh/<keyname>
The .pub file (public key) is fine at 644 — only the private key needs restricting

Never ssh-add the public key

ssh-add only takes private keys — adding the .pub file is an error
The public key goes to the server's authorized_keys / Forgejo UI; the private key goes to the agent

git divergent branches need a reconcile strategy

If local and remote have both moved forward independently, git pull alone fails with "need to specify how to reconcile divergent branches"
For a personal repo where you're the only user: git pull --rebase origin main is the clean fix
Rebase replays your local commits on top of the remote ones — no merge commit noise

fzf gives Fish-like UX in plain Bash

No need to switch shells for autocomplete ergonomics
fzf --bash adds: fuzzy history search (Ctrl+R), fuzzy file search (Ctrl+T), fuzzy directory jump (Alt+C)
Same config works on Arch (pacman) and Debian (apt) — only the install command differs

`man` is faster than googling for system concepts

man <command> — full manual for any system tool
/SEARCHTERM inside man — jumps to first match; n for next match
man 7 locale — section 7 is "concepts"; use section numbers when a manpage references another like locale(7)
Config files often document themselves — scroll to the top before editing any system config file

"Registered in the EU" isn't the same as "EU-controlled"

A company's legal registration doesn't guarantee where its actual development team or control sits. Both OnlyOffice (Latvian registration, Russia-based dev team per a 2026 consortium fork) and WPS Office (global product, Chinese parent company subject to China's cybersecurity/content laws) showed this gap. When sovereignty matters, check who actually builds and can compel the software, not just where it's incorporated.

Swiss ≠ EU

Proton is a Swiss company. Switzerland has an EU adequacy decision and GDPR-comparable law (FADP), but it is not the EU. If a project's hosting rule is meant as a literal "EU soil" boundary rather than a "GDPR-equivalent is fine" boundary, decide that explicitly — don't let "privacy-focused" quietly substitute for "EU-hosted."

Closed-source + offline-only still isn't as good as open-source + offline-only

If cloud sync is disabled anyway (no data leaves the device), a closed-source app offers no privacy advantage over an open-source one with the same offline model — and loses the ability to audit what it does. When the choice is "local-only closed-source" vs. "local-only open-source" (e.g. WPS offline vs. LibreOffice), the open-source option wins with no trade-off.

Compile-time headers vs. runtime libraries are separate concerns

A working driver does not mean the matching build-time headers are installed, and vice versa. vulkan-icd-loader + vulkan-radeon let an already-compiled program run against the GPU right now (games, desktop compositor, etc.); vulkan-headers + spirv-headers are plain .h declaration files the compiler needs to build new source code that calls those same functions. Hit both errors in sequence while building llama.cpp despite Vulkan already "working" on the desktop. Applies generally to any -dev/-headers package split, not just Vulkan.

The "official" vendor stack isn't automatically the fastest on brand-new hardware

On a just-released GPU architecture, driver/kernel maturity for that specific chip can lag behind a more generic, broadly-supported API — even when the generic API isn't the vendor's primary compute path. On the RX 9070 XT (RDNA4, gfx1201), confirmed via head-to-head llama-bench in Chat 9: Vulkan beat HIP/ROCm by ~53–54% on both prompt processing and token generation, the reverse of the usual pattern on older AMD cards — because ROCm's gfx1201 support only matured in March 2026. Worth checking current per-GPU-generation benchmarks before assuming "the vendor's official stack" wins by default, and worth actually running the benchmark rather than stopping at a prediction.

A clean compile is not proof of real GPU execution at runtime

Some ROCm/llama.cpp combinations are known to silently fall back to CPU with zero error message, while startup logs still claim GPU offload. Neither "it compiled" nor "it ran and produced output" rules this out — token speed alone is suggestive but not proof. The reliable check is a GPU-only runtime code path actually firing in the logs (e.g. CUDA-graph reuse lines for llama.cpp's HIP backend) — something that structurally cannot appear if compute silently fell back to CPU.

ggml's HIP backend logs say "CUDA" — this is expected, not a bug

llama.cpp's ggml HIP backend is generated from the same source as its CUDA backend via AMD's HIPIFY translation tool. Internal log lines and function names (CUDA Graph ... reused, ggml_backend_cuda_graph_compute) still say "cuda" even when genuinely compiled and running via HIP/ROCm on an AMD GPU. Don't mistake this for a misconfiguration — it's shared codegen, and it's actually useful: those CUDA-only code paths are the evidence that real GPU execution is happening (see previous lesson).

HuggingFace's local cache nests the actual model file — point `-m` at the file, not the folder

huggingface_hub's local cache stores a model under models--<org>--<name>/snapshots/<hash>/<actual-file>.gguf, not directly in the model's named folder. Pointing llama-cli -m at the folder itself fails with gguf_init_from_reader: failed to read magic — the loader tried to read a directory as if it were a GGUF file's byte header. Use find ~ -iname "*modelname*" to locate the actual .gguf file when unsure, then pass that exact path.

Cross-check model VRAM claims against your own confirmed hardware numbers, not just the model card

A model recommendation this session (Qwen 3.6-27B at ~17GB Q4) was made against a general "16GB-class" guideline without re-checking it against the desktop's own already-confirmed figure (16304 MiB / ~16GB exactly, from rocminfo in Chat 8) — and didn't fit. Caught before any download happened, but the lesson generalizes: once a machine's exact hardware limit is known, use that exact number for sizing decisions, not the nearest round-number tier from a guide.

Sequential on-demand model loading resolves VRAM-constrained multi-role tension

On one VRAM-limited GPU, wanting (a) multiple distinct model "roles," (b) real headroom, and (c) at least one genuinely large/high-quality model, cannot all be satisfied simultaneously. Loading models sequentially, on-demand (via a swapping proxy like llama-swap) resolves this — headroom then applies per active model, not across a permanently-loaded roster. The trade-off is a few seconds' load delay when switching roles, not a compromise on any of the three goals.

Not every task needs a dedicated model file — an alias with a different system prompt can be enough

Two tasks needing the same underlying skill (e.g. debugging and code review both need deep, careful code understanding) don't need two separate downloaded models. A swapping proxy can expose the same weights under two names, each launched with a different system prompt — zero extra VRAM or disk cost. Reserve genuinely separate model files for roles needing a different skill (e.g. fast FIM completion vs. deep multi-step reasoning).

Front-load a readiness audit before installing/building anything

Checking all prerequisites (kernel driver bound correctly, required packages present, disk space, kernel version, group/permission requirements) before starting an install or build catches missing pieces individually and cheaply, instead of hitting them one at a time mid-build where each failure costs a re-run. Directly reduced debugging time on the llama.cpp build — only two small, expected gaps (cmake, vulkan-headers) surfaced, both caught before any download or compile started.

Go release binaries use `amd64`/`arm64`, not `x86_64`/`aarch64`

Go (and Docker) inherited this naming from the old AMD64 instruction-set lineage. Relevant for identifying the correct asset on any future prebuilt Go-binary download, not just llama-swap.

`radeontop` needs an explicit `-b <bus>` on multi-GPU machines

Without it, radeontop can silently monitor the wrong device. On this desktop it defaulted to the Raphael iGPU (bus 12:00.0, reported UNKNOWN_CHIP, ~488MB "VRAM") instead of the actual RX 9070 XT (03:00.0, 16GB) — a reading that looked plausible enough to almost be trusted at face value. Confirm the correct bus with lspci | grep -i vga first on any machine with more than one GPU, then always pass -b.

`mv`-ing a Hugging Face cache "file" moves a broken symlink, not the model

The HF cache is content-addressable: what looks like a named .gguf file under snapshots/<hash>/ is actually a relative symlink pointing at a hash-named blob elsewhere. mv relocates the symlink itself, not its target, and the relative path then resolves incorrectly from the new location — silently broken until something tries to read it. Fix: don't move cache entries at all. Download directly with curl -L -o straight into the destination folder, bypassing the cache mechanism entirely. (Extends the Chat 9 lesson about pointing -m at the file inside snapshots/<hash>/, not the named folder — same underlying cache structure, a different failure mode.)

`--n-cpu-moe`, not `-ngl`, is the right lever for partial MoE offload

For Mixture-of-Experts models, -ngl offloads whole layers indiscriminately — attention, shared weights, and experts alike. --n-cpu-moe N specifically pushes N layers' worth of expert weights to CPU RAM while keeping always-active layers on GPU. Because only a small fraction of experts are active per token regardless of where they physically sit, the performance cost per GB of VRAM freed this way is much smaller than offloading whole layers.

An unset `--ctx-size` can dominate VRAM usage more than any offload setting

llama-server defaults to a model's native training context if --ctx-size isn't set — for Qwen3.6-35B-A3B that's 262K tokens (194K once auto-split across 4 parallel slots), pre-allocated as KV cache in VRAM at load time. This dwarfed the effect of --n-cpu-moe in initial testing (going from 8→12 barely moved VRAM), making the offload flag look far less effective than it actually was until context was constrained first and isolated as its own variable.

llama.cpp's KV cache is allocated once, in full, at load — not grown incrementally

A VRAM reading taken right after a model loads with an empty context is the number for the entire session up to that --ctx-size limit, not a floor that creeps upward as a conversation fills in. No hidden risk of a session that starts safely and later runs out of memory mid-task.

llama-swap's `aliases`, not `groups`, is for "same backend, multiple names"

llama-swap treats every models: entry as an independent backend by default — even two entries with byte-identical cmd: strings get separately stopped/started, costing a full reload on every switch between them. groups is for running genuinely different models concurrently (not useful on a VRAM-constrained single-GPU desktop). aliases is the correct tool for "one resident backend, reachable under more than one name" — confirmed via server logs, not just correct output, since a correct response doesn't by itself prove no reload happened.

A quantizer's own "not recommended" note is worth reading before downloading

bartowski's quant tables flag specific quant levels as lower quality even within the same size class — Q3_K_M was explicitly flagged "low quality" while Q3_K_L, in the same table right next to it, was flagged "recommended" for only ~660MB more. Worth checking the actual per-quant notes in the model card, not just picking a quant level by name or habit.

Duplicate disk labels are a real collision risk, not just cosmetic

Two entirely different physical disks ended up sharing the label GG — a newly-formatted internal partition and a pre-existing external drive — purely by coincidental naming habit. Anything that mounts "by label" going forward would have had a genuine chance of grabbing the wrong device. Caught by noticing lsblk showed the same label twice; worth checking for collisions immediately after naming any new partition.

Open-source license alone doesn't guarantee a tool keeps shipping updates

Continue.dev being Apache 2.0 means it keeps running even after its acquisition and the original team's departure — but the license alone doesn't prevent the project going stale. "Actively maintained" and "still technically works today" are separate questions. For a project that explicitly prioritizes long-term sustainability over short-term convenience, check which one actually matters before building around a tool, not after.

Docker container → host traffic is a distinct UFW gap from published-port traffic

The existing "three firewall layers" lesson (above) covers inbound traffic from the internet into a published container port. There's a separate, easily-missed case: a container reaching back out to a service running on its own host (e.g. via host.docker.internal, which resolves to the Docker bridge gateway) arrives on the docker0 interface — and needs its own explicit UFW allow rule, independent of whatever rule was already given to the mesh (wt0) or LAN interfaces. Confirmed via sudo journalctl -k | grep -i block showing UFW blocking the request specifically on docker0.

Hang/timeout vs. instant refusal is a real firewall diagnostic signal

When testing whether a UFW rule genuinely restricts a port to one interface, a blocked connection attempt hangs or times out rather than being refused instantly (DROP vs. REJECT behavior). Confirmed on the desktop: mesh (wt0) access to llama-swap worked normally, while a raw LAN attempt from the laptop hung instead of failing fast. Useful for confirming a restriction is actually in effect, not just inferring it from the rule's presence in ufw status.

A kernel update without a reboot can break VPN/tunnel features specifically, even when the rest of the system looks fine

Hit during the NetBird mesh setup: running a kernel while a different version's modules are still the ones loaded breaks kernel-dependent features like TUN/WireGuard, while most of the system continues working normally — making it a non-obvious cause to suspect. A reboot, not a service restart, is required after any kernel update, especially before troubleshooting VPN connectivity from first principles.

Negative/conditional instructions are weak levers on small local models (side quest, Chat 13)

"Don't do X unless Y" competes poorly against a 7B–22B model's strong training-data prior toward its default behavior. A positive, structural instruction — "output only a numbered list, stop after it" — with an explicit alternative format to produce, rather than just a thing to avoid, holds far more reliably. Applies to any small local-model prompt-engineering work, not just the design-questions use case this was found on.

Isolate one variable per test, especially when the user's own message changes two things at once (side quest, Chat 13)

An early test mixed an intentional test condition (a calm, explicit permission change) with an unplanned follow-up (a very specific direct request), muddying which fix was actually responsible for the observed behavior. Worth scripting test conversations as deliberately as the fix itself — one signal at a time — rather than reusing a natural, multi-purpose conversation and trying to disentangle results after the fact.

A scripted API harness removes retyping and wording drift between manual prompt-engineering test runs (side quest, Chat 13)

Manually retyping or copy-pasting the same test conversation into a chat UI across multiple runs risks small wording differences that can themselves change model behavior, confounding the actual variable under test. A short script sending the exact same fixed message sequence straight to the model's own OpenAI-compatible API (e.g. via llama-swap) removes that variance entirely — cheap to build once a prompt's iteration has stabilized enough to be worth repeat-testing.

A chat UI's own agentic features can confound a clean behavioral test (side quest, Chat 13)

Open WebUI's memory system can call tools (add_memory, search_chats) mid-conversation on models with native function calling enabled — a second mechanism competing with the system prompt for the model's attention. Worth checking for, and disabling per-model rather than globally if possible, before trusting any test of system-prompt behavior in a chat UI with agentic features turned on.

Open Items / Next Steps

Offsite Backup (blocked — depends on desktop + mesh)

Architecture decided: VPS → desktop over NetBird mesh
What needs backing up offsite: CheckPoint SQLite DB, Forgejo data (repos + Forgejo DB + config), Caddy config
Blocker: Desktop must be set up as LLM host first; NetBird mesh must be running; then backup job can be configured
Gap surfaced Chat 11: the mesh is live across desktop, laptop, and phone, but the VPS itself is not yet a mesh member — joining it is a prerequisite not previously called out explicitly, and still needs to happen before this backup job can be built.
⚠️ Until this is done, a full VPS loss means losing everything. The CheckPoint daily backup currently lives on the VPS itself.

Nextcloud Deployment (decided — not yet sequenced)

Decision made: self-host Nextcloud on the VPS as the long-term Google Docs / MS Office replacement (see Architecture Decisions above for the full comparison and reasoning).
Not yet decided: editor engine — Collabora vs. self-hosted OnlyOffice vs. wait for Euro-Office (stable release targeted summer 2026).
Not yet sequenced: where this fits relative to the desktop LLM stack / NetBird mesh / offsite backup work already queued below. Needs a session to place it in the roadmap.

Desktop LLM Stack (deployed — minor items remain)

Decisions made: llama.cpp (not Ollama); Option C benchmark complete — Vulkan confirmed as active backend, ROCm/HIP kept installed as fallback; 5-tier local model roster designed and now fully deployed via llama-swap; Open WebUI running as the browser interface; Cline (not Continue.dev — acquired by Cursor, see Architecture Decisions) as the coding-agent interface, Plan Mode → architect, Act Mode → debug.
✅ Vulkan backend built and verified GPU-accelerated (Chat 8)
✅ HIP/ROCm backend built, verified genuinely GPU-active, benchmarked against Vulkan, backend decision finalized (Chat 9)
✅ video/render group membership confirmed active (Chat 9)
✅ Model selection unblocked and resolved into a designed 5-tier roster (Chat 9) — see Architecture Decisions
✅ llama-swap installed, dedicated /mnt/data storage set up, full roster downloaded and tuned, Open WebUI and Cline both connected and confirmed working end-to-end (Chat 10) — see What Is Confirmed Working and Key Configs
Not yet done:
- TTL/auto-unload tuning for llama-swap (currently untouched defaults)
- Decide coder-fim (B)'s role now that Cline has no FIM-autocomplete equivalent — options include leaving it as a manual-pick model, adding a dedicated autocomplete tool (e.g. Tabby) alongside Cline, or retiring the tier
- Security item surfaced Chat 10, partially addressed Chat 11: llama-swap's own startup log flags it as reachable on all network interfaces, not restricted to loopback. Mitigated at the network layer — UFW now restricts port 8080 to the wt0 (NetBird) interface only, confirmed working. llama-swap itself still listens on all interfaces at the application layer, so -listen localhost:8080 remains a defense-in-depth option, but VPN-only access is no longer unenforced.
- Vault entries for this session's work not yet drafted/pushed — candidates already identified: MoE offload tuning (--n-cpu-moe vs. -ngl), llama-swap aliases vs. groups, the Hugging Face cache symlink gotcha
- Confirm the four Chat 8/9 vault files actually reached Forgejo (still outstanding from Chat 9)

Desktop — Next Major Phase

~~Finish LLM stack setup~~ — done (Chat 10); minor items above remain
~~NetBird mesh setup (desktop + laptop + phone)~~ — done (Chat 11, reconstructed from a continuation document — see Session Log). VPS is not yet a mesh member — still needed before offsite backup can be built.
Offsite backup — blocked on VPS joining the mesh (see above and the Offsite Backup item below)

Phone Access to Desktop Open WebUI (Chat 11 → 12, in progress)

Scope decided (Chat 12): the phone reaches Open WebUI via the desktop, not the laptop — see Open WebUI Topology under Architecture Decisions.
🔴 Immediate blocker: the desktop's own Open WebUI container can't reach the desktop's own llama-swap. Root cause confirmed (UFW blocks the docker0 interface); fix identified but not yet applied — see Key Configs (docker0 Fix). This is now a hard prerequisite for phone access, not a side issue.
Awaiting Patsy's answers (Chat 12), in order:
1. Proceed with the docker0 UFW fix as identified, or has anything changed since it was identified?
2. How is the desktop's Open WebUI web port currently published (0.0.0.0, unbound, something else)? Determines what firewall work is still needed for the phone specifically, once the docker0 fix lands.
3. Keep the laptop's Open WebUI running for the laptop's own local use, or decommission it now that it's not the phone's target?
De-prioritized, not cancelled: rebinding the laptop's Open WebUI (currently 0.0.0.0:3000) to its specific mesh IP. No longer load-bearing for the phone goal, but still worth doing if the laptop instance stays in service (depends on the answer to Q3 above).
SSH remote access to the desktop — deliberately deferred by Patsy. SSH daemon is running, but UFW has no rule for port 22 at all (local keyboard access only), and no key-based auth is set up yet. Would need a new ed25519 keypair on the laptop (per the per-device key policy) plus a UFW rule — likely scoped to the home LAN subnet specifically, not just wt0, so it still works as a fallback if NetBird itself is ever the thing that's broken.
Docker's general iptables/UFW interaction (the DOCKER-USER chain) — flagged as worth understanding properly later, since it will likely recur with future Dockerized services. Not currently blocking anything — the fixes hit so far have had narrower, sufficient solutions.
Vault entry candidates from the NetBird session — offered, none yet decided:
- Kernel/module mismatch after an unrebooted update (general Arch lesson)
- Instant refusal vs. hang/timeout as a firewall diagnostic signal
- Docker bypassing UFW for published ports (the inbound case — distinct from the docker0-outbound case being fixed here)

Laptop / Desktop (parked, pick up in order)

VS Code extensions — what's worth installing for Flask dev + .md notes
.md files — are there better options for the knowledge vault use case
Pandoc styled output — Pan Am CSS downloaded and confirmed valid; next step is confirming --self-contained flag produces a properly styled standalone HTML file; PDF export not yet attempted
File naming convention — YYMMDD_name1_name2_name3 feels wonky; needs maintainable instructions as part of weekly routine
Zed editor review — low priority, revisit in ~6 months

`architect` Model System Prompt (Chat 13, side quest — no VPS work)

Confirm the final 5-rule system prompt (see Key Configs) is actually saved in Open WebUI's persistent model settings for architect, not just used ad-hoc via the API test script
Stress-test Rule 5 (scope discipline) with a larger edit than the one-line change tested so far — e.g. "add a second button" or "convert the header into a nav bar"
Bug B (frustration causing a dropped content constraint) was tested once and not reproduced — treat as low-priority/possibly not a real mechanism unless it resurfaces during real use, rather than as confirmed-fixed
Decide whether the test-driven prompt-debugging approach itself (isolate one variable per test, scripted API harness for repeatable runs) is worth a vault entry — not yet drafted
test3_scripted.py delivered as a file this session, not yet pushed to Forgejo

Session Log

Chat 1 — VPS Rebuild from Scratch

Covered: Full rebuild decision, shell choice, SSH key architecture, VPS hardening, multi-app structure decision, base install, CheckPoint deployment, Forgejo setup. Ended at: Forgejo SSH debugging — authorized_keys file not found anywhere on VPS.

Chat 2 — Forgejo SSH Fix

Covered: Diagnosed missing Docker port mapping as root cause. Learned three-layer firewall model. Opened port 2222 in IONOS, UFW, and compose.yaml. Restarted container detached. Confirmed correct port mapping in docker ps. Ended at: ssh -T git@git.bujour.de -p 2222 test — not yet run.

Chat 4 — Git Sync, Vault Method, Pandoc

Note: No VPS work this session. Focused on laptop tooling, vault workflow, and Pandoc setup. Covered:

Git pull/push rhythm clarified — git pull origin main confirmed working for syncing vault across desktop and laptop
git fetch vs git pull distinction learned and vaulted
Vault method consolidated — vault prompt finalised and saved as 04_Vault_Prompt.md; Claude now asks "should this go into the vault?" at meaningful milestones
git_basics.md vault note updated with fetch/pull distinction and day-to-day sync rhythm
Pandoc confirmed already installed (v3.6) on laptop
Pan Am CSS (styling.css) downloaded with curl -LO (first attempt failed — -L flag missing, saved HTML redirect instead)
--self-contained flag explored for portable HTML output — not yet confirmed working in browser; carry over to next session Ended at: Pandoc --self-contained output — file size not yet checked; browser rendering not yet confirmed.

Chat 5 — VPS Status Review & Backup Architecture

Note: No hands-on work this session. Status review and next-phase planning. Covered:

Confirmed CheckPoint Ehrenamt is deployed and running ✅
Confirmed Forgejo is running at git.bujour.de ✅
Confirmed Forgejo SSH and vault sync already working (open items from previous sessions now closed)
CheckPoint daily backup exists but lives on the VPS itself — identified as not a real backup (single copy, same machine)
Offsite backup architecture decided: VPS → desktop over NetBird mesh
What needs offsite: CheckPoint SQLite DB, Forgejo data + DB + config, Caddy config
Identified dependency chain: desktop LLM setup → NetBird mesh → offsite backup
Desktop will stay on permanently (LLM host, backup destination — dual purpose, good architecture)
⚠️ Until offsite backup is implemented, full VPS loss = total data loss Ended at: Planning complete. Next session starts with desktop LLM stack setup.

Chat 6 — Shell UX, SSH Agent, Bash Config, VPS Structure Prompt

Note: Mix of VPS housekeeping and laptop tooling. No new app deployments. Includes two side quests (shell switch attempt, corrupted .bashrc). Covered:

Continuation prompt redesigned — new self-updating VPS_Setup_Guide.md workflow replaces the old static prompt; Claude now updates the guide at end of session
VPS server structure prompt written (04_VPS_Serverstruktur_Prompt.md) — reusable context block for future deployment chats covering directory layout, running containers, Caddy routing pattern, networking, and hard rules
Shell decision finalised: staying on Bash — fzf + bash-completion gives Fish/Zsh-like ergonomics without any compatibility risk
- (Side quest: attempted chsh to Zsh — failed due to PAM auth issue; usermod -s worked but login shell still showed Bash after re-login, likely terminal emulator override. Abandoned — Bash is the right call anyway)
fzf + bash-completion installed and activated on laptop and VPS — Ctrl+R fuzzy history, Ctrl+T file search, Alt+C directory jump
SSH agent auto-start block added to laptop ~/.bashrc
forgejo_laptop private key permissions fixed (chmod 600) — was 644, causing SSH to refuse the key
Vault file bash_fzf_setup.md written and pushed to Forgejo
- (Side quest: git push rejected — divergent branches; fixed with git pull --rebase origin main)
- (Side quest: corrupted .bashrc on VPS — a paste injected an API key/token string into 5 lines including an esac keyword; fixed manually in nano using Ctrl+_ to jump to line and Ctrl+W to search; ⚠️ the pasted string may be a compromised secret — rotate if known) Ended at: Guide update. CheckPoint deployment is next.

Chat 3 — Laptop Setup, Knowledge Vault, Locale, VS Code

Note: This session covered the laptop (EndeavourOS, heymdall-750xed), not the VPS directly. All items are infrastructure/tooling for the wider project. Covered:

Bluetooth autostart missing — diagnosed with systemctl is-enabled, fixed with systemctl enable bluetooth
Knowledge vault created at ~/Schreibtisch/05_vault/ — flat .md files, hand-maintained index.md; filing system: 01_INBOX, 02_DEV, 03_PRIVAT, 04_Arbeit, 05_vault, 99_ARCHIV
Locale investigation — system was set to de_DE in /etc/locale.conf, but KDE was overriding with en_GB at login; en_GB.UTF-8 not installed; fixed via locale-gen + KDE System Settings → Region & Language; result: en_GB language, de_DE formats
VS Code crashing on close of unsaved files — root cause: code pacman build; fixed by uninstalling and installing visual-studio-code-bin via yay
VS Code workspaces explained and set up — 05_vault + 02_DEV open simultaneously
Vault prompt created — reusable end-of-session prompt to generate separate vault notes per topic
Continuation prompt created for this project (VPS_Setup_Guide.md workflow) Ended at: VS Code extensions — not yet covered; parked for next session.

Chat 7 — Office Suite Research & Nextcloud Decision (side quest)

Note: No hands-on VPS work this session — this started as a tangent from the VPS project (self-hosted productivity alternatives) but produced a real architecture decision that feeds back into the multi-app structure. Marked as a side quest per session convention. Covered:

Compared five privacy/EU-oriented alternatives to Google Docs/MS Office against Patsy's own setup: Proton Docs (already paid, Swiss not EU), self-hosted Nextcloud + Collabora, self-hosted OnlyOffice, LibreOffice (local-only), CryptPad (zero-knowledge, French/EU)
Flagged the Swiss-vs-EU nuance on Proton explicitly, since Patsy's stated hosting rule elsewhere in the project is "EU hosting, own server"
Surfaced the March 2026 Euro-Office fork: a consortium including IONOS (Patsy's own VPS host), Nextcloud, and Proton forked OnlyOffice over concerns its dev team is Russia-based despite Latvian registration; Euro-Office isn't production-ready yet (stable release targeted summer 2026)
Follow-up deep dive on WPS Office (prompted by an article Patsy read): Chinese company (Kingsoft / Beijing Kingsoft Office Software), not end-to-end encrypted by default, documented history of content-scanning/access-blocking on cloud-stored files in China, a 2023 privacy-policy clause about AI training that was walked back after backlash, and a 2020 US national-security executive order that targeted it. Concluded it has no advantage over LibreOffice even in offline-only mode, since LibreOffice gives the same exposure model but is auditable
Decision made: Patsy has decided, long-term, to self-host Nextcloud on the VPS as the Office/Docs replacement — now a confirmed future app for the multi-app structure (see Architecture Decisions)
Editor engine choice (Collabora vs. OnlyOffice vs. Euro-Office) explicitly left open as its own options-first discussion Ended at: Patsy uploaded VPS_Setup_Guide.md and requested this update, with an explicit instruction not to auto-continue from the session log — next session starts wherever Patsy directs it.

Chat 8 — Desktop LLM Stack: llama.cpp + Vulkan Build

Note: Desktop work, not VPS directly — this is the "desktop LLM stack setup" phase flagged as next in Chat 5. No side quests this session; straightforward start-to-finish build. Covered:

Confirmed desktop hardware via hwinfo: AMD Ryzen 7 7700X, AMD Radeon RX 9070 XT (RDNA4, gfx1201) + Raphael iGPU
Decision: llama.cpp instead of Ollama — Patsy wants full control for the most efficient possible coding/agent-work setup (see Architecture Decisions)
Decision: Option C — build both Vulkan and HIP/ROCm backends and benchmark on real hardware, rather than assume ROCm wins by default; researched and flagged that gfx1201 ROCm support only matured in ROCm 7.2 (March 2026), current data shows Vulkan can outperform HIP on this generation, and HIP has a known RDNA4 idle-power bug relevant to an always-on machine (see Architecture Decisions)
Time-estimated all three options (Vulkan-only, HIP/ROCm-only, both+benchmark) before committing, at Patsy's request
Ran a full readiness audit before installing anything: kernel driver (amdgpu, bound correctly to both GPUs), Vulkan stack, build tools, disk space (157G free), kernel version (7.0.12-arch1-1), group membership
Discovered kernel 7.0 (released April 2026) shipped native ROCm packaging support — lower risk for the upcoming HIP build than expected
Found and installed missing packages: cmake, vulkan-headers, ninja
Hit and fixed a second missing-dependency build error: spirv-headers (separate from the shaderc/glslc shader compiler already present)
Added heimdall to video,render groups — confirmed not yet active (needs logout/reboot), confirmed not currently blocking since /dev/kfd and render nodes are already world-accessible
Cloned llama.cpp to ~/apps/llama.cpp (mirrors the VPS's ~/apps/ convention)
Configured with cmake -B build -G Ninja -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release, built with cmake --build build -j$(nproc) — clean build, only harmless warnings
Noted current llama.cpp bundles its own SvelteKit-based browser chat UI in the build, in addition to the API
Verified full binary set in build/bin/ (llama-server, llama-cli, llama-bench, plus a new consolidated llama binary) and confirmed libggml-vulkan.so fully populated (~50MB)
Smoke-tested with gemma-3-1b-it-GGUF via llama-cli -ngl 99 — GPU offload confirmed by generation speed (~150–260 t/s, well above CPU-only range for that model size)
Two vault entries drafted: build_deps_headers_vs_runtime.md (headers-vs-runtime distinction), llama_cpp_vulkan_rx9070xt.md (backend decision + build steps + verification)
Model selection for daily use explicitly parked mid-session at Patsy's request, to be revisited properly later Ended at: Option A (Vulkan) fully built and verified working, per the explicit "we finish Option A no matter what" agreement at session start. Patsy then introduced this VPS_Setup_Guide.md file for future sessions, replacing the standalone continuation-prompt approach used for the desktop LLM thread up to this point.

Chat 9 — Desktop LLM Stack: HIP/ROCm Benchmark, Backend Decision & Local Model Roster Design

Note: Desktop work only, no VPS work this session — direct continuation of the Chat 8 "Desktop LLM Stack" thread (started from the standalone desktop continuation prompt, not yet the VPS_Setup_Guide.md workflow at session start). No side quests — a single continuous thread from backend benchmarking through model-roster design. Covered:

Confirmed heimdall's video/render group membership now active (groups output showed both — relogin/reboot happened between sessions)
Installed ROCm 7.2.4-1 via pacman; rocminfo confirmed gfx1201 cleanly detected with no known "2 ISAs" HIP-rejection bug, and surfaced the Raphael iGPU as its own separate HSA agent (gfx1036)
Researched the current gfx1201/RDNA4 ROCm support landscape before building — found a real, documented risk of silent CPU fallback dressed as GPU success in some ROCm/llama.cpp combinations, which shaped the verification approach below
Built llama.cpp's HIP backend into a separate build-hip/ directory (HIPCXX/HIP_PATH from hipconfig, GGML_HIP=ON, GPU_TARGETS=gfx1201) — kept the working Vulkan build/ untouched
Verified via ldd that the binary linked against real ROCm libraries (libamdhip64, librocblas, etc.) — ruled out "compiled without real HIP support"
Verified genuine runtime GPU execution via CUDA Graph ... reused log lines during inference; learned this is expected naming (ggml's HIP backend is HIPIFY-translated from CUDA source) rather than a misconfiguration
Hit and fixed a model-path pitfall: HuggingFace's local cache nests the actual .gguf file under snapshots/<hash>/, not directly in the named model folder — -m needs the file, not the folder (symptom: failed to read magic)
Ran llama-bench head-to-head, same model, both backends: Vulkan beat HIP/ROCm by ~53–54% on both prompt processing (17,596 vs. 11,512 t/s) and token generation (332.8 vs. 215.6 t/s) — confidence intervals didn't overlap, a real result
Decision finalized: Vulkan is the active inference backend; ROCm/HIP kept installed as a fallback (e.g. future training workloads), not used for inference
Moved into model selection (parked since Chat 8) — worked through a full task taxonomy covering the coding pipeline (architecture discussion, active coding, documentation, debugging/review) plus daily ADHD/task-support use, then designed a 5-tier local model roster around it (full detail under Architecture Decisions)
Decision: sequential on-demand model swapping instead of concurrent loading, to resolve the headroom-vs-quality-vs-split-role tension on the 16GB card
Decision: llama-swap as the orchestration proxy (single OpenAI-compatible endpoint, auto load/unload by model name, TTL + groups)
Decision: Open WebUI as the browser-based daily-use interface, chosen over a CLI client for its conversation history
Decision: code review reuses the debugging tier's weights under a different alias/system prompt rather than a dedicated model
Decision: documentation/vault-entry generation gets no dedicated model — reuses whichever tier is already loaded
Self-caught correction mid-session: an initial model suggestion (Qwen 3.6-27B, ~17GB) exceeded the desktop's own already-confirmed 16GB VRAM budget (16304 MiB per rocminfo, Chat 8) — corrected before any download happened; vaulted as a lesson about checking exact hardware numbers over round-number guideline tiers
Two vault entries drafted: vulkan_vs_hip_benchmark_gfx1201.md (backend build/verification/benchmark), local_llm_multi_model_roster.md (task taxonomy, roster, llama-swap/Open WebUI decisions) — not yet confirmed pushed to Forgejo
Session closed with a bullet-journal entry and a draft continuation prompt; Patsy then introduced this chat's transcript into the VPS_Setup_Guide.md workflow going forward Ended at: About to install llama-swap — identifying the correct GitHub release asset for x86_64 Linux — when the session was closed. Next session picks up there.

Chat 10 — Desktop LLM Stack: llama-swap Install, Full Roster Deployment, Editor Integration

Note: Desktop work only, no VPS work this session — direct continuation of the Chat 9 "about to install llama-swap" cliffhanger. One notable side quest (storage partition discovery/wipe); otherwise a single continuous build-out from install through two fully working end-to-end interfaces. Covered:

Installed llama-swap v240 as a static Go binary (not Docker/Homebrew — keeps using the already-built-and-benchmarked Vulkan llama-server) to ~/apps/llama-swap/
(Side quest) Deciding where to store GGUF model files led to discovering an entire old, untouched Btrfs installation surviving on nvme0n1p2 (1.6T) from before the EndeavourOS reinstall — complete with a real @home/freya user directory. Confirmed externally backed up, then wiped, reformatted to ext4, and mounted permanently at /mnt/data via /etc/fstab. Caught and resolved a duplicate-partition-label collision (GG used for both the new internal partition and a pre-existing external drive) before it became a real risk.
Wrote a minimal one-model llama-swap config, verified a request reaches llama-server and returns a real response (smoke test via gemma-3-1b-it)
Downloaded and wired in the full 5-tier roster: daily (Qwen3-8B Q4_K_M), coder-fim (Qwen2.5-Coder-7B-Instruct Q6_K), architect (gpt-oss-20b Q6_K), debug/review (Qwen3.6-35B-A3B Q3_K_L, shared backend)
Hit and fixed a Hugging Face cache gotcha: mv-ing a cached model "file" actually moves a broken symlink (target is a hash-named blob elsewhere) — fixed by deleting and re-downloading directly via curl -L -o into /mnt/data/models/, and used that method for every subsequent download
Decision: upgraded A2's quant from the originally-planned Q3_K_M to Q3_K_L — bartowski's own quant table flags Q3_K_M as "low quality" (not recommended) while Q3_K_L is flagged "recommended" for ~660MB more, effectively free given the model was never fitting fully in 16GB VRAM at any Q3-tier quant regardless
Found and fixed Qwen3-8B's default "thinking" behavior burning ~130 tokens on trivial prompts, defeating daily's fast-response purpose — fixed with --chat-template-kwargs '{"enable_thinking":false}', applied only to daily (architect/debug are supposed to reason — confirmed that behavior as correct, not a bug, on those tiers)
Empirically tuned A2 (debug/review) VRAM usage via radeontop -b 03:00.0 — discovered the real lever was an unset --ctx-size defaulting to the model's full 262K native context (194K total across 4 auto-provisioned slots), not --n-cpu-moe as initially assumed; settled on --n-cpu-moe 16 --ctx-size 65536 --parallel 1 → 85% VRAM, confirmed stable for the whole session (KV cache allocated once at load, not incrementally)
Diagnosed and fixed a reload gap between debug and review (two config entries with identical cmd: were triggering a full ~20s+ reload on every switch) using llama-swap's aliases: field — confirmed fixed via server logs (single health-check event) and response timing, not just inference from a correct response
Installed Docker on the desktop for the first time — architecture decision made explicitly (Docker vs. pip/uv for Open WebUI), chose Docker
Deployed Open WebUI in Docker, avoided a port-8080 collision with llama-swap by mapping to host port 3000 instead of --network=host; connected via http://host.docker.internal:8080/v1 (required --add-host=host.docker.internal:host-gateway — not automatic on Linux, unlike Docker Desktop)
Finding that changed the plan: Continue.dev was acquired by Cursor in June 2026 — final release shipped, GitHub repo now read-only, no further updates from the original team. Verified via multiple independent sources before acting on it, rather than proceeding to configure a tool on a foundation that had just changed.
Requested and received a fair, evenhanded comparison of alternatives (Cline, Tabby, Aider, Roo Code) plus a specific check on VS Code's own native BYOK/Agent Mode (real and current, but same "no dedicated autocomplete" limitation as Cline, and tied to Microsoft's evolving Copilot product terms)
Decision: Cline — independent, actively maintained, community-owned (Apache 2.0), can't be pulled out from under the project by a vendor's roadmap the way a built-in editor feature could
Installed Cline, configured as an OpenAI-Compatible provider pointed at http://localhost:8080/v1 (plain localhost — Cline runs natively, unlike Open WebUI's containerized path)
Decision: mapped Cline's Plan/Act modes onto the existing roster — Plan Mode → architect, Act Mode → debug — since Cline has no equivalent to Continue's dedicated FIM-autocomplete role
Left open: coder-fim (B) no longer has the role it was originally designed for; still installed and usable as a manual pick for lightweight coding asks, but its long-term role is undecided (see Open Items)
Generated two documents at session end: a consolidated session-summary markdown (storage, llama-swap, full roster, tuning history, final config) and a separate copy-paste continuation prompt scoped specifically to the next phase (NetBird mesh, laptop access) — both delivered as downloadable files, not yet pushed to Forgejo or folded into the vault's one-topic-per-file convention Ended at: Full stack confirmed working end-to-end — Open WebUI and Cline both successfully reaching all 5 models through llama-swap. Session closed with this guide-update request; next session's scope not yet chosen (per this guide's own instruction not to auto-continue).

Chat 11 — NetBird Mesh: Desktop/Laptop/Phone, llama-swap Hardening, Laptop Open WebUI (reconstructed from a continuation document, not a live transcript in this guide)

Note: This session (or sessions — the source document itself opens "picked up mid-way through fixing one specific bug," implying at least one earlier unlogged session started this thread) was tracked via a standalone continuation prompt (STATUS-netbird-mesh-continuation.md) instead of this guide's workflow, per Chat 10's closing note. Patsy uploaded that document in Chat 12 and it's folded in here for the first time. Because it's a summary document rather than a full transcript, internal side-quest structure (if any) isn't preserved — everything below is presented as one continuous thread. Covered:

NetBird Cloud mesh set up and confirmed across desktop, laptop, and phone — 3 peers, direct peer-to-peer connectivity verified (not relayed)
llama-swap converted from a manually-run process to a systemd service (llama-swap.service, Restart=on-failure, enabled for boot)
UFW installed/configured on the desktop: default-deny incoming / default-allow outgoing, port 8080 (llama-swap) restricted to the wt0 interface only — verified from the laptop
Desktop sleep prevention hardened: confirmed KDE/systemd-logind already had no idle action, then additionally hard-masked all sleep/suspend/hibernate targets at the systemd level
Open WebUI newly deployed on the laptop (Docker), connected to the desktop's llama-swap over the mesh (http://100.105.210.21:8080/v1) — all 5 models tested with real inference
Cline on the laptop repointed at the desktop's llama-swap over the mesh — confirmed working
Phone added to the mesh, confirmed as a connected peer — not yet pointed at any service
Bug diagnosed, not yet fixed: the desktop's own Open WebUI container (from Chat 10) can't reach the desktop's own llama-swap. Root cause confirmed via sudo journalctl -k | grep -i block — UFW blocks the request on the docker0 interface, since only wt0 had been given an allow rule. Fix identified (sudo ufw allow in on docker0 to any port 8080 proto tcp) but not yet run when the session closed.
Three vault-entry candidates identified and offered to Patsy, no decision made on which (if any) to draft: kernel/module mismatch after an unrebooted update, hang/timeout-vs-refusal as a firewall diagnostic, Docker bypassing UFW for published ports (inbound case) Ended at: Mid-fix on the docker0 bug — this is where the continuation document itself stops. Also left open: laptop Open WebUI still bound to 0.0.0.0:3000 (rebind proposed, not executed), SSH remote access to the desktop (deliberately deferred), and the DOCKER-USER chain as a topic worth understanding later.

Chat 12 — Scoping: Phone Access via Desktop Open WebUI (in progress — no commands run)

Covered:

Patsy clarified the phone-access goal: the phone should reach Open WebUI through the desktop, explicitly not the laptop.
Decision: this de-prioritizes (without cancelling) two Chat 11 open items — rebinding the laptop's Open WebUI, and testing phone → laptop — since neither is on the critical path for a desktop-only target.
Consequence flagged: the docker0/UFW bug from Chat 11 (desktop's own Open WebUI can't reach desktop's own llama-swap) is now a direct blocker for this goal, not a side issue, since the desktop's Open WebUI has to work locally before it can be exposed to the phone at all.
Three clarifying questions raised (proceed with the identified docker0 fix? how is the desktop's Open WebUI port currently published? keep or decommission the laptop's Open WebUI?) — posed as a numbered list per Patsy's standing convention, first answer not yet given.
Patsy then uploaded VPS_Setup_Guide.md and requested this guide update, with an explicit instruction not to auto-continue the phone-access work and to wait for direction on what to work on next. Ended at: Guide update delivered. No commands executed this session. Next session resumes the numbered question list (starting with Q1) once Patsy chooses to continue it — per the guide's standing instruction, not assumed automatically.

Chat 13 — Side Quest: Test-Driven System-Prompt Debugging for the `architect` Model (Open WebUI)

Note: Entirely a side quest — no VPS commands were run this session, and no VPS/infrastructure state changed. Scope was debugging the conversational behavior of the architect model (gpt-oss-20b, already deployed via llama-swap — see Local LLM Multi-Model Roster) as used through Open WebUI and, by extension, Cline's Plan Mode. Folded into this guide per Patsy's explicit "include side quests but mark them" instruction. Covered:

Diagnosed Cline's inline chat behavior (auto-generating full code even when asked not to) as a small-model instruction-following limitation rather than a Cline bug — negative/conditional instructions ("don't do X unless Y") are weaker levers on 7B–22B models than positive, structural ones ("output only a numbered list, stop after it").
Clarified the actual target of the fix: Open WebUI's own chat with the architect model in Cline's Plan Mode role, not Cline's Edit/Agent mode. Confirmed via conversation_search that Patsy had switched editor tools from Continue.dev to Cline since the relevant earlier session (see Chat 10) before proceeding.
Confound identified and removed: Open WebUI's agentic memory tools (add_memory, search_chats) were firing mid-conversation in an early test transcript, competing with the system prompt for the model's attention. Looked up the current fix via Open WebUI's docs (the memory toggle is a per-model Builtin Tools setting, not only a single global switch) and disabled it for the architect model specifically before further testing.
Decision: set the system prompt persistently in the model's own settings (Workspace → Models → Edit → architect → System Prompt) rather than pasting it into each chat — removes wording drift between test runs.
Test-driven iteration across four rounds, using real transcripts pasted back after each run:
- Bug A (generic templated questions that ignored prior answers): confirmed present in the original unscripted transcript; fixed with an explicit "propose 2–3 concrete options built from the user's own answers" rule — confirmed fixed in Test 1.
- Test 1 side effect: the same transcript also surfaced an unplanned second bug — a flat, unexplained refusal to produce HTML even after Patsy had calmly granted permission earlier in that same conversation.
- Bug B hypothesis (frustration causing the model to drop an unrelated content constraint, e.g. "no code"): a dedicated pacing-only test (Test 2) was designed to isolate it from Bug A. Not reproduced — a pure "can we move faster" message correctly compressed pacing only, leaving the "no code" constraint untouched.
- Rule 3 (the actual refusal bug) rewritten to explicitly separate PROCESS signals (pacing/frustration — never touch content rules) from CONTENT signals (explicit statements like "an HTML page is fine now" — must be honored, with a mandatory one-sentence explanation if declining anything).
- Manual Test 3: Patsy skipped the planned "calm, separate permission statement" step and instead asked directly and very specifically for a full HTML template. No refusal occurred — but every subsequent edit request in that same thread ("adapt the light theme," "fix the contrast," "where did the mint green go") triggered a full-file regeneration from scratch instead of a scoped change. This surfaced a new, more consequential problem than the original two bugs: the same "produce the biggest possible artifact" instinct behind Patsy's original Cline complaint, resurfacing at the deliverable stage rather than the questioning stage.
- Rule 5 added (scope discipline: output only the changed portion on edit requests, match response size to request size, ask one sentence if genuinely unsure before regenerating anything large) — confirmed fixed via Test 4 (a trivial one-line edit: "change the button color to teal" returned only the button markup, not the full file). Not yet stress-tested on a larger edit.
- Scripted Test 3 re-run, using a purpose-built API harness (below) with the complete 5-rule prompt and the originally-planned sequence (calm permission statement as its own turn, then a direct request): confirmed immediate compliance with no refusal — the specific condition Rule 3 was written to fix. One minor scope-creep note (an unrequested "Features" section in a first-time deliverable) flagged as low-priority, since Rule 5 governs edits to existing output, not first-time generation.
Built a reusable Python test harness (test3_scripted.py, delivered as a downloadable file) that sends a fixed sequence of user turns straight to the architect model via llama-swap's OpenAI-compatible API — removes manual retyping and wording variance between test runs; used successfully to re-run and confirm the Rule 3 fix. Status: Final combined 5-rule system prompt exists and has been used successfully via direct API calls against the live architect model — see Key Configs. Not yet confirmed: that this exact text is saved persistently in Open WebUI's model settings (the harness sends it per-request via the API, independent of whatever's currently saved in the UI). Ended at: Rule 5 confirmed on a trivial one-line edit only. Two items intentionally left open rather than assumed fixed, per Patsy's test-driven approach: Bug B's single clean (non-reproducing) run, and Rule 5 untested on a more substantial edit. Session closed with this guide-update request.

91 KiB Raw Permalink Blame History Unescape Escape