212 lines
10 KiB
Markdown
212 lines
10 KiB
Markdown
# vault-renamer — shared core
|
|
|
|
This is step one: the shared logic both front ends (GUI and batch/spreadsheet)
|
|
will sit on top of. Nothing here touches your real files yet except
|
|
`preview_scan.py`, which only *reads*.
|
|
|
|
## Layout
|
|
|
|
```
|
|
vault-renamer/
|
|
├── core/
|
|
│ ├── config.py # junk patterns to ignore, "already conventioned" regex
|
|
│ ├── sanitiser.py # filename-building logic, ported from Dokumenten_Beschrifter.html
|
|
│ ├── model.py # FileRecord — one row per file, shared by both front ends
|
|
│ ├── scanner.py # walks a folder, builds FileRecords, discovers destination folders live
|
|
│ ├── suggest.py # simple keyword-overlap folder suggestion (no ML, no network)
|
|
│ └── executor.py # applies a reviewed batch to disk — dry-run by default, always logged
|
|
└── preview_scan.py # read-only sanity check, run this first
|
|
```
|
|
|
|
## LLM-assisted suggestions (new) — optional
|
|
|
|
Uses your local llama-swap instance to suggest topic/subtopic/destination
|
|
by actually reading each file, not just its filename. Runs entirely
|
|
against `http://localhost:8080` — nothing leaves your machine.
|
|
|
|
**Setup, only needed for `.pdf`/`.docx`/`.xlsx` content reading** (`.md`/`.txt`
|
|
work with no setup):
|
|
```bash
|
|
pip install pypdf python-docx openpyxl --break-system-packages
|
|
```
|
|
Missing one of these doesn't break anything — that specific format just
|
|
falls back to filename-only until it's installed.
|
|
|
|
**In the batch GUI:** select rows, click "Suggest via LLM (selected)".
|
|
Runs in a background thread — the window stays responsive while it waits
|
|
on the model. Topic/subtopic/destination fields fill in as each result
|
|
comes back; a file the model couldn't help with is left completely
|
|
unchanged, never blanked. A destination the model invents that isn't one
|
|
of your real folders is discarded rather than applied.
|
|
|
|
**Defaults to the `daily` model** (fast, good for this) **against
|
|
`localhost:8080`** (llama-swap). Override either without touching code:
|
|
```bash
|
|
VAULT_RENAMER_LLM_MODEL=architect VAULT_RENAMER_LLM_ENDPOINT=http://localhost:8080/v1/chat/completions python3 gui/batch_app.py /path
|
|
```
|
|
|
|
## Batch GUI (new) — a table, not a spreadsheet round-trip
|
|
|
|
```bash
|
|
python3 gui/batch_app.py /path/to/folder
|
|
```
|
|
|
|
This exists specifically because the CSV workflow's round-trip through
|
|
LibreOffice caused real problems (BOM characters, auto-reformatted dates,
|
|
autocorrected header casing). Editing happens directly on the same
|
|
in-memory data `apply_batch()` acts on — nothing is ever serialized to a
|
|
file and re-parsed in between, so none of that class of bug can happen here.
|
|
|
|
How it behaves:
|
|
- One row per file, all visible at once — not one-at-a-time like `gui/app.py`.
|
|
- **Double-click** a Date/Topic/Subtopic/Extra/More/Version/Destination
|
|
cell to edit it in place. **Single-click** the "Keep name" cell to toggle it.
|
|
- **Click a column header to sort by it** — sort by File to group
|
|
everything from one folder together before working through it, the
|
|
same trick that helped in the spreadsheet.
|
|
- Select one or many rows (ctrl/shift-click, like any file manager) and
|
|
use the toolbar to mark them **Confirmed** / **Skip** / **Pending** —
|
|
no risk of a typo'd status value, unlike a free-text CSV column. Row
|
|
colour reflects status at a glance.
|
|
- **Suggest folder (selected)** applies each selected row's own keyword
|
|
suggestion individually. **Set folder (selected)…** applies one chosen
|
|
folder to every selected row at once — useful after sorting a whole
|
|
folder's worth of files together.
|
|
- **Preview (dry run)** and **Apply for real** act on every row currently
|
|
marked Confirmed. A collision (target already exists) resets that row
|
|
back to Pending rather than leaving it in limbo, so it stays visible
|
|
and you can fix and retry it.
|
|
- Shares `.vault_renamer_state.json` / `.vault_renamer_log.csv` with both
|
|
other front ends — confirmed by actually running this after the GUI
|
|
and the CSV batch tool on the same folder and checking all three agree.
|
|
|
|
**A real limitation worth knowing:** if two files are genuinely
|
|
byte-identical (same size, same content — true duplicates, or two empty
|
|
placeholder files), they get the same internal identity, because there's
|
|
nothing in their content to tell them apart. All three front ends now
|
|
warn you when this is detected. If you process one and not the other,
|
|
progress-tracking may not distinguish them correctly. The actual fix is
|
|
deduplicating true duplicates as a separate pass, not a cleverer identity
|
|
scheme — no content-based approach can distinguish files that are
|
|
genuinely, byte-for-byte the same.
|
|
|
|
## Batch / spreadsheet stager
|
|
|
|
Two scripts instead of one app — generate a CSV, edit it in Calc, apply it.
|
|
|
|
```bash
|
|
python3 batch/stage.py /path/to/folder # writes staging.csv
|
|
# ... edit staging.csv in LibreOffice Calc / Excel / whatever ...
|
|
python3 batch/apply.py /path/to/folder staging.csv # dry run — just previews
|
|
python3 batch/apply.py /path/to/folder staging.csv --apply # actually does it
|
|
```
|
|
|
|
How it behaves:
|
|
- `stage.py` writes one row per file not yet marked `done`. Files already
|
|
`done` (by either front end) are left out entirely; files marked `skip`
|
|
are left out too unless you pass `--include-skipped`. It also writes a
|
|
`..._folders.txt` alongside the CSV — the full list of real destination
|
|
folders, to copy/paste into `dest_folder` cells.
|
|
- Every row starts with `status = pending`. Editing name/destination
|
|
fields does nothing by itself — a row is only acted on once you change
|
|
its `status` to `confirmed`. Set it to `skip` to mark a file as
|
|
deliberately left alone without touching it.
|
|
- `apply.py` defaults to a **dry run** — it prints and logs exactly what
|
|
it would do without touching a single file. Only `--apply` does it for real.
|
|
- Rows you leave as `pending` are completely ignored — nothing is lost,
|
|
they'll show up in the next `stage.py` run (or in the GUI) unchanged.
|
|
- **Shares progress with the GUI** — same `.vault_renamer_state.json`,
|
|
same `.vault_renamer_log.csv`, same identity-based matching. A file
|
|
processed in one front end shows up as `done` in the other. Confirmed
|
|
by actually running one, then the other, on the same folder — not
|
|
assumed.
|
|
- `identity` and `current_path` columns are there so `apply.py` can find
|
|
the right file again — don't edit them. If a row's file can't be found
|
|
anymore (e.g. moved by the GUI in the meantime), `apply.py` reports it
|
|
and skips that row rather than guessing.
|
|
|
|
## GUI (new)
|
|
|
|
```bash
|
|
python3 gui/app.py /path/to/some/folder
|
|
```
|
|
|
|
On Arch/EndeavourOS, Tkinter needs the system `tk` package if it's not
|
|
already installed:
|
|
```bash
|
|
sudo pacman -S tk
|
|
```
|
|
|
|
How it behaves:
|
|
- One file at a time. The left list shows everything still pending;
|
|
click one to load it into the form on the right.
|
|
- Every field defaults to "no change" — an untouched file stays exactly
|
|
as it is. Files that already match the naming convention default to
|
|
**"keep current filename (only move)"** ticked, so opening one just to
|
|
relocate it doesn't blank out a name that was already fine. Typing into
|
|
any name field automatically unticks that box.
|
|
- The preview always shows the exact new name and destination *before*
|
|
you click Confirm.
|
|
- **Confirm && Move applies immediately** — this isn't a queue you review
|
|
later, it's the same as renaming the file yourself, just faster. That's
|
|
deliberate: you've already seen the exact result in the preview.
|
|
- Progress and every field you've typed are saved automatically to
|
|
`.vault_renamer_state.json` in the scanned folder. Close the app
|
|
mid-batch, reopen later (even after a restart), and it picks up exactly
|
|
where you left off — done files stay done even though they've since
|
|
moved to a different path, because progress is tracked by a content
|
|
fingerprint (`core/identity.py`), not by location.
|
|
- Every move — real or attempted — is appended to
|
|
`.vault_renamer_log.csv` in the scanned folder, so there's a full audit
|
|
trail even if a session gets interrupted.
|
|
- "Skip" marks a file as deliberately left alone (distinct from "haven't
|
|
gotten to it yet"). Tick "show skipped too" to bring them back into view.
|
|
- If a computed target filename already exists, the file is left
|
|
untouched and you get a warning — never silently overwritten.
|
|
|
|
Both `.vault_renamer_state.json` and `.vault_renamer_log.csv` are
|
|
automatically excluded from the scan itself, so the tool never tries to
|
|
rename its own bookkeeping files.
|
|
|
|
## Try it first — read-only, changes nothing
|
|
|
|
```bash
|
|
cd vault-renamer
|
|
python3 preview_scan.py /path/to/some/folder
|
|
```
|
|
|
|
Start with something small — `03_PRIVAT` rather than the whole vault — and check:
|
|
- Does the file count look right (junk correctly excluded — `.stfolder`,
|
|
`.sync-conflict-*`, `Thumbs.db`, LibreOffice lock files)?
|
|
- Does the filesystem date look like a sane *starting point* (not
|
|
necessarily correct — you're always meant to review/override it)?
|
|
- Does the "already conventioned" count roughly match what you'd expect?
|
|
|
|
This is exactly the kind of small, cheap test worth running before either
|
|
front end gets built on top — if the date-guessing or junk-filtering is off
|
|
in some way I haven't anticipated, better to find that now on read-only code
|
|
than after a GUI is built on it.
|
|
|
|
## What's deliberately NOT here yet
|
|
|
|
- No folder taxonomy hardcoded anywhere — `discover_folders()` reads your
|
|
actual directory tree fresh every run, so it stays correct as your
|
|
folders change.
|
|
- No GUI, no CSV/spreadsheet staging generator, no actual renaming happening
|
|
anywhere — those are the next two pieces, once this core checks out.
|
|
- The suggestion engine (`suggest.py`) is intentionally simple: it only
|
|
matches shared words between what you type and folder names. It won't
|
|
read file *content* to guess a folder — that's a reasonable next upgrade
|
|
for `.md`/`.txt` files if the simple version feels too weak once you've
|
|
tried it for real, but not worth building before we know it's needed.
|
|
|
|
## Safety notes baked in
|
|
|
|
- `executor.apply_batch()` defaults to `dry_run=True`. A real run requires
|
|
explicitly passing `dry_run=False`.
|
|
- Every run — dry or real — writes the same CSV log, so a dry-run's log
|
|
is the exact preview of what a real run would do.
|
|
- A target path that already exists is skipped with a clear reason, never
|
|
silently overwritten.
|
|
- Only rows explicitly marked `status = "confirmed"` are touched at all;
|
|
everything else is left alone.
|