noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Serve the document from a Django backend, split into three tiers

Browse source 
Step 9. The tier split was the work; Django was the easy half.

Tier 1 — the authored scene — is the document, and it is addressed as
independently versioned leaves rather than saved whole, so one vertex drag
cannot clobber a collaborator's keying. `domain/leaf` is the document as
path -> value; `domain/wire` puts it on the wire as transit, because JSON
has neither integer map keys nor keywords and a save would quietly turn
`{0 v}` into `{"0" v}`.

Tier 2 — the dense channel blocks — is content-addressed by a hash over
every input, with the detector version inside every key through the
analysis the block descriptor names. `flow/address`'s `block-knobs` is the
invalidation table, and `address-test` does not trust it: it re-freezes the
take once per knob and asserts the biconditional, that a block's bytes
changed if and only if its key changed. That found `brow-pos` not depending
on `contour-avg` — the brow ring is smoothed, the raise is not.

Tier 3 — frames and audio — is served by the hash of its bytes out of the
same store. A manifest now names frames and carries a URL for each, so the
frame layout stopped being a shared secret between a shell script and a
ClojureScript namespace, and the `?v=` cache-buster went with it: a blob's
name is the hash of its contents, so a stale copy is not a thing that can
happen. The synthetic take's `audio.wav` moved to `static/arthur/` — an
asset the project owns, not an extraction that churns.

The server verifies rather than trusting a name it was handed: it
recomputes every key from the descriptor stored beside it, refuses an
analysis that declares no detector version, and refuses a document naming
blocks it does not hold. It hashes the descriptor TEXT, because JS prints
an integral double as `1` and Python as `1.0`, and a scheme where both ends
re-render the numbers disagrees on the first parameter that happens to be
whole.

Two loose ends from step 8 closed on the way. `pack` no longer takes a
`(track, frame)` predicate whose call sites each re-derived a feature from
an index — every track names the feature it follows, which deleted five
hand-maintained mappings. And `:dev-http` is gone: Django serves the page,
shadow-cljs only builds into the staticfiles tree.

227 CLJS tests, 31 Django tests, green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Olive Vaughn 2026-09-28 01:11:41 -04:00
parent b6517f837a
commit 9cd5243983
61 changed files with 4694 additions and 269 deletions
Download patch file Download diff file Expand all files Collapse all files

122
frontend/README.md
View file

@ -6,7 +6,9 @@ what order; this file is only how to run it.
## Once
```sh
mise install # from the REPO ROOT: java 21+, node 20, clojure, python
mise install # from the REPO ROOT
pip install -r requirements.txt # the Django half; one dependency
mise exec -- python manage.py migrate # the document database
cd frontend && npm install
```
@ -33,6 +35,18 @@ Run them separately if a compile error is in the way.
must be 21+ and node 20.19+. On an nvm node 20.11 shadowing the pinned one,
things fail in ways that read like the code being broken and are not.
### And the Django one
```sh
mise exec -- python manage.py test clips # from the REPO ROOT
```
Thirty-one tests over the API: the blob store, key verification, the load/save
round trip, the conditional write, and the footage manifest. The two groups worth
reading are the ones that make the tier split a property of the system rather than
a convention in ClojureScript — the server recomputes every tier-2 key it is
handed, and refuses a block whose analysis does not declare a detector version.
### And the browser one
Step 5's done-criterion is a PICTURE, and no assertion in `cljs.test` can check
@ -40,14 +54,21 @@ one: a take that resolves to the right numbers and draws nothing would pass ever
test in `arthur.flow.freeze-test`. A blank canvas under a perfectly correct
transport is the bug class unit tests miss, and it has happened here once.
So there is a second suite that drives a real Chrome over CDP. It needs the dev
server up:
Step 9's is a picture too, for a different reason: the ways a document survives a
round trip LOOKING correct are the interesting ones. So the suite now also saves
the take, reopens it, and checks the frames are the same pixels.
It drives a real Chrome over CDP, and needs both processes up:
```sh
mise exec -- python manage.py runserver 8778 # from the REPO ROOT
cd frontend && mise exec -- npx shadow-cljs watch app # in one shell
cd frontend && mise exec -- npm run browser # in another
```
`ARTHUR_URL` overrides the page it drives; it defaults to
`http://localhost:8778/index.html`, which since step 9 is Django's.
No dependencies. Playwright is not installed and CDP needs none —
`node --experimental-websocket` has a global `WebSocket` and
`--headless=new --remote-debugging-port=N` is the whole of the other side. It
@ -58,13 +79,21 @@ eye as well as by count.
## The app
Two processes, which do not talk to each other:
```sh
mise exec -- python manage.py runserver 8778 # from the REPO ROOT
cd frontend && mise exec -- npx shadow-cljs watch app
```
Then open **<http://localhost:8778/index.html>** — with the `/index.html`, not
bare `/`. This shadow-cljs does no directory-index resolution, so `/` is a 404
whatever the roots are.
Then open **<http://localhost:8778/>**. Django serves the page from
`clips/templates/clips/index.html`, and staticfiles serves the bundle out of
`static/arthur/js`, where `shadow-cljs` already writes it — so nothing copies files
between the two.
`/index.html` still works, and that is deliberate: it is the URL the browser suite
has used since step 5, when shadow-cljs's `:dev-http` did no directory-index
resolution and the suite learned to ask for the file.
Four built-in clips, on buttons in the transport:
@ -82,29 +111,44 @@ The demo scene itself is `src/arthur/demo/scene.edn`. Both the synthetic take
and real footage use `src/arthur/flow/take.cljs` for the measurement order and
`src/arthur/flow/freeze.cljs` for the landmark-to-channel conversion.
### Real footage (port steps 6–7)
### Real footage (port steps 6–7, served by the backend since step 9)
From the repo root, extract a clip, then click **load frames** in the CLJS app:
Two commands from the repo root, then pick the take in the app and click
**load frames**:
```sh
./extract.sh /path/to/clip.mov
./extract.sh /path/to/clip.mov # decode to frames + audio + manifest
mise exec -- python manage.py ingest_bundle
```
This keeps every source frame and writes `frames/0001.png` onward, `audio.wav`,
and `manifest.json` at the repo root. The manifest supplies the exact frame
count, source fps and audio path. Variable frame rate sources are rejected until
the manifest and clock carry per-frame timestamps.
`extract.sh` keeps every source frame and writes `frames/0001.png` onward,
`audio.wav` and `manifest.json`. Variable frame rate sources are rejected until the
manifest and clock carry per-frame timestamps.
To keep multiple takes or compare with a previous extraction, pass a bundle
directory and enter its manifest path in the app:
`ingest_bundle` then hashes all of it into the content-addressed blob store under
`var/blobs` — by hard link, so 112MB of PNGs is not copied — and registers one
`Footage` row. From then on the frames are the backend's: `GET /api/footage/<id>`
answers with a manifest carrying **a URL per frame**, and the app fetches those.
That replaced a shared secret. Until step 9 the page fetched `/manifest.json` off
the filesystem and built `frames/0001.png` itself, with shadow-cljs serving the
repo root — so the frame layout was agreed between a shell script and a
ClojureScript namespace, and "where are the frames" was answered by a directory
listing. The cache-busting `?v=` that used to hang off every frame URL went with
it: a blob's name is the hash of its bytes, so re-extracting gives a frame a
different URL rather than overwriting one.
To keep several takes, pass a bundle directory; each ingests separately and both
stay selectable in the app.
```sh
./extract.sh /path/to/clip.mov scratch/my-take
# source manifest field: /scratch/my-take/manifest.json
mise exec -- python manage.py ingest_bundle scratch/my-take
```
`scratch/` is ignored by Git. The directory contains its own frames, audio and
manifest, so extracting it does not replace another take's files.
`scratch/` is ignored by Git, as are `frames/`, `audio.wav` and `manifest.json` at
the root — all of it is extraction output, and tier 3 does not belong in the repo.
Loading detects one face per frame, measures the mouth, eyes and brows from
landmarks and the teeth from source pixels, then freezes them into channels,
and adds a button for the footage clip. Detection happens once when you load;
@ -124,13 +168,42 @@ inclusive, matching PNG filenames. The eye remains the same feature when it
reappears; the other eye and the mouth continue through the gap. This is an
input annotation, with no UI for editing it yet.
MediaPipe's JS, wasm and model are under `public/mediapipe/` and served locally.
No CDN is used by this app. See that directory's README for provenance.
MediaPipe's JS, wasm and model are under `public/mediapipe/`, served by Django's
staticfiles under `/static/mediapipe/`. No CDN is used by this app. See that
directory's README for provenance.
The server reports what it serves at `GET /api/detector`: the package version plus
the **sha256 of the model asset**, and that string goes inside the content address
of every block a detection produces. Asked rather than assumed, because a version
constant in the client is one somebody has to remember to bump — and
`docs/architecture.md` is explicit that a model upgrade silently reusing old
landmarks presents as "the tool got worse", with no event to attach it to.
Port 8778 is deliberately not 8777. `python3 serve.py` from the repo root still
runs the old JS tool on 8777, and the two are meant to run side by side.
From step 9 Django serves the page and `:dev-http` goes away.
## Saving
**save** and **open** in the transport. A save is three requests, in an order that
is the tier split:
1. the **analysis** record, so every block stored afterwards can name the detector
version that produced it. The server refuses a block whose analysis it does not
know.
2. ask which **blocks** are missing, and upload only those.
3. the **document** — tier 1, as leaves. The server refuses a clip that names
blocks it does not hold, so a saved document cannot load into a blank stage
somewhere else.
The status line says what happened: `saved r3 · 64 leaves · 8 blocks`. Saving an
unchanged document says `0 leaves · 0 blocks`, which is both halves of the
addressing working at once — an unchanged leaf keeps its version, and a
content-addressed block is already there.
Two things are deliberately visible as failures. Saving `swarm` is refused,
because its blocks have hand-written names and a document may only name content
addresses. And **open** takes the most recently updated project and shows its first
clip: there is no project browser, and the store holds one clip at a time.
## The oracle, which is finished
@ -152,6 +225,7 @@ and pixel extraction. Its comments encode bugs that actually happened.
```
src/arthur/domain/ pure. No re-frame, no DOM, no flow/.
src/arthur/fx/ the only namespaces that talk to the network
src/arthur/flow/ the stages. `(f params inputs) -> output`, no state.
src/arthur/synth.cljs the synthetic track. In src/ because the take PLAYS it —
it stands in for flow/detect, and a tool that needs a
@ -160,10 +234,14 @@ src/arthur/synth.cljs the synthetic track. In src/ because the take PLAYS it
src/arthur/demo.cljs the hand-written scene, read from demo/scene.edn
src/arthur/demo/take.cljs the synthetic source for the shared flow/take path
src/arthur/ui/canvas.cljs indexed raster blit to the display canvas
test/arthur/support/ machinery shared between suites; not tests itself
test/browser/ drives a real Chrome over CDP. Not run by `npm test`.
public/index.html dev host page. Django replaces it at step 9.
public/mediapipe/ vendored wasm and model, served under /static/mediapipe/
```
`public/` holds nothing but those assets now. The host page that used to sit beside
them is `clips/templates/clips/index.html`.
## Two evaluators, on purpose
`domain/scene` has both `eval-frame` and `resolver`, and they are not