noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add video upload, extraction progress, and reusable analysis sources

Browse source
This commit is contained in:
Olive Vaughn 2026-09-28 09:38:49 -04:00
parent 690de21fa4
commit 686f897401
24 changed files with 927 additions and 137 deletions
Download patch file Download diff file Expand all files Collapse all files

25
frontend/README.md
View file

@ -41,8 +41,9 @@ things fail in ways that read like the code being broken and are not.
mise exec -- python manage.py test clips # from the REPO ROOT
```
Thirty-one tests over the API: the blob store, key verification, the load/save
round trip, the conditional write, and the footage manifest. The two groups worth
Tests cover the API: the blob store, key verification, the load/save round trip,
the conditional write, source analysis blocks, and video upload and extraction.
The two groups worth
reading are the ones that make the tier split a property of the system rather than
a convention in ClojureScript — the server recomputes every tier-2 key it is
handed, and refuses a block whose analysis does not declare a detector version.
@ -111,10 +112,16 @@ The demo scene itself is `src/arthur/demo/scene.edn`. Both the synthetic take
and real footage use `src/arthur/flow/take.cljs` for the measurement order and
`src/arthur/flow/freeze.cljs` for the landmark-to-channel conversion.
### Real footage (port steps 6–7, served by the backend since step 9)
### Real footage
Two commands from the repo root, then pick the take in the app and click
**load frames**:
Choose a video in the **footage** file input. The server probes it, extracts one
PNG per source frame and WAV audio, then makes the resulting footage selectable.
Click **load frames** to detect and freeze it. Extraction progress is currently
read from `/api/extractions/<key>`; a future WebSocket can push the same job state.
The uploaded bytes, extraction job, and decoded footage have separate records, so
the same uploaded video can be reopened without decoding it again.
The command-line route is also available for an existing extracted bundle:
```sh
./extract.sh /path/to/clip.mov # decode to frames + audio + manifest
@ -161,6 +168,9 @@ stays 320×200 regardless of the footage dimensions. Real
footage starts at the source picture rate. The **picture fps** buttons sample the
frozen roto at lower rates while the source track, duration and audio clock stay
unchanged. Picking frames to trace into cels is a separate future editing step.
**save** also stores the detection mask, dense landmarks and raw RGBA mouth crops
as three analysis blocks. **open** restores these without running MediaPipe or
loading source PNGs. The frozen shapes remain separate channel blocks.
For known occlusion intervals, an extracted manifest may add
`"feature-absence": {"eye-r": [[10, 14]]}`. Frame numbers are one-based and
@ -184,13 +194,14 @@ runs the old JS tool on 8777, and the two are meant to run side by side.
## Saving
**save** and **open** in the transport. A save is three requests, in an order that
**save** and **open** in the transport. A save has three ordered stages:
is the tier split:
1. the **analysis** record, so every block stored afterwards can name the detector
version that produced it. The server refuses a block whose analysis it does not
know.
2. ask which **blocks** are missing, and upload only those.
2. ask which **blocks** are missing, upload the source analysis blocks and frozen
channel blocks, then link the source blocks to the analysis.
3. the **document** — tier 1, as leaves. The server refuses a clip that names
blocks it does not hold, so a saved document cannot load into a blank stage
somewhere else.