noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Split the shell into topbar, pool, stage, timeline and params panes

Browse source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Olive Vaughn 2026-09-29 12:25:46 -04:00
parent ddabfbeaa8
commit 179770d7d4
24 changed files with 2138 additions and 611 deletions
Download patch file Download diff file Expand all files Collapse all files

94
frontend/README.md
View file

@ -88,18 +88,60 @@ cd frontend && mise exec -- npx shadow-cljs watch app
```
Then open **<http://localhost:8778/>**. Django serves the page from
`clips/templates/clips/index.html`, and staticfiles serves the bundle out of
`static/arthur/js`, where `shadow-cljs` already writes it — so nothing copies files
between the two.
`clips/templates/clips/index.html`, its styles from `static/arthur/app.css`, and
the bundle out of `static/arthur/js`, where `shadow-cljs` already writes it — so
nothing copies files between the two.
### The window
One screen, five panes, no scrolling page. `src/arthur/ui/shell.cljs` is the grid
and nothing else; each pane owns its own subscriptions.
```
top the document: its name and last status, export, new / open / save
left media pool — the open document's symbols, and footage on the server
centre the palette strip (16 slots) above the stage
right inspector — the clip, the selected node, the tracked objects
bottom timeline — transport, ruler, a row per node
```
**It opens on a blank document**, and **new** makes another one. Nothing is
loaded until it is asked for.
**Whole documents live under `open ▾`, not in the media pool**, and the split is
load-bearing rather than tidy. Opening a project REPLACES the stage; everything
in the pool is a thing to put ON it. Listing documents beside the symbols inside
one of them makes them read as two kinds of the same thing. The menu lists the
projects the server holds; the built-in scenes are under their own heading,
italic, and are not projects — they are compiled into the bundle and the server
has never heard of them.
Selection lives in app-db under `:ui`, as `[:node <timeline> <node>]`,
`[:timeline <id>]` or `[:subject|:feature|:group <id>]` — four panes ask what is
selected, and a ratom private to one of them can only be shared by making the
other three require it.
**Drop a video on the media pool** and it uploads, extracts and goes straight on
into detection. Dragging a symbol out of the pool onto the stage places an
instance of it at the playhead.
The timeline's rows are the open clip's nodes, front-most first, with a dot per
keyframe and a bar over the frames the node exists on; a dense channel is hatched
rather than ticked, because one value per frame is a solid block that says less
than the bar does. Opening a row shows its channels; opening a **symbol** row
shows the timeline it instances, with every frame number mapped back into the
stage's own frame space — see the namespace docstring in `ui/timeline.cljs`, which
is where that mapping is argued.
### Paint sketch
Click **new polygon**, place at least three vertices on the stage, then click
**finish shape**. Select a shape to drag its vertices. Scrub to another frame and
click **new drawing key** to copy the visible outline there; the previous drawing
holds until that key. The numbered drawing-key buttons jump to editable keys.
The transition control between two drawing keys can switch that gap between a
hold and linear vertex tweening. Other gaps keep their own timing. Tweening works
Pick a tone from the palette strip, click **polygon**, place at least three
vertices on the stage, then click **finish**. Select a shape — on the stage, or by
its timeline row — to drag its vertices. Scrub to another frame and click
**drawing key here** in the inspector to copy the visible outline there; the
previous drawing holds until that key. The numbered key buttons jump to editable
keys. The transition control between two drawing keys can switch that gap between
a hold and linear vertex tweening. Other gaps keep their own timing. Tweening works
best when the same vertex
keeps the same meaning in every drawing. Paint shapes use the timeline clock
directly, so the roto exposure grid does not delay a drawing key or step its
@ -109,7 +151,7 @@ tween. Use the project **save** button to persist the drawings.
has used since step 5, when shadow-cljs's `:dev-http` did no directory-index
resolution and the suite learned to ask for the file.
Four built-in clips, on buttons in the transport:
Four built-in clips, under **built-in examples** in the open menu:
| | |
| --- | --- |
@ -125,14 +167,14 @@ The demo scene itself is `src/arthur/demo/scene.edn`. Both the synthetic take
and real footage use `src/arthur/flow/take.cljs` for the measurement order and
`src/arthur/flow/freeze.cljs` for the landmark-to-channel conversion.
**stage 8625** loads the locally saved `IMG_8625.MOV` project and places its
**8625 stage study**, in the open menu, loads the locally saved `IMG_8625.MOV` project and places its
post-processed timeline twice. The stage layout is
`src/arthur/demo/stage_8625.edn`: the right picture and sound start at frame 48,
and the two pictures overlap slightly in stage space. Audio has its own timeline
nodes, linked to the picture instances but with independent spans and gain
channels. The right sound swells and pans across the stage, then fades out at
frame 260 while its picture continues to
frame 280. The button needs that saved 8625 project in the local server database.
frame 280. The row needs that saved 8625 project in the local server database.
### Projects and the EDN fixtures
@ -148,16 +190,17 @@ the same ClojureScript clip.
The intended editor creates and changes that in-memory clip directly: a project
browser and **new stage** action, timeline instance placement, node and channel
editors, then the existing save path. EDN remains useful for checked-in examples
and reproducible studies. The current UI has save and open, but no project
browser, blank-stage action, or authoring controls yet; open chooses the most
recent project.
and reproducible studies. The UI now has the blank-stage action (**new**), a
project browser (`open ▾`), and placement by dragging a symbol out of the media
pool; node and channel editors are still to come — the inspector reports a
channel's shape but has nowhere to change its values.
### Real footage
Choose a video in the **footage** file input. The server probes it, re-encodes it
Drop a video on the media pool, or use its **+** button. The server probes it, re-encodes it
to an H.264 proxy and a raw stream of the same coded frames, pulls WAV audio and one tracing JPEG per frame,
then makes the resulting footage selectable. Click **load frames** to detect and
freeze it. Extraction progress is currently read from `/api/extractions/<key>`; a
then makes the resulting footage selectable and runs detection on it. **roto**, in
the pool's header, does the same for footage that is already there. Extraction progress is currently read from `/api/extractions/<key>`; a
future WebSocket can push the same job state. The uploaded bytes, extraction job,
and decoded footage have separate records, so the same uploaded video can be
reopened without decoding it again.
@ -216,14 +259,14 @@ the root — all of it is extraction output, and tier 3 does not belong in the r
Loading detects one face per frame, measures the mouth, eyes and brows from
landmarks and the teeth from source pixels, then freezes them into channels,
and adds a button for the footage clip. Detection happens once when you load;
and opens the footage clip. Detection happens once when you load;
playback only resolves channels and paints. Frames without a detection remain
marked absent even though their neighbouring poses are used to condition the
track. The scene now records stable subject and feature IDs and explicit eye
pairs; dense channels can mark one feature absent while another is observed.
Current MediaPipe loading supplies only the full-face detection mask. The stage
stays 320×200 regardless of the footage dimensions. Real
footage starts at the source picture rate. The **picture fps** buttons sample the
footage starts at the source picture rate. The **picture** buttons in the inspector sample the
frozen roto at lower rates while the source track, duration and audio clock stay
unchanged. Picking frames to trace into cels is a separate future editing step.
**save** also stores the detection mask, dense landmarks and raw RGBA mouth crops
@ -252,7 +295,7 @@ runs the old JS tool on 8777, and the two are meant to run side by side.
## Saving
**save** and **open** in the transport. A save has three ordered stages:
**new**, **open** and **save** in the top bar. A save has three ordered stages:
is the tier split:
1. the **analysis** record, so every block stored afterwards can name the detector
@ -269,10 +312,11 @@ unchanged document says `0 leaves · 0 blocks`, which is both halves of the
addressing working at once — an unchanged leaf keeps its version, and a
content-addressed block is already there.
Two things are deliberately visible as failures. Saving `swarm` is refused,
because its blocks have hand-written names and a document may only name content
addresses. And **open** takes the most recently updated project and shows its first
clip: there is no project browser, and the store holds one clip at a time.
Saving `swarm` is deliberately visible as a failure: its blocks have
hand-written names and a document may only name content addresses.
`open ▾` lists every project the server holds, newest first, and shows the first
clip of whichever one is picked — the store holds one clip at a time.
## The oracle, which is finished