noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Port step 5: freeze measured mouth into playable channels

Browse source
This commit is contained in:
Olive Vaughn 2026-09-27 19:01:39 -04:00
parent 942e2f38ab
commit 8a06835895
24 changed files with 1625 additions and 506 deletions
Download patch file Download diff file Expand all files Collapse all files

108
frontend/README.md
View file

@ -17,85 +17,107 @@ shadow-cljs bug and is not one. `mise install` is what prevents it.
## The tests
```sh
cd frontend && npm test
cd frontend && mise exec -- npm test
```
That is three things in order: regenerate the JS oracle's answers, compile the
`:test` build, run it under node.
Two things: compile the `:test` build, run it under node.
```
node test/parity/oracle.mjs # drives js/ and writes test/parity/oracle.json
shadow-cljs compile test
node out/node-tests.js
```
Run them separately if a compile error is in the way. The oracle JSON is
generated, not committed.
Run them separately if a compile error is in the way.
**Run them through `mise`**, or make sure `mise`'s node is first on PATH. The
oracle imports `js/*.js` directly, and those are ES modules in a directory with
no `package.json`, so node needs the module detection that became default in
20.19. On an older node 20 — an nvm install shadowing the pinned one is the easy
way to get there — every import fails with "Named export not found ... is a
CommonJS module", which reads like the oracle being broken and is not.
**Run them through `mise`**, or make sure `mise`'s node is first on PATH. `java`
must be 21+ and node 20.19+. On an nvm node 20.11 shadowing the pinned one,
things fail in ways that read like the code being broken and are not.
### And the browser one
Step 5's done-criterion is a PICTURE, and no assertion in `cljs.test` can check
one: a take that resolves to the right numbers and draws nothing would pass every
test in `arthur.flow.freeze-test`. A blank canvas under a perfectly correct
transport is the bug class unit tests miss, and it has happened here once.
So there is a second suite that drives a real Chrome over CDP. It needs the dev
server up:
```sh
cd frontend && mise exec -- npx shadow-cljs watch app # in one shell
cd frontend && mise exec -- npm run browser # in another
```
No dependencies. Playwright is not installed and CDP needs none —
`node --experimental-websocket` has a global `WebSocket` and
`--headless=new --remote-debugging-port=N` is the whole of the other side. It
reads the canvas's own pixels rather than a screenshot, because the CSS scales
the stage up by 2 and a screenshot is four pixels per raster pixel; it writes
PNGs into `test/browser/out/` anyway, so "it drew something" can be checked by
eye as well as by count.
## The app
```sh
cd frontend && npx shadow-cljs watch app
cd frontend && mise exec -- npx shadow-cljs watch app
```
Then open **<http://localhost:8778/index.html>** — with the `/index.html`, not
bare `/`. This shadow-cljs does no directory-index resolution, so `/` is a 404
whatever the roots are.
The page is the hand-written scene from step 2, scrubbed by hand. There is
deliberately no clock: the audio clock, the rAF loop, the `::resolver`
subscription and the transport are step 3, and the question this step answers is
whether the data model evaluates correctly, not whether it evaluates at 30fps.
A scrubber answers the first and nothing else, which is what makes a failure
here unambiguous.
Four clips, on buttons in the transport:
The scene itself is `src/arthur/demo/scene.edn`, and it is not a face. It is the
smallest scene that exercises every mechanism the model claims to have —
inherited exposure, a sparse held `[:xform :pos]`, composition through a group,
rotation about an anchor, a stencil chain, a keyed `[:vis]`, a `:span`, and
fractional z among siblings — chosen so that each one is visible when it breaks.
| | |
| --- | --- |
| `take` | the synthetic take, head **as filmed**. Step 5's deliverable: a moving mouth, frozen into dense channels, with no video file anywhere. |
| `locked` | the same freeze, head **locked**. The same blocks — `:head`'s channels are written as framed identity instead of as a dense track, and nothing in tier 2 differs. |
| `demo` | the hand-written scene from step 2. Not a face: the smallest scene that exercises every mechanism the model claims to have, so that each one is visible when it breaks. |
| `swarm` | a hundred and twenty dense nodes. Not useful; it is the load test. |
`take` and `locked` are the pair worth looking at together, because switching
between them is the whole of what "stabilisation is a channel, not a mode" means.
The demo scene itself is `src/arthur/demo/scene.edn`; the take is built in
`src/arthur/demo/take.cljs`, which is also the only place the seven stages are
composed in order.
Port 8778 is deliberately not 8777. `python3 serve.py` from the repo root still
runs the old JS tool on 8777, and the two are meant to run side by side — that is
the whole reason `js/` is still in the tree.
runs the old JS tool on 8777, and the two are meant to run side by side.
From step 9 Django serves the page and `:dev-http` goes away.
## The oracle
## The oracle, which is finished
`js/` is the numeric oracle, not dead weight. `test/parity/` runs both
implementations on the same synthetic track and diffs them: `fit-similarity` and
`procrustes-mean` agree to 1e-9, the raster pixel-for-pixel, and `stabilize`'s
whole output agrees across the three namespaces it was split into.
`js/` was the numeric oracle through step 4: `test/parity/` ran both
implementations on the same synthetic track and diffed `fit-similarity`,
`procrustes-mean`, the raster and `stabilize` to 1e-9.
The oracle drives `stabilize` at aspect 0.5625 as well as at 1. Aspect 1 makes the
anisotropy correction the identity, so a port that dropped it entirely would pass
— which is the one thing a parity suite on normalised landmarks can be blind to.
**It was deleted at step 5, on purpose.** Parity proves the port is FAITHFUL, not
that the answer is RIGHT. The JS is a prototype and several of its conclusions
contradict each other; a parity test pins behaviour while code moves, and keeping
it afterwards would bake the prototype's mistakes into the rewrite and make them
permanent. `docs/port-plan.md` says to delete it in one commit once the CLJS
player renders the synthetic take, and that is what happened.
Both sides get the identical track because `js/synth.js` reads `Math.random` at
call time, so `oracle.mjs` stubs it to a constant and the CLJS side passes
`:rand-fn (constantly 0.5)`. `js/` itself is never modified.
**`test/parity/` and `arthur.parity-test` get deleted in one commit at step 5.**
A parity test pins behaviour while code moves; keeping it afterwards would bake
the prototype's mistakes into the rewrite and make them permanent.
`js/` itself stays. It is not an oracle any more, it is the SOURCE for steps 6
and 7 — the MediaPipe setup, the eye and brow signals, the interior extraction —
and its comments encode bugs that actually happened.
## Layout
```
src/arthur/domain/ pure. No re-frame, no DOM, no flow/.
src/arthur/flow/ the stages. `(f params inputs) -> output`, no state.
src/arthur/synth.cljs the synthetic track. In src/ because the take PLAYS it —
it stands in for flow/detect, and a tool that needs a
video file before it shows you anything is one you
cannot debug.
src/arthur/demo.cljs the hand-written scene, read from demo/scene.edn
src/arthur/demo/take.cljs the seven stages composed in order, and the only place
they are
src/arthur/ui/canvas.cljs the one imperative sink — the only DOM canvas call
test/arthur/synth.cljs the synthetic track — test infrastructure, not src
test/parity/ the JS oracle harness. Deletable at step 5.
test/browser/ drives a real Chrome over CDP. Not run by `npm test`.
public/index.html dev host page. Django replaces it at step 9.
```