noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Port steps 2-3: the data model and the player

Browse source 
Steps 2 and 3 land together because the model revisions in the middle changed
code from both, and splitting them now would invent intermediate states that
never built.

  domain/channel  value-at across framed/keyed/dense, plus a cursor
  domain/node     decomposed transform, composition order, time maps
  domain/scene    topological order, z paths, eval-frame and resolver
  clock           audio-clocked frame derivation, outside app-db
  db/events/subs  re-frame arrives; the playhead is document state
  ui/player       the rAF loop; reads, blits, dispatches (almost) nothing
  ui/shell        transport

133 tests, 1158 assertions. The scene plays at 30fps against audio, scrubs, and
runs at 1/4x through 4x; verified by driving a real browser over CDP rather than
by assertion.

Two evaluators, on purpose. `eval-frame` is the specification -- allocating,
order-free, obviously correct. `resolver` is what playback uses: cached topo
order and z paths, a cursor per channel, a preallocated point buffer per node.
Both run the same walk, parameterised only by how a channel is read and where
points are written, because two independent implementations of frame evaluation
would drift and the drift would read as a rendering bug rather than as two
functions disagreeing. scene-test asserts they agree frame for frame in forward,
backward and random order.

Deviations and decisions, each with a reason:

- raster/fill-poly! is now a thin wrapper over fill-poly-buf!, which takes a flat
  preallocated buffer. ONE scanline fill serves the analysis stages, which speak
  {:x :y}, and frame evaluation, which hands over a buffer it owns. The parity
  suite still passes pixel-for-pixel, which is what makes the rewrite safe.

- The state mask carries ABSENCE ONLY. An earlier draft gave it a hidden bit too,
  per architecture.md's "hidden flag + palette index", and that bit was a dense
  [:vis] wearing a different hat -- two mechanisms for one question, which is how
  a part ends up hidden by one and shown by the other.

- The palette is a parameter of evaluation, not a global. A node names a TONE;
  which ramp that tone is read in belongs to the timeline it sits in.

- :over layers and a symbol :rate THROW rather than being ignored. Neither is
  built and nothing can produce one, so this can only fire on data that has run
  ahead of the code. A silently dropped override is a hand correction the user
  made once, watched fail, and has no reason to trust again.

Three findings the model produced rather than received:

- Presence propagates asymmetrically. An absent transform drops the subtree; an
  absent [:geom :pts] drops only that node, because an absent mouth outline has
  nothing to draw but the head it hangs off has not moved. That asymmetry is the
  reason presence is tracked per channel and not per node.

- Z paths need lexicographic compare, not `compare`, which orders vectors by
  count first -- so a cel three levels under "a1" would jump in front of a bare
  "a2" and the layer order would mostly work.

- A node stencilled by something that drew nothing is dropped, not drawn
  unclipped: an iris floating over the cheek is worse than a missing iris.

docs/ revised alongside, and those revisions are the load-bearing part:

- A scene, a timeline and a symbol are one type. The doc had two structures with
  the same fields and never said so. Two axes of nesting are now separated --
  parent/child within a timeline is flat with parent pointers, instance nesting
  is by reference -- which is why "nestable" and "flat" only sounded
  contradictory.

- Palettes are named, live on the project, and are ENABLED on a timeline as a
  channel. Absent inherits; present travels with the timeline, so a symbol
  authored against :night stays night wherever it is placed. The output index
  space is the concatenation of the named ramps, which keeps one buffer and one
  flat table and incidentally stops two nodes in different palettes colliding on
  a stencil.

- Stabilisation is a channel, not a mode: {s, theta, tx, ty} IS [:xform :*], so
  the normalise on/off/per-plate toggle is which of the three channel shapes the
  :head node carries. Always measure and always store factored -- smoothing and
  velocity-minimum key selection both need the split to exist in storage.

- There is no camera node and none is needed. Placement is a node transform, the
  stage clips what hangs off it, and project dimensions are independent of the
  footage. `makeXform` is therefore not to be ported: it bakes a cropping
  decision into every stored vertex.

- Export is removed. The .take writer was for an Animator Pro render script; the
  target is encoding video in the browser, and step 9 now says not to port the
  old one.

demo/swarm is 120 shapes on six orbits, entirely dense blocks behind store
handles -- the shape freeze produces at step 5, and the first thing to exercise
that path under load. It plays at 30fps, and bench-test keeps a deliberately
loose floor under it because a performance regression here does not announce
itself: the picture stays correct and merely arrives late.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDfHGdV39zu6rvgbBTfDaT
This commit is contained in:
Olive Vaughn 2026-09-27 17:28:05 -04:00
parent eb06be005c
commit 18d6495592
27 changed files with 3395 additions and 90 deletions
Download patch file Download diff file Expand all files Collapse all files

346
docs/animation-model.md
View file

@ -14,7 +14,7 @@ primitive, scalar — each with its own source, vocabulary and interpolation. Th
table is a good description of **where data comes from** and a bad description of
**what data is**, and the current code follows it too literally: eyes, brows,
teeth and mouth each get their own build function, their own key shape and their
own path through `take.js`.
own path through the prototype's writer.
They are all one thing. A part is a **node** with **channels**, and the five
kinds collapse into differences of which channels exist and who filled them in.
@ -164,6 +164,43 @@ This is what `docs/design.md` means by an override layer, and it is why
re-freezing is safe: the base is regenerated, the layers are untouched. It is
Blender's NLA blending and AE's effect stack at one property.
Layers are what "set it by hand" means for anything measured, and the measured
channel does not need to know. A hand-set gaze is an `:over` on
`[:xform :pos]` of the iris; a hand-set mouth shape is an `:over` on
`[:geom :pts]`. Turning the gaze-step or gaze-dwell knob regenerates the base and
leaves the correction alone, which is the entire reason a correction is stored as
a layer rather than written into the track.
**A `:replace` layer overrides absence, an `:offset` layer does not.** Sampling a
channel is: read the base, then apply the layers — and the base coming back
`absent` does not short-circuit that. `:replace` is explicitly for the frame
where detection failed, so it has to be able to supply a value where there is
none; `:offset` is a delta, and there is nothing to nudge, so an offset over an
absent base stays absent. Implemented the obvious way — bail out on absence
before reaching the layers — the one case the feature exists for is the one case
it would not cover.
### One signal, two nodes
Gaze is deliberately **one measurement shared by both eyes**: at this size the
per-eye difference is noise, and independent noise reads as wall-eyed
immediately, which is the most expensive artefact on a face. But it is stored as
`[:xform :pos]` on `:iris-r` and on `:iris-l`, which are two channels on two
nodes with two different parents — so the invariant lives in `measure` and
nothing in the document enforces it.
That matters as soon as either one can be overridden by hand, because an `:over`
on one iris alone reproduces exactly the artefact the shared measurement exists
to prevent. Until drivers exist, **the override is on both or on neither**, and
that is a rule the UI has to keep rather than one the data can.
This is the case that will eventually justify **drivers** — one value, evaluated
once, feeding several channels — which is why gaze is named in Deferred as the
obvious first one. Nothing here forecloses it: a driver needs a place in the
document and a `:driven-by` on a channel, both of which are additive, and an
absent key means "not driven". So it stays deferred, and the shape does not have
to change to allow it.
## Transform: decomposed, never a matrix
```clojure
@ -194,6 +231,101 @@ of thing that makes a parenting feature feel broken.
`[:xform :pos]` with no conversion. The analysis output and the animation model
meet without an adapter, which is a sign the decomposition is the right one.
## What space geometry is in
**`[:geom :pts]` is always in the node's own local space, and the transform
chain says what that means.** There is no global geometry space and no decision
to make about one.
| Node | Its local space | Why that one |
| --- | --- | --- |
| a rotoscoped feature | head-local, isotropic, unit = one image height | what the anchor fit already produces; the `xform` to raster is not applied and not stored |
| a painted cel | the stage, in pixels, grid-snapped | the artist is placing pixels, so the pixel grid is the thing being authored |
| a primitive under a feature | its parent's | the iris is positioned on the lid ring, not on the stage |
This looks like a small clarification and it removes a whole class of argument.
The prototype bakes the framing into the numbers: `toRasterRing` applies
`makeXform`, which centres on the face oval's bounding box and zooms until the
face is 80% of the raster height, so **every stored vertex carries a cropping
decision** that was made once, at analysis time, from one frame's landmarks.
Dropping that step is a deletion, not a feature, and after it the framing is
simply a transform on a node.
Grid snapping belongs to the cel and not to the roto, for the same reason: a cel
is authored on the grid and a traced contour is not. So it is a property of a
node's space rather than a rule about all geometry, and the tension between
"integer polygons" and "arbitrary placement" was never real.
Each dense block therefore carries its own **fixed-point scale** in its header,
because a block in image-height units and a block in stage pixels need different
ones to fill an `Int16` usefully.
### There is no camera node
A camera is a global transform over everything, and nothing here wants one.
Placing the face on the stage is a transform on a node, which already exists;
what is not on the stage hangs off the edges and the canvas clips it. Every fill
in `domain/raster` already clamps rather than assuming it is inside, so drawing
past the edge is not a feature to add.
Project dimensions are therefore **independent of the footage**. A 1440x1920
portrait clip composited onto a 320x200 stage is not a problem to solve — the
head is placed and scaled where it belongs and the rest of the frame is simply
not on stage. The full frame stays *available* for tracing without being
*visible*, and those are different requirements.
## The anchor: stabilisation is a channel, not a mode
`stabilize` produces `{s, θ, tx, ty}` per frame, which is exactly
`[:xform :scale]`, `[:xform :rot]` and `[:xform :pos]`. So removing the head's
motion is not a pipeline setting — it is a question of **which node holds that
motion**, and the answer is one channel definition:
```clojure
;; locked: the head sits still, for tracing and for judging articulation
[:xform :pos] {:animated? false :value [0.0 0.0]}
;; as filmed: the head moves around the stage
[:xform :pos] {:animated? true :interp :hold
:dense {:store "sha256:…" :stride 2 :frames 600}
:generated {:by :anchor/similarity}}
;; per plate: the head snaps at each selected frame and holds
[:xform :pos] {:animated? true :interp :hold :keys {0 […], 12 […], 23 […]}}
```
The three modes are the three channel shapes, on one channel, on one node. The
third is the one a plate strip wants — the head pose is stable for exactly as
long as a drawing is on screen — and it costs nothing because `:keys` already
exists. Its frame set is the kept-frame set, which is `suggestPlateFrames` in the
prototype and belongs to painting rather than to measurement.
**Always measure, always store factored, toggle the parent.** The fit is computed
and the geometry is stored head-local in every mode, and only the parent's
channel changes. Two things downstream require it, and both would be lost by
making this an analysis-time switch:
- *Smoothing.* "Smooth the transform, never the contour" only means anything
while the two are separate.
- *Key selection.* A velocity minimum is "articulation paused" in head-local
space and "the head happened to be still" in image space.
It also makes the toggle an edit to the document rather than a reason to
re-analyse: tier 1, undoable, syncable, and instant.
### Two nodes, because two different things want that transform
```
:face group — AUTHORED. where the face sits on the stage, and how big.
:head group — MEASURED. the head's motion, or identity.
:mouth :mouth-in :teeth :lid-r :lid-l :brow-r :brow-l …
```
Switching modes rewrites `:head` and never touches `:face`, so it cannot move
something that was placed by hand. A group node is free, and keeping the authored
and the measured transform apart is the whole reason the transform is decomposed
in the first place.
## Time maps — exposure, lead and symbol timing are one thing
Every node may map the frame it is evaluated at:
@ -221,27 +353,79 @@ different rules:
*not* to the plate, which is the whole point of it — so the offset genuinely
belongs at the node, not the clip.
## Symbols
## Timelines, and why a scene is one
The Flash idea, kept:
A **timeline** is an ordered bag of nodes in its own frame space:
```clojure
:library
{:sym/head {:kind :poly :channels {...}}
:sym/blink {:kind :timeline :frames 3 :nodes [...]}}
{:frames 91
:palette {...} ; see Palettes
:nodes {id -> node}}
```
A node with `:symbol :sym/blink` is an **instance**. Its own channels compose
*over* the symbol's, so one definition can be placed many times and tinted,
offset or retimed at each placement. A `:timeline` symbol has its own frame space,
which the instance's `:time` maps into — that is Flash's MovieClip, Lottie's
precomp and AE's pre-comp, and it is how a three-frame blink gets reused at
frames 40, 88 and 200 without copying it.
That is the whole type, and **everything that holds nodes is one of these**:
- a clip's **scene** is its root timeline,
- a **symbol** in the library is a timeline,
- a node with `:kind :symbol` is an **instance** of one.
An earlier draft of this document had a scene and a `:kind :timeline` symbol as
two structures with the same fields and never said they were the same thing.
They are. Flash's `_root` is a MovieClip; After Effects' "a pre-comp is just a
layer" is already in the prior-art table above. Collapsing them is what makes
nesting arbitrary and free, rather than a feature to be added.
### Two axes of nesting, and they are different
This is the distinction the flat-storage rule is about, and conflating the two is
why "nested" and "flat with parent pointers" sound contradictory when they are
not:
| Axis | What nests | How it is stored |
| --- | --- | --- |
| **parent / child** | transform composition within one timeline | **flat, with parent pointers** — never nested maps |
| **instance** | a timeline inside another timeline | by reference into the library |
Each timeline is flat. Timelines nest. Every argument for flat storage —
addressability, one-field reparenting, structural sharing, per-node sync leaves —
is about the first axis and is untouched by the second.
The instance boundary is also **the only place the frame space changes.** Within
a timeline, `:time` is exposure and lead: a shift inside one space. At an
instance it is `(f - at)·rate + in`, into a different one. That is why `:rate` is
meaningless on an ordinary node and why sampling one must fail loudly rather than
be ignored.
### What is scoped to a timeline
Three fields on a node only have meaning relative to a timeline, and the answer
for all three is the same — **their own**:
- **`:z`** orders among siblings; a node cannot interleave with nodes inside a
nested instance. The instance occupies one position in its parent's order and
its contents sort beneath it, which the z path gives for free by being a
vector.
- **`:stencil`** names a node in the same timeline. A colour key does not
naturally respect a boundary — it is just pixels — so this is a rule rather
than a consequence, and it is Flash's rule for masks.
- **`:span`** is in the parent node's frame space.
### Instances
A node with `:kind :symbol` and `:of :sym/blink` places one. Its own channels
compose *over* the symbol's, so one definition is placed many times and tinted,
offset or retimed at each placement — that is how a three-frame blink is reused
at frames 40, 88 and 200 without copying it.
This is also where `docs/design.md`'s "closed vocabulary is right for the head"
lands: a plate library is a set of `:sym/head-*` definitions, and the strip
chooses which instance is placed on which frame. The take format's `plate=` field
becomes an instance reference.
lands: a plate library is a set of `:sym/head-*` timelines, and the strip chooses
which is instanced on which frame.
**Cursors and point buffers are per-instance, not per-node.** Two instances of
one symbol sit at different frames in their own space, so they cannot share a
reading head over the same channel. The resolver keys its caches by the instance
path, not by node id — which is a detail of `Making it fast` below, and the one
place symbol nesting is not free.
## Evaluating a frame
@ -266,6 +450,26 @@ becomes an instance reference.
The op list is the boundary with stage 7 in `docs/architecture.md`: the
rasteriser takes ops and knows nothing about nodes, channels or time.
**A photographic underlay is not an op.** The registered source frame that an
animator traces over is a reference, not output, and it may not enter the indexed
buffer — the same rule `docs/architecture.md` already sets for handles and
vertex boxes. It is a `drawImage` at an affine on a separate canvas, which clips
at the canvas edge for free, and the only thing it needs from the model is the
world transform of the node it rides:
```clojure
(world-of resolver :head) ;; -> Float64Array[6]
```
Composed with image-pixels-to-local — **both axes divided by `imgH`**, never by
their own dimension — the photo is registered with the shapes by construction,
and an unregistered underlay is merely decorative. Which frame it shows, the
current one or the held plate frame, is a UI choice and not a stored one.
A photo that has to sit *between* two drawn layers is the case that would make it
a `:bitmap` node with an op of its own. Nothing wants that yet: a reference is
either under everything or over everything at low alpha.
### Making it fast in CLJS
Three things, and only these three matter:
@ -278,8 +482,16 @@ Three things, and only these three matter:
on a seek. This is the difference between a `rsubseq` allocation per channel per
frame and none.
- **Preallocated point buffers per node.** Fixed topology means the size is known
at freeze time, so a frame allocates nothing. At 30fps, per-frame allocation is
the only thing that will make this stutter.
at freeze time, so the vertices — the overwhelming majority of the per-frame
bytes — are written into a buffer the node already owns. A frame still
allocates its op maps and the sorted op vector; that is a dozen small objects
against hundreds of points, and pooling them would buy nothing and cost the
ability to pass an op list around as plain data. At 30fps, per-vertex
allocation is the thing that will make this stutter.
Because the buffers are reused, **ops must be consumed before the next frame is
asked for.** That is the contract the rAF loop wants anyway: it reads, blits,
and dispatches nothing.
### What is in app-db, and what is not
@ -309,6 +521,9 @@ Proof that it covers what exists, not just what is wanted:
| square pupil | node `:pupil-r`, `:kind :rect`, parent `:iris-r`, stencil `:iris-r` |
| brow ring + quantised raise | node `:brow-r`, `[:geom :pts]` dense (the traced ring with height removed), `[:xform :pos]` dense (the quantised raise). **The decomposition design.md insists on is two channels.** |
| head plate, kept frames | node `:head`, `:symbol` per instance, keys on `[:symbol]` at kept frames |
| `makeXform` face-oval crop | **gone.** Placement is `[:xform :*]` on `:face`; the stage clips |
| `stabilize` transforms | `[:xform :*]` on `:head` — framed identity, dense, or keyed at kept frames |
| registered underlay | not data — a UI layer riding `(world-of resolver :head)` |
| painted background cel | node per layer, `[:geom :pts]` **framed**, `[:style :color]` framed |
| `mouth lead` | `:time {:offset k}` on performance nodes only |
| `exposure` | `:time {:expose n}` on the clip root, inherited |
@ -320,21 +535,106 @@ measured *out* and put *back* or the brow moves twice. In this model that is not
an argument to remember — it is two channels on one node, and getting it wrong
would mean writing the height into both.
## Palettes
Three levels, and keeping them apart is what makes a palette swap a
**reinterpretation** rather than an edit:
| Level | Holds | Lives on |
| --- | --- | --- |
| **tone** | which mark this is — `:skin-dark` | `[:style :color]`, a channel on the node |
| **ramp** | what that tone looks like *here* | `:palette`, a channel on the timeline |
| **the ramps** | every named palette | the project |
A node names a **tone**, never a colour and never a ramp. Which ramp the tone is
read in is decided by the timeline the node is in. So the same drawing reads day
or night without one stored value changing — which is the entire payoff of
indexed colour, and is why `docs/design.md` forbids sampled RGB: once a shape
holds a measured colour there is nothing left to reinterpret.
Named palettes are **variants over one tone vocabulary**, not arbitrary colour
lists. `:day` and `:night` both define `:skin-dark`; that is what keeps a swap
total and keeps `docs/design.md`'s closed vocabulary closed. A tone the ramp in
scope does not define resolves to the loud magenta, like any other missing index.
### The scope rule
`:palette` on a timeline is a channel like any other:
```clojure
{:frames 91
:palette {:animated? true :interp :hold :keys {0 :day, 48 :dusk, 72 :night}}
:nodes {...}}
```
**Absent means inherit** from the instancing context. **Present means this
timeline's content is read in that ramp, and it travels with the timeline** — a
symbol authored against `:night` stays night wherever it is placed. That is
lexical scope, and deliberately: a character with their own palette is a
character, not a decoration of whichever scene they were dropped into.
Composition is the same walk as `:time` — down the instance chain, **innermost
set palette wins**. An enclosing timeline's palette therefore applies to
everything inside it that does not set its own, which is adjustment-layer
behaviour with no adjustment layer in it. It is just scope.
And because it is an ordinary channel, a project switches palette over time with
keys on the root timeline, a child timeline switches on its own, and neither
knows about the other.
### One index space, partitioned by palette
A raster is one `Uint8Array` and an index means one colour in it, so two ramps in
one frame cannot both own index 2. The resolution: **the output index space is
the concatenation of the named palettes**, and a tone resolves to
`palette-base + tone-index`.
Everything downstream is then unchanged — one buffer, one flat table for
`->rgba`, no per-frame palette construction, and an index does not change meaning
between frames, so bakes and thumbnails stay valid.
Two consequences worth stating rather than discovering:
- **The limit is real and reachable.** 256 indices over a nine-tone vocabulary is
twenty-eight palettes. Detect it and say so; do not let it arrive as wrapped
colour.
- **It makes the stencil sharper.** A stencil is a colour key, so two nodes
sharing a tone share a stencil — a genuine weakness of the technique.
Partitioning the index space by palette means two nodes in *different* palettes
no longer collide at all, and the resolved stencil picks up whichever index the
stencil node actually drew in.
### Where it is resolved
At the op boundary, and nowhere else. `[:style :color]` holds a keyword all the
way through evaluation; the walk carries the palette in scope the same way it
carries the parent transform and the local frame; the op carries a resolved
index. The rasteriser never sees a tone name and the node never sees an index.
This also means the palette is a **parameter of evaluation**, not a global. The
resolver takes it alongside the store.
## Format on disk and on the wire
Tier 1 is EDN/transit: the node tree, channel definitions, framed values, keys,
layers, library. Kilobytes, human-readable, diffable, and leaf-addressable for
sync.
Dense blocks are separate content-addressed binaries — `Int16Array` for raster
Dense blocks are separate content-addressed binaries — `Int16Array` for
geometry, `Float32Array` for transforms — with a small header naming the channel
path, frame count and stride.
path, frame count, stride, and the **fixed-point scale** of the node-local space
the block is in. Geometry is stored in the node's own space, not in raster space;
see "What space geometry is in".
**Not Lottie internally**, despite the property shape being borrowed from it.
Lottie has no palette-indexed colour, its shapes are bezier with in/out tangents
where these are integer polygons, and its interpolation defaults are the opposite
of what is wanted. It is a good **export** target later, next to the `.take`
writer, and a bad internal format.
of what is wanted. It is a fine thing to write out one day and a bad thing to
store.
Output is deliberately not specified here. The target is encoding video in the
browser, which touches the op list and nothing above it — a writer consumes
frames, and frames are what stage 7 already produces.
## Deferred
@ -348,4 +648,6 @@ writer, and a bad internal format.
- **Instance channel overrides on symbols.** Compose-over is specified; only
colour and transform need it at first.
- **Constraints and drivers.** Blender's other half. A gaze that aims at a null
object is the obvious first one, and it is a long way off.
object is the obvious first one, and it is a long way off. Until then the one
gaze shared by two iris nodes is a UI rule, not a stored relationship — see
"One signal, two nodes".

14
docs/architecture.md
View file

@ -14,7 +14,7 @@ decision that is currently made in two places at once.
## What is actually wrong with the current shape
The pure modules are fine. `pipeline.js`, `mathutil.js`, `landmarks.js`,
`interior.js`, `raster.js` and `take.js` are functions over data with the
`interior.js` and `raster.js` are functions over data with the
reasoning written down next to them, and they port nearly verbatim.
`app.js` is the whole problem, and not because it is long. It is long because
@ -26,7 +26,7 @@ six unrelated jobs are braided together in it:
- resolving what is on screen at a frame (`plateIndex`, `perfIndex`, `leadIndex`),
- rasterising (`renderFrame`, `compositeRender`),
- driving the clock (`tick`),
- persisting (`saveCels`, `loadCels`, `exportTake`).
- persisting (`saveCels`, `loadCels`).
Every one of those is a different layer, and `rebuild` recomputes all of them
whenever any knob moves. That is affordable at one clip and 72 frames and is
@ -90,7 +90,7 @@ Renaming this later costs a day.
## The node, decomposed
`take.js` today gives a part a `parent` and a `clip`, and `parent` is doing
The prototype today gives a part a `parent` and a `clip`, and `parent` is doing
nothing except documenting intent — `mouth_in` is already in the same
head-local raster space as `mouth`, so composing its transform would be
composing identity. The moment a painted cel is attached to a head plate that
@ -100,7 +100,7 @@ two fields have to come apart:
| Field | What it does | Wrong to conflate because |
| --- | --- | --- |
| `:parent` | Transform composition. Child geometry is in parent's local space. | A node can be drawn over its parent without inheriting its motion. |
| `:stencil` | Colour-key clip, the `clip=` of the take format. | The iris is stencilled by the sclera and parented to the lid ring; those are different nodes. |
| `:stencil` | Colour-key clip: write only where the buffer already holds that index. | The iris is stencilled by the sclera and parented to the lid ring; those are different nodes. |
| `:z` | Draw order. | Order is authored per scene, not implied by the tree. |
A node is then:
@ -125,7 +125,7 @@ data shape that keeps the promise.
Each node also gets an optional local transform channel — translate, rotate,
scale, squash. That is the thing the current tool cannot express and that the
`slot_mouth` / `scale` / `rot` / `squash` fields in `take.js` are reaching for.
`slot_mouth` / `scale` / `rot` / `squash` fields in the prototype are reaching for.
## The flow, in seven stages
@ -165,7 +165,6 @@ dragging that slider does not re-run the interior extraction.
| `toRasterRing`, iris placement at ring slots, `offsetRing` | 6 resolve |
| `IndexedRaster`, `drawCel` | 7 palette |
| `drawRegistered`, `posterizeInto` | reference only, off to the side |
| `writeTake` | serialization, off the end |
Three things in `app.js` currently straddle a boundary, and each straddle is a
bug waiting for a bigger project:
@ -212,7 +211,6 @@ src/arthur/
clip.cljs clip entity; the frame-space conversions
cel.cljs painted vector layers
timeline.cljs sequence: clip placement, qf <-> cf
take.cljs take-file writer (port take.js)
flow/
ingest.cljs
detect.cljs the MediaPipe boundary, and the only one
@ -237,7 +235,7 @@ src/arthur/
mode/ one ns per tool mode
panel/ strip, worksheet, readouts, palette, layers
canvas.cljs the one imperative sink
fx/ mediapipe, files, audio, persistence, export
fx/ mediapipe, files, audio, persistence
```
Two rules about this tree. `domain/` may not require `flow/`, and neither may

36
docs/port-plan.md
View file

@ -265,13 +265,30 @@ Port `stabilize` and the lip rings out of `pipeline.js`. Split **condition**
(`smoothTransforms`, `smoothContours`) into its own stage so the two smoothing
knobs do not re-run measurement.
**Done:** measured numbers match the JS on the synthetic track.
**Done:** measured numbers match the JS on the synthetic track. Note that
parity here is on `stabilize`'s output, not on `toRasterRing`'s — the framing
step is being deleted, not ported.
### 5 — freeze
The new module, and the heart of this work: measurements → channels. A dense
`[:geom :pts]` block per node — `Int16Array[frames × verts × 2]`, raster space,
grid-snapped — with `:generated` attached. Fixed topology is what makes this a
rectangular array with no per-frame header.
`[:geom :pts]` block per node — `Int16Array[frames × verts × 2]` in the node's
own local space, with the block's fixed-point scale in its header — plus
`:generated`. Fixed topology is what makes this a rectangular array with no
per-frame header.
**Do not port `makeXform`.** The prototype bakes the framing into the stored
numbers: it centres on the face oval's bbox and zooms until the face is 80% of
the raster height, so every vertex carries a cropping decision made once from one
frame's landmarks. Dropping it is a deletion. Placement becomes `[:xform :*]` on
an authored `:face` node, the stage clips whatever hangs off, and project
dimensions stop being tied to the footage. See "What space geometry is in" in
`docs/animation-model.md`.
The anchor transform freezes onto `:head`, one level under `:face`, and the
normalise on/off/per-plate toggle is which of the three channel shapes that node
carries. Always measure and always store factored, whatever the toggle says:
smoothing and velocity-minimum key selection both require the split to exist in
storage.
**Done:** the synthetic take plays back as a moving mouth. Full vertical slice.
@ -299,10 +316,15 @@ The parameter UI, as leaf-addressed params in app-db
(`clip/:cid/params/:subject/:area` — see below), so the sync layer added later has
nothing to retrofit.
### 9 — backend and take export
### 9 — backend
Django project, the `clips` app, models for Project/Clip/Footage/Analysis/Leaf,
and project load/save. Port `take.js` — sixty-five lines, and it is the proof the
model serialises.
and project load/save. Round-tripping a project through the server is the proof
the model serialises.
**Output is not in this plan.** The `.take` writer in `js/take.js` was for
driving an Animator Pro render script and it is not where this is going: the
target is encoding video in the browser, and that is a separate piece of design
nobody should pre-empt by porting the old one.
## Two things to not foreclose