arthur/docs/animation-model.md

arthur — the animation model

The data that describes a moving picture: what the primitives are, how they nest, how they change over time, and how rotoscoped and hand-authored work end up being the same thing with one flag between them.

docs/design.md is the aesthetic argument. docs/architecture.md is where the code goes. This is the type that both of them are about.

What this replaces

docs/design.md has a table of five kinds of part — plate, feature, interior, primitive, scalar — each with its own source, vocabulary and interpolation. That table is a good description of where data comes from and a bad description of what data is, and the current code follows it too literally: eyes, brows, teeth and mouth each get their own build function, their own key shape and their own path through take.js.

They are all one thing. A part is a node with channels, and the five kinds collapse into differences of which channels exist and who filled them in.

Prior art, and what each one gets right

System	The idea worth taking
Flash / SWF	A library of symbols and a timeline of instances at depths. "Framed" content that simply exists on a frame, versus tweened content. DefineMorphShape requires matching vertex counts — the fixed-topology rule, arrived at from the other direction.
Blender	Animation is addressed by path into the data (location[0]), not stored as fields on the object. An Action is a bag of F-Curves. That decoupling is what makes the dope sheet, the graph editor and the NLA three views of one dataset. Also: parenting captures a parent_inverse so the child does not jump.
After Effects	Every leaf property is animatable, uniformly. Property groups form a tree. Pre-comps nest arbitrarily and a pre-comp is just a layer.
Lottie	The uniform property shape: {a: 0, k: <value>} or {a: 1, k: [<keys>]}. One representation for static and animated, which is exactly "framed or keyframed".
Grease Pencil	A 2D layer holds frames at frame numbers, and a frame holds until the next one. Hold is the default, not a special case.

What none of them get right for this project: colour. All four store RGB on the shape. docs/design.md forbids that, so colour is a palette index here and it is a channel like any other.

The one idea

Analysis is a channel generator. It does not produce a different kind of data; it produces keys, densely, on the same channels a hand would fill in sparsely. So:

footage ──▶ analysis ──▶ FREEZE ──▶ channels on nodes ──▶ evaluate ──▶ raster
                                        ▲
                       hand authoring ──┘

Freezing is not a conversion into a second format. There is one format, and freezing fills it in. That is what makes "the only difference is a special flag" literally true: the flag is provenance on a channel, and nothing in the renderer reads it.

Node

A node is an instance in the scene. The tree is stored flat, with parent pointers — never as nested maps.

{:id      :mouth
 :name    "mouth"
 :kind    :poly                  ; :poly :disc :rect :group :bitmap :symbol
 :parent  :head                  ; nil at the root
 :z       "a3"                   ; fractional index, ordered among all siblings
 :symbol  nil                    ; or :sym/blink — see Symbols
 :stencil :mouth-in              ; colour-key clip; structural, not a channel
 :span    [0 240]                ; in/out in the parent's frame space
 :pinv    [1 0 0 1 0 0]          ; parent-inverse, captured when parented
 :channels {...}}

Flat with pointers, for four reasons that all point the same way: any node is addressable without a walk; reparenting is a one-field write rather than a subtree move; an edit to a leaf does not change the identity of its ancestors, so re-frame's structural sharing keeps ancestor subs from invalidating; and it is what lets every node be its own sync leaf. Flash, Blender and AE all store it this way.

:span is Lottie's ip/op and Flash's PlaceObject/RemoveObject: the range over which the node exists at all. Distinct from a [:vis] channel, which blinks an existing node on and off.

Channel

Every animatable property is a channel, and channels are addressed by path:

:channels
{[:xform :pos]   {:animated? false :value [0.0 0.0]}
 [:xform :rot]   {:animated? false :value 0.0}
 [:xform :scale] {:animated? false :value [1.0 1.0]}
 [:xform :skew]  {:animated? false :value [0.0 0.0]}
 [:xform :anchor]{:animated? false :value [0.0 0.0]}
 [:geom :pts]    {:animated? true :interp :hold :dense {...} :generated {...}}
 [:style :color] {:animated? false :value :skin-dark}
 [:vis]          {:animated? true :interp :hold :keys {0 true, 37 false}}}

A path is a vector, not a string — CLJS maps take vectors as keys natively, so Blender's data_path idea arrives with no parsing. The set of valid paths for a node follows from its :kind, and that is a spec, not a schema migration.

Three channel shapes, and the uniformity across them is the point:

;; FRAMED — one static thing. No animation, no vertex correspondence to worry
;; about. A painted background cel is this.
{:animated? false :value v}

;; KEYED — sparse, authored, in the document. Undoable and syncable.
{:animated? true :interp :hold :keys {0 v, 4 v, 12 v}}

;; DENSE — generated, one value per frame, held in tier 2 as a typed array.
{:animated? true :interp :hold
 :dense {:store "sha256:…" :offset 0 :stride 40 :frames 600}
 :generated {...}}

:interp defaults to :hold, which docs/design.md requires of every cut part. A key may carry its own :interp to override the channel's, which is how Lottie and Blender both do per-key easing; nothing uses it yet and the door is cheap to leave open.

Keys are a map by frame, not a list

Already argued in docs/architecture.md for merge reasons; here it also gives "the most recent key at or before f" as a rsubseq on a sorted map instead of a scan. Store a plain map in the document — transit and JSON both lose sortedness — and build the sorted index in the resolver.

The flag lives on the channel, not the node

:generated {:by       :roto/lips-outer
            :analysis "sha256:…"          ; which analysis artifact
            :params   {:verts 8 :contour-avg 1 :aperture-cut 0.004}}

Present means the UI offers a parameter panel and a re-freeze button. Absent means the UI offers the keys directly. The renderer never reads it.

It belongs on the channel rather than the node because a node routinely wants both at once: a mouth whose [:geom :pts] is rotoscoped and whose [:xform :pos] is hand-animated to sit on a plate. Putting the flag on the node would forbid the most useful thing in the model.

Channels are layered

A channel is a base plus optional override layers, and a layer declares how it combines:

{:animated? true :interp :hold
 :dense {...} :generated {...}
 :over  [{:blend :offset  :keys {88 [2 0], 96 [0 0]}}
         {:blend :replace :keys {104 [[3 7] [4 7] …]}}]}

• :offset adds a delta to the base. "Nudge the mouth two pixels right for ten frames" survives a re-freeze at different parameters, because it was never a position — it was a correction.
• :replace wins outright. For the frame where detection simply failed.

This is what docs/design.md means by an override layer, and it is why re-freezing is safe: the base is regenerated, the layers are untouched. It is Blender's NLA blending and AE's effect stack at one property.

Transform: decomposed, never a matrix

{:pos [x y] :rot θ :scale [sx sy] :skew [kx ky] :anchor [ax ay]}

Stored decomposed for two reasons. Each component has to be independently keyframable, which is the entire point of channels. And interpolating matrix entries is meaningless — a rotation tweened through its matrix shears on the way.

Composition, per node:

local = T(pos) · T(anchor) · R(rot) · K(skew) · S(scale) · T(-anchor)
world = world(parent) · pinv · local

:anchor is Flash's registration point and Blender's origin: rotation and scale happen about it, and getting it wrong is why hand-placed parts swing rather than turn.

:pinv is Blender's parent_inverse, captured at the moment of parenting so the child does not jump when it acquires a parent. Small, and its absence is the kind of thing that makes a parenting feature feel broken.

The similarity fit already produces a decomposition. fitSimilarity returns {s θ tx ty}, which drops straight into [:xform :scale], [:xform :rot] and [:xform :pos] with no conversion. The analysis output and the animation model meet without an adapter, which is a sign the decomposition is the right one.

Time maps — exposure, lead and symbol timing are one thing

Every node may map the frame it is evaluated at:

:time {:mode :inherit}                        ; the default, and almost always right
:time {:mode :map :expose 2 :offset -1 :rate 1.0 :loop? false}

Three features that look unrelated are this one mechanism:

• exposure is ⌊f/n⌋·n,
• mouth lead is f + k,
• a symbol instance's timing is (f - at)·rate + in, with optional looping.

Composed along the nesting chain, outermost first. Two rules follow, and they are different rules:

• Exposure inherits strictly. docs/design.md is emphatic that everything rides one grid, because a head cutting on odd frames against a mouth cutting on even ones reads as two performances. The model permits a per-node grid; the default must be :inherit, and setting it lower is a deliberate act the UI should make feel like one.
• Offset is per-node by design. Mouth lead applies to performance nodes and not to the plate, which is the whole point of it — so the offset genuinely belongs at the node, not the clip.

Symbols

The Flash idea, kept:

:library
{:sym/head        {:kind :poly :channels {...}}
 :sym/blink       {:kind :timeline :frames 3 :nodes [...]}}

A node with :symbol :sym/blink is an instance. Its own channels compose over the symbol's, so one definition can be placed many times and tinted, offset or retimed at each placement. A :timeline symbol has its own frame space, which the instance's :time maps into — that is Flash's MovieClip, Lottie's precomp and AE's pre-comp, and it is how a three-frame blink gets reused at frames 40, 88 and 200 without copying it.

This is also where docs/design.md's "closed vocabulary is right for the head" lands: a plate library is a set of :sym/head-* definitions, and the strip chooses which instance is placed on which frame. The take format's plate= field becomes an instance reference.

Evaluating a frame

(defn eval-frame
  "Scene at clip frame f -> draw ops in z order. Pure."
  [scene f] ...)

1. Walk nodes in topological order by parent depth (cached; recompute only when parentage changes).
2. Skip nodes outside :span.
3. Apply the node's time map to get its own local frame fn.
4. Sample each channel at fn: a map lookup for framed, a sorted-index lookup for keyed, an array read for dense. Then apply :over layers.
5. Compose world from the parent's.
6. Transform geometry into raster space, writing into a preallocated buffer owned by the node.
7. Emit {:kind :poly :pts buf :n 20 :color idx :stencil id}.
8. Sort by resolved z.

The op list is the boundary with stage 7 in docs/architecture.md: the rasteriser takes ops and knows nothing about nodes, channels or time.

Making it fast in CLJS

Three things, and only these three matter:

• Decomposed and persistent for storage; flat and mutable for evaluation. Composed transforms are 6-element Float64Arrays, not maps. Every renderer does this; the storage form and the evaluation form are allowed to differ.
• A cursor per channel. Playback is sequential, so "most recent key at or before f" is an advance of a saved index, O(1) amortised. Binary search only on a seek. This is the difference between a rsubseq allocation per channel per frame and none.
• Preallocated point buffers per node. Fixed topology means the size is known at freeze time, so a frame allocates nothing. At 30fps, per-frame allocation is the only thing that will make this stutter.

What is in app-db, and what is not

In app-db (tier 1)	In tier 2, behind a handle
nodes, parentage, z, spans, stencils	dense channel blocks
channel definitions, :interp, :generated	analysis artifacts
framed values, keyed keys, :over layers	preallocated eval buffers
library / symbol definitions	composed transform scratch

The rule: anything a human placed is in the document; anything a generator produced is a handle. Which is the same line docs/architecture.md draws for sync and baking, arrived at again from the renderer's side.

The current parts, in this model

Proof that it covers what exists, not just what is wanted:

Now	Becomes
mouth outer ring, every frame	node :mouth, [:geom :pts] dense, :generated {:by :roto/lips-outer}
mouth_in, hidden below aperture	node :mouth-in, parent :mouth, [:geom :pts] dense + [:vis] dense
teeth from image content	node :teeth, stencil :mouth-in, [:geom :pts] dense, :generated {:by :interior/teeth}
lid rings	nodes :lid-r/-l, [:geom :pts] dense
lash line (offsetRing)	not data — a stage-6 parameter on the node, {:grow px}
iris disc	node :iris-r, :kind :disc, parent :lid-r, stencil :sclera-r, [:xform :pos] dense (quantised at freeze), radius framed
square pupil	node :pupil-r, :kind :rect, parent :iris-r, stencil :iris-r
brow ring + quantised raise	node :brow-r, [:geom :pts] dense (the traced ring with height removed), [:xform :pos] dense (the quantised raise). The decomposition design.md insists on is two channels.
head plate, kept frames	node :head, :symbol per instance, keys on [:symbol] at kept frames
painted background cel	node per layer, [:geom :pts] framed, [:style :color] framed
mouth lead	:time {:offset k} on performance nodes only
exposure	:time {:expose n} on the clip root, inherited
hand correction	an :over layer, :offset or :replace

The brow row is the one worth looking at twice. docs/design.md argues at length that the traced ring already contains the height, so the quantised raise must be measured out and put back or the brow moves twice. In this model that is not an argument to remember — it is two channels on one node, and getting it wrong would mean writing the height into both.

Format on disk and on the wire

Tier 1 is EDN/transit: the node tree, channel definitions, framed values, keys, layers, library. Kilobytes, human-readable, diffable, and leaf-addressable for sync.

Dense blocks are separate content-addressed binaries — Int16Array for raster geometry, Float32Array for transforms — with a small header naming the channel path, frame count and stride.

Not Lottie internally, despite the property shape being borrowed from it. Lottie has no palette-indexed colour, its shapes are bezier with in/out tangents where these are integer polygons, and its interpolation defaults are the opposite of what is wanted. It is a good export target later, next to the .take writer, and a bad internal format.

Deferred

• Per-key easing. The structure allows it; nothing should use it until a parented transform on a painted cel asks for it.
• More than two channel layers. The :over vector is already a list; a real blend stack with weights is the NLA, and it is not needed to fix a bad frame.
• Skew beyond the field. [:xform :skew] is in the transform and in the composition order from the start, because adding a component to a decomposition later means migrating every stored transform.
• Instance channel overrides on symbols. Compose-over is specified; only colour and transform need it at first.
• Constraints and drivers. Blender's other half. A gaze that aims at a null object is the obvious first one, and it is a long way off.