# arthur — the animation model The data that describes a moving picture: what the primitives are, how they nest, how they change over time, and how rotoscoped and hand-authored work end up being the same thing with one flag between them. `docs/design.md` is the aesthetic argument. `docs/architecture.md` is where the code goes. This is the type that both of them are about. ## What this replaces `docs/design.md` has a table of five kinds of part — plate, feature, interior, primitive, scalar — each with its own source, vocabulary and interpolation. That table is a good description of **where data comes from** and a bad description of **what data is**, and the current code follows it too literally: eyes, brows, teeth and mouth each get their own build function, their own key shape and their own path through `take.js`. They are all one thing. A part is a **node** with **channels**, and the five kinds collapse into differences of which channels exist and who filled them in. ## Prior art, and what each one gets right | System | The idea worth taking | | --- | --- | | **Flash / SWF** | A **library of symbols** and a timeline of **instances** at depths. "Framed" content that simply exists on a frame, versus tweened content. `DefineMorphShape` requires matching vertex counts — the fixed-topology rule, arrived at from the other direction. | | **Blender** | Animation is **addressed by path into the data** (`location[0]`), not stored as fields on the object. An Action is a bag of F-Curves. That decoupling is what makes the dope sheet, the graph editor and the NLA three views of one dataset. Also: parenting captures a `parent_inverse` so the child does not jump. | | **After Effects** | Every leaf property is animatable, uniformly. Property groups form a tree. Pre-comps nest arbitrarily and a pre-comp is just a layer. | | **Lottie** | The uniform property shape: `{a: 0, k: }` or `{a: 1, k: []}`. One representation for static and animated, which is exactly "framed or keyframed". | | **Grease Pencil** | A 2D layer holds frames at frame numbers, and a frame **holds until the next one**. Hold is the default, not a special case. | What none of them get right for this project: colour. All four store RGB on the shape. `docs/design.md` forbids that, so colour is a palette index here and it is a channel like any other. ## The one idea **Analysis is a channel generator.** It does not produce a different kind of data; it produces keys, densely, on the same channels a hand would fill in sparsely. So: ``` footage ──▶ analysis ──▶ FREEZE ──▶ channels on nodes ──▶ evaluate ──▶ raster ▲ hand authoring ──┘ ``` Freezing is not a conversion into a second format. There is one format, and freezing fills it in. That is what makes "the only difference is a special flag" literally true: the flag is provenance on a channel, and nothing in the renderer reads it. ## Node A node is an instance in the scene. The tree is stored **flat, with parent pointers** — never as nested maps. ```clojure {:id :mouth :name "mouth" :kind :poly ; :poly :disc :rect :group :bitmap :symbol :parent :head ; nil at the root :z "a3" ; fractional index, ordered among all siblings :symbol nil ; or :sym/blink — see Symbols :stencil :mouth-in ; colour-key clip; structural, not a channel :span [0 240] ; in/out in the parent's frame space :pinv [1 0 0 1 0 0] ; parent-inverse, captured when parented :channels {...}} ``` Flat with pointers, for four reasons that all point the same way: any node is addressable without a walk; reparenting is a one-field write rather than a subtree move; an edit to a leaf does not change the identity of its ancestors, so re-frame's structural sharing keeps ancestor subs from invalidating; and it is what lets every node be its own sync leaf. Flash, Blender and AE all store it this way. `:span` is Lottie's `ip`/`op` and Flash's `PlaceObject`/`RemoveObject`: the range over which the node exists at all. Distinct from a `[:vis]` channel, which blinks an existing node on and off. ## Channel Every animatable property is a channel, and channels are addressed **by path**: ```clojure :channels {[:xform :pos] {:animated? false :value [0.0 0.0]} [:xform :rot] {:animated? false :value 0.0} [:xform :scale] {:animated? false :value [1.0 1.0]} [:xform :skew] {:animated? false :value [0.0 0.0]} [:xform :anchor]{:animated? false :value [0.0 0.0]} [:geom :pts] {:animated? true :interp :hold :dense {...} :generated {...}} [:style :color] {:animated? false :value :skin-dark} [:vis] {:animated? true :interp :hold :keys {0 true, 37 false}}} ``` A path is a **vector**, not a string — CLJS maps take vectors as keys natively, so Blender's `data_path` idea arrives with no parsing. The set of valid paths for a node follows from its `:kind`, and that is a spec, not a schema migration. Three channel shapes, and the uniformity across them is the point: ```clojure ;; FRAMED — one static thing. No animation, no vertex correspondence to worry ;; about. A painted background cel is this. {:animated? false :value v} ;; KEYED — sparse, authored, in the document. Undoable and syncable. {:animated? true :interp :hold :keys {0 v, 4 v, 12 v}} ;; DENSE — generated, one value per frame, held in tier 2 as a typed array. {:animated? true :interp :hold :dense {:store "sha256:…" :offset 0 :stride 40 :frames 600} :generated {...}} ``` `:interp` defaults to `:hold`, which `docs/design.md` requires of every cut part. A key may carry its own `:interp` to override the channel's, which is how Lottie and Blender both do per-key easing; nothing uses it yet and the door is cheap to leave open. ### Keys are a map by frame, not a list Already argued in `docs/architecture.md` for merge reasons; here it also gives "the most recent key at or before `f`" as a `rsubseq` on a sorted map instead of a scan. **Store a plain map** in the document — transit and JSON both lose sortedness — and build the sorted index in the resolver. ### The flag lives on the channel, not the node ```clojure :generated {:by :roto/lips-outer :analysis "sha256:…" ; which analysis artifact :params {:verts 8 :contour-avg 1 :aperture-cut 0.004}} ``` Present means the UI offers a parameter panel and a re-freeze button. Absent means the UI offers the keys directly. **The renderer never reads it.** It belongs on the channel rather than the node because a node routinely wants both at once: a mouth whose `[:geom :pts]` is rotoscoped and whose `[:xform :pos]` is hand-animated to sit on a plate. Putting the flag on the node would forbid the most useful thing in the model. ### Channels are layered A channel is a base plus optional override layers, and a layer declares how it combines: ```clojure {:animated? true :interp :hold :dense {...} :generated {...} :over [{:blend :offset :keys {88 [2 0], 96 [0 0]}} {:blend :replace :keys {104 [[3 7] [4 7] …]}}]} ``` - **`:offset`** adds a delta to the base. "Nudge the mouth two pixels right for ten frames" survives a re-freeze at different parameters, because it was never a position — it was a correction. - **`:replace`** wins outright. For the frame where detection simply failed. This is what `docs/design.md` means by an override layer, and it is why re-freezing is safe: the base is regenerated, the layers are untouched. It is Blender's NLA blending and AE's effect stack at one property. ## Transform: decomposed, never a matrix ```clojure {:pos [x y] :rot θ :scale [sx sy] :skew [kx ky] :anchor [ax ay]} ``` Stored decomposed for two reasons. Each component has to be independently keyframable, which is the entire point of channels. And interpolating matrix entries is meaningless — a rotation tweened through its matrix shears on the way. Composition, per node: ``` local = T(pos) · T(anchor) · R(rot) · K(skew) · S(scale) · T(-anchor) world = world(parent) · pinv · local ``` `:anchor` is Flash's registration point and Blender's origin: rotation and scale happen about it, and getting it wrong is why hand-placed parts swing rather than turn. `:pinv` is Blender's `parent_inverse`, captured at the moment of parenting so the child does not jump when it acquires a parent. Small, and its absence is the kind of thing that makes a parenting feature feel broken. **The similarity fit already produces a decomposition.** `fitSimilarity` returns `{s θ tx ty}`, which drops straight into `[:xform :scale]`, `[:xform :rot]` and `[:xform :pos]` with no conversion. The analysis output and the animation model meet without an adapter, which is a sign the decomposition is the right one. ## Time maps — exposure, lead and symbol timing are one thing Every node may map the frame it is evaluated at: ```clojure :time {:mode :inherit} ; the default, and almost always right :time {:mode :map :expose 2 :offset -1 :rate 1.0 :loop? false} ``` Three features that look unrelated are this one mechanism: - **exposure** is `⌊f/n⌋·n`, - **mouth lead** is `f + k`, - **a symbol instance's timing** is `(f - at)·rate + in`, with optional looping. Composed along the nesting chain, outermost first. Two rules follow, and they are different rules: - **Exposure inherits strictly.** `docs/design.md` is emphatic that everything rides one grid, because a head cutting on odd frames against a mouth cutting on even ones reads as two performances. The model permits a per-node grid; the default must be `:inherit`, and setting it lower is a deliberate act the UI should make feel like one. - **Offset is per-node by design.** Mouth lead applies to performance nodes and *not* to the plate, which is the whole point of it — so the offset genuinely belongs at the node, not the clip. ## Symbols The Flash idea, kept: ```clojure :library {:sym/head {:kind :poly :channels {...}} :sym/blink {:kind :timeline :frames 3 :nodes [...]}} ``` A node with `:symbol :sym/blink` is an **instance**. Its own channels compose *over* the symbol's, so one definition can be placed many times and tinted, offset or retimed at each placement. A `:timeline` symbol has its own frame space, which the instance's `:time` maps into — that is Flash's MovieClip, Lottie's precomp and AE's pre-comp, and it is how a three-frame blink gets reused at frames 40, 88 and 200 without copying it. This is also where `docs/design.md`'s "closed vocabulary is right for the head" lands: a plate library is a set of `:sym/head-*` definitions, and the strip chooses which instance is placed on which frame. The take format's `plate=` field becomes an instance reference. ## Evaluating a frame ```clojure (defn eval-frame "Scene at clip frame f -> draw ops in z order. Pure." [scene f] ...) ``` 1. Walk nodes in **topological order** by parent depth (cached; recompute only when parentage changes). 2. Skip nodes outside `:span`. 3. Apply the node's time map to get its own local frame `fn`. 4. **Sample** each channel at `fn`: a map lookup for framed, a sorted-index lookup for keyed, an array read for dense. Then apply `:over` layers. 5. Compose `world` from the parent's. 6. Transform geometry into raster space, writing into a **preallocated buffer** owned by the node. 7. Emit `{:kind :poly :pts buf :n 20 :color idx :stencil id}`. 8. Sort by resolved `z`. The op list is the boundary with stage 7 in `docs/architecture.md`: the rasteriser takes ops and knows nothing about nodes, channels or time. ### Making it fast in CLJS Three things, and only these three matter: - **Decomposed and persistent for storage; flat and mutable for evaluation.** Composed transforms are 6-element `Float64Array`s, not maps. Every renderer does this; the storage form and the evaluation form are allowed to differ. - **A cursor per channel.** Playback is sequential, so "most recent key at or before `f`" is an advance of a saved index, O(1) amortised. Binary search only on a seek. This is the difference between a `rsubseq` allocation per channel per frame and none. - **Preallocated point buffers per node.** Fixed topology means the size is known at freeze time, so a frame allocates nothing. At 30fps, per-frame allocation is the only thing that will make this stutter. ### What is in app-db, and what is not | In app-db (tier 1) | In tier 2, behind a handle | | --- | --- | | nodes, parentage, z, spans, stencils | dense channel blocks | | channel definitions, `:interp`, `:generated` | analysis artifacts | | **framed** values, **keyed** keys, `:over` layers | preallocated eval buffers | | library / symbol definitions | composed transform scratch | The rule: **anything a human placed is in the document; anything a generator produced is a handle.** Which is the same line `docs/architecture.md` draws for sync and baking, arrived at again from the renderer's side. ## The current parts, in this model Proof that it covers what exists, not just what is wanted: | Now | Becomes | | --- | --- | | `mouth` outer ring, every frame | node `:mouth`, `[:geom :pts]` dense, `:generated {:by :roto/lips-outer}` | | `mouth_in`, hidden below aperture | node `:mouth-in`, parent `:mouth`, `[:geom :pts]` dense + `[:vis]` dense | | `teeth` from image content | node `:teeth`, stencil `:mouth-in`, `[:geom :pts]` dense, `:generated {:by :interior/teeth}` | | lid rings | nodes `:lid-r/-l`, `[:geom :pts]` dense | | lash line (`offsetRing`) | not data — a stage-6 parameter on the node, `{:grow px}` | | iris disc | node `:iris-r`, `:kind :disc`, parent `:lid-r`, stencil `:sclera-r`, `[:xform :pos]` dense (quantised at freeze), radius framed | | square pupil | node `:pupil-r`, `:kind :rect`, parent `:iris-r`, stencil `:iris-r` | | brow ring + quantised raise | node `:brow-r`, `[:geom :pts]` dense (the traced ring with height removed), `[:xform :pos]` dense (the quantised raise). **The decomposition design.md insists on is two channels.** | | head plate, kept frames | node `:head`, `:symbol` per instance, keys on `[:symbol]` at kept frames | | painted background cel | node per layer, `[:geom :pts]` **framed**, `[:style :color]` framed | | `mouth lead` | `:time {:offset k}` on performance nodes only | | `exposure` | `:time {:expose n}` on the clip root, inherited | | hand correction | an `:over` layer, `:offset` or `:replace` | The brow row is the one worth looking at twice. `docs/design.md` argues at length that the traced ring already contains the height, so the quantised raise must be measured *out* and put *back* or the brow moves twice. In this model that is not an argument to remember — it is two channels on one node, and getting it wrong would mean writing the height into both. ## Format on disk and on the wire Tier 1 is EDN/transit: the node tree, channel definitions, framed values, keys, layers, library. Kilobytes, human-readable, diffable, and leaf-addressable for sync. Dense blocks are separate content-addressed binaries — `Int16Array` for raster geometry, `Float32Array` for transforms — with a small header naming the channel path, frame count and stride. **Not Lottie internally**, despite the property shape being borrowed from it. Lottie has no palette-indexed colour, its shapes are bezier with in/out tangents where these are integer polygons, and its interpolation defaults are the opposite of what is wanted. It is a good **export** target later, next to the `.take` writer, and a bad internal format. ## Deferred - **Per-key easing.** The structure allows it; nothing should use it until a parented transform on a painted cel asks for it. - **More than two channel layers.** The `:over` vector is already a list; a real blend stack with weights is the NLA, and it is not needed to fix a bad frame. - **Skew beyond the field.** `[:xform :skew]` is in the transform and in the composition order from the start, because adding a component to a decomposition later means migrating every stored transform. - **Instance channel overrides on symbols.** Compose-over is specified; only colour and transform need it at first. - **Constraints and drivers.** Blender's other half. A gaze that aims at a null object is the obvious first one, and it is a long way off.