# Plan: tracing is a symbol Status: built, 2026-10-03, on branch `worktree-tracing-layers`. Where the build differs from the plan below, the build wins: - The head's field is `:reads` (`{:holds [...]}` or `{:holds-of :plate}`), not `:follow`. - A still is not one frame. It gets as many frames as remain in the symbol it is dropped into, at that symbol's rate, and is trimmed like any clip. - An image's `:media` is `{:image }`, served at `/blob/`. The server's `Image` row exists for the pool's list and labels, not for identity. - Dropping media that a tracing symbol in the document already shows reuses that symbol. - Nothing can be created or dropped inside a tracing symbol (`creation/target`, `drop-destination-at`, `nest/move-refusal`), and one cannot be opened in a tab. - Schema 6. Every project is marked 6. One that still carries `:trace` on a node is refused when opened, with what to do about it, rather than all old projects being refused. No backward compatibility (see `lane-model.md`, "Goal and compatibility policy"). ## What is wrong today Tracing is not a thing in the document. It is three mechanisms that each know about faces: - **`:trace {:frames :origin}` on a face's `:head`.** It decides which measured frame the head reads (`symbol/base-channel-frame` → `trace/held-frame`), and the underlay also reads it to decide which photo to show (`trace/photo-frame`). - **`ui/underlay`**, a painter that walks `trace/shown` → `trace/faces` (a separate instance walk), looks up each face's subject → analysis → footage, asks the resolver where `[...path :head]` went, and builds the photo's matrix by hand: `world(head) · M(p)⁻¹ · 1/imageH` (`trace/photo-matrix`). - **`[:ui :trace {:faces #{} :opacity}]`**, a per-face switch in editor state, plus the special "opening a face shows its footage, opening a take does not" rule (`trace/showing-for`). What you cannot do: place footage or a still as a reference where you like, move it, scale it, turn it, trim it, hold it, put it in a lane, or trace something that is not a tracked face. The photo is not selectable and has no row. The one case that works (a face over its own footage) runs on code that nothing else uses. ## The model in one paragraph A **tracing symbol** is a symbol with `:type :trace`. It has no nodes. It names media (a footage range or a still image) and has that media's frame count, fps and pixel size. It is **placed by an ordinary instance**, so lanes, spans, trim, split, move, playback (`:in :speed :end`), transform, nesting, selection, picking and gestures all work on it with no new code. It is evaluated to a single **`:trace` op**, which never reaches the raster or an export. A face's footage is not a special case. It is one of these instances, placed inside the face as a child of `:head` and carrying the measured registration transform. The trace keys become **holds on that instance's time**. The head's "origin" becomes the head saying **which node's frames it follows**. This follows the precedent of `:type :palette` symbols (no nodes, they name an asset, and they are placed by instances in a lane), so the uniformity rule holds (see the `model-uniformity` memory). Nothing new holds nodes, and no special instance kind is added. ## Data model ### The tracing symbol ```clojure :footage-8625 ; a symbol id like any other {:id :footage-8625 :name "8625.mov" :type :trace :media {:footage #uuid "f8ca…" :range [12 241]} ; or {:image "sha256:…"} :frames 229 ; the range's length; 1 for a still :fps 30 ; the footage's own rate, so cadence handles 30→12 for free :width 1440 :height 1920 ; pixel size = the symbol's own stage :audio {:footage #uuid "f8ca…"} ; optional, the existing "a symbol says what it sounds like" :nodes {}} ``` - `:media` is the one new symbol key. Add it to `symbol/symbol-keys` and to the symbol leaf's `select-keys` in `leaf/leaves`. - The symbol's local space is **pixels of the media**: `[0 w) × [0 h)`. `clip/center` of a node-less symbol already returns its stage middle, which here is `[w/2 h/2]`. So `place-symbol` puts the anchor at the image centre with no new code. - `symbol/problems`: `:type :trace` needs `:media` with exactly one of `:footage`/`:image`, needs `(empty? nodes)`, and for footage needs `(= frames (- end start))`. - **One tracing symbol per footage range, shared.** Two faces from one take place the same symbol, and so does a hand-placed reference. Like any symbol it is reused by reference, and `bring/symbols` copies it like any other. ### Placing it It is an ordinary `:kind :instance` with `:source {:symbol :footage-8625}`: - **Start and end** are `:span` (in its own frames) and `:time :at`. You get these from the existing trim, split, move and roll. - **Which frame shows** comes from `:playback {:in :speed :end}`. Footage plays with speed 1, a frozen frame has speed 0, and a still is 1 frame with `:end :hold`. - **Placement in space** is `[:xform …]`, the same as every node: gestures, inspector and keys. - **In a lane, or on its own**: it is a parent-less spanned node, so a `:display :lane` symbol draws it as a block like any cel. It can also sit as a free node or under a group. To have a reference with its own tab, wrap it in an ordinary symbol. **Default on creation** (in the drop event, not in `place-symbol`): scale so the image's height fits the stage height, centred on the drop point. This is a creation default like "use center anchor when dropping", and nothing updates it afterwards. ### Holds: the one new time feature `:time {:holds [0 12 30]}` floors a node's local frame to the last hold at or before it. Before the first hold, the first hold applies. This is `:expose` generalized from a regular grid to authored frames. It is applied in `node/local-frame` at the same point as `:expose`, and is inherited in the same way. It is in the node's **own** frames. On a tracing instance with `:in 0 :speed 1`, own frames are source frames, so the hold list is the set of traced frames. It is general on purpose. Holding a playing symbol on chosen drawings is the same feature. It costs about three lines in `local-frame`, one `problems` clause (sorted, distinct, finite), and the timeline drawing hold frames as marks on the row. `symbol/frame-map` refuses floors (`:expose > 1`). It must also refuse a non-empty `:holds`, for the same reason. ## The face: "a child symbol that represents the trace" The face symbol after a freeze: ``` :place group authored source→stage mapping (image heights → stage px) :head group measured M(p) :reads — see below :mouth … parts generated, every frame :plate instance of :footage-8625 ← the trace measured channels: fit(p) = M(p)⁻¹ · S(1/imageH) :time {:holds [0 12 30]} ← the trace keys ``` ### Registration comes from the parent The plate is a child of `:head`. Its own measured channels are the **stabilizing fit** at frame p: the inverse of the head's measured transform, with pixels → image heights folded into the scale (it stays a similarity, so it decomposes into `pos`/`rot`/`scale`). Freeze already computes the fit; the head's measured channels are its inverse (`freeze/invert`). The plate gets a second dense block, which is three small channels. Because holds are a **time** floor, the plate's channels and the frame its content shows are read at the **same** held frame q. Its world transform is: ``` world(plate) = place · M(p_head) · M(q)⁻¹ · S(1/H) ``` That is exactly `trace/photo-matrix`, but now it falls out of the ordinary walk. It is registered to whatever the head is doing, by construction: | head reads (p_head) | plate shows (q) | result | | --- | --- | --- | | f (continuous) | f (no holds) | footage where filmed | | f (continuous) | held key | held photo rides the moving head (today's behaviour) | | held key (same as plate) | held key | `M(q)·M(q)⁻¹ = I`: photo sits where filmed | | 0 (start) | f or held | stabilized footage under a still head | If a frame has no measurement, the plate's dense channel has nothing there, so `xform-at` gives nil, so the plate is not placed and no photo shows. That is what happens today too. ### Trace keys versus origin: who owns what The two decisions are separate and stay separate: - **Trace keys** are which footage frames get drawn over (the "plate drawings" in `frame-selection.md`). They belong to the **plate**, as `:time :holds`. A hand-placed tracing layer with holds is the *same thing*: the face's plate is an ordinary tracing placement and nothing more. - **Origin** is how the head moves between kept frames. `frame-selection.md` already says `:origin` "is not a tracing setting" but a performance one, so it belongs to the **head**: ```clojure :head {…} ; continuous — reads its own frame :head {… :reads {:holds [0]}} ; start :head {… :reads {:holds-of :plate}} ; at keys — reads its measured channels at ; the frames :plate's holds select ``` `{:holds [...]}` is also what a face with no footage uses for "at keys", since it has no plate to follow. `:reads` changes only the **head's own channel reads**, not its children's frames. The parts must keep running every frame, which is why this cannot be a `:time` hold on the head. It is today's `traces` branch of `base-channel-frame` with the hold list read from the named node. A node reference has precedent (`:stencil`, `:pose-group`). It points from the follower to the thing followed, so there is still one stored list of frames and nothing to keep in sync. Validate it in `symbol/problems` the way `:stencil` is validated: the target exists in the symbol, and it is not the head itself or an ancestor of the head. Rejected alternatives, so they are not re-proposed: - **Keys on the head, and the plate reads them.** This is today's direction. It makes the plate special: a free tracing layer could not have keys that a face's plate also understands. - **Hold `:time` on `:head`.** Exposure inherits strictly, so the mouth and eyes would freeze along with the head. - **A wrapper "registered footage" symbol holding the fit.** It is correct but adds a symbol per face. The time-floor holds already put the fit read and the content read on the same frame, so the wrapper buys nothing. - **Plate as a sibling of `:head` at identity.** It is only registered when the head and the photo read the same frame, so it breaks the continuous-plus-holds and start rows above. ## Evaluation: an op that is never rendered - **`clip/resolver`**, in the instance branch: when the source symbol has `:type :trace`, it does not recurse. If `(:tracing? opts)` is set, it emits one op: ```clojure {:kind :trace :node [id] :m :media … :frame shown-frame :size [w h]} ``` The frame comes from the same `placed-frame` path as any instance, so playback, holds and the fps cadence apply. Do not build a child resolver for a trace symbol. - **`transform-op`** gets a `:trace` case that composes the matrix. Row paths, solo filtering and nesting at any depth then work for free. - **The output guarantee is structural.** `:tracing?` defaults to false. Only the stage's `::render/resolver` passes true. Export, `clip/center` (a big photo must not pull a symbol's pivot), thumbnails and the bench never ask for trace ops. `raster/draw-ops!` keeps throwing on unknown kinds, so a leak fails loudly. - **`ui/player`** sends picture ops to the raster and `:trace` ops to the painter. - **`ui/underlay` becomes `ui/tracing`.** It paints `:trace` ops in draw order with `drawImage` at `op.m` and the global opacity. The media URL is the footage manifest's `urls[range-start + frame]`, or the image blob URL. The three steadiness fixes stay: LRU cache, hold the last still per op `:node`, and read ahead while playing. Everything that walked faces is deleted. - **`pick`**: a `:trace` op is hit when the point, mapped through `m⁻¹`, falls in `[0 w) × [0 h)`. Picture ops are tested first and trace ops only if nothing drawn is under the pointer, so a full-frame photo does not steal every click. When the global switch is off, traces are neither painted nor picked. - **Gestures**: no change. The face's plate is measured, so `gesture/refusal` already says "place the instance it is in". A hand-placed layer is authored and moves, turns and scales like anything else. ## On and off All of it is EDITOR STATE, as ed88c5e decided for the per-face switch: showing a reference is a way of looking at the stage, so it is not an undo step, does not travel to collaborators, and cannot reach an export. ```clojure [:ui :tracing {:on? true :opacity 0.5 :hidden #{[sid node-id] …}}] ``` - **One layer** is an entry in `:hidden`, keyed by the symbol the tracing instance is in and its node id. The symbol id is needed because every face's plate is called `:plate`. Keyed this way, hiding a face's plate hides it in every placement of that face, which is what the per-face switch did. Shown is the default, so opening a face shows its footage with no setup. - **How it's applied**: `clip/resolver` already knows which symbol and node a trace op comes from, so the op carries `:layer [sid id]`. The painter and `pick` skip ops whose layer is hidden. The resolver does not change when you toggle a layer, so nothing is rebuilt. - **Global**: `:on?` and `:opacity`, a toggle plus an opacity slider in `ui/palette/bar` next to the palette controls. When it is off, traces are neither painted nor picked. - **Switching one layer on also switches the global setting on.** This applies from the tracing instance's inspector, from its timeline row, and from the face/roto section. A layer you just enabled must not stay invisible behind a switch you forgot. Turning one layer off never touches the global switch. It is one event (`::ui/show-trace layer on?`) that updates `:hidden` and, when turning on, sets `:on? true` in the same handler, so the three places cannot behave differently. - **`[:vis]`** still works on a tracing instance like on any node: key it to show a reference only over part of the shot. It is not the on/off switch. - **Deleted**: `[:ui :trace :faces]`, `::trace-face`, `::trace-faces`, `trace/showing-for`, `traceable-faces`, `shown`, `faces`, and the "a take shows nothing by default" rule. ## UI - **Timeline.** A tracing instance is an ordinary row or clip block, styled to read as reference-only (hatched block, an eye icon in place of the colour chip). Hold frames are marks on the row. The row's eye button dispatches `::ui/show-trace`, which replaces the face row's `tl-trace` button. - **Inspector, tracing instance.** Show the media (footage name and range, or image), an on/off control (which goes through `::ui/show-trace`), and the hold list ("hold here" / "remove hold" plus seek buttons, which is `trace-keys` re-aimed at `:time :holds`). Playback, transform and span use the existing sections. - **Inspector, face/roto section.** The same hold controls, aimed at the face's `:plate`. Origin buttons write `:head :reads`. The section is found by "this face has a `:plate`", not by `trace/traceable?`. - **Palette bar.** The global toggle and opacity slider. - **Making one.** - Footage: the convert dialog gets a choice between "animate faces" (today's flow) and "tracing layer" (no detection). The tracing-layer choice makes the `:type :trace` symbol for the chosen range and places it where the video was dropped. - Footage can also be dragged from the pool with a modifier or as a second drag kind. - A still image: drop it on the pool and it uploads; drop it on the stage or timeline and it is placed with `:end :hold` and a span of the host's remaining frames. ## Freeze, bring, regenerate - **`freeze/subject-part`** emits the `:plate` instance under `:head`, with fit measured channels and `:time {:holds []}`, and emits one `:type :trace` symbol for the analysed footage range. `bring/take` already copies every symbol the take reaches and rewrites `:source :symbol` references, so the tracing symbol comes along. It gets the footage's `:audio` too, so a dropped tracing layer can bring its sound through the existing link. - **`head-mode`** writes `:reads` on the head and `:holds` on the plate, in place of `:trace`. The `frame-selection.md` plate proposal materializes into the plate's `:time :holds` in place of `:trace :frames`. - **Regenerate** replaces both measured blocks (head and plate) and keeps `:holds` and `:reads`. Check that `regenerate-head`'s measured/authored comparison handles a second measured node. ## Server - `Image` endpoints mirroring `Sound`: `POST /api/images` stores a `Blob` and returns `{id, width, height, url}`, and `GET /api/images` lists them. The migration is a model only. Bump `schema_version` and refuse older documents clearly. Do not convert them. ## Deleted outright `trace/of`, `prepare`, `held-frame`, `problems`, `toggle-frame`, `photo-frame`, `measured-local`, `photo-matrix`, `traceable?`, `faces`, `traceable-faces`, `showing-for`, `shown`, `opacity-default`. `domain/trace.cljs` probably goes away entirely; if anything is left, it is a hold helper that belongs in `node`. Also deleted: `symbol/prepared-traces` and the `traces` arm of `base-channel-frame`, which becomes the `:reads` lookup. `:trace` on nodes (and `symbol/problems` reports it as removed). `::project/set-trace`. `::render/underlay` and `::render/tracing` become one `::render/tracing` that returns `[:ui :tracing]`. The face lookup in `params/view`. Expected effect on code size: net negative. One time-floor clause, one op kind, one resolver branch, one pick case and a `:reads` lookup replace the face walk, the hand-built photo matrix, per-face showing state and the underlay's face bookkeeping. Measure the change and report the number honestly (see the `cljs-style` memory). ## Tests **Domain (node).** - `:holds` in `local-frame`, with eval-frame and resolver agreeing forward, backward and in random order. - A trace op appears only with `:tracing?`, and `export/run!` output contains no `:trace` op. - `transform-op` composes the matrix through two nesting levels. - `clip/center` ignores traces, and is `[w/2 h/2]` for a tracing symbol. - `pick` returns picture ops before traces and inverse-maps through a rotated layer. - `symbol/problems` covers `:type :trace`, `:reads` targets and cycles, and `:holds` shape. - Registration: port `trace_test`'s photo-matrix assertions onto the walk. For every row of the registration table, the plate's world transform equals the expected matrix. At a held key with `:reads {:holds-of :plate}`, it equals `world(:place) · S(1/H)`. - Freeze emits the plate and the tracing symbol. Regeneration keeps the holds. - The `::ui/show-trace` event sets the global switch on when turning a layer on and leaves it alone when turning one off. **Browser** (CDP, see the `arthur-verify-dont-guess` memory): - Drop a still, then drag, turn and scale it. - Toggle a layer on with the global switch off, and confirm both are on and the photo paints. - Export a frame and check it has no photo pixels. - Open a face: the plate is registered at a hold with origin "at keys", and stabilized with origin "start". ## Order of work, each step green 1. `:time :holds` in `node/local-frame`, `problems`, `frame-map`, and timeline marks. 2. `:type :trace` symbol, `:media`, the `:trace` op behind `:tracing?`, `transform-op`, the player split, `ui/tracing` painter and `pick`. Footage media only, placed by hand from a REPL or test document. 3. The face: freeze and bring emit the plate and tracing symbol, `:reads` replaces `:trace`, delete the old trace and underlay paths, and re-aim the inspector's face section. 4. On/off: the `:hidden` set and `:layer` on trace ops, the row eye, the `::ui/show-trace` rule, the global toggle and opacity in the palette bar, and delete `[:ui :trace :faces]`. 5. Creation: "tracing layer" in the convert dialog, pool drag, then the image endpoint, pool images and image drops. 6. Docs: `animation-model.md` "A photographic underlay is not an op" becomes "a trace is an op that never reaches the raster". Update `frame-selection.md` (where plate keys live) and `lane-model.md` (tracing clips in lanes). ## Open questions 1. **"A take shows its faces' footage."** Dropping the old "only in a face's own tab" default means opening a take shows every face's plate if the global switch is on. The recommendation is to accept that, since the switch is one click. 2. **Lane-model audio rule.** `lane-problems` requires visual cels. A tracing cel is visual for this purpose, but say so explicitly when a lane may hold both tracing and drawing cels.