arthur/docs/tracing-symbol-plan.md

# Plan: tracing is a symbol

Status: built, 2026-10-03, on branch `worktree-tracing-layers`. Where the build
differs from the plan below, the build wins:

- The head's field is `:reads` (`{:holds [...]}` or `{:holds-of :plate}`), not
  `:follow`.
- A still is not one frame. It gets as many frames as remain in the symbol it is
  dropped into, at that symbol's rate, and is trimmed like any clip.
- An image's `:media` is `{:image <blob sha256>}`, served at `/blob/<sha>`. The
  server's `Image` row exists for the pool's list and labels, not for identity.
- Dropping media that a tracing symbol in the document already shows reuses that
  symbol.
- Nothing can be created or dropped inside a tracing symbol (`creation/target`,
  `drop-destination-at`, `nest/move-refusal`), and one cannot be opened in a tab.
- Schema 6. Every project is marked 6. One that still carries `:trace` on a
  node is refused when opened, with what to do about it, rather than all old
  projects being refused.

No backward compatibility (see `lane-model.md`, "Goal and compatibility policy").

## What is wrong today

Tracing is not a thing in the document. It is three mechanisms that each know
about faces:

- **`:trace {:frames :origin}` on a face's `:head`.** It decides which measured
  frame the head reads (`symbol/base-channel-frame` → `trace/held-frame`), and
  the underlay also reads it to decide which photo to show (`trace/photo-frame`).
- **`ui/underlay`**, a painter that walks `trace/shown` → `trace/faces` (a
  separate instance walk), looks up each face's subject → analysis → footage,
  asks the resolver where `[...path :head]` went, and builds the photo's matrix
  by hand: `world(head) · M(p)⁻¹ · 1/imageH` (`trace/photo-matrix`).
- **`[:ui :trace {:faces #{} :opacity}]`**, a per-face switch in editor state,
  plus the special "opening a face shows its footage, opening a take does not"
  rule (`trace/showing-for`).

What you cannot do: place footage or a still as a reference where you like, move
it, scale it, turn it, trim it, hold it, put it in a lane, or trace something
that is not a tracked face. The photo is not selectable and has no row. The one
case that works (a face over its own footage) runs on code that nothing else
uses.

## The model in one paragraph

A **tracing symbol** is a symbol with `:type :trace`. It has no nodes. It names
media (a footage range or a still image) and has that media's frame count, fps and
pixel size. It is **placed by an ordinary instance**, so lanes, spans, trim,
split, move, playback (`:in :speed :end`), transform, nesting, selection, picking
and gestures all work on it with no new code. It is evaluated to a single
**`:trace` op**, which never reaches the raster or an export. A face's footage is
not a special case. It is one of these instances, placed inside the face as a
child of `:head` and carrying the measured registration transform. The trace keys
become **holds on that instance's time**. The head's "origin" becomes the head
saying **which node's frames it follows**.

This follows the precedent of `:type :palette` symbols (no nodes, they name an
asset, and they are placed by instances in a lane), so the uniformity rule holds
(see the `model-uniformity` memory). Nothing new holds nodes, and no special
instance kind is added.

## Data model

### The tracing symbol

```clojure
:footage-8625                                 ; a symbol id like any other
{:id :footage-8625 :name "8625.mov"
 :type :trace
 :media {:footage #uuid "f8ca…" :range [12 241]}   ; or {:image "sha256:…"}
 :frames 229          ; the range's length; 1 for a still
 :fps 30              ; the footage's own rate, so cadence handles 30→12 for free
 :width 1440 :height 1920   ; pixel size = the symbol's own stage
 :audio {:footage #uuid "f8ca…"}   ; optional, the existing "a symbol says what it sounds like"
 :nodes {}}
```

- `:media` is the one new symbol key. Add it to `symbol/symbol-keys` and to the
  symbol leaf's `select-keys` in `leaf/leaves`.
- The symbol's local space is **pixels of the media**: `[0 w) × [0 h)`.
  `clip/center` of a node-less symbol already returns its stage middle, which here
  is `[w/2 h/2]`. So `place-symbol` puts the anchor at the image centre with no
  new code.
- `symbol/problems`: `:type :trace` needs `:media` with exactly one of
  `:footage`/`:image`, needs `(empty? nodes)`, and for footage needs
  `(= frames (- end start))`.
- **One tracing symbol per footage range, shared.** Two faces from one take
  place the same symbol, and so does a hand-placed reference. Like any symbol it
  is reused by reference, and `bring/symbols` copies it like any other.

### Placing it

It is an ordinary `:kind :instance` with `:source {:symbol :footage-8625}`:

- **Start and end** are `:span` (in its own frames) and `:time :at`. You get
  these from the existing trim, split, move and roll.
- **Which frame shows** comes from `:playback {:in :speed :end}`. Footage plays
  with speed 1, a frozen frame has speed 0, and a still is 1 frame with `:end :hold`.
- **Placement in space** is `[:xform …]`, the same as every node: gestures,
  inspector and keys.
- **In a lane, or on its own**: it is a parent-less spanned node, so a `:display
  :lane` symbol draws it as a block like any cel. It can also sit as a free node
  or under a group. To have a reference with its own tab, wrap it in an ordinary
  symbol.

**Default on creation** (in the drop event, not in `place-symbol`): scale so the
image's height fits the stage height, centred on the drop point. This is a
creation default like "use center anchor when dropping", and nothing updates it
afterwards.

### Holds: the one new time feature

`:time {:holds [0 12 30]}` floors a node's local frame to the last hold at or
before it. Before the first hold, the first hold applies. This is `:expose`
generalized from a regular grid to authored frames. It is applied in
`node/local-frame` at the same point as `:expose`, and is inherited in the same
way. It is in the node's **own** frames. On a tracing instance with
`:in 0 :speed 1`, own frames are source frames, so the hold list is the set of
traced frames.

It is general on purpose. Holding a playing symbol on chosen drawings is the
same feature. It costs about three lines in `local-frame`, one `problems` clause
(sorted, distinct, finite), and the timeline drawing hold frames as marks on the
row.

`symbol/frame-map` refuses floors (`:expose > 1`). It must also refuse a
non-empty `:holds`, for the same reason.

## The face: "a child symbol that represents the trace"

The face symbol after a freeze:

```
:place    group     authored source→stage mapping (image heights → stage px)
  :head   group     measured M(p)                      :reads   — see below
    :mouth … parts  generated, every frame
    :plate instance  of :footage-8625                   ← the trace
                     measured channels: fit(p) = M(p)⁻¹ · S(1/imageH)
                     :time {:holds [0 12 30]}           ← the trace keys
```

### Registration comes from the parent

The plate is a child of `:head`. Its own measured channels are the
**stabilizing fit** at frame p: the inverse of the head's measured transform,
with pixels → image heights folded into the scale (it stays a similarity, so it
decomposes into `pos`/`rot`/`scale`). Freeze already computes the fit; the
head's measured channels are its inverse (`freeze/invert`). The plate gets a
second dense block, which is three small channels.

Because holds are a **time** floor, the plate's channels and the frame its
content shows are read at the **same** held frame q. Its world transform is:

```
world(plate) = place · M(p_head) · M(q)⁻¹ · S(1/H)
```

That is exactly `trace/photo-matrix`, but now it falls out of the ordinary walk.
It is registered to whatever the head is doing, by construction:

| head reads (p_head) | plate shows (q) | result |
| --- | --- | --- |
| f (continuous) | f (no holds) | footage where filmed |
| f (continuous) | held key | held photo rides the moving head (today's behaviour) |
| held key (same as plate) | held key | `M(q)·M(q)⁻¹ = I`: photo sits where filmed |
| 0 (start) | f or held | stabilized footage under a still head |

If a frame has no measurement, the plate's dense channel has nothing there, so
`xform-at` gives nil, so the plate is not placed and no photo shows. That is what
happens today too.

### Trace keys versus origin: who owns what

The two decisions are separate and stay separate:

- **Trace keys** are which footage frames get drawn over (the "plate drawings"
  in `frame-selection.md`). They belong to the **plate**, as `:time :holds`. A
  hand-placed tracing layer with holds is the *same thing*: the face's plate is
  an ordinary tracing placement and nothing more.
- **Origin** is how the head moves between kept frames. `frame-selection.md`
  already says `:origin` "is not a tracing setting" but a performance one, so it
  belongs to the **head**:

```clojure
:head {…}                            ; continuous — reads its own frame
:head {… :reads {:holds [0]}}        ; start
:head {… :reads {:holds-of :plate}}  ; at keys — reads its measured channels at
                                     ; the frames :plate's holds select
```

`{:holds [...]}` is also what a face with no footage uses for "at keys", since it
has no plate to follow.

`:reads` changes only the **head's own channel reads**, not its children's
frames. The parts must keep running every frame, which is why this cannot be a
`:time` hold on the head. It is today's `traces` branch of
`base-channel-frame` with the hold list read from the named node. A node
reference has precedent (`:stencil`, `:pose-group`). It points from the follower
to the thing followed, so there is still one stored list of frames and nothing to
keep in sync. Validate it in `symbol/problems` the way `:stencil` is validated:
the target exists in the symbol, and it is not the head itself or an ancestor of
the head.

Rejected alternatives, so they are not re-proposed:

- **Keys on the head, and the plate reads them.** This is today's direction. It
  makes the plate special: a free tracing layer could not have keys that a face's
  plate also understands.
- **Hold `:time` on `:head`.** Exposure inherits strictly, so the mouth and eyes
  would freeze along with the head.
- **A wrapper "registered footage" symbol holding the fit.** It is correct but
  adds a symbol per face. The time-floor holds already put the fit read and the
  content read on the same frame, so the wrapper buys nothing.
- **Plate as a sibling of `:head` at identity.** It is only registered when the
  head and the photo read the same frame, so it breaks the continuous-plus-holds
  and start rows above.

## Evaluation: an op that is never rendered

- **`clip/resolver`**, in the instance branch: when the source symbol has
  `:type :trace`, it does not recurse. If `(:tracing? opts)` is set, it emits one op:

  ```clojure
  {:kind :trace :node [id] :m <copy of world> :media … :frame shown-frame :size [w h]}
  ```

  The frame comes from the same `placed-frame` path as any instance, so playback,
  holds and the fps cadence apply. Do not build a child resolver for a trace
  symbol.
- **`transform-op`** gets a `:trace` case that composes the matrix. Row paths,
  solo filtering and nesting at any depth then work for free.
- **The output guarantee is structural.** `:tracing?` defaults to false. Only the
  stage's `::render/resolver` passes true. Export, `clip/center` (a big photo
  must not pull a symbol's pivot), thumbnails and the bench never ask for trace
  ops. `raster/draw-ops!` keeps throwing on unknown kinds, so a leak fails loudly.
- **`ui/player`** sends picture ops to the raster and `:trace` ops to the
  painter.
- **`ui/underlay` becomes `ui/tracing`.** It paints `:trace` ops in draw order
  with `drawImage` at `op.m` and the global opacity. The media URL is the footage
  manifest's `urls[range-start + frame]`, or the image blob URL. The three
  steadiness fixes stay: LRU cache, hold the last still per op `:node`, and read
  ahead while playing. Everything that walked faces is deleted.
- **`pick`**: a `:trace` op is hit when the point, mapped through `m⁻¹`, falls in
  `[0 w) × [0 h)`. Picture ops are tested first and trace ops only if nothing
  drawn is under the pointer, so a full-frame photo does not steal every click.
  When the global switch is off, traces are neither painted nor picked.
- **Gestures**: no change. The face's plate is measured, so `gesture/refusal`
  already says "place the instance it is in". A hand-placed layer is authored and
  moves, turns and scales like anything else.

## On and off

All of it is EDITOR STATE, as ed88c5e decided for the per-face switch: showing
a reference is a way of looking at the stage, so it is not an undo step, does not
travel to collaborators, and cannot reach an export.

```clojure
[:ui :tracing {:on? true :opacity 0.5 :hidden #{[sid node-id] …}}]
```

- **One layer** is an entry in `:hidden`, keyed by the symbol the tracing
  instance is in and its node id. The symbol id is needed because every face's
  plate is called `:plate`. Keyed this way, hiding a face's plate hides it in
  every placement of that face, which is what the per-face switch did. Shown is
  the default, so opening a face shows its footage with no setup.
- **How it's applied**: `clip/resolver` already knows which symbol and node a
  trace op comes from, so the op carries `:layer [sid id]`. The painter and
  `pick` skip ops whose layer is hidden. The resolver does not change when you
  toggle a layer, so nothing is rebuilt.
- **Global**: `:on?` and `:opacity`, a toggle plus an opacity slider in
  `ui/palette/bar` next to the palette controls. When it is off, traces are
  neither painted nor picked.
- **Switching one layer on also switches the global setting on.** This applies
  from the tracing instance's inspector, from its timeline row, and from the
  face/roto section. A layer you just enabled must not stay invisible behind a
  switch you forgot. Turning one layer off never touches the global switch. It is
  one event (`::ui/show-trace layer on?`) that updates `:hidden` and, when
  turning on, sets `:on? true` in the same handler, so the three places cannot
  behave differently.
- **`[:vis]`** still works on a tracing instance like on any node: key it to
  show a reference only over part of the shot. It is not the on/off switch.
- **Deleted**: `[:ui :trace :faces]`, `::trace-face`, `::trace-faces`,
  `trace/showing-for`, `traceable-faces`, `shown`, `faces`, and the "a take shows
  nothing by default" rule.

## UI

- **Timeline.** A tracing instance is an ordinary row or clip block, styled to
  read as reference-only (hatched block, an eye icon in place of the colour
  chip). Hold frames are marks on the row. The row's eye button dispatches
  `::ui/show-trace`, which replaces the face row's `tl-trace` button.
- **Inspector, tracing instance.** Show the media (footage name and range, or
  image), an on/off control (which goes through `::ui/show-trace`), and the hold
  list ("hold here" / "remove hold" plus seek buttons, which is `trace-keys`
  re-aimed at `:time :holds`). Playback, transform and span use the existing
  sections.
- **Inspector, face/roto section.** The same hold controls, aimed at the face's
  `:plate`. Origin buttons write `:head :reads`. The section is found by "this
  face has a `:plate`", not by `trace/traceable?`.
- **Palette bar.** The global toggle and opacity slider.
- **Making one.**
  - Footage: the convert dialog gets a choice between "animate faces" (today's
    flow) and "tracing layer" (no detection). The tracing-layer choice makes the
    `:type :trace` symbol for the chosen range and places it where the video was
    dropped.
  - Footage can also be dragged from the pool with a modifier or as a second
    drag kind.
  - A still image: drop it on the pool and it uploads; drop it on the stage or
    timeline and it is placed with `:end :hold` and a span of the host's
    remaining frames.

## Freeze, bring, regenerate

- **`freeze/subject-part`** emits the `:plate` instance under `:head`, with fit
  measured channels and `:time {:holds []}`, and emits one `:type :trace` symbol
  for the analysed footage range. `bring/take` already copies every symbol the
  take reaches and rewrites `:source :symbol` references, so the tracing symbol
  comes along. It gets the footage's `:audio` too, so a dropped tracing layer
  can bring its sound through the existing link.
- **`head-mode`** writes `:reads` on the head and `:holds` on the plate, in
  place of `:trace`. The `frame-selection.md` plate proposal materializes into the
  plate's `:time :holds` in place of `:trace :frames`.
- **Regenerate** replaces both measured blocks (head and plate) and keeps
  `:holds` and `:reads`. Check that `regenerate-head`'s measured/authored
  comparison handles a second measured node.

## Server

- `Image` endpoints mirroring `Sound`: `POST /api/images` stores a `Blob` and
  returns `{id, width, height, url}`, and `GET /api/images` lists them. The
  migration is a model only. Bump `schema_version` and refuse older documents
  clearly. Do not convert them.

## Deleted outright

`trace/of`, `prepare`, `held-frame`, `problems`, `toggle-frame`, `photo-frame`,
`measured-local`, `photo-matrix`, `traceable?`, `faces`, `traceable-faces`,
`showing-for`, `shown`, `opacity-default`. `domain/trace.cljs` probably goes
away entirely; if anything is left, it is a hold helper that belongs in `node`.

Also deleted: `symbol/prepared-traces` and the `traces` arm of
`base-channel-frame`, which becomes the `:reads` lookup. `:trace` on nodes (and
`symbol/problems` reports it as removed). `::project/set-trace`.
`::render/underlay` and `::render/tracing` become one `::render/tracing` that
returns `[:ui :tracing]`. The face lookup in `params/view`.

Expected effect on code size: net negative. One time-floor clause, one op kind,
one resolver branch, one pick case and a `:reads` lookup replace the face walk,
the hand-built photo matrix, per-face showing state and the underlay's face
bookkeeping. Measure the change and report the number honestly (see the
`cljs-style` memory).

## Tests

**Domain (node).**

- `:holds` in `local-frame`, with eval-frame and resolver agreeing forward,
  backward and in random order.
- A trace op appears only with `:tracing?`, and `export/run!` output contains no
  `:trace` op.
- `transform-op` composes the matrix through two nesting levels.
- `clip/center` ignores traces, and is `[w/2 h/2]` for a tracing symbol.
- `pick` returns picture ops before traces and inverse-maps through a rotated
  layer.
- `symbol/problems` covers `:type :trace`, `:reads` targets and cycles, and
  `:holds` shape.
- Registration: port `trace_test`'s photo-matrix assertions onto the walk.
  For every row of the registration table, the plate's world transform equals
  the expected matrix. At a held key with `:reads {:holds-of :plate}`, it equals
  `world(:place) · S(1/H)`.
- Freeze emits the plate and the tracing symbol. Regeneration keeps the holds.
- The `::ui/show-trace` event sets the global switch on when turning a layer on
  and leaves it alone when turning one off.

**Browser** (CDP, see the `arthur-verify-dont-guess` memory):

- Drop a still, then drag, turn and scale it.
- Toggle a layer on with the global switch off, and confirm both are on and the
  photo paints.
- Export a frame and check it has no photo pixels.
- Open a face: the plate is registered at a hold with origin "at keys", and
  stabilized with origin "start".

## Order of work, each step green

1. `:time :holds` in `node/local-frame`, `problems`, `frame-map`, and timeline
   marks.
2. `:type :trace` symbol, `:media`, the `:trace` op behind `:tracing?`,
   `transform-op`, the player split, `ui/tracing` painter and `pick`. Footage
   media only, placed by hand from a REPL or test document.
3. The face: freeze and bring emit the plate and tracing symbol, `:reads`
   replaces `:trace`, delete the old trace and underlay paths, and re-aim the
   inspector's face section.
4. On/off: the `:hidden` set and `:layer` on trace ops, the row eye, the
   `::ui/show-trace` rule, the global toggle and opacity in the palette bar, and
   delete `[:ui :trace :faces]`.
5. Creation: "tracing layer" in the convert dialog, pool drag, then the image
   endpoint, pool images and image drops.
6. Docs: `animation-model.md` "A photographic underlay is not an op" becomes "a
   trace is an op that never reaches the raster". Update `frame-selection.md`
   (where plate keys live) and `lane-model.md` (tracing clips in lanes).

## Open questions

1. **"A take shows its faces' footage."** Dropping the old "only in a face's own
   tab" default means opening a take shows every face's plate if the global
   switch is on. The recommendation is to accept that, since the switch is one
   click.
2. **Lane-model audio rule.** `lane-problems` requires visual cels. A tracing cel
   is visual for this purpose, but say so explicitly when a lane may hold both
   tracing and drawing cels.