arthur/docs/timing-handoff.md

# Timing and frame-selection handoff

Status (2026-09-29): the multi-face representation is complete. Each face has a
local drawing timeline and an ordinary symbol instance. Keep that model; the next
feature is performance-pose selection, followed by plate drawings and tracing.
See [multi-face representation](multi-face-representation.md) for verification
and compatibility limits.

## Next steps, in order

1. **Commit the verified checkpoint.** Representation, scoped regeneration,
   nested stage composition and source persistence are implemented and tested.
2. **Exercise real two-person footage.** Include crossings, late arrivals and
   disappearances. Assignment is still a nearest-centroid heuristic; inspect
   whether identities, landmarks and mouth crops stay together. Correcting an
   assignment requires measuring again. Do not redesign the representation to
   compensate for an assignment failure.
3. **Build performance-pose selection.** Propose frames from a target picture
   rate, allow explicit Keep/Drop edits, and apply requests per instance. Reuse
   the existing held-frame lookup and generated pose groups. Keep authored keys
   and audio timing intact.
4. **Then build plate drawings and tracing.** Suggest drawing frames from head
   displacement, allow manual choices, and give each cel an independently
   selectable tracing reference.

Older flat captures need reanalysis for the new regeneration path. Migrating their
existing authored edits is separate work; it is not implemented by this checkpoint.

## Timing decisions

Keep the dense analyzed frames. Generated motion holds the most recent selected
source pose; removing a selected pose never deletes source data or shortens the
clip. Store edits in the animation's local frame space, so moving an instance
does not move its edits. Authored keys follow intentional instance retiming but
must not be quantized by a picture-rate request. Clip FPS and audio duration stay
fixed.

There are two selections with different owners, sharing held-frame lookup:

- **Performance poses:** propose a kept-frame list from the target picture rate,
  then apply explicit keep/drop edits. A parent instance may request a lower
  rate. Mouth outline, interior, teeth and visibility read the same selected
  source frame; likewise each eye's coupled parts. Use group overrides when
  needed, rather than a setting on every channel. Head motion currently has its
  own anchor selection; do not silently put it under mouth timing.
- **Plate drawings:** start with frame 0, walk measured rigid head poses, and
  suggest a frame when maximum landmark displacement from the last kept pose
  exceeds tolerance. Let the artist add/remove frames. A removed drawing stays
  stored so it can reappear if restored. This selection does not thin the mouth.

A target rate is approximate. Pin a useful closed-mouth pose at its actual frame,
even if that produces more changes than the target. Do not show a future pose
early to fit a grid. Manual drop wins over an automatic suggestion; make removal
of the only closed pose in a beat visible in the UI. Skip missing detections when
suggesting a replacement. Keep a frame-zero selection and hold the last selection
through the end. A skipped pose (hold), `[:vis] false` (hidden), and an absent
measurement remain different facts.

Store manual edits separately from generated proposals so changing the rate or
tolerance retains hand decisions. Selection edits change the document, not dense
blocks or analysis addresses. Verify save/open for every new field; extend leaf
handling and the relevant key whitelist if its storage location requires it.

## Current code: reuse these mechanisms

- `domain/pose.cljs` already has `prepare`, `held-frame` and `source-frame`.
  Instance `:playback :tracks` map local change frames to held source frames,
  keyed by pose group. Reuse this lookup; frame suggestion and Keep/Drop policy
  are the missing layer. An explicit cut is not itself a complete selection UI.
- `freeze/performance-nodes` marks generated animated channels with
  `:pose-sampled?` and local `:pose-group` names. This includes keyed visibility
  as well as dense geometry. `:generated` remains provenance for regeneration.
- `timeline/channel-frame` already applies explicit pose choices and default
  picture sampling to marked channels. Playback and export both use
  `clip/resolver` with `:picture-fps`; there is no need for a second sampling
  implementation. Export's pose count is still a rate-based estimate.
- The picture-rate option is currently passed through the resolver tree
  unchanged. Instance-specific parent requests are still to be implemented.
  Instance offset/rate must apply before selecting the local source pose.
- `pose/put-cut` and `remove-cut` currently address instances in `:main`.
  A take's face instances are there, but a composed stage nests them inside a
  shared source timeline. Make the editing scope explicit when adding nested
  controls. A request on one outer placement must not rewrite the shared
  drawing's playback settings for every placement.
- Generic root `:time :expose` still retimes descendants, and frozen takes still
  store it. Paint nodes are rootless to escape it. When the selection path
  replaces take picture cadence, remove that redundant quantization from the
  take default; preserve intentional generic time maps. Moving exposure to
  `:head` would still retime authored children.
- `freeze/head-mode` supports `:free` and `:anchored`. It keeps measured channels
  dense and writes optional per-subject `:anchors` maps; it does **not** implement
  `:per-plate` mode or materialize transform keys from `:kept`. Plate selection
  should reuse held measured-frame addresses where appropriate, without
  rerunning analysis or copying the measurements.
- `:over` hand corrections are currently refused by the channel reader. Their
  future application belongs after generated pose selection.

## Performance-pose implementation sequence

1. Add pure proposal and Keep/Drop policy around the existing held-frame lookup.
   Cover frame zero, nondivisible rates, manual precedence, missing poses and a
   protected mouth closure. Preserve all source frames.
2. Feed instance requests and group selections into the existing channel read
   path. Cover two faces, two differently timed placements of one source, nested
   instances, coupled visibility/geometry, and authored keys at their normal time.
   Use this same path for preview and export; keep audio duration unchanged.
3. Wire the performance strip's Suggest/Keep/Drop controls and persistence.
   Replace the export pose estimate with the actual selection count. Retire the
   take's redundant root exposure only when this path replaces its behavior.

## Plate drawings and tracing, afterward

The old suggestion algorithm is `js/pipeline.js:suggestPlateFrames`; the strip,
worksheet and tracing photo are in `js/app.js`. Port the useful policy over the
measured head poses and reuse held-frame lookup for the resulting drawing set.

Give a cel an editor-only source-frame reference, defaulting to its plate frame
but independently changeable. It may point to a frame omitted from either rendered
selection. Register the photo using that source frame's measured transform. The
old prototype coupled photo and cel addresses; independent tracing is new work.

The old iris socket lock, gaze origin, CLJS head anchors and registration pivot
are separate settings. Clarify what an "origin-lock" request means before adding
that control.

Keep the UI to two scopes: **performance poses** and **plate drawings**, each with
Suggest/Keep/Drop. Tracing reference and lock controls live with the cel or feature
they affect. No general keyframe framework is needed for this work.