309c47e0a7
A face's :head carries :trace {:frames :origin}: the frames its photo holds
on, and whether the head reads every frame, jumps to the trace frames, or
holds frame 0. It replaces :anchors, so which measured frame a head reads is
one stored fact. An instance's :underlay shows the tracing stills over every
face at or below it, registered through each face's own head, at an opacity,
unkeyed. The clip resolver answers where a row path went on its last frame,
so the paint loop reads the photo's matrix instead of resolving again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
87 lines
4.4 KiB
Markdown
87 lines
4.4 KiB
Markdown
# Timing model
|
|
|
|
The source footage, authored drawings, generated face motion, and stage placement
|
|
have different frame decisions. They share a clock but do not share one kept-frame
|
|
list. `timing-handoff.md` records earlier implementation notes.
|
|
|
|
## Frame spaces
|
|
|
|
- A source frame addresses a decoded image and its measured face data. Keep the
|
|
source cadence and, for variable-rate video, its presentation timestamp.
|
|
- A timeline frame addresses authored keys in the clip or symbol's local space.
|
|
- A stage frame is mapped through the symbol instance's offset and rate before
|
|
local frame decisions are read. Moving a placement does not rewrite its keys.
|
|
|
|
The analyzed source poses remain dense. A lower picture rate or a skipped pose
|
|
never removes source data or shortens audio.
|
|
|
|
## Head placement
|
|
|
|
Analysis fits each source frame's rigid landmarks into one common head-local
|
|
space. Its inverse is the measured head transform, stored densely on `:head`.
|
|
The head node has one optional anchor map:
|
|
|
|
```clojure
|
|
;; no :anchors free movement: read measured frame f at f
|
|
:anchors {0 12} ; one lock: use frame 12's transform throughout
|
|
:anchors {0 12, 40 42} ; keyed locks: switch to frame 42 at local frame 40
|
|
```
|
|
|
|
A key is `(local change frame -> measured source frame)`. Its value holds to the
|
|
next key. The map chooses position, rotation and scale together. Frame zero must
|
|
have a key when the map exists. The dense transform blocks remain intact, so
|
|
editing anchors is a small document change and re-freezing can replace the
|
|
measurements without losing the anchor choices.
|
|
|
|
A source image used for tracing should be registered with that image's measured
|
|
stabilizing transform, then the selected head transform, then the authored
|
|
`:face` placement. This makes the photo and head-local vectors share the same
|
|
orientation and position. Tracing-photo selection is a separate editor address;
|
|
it does not choose the head anchor.
|
|
|
|
The prototype stabilizes into the shot's mean rigid pose and uses an early
|
|
closed-mouth frame for raster framing. Those are internal analysis and framing
|
|
choices. The authored head-anchor map above controls which measured head pose is
|
|
shown over each range. It is independent of plate drawing starts.
|
|
|
|
## Performance poses
|
|
|
|
Generated mouth, eye, and brow channels can be sampled at a lower picture rate
|
|
without retiming authored keys. The normal rule picks the latest available pose
|
|
at or before a picture-grid time. A future performance policy may add important
|
|
closed-mouth poses and store manual keeps/drops separately from the rate's
|
|
proposal. Related parts should share a selected pose by default: a mouth outline,
|
|
interior, teeth and generated visibility must not disagree about its frame.
|
|
|
|
## Stage placement
|
|
|
|
A symbol placement has optional pose-cut tracks, separate from head anchors:
|
|
|
|
```clojure
|
|
:playback {:tracks {:mouth {0 12, 8 27}
|
|
:eye-r {0 0, 4 6}}}
|
|
```
|
|
|
|
These maps are also `(local change frame -> source pose frame)`. They select which
|
|
baked/generated shape pose appears on that placement. Before the first explicit
|
|
cut, normal generated motion continues. Cuts hold, without interpolation, until
|
|
the next cut. A `[:node id]` track can override one shape in a shared group.
|
|
Authored cels, transforms, and audio remain on their normal local time.
|
|
|
|
The current implementation reads retained frozen channels. A separate resolved
|
|
geometry bake is not implemented; when added, it must preserve addressable
|
|
candidate poses so stage cuts can still select any of them.
|
|
|
|
## Ownership
|
|
|
|
| Choice | Owner | Current state |
|
|
| --- | --- | --- |
|
|
| Source frames and timestamps | Footage/analysis | Constant-rate frame indexing exists; variable timestamps remain future work |
|
|
| Head anchor map | `:head` node | Implemented, stored with the node |
|
|
| Trace frames (photo address) and origin | Face symbol's `:head` `:trace`, written through `:anchors` | Implemented, see `domain/trace` |
|
|
| Showing the tracing photo, and its opacity | Face instance's `:underlay` | Implemented; a drawing aid, not keyed |
|
|
| Generated picture-rate proposal and closure protection | Roto clip/symbol | Generated-only picture sampling exists; closure protection remains future work |
|
|
| Stage pose cuts | Symbol instance | Implemented, stored with the instance |
|
|
|
|
Preview and export use the same resolver for generated picture sampling and stage
|
|
cuts. Export still emits every timeline frame at the clip's audio rate.
|