Model head anchors and independent pose timing
This commit is contained in:
parent
a45e89f4e4
commit
49ece8dee6
17 changed files with 521 additions and 215 deletions
86
docs/timing-model.md
Normal file
86
docs/timing-model.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
# Timing model
|
||||
|
||||
The source footage, authored drawings, generated face motion, and stage placement
|
||||
have different frame decisions. They share a clock but do not share one kept-frame
|
||||
list. `timing-handoff.md` records earlier implementation notes.
|
||||
|
||||
## Frame spaces
|
||||
|
||||
- A source frame addresses a decoded image and its measured face data. Keep the
|
||||
source cadence and, for variable-rate video, its presentation timestamp.
|
||||
- A timeline frame addresses authored keys in the clip or symbol's local space.
|
||||
- A stage frame is mapped through the symbol instance's offset and rate before
|
||||
local frame decisions are read. Moving a placement does not rewrite its keys.
|
||||
|
||||
The analyzed source poses remain dense. A lower picture rate or a skipped pose
|
||||
never removes source data or shortens audio.
|
||||
|
||||
## Head placement
|
||||
|
||||
Analysis fits each source frame's rigid landmarks into one common head-local
|
||||
space. Its inverse is the measured head transform, stored densely on `:head`.
|
||||
The head node has one optional anchor map:
|
||||
|
||||
```clojure
|
||||
;; no :anchors free movement: read measured frame f at f
|
||||
:anchors {0 12} ; one lock: use frame 12's transform throughout
|
||||
:anchors {0 12, 40 42} ; keyed locks: switch to frame 42 at local frame 40
|
||||
```
|
||||
|
||||
A key is `(local change frame -> measured source frame)`. Its value holds to the
|
||||
next key. The map chooses position, rotation and scale together. Frame zero must
|
||||
have a key when the map exists. The dense transform blocks remain intact, so
|
||||
editing anchors is a small document change and re-freezing can replace the
|
||||
measurements without losing the anchor choices.
|
||||
|
||||
A source image used for tracing should be registered with that image's measured
|
||||
stabilizing transform, then the selected head transform, then the authored
|
||||
`:face` placement. This makes the photo and head-local vectors share the same
|
||||
orientation and position. Tracing-photo selection is a separate editor address;
|
||||
it does not choose the head anchor.
|
||||
|
||||
The prototype stabilizes into the shot's mean rigid pose and uses an early
|
||||
closed-mouth frame for raster framing. Those are internal analysis and framing
|
||||
choices. The authored head-anchor map above controls which measured head pose is
|
||||
shown over each range. It is independent of plate drawing starts.
|
||||
|
||||
## Performance poses
|
||||
|
||||
Generated mouth, eye, and brow channels can be sampled at a lower picture rate
|
||||
without retiming authored keys. The normal rule picks the latest available pose
|
||||
at or before a picture-grid time. A future performance policy may add important
|
||||
closed-mouth poses and store manual keeps/drops separately from the rate's
|
||||
proposal. Related parts should share a selected pose by default: a mouth outline,
|
||||
interior, teeth and generated visibility must not disagree about its frame.
|
||||
|
||||
## Stage placement
|
||||
|
||||
A symbol placement has optional pose-cut tracks, separate from head anchors:
|
||||
|
||||
```clojure
|
||||
:playback {:tracks {:mouth {0 12, 8 27}
|
||||
:eye-r {0 0, 4 6}}}
|
||||
```
|
||||
|
||||
These maps are also `(local change frame -> source pose frame)`. They select which
|
||||
baked/generated shape pose appears on that placement. Before the first explicit
|
||||
cut, normal generated motion continues. Cuts hold, without interpolation, until
|
||||
the next cut. A `[:node id]` track can override one shape in a shared group.
|
||||
Authored cels, transforms, and audio remain on their normal local time.
|
||||
|
||||
The current implementation reads retained frozen channels. A separate resolved
|
||||
geometry bake is not implemented; when added, it must preserve addressable
|
||||
candidate poses so stage cuts can still select any of them.
|
||||
|
||||
## Ownership
|
||||
|
||||
| Choice | Owner | Current state |
|
||||
| --- | --- | --- |
|
||||
| Source frames and timestamps | Footage/analysis | Constant-rate frame indexing exists; variable timestamps remain future work |
|
||||
| Head anchor map | `:head` node | Implemented, stored with the node |
|
||||
| Tracing cel starts and photo address | Authored cel | Separate future work |
|
||||
| Generated picture-rate proposal and closure protection | Roto clip/symbol | Generated-only picture sampling exists; closure protection remains future work |
|
||||
| Stage pose cuts | Symbol instance | Implemented, stored with the instance |
|
||||
|
||||
Preview and export use the same resolver for generated picture sampling and stage
|
||||
cuts. Export still emits every timeline frame at the clip's audio rate.
|
||||
Loading…
Add table
Add a link
Reference in a new issue