a4ce750be2
The source-to-stage mapping moves off :main's :face group and onto each face's own :place, above its head. `face-placement` computes exactly what it computed before, over every subject together, so two faces filmed side by side keep their filmed relation — it is written into each face instead of onto a group above them all. Same transform, same subtree, one level lower, and the composite is identical to the pixel: a digest over every op :main emits across the whole take is unchanged either way. THE OWNER IS THE POINT. A face carrying its own mapping is the right size wherever it is put — dropped into another symbol, or opened in its own tab to be drawn over — and the take that holds it needs to know nothing. On a group above the instances the scale belonged to the take, so a face taken out of it had no size at all and drew at a fraction of a pixel. The pool's thumbnails drop the workaround that knew about this: a symbol is rendered rooted at itself again, because a face now carries the placement that makes that honest, so the pool needs to know nothing about where a symbol happens to be used. `domain/node` and `arthur.export` leave its requires with it. The tests here were reading the placement off :main. The photo registration test changes shape rather than location: its premise was that face-1's head is its own root, so a photo sitting where it was filmed was image pixels over image height and nothing else. The head still cancels — that is what the test is about — but it now cancels against the face's own placement, which is why the photo comes with the face into its own tab instead of sitting at a fraction of a pixel beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
91 lines
4.6 KiB
Markdown
91 lines
4.6 KiB
Markdown
# Timing model
|
|
|
|
[The Lane Model](lane-model.md) defines the revised target for occurrence timing,
|
|
source playback, sampling scope, and inverse editing. It supersedes conflicting
|
|
proposals here; the sections below describe earlier implementation decisions.
|
|
|
|
The source footage, authored drawings, generated face motion, and stage placement
|
|
have different frame decisions. They share a clock but do not share one kept-frame
|
|
list. `timing-handoff.md` records earlier implementation notes.
|
|
|
|
## Frame spaces
|
|
|
|
- A source frame addresses a decoded image and its measured face data. Keep the
|
|
source cadence and, for variable-rate video, its presentation timestamp.
|
|
- A timeline frame addresses authored keys in the clip or symbol's local space.
|
|
- A stage frame is mapped through the symbol instance's offset and rate before
|
|
local frame decisions are read. Moving a placement does not rewrite its keys.
|
|
|
|
The analyzed source poses remain dense. A lower picture rate or a skipped pose
|
|
never removes source data or shortens audio.
|
|
|
|
## Head placement
|
|
|
|
Analysis fits each source frame's rigid landmarks into one common head-local
|
|
space. Its inverse is the measured head transform, stored densely on `:head`.
|
|
The head node has one optional anchor map:
|
|
|
|
```clojure
|
|
;; no :anchors free movement: read measured frame f at f
|
|
:anchors {0 12} ; one lock: use frame 12's transform throughout
|
|
:anchors {0 12, 40 42} ; keyed locks: switch to frame 42 at local frame 40
|
|
```
|
|
|
|
A key is `(local change frame -> measured source frame)`. Its value holds to the
|
|
next key. The map chooses position, rotation and scale together. Frame zero must
|
|
have a key when the map exists. The dense transform blocks remain intact, so
|
|
editing anchors is a small document change and re-freezing can replace the
|
|
measurements without losing the anchor choices.
|
|
|
|
A source image used for tracing should be registered with that image's measured
|
|
stabilizing transform, then the selected head transform, then the authored
|
|
`:place` placement the face carries. This makes the photo and head-local vectors share the same
|
|
orientation and position. Tracing-photo selection is a separate editor address;
|
|
it does not choose the head anchor.
|
|
|
|
The prototype stabilizes into the shot's mean rigid pose and uses an early
|
|
closed-mouth frame for raster framing. Those are internal analysis and framing
|
|
choices. The authored head-anchor map above controls which measured head pose is
|
|
shown over each range. It is independent of plate drawing starts.
|
|
|
|
## Performance poses
|
|
|
|
Generated mouth, eye, and brow channels can be sampled at a lower picture rate
|
|
without retiming authored keys. The normal rule picks the latest available pose
|
|
at or before a picture-grid time. A future performance policy may add important
|
|
closed-mouth poses and store manual keeps/drops separately from the rate's
|
|
proposal. Related parts should share a selected pose by default: a mouth outline,
|
|
interior, teeth and generated visibility must not disagree about its frame.
|
|
|
|
## Stage placement
|
|
|
|
A symbol placement has optional pose-cut tracks, separate from head anchors:
|
|
|
|
```clojure
|
|
:playback {:tracks {:mouth {0 12, 8 27}
|
|
:eye-r {0 0, 4 6}}}
|
|
```
|
|
|
|
These maps are also `(local change frame -> source pose frame)`. They select which
|
|
baked/generated shape pose appears on that placement. Before the first explicit
|
|
cut, normal generated motion continues. Cuts hold, without interpolation, until
|
|
the next cut. A `[:node id]` track can override one shape in a shared group.
|
|
Authored cels, transforms, and audio remain on their normal local time.
|
|
|
|
The current implementation reads retained frozen channels. A separate resolved
|
|
geometry bake is not implemented; when added, it must preserve addressable
|
|
candidate poses so stage cuts can still select any of them.
|
|
|
|
## Ownership
|
|
|
|
| Choice | Owner | Current state |
|
|
| --- | --- | --- |
|
|
| Source frames and timestamps | Footage/analysis | Constant-rate frame indexing exists; variable timestamps remain future work |
|
|
| Head anchor map | `:head` node | Implemented, stored with the node |
|
|
| Trace frames (photo address) and origin | Face symbol's `:head` `:trace`, written through `:anchors` | Implemented, see `domain/trace` |
|
|
| Showing the tracing photo, and its opacity | Face instance's `:underlay` | Implemented; a drawing aid, not keyed |
|
|
| Generated picture-rate proposal and closure protection | Roto clip/symbol | Generated-only picture sampling exists; closure protection remains future work |
|
|
| Stage pose cuts | Symbol instance | Implemented, stored with the instance |
|
|
|
|
Preview and export use the same resolver for generated picture sampling and stage
|
|
cuts. Export still emits every timeline frame at the clip's audio rate.
|