Model head anchors and independent pose timing

This commit is contained in:
Olive Vaughn 2026-09-29 00:46:08 -04:00
parent a45e89f4e4
commit 49ece8dee6
17 changed files with 521 additions and 215 deletions

View file

@ -339,54 +339,47 @@ head is placed and scaled where it belongs and the rest of the frame is simply
not on stage. The full frame stays *available* for tracing without being not on stage. The full frame stays *available* for tracing without being
*visible*, and those are different requirements. *visible*, and those are different requirements.
## The anchor: stabilisation is a channel, not a mode ## Head motion: free or anchored to measured frames
`stabilize` produces `{s, θ, tx, ty}` per frame, which is exactly `stabilize` produces `{s, θ, tx, ty}` per source frame. Its inverse is stored
`[:xform :scale]`, `[:xform :rot]` and `[:xform :pos]`. So removing the head's densely on `:head`'s position, rotation and scale channels. The same measured
motion is not a pipeline setting — it is a question of **which node holds that track serves every placement choice:
motion**, and the answer is one channel definition:
```clojure ```clojure
;; locked: the head sits still, for tracing and for judging articulation ;; no :anchors — free: read the measured transform at the current frame
[:xform :pos] {:animated? false :value [0.0 0.0]} ;; one key — lock to a chosen measured frame throughout
:anchors {0 12}
;; as filmed: the head moves around the stage ;; several keys — cut to another measured head transform at frame 40
[:xform :pos] {:animated? true :interp :hold :anchors {0 12, 40 42}
:dense {:store "sha256:…" :stride 2 :frames 600}
:generated {:by :anchor/similarity}}
;; per plate: the head snaps at each selected frame and holds
[:xform :pos] {:animated? true :interp :hold :keys {0 […], 12 […], 23 […]}}
``` ```
The three modes are the three channel shapes, on one channel, on one node. The The map is `local change frame -> measured source frame`. A single lock is a
third is the one a plate strip wants — the head pose is stable for exactly as one-key map. Position, rotation and scale read the same held source frame. The
long as a drawing is on screen — and it costs nothing because `:keys` already frame set belongs to head placement, independently of plate drawings and stage
exists. Its frame set is the kept-frame set, which is `suggestPlateFrames` in the pose cuts. No measured block is copied into authored transform keys.
prototype and belongs to painting rather than to measurement.
**Always measure, always store factored, toggle the parent.** The fit is computed **Always measure, always store factored.** The fit is computed and the geometry
and the geometry is stored head-local in every mode, and only the parent's is stored head-local in every mode. Only the frame address used to read the
channel changes. Two things downstream require it, and both would be lost by head's measured transform changes. Two things downstream require that split:
making this an analysis-time switch:
- *Smoothing.* "Smooth the transform, never the contour" only means anything - *Smoothing.* "Smooth the transform, never the contour" only means anything
while the two are separate. while the two are separate.
- *Key selection.* A velocity minimum is "articulation paused" in head-local - *Key selection.* A velocity minimum is "articulation paused" in head-local
space and "the head happened to be still" in image space. space and "the head happened to be still" in image space.
It also makes the toggle an edit to the document rather than a reason to This is a document edit, not a reason to re-analyse. A registered tracing photo
re-analyse: tier 1, undoable, syncable, and instant. will use its own source frame's stabilising transform followed by the same
selected head placement, so it aligns with the vectors drawn over it.
### Two nodes, because two different things want that transform ### Two nodes, because two different things want that transform
``` ```
:face group — AUTHORED. where the face sits on the stage, and how big. :face group — AUTHORED. where the face sits on the stage, and how big.
:head group — MEASURED. the head's motion, or identity. :head group — MEASURED. dense head motion read at the selected frame.
:mouth :mouth-in :teeth :lid-r :lid-l :brow-r :brow-l … :mouth :mouth-in :teeth :lid-r :lid-l :brow-r :brow-l …
``` ```
Switching modes rewrites `:head` and never touches `:face`, so it cannot move Changing anchor keys edits `:head` and never touches `:face`, so it cannot move
something that was placed by hand. A group node is free, and keeping the authored something that was placed by hand. A group node is free, and keeping the authored
and the measured transform apart is the whole reason the transform is decomposed and the measured transform apart is the whole reason the transform is decomposed
in the first place. in the first place.
@ -627,12 +620,12 @@ Proof that it covers what exists, not just what is wanted:
| brow ring + quantised raise | node `:brow-r`, `[:geom :pts]` dense (the traced ring with height removed), `[:xform :pos]` dense (the quantised raise). **The decomposition design.md insists on is two channels.** | | brow ring + quantised raise | node `:brow-r`, `[:geom :pts]` dense (the traced ring with height removed), `[:xform :pos]` dense (the quantised raise). **The decomposition design.md insists on is two channels.** |
| head plate, kept frames | node `:head`, `:symbol` per instance, keys on `[:symbol]` at kept frames | | head plate, kept frames | node `:head`, `:symbol` per instance, keys on `[:symbol]` at kept frames |
| `makeXform` face-oval crop | **gone.** Placement is `[:xform :*]` on `:face`; the stage clips | | `makeXform` face-oval crop | **gone.** Placement is `[:xform :*]` on `:face`; the stage clips |
| `stabilize` transforms | `[:xform :*]` on `:head` — framed identity, dense, or keyed at kept frames | | `stabilize` transforms | dense `[:xform :*]` on `:head`, read through its optional `:anchors` map |
| registered underlay | not data — a UI layer riding `(world-of resolver :head)` | | registered underlay | not data — a UI layer riding `(world-of resolver :head)` |
| painted background cel | node per layer, `[:geom :pts]` **framed**, `[:style :color]` framed | | painted background cel | node per layer, `[:geom :pts]` **framed**, `[:style :color]` framed |
| `mouth lead` | `:time {:offset k}` on performance nodes only | | `mouth lead` | `:time {:offset k}` on performance nodes only |
| `exposure` | `:time {:expose n}` on the clip root, inherited | | `exposure` | `:time {:expose n}` on the clip root, inherited |
| picture fps | `:time {:source-fps s :sample-fps p}` on the clip root, applied after analysis | | picture fps | resolver samples marked generated channels at the picture rate; authored keys keep their own time |
| hand correction | an `:over` layer, `:offset` or `:replace` | | hand correction | an `:over` layer, `:offset` or `:replace` |
The brow row is the one worth looking at twice. `docs/design.md` argues at length The brow row is the one worth looking at twice. `docs/design.md` argues at length

86
docs/timing-model.md Normal file
View file

@ -0,0 +1,86 @@
# Timing model
The source footage, authored drawings, generated face motion, and stage placement
have different frame decisions. They share a clock but do not share one kept-frame
list. `timing-handoff.md` records earlier implementation notes.
## Frame spaces
- A source frame addresses a decoded image and its measured face data. Keep the
source cadence and, for variable-rate video, its presentation timestamp.
- A timeline frame addresses authored keys in the clip or symbol's local space.
- A stage frame is mapped through the symbol instance's offset and rate before
local frame decisions are read. Moving a placement does not rewrite its keys.
The analyzed source poses remain dense. A lower picture rate or a skipped pose
never removes source data or shortens audio.
## Head placement
Analysis fits each source frame's rigid landmarks into one common head-local
space. Its inverse is the measured head transform, stored densely on `:head`.
The head node has one optional anchor map:
```clojure
;; no :anchors free movement: read measured frame f at f
:anchors {0 12} ; one lock: use frame 12's transform throughout
:anchors {0 12, 40 42} ; keyed locks: switch to frame 42 at local frame 40
```
A key is `(local change frame -> measured source frame)`. Its value holds to the
next key. The map chooses position, rotation and scale together. Frame zero must
have a key when the map exists. The dense transform blocks remain intact, so
editing anchors is a small document change and re-freezing can replace the
measurements without losing the anchor choices.
A source image used for tracing should be registered with that image's measured
stabilizing transform, then the selected head transform, then the authored
`:face` placement. This makes the photo and head-local vectors share the same
orientation and position. Tracing-photo selection is a separate editor address;
it does not choose the head anchor.
The prototype stabilizes into the shot's mean rigid pose and uses an early
closed-mouth frame for raster framing. Those are internal analysis and framing
choices. The authored head-anchor map above controls which measured head pose is
shown over each range. It is independent of plate drawing starts.
## Performance poses
Generated mouth, eye, and brow channels can be sampled at a lower picture rate
without retiming authored keys. The normal rule picks the latest available pose
at or before a picture-grid time. A future performance policy may add important
closed-mouth poses and store manual keeps/drops separately from the rate's
proposal. Related parts should share a selected pose by default: a mouth outline,
interior, teeth and generated visibility must not disagree about its frame.
## Stage placement
A symbol placement has optional pose-cut tracks, separate from head anchors:
```clojure
:playback {:tracks {:mouth {0 12, 8 27}
:eye-r {0 0, 4 6}}}
```
These maps are also `(local change frame -> source pose frame)`. They select which
baked/generated shape pose appears on that placement. Before the first explicit
cut, normal generated motion continues. Cuts hold, without interpolation, until
the next cut. A `[:node id]` track can override one shape in a shared group.
Authored cels, transforms, and audio remain on their normal local time.
The current implementation reads retained frozen channels. A separate resolved
geometry bake is not implemented; when added, it must preserve addressable
candidate poses so stage cuts can still select any of them.
## Ownership
| Choice | Owner | Current state |
| --- | --- | --- |
| Source frames and timestamps | Footage/analysis | Constant-rate frame indexing exists; variable timestamps remain future work |
| Head anchor map | `:head` node | Implemented, stored with the node |
| Tracing cel starts and photo address | Authored cel | Separate future work |
| Generated picture-rate proposal and closure protection | Roto clip/symbol | Generated-only picture sampling exists; closure protection remains future work |
| Stage pose cuts | Symbol instance | Implemented, stored with the instance |
Preview and export use the same resolver for generated picture sampling and stage
cuts. Export still emits every timeline frame at the clip's audio rate.

View file

@ -21,11 +21,9 @@
anything that reads a source pixel, because that part takes the landmarks and anything that reads a source pixel, because that part takes the landmarks and
the frames and never the transform. the frames and never the transform.
TWO CLIPS, ONE STORE. `:take` carries the head as filmed and `:take-locked` TWO CLIPS, ONE STORE. `:take` reads each measured head transform in time;
carries it locked, and they are the same dense blocks with one node's channels `:take-locked` holds the measured transform from frame zero. They share the
written two ways. That is the claim \"stabilisation is a channel, not a mode\" same dense blocks; only the head node's anchor map differs."
made checkable by eye: switching between them is a document edit, tier 1, and
not one byte of tier 2 differs."
(:require [arthur.flow.address :as address] (:require [arthur.flow.address :as address]
[arthur.flow.freeze :as freeze] [arthur.flow.freeze :as freeze]
[arthur.flow.take :as take] [arthur.flow.take :as take]
@ -74,7 +72,7 @@
:expose 2 :expose 2
;; The head as filmed. `:take-locked` is the same freeze with this one ;; The head as filmed. `:take-locked` is the same freeze with this one
;; field changed, which is the point. ;; field changed, which is the point.
:head :as-filmed :head :free
;; Provenance, and now a content address. There is no detector here, so ;; Provenance, and now a content address. There is no detector here, so
;; the generator IS the detector and its seed is the source: two synth ;; the generator IS the detector and its seed is the source: two synth
;; takes at different seeds are different analyses, which is the same ;; takes at different seeds are different analyses, which is the same
@ -94,5 +92,5 @@
(def clip (delay (:clip @frozen))) (def clip (delay (:clip @frozen)))
(def locked (def locked
"The same blocks, with `:head` written as framed identity instead." "The same blocks, with `:head` held at measured frame zero."
(delay (freeze/head-mode {:mode :locked} @frozen))) (delay (freeze/head-mode {:mode :anchored :anchors {0 0}} @frozen)))

View file

@ -36,6 +36,7 @@
(:require [arthur.domain.feature :as feature] (:require [arthur.domain.feature :as feature]
[arthur.domain.node :as node] [arthur.domain.node :as node]
[arthur.domain.palette :as pal] [arthur.domain.palette :as pal]
[arthur.domain.pose :as pose]
[arthur.domain.timeline :as timeline])) [arthur.domain.timeline :as timeline]))
(def ^:const root-id (def ^:const root-id
@ -118,8 +119,9 @@
inside it still resolve, because this is the function that knows how to do that." inside it still resolve, because this is the function that knows how to do that."
([clip store] (resolver clip store pal/index-of root-id)) ([clip store] (resolver clip store pal/index-of root-id))
([clip store palette] (resolver clip store palette root-id)) ([clip store palette] (resolver clip store palette root-id))
([clip store palette root] ([clip store palette root] (resolver clip store palette root nil))
(letfn [(build [tid chain] ([clip store palette root {:keys [picture-fps] :as opts}]
(letfn [(build [tid chain pose-tracks]
(when (some #{tid} chain) (when (some #{tid} chain)
(throw (ex-info "symbol timeline cycle" {:chain (conj chain tid)}))) (throw (ex-info "symbol timeline cycle" {:chain (conj chain tid)})))
(let [tl (or (timeline clip tid) (let [tl (or (timeline clip tid)
@ -127,10 +129,12 @@
nodes (:nodes tl) nodes (:nodes tl)
rank (timeline/draw-rank nodes (timeline/order nodes)) rank (timeline/draw-rank nodes (timeline/order nodes))
ids (sort-by rank (keys nodes)) ids (sort-by rank (keys nodes))
own (timeline/resolver tl store palette) own (timeline/resolver tl store palette pose-tracks
(assoc opts :source-fps (:fps clip)))
children (into {} children (into {}
(for [[id n] nodes :when (= :symbol (:kind n))] (for [[id n] nodes :when (= :symbol (:kind n))]
[id (build (:of n) (conj chain tid))]))] [id (build (:of n) (conj chain tid)
(get-in n [:playback :tracks]))]))]
(fn [f] (fn [f]
(let [by-id (into {} (map (juxt :node identity)) (own f))] (let [by-id (into {} (map (juxt :node identity)) (own f))]
(into [] (into []
@ -151,7 +155,7 @@
[])) []))
(when-let [op (get by-id id)] [op])))) (when-let [op (get by-id id)] [op]))))
ids))))))] ids))))))]
(build root [])))) (build root [] nil))))
(defn problems (defn problems
"Human-readable reasons this clip will not evaluate or will not save. Empty "Human-readable reasons this clip will not evaluate or will not save. Empty
@ -184,6 +188,19 @@
(not (contains? (:timelines clip) (:of n))))] (not (contains? (:timelines clip) (:of n))))]
(str "timeline " (pr-str tid) " symbol " (pr-str id) (str "timeline " (pr-str tid) " symbol " (pr-str id)
" names missing timeline " (pr-str (:of n)))) " names missing timeline " (pr-str (:of n))))
(for [[tid tl] (:timelines clip)
[id n] (:nodes tl)
:when (= :symbol (:kind n))
:let [target (get-in clip [:timelines (:of n)])
active (filter (fn [node]
(some :pose-sampled? (vals (:channels node))))
(vals (:nodes target)))
groups (set (concat
(map #(or (:pose-group %) (:id %)) active)
(map #(vector :node (:id %)) active)))]
p (pose/problems (get-in n [:playback :tracks])
(:frames target) groups)]
(str "timeline " (pr-str tid) " symbol " (pr-str id) ": " p))
(for [[tid tl] (:timelines clip) (for [[tid tl] (:timelines clip)
[id n] (:nodes tl) [id n] (:nodes tl)
:when (and (= :audio (:kind n)) (:linked-to n) :when (and (= :audio (:kind n)) (:linked-to n)

View file

@ -44,7 +44,9 @@
WHY `measured` IS ONE LEAF AND CHANNELS ARE NOT. `:head`'s measured channels are WHY `measured` IS ONE LEAF AND CHANNELS ARE NOT. `:head`'s measured channels are
not authored: they are written together by a freeze and replaced together by a not authored: they are written together by a freeze and replaced together by a
re-freeze, and `head-mode` reads them to write `:channels`. A leaf per measured re-freeze, and `head-mode` exposes them through `:channels`. The optional
`:anchors` map on the head node chooses which measured frame those channels
read. A leaf per measured
channel would offer a write nobody can make. The authored channels beside them channel would offer a write nobody can make. The authored channels beside them
are one leaf each, because a hand writes one at a time. are one leaf each, because a hand writes one at a time.

View file

@ -0,0 +1,94 @@
(ns arthur.domain.pose
"An instance's explicit, held choices of source pose for each shape group.
A track is {local-frame -> source-frame}. The key is when the cut happens;
the value is the frozen pose to read. Skipped source frames remain available.")
(defn prepare
"Sort exposure tracks once when building a resolver."
[tracks]
(into {}
(map (fn [[group entries]]
[group (vec (sort-by first entries))]))
tracks))
(defn held-frame
"Last value keyed at or before f, or default before the first key."
[entries f default-frame]
(loop [lo 0 hi (dec (count entries)) hit nil]
(if (> lo hi)
(if (some? hit) (second (nth entries hit)) default-frame)
(let [mid (bit-shift-right (+ lo hi) 1)]
(if (<= (first (nth entries mid)) f)
(recur (inc mid) hi mid)
(recur lo (dec mid) hit))))))
(defn source-frame
"Read an explicit cut if one has happened; otherwise read the default pose.
This keeps the existing motion before the first edited cut."
[prepared group f default-frame]
(if-let [entries (get prepared group)]
(held-frame entries f default-frame)
default-frame))
(defn put-cut
"Set one held pose on a symbol instance. Earlier motion stays untouched."
[clip instance group at source]
(let [node (get-in clip [:timelines :main :nodes instance])
symbol (get-in clip [:timelines (:of node)])
length (:frames symbol)
active (filter (fn [n] (some :pose-sampled? (vals (:channels n))))
(vals (:nodes symbol)))
groups (set (map #(or (:pose-group %) (:id %)) active))
ids (set (map :id active))]
(when-not (and (= :symbol (:kind node))
(or (contains? groups group)
(and (vector? group) (= 2 (count group))
(= :node (first group))
(contains? ids (second group))))
(integer? at) (<= 0 at) (< at length)
(integer? source) (<= 0 source) (< source length))
(throw (ex-info "invalid stage pose cut"
{:instance instance :group group :at at :source source})))
(update-in clip [:timelines :main :nodes instance :playback :tracks group]
#(assoc (or % {}) at source))))
(defn remove-cut
"Remove a cut; an empty track again follows the normal generated motion."
[clip instance group at]
(let [path [:timelines :main :nodes instance :playback :tracks group]]
(if-let [entries (get-in clip path)]
(if-let [remaining (not-empty (dissoc entries at))]
(assoc-in clip path remaining)
(update-in clip [:timelines :main :nodes instance :playback :tracks]
dissoc group))
clip)))
(defn problems
"Errors in one symbol instance's exposure tracks."
[tracks source-frames groups]
(cond
(nil? tracks) []
(not (map? tracks)) [":playback :tracks must be a map"]
:else
(vec
(mapcat
(fn [[group entries]]
(cond
(not (contains? groups group))
[(str "pose track " (pr-str group) " names no generated shape")]
(not (map? entries))
[(str "pose track " (pr-str group) " must map local frames to source frames")]
:else
(concat
(when-not (every? #(and (integer? %) (<= 0 %)
(or (nil? source-frames) (< % source-frames)))
(keys entries))
[(str "pose track " (pr-str group) " has an invalid change frame")])
(when-not (every? #(and (integer? %) (<= 0 %)
(or (nil? source-frames) (< % source-frames)))
(vals entries))
[(str "pose track " (pr-str group) " names a pose outside the source")]))))
tracks))))

View file

@ -55,6 +55,7 @@
fill in the same channel rather than convert into a second format." fill in the same channel rather than convert into a second format."
(:require [arthur.domain.channel :as ch] (:require [arthur.domain.channel :as ch]
[arthur.domain.node :as node] [arthur.domain.node :as node]
[arthur.domain.pose :as pose]
[arthur.domain.palette :as pal])) [arthur.domain.palette :as pal]))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
@ -335,6 +336,29 @@
{:keys (vec (sort-by str (keys tl)))}))) {:keys (vec (sort-by str (keys tl)))})))
nodes)) nodes))
(defn- channel-frame
"Only marked generated channels read picture cadence and instance pose cuts."
[choices head-anchors nodes source-fps picture-fps id path c lf]
(cond
(and (= id :head) (contains? #{[:xform :pos] [:xform :rot] [:xform :scale]} path)
(contains? head-anchors id))
(pose/held-frame (get head-anchors id) lf lf)
(:pose-sampled? c)
(pose/source-frame choices
(if (contains? choices [:node id])
[:node id]
(or (:pose-group (get nodes id)) id))
lf
(node/sample-frame lf source-fps picture-fps))
:else lf))
(defn- prepared-head-anchors [nodes]
(into {}
(for [[id n] nodes :when (seq (:anchors n))]
[id (vec (sort-by first (:anchors n)))])))
(defn- eval-into (defn- eval-into
"One frame, as a fold over the nodes in topological order. "One frame, as a fold over the nodes in topological order.
@ -381,10 +405,17 @@
This is the definition of what a frame means. `resolver` is what plays it." This is the definition of what a frame means. `resolver` is what plays it."
([tl f] (eval-frame tl f nil pal/index-of)) ([tl f] (eval-frame tl f nil pal/index-of))
([tl f store] (eval-frame tl f store pal/index-of)) ([tl f store] (eval-frame tl f store pal/index-of))
([tl f store palette] ([tl f store palette] (eval-frame tl f store palette nil nil))
([tl f store palette pose-tracks opts]
(let [nodes (nodes-of tl) (let [nodes (nodes-of tl)
choices (pose/prepare pose-tracks)
head-anchors (prepared-head-anchors nodes)
{:keys [source-fps picture-fps]} opts
ord (order nodes)] ord (order nodes)]
(eval-into {:read (fn [_id _path c lf] (ch/value-at c lf store)) (eval-into {:read (fn [id path c lf]
(ch/value-at c (channel-frame choices head-anchors nodes
source-fps picture-fps id path c lf)
store))
:palette palette :palette palette
:mat-for (fn [_id] (node/mat)) :mat-for (fn [_id] (node/mat))
:pinv-for (fn [id] (node/pinv (get nodes id))) :pinv-for (fn [id] (node/pinv (get nodes id)))
@ -435,10 +466,14 @@
The op maps themselves are allocated fresh, and deliberately: there are a dozen The op maps themselves are allocated fresh, and deliberately: there are a dozen
of them per frame against hundreds of points, so pooling them would buy of them per frame against hundreds of points, so pooling them would buy
nothing and cost the ability to hand an op list around as plain data." nothing and cost the ability to hand an op list around as plain data."
([tl] (resolver tl nil pal/index-of)) ([tl] (resolver tl nil pal/index-of nil nil))
([tl store] (resolver tl store pal/index-of)) ([tl store] (resolver tl store pal/index-of nil nil))
([tl store palette] ([tl store palette] (resolver tl store palette nil nil))
([tl store palette pose-tracks] (resolver tl store palette pose-tracks nil))
([tl store palette pose-tracks {:keys [source-fps picture-fps]}]
(let [nodes (nodes-of tl) (let [nodes (nodes-of tl)
choices (pose/prepare pose-tracks)
head-anchors (prepared-head-anchors nodes)
ord (order nodes) ord (order nodes)
rank (draw-rank nodes ord) rank (draw-rank nodes ord)
cursors (into {} cursors (into {}
@ -460,7 +495,10 @@
;; eval-into having to report it — and it covers groups, which are ;; eval-into having to report it — and it covers groups, which are
;; placed but emit no op, and which are exactly what an underlay rides. ;; placed but emit no op, and which are exactly what an underlay rides.
placed (volatile! {}) placed (volatile! {})
ctx {:read (fn [id path _c lf] (ch/sample! (get-in cursors [id path]) lf)) ctx {:read (fn [id path c lf]
(ch/sample! (get-in cursors [id path])
(channel-frame choices head-anchors nodes
source-fps picture-fps id path c lf)))
:palette palette :palette palette
:mat-for (fn [id] (get mats id)) :mat-for (fn [id] (get mats id))
:on-place (fn [id p] (vswap! placed assoc id p)) :on-place (fn [id p] (vswap! placed assoc id p))
@ -521,6 +559,18 @@
(into (for [[id n] nodes (into (for [[id n] nodes
p (node/problems n)] p (node/problems n)]
(str "node " (pr-str id) ": " p))) (str "node " (pr-str id) ": " p)))
(into (for [[id n] nodes
:let [anchors (:anchors n)]
:when (some? anchors)
:when (not (and (= id :head)
(map? anchors) (contains? anchors 0)
(integer? (:frames tl))
(every? #(and (integer? %) (<= 0 %)
(< % (:frames tl)))
(concat (keys anchors) (vals anchors)))
(= (:channels n) (:measured n))))]
(str "node " (pr-str id)
": :anchors must start at frame 0, name valid measured frames, and read the measured head channels")))
(into (for [k (remove timeline-keys (keys tl))] (into (for [k (remove timeline-keys (keys tl))]
(str "timeline has a field with no leaf to save it in: " (pr-str k)))) (str "timeline has a field with no leaf to save it in: " (pr-str k))))
(into (when-not (or (nil? (:frames tl)) (and (integer? (:frames tl)) (pos? (:frames tl)))) (into (when-not (or (nil? (:frames tl)) (and (integer? (:frames tl)) (pos? (:frames tl))))

View file

@ -12,7 +12,8 @@
reachable and neither should be the other's special case. reachable and neither should be the other's special case.
What is genuinely shared is everything above the sink, and it is most of the What is genuinely shared is everything above the sink, and it is most of the
work: rooting the resolver at the chosen timeline, the picture-rate time map, work: rooting the resolver at the chosen timeline, generated-channel picture
sampling,
the raster, the frame loop, the audio mix, the progress reporting and the the raster, the frame loop, the audio mix, the progress reporting and the
yielding that lets the page paint. So `run!` owns all of that and calls three yielding that lets the page paint. So `run!` owns all of that and calls three
methods. methods.
@ -126,22 +127,6 @@
(update tl :nodes select-keys (kin (:nodes tl) id)) (update tl :nodes select-keys (kin (:nodes tl) id))
tl)) tl))
(defn- sampled
"The timeline with picture-rate sampling written onto its root node's time map.
The same transform `subs/render/::timeline` applies for playback, and
deliberately the same one: an export that sampled poses differently from the
preview would make the preview a lie about the deliverable. Only the root node's
time map changes — the dense source track stays at its native rate, and the
frame COUNT is untouched, which is what keeps the duration and therefore the
audio right."
[tl source-fps picture-fps]
(if (and picture-fps source-fps (< picture-fps source-fps))
(-> tl
(assoc-in [:nodes :root :time :source-fps] source-fps)
(assoc-in [:nodes :root :time :sample-fps] picture-fps))
tl))
(defn- yield! (defn- yield!
"Hand the event loop a turn between frames. "Hand the event loop a turn between frames.
@ -183,8 +168,8 @@
:width (* (:width clip) zoom) :width (* (:width clip) zoom)
:height (* (:height clip) zoom) :height (* (:height clip) zoom)
:seconds (/ (:frames tl) (:fps clip)) :seconds (/ (:frames tl) (:fps clip))
;; The poses the picture actually holds, which is what a lower picture rate ;; The unedited picture-grid count. A per-instance pose track can add or
;; changes. The frame COUNT does not move — see the namespace docstring. ;; remove changes, so this is only the grid's nominal count.
:poses (if (and picture-fps (< picture-fps (:fps clip))) :poses (if (and picture-fps (< picture-fps (:fps clip)))
(js/Math.ceil (* (/ (:frames tl) (:fps clip)) picture-fps)) (js/Math.ceil (* (/ (:frames tl) (:fps clip)) picture-fps))
(:frames tl))}))) (:frames tl))})))
@ -207,9 +192,9 @@
;; Rooted at the chosen timeline, so exporting a symbol is exporting a ;; Rooted at the chosen timeline, so exporting a symbol is exporting a
;; clip whose root that symbol is. Nested symbols inside it still ;; clip whose root that symbol is. Nested symbols inside it still
;; resolve — clip/resolver is the function that knows how. ;; resolve — clip/resolver is the function that knows how.
doc (assoc-in clip [:timelines timeline] doc (assoc-in clip [:timelines timeline] tl)
(sampled tl (:fps clip) picture-fps)) resolve-frame (clip/resolver doc store palette timeline
resolve-frame (clip/resolver doc store palette timeline) {:picture-fps picture-fps})
ras (raster/make (:width clip) (:height clip)) ras (raster/make (:width clip) (:height clip))
bg (get palette :bg 0)] bg (get palette :bg 0)]
(-> (audio! doc timeline store audio-url) (-> (audio! doc timeline store audio-url)

View file

@ -245,65 +245,34 @@
:tx (* (- s') (+ (* c tx) (* sn ty))) :tx (* (- s') (+ (* c tx) (* sn ty)))
:ty (* (- s') (+ (* (- sn) tx) (* c ty)))})) :ty (* (- s') (+ (* (- sn) tx) (* c ty)))}))
(def ^:private head-modes #{:locked :as-filmed :per-plate}) (def ^:private head-modes #{:free :anchored})
(defn head-mode (defn head-mode
"Rewrite `:head`'s transform channels into one of the three shapes. "Keep the measured transform dense; optionally hold chosen source frames.
STABILISATION IS A CHANNEL, NOT A MODE. `{s θ tx ty}` per frame IS A nil anchor map reads measured frame f at frame f (free movement).
`[:xform :scale]`, `[:xform :rot]` and `[:xform :pos]`, so removing the head's `{0 12}` locks to the measured transform of source frame 12. `{0 12, 40 42}`
motion is not a pipeline setting — it is a question of which shape one node's cuts to source frame 42 at local frame 40. The same map selects position,
channels carry: rotation and scale, so the head and registered photo cannot drift apart.
No analysis block or authored face placement changes."
:locked framed identity — the head sits still, for tracing and for [{:keys [mode anchors]} {:keys [clip]}]
judging articulation
:as-filmed dense — the head moves around the stage
:per-plate keyed at the kept frames — the head pose is stable for exactly as
long as a drawing is on screen, which is what a plate strip wants
ALWAYS MEASURE, ALWAYS STORE FACTORED, whatever the toggle says. The blocks are
written once and every mode reads them or ignores them; making this an
analysis-time switch would lose two things downstream. \"Smooth the transform,
never the contour\" only means anything while the two are separate. And a
velocity minimum is \"articulation paused\" in head-local space and \"the head
happened to be still\" in image space, so key selection needs the split to exist
in storage.
So this is a DOCUMENT EDIT: tier 1, undoable, syncable, instant, and not a
reason to re-analyse. It rewrites `:head` and touches nothing else — in
particular not `:face`, which a hand placed, and not one byte of any block.
`:per-plate` samples the dense track at the kept frames and stores plain
vectors. It may not store what `value-at` handed it: a wide dense read is a
VIEW into the block, and a view into tier 2 sitting in the document is a value
that changes when a re-freeze rewrites the array under it."
[{:keys [mode kept]} {:keys [clip store]}]
(when-not (contains? head-modes mode) (when-not (contains? head-modes mode)
(throw (ex-info "head mode is not one of the three channel shapes" (throw (ex-info "head mode must be free or anchored"
{:mode mode :modes head-modes}))) {:mode mode :modes head-modes})))
(let [base (get-in (clip/root clip) [:nodes :head :measured]) (when (and (= mode :free) (some? anchors))
at (fn [path f] (throw (ex-info "free head motion has no anchors" {:anchors anchors})))
(let [v (ch/value-at (get base path) f store) (let [frames (clip/frames clip)]
n (:stride (:dense (get base path)))] (when (and (= mode :anchored)
(if (= 1 n) v (mapv #(ch/component v %) (range n))))) (not (and (map? anchors) (contains? anchors 0)
keys-of (fn [path] (every? #(and (integer? %) (<= 0 %) (< % frames))
(-> (get base path) (concat (keys anchors) (vals anchors))))))
(dissoc :dense) (throw (ex-info "anchored head needs a frame-zero key and valid source frames"
(assoc :keys (into {} (map (juxt identity #(at path %))) (sort kept)))))] {:anchors anchors :frames frames})))
(when (and (= mode :per-plate) (empty? kept)) (update-in clip [:timelines clip/root-id :nodes :head]
(throw (ex-info "the per-plate head mode needs a kept-frame set; it is the plate strip's, not measurement's" (fn [n]
{:mode mode}))) (cond-> (assoc n :channels (:measured n))
(assoc-in clip [:timelines clip/root-id :nodes :head :channels] (= mode :anchored) (assoc :anchors anchors)
(case mode (= mode :free) (dissoc :anchors))))))
;; No `:generated` on the locked shape, and that is not an
;; oversight: nothing generated this identity. It is a decision,
;; and provenance that claimed otherwise would offer a re-freeze
;; button that could only undo it.
:locked {[:xform :pos] (ch/framed [0.0 0.0])
[:xform :rot] (ch/framed 0.0)
[:xform :scale] (ch/framed [1.0 1.0])}
:as-filmed base
:per-plate (into {} (map (juxt key #(keys-of (key %)))) base)))))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; the aperture, onto [:vis] of :mouth-in ;; the aperture, onto [:vis] of :mouth-in
@ -347,6 +316,32 @@
(range (count values)))) (range (count values))))
:generated generated)) :generated generated))
(def ^:private pose-groups
{:mouth :mouth :mouth-in :mouth :teeth :mouth
:eye-r :eye-r :eye-r-in :eye-r :iris-r :eye-r :pupil-r :eye-r
:eye-l :eye-l :eye-l-in :eye-l :iris-l :eye-l :pupil-l :eye-l
:brow-r :brow-r :brow-l :brow-l})
(defn- performance-nodes
"Mark the generated moving channels that may read an instance's pose choice.
Provenance helps identify them here; the renderer reads only :pose-sampled?."
[nodes]
(into {}
(map (fn [[id n]]
[id (cond-> n
(contains? pose-groups id)
(assoc :pose-group (get pose-groups id))
(contains? pose-groups id)
(update :channels
(fn [channels]
(into {}
(map (fn [[path ch]]
[path (cond-> ch
(and (:generated ch) (:animated? ch))
(assoc :pose-sampled? true))]))
channels))))]))
nodes))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; the face, onto the stage ;; the face, onto the stage
@ -586,12 +581,13 @@
(and (contains? presence id) (and (contains? presence id)
(not (nth (get presence id) f)))))) (not (nth (get presence id) f))))))
obs {:detected detected :presence presence}] obs {:detected detected :presence presence}]
(case area (update (case area
:mouth (mouth-part absent? obs params measured) :mouth (mouth-part absent? obs params measured)
:eye (feature-parts absent? obs params (select-keys measured [:eyes])) :eye (feature-parts absent? obs params (select-keys measured [:eyes]))
:brow (feature-parts absent? obs params (select-keys measured [:brows])) :brow (feature-parts absent? obs params (select-keys measured [:brows]))
:teeth (interior-part params absent? obs (:teeth measured)) :teeth (interior-part params absent? obs (:teeth measured))
(throw (ex-info "unknown frozen feature type" {:area area}))))) (throw (ex-info "unknown frozen feature type" {:area area})))
:nodes performance-nodes)))
(defn head-part (defn head-part
"Freeze only the measured head transform from an already conditioned anchor." "Freeze only the measured head transform from an already conditioned anchor."
@ -638,8 +634,8 @@
:brow-verts the brow rings' vertex budget :brow-verts the brow rings' vertex budget
:aperture-cut fraction of the take's peak aperture below which the mouth :aperture-cut fraction of the take's peak aperture below which the mouth
interior is not present interior is not present
:head :locked | :as-filmed | :per-plate :head :free | :anchored
:kept frames, for :per-plate only :anchors optional map of local change frame -> measured source frame
:analysis the analysis record these measurements came from — detector, :analysis the analysis record these measurements came from — detector,
VERSION, source, source cadence and pixel aspect. Its `:id` is VERSION, source, source cadence and pixel aspect. Its `:id` is
a content address over all of that, every block's key is a hash a content address over all of that, every block's key is a hash
@ -682,9 +678,8 @@
painted cel rides a moving plate it will be. painted cel rides a moving plate it will be.
`:head` keeps its measured channels under `:measured` as well as in `:head` keeps its measured channels under `:measured` as well as in
`:channels`, so `head-mode` can switch shapes without the blocks or the `:channels`; its optional `:anchors` map changes only their frame address."
provenance having to be rebuilt." [{:keys [name fps stage expose verts aperture-cut head anchors
[{:keys [name fps stage expose verts aperture-cut head kept
analysis anchor-avg contour-avg] :as params} analysis anchor-avg contour-avg] :as params}
{:keys [transforms outer inner detected presence eyes brows teeth] :as inputs}] {:keys [transforms outer inner detected presence eyes brows teeth] :as inputs}]
(let [nf (count outer) (let [nf (count outer)
@ -796,4 +791,7 @@
(throw (ex-info "presence track names no feature in this clip" (throw (ex-info "presence track names no feature in this clip"
{:feature id :features (keys (:features built))})))) {:feature id :features (keys (:features built))}))))
{:store store {:store store
:clip (head-mode {:mode head :kept kept} {:clip built :store store})})) :clip (head-mode {:mode head :anchors anchors}
{:clip (update-in built [:timelines clip/root-id :nodes]
performance-nodes)
:store store})}))

View file

@ -112,7 +112,7 @@
{:name (or (:source manifest) "footage") {:name (or (:source manifest) "footage")
:fps (:fps manifest) :aspect (/ w h) :fps (:fps manifest) :aspect (/ w h)
:stage [320 200] :fit-motion? true :stage [320 200] :fit-motion? true
:expose 1 :head :as-filmed :expose 1 :head :free
;; The detector's identity comes from the server, which ;; The detector's identity comes from the server, which
;; hashes the model asset it serves rather than trusting a ;; hashes the model asset it serves rather than trusting a
;; version string somebody has to remember to bump. See ;; version string somebody has to remember to bump. See

View file

@ -26,21 +26,7 @@
(rf/reg-sub (rf/reg-sub
::timeline ::timeline
:<- [::clip] :<- [::clip]
:<- [::playback/display-fps] (fn [clip _] (some-> clip clip/root)))
(fn [[clip picture-fps] _]
;; The ROOT timeline, with picture sampling written onto its root node's time
;; map. Only that node's time map changes: the dense source track stays at its
;; native rate and the audio clock still advances through source time.
;;
;; `:fps` is read off the CLIP and `:frames` off the timeline, which is the
;; whole reason the two came apart — the source cadence is a fact about how fast
;; the clip plays against its audio, and a nested timeline will not have one.
(when-let [tl (clip/root clip)]
(if (and picture-fps (< picture-fps (:fps clip)))
(-> tl
(assoc-in [:nodes :root :time :source-fps] (:fps clip))
(assoc-in [:nodes :root :time :sample-fps] picture-fps))
tl))))
(rf/reg-sub (rf/reg-sub
::exposure ::exposure
@ -85,6 +71,8 @@
:<- [::timeline] :<- [::timeline]
:<- [::store] :<- [::store]
:<- [::palette] :<- [::palette]
(fn [[document tl store palette] _] :<- [::playback/display-fps]
(fn [[document tl store palette picture-fps] _]
(when tl (clip/resolver (assoc-in document [:timelines :main] tl) (when tl (clip/resolver (assoc-in document [:timelines :main] tl)
store palette)))) store palette clip/root-id
{:picture-fps picture-fps}))))

View file

@ -60,7 +60,8 @@
;; leaf carries structure and no geometry. ;; leaf carries structure and no geometry.
(let [ls (leaf/leaves :c1 @take/clip) (let [ls (leaf/leaves :c1 @take/clip)
n (get ls "clip/c1/timeline/main/node/mouth")] n (get ls "clip/c1/timeline/main/node/mouth")]
(is (= {:id :mouth :name "mouth" :kind :poly :parent :head :z "a1"} n)) (is (= {:id :mouth :name "mouth" :kind :poly :parent :head :z "a1"
:pose-group :mouth} n))
(is (nil? (:channels n))) (is (nil? (:channels n)))
(is (:animated? (get ls "clip/c1/timeline/main/channel/mouth/geom.pts"))))) (is (:animated? (get ls "clip/c1/timeline/main/channel/mouth/geom.pts")))))

View file

@ -4,7 +4,10 @@
[arthur.domain.channel :as ch] [arthur.domain.channel :as ch]
[arthur.domain.clip :as clip] [arthur.domain.clip :as clip]
[arthur.domain.leaf :as leaf] [arthur.domain.leaf :as leaf]
[arthur.domain.node :as node])) [arthur.domain.node :as node]
[arthur.domain.pose :as pose]
[arthur.domain.palette :as pal]
[arthur.domain.timeline :as timeline]))
(def source (def source
{:name "source" :fps 30 :width 320 :height 200 {:name "source" :fps 30 :width 320 :height 200
@ -40,6 +43,97 @@
(is (= [[[:right :mark] 150]] (at 5))) (is (= [[[:right :mark] 150]] (at 5)))
(is (= document (leaf/clip "stage" (leaf/leaves "stage" document)))))) (is (= document (leaf/clip "stage" (leaf/leaves "stage" document))))))
(deftest a-placement-holds-and-cuts-each-generated-shape-independently
(let [values (js/Int16Array. (clj->js (range 2 32)))
visible (ch/keyed {0 true 20 true 21 false})
dense {:animated? true :interp :hold
:dense {:store "sizes" :offset 0 :stride 1 :frames 30}
:pose-sampled? true}
shape (fn [id z group]
{:id id :kind :rect :parent :root :z z :pose-group group
:channels {[:xform :pos] (ch/keyed {0 [0 0] 8 [8 0]})
[:geom :size] dense
[:vis] (assoc visible :pose-sampled? true)
[:style :color] (ch/framed :brow)}})
symbol {:id :sym/poses :frames 30
:nodes {:root {:id :root :kind :group :z "a1"}
:mouth (shape :mouth "a1" :mouth)
:mouth-detail (shape :mouth-detail "a2" :mouth)
:eye (shape :eye "a3" :eye)
:brow (shape :brow "a4" :brow)}}
document {:fps 30 :width 320 :height 200
:timelines
{:main {:id :main :frames 30
:nodes {:root {:id :root :kind :group :z "a1"}
:first {:id :first :kind :symbol :of :sym/poses
:parent :root :z "a1"
:playback {:tracks {:mouth {0 0, 8 20, 9 21}
[:node :mouth-detail] {0 0, 8 4}
:eye {0 0, 4 4}}}}
:second {:id :second :kind :symbol :of :sym/poses
:parent :root :z "a2"
:playback {:tracks {:mouth {0 0, 8 8}}}}}}
:sym/poses symbol}}
resolve (clip/resolver document {"sizes" {:data values}})
low-resolve (clip/resolver document {"sizes" {:data values}}
pal/index-of :main {:picture-fps 8})
at (fn [f] (into {} (map (fn [op] [(:node op) op])) (resolve f)))
low-at (fn [f] (into {} (map (fn [op] [(:node op) op])) (low-resolve f)))]
(is (empty? (clip/problems document)))
(is (= document (leaf/clip "stage" (leaf/leaves "stage" document))))
(is (= 2 (:size (get (at 7) [:first :mouth]))) "eight static frames")
(is (= 22 (:size (get (at 8) [:first :mouth]))) "cut to source pose 20")
(is (= 6 (:size (get (at 8) [:first :mouth-detail])))
"one node may depart from its shared mouth group")
(is (= 10 (:size (get (at 8) [:second :mouth]))) "other instance chooses pose 8")
(is (= 6 (:size (get (at 7) [:first :eye]))) "eye has its own timing")
(is (= 8 (:cx (get (at 8) [:first :eye]))) "authored position still reads stage time")
(is (nil? (get (at 9) [:first :mouth]))
"generated visibility is read from the same selected pose")
(is (some? (get (at 9) [:second :mouth])))
(is (= 5 (:size (get (low-at 7) [:first :brow])))
"picture rate samples only generated motion")
(is (= 22 (:size (get (low-at 8) [:first :mouth])))
"an explicit cut occurs at its exact local frame, even off the picture grid")
(is (= 8 (:cx (get (low-at 8) [:first :eye])))
"authored position ignores the picture grid")
(let [tl (get-in document [:timelines :sym/poses])
opts {:source-fps 30 :picture-fps 8}]
(is (= (mapv #(select-keys % [:node :cx :size])
(timeline/eval-frame tl 8 {"sizes" {:data values}}
pal/index-of {:mouth {0 0, 8 20}} opts))
(mapv #(select-keys % [:node :cx :size])
((timeline/resolver tl {"sizes" {:data values}}
pal/index-of {:mouth {0 0, 8 20}} opts) 8)))
"pure evaluation and playback apply the same pose choice"))))
(deftest stage-pose-edits-preserve-earlier-motion-and-survive-save
(let [document (-> source
(assoc-in [:timelines :main :nodes :placed]
{:id :placed :kind :symbol :of :sym/test :parent :root
:z "a2"})
(assoc-in [:timelines :sym/test]
{:id :sym/test :frames 4
:nodes {:root {:id :root :kind :group :z "a1"}
:mark {:id :mark :kind :rect :parent :root
:z "a1" :pose-group :mark
:channels {[:geom :size]
{:animated? true :interp :hold
:keys {0 2 1 3 2 4 3 5}
:pose-sampled? true}}}}})
(pose/put-cut :placed :mark 2 3))
cuts (get-in document [:timelines :main :nodes :placed :playback :tracks :mark])]
(is (= {2 3} cuts))
(is (= 1 (pose/source-frame (pose/prepare {:mark cuts}) :mark 1 1))
"before the first cut, dense motion continues")
(is (= 3 (pose/source-frame (pose/prepare {:mark cuts}) :mark 2 2)))
(is (= document (leaf/clip "stage" (leaf/leaves "stage" document))))
(is (nil? (get-in (pose/remove-cut document :placed :mark 2)
[:timelines :main :nodes :placed :playback :tracks :mark])))
(is (seq (clip/problems (assoc-in document
[:timelines :main :nodes :placed :playback :tracks :mark]
{4 3}))))))
(defn- uuid-of (defn- uuid-of
"The uuid the layout authors for the placement whose handle is `id`. "The uuid the layout authors for the placement whose handle is `id`.

View file

@ -25,12 +25,8 @@
:channels {[:geom :pts] (ch/framed pts) [:style :color] (ch/framed color)}}) :channels {[:geom :pts] (ch/framed pts) [:style :color] (ch/framed color)}})
(defn- a-timeline (defn- a-timeline
"One square under a `:root` group. "One authored square under a `:root` group. Picture sampling now applies to
marked generated channels in the shared resolver, leaving this square alone."
The group is not decoration: a reduced picture rate is applied by writing a time
map onto the node called `:root` — `export/sampled` and `subs/render` both do it,
which is what keeps the export's poses identical to the preview's — so a timeline
without one is not a shape this tool produces."
[frames] [frames]
{:frames frames {:frames frames
:nodes {:root {:id :root :kind :group :z "a1"} :nodes {:root {:id :root :kind :group :z "a1"}

View file

@ -36,7 +36,7 @@
(defn- params-at [overrides] (defn- params-at [overrides]
(merge take/knobs (merge take/knobs
{:name "addr" :fps fps :aspect 1 :stage [320 200] {:name "addr" :fps fps :aspect 1 :stage [320 200]
:expose 1 :head :as-filmed} :expose 1 :head :free}
overrides)) overrides))
(defn- freeze-at (defn- freeze-at

View file

@ -13,6 +13,7 @@
[arthur.domain.channel :as ch] [arthur.domain.channel :as ch]
[arthur.domain.clip :as clip] [arthur.domain.clip :as clip]
[arthur.domain.geom :as geom] [arthur.domain.geom :as geom]
[arthur.domain.leaf :as leaf]
[arthur.domain.node :as node] [arthur.domain.node :as node]
[arthur.domain.palette :as pal] [arthur.domain.palette :as pal]
[arthur.domain.raster :as raster] [arthur.domain.raster :as raster]
@ -77,10 +78,11 @@
;; before the data is trusted — which is what it is for. It checks the clip's ;; before the data is trusted — which is what it is for. It checks the clip's
;; fields, every timeline in it and the tracking identities, so it is the whole ;; fields, every timeline in it and the tracking identities, so it is the whole
;; of what a save would refuse. ;; of what a save would refuse.
(doseq [mode [:as-filmed :locked]] (doseq [spec [{:mode :free} {:mode :anchored :anchors {0 0}}]]
(let [c (freeze/head-mode {:mode mode} @frozen)] (let [c (freeze/head-mode spec @frozen)]
(is (empty? (clip/problems c)) (str mode ": " (pr-str (clip/problems c)))))) (is (empty? (clip/problems c)) (str spec ": " (pr-str (clip/problems c))))))
(let [c (freeze/head-mode {:mode :per-plate :kept #{0 12 40 88 150}} @frozen)] (let [c (freeze/head-mode {:mode :anchored
:anchors {0 12, 40 88, 150 150}} @frozen)]
(is (empty? (clip/problems c)) (pr-str (clip/problems c))))) (is (empty? (clip/problems c)) (pr-str (clip/problems c)))))
(deftest the-tree-is-the-one-the-model-specifies (deftest the-tree-is-the-one-the-model-specifies
@ -196,7 +198,7 @@
;; of the two says it is. A test that recomputed the chain would only be ;; of the two says it is. A test that recomputed the chain would only be
;; checking arithmetic against itself; this checks `node/local!`, `node/world!` ;; checking arithmetic against itself; this checks `node/local!`, `node/world!`
;; and `emit` as well. ;; and `emit` as well.
(let [c (freeze/head-mode {:mode :as-filmed} @frozen) (let [c (freeze/head-mode {:mode :free} @frozen)
res (timeline/resolver (clip/root c) @store pal/index-of) res (timeline/resolver (clip/root c) @store pal/index-of)
k (first (:value (chan :face [:xform :scale]))) k (first (:value (chan :face [:xform :scale])))
anc (:value (chan :face [:xform :anchor])) anc (:value (chan :face [:xform :anchor]))
@ -234,47 +236,48 @@
"], the composition says [" wx " " wy "]")))))) "], the composition says [" wx " " wy "]"))))))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; the three modes are the three channel shapes ;; one dense measurement, with optional held anchor frames
(deftest the-three-head-modes-are-the-three-channel-shapes (deftest head-anchors-select-measured-frames-without-copying-channels
(let [kept #{0 12 40 88 150} (let [free (freeze/head-mode {:mode :free} @frozen)
one (freeze/head-mode {:mode :anchored :anchors {0 12}} @frozen)
keyed (freeze/head-mode {:mode :anchored
:anchors {0 12, 40 88, 150 150}} @frozen)
of (fn [c path] (get-in (clip/nodes c) [:head :channels path]))] of (fn [c path] (get-in (clip/nodes c) [:head :channels path]))]
(testing "locked is framed identity" (doseq [c [free one keyed]
(let [sc (freeze/head-mode {:mode :locked} @frozen)] path [[:xform :pos] [:xform :rot] [:xform :scale]]]
(is (= [:framed :framed :framed] (is (= :dense (ch/describe (of c path))))
(mapv #(ch/describe (of sc %)) (is (= (of free path) (of c path)) "anchor edits do not copy measurements"))
[[:xform :pos] [:xform :rot] [:xform :scale]]))) (is (nil? (get-in (clip/nodes free) [:head :anchors])))
(is (= [0.0 0.0] (:value (of sc [:xform :pos])))) (is (= {0 12} (get-in (clip/nodes one) [:head :anchors])))
(is (= 0.0 (:value (of sc [:xform :rot])))) (is (= {0 12, 40 88, 150 150}
(is (= [1.0 1.0] (:value (of sc [:xform :scale])))))) (get-in (clip/nodes keyed) [:head :anchors])))
(testing "as filmed is dense" (is (= keyed (leaf/clip "head" (leaf/leaves "head" keyed)))
(let [sc (freeze/head-mode {:mode :as-filmed} @frozen)] "anchor source addresses survive the document round trip")))
(is (= [:dense :dense :dense]
(mapv #(ch/describe (of sc %)) (deftest head-anchor-keys-hold-the-whole-measured-transform
[[:xform :pos] [:xform :rot] [:xform :scale]]))))) (let [free (timeline/resolver (clip/root
(testing "per plate is keyed at exactly the kept frames" (freeze/head-mode {:mode :free} @frozen))
(let [sc (freeze/head-mode {:mode :per-plate :kept kept} @frozen)] @store pal/index-of)
(doseq [path [[:xform :pos] [:xform :rot] [:xform :scale]]] held (timeline/resolver (clip/root
(is (= :keyed (ch/describe (of sc path)))) (freeze/head-mode {:mode :anchored
(is (= (sort kept) (sort (keys (:keys (of sc path)))))) :anchors {0 12, 40 88}} @frozen))
;; A dense read is a VIEW into tier 2. Storing one in the document would @store pal/index-of)
;; be storing a value that changes when a re-freeze rewrites the array world (fn [resolver frame]
;; under it, so the keys hold plain data. (resolver frame)
(doseq [[_ v] (:keys (of sc path))] (vec (array-seq (timeline/world-of resolver :head))))]
(is (or (number? v) (vector? v)) (str path " key is " (pr-str v))))) (is (= (world free 12) (world held 0)))
;; And the keys are the dense track sampled at those frames, which is the (is (= (world free 12) (world held 38)))
;; whole of what "per plate" means. (is (= (world free 88) (world held 40)))
(is (= (mapv #(ch/value-at (get-in (node :head) [:measured [:xform :rot]]) % @store) (is (= (world free 88) (world held 100)))))
(sort kept))
(mapv (:keys (of sc [:xform :rot])) (sort kept))))))))
(deftest switching-modes-rewrites-the-head-and-nothing-else (deftest switching-modes-rewrites-the-head-and-nothing-else
;; It has to be impossible for the toggle to move something a hand placed, and ;; It has to be impossible for the toggle to move something a hand placed, and
;; it has to be a DOCUMENT edit: tier 1, undoable, syncable, instant, and not a ;; it has to be a DOCUMENT edit: tier 1, undoable, syncable, instant, and not a
;; reason to re-analyse. ;; reason to re-analyse.
(let [a (freeze/head-mode {:mode :as-filmed} @frozen) (let [a (freeze/head-mode {:mode :free} @frozen)
b (freeze/head-mode {:mode :locked} @frozen) b (freeze/head-mode {:mode :anchored :anchors {0 0}} @frozen)
c (freeze/head-mode {:mode :per-plate :kept #{0 40}} @frozen)] c (freeze/head-mode {:mode :anchored :anchors {0 0, 40 40}} @frozen)]
(doseq [x [b c]] (doseq [x [b c]]
(is (= (get (clip/nodes a) :face) (get (clip/nodes x) :face)) (is (= (get (clip/nodes a) :face) (get (clip/nodes x) :face))
":face moved") ":face moved")
@ -288,13 +291,14 @@
;; channels, so the clip's own fields and its other timelines are untouched. ;; channels, so the clip's own fields and its other timelines are untouched.
(is (= (dissoc a :timelines) (dissoc x :timelines)))))) (is (= (dissoc a :timelines) (dissoc x :timelines))))))
(deftest a-head-mode-that-is-not-one-of-the-three-is-refused (deftest invalid-head-anchor-maps-are-refused
(is (thrown-with-msg? ExceptionInfo #"not one of the three channel shapes" (is (thrown-with-msg? ExceptionInfo #"free or anchored"
(freeze/head-mode {:mode :stabilised} @frozen))) (freeze/head-mode {:mode :stabilised} @frozen)))
;; The kept-frame set belongs to the plate strip, not to measurement, so freeze (doseq [anchors [nil {} {12 12} {0 take/frames} {0 0, 10 -1}]]
;; cannot invent one. (is (thrown-with-msg? ExceptionInfo #"frame-zero key"
(is (thrown-with-msg? ExceptionInfo #"kept-frame set" (freeze/head-mode {:mode :anchored :anchors anchors} @frozen))))
(freeze/head-mode {:mode :per-plate} @frozen)))) (is (thrown-with-msg? ExceptionInfo #"has no anchors"
(freeze/head-mode {:mode :free :anchors {0 0}} @frozen))))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; the face: authored, and what makes makeXform deletable ;; the face: authored, and what makes makeXform deletable
@ -597,7 +601,7 @@
;; from different beats are genuinely different mouths and frames inside one ;; from different beats are genuinely different mouths and frames inside one
;; beat are not. This is the numeric half of step 5's done-criterion; the other ;; beat are not. This is the numeric half of step 5's done-criterion; the other
;; half is a picture and lives in test/browser/take.mjs. ;; half is a picture and lives in test/browser/take.mjs.
(let [locked (freeze/head-mode {:mode :locked} @frozen) (let [locked (freeze/head-mode {:mode :anchored :anchors {0 0}} @frozen)
shot (fn [f] shot (fn [f]
(let [r (raster/make W H) (let [r (raster/make W H)
mouth (filter #(= :mouth (:node %)) (ops-at (clip/root locked) f))] mouth (filter #(= :mouth (:node %)) (ops-at (clip/root locked) f))]
@ -614,6 +618,6 @@
(is (< (differ (shot 10) (shot 12)) 200) (is (< (differ (shot 10) (shot 12)) 200)
"a held pose is moving more than the detector noise it should have lost")) "a held pose is moving more than the detector noise it should have lost"))
;; As filmed, the head carries it around the stage as well. ;; As filmed, the head carries it around the stage as well.
(let [filmed (freeze/head-mode {:mode :as-filmed} @frozen)] (let [filmed (freeze/head-mode {:mode :free} @frozen)]
(is (> (count (remove true? (map = (render filmed 10) (render filmed 120)))) 300) (is (> (count (remove true? (map = (render filmed 10) (render filmed 120)))) 300)
"the head does not move across the take"))) "the head does not move across the take")))

View file

@ -17,7 +17,7 @@
(delay (delay
(let [p (merge take/knobs (let [p (merge take/knobs
{:name "regen" :fps 30 :aspect 1 :stage [320 200] {:name "regen" :fps 30 :aspect 1 :stage [320 200]
:expose 1 :head :as-filmed :expose 1 :head :free
:analysis (address/analysis :analysis (address/analysis
{:detector "synth" :version "mulberry32" {:detector "synth" :version "mulberry32"
:seed 9 :frames frames :fps 30 :aspect 1})})] :seed 9 :frames frames :fps 30 :aspect 1})})]
@ -27,7 +27,7 @@
(take/build (take/build
(merge take/knobs (merge take/knobs
{:name "regen" :fps 30 :aspect 1 :stage [320 200] {:name "regen" :fps 30 :aspect 1 :stage [320 200]
:expose 1 :head :as-filmed :expose 1 :head :free
:analysis (get-in @initial [:clip :analysis])} :analysis (get-in @initial [:clip :analysis])}
change) change)
@inputs)) @inputs))