Give features stable identity, eye pairs and per-feature presence

Step 8's data model, ahead of its controls. Nothing here is a UI.

domain/params holds every knob's definition once — default, applicable area,
value constraints and the areas a change would force to regenerate. flow/take's
literal knob map becomes a view of it, so the take's defaults and the future
parameter panel cannot drift apart.

domain/feature adds subjects, features and groups as document data the renderer
never reads. A feature ID is stable for the whole clip, across occlusion: a run
of visible frames is not a new identity. An eye pair is an explicit group of one
or two eyes of the same subject, so a profile view with one identified eye needs
no invented partner. Settings resolve area -> subject -> group -> feature, and
dropping an eye from a pair materialises its effective values first so playback
does not jump. scene/problems now validates all of it.

Presence becomes per-feature rather than per-subject. freeze's :absent predicate
takes a track as well as a frame, so one occluded eye can be absent while its
partner still has a value; a full-face miss still marks everything absent. A
manifest may annotate known gaps as one-based inclusive intervals, which ingest
expands into observation tracks before measurement. An unobserved eye then gets
no vote in the iris pairing and cannot steer the shared gaze — gaze falls back to
whichever eye is visible. Temporal filters still see a sample on every frame,
held from the last observed one, because the numbers are a rectangular buffer;
the state mask, not the buffer, is what says the frame has no value.

js/app.js gets the same occlusion lesson: leading nulls from a face that starts
occluded used to throw away the whole take, and the neutral frame could be chosen
from a held duplicate pose.

Parameter editing, scoped regeneration and a feature-level detector remain. Until
one exists, footage without annotations falls back to the full-face mask rather
than claiming occlusions it cannot see.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B87NVmiU36qQmN9gmFYnJ9

README.md

@@ -18,7 +18,8 @@ modern conveniences belong in the workflow, not the output. See
 
 The active port has reached [step 7 of the port plan](docs/port-plan.md): it plays
 the synthetic take and can load extracted real footage with mouth, eyes, brows,
-and pixel-derived teeth.
+and pixel-derived teeth. The step 8 data model now represents persistent feature
+IDs, eye pairs and feature-level observation gaps; its controls are still pending.
 See [frontend/README.md](frontend/README.md) for setup and the **load frames**
 workflow. The rest of this README describes the older JS prototype, which still
 runs separately on port 8777.

docs/animation-model.md

@@ -79,6 +79,59 @@ this way.
 over which the node exists at all. Distinct from a `[:vis]` channel, which
 blinks an existing node on and off.
 
+### Subjects and tracked features
+
+Scene nodes describe drawings, not tracking identity. A scene may also carry a
+flat `:features` map. A feature ID stays stable for the whole clip, including
+frames where that feature is occluded and later reappears:
+
+```clojure
+:subjects {:face-1 {:id :face-1}}
+:features
+{:eye-r {:id :eye-r :subject :face-1 :area :eye
+         :nodes [:eye-r :eye-r-in :iris-r :pupil-r] :params {}}
+ :eye-l {:id :eye-l :subject :face-1 :area :eye
+         :nodes [:eye-l :eye-l-in :iris-l :pupil-l] :params {}}
+ :mouth {:id :mouth :subject :face-1 :area :mouth
+         :nodes [:mouth :mouth-in :teeth] :params {}}}
+:groups
+{:eyes-1 {:id :eyes-1 :kind :eye-pair :subject :face-1
+          :members [:eye-r :eye-l] :params {}}}
+```
+
+An eye pair is an explicit relationship between one or two eyes of the **same
+subject**. It may have one member when only one eye has been identified; it does
+not invent a second eye. Five subjects with nine identified eyes can have four
+two-member pairs and one one-member pair. Each eye still has its own feature ID
+and presence track. A group is a settings association, not a scene parent or a
+tracking ID. Membership lives only on the group, avoiding a second pointer on
+the feature that could disagree with it.
+
+Each feature resolves settings from its area's definitions, then its group,
+then its own `:params`. An eye can therefore inherit a pair setting or override
+it without changing its partner. Removing it from a pair copies its effective
+values into the feature first, so the result does not jump. Feature identity
+and pair membership are clip-wide; a future parameter track can vary values
+over time without splitting a feature at an observation gap.
+
+Parameter definitions live in one registry: key, default, applicable area,
+value constraints and affected areas. The registry supplies the take's defaults
+today. The parameter UI and regeneration from edited values are later work.
+
+Dense channel state records whether a measurement exists **for that feature on
+that frame**. Occlusion means absent data on that frame, not a false `[:vis]`
+value and not the end of the feature's identity. A full-face detection failure
+makes all its features absent. A single occluded eye need only make that eye's
+channels absent. Footage can carry explicit feature absence intervals in its
+manifest, with one-based inclusive source frame numbers, for example
+`"feature-absence": {"eye-r": [[10, 14]]}`. The loader expands these into
+per-frame observation tracks before measurement. Unobserved landmarks may fill
+rectangular numeric buffers, but they cannot contribute to an eye's contour,
+blink or shared gaze. When one eye is absent, gaze uses the observed eye.
+Until a detector supplies feature-level confidence, footage without annotations
+uses the full-face detection mask as the fallback; it must not claim to detect
+individual occlusions that it cannot see.
+
 ## Channel
 
 Every animatable property is a channel, and channels are addressed **by path**:

docs/port-plan.md

@@ -7,8 +7,11 @@ Step 6 reads extracted footage from the manifest, detects landmarks with local
 MediaPipe assets at full source cadence, and runs the same freeze path as the
 synthetic take. The scene time map can sample the frozen roto at a lower picture
 fps without changing source analysis, duration or audio. Step 7 adds dense
-eyelids, shared gaze, brows and pixel-derived teeth. Step 8 is next: authored
+eyelids, shared gaze, brows and pixel-derived teeth. Step 8's data model now
-parameter controls and their scoped recomputation.
+has stable feature identity, feature-level presence, explicit eye pairs and
+shared parameter definitions. A manifest can now supply known feature absence
+intervals through measurement and freeze. Automatic per-feature detection,
+parameter editing and scoped regeneration remain.
 
 ## What arthur is
 
@@ -326,9 +329,28 @@ minus paint. The fixed pixel thresholds remain provisional; step 8 exposes their
 parameters for tuning without changing the source track or picture timing.
 
 ### 8 — knobs
-The parameter UI, as leaf-addressed params in app-db
+Build the parameter model before its UI. Define each parameter once with its
-(`clip/:cid/params/:subject/:area` — see below), so the sync layer added later has
+default, validation, applicable area and regeneration dependencies. Store values
-nothing to retrofit.
+by stable subject and feature ID. Represent an eye pair as one group with one or
+two eye member IDs from the same subject; a profile view with one identified eye
+needs no invented partner. Each eye may override a pair value. Removing an eye
+from a pair materialises its effective values so playback does not change. Keep the
+existing frozen channels as renderer input; settings and provenance do not enter
+the render path.
+
+Carry feature-level presence through freeze and dense channel state. The same
+feature ID covers every observed run across occlusion; a missing measurement
+has no channel value on that frame. Full-face detection is the fallback mask
+until there is a feature-level detector or authored presence data. A shared gaze
+measurement may still feed two independently identified eyes. A small manifest
+annotation can supply feature absence intervals now: the loader expands them
+before measurement, so invalid eye landmarks are ignored and gaze uses the
+visible eye. This is an input format, not a control UI or an automatic detector.
+
+Use leaf-addressable settings under the clip, subject, feature and optional
+group. Retain source measurements so a setting change can regenerate affected
+channels without re-detecting footage. Time-varying parameter values and all
+parameter controls are deferred to the UI pass.
 
 ### 9 — backend
 Django project, the `clips` app, models for Project/Clip/Footage/Analysis/Leaf,
@@ -342,16 +364,14 @@ nobody should pre-empt by porting the old one.
 
 ## Two things to not foreclose
 
-The feature controls will later be rethought to handle more than one face,
+Feature controls will later handle more than one face and editing presence.
-periodic occlusion, stable identity across frames, and feature groups with their
+The underlying identity, occlusion and group association model begins in step 8:
-own parameters. That design can wait; two decisions here are free now and
-annoying to reverse:
 
-- **Presence is not visibility.** An occluded subject has *no value* on a frame,
+- **Presence is not visibility.** An occluded feature has *no value* on a frame,
-  which is different from a part being hidden. Give every dense block a
+  which is different from a part being hidden. Dense blocks carry a per-track,
-  `Uint8Array` state mask per frame and let it mean *absent* as well as hidden.
+  per-frame absence mask; `[:vis]` remains the sole hiding mechanism.
-- **Params carry a subject segment.** `clip/:cid/params/:subject/:area`, with one
+- **Params carry stable identity.** A subject and its features keep their IDs
-  subject today. Adding a path segment later touches every read and write.
+  across observation gaps. A run of visible frames is not a new identity.
 
 The identity tracker, when it comes, should use the same pattern the iris and brow
 correspondences already use: vote across every frame rather than trusting one.

frontend/README.md

@@ -110,11 +110,20 @@ landmarks and the teeth from source pixels, then freezes them into channels,
 and adds a button for the footage clip. Detection happens once when you load;
 playback only resolves channels and paints. Frames without a detection remain
 marked absent even though their neighbouring poses are used to condition the
-track. The stage stays 320×200 regardless of the footage dimensions. Real
+track. The scene now records stable subject and feature IDs and explicit eye
+pairs; dense channels can mark one feature absent while another is observed.
+Current MediaPipe loading supplies only the full-face detection mask. The stage
+stays 320×200 regardless of the footage dimensions. Real
 footage starts at the source picture rate. The **picture fps** buttons sample the
 frozen roto at lower rates while the source track, duration and audio clock stay
 unchanged. Picking frames to trace into cels is a separate future editing step.
 
+For known occlusion intervals, an extracted manifest may add
+`"feature-absence": {"eye-r": [[10, 14]]}`. Frame numbers are one-based and
+inclusive, matching PNG filenames. The eye remains the same feature when it
+reappears; the other eye and the mouth continue through the gap. This is an
+input annotation, with no UI for editing it yet.
+
 MediaPipe's JS, wasm and model are under `public/mediapipe/` and served locally.
 No CDN is used by this app. See that directory's README for provenance.
 

frontend/src/arthur/domain/feature.cljs

@@ -0,0 +1,85 @@
+(ns arthur.domain.feature
+  "Stable tracked identities and explicit eye-pair settings associations.
+  These maps are document data. Rendering only reads the nodes and channels."
+  (:require [arthur.domain.params :as params]))
+
+(defn group-for [scene feature-id]
+  (first (filter (fn [[_ group]] (some #{feature-id} (:members group)))
+                 (:groups scene))))
+
+(defn effective-params
+  "Resolve static settings for one feature. A future parameter channel can
+  replace a scalar at this boundary without changing feature or pair identity."
+  [scene feature-id]
+  (let [{:keys [subject area params] :as feature} (get-in scene [:features feature-id])
+        [_ group] (group-for scene feature-id)]
+    (when-not feature
+      (throw (ex-info "unknown feature" {:feature feature-id})))
+    (merge (params/for-area :subject)
+           (get-in scene [:subjects subject :params])
+           (params/for-area area)
+           (:params group)
+           params)))
+
+(defn remove-from-pair
+  "Keep the eye's current settings when its association is removed. Empty pairs
+  are removed; a one-eye pair remains valid and can acquire a partner later."
+  [scene feature-id]
+  (if-let [[group-id group] (group-for scene feature-id)]
+    (let [area (get-in scene [:features feature-id :area])
+          values (select-keys (effective-params scene feature-id)
+                              (keys (params/for-area area)))
+          members (vec (remove #{feature-id} (:members group)))]
+      (-> scene
+          (assoc-in [:features feature-id :params] values)
+          (update :groups (fn [groups]
+                            (if (seq members)
+                              (assoc-in groups [group-id :members] members)
+                              (dissoc groups group-id))))))
+    scene))
+
+(defn problems
+  "Check identity references and pair membership before storing a scene."
+  [scene]
+  (let [subjects (:subjects scene)
+        features (:features scene)
+        groups (:groups scene)
+        memberships (mapcat (comp :members val) groups)
+        node-owners (mapcat (comp :nodes val) features)]
+    (vec
+     (concat
+      (for [[id s] subjects :when (not= id (:id s))]
+        (str "subject " (pr-str id) " has a different :id"))
+      (for [[id s] subjects
+            :when (not (params/valid-settings? :subject (or (:params s) {})))]
+        (str "subject " (pr-str id) " has invalid settings"))
+      (for [[id f] features :when (not= id (:id f))]
+        (str "feature " (pr-str id) " has a different :id"))
+      (for [[id f] features :when (not (contains? subjects (:subject f)))]
+        (str "feature " (pr-str id) " has no subject"))
+      (for [[id f] features :when (not (contains? (disj params/areas :subject) (:area f)))]
+        (str "feature " (pr-str id) " has an unknown area"))
+      (for [[id f] features
+            :when (not (params/valid-settings? (:area f) (or (:params f) {})))]
+        (str "feature " (pr-str id) " has invalid settings for " (pr-str (:area f))))
+      (for [[id f] features node-id (:nodes f)
+            :when (not (contains? (:nodes scene) node-id))]
+        (str "feature " (pr-str id) " refers to missing node " (pr-str node-id)))
+      (for [[id n] (frequencies node-owners) :when (> n 1)]
+        (str "node " (pr-str id) " belongs to more than one feature"))
+      (for [[id g] groups :when (not= id (:id g))]
+        (str "group " (pr-str id) " has a different :id"))
+      (for [[id g] groups
+            :when (not (and (= :eye-pair (:kind g))
+                            (<= 1 (count (:members g)) 2)
+                            (= (count (:members g)) (count (distinct (:members g))))))]
+        (str "group " (pr-str id) " must be an eye pair of one or two distinct eyes"))
+      (for [[id g] groups
+            :when (not (params/valid-settings? :eye (or (:params g) {})))]
+        (str "group " (pr-str id) " has invalid eye settings"))
+      (for [[id g] groups member (:members g)
+            :let [f (get features member)]
+            :when (not (and f (= :eye (:area f)) (= (:subject g) (:subject f))))]
+        (str "group " (pr-str id) " has an eye from another subject or an unknown feature"))
+      (for [[id n] (frequencies memberships) :when (> n 1)]
+        (str "feature " (pr-str id) " belongs to more than one group"))))))

frontend/src/arthur/domain/params.cljs

@@ -0,0 +1,61 @@
+(ns arthur.domain.params
+  "Definitions for generated settings. Defaults, scope and value constraints live
+  here once; UI metadata and regeneration rules can be added to each entry.")
+
+(def definitions
+  {:anchor-avg    {:area :subject :default 2 :type :integer :min 0
+                   :affects #{:head :mouth :eye :brow :teeth}}
+   :contour-avg   {:area :subject :default 1 :type :integer :min 0
+                   :affects #{:mouth :eye :brow}}
+   :verts         {:area :mouth :default 8 :type :integer :min 4 :even? true}
+   :aperture-cut  {:area :mouth :default 0.12 :type :number :min 0 :max 1}
+   :eye-verts     {:area :eye :default 8 :type :integer :min 4 :even? true}
+   :blink-cut     {:area :eye :default 0.13 :type :number :min 0}
+   :gaze-gain     {:area :eye :default 1 :type :number :min 0}
+   :gaze-step     {:area :eye :default 0.08 :type :number :min 0}
+   :iris-size     {:area :eye :default 0.42 :type :number :min 0}
+   :lash-weight   {:area :eye :default 0.06 :type :number :min 0}
+   :pupil-size    {:area :eye :default 0.15 :type :number :min 0}
+   :brow-verts    {:area :brow :default 6 :type :integer :min 4 :even? true}
+   :brow-gain     {:area :brow :default 1 :type :number :min 0}
+   :brow-step     {:area :brow :default 0.08 :type :number :min 0}
+   :brow-weight   {:area :brow :default 0.05 :type :number :min 0}
+   :cavity-erode  {:area :teeth :default 0.18 :type :number :min 0}
+   :tongue-reject {:area :teeth :default 0.18 :type :number :min 0}
+   :blob-grow     {:area :teeth :default 0 :type :integer}
+   :top-bias      {:area :teeth :default 0.6 :type :number}
+   :teeth-verts   {:area :teeth :default 10 :type :integer :min 4}
+   :min-area      {:area :teeth :default 12 :type :integer :min 0}
+   :teeth-on      {:area :teeth :default 0.12 :type :number :min 0}
+   :teeth-smooth  {:area :teeth :default 1 :type :integer :min 0}})
+
+(def defaults
+  (into {} (map (fn [[id spec]] [id (:default spec)])) definitions))
+
+(def areas #{:subject :mouth :eye :brow :teeth})
+
+(defn affected-areas [id]
+  (when-let [spec (get definitions id)]
+    (or (:affects spec) #{(:area spec)})))
+
+(defn for-area [wanted-area]
+  (into {} (keep (fn [[id {:keys [area default]}]]
+                   (when (= wanted-area area) [id default])))
+        definitions))
+
+(defn valid-value? [id value]
+  (when-let [{:keys [type min max] must-even? :even?} (get definitions id)]
+    (and (case type
+           :integer (integer? value)
+           :number (number? value)
+           false)
+         (or (nil? min) (<= min value))
+         (or (nil? max) (<= value max))
+         (or (not must-even?) (and (integer? value) (even? value))))))
+
+(defn valid-settings? [area settings]
+  (and (map? settings)
+       (every? (fn [[id value]]
+                 (and (= area (get-in definitions [id :area]))
+                      (valid-value? id value)))
+               settings)))

frontend/src/arthur/domain/scene.cljs

@@ -28,6 +28,7 @@
   vector of numbers, and they read the same way, which is what makes freezing
   fill in the same channel rather than convert into a second format."
   (:require [arthur.domain.channel :as ch]
+            [arthur.domain.feature :as feature]
             [arthur.domain.node :as node]
             [arthur.domain.palette :as pal]))
 
@@ -451,8 +452,8 @@
           (into (for [[id n] nodes
                       p      (node/problems n)]
                   (str "node " (pr-str id) ": " p)))
+          (into (feature/problems scene))
           (into (try
                   (doall (map #(depth nodes %) (keys nodes)))
                   nil
                   (catch :default e [(ex-message e)])))))))
-

frontend/src/arthur/events/footage.cljs

@@ -58,7 +58,8 @@
 (defn- build-clip [manifest {:keys [dense detected dimensions interior missing first-real]}]
   (let [[w h] dimensions
         frozen (take/footage manifest {:dense dense :detected detected
-                                      :dimensions dimensions :interior interior})
+                                      :dimensions dimensions :interior interior
+                                      :presence (:presence manifest)})
         scene (:scene frozen)]
     (assoc (select-keys scene [:fps :frames :width :height])
            :display-fps (:fps scene)

frontend/src/arthur/flow/condition/eyes.cljs

@@ -13,20 +13,47 @@
      :y (/ (+ (:y a) (:y b)) 2)
      :w (distance a b)}))
 
-(defn- mean-width [lids]
+(defn- hold-observed
-  (/ (reduce + (map (comp :w socket) lids)) (count lids)))
+  "Give temporal filters real samples at every index. The freeze mask still
+  marks the gap absent; these held values are only numeric placeholders."
+  [values observed fallback]
+  (let [values (vec values)
+        seed (or (first (keep-indexed (fn [i v]
+                                       (when (and (nth observed i) (some? v)) v))
+                                     values))
+                 fallback (first values))]
+    (loop [f 0 last-value seed out []]
+      (if (= f (count values))
+        out
+        (let [v (nth values f)
+              next-value (if (and (nth observed f) (some? v)) v last-value)]
+          (recur (inc f) next-value (conj out next-value)))))))
+
+(defn- mean-width [lids observed]
+  (let [valid (seq (keep-indexed (fn [i lid] (when (nth observed i) lid)) lids))
+        rings (or valid lids)]
+    (/ (reduce + (map (comp :w socket) rings)) (count rings))))
 
 (defn apply-defaults
   [{:keys [fps contour-avg blink-cut gaze-gain gaze-step iris-size
-           lash-weight pupil-size]} {:keys [lid-r lid-l open-r open-l gaze]
+           lash-weight pupil-size]} {:keys [lid-r lid-l open-r open-l gaze
+                                            observed-r observed-l gaze-observed]
                                     :as measured}]
   (let [fps (or fps 30)
-        right (condition/contours {:contour-avg contour-avg} lid-r)
+        observed-r (or observed-r (vec (repeat (count lid-r) true)))
-        left (condition/contours {:contour-avg contour-avg} lid-l)
+        observed-l (or observed-l (vec (repeat (count lid-l) true)))
-        w-r (mean-width right)
+        gaze-observed (or gaze-observed (vec (repeat (count gaze) true)))
-        w-l (mean-width left)
+        right (condition/contours {:contour-avg contour-avg}
+                                  (hold-observed lid-r observed-r nil))
+        left (condition/contours {:contour-avg contour-avg}
+                                 (hold-observed lid-l observed-l nil))
+        w-r (mean-width right observed-r)
+        w-l (mean-width left observed-l)
         w (/ (+ w-r w-l) 2)
-        rest (condition/median-point gaze)
+        valid-gaze (seq (keep-indexed (fn [i point]
+                                       (when (nth gaze-observed i) point)) gaze))
+        gaze (hold-observed gaze gaze-observed {:x 0 :y 0})
+        rest (if valid-gaze (condition/median-point valid-gaze) {:x 0 :y 0})
         px (mapv (fn [{:keys [x y]}]
                    {:x (* (- x (:x rest)) gaze-gain w)
                     :y (* (- y (:y rest)) gaze-gain w)}) gaze)
@@ -42,8 +69,10 @@
            :lid-r right :lid-l left
            :lash-r (mapv #(ring/offset-ring % (* lash-weight w-r)) right)
            :lash-l (mapv #(ring/offset-ring % (* lash-weight w-l)) left)
-           :shut-r (condition/resolve-blink open-r blink)
+           :shut-r (condition/resolve-blink
-           :shut-l (condition/resolve-blink open-l blink)
+                    (hold-observed open-r observed-r nil) blink)
+           :shut-l (condition/resolve-blink
+                    (hold-observed open-l observed-l nil) blink)
            :iris-r (centres right) :iris-l (centres left)
            :radius-r (* 0.5 iris-size w-r)
            :radius-l (* 0.5 iris-size w-l)

frontend/src/arthur/flow/freeze.cljs

@@ -76,10 +76,9 @@
 
   `:ctor` makes the array — a function and not a type, because `new` is not
   something a value can carry — `:scale` is the fixed-point scale or nil, and
-  `:absent` an optional per-frame predicate. Absence is the SUBJECT's, not the
+  `:absent` an optional (track, frame) predicate. The state is per track, so one
-  part's, so it is written into every track of the block: the mouth outline and
+  occluded eye can be absent while its partner still has a value. A full-face
-  the mouth interior are one face, and a frame that face was not on has no pose
+  miss marks every track absent.
-  for either.
 
   Written with `dotimes` and `aset` rather than as a fold, and that is the
   exception rather than the rule in this codebase: the destination is a typed
@@ -110,7 +109,7 @@
                                              :track i :frame f :component k})))
                           q)
                         v))))
-            (when (and state (absent f))
+            (when (and state (absent i f))
               (aset state (+ (* i nf) f) ch/absent-bit))))))
     {:data data :state state :stride stride :frames nf :scale scale
      :offsets (mapv #(* % nf stride) (range n))}))
@@ -372,16 +371,24 @@
                      {:by by :analysis analysis
                       :params (merge {:anchor-avg anchor-avg :contour-avg contour-avg}
                                      extra)})
-        eye-block (pack {:ctor #(js/Int16Array. %) :scale geom-scale :absent absent}
+        eye-block (pack {:ctor #(js/Int16Array. %) :scale geom-scale
+                         :absent (when absent (fn [i f]
+                                                (absent (if (< i 2) :eye-r :eye-l) f)))}
                         (mapv #(rings->flat % eye-verts)
                               [(:lash-r eyes) (:lid-r eyes)
                                (:lash-l eyes) (:lid-l eyes)]))
-        iris-block (pack {:ctor #(js/Float32Array. %) :absent absent}
+        iris-block (pack {:ctor #(js/Float32Array. %)
+                          :absent (when absent (fn [i f]
+                                                 (absent (if (zero? i) :eye-r :eye-l) f)))}
                          [(:iris-r eyes) (:iris-l eyes)])
-        brow-block (pack {:ctor #(js/Int16Array. %) :scale geom-scale :absent absent}
+        brow-block (pack {:ctor #(js/Int16Array. %) :scale geom-scale
+                          :absent (when absent (fn [i f]
+                                                 (absent (if (zero? i) :brow-r :brow-l) f)))}
                          (mapv #(rings->flat % brow-verts)
                                [(:ring-r brows) (:ring-l brows)]))
-        brow-pos-block (pack {:ctor #(js/Float32Array. %) :absent absent}
+        brow-pos-block (pack {:ctor #(js/Float32Array. %)
+                              :absent (when absent (fn [i f]
+                                                     (absent (if (zero? i) :brow-r :brow-l) f)))}
                               [(:pos-r brows) (:pos-l brows)])
         eye-node (fn [id z track]
                    {:id id :name (clojure.core/name id) :kind :poly :parent :head :z z
@@ -434,11 +441,11 @@
   "Freeze the pixel-derived radial contour under the mouth cavity. Missing
   contours use the dense block's absence bit; contrast decides editable :vis."
   [name {:keys [analysis teeth-verts cavity-erode tongue-reject blob-grow
-                top-bias teeth-on teeth-smooth]} detected
+                top-bias teeth-on teeth-smooth]} absent-feature
    {:keys [contours shown]}]
   (let [key (str name "/teeth")
-        absent #(or (nil? (nth contours %))
+        absent (fn [_ f] (or (nil? (nth contours f))
-                    (and detected (not (nth detected %))))
+                             (and absent-feature (absent-feature :teeth f))))
         empty-points (vec (repeat (* 2 teeth-verts) 0))
         values (mapv (fn [ring]
                        (if ring
@@ -495,7 +502,9 @@
     :eyes :brows   the CONDITIONED head-local feature measurements
     :teeth         optional pixel-derived, conditioned radial contour
     :aperture      head-local aperture per frame
-    :detected      optional per-frame booleans; absent frames get the state mask
+    :detected      optional per-frame face booleans
+    :presence      optional feature-id -> per-frame booleans; false means an
+                   unobserved feature, even if the rest of the face was found
 
   The node tree is the one docs/animation-model.md specifies, and the two groups
   are two different things wanting the same transform:
@@ -522,21 +531,32 @@
   provenance having to be rebuilt."
   [{:keys [name fps stage expose verts aperture-cut head kept
            analysis anchor-avg contour-avg] :as params}
-   {:keys [transforms outer inner detected eyes brows teeth] :as inputs}]
+   {:keys [transforms outer inner detected presence eyes brows teeth] :as inputs}]
   (let [nf      (count outer)
-        absent  (when detected #(not (nth detected % true)))
+        _       (doseq [[id track] presence]
+                  (when (not= nf (count track))
+                    (throw (ex-info "feature presence track must match the clip"
+                                    {:feature id :frames nf :actual (count track)}))))
+        absent  (when (or detected presence)
+                  (fn [id f]
+                    (or (and detected (not (nth detected f true)))
+                        (and (contains? presence id)
+                             (not (nth (get presence id) f))))))
+        head-absent (when detected (fn [_ f] (not (nth detected f true))))
         geom-k  (str name "/geom")
         head-k  #(str name "/head-" %)
         prov    (fn [by extra]
                   {:by by :analysis analysis
                    :params (merge {:anchor-avg anchor-avg} extra)})
-        rings   (pack {:ctor #(js/Int16Array. %) :scale geom-scale :absent absent}
+        rings   (pack {:ctor #(js/Int16Array. %) :scale geom-scale
+                       :absent (when absent (fn [_ f] (absent :mouth f)))}
                       [(rings->flat outer verts) (rings->flat inner verts)])
         ;; The anchor, inverted and split into its three components. Three blocks
         ;; and not one: they are three channels, they have three strides, and a
         ;; single block would need a per-component offset table to say so.
         inv     (mapv invert transforms)
-        xf      (fn [f] (pack {:ctor #(js/Float32Array. %) :absent absent} [(mapv f inv)]))
+        xf      (fn [f] (pack {:ctor #(js/Float32Array. %) :absent head-absent}
+                              [(mapv f inv)]))
         pos     (xf (fn [t] [(:tx t) (:ty t)]))
         rot     (xf (fn [t] [(:theta t)]))
         scale   (xf (fn [t] [(:s t) (:s t)]))
@@ -549,10 +569,31 @@
         anchor-prov (prov :anchor/similarity nil)
         roto    (fn [by] (prov by {:verts verts :contour-avg contour-avg}))
         features (when (and eyes brows) (feature-parts name absent params inputs))
-        interior (when teeth (interior-part name params detected teeth))
+        interior (when teeth (interior-part name params absent teeth))
         scene   {:name   name
                  :frames nf
                  :fps    fps
+                 ;; A gap changes channel state, never these IDs or pair links.
+                 :subjects {:face-1 {:id :face-1 :params {}}}
+                 :features (merge
+                            {:mouth {:id :mouth :subject :face-1 :area :mouth
+                                     :nodes [:mouth :mouth-in] :params {}}}
+                            (when features
+                              {:eye-r {:id :eye-r :subject :face-1 :area :eye
+                                       :nodes [:eye-r :eye-r-in :iris-r :pupil-r] :params {}}
+                               :eye-l {:id :eye-l :subject :face-1 :area :eye
+                                       :nodes [:eye-l :eye-l-in :iris-l :pupil-l] :params {}}
+                               :brow-r {:id :brow-r :subject :face-1 :area :brow
+                                        :nodes [:brow-r] :params {}}
+                               :brow-l {:id :brow-l :subject :face-1 :area :brow
+                                        :nodes [:brow-l] :params {}}})
+                            (when teeth
+                              {:teeth {:id :teeth :subject :face-1 :area :teeth
+                                       :nodes [:teeth] :params {}}}))
+                 :groups (if features
+                           {:eyes-1 {:id :eyes-1 :kind :eye-pair :subject :face-1
+                                     :members [:eye-r :eye-l] :params {}}}
+                           {})
                  ;; Stage dimensions, on the clip and not on the footage. See
                  ;; face-placement: nothing below here knows the frame size.
                  :width  (first stage)
@@ -591,6 +632,10 @@
                                                  (prov :roto/mouth-aperture
                                                        {:aperture-cut aperture-cut}))}}}
                  (:nodes features) (:nodes interior))}
+        presence-check (doseq [id (keys presence)]
+                         (when-not (contains? (:features scene) id)
+                           (throw (ex-info "presence track names no feature in this scene"
+                                           {:feature id :features (keys (:features scene))}))))
         ;; Tier 2, behind a handle. The keys are descriptive because there is no
         ;; hashing yet; they become the blocks' sha256 when the backend arrives
         ;; and nothing above here changes, which is the point of a handle.

frontend/src/arthur/flow/ingest.cljs

@@ -2,6 +2,32 @@
   "Read a pre-extracted take. The manifest owns timing and the exact frame count."
   (:require [clojure.string :as str]))
 
+(defn feature-presence
+  "Expand one-based, inclusive absence intervals from a manifest into boolean
+  observation tracks. Unknown features are rejected by freeze, where the scene
+  knows its feature IDs."
+  [frames absence]
+  (when (some? absence)
+    (when-not (map? absence)
+      (throw (ex-info "feature-absence must be a map of feature IDs to intervals"
+                      {:feature-absence absence})))
+    (into {}
+          (map (fn [[id intervals]]
+                 (when-not (and (keyword? id) (sequential? intervals)
+                                (every? (fn [span]
+                                          (and (vector? span) (= 2 (count span))
+                                               (every? integer? span)
+                                               (<= 1 (first span) (second span) frames)))
+                                        intervals))
+                   (throw (ex-info "feature-absence intervals must be [first last] source frames"
+                                   {:feature id :intervals intervals :frames frames})))
+                 [id (mapv (fn [frame]
+                             (not-any? (fn [[first-frame last-frame]]
+                                         (<= first-frame frame last-frame))
+                                       intervals))
+                           (range 1 (inc frames)))]))
+          absence)))
+
 (defn- valid-manifest [m]
   (let [fps (js/Number (:fps m))
         frames (js/Number (:frames m))]
@@ -10,7 +36,8 @@
                    (string? (:dir m)) (seq (:dir m))
                    (string? (:audio m)) (seq (:audio m)))
       (throw (ex-info "manifest.json needs fps, frames (1–900), dir and audio" {:manifest m})))
-    (assoc m :fps fps :frames frames)))
+    (assoc m :fps fps :frames frames
+           :presence (feature-presence frames (:feature-absence m)))))
 
 (defn manifest!
   [path]

frontend/src/arthur/flow/measure/eyes.cljs

@@ -16,22 +16,38 @@
           (geom/apply-sim-all tf (anchor/pick frame table aspect)))
         dense transforms))
 
-(defn- iris-pair [iris-a iris-b corners-r]
+(defn- observed-track [frames detected presence feature-id]
+  (let [part (get presence feature-id)]
+    (mapv (fn [f]
+            (and (or (nil? detected) (nth detected f))
+                 (or (nil? part) (nth part f))))
+          (range frames))))
+
+(defn- iris-pair [iris-a iris-b corners-r corners-l observed-r observed-l]
   (when iris-a
+    ;; An occluded eye may still get a plausible iris from MediaPipe. It gets no
+    ;; vote. Either eye can establish the pairing when its partner is absent.
     (let [votes (reduce +
-                        (map (fn [a b [outer inner]]
+                        (for [f (range (count iris-a))
-                               (if (< (distance (first a) (midpoint outer inner))
+                              [seen corners sign] [[(nth observed-r f) (nth corners-r f) 1]
-                                      (distance (first b) (midpoint outer inner)))
+                                                   [(nth observed-l f) (nth corners-l f) -1]]
-                                 1 -1))
+                              :when seen]
-                             iris-a iris-b corners-r))]
+                          (let [a (first (nth iris-a f))
+                                b (first (nth iris-b f))
+                                [outer inner] corners]
+                            (if (< (distance a (midpoint outer inner))
+                                   (distance b (midpoint outer inner)))
+                              sign (- sign)))))]
       (if (pos? votes) {:right :a :left :b} {:right :b :left :a}))))
 
 (defn measure
   "All rings and iris coordinates use the anchor's head-local image-height unit.
   Eye openness and gaze use each eye's rigid corner width as their unit. Iris
-  block identity is voted from proximity over the whole take."
+  block identity is voted from observed eyes over the whole take."
-  [{:keys [aspect debug?]} {:keys [dense transforms]}]
+  [{:keys [aspect debug?]} {:keys [dense transforms detected presence]}]
   (let [track (partial local-track dense transforms aspect)
+        observed-r (observed-track (count dense) detected presence :eye-r)
+        observed-l (observed-track (count dense) detected presence :eye-l)
         lid-r (track lm/EYE-R-RING)
         lid-l (track lm/EYE-L-RING)
         corners-r (track lm/EYE-R-CORNERS)
@@ -41,7 +57,7 @@
         has-iris? (every? #(> (count %) (last lm/IRIS-B)) dense)
         iris-a (when has-iris? (track lm/IRIS-A))
         iris-b (when has-iris? (track lm/IRIS-B))
-        pairing (iris-pair iris-a iris-b corners-r)
+        pairing (iris-pair iris-a iris-b corners-r corners-l observed-r observed-l)
         iris-for (fn [side f]
                    (first (nth (if (= (get pairing side) :a) iris-a iris-b) f)))
         openness (fn [corners lids]
@@ -57,17 +73,25 @@
                                          iris (iris-for side f)]
                                      {:x (/ (- (:x iris) (:x c)) w)
                                       :y (/ (- (:y iris) (:y c)) w)}))
-                             r (one :right corners-r)
+                             r (when (nth observed-r f) (one :right corners-r))
-                             l (one :left corners-l)]
+                             l (when (nth observed-l f) (one :left corners-l))]
-                         (midpoint r l)))
+                         (cond
+                           (and r l) (midpoint r l)
+                           r r
+                           l l
+                           :else nil)))
                      (range (count dense)))
                (vec (repeat (count dense) {:x 0 :y 0})))]
     {:lid-r lid-r :lid-l lid-l
      :corners-r corners-r :corners-l corners-l
      :open-r (openness corners-r lids-r)
      :open-l (openness corners-l lids-l)
+     :observed-r observed-r :observed-l observed-l
+     :gaze-observed (if pairing (mapv #(or %1 %2) observed-r observed-l)
+                        (vec (repeat (count dense) true)))
      :gaze gaze :has-iris? (boolean pairing)
      :iris-pair pairing
      :debug (when debug? {:open-r (openness corners-r lids-r)
                           :open-l (openness corners-l lids-l)
+                          :observed-r observed-r :observed-l observed-l
                           :gaze gaze :iris-pair pairing})}))

frontend/src/arthur/flow/take.cljs

@@ -9,24 +9,25 @@
             [arthur.flow.measure.brows :as brows]
             [arthur.flow.measure.eyes :as eyes]
             [arthur.flow.measure.interior :as interior]
-            [arthur.flow.measure.mouth :as mouth]))
+            [arthur.flow.measure.mouth :as mouth]
+            [arthur.domain.params :as params]))
 
-(def knobs
+(def knobs params/defaults)
-  {:anchor-avg 2 :contour-avg 1 :verts 8 :aperture-cut 0.12
-   :eye-verts 8 :blink-cut 0.13 :gaze-gain 1 :gaze-step 0.08
-   :iris-size 0.42 :lash-weight 0.06 :pupil-size 0.15
-   :brow-verts 6 :brow-gain 1 :brow-step 0.08 :brow-weight 0.05
-   :cavity-erode 0.18 :tongue-reject 0.18 :blob-grow 0 :top-bias 0.6
-   :teeth-verts 10 :min-area 12 :teeth-on 0.12 :teeth-smooth 1})
 
 (defn measure
   "Condition the anchor before measuring rings through it."
-  [{:keys [aspect] :as params} {:keys [dense detected interior]}]
+  [{:keys [aspect] :as params} {:keys [dense detected interior presence]}]
-  (let [fitted   (anchor/fit {:aspect aspect} {:dense dense})
+  (let [_        (doseq [[id track] presence]
+                   (when (not= (count dense) (count track))
+                     (throw (ex-info "feature presence track must match the source"
+                                     {:feature id :frames (count dense)
+                                      :actual (count track)}))))
+        fitted   (anchor/fit {:aspect aspect} {:dense dense})
         anchored (condition/anchor params fitted)
         rings    (mouth/measure {:aspect aspect}
                                 {:dense dense :transforms (:transforms anchored)})
-        eye-data (eyes/measure params {:dense dense :transforms (:transforms anchored)})
+        eye-data (eyes/measure params {:dense dense :transforms (:transforms anchored)
+                                      :detected detected :presence presence})
         brow-data (brows/measure params {:dense dense :transforms (:transforms anchored)})
         inner-data (when interior
                      (condition-interior/apply-defaults
@@ -41,7 +42,8 @@
            :eyes (condition-eyes/apply-defaults params eye-data)
            :brows (condition-brows/apply-defaults params brow-data eye-data)
            :teeth inner-data
-           :detected detected)))
+           :detected detected
+           :presence presence)))
 
 (defn build
   [params inputs]
@@ -51,11 +53,12 @@
   "A real manifest and its detected landmarks through the same measurement and
   freeze path as the synthetic take. The source cadence stays in :fps; picture
   sampling is a root time map applied only after this artifact exists."
-  [manifest {:keys [dense detected dimensions interior]}]
+  [manifest {:keys [dense detected dimensions interior presence]}]
   (let [[w h] dimensions
         params (merge knobs
                       {:name "footage" :fps (:fps manifest) :aspect (/ w h)
                        :stage [320 200] :fit-motion? true
                        :expose 1 :head :as-filmed
                        :analysis (str "mediapipe:1.0.1/" (:source manifest))})]
-    (build params {:dense dense :detected detected :interior interior})))
+    (build params {:dense dense :detected detected :interior interior
+                   :presence presence})))

frontend/test/arthur/domain/feature_test.cljs

@@ -0,0 +1,54 @@
+(ns arthur.domain.feature-test
+  (:require [cljs.test :refer [deftest is]]
+            [arthur.domain.feature :as feature]
+            [arthur.domain.params :as params]))
+
+(defn- nine-eyes []
+  (let [people (mapv #(keyword (str "person-" %)) (range 5))
+        eyes (mapv #(keyword (str "eye-" %)) (range 9))
+        members [[:eye-0 :eye-1] [:eye-2 :eye-3] [:eye-4 :eye-5]
+                 [:eye-6 :eye-7] [:eye-8]]]
+    {:subjects (into {} (map (fn [id] [id {:id id :params {}}]) people))
+     :features (into {} (map-indexed (fn [i id]
+                                      [id {:id id :subject (nth people (quot i 2))
+                                           :area :eye :nodes [] :params {}}]) eyes))
+     :groups (into {} (map-indexed (fn [i ids]
+                                     (let [id (keyword (str "pair-" i))]
+                                       [id {:id id :kind :eye-pair
+                                            :subject (nth people i)
+                                            :members ids :params {}}])) members))
+     :nodes {}}))
+
+(deftest five-people-can-have-nine-identified-eyes
+  (let [scene (nine-eyes)]
+    (is (empty? (feature/problems scene)))
+    (is (= [:eye-8] (get-in scene [:groups :pair-4 :members])))
+    (is (= :person-4 (get-in scene [:features :eye-8 :subject])))))
+
+(deftest parameter-definitions-drive-the-current-take-defaults
+  (is (= #{:eye} (params/affected-areas :blink-cut)))
+  (is (= #{:head :mouth :eye :brow :teeth}
+         (params/affected-areas :anchor-avg)))
+  (is (params/valid-settings? :eye {:blink-cut 0.2}))
+  (is (not (params/valid-settings? :eye {:verts 8}))))
+
+(deftest pair-values-are-shared-but-eye-overrides-are-independent
+  (let [scene (-> (nine-eyes)
+                  (assoc-in [:groups :pair-0 :params :blink-cut] 0.2)
+                  (assoc-in [:features :eye-1 :params :blink-cut] 0.3))]
+    (is (= 0.2 (:blink-cut (feature/effective-params scene :eye-0))))
+    (is (= 0.3 (:blink-cut (feature/effective-params scene :eye-1))))
+    (is (= (:eye-verts params/defaults)
+           (:eye-verts (feature/effective-params scene :eye-0))))
+    (let [before (feature/effective-params scene :eye-0)
+          unpaired (feature/remove-from-pair scene :eye-0)]
+      (is (= before (feature/effective-params unpaired :eye-0)))
+      (is (= [:eye-1] (get-in unpaired [:groups :pair-0 :members])))
+      (is (empty? (feature/problems unpaired))))))
+
+(deftest a-pair-cannot-cross-subjects-or-own-an-eye-twice
+  (let [scene (nine-eyes)]
+    (is (seq (feature/problems
+              (assoc-in scene [:groups :pair-0 :members] [:eye-0 :eye-2]))))
+    (is (seq (feature/problems
+              (assoc-in scene [:groups :pair-1 :members] [:eye-0 :eye-3]))))))

frontend/test/arthur/flow/eye_occlusion_test.cljs

@@ -0,0 +1,67 @@
+(ns arthur.flow.eye-occlusion-test
+  (:require [cljs.test :refer [deftest is]]
+            [arthur.demo.take :as take]
+            [arthur.domain.channel :as ch]
+            [arthur.domain.landmarks :as lm]
+            [arthur.domain.palette :as pal]
+            [arthur.domain.scene :as scene]
+            [arthur.flow.condition.eyes :as condition-eyes]
+            [arthur.flow.ingest :as ingest]
+            [arthur.flow.measure.eyes :as eyes]
+            [arthur.flow.take :as flow-take]))
+
+(deftest an-unobserved-eye-cannot-steer-the-shared-gaze
+  (let [frame 11
+        params (assoc take/params :aspect 1)
+        presence (ingest/feature-presence take/frames {:eye-r [[10 14]]})
+        inputs {:dense @take/analysis
+                :transforms (:transforms @take/measured)
+                :presence presence}
+        clean (eyes/measure params inputs)
+        iris-ids (if (= :a (:right (:iris-pair clean))) lm/IRIS-A lm/IRIS-B)
+        bad-ids (distinct (concat lm/EYE-R-RING iris-ids))
+        corrupted (mapv (fn [i source]
+                          (if (<= 9 i 13)
+                            (reduce (fn [points id]
+                                      (update-in points [id :x] + 0.25))
+                                    source bad-ids)
+                            source))
+                        (range take/frames) @take/analysis)
+        masked (eyes/measure params (assoc inputs :dense corrupted))
+        unmasked (eyes/measure params
+                               (dissoc (assoc inputs :dense corrupted) :presence))
+        drawn-clean (condition-eyes/apply-defaults params clean)
+        drawn-masked (condition-eyes/apply-defaults params masked)]
+    (is (= lm/NUM-LANDMARKS (count (nth corrupted frame))))
+    (is (:has-iris? masked) (str "pair " (:iris-pair masked)))
+    (is (false? (nth (:observed-r masked) frame)))
+    (is (true? (nth (:observed-l masked) frame)))
+    (is (= (nth (:gaze clean) frame) (nth (:gaze masked) frame)))
+    (is (not= (nth (:gaze clean) frame) (nth (:gaze unmasked) frame)))
+    (is (= (nth (:lid-r drawn-clean) frame)
+           (nth (:lid-r drawn-masked) frame)))
+    (is (= (nth (:iris-l drawn-clean) frame)
+           (nth (:iris-l drawn-masked) frame)))))
+
+(deftest the-full-take-hides-only-the-annotated-eye-and-returns-to-the-same-id
+  (let [presence (ingest/feature-presence take/frames {:eye-r [[10 14]]})
+        {:keys [scene store]} (flow-take/build
+                               (assoc take/params :aspect 1 :name "observed-gap")
+                               {:dense @take/analysis :presence presence})
+        sample (fn [id frame]
+                 (ch/value-at (get-in scene [:nodes id :channels [:geom :pts]])
+                              frame store))]
+    (is (empty? (scene/problems scene)))
+    (is (= [:eye-r :eye-l] (get-in scene [:groups :eyes-1 :members])))
+    (doseq [f (range 9 14)]
+      (is (ch/nothing? (sample :eye-r f)))
+      (is (not (ch/nothing? (sample :eye-l f))))
+      (is (not (ch/nothing? (sample :mouth f)))))
+    (let [drawn (into #{} (map :node)
+                      ((scene/resolver scene store pal/index-of) 11))]
+      (is (not (contains? drawn :eye-r)))
+      (is (not (contains? drawn :iris-r)))
+      (is (contains? drawn :eye-l))
+      (is (contains? drawn :mouth)))
+    (is (not (ch/nothing? (sample :eye-r 8))))
+    (is (not (ch/nothing? (sample :eye-r 14))))))

frontend/test/arthur/flow/freeze_test.cljs

@@ -381,6 +381,27 @@
 ;; ---------------------------------------------------------------------------
 ;; presence is not visibility
 
+(deftest an-eye-can-disappear-and-return-under-the-same-identity
+  (let [gap (set (range 40 60))
+        presence {:eye-r (mapv #(not (contains? gap %)) (range take/frames))}
+        c (freeze/clip (assoc take/params :name "one-eye-gappy")
+                       (assoc @take/measured :presence presence))
+        sc (:scene c)
+        at (fn [id f]
+             (ch/value-at (get-in sc [:nodes id :channels [:geom :pts]]) f (:store c)))]
+    (is (= [:eye-r :eye-l] (get-in sc [:groups :eyes-1 :members])))
+    (is (= :eye-r (get-in sc [:features :eye-r :id])))
+    (is (not (ch/nothing? (at :eye-r 39))))
+    (is (ch/nothing? (at :eye-r 50)))
+    (is (not (ch/nothing? (at :eye-r 60))))
+    (is (not (ch/nothing? (at :eye-l 50))))
+    (is (not (ch/nothing? (at :mouth 50))))
+    (let [drawn-nodes (into #{} (map :node) ((scene/resolver sc (:store c) pal/index-of) 50))]
+      (is (not (contains? drawn-nodes :eye-r)))
+      (is (contains? drawn-nodes :eye-l))
+      (is (contains? drawn-nodes :mouth)))
+    (is (empty? (scene/problems sc)))))
+
 (deftest an-undetected-frame-has-no-pose-at-all
   ;; A subject that was not on the frame has NO VALUE, which is different from a
   ;; part being switched off. The mask lands on every block of the freeze, so an
@@ -447,10 +468,15 @@
   ;; beat are not. This is the numeric half of step 5's done-criterion; the other
   ;; half is a picture and lives in test/browser/take.mjs.
   (let [locked (freeze/head-mode {:mode :locked} @clip)
-        shot   (fn [f] (render locked f))
+        shot   (fn [f]
+                 (let [r (raster/make W H)
+                       mouth (filter #(= :mouth (:node %)) (ops-at locked f))]
+                   (raster/clear! r (get pal/index-of :bg))
+                   (raster/draw-ops! r mouth)
+                   (vec (array-seq (:buf r)))))
         differ (fn [a b] (count (remove true? (map = a b))))]
-    ;; Beat 1 is wide open and beat 3 is shut. Rendered with the head LOCKED, so
+    ;; Beat 1 is wide open and beat 3 is shut. Compare only the mouth outline:
-    ;; what differs is articulation and not the head wandering across the stage.
+    ;; eyes and brows added in step 7 have their own motion within the beat.
     (is (> (differ (shot 10) (shot 28)) 300)
         "the open and the shut mouth rasterise the same")
     ;; And within a beat, on the exposure grid, it holds.

frontend/test/arthur/flow/ingest_test.cljs

@@ -0,0 +1,15 @@
+(ns arthur.flow.ingest-test
+  (:require [cljs.test :refer [deftest is]]
+            [arthur.flow.ingest :as ingest]))
+
+(deftest feature-absence-uses-the-source-frame-numbers
+  (let [tracks (ingest/feature-presence
+                8 {:eye-r [[3 5]] :eye-l [[1 1] [8 8]]})]
+    (is (= [true true false false false true true true] (:eye-r tracks)))
+    (is (= [false true true true true true true false] (:eye-l tracks)))))
+
+(deftest malformed-feature-absence-fails-before-analysis
+  (doseq [bad [{:eye-r [[0 2]]}
+               {:eye-r [[6 9]]}
+               {:eye-r [[5 3]]}]]
+    (is (thrown? ExceptionInfo (ingest/feature-presence 8 bad)))))

js/app.js

@@ -51,6 +51,7 @@ const state = {
   lead: 0,                  // performance-track offset in frames
   exposure: 1,              // 1 = on 1s, 2 = on 2s. Picture holds; audio does not.
   interior: null,           // per-frame teeth measurement from image content
+  detected: null,           // per-frame: did MediaPipe really see a face here?
   teeth: null,              // resolved per-frame {show, t} after knobs
   eyes: null,               // resolved per-frame lid rings, shut flags, iris discs
   brows: null,              // resolved per-frame brow rings after quantised raise
@@ -167,18 +168,29 @@ async function initLandmarker() {
 async function detectAll(images) {
   const lm = await initLandmarker();
   const cv = document.createElement('canvas');
-  const dense = [], missing = [];
+  const dense = [], missing = [], detected = [];
   for (let i = 0; i < images.length; i++) {
     const im = images[i];
     cv.width = im.naturalWidth; cv.height = im.naturalHeight;
     cv.getContext('2d').drawImage(im, 0, 0);
     const out = lm.detect(cv);
-    if (out.faceLandmarks && out.faceLandmarks.length) dense.push(out.faceLandmarks[0]);
+    if (out.faceLandmarks && out.faceLandmarks.length) {
-    else { missing.push(i); dense.push(dense.length ? dense[dense.length - 1] : null); }
+      dense.push(out.faceLandmarks[0]); detected.push(true);
+    } else {
+      missing.push(i); detected.push(false);
+      dense.push(dense.length ? dense[dense.length - 1] : null);
+    }
     if (i % 4 === 0) status(`detecting… ${i + 1}/${images.length}`);
   }
-  if (dense[0] === null) throw new Error('no face found in the first frame');
+  // A gap mid-take holds the previous frame, but a gap at the TOP has no
-  return { dense, missing };
+  // previous to hold - a face that starts occluded or walks in late left
+  // leading nulls, and this used to throw and discard the whole take. Back-fill
+  // from the first real detection: the mirror of hold-previous, and the only
+  // fill that is a real pose from this take rather than an invention.
+  const firstReal = dense.findIndex((d) => d !== null);
+  if (firstReal < 0) throw new Error('no face found in any frame — check framing and light');
+  for (let i = 0; i < firstReal; i++) dense[i] = dense[firstReal];
+  return { dense, missing, detected, firstReal };
 }
 
 // Interior measurement is a function of pixels alone, so it runs once with
@@ -217,10 +229,19 @@ function rebuild(resetKeep) {
 
   state.stab = stabilize(state.dense, o.smoothWin, state.aspect);
 
+  // The neutral drives calibration and the placeholder plate, so it has to be a
+  // frame the camera actually saw: a back-filled or held frame is a duplicate
+  // pose, and letting one win this contest would calibrate the whole take
+  // against a landmark set that belongs to some other moment.
   const ap = state.stab.aperture;
+  const real = state.detected;
+  const shut = (a, b) => (b < 0 || ap[a] < ap[b] ? a : b);
   const head = Math.max(1, Math.floor(N / 4));
-  let neutral = 0;
+  let neutral = -1;
-  for (let i = 0; i < head; i++) if (ap[i] < ap[neutral]) neutral = i;
+  for (let i = 0; i < head; i++) if (!real || real[i]) neutral = shut(i, neutral);
+  // Whole head of the take occluded: widen to any real frame rather than give up.
+  if (neutral < 0) for (let i = 0; i < N; i++) if (!real || real[i]) neutral = shut(i, neutral);
+  if (neutral < 0) neutral = 0;
   state.neutral = neutral;
   state.xform = makeXform(state.stab, neutral);
 
@@ -1072,8 +1093,8 @@ async function runFrames() {
       status(`no frames in ${el('framedir').value}/ — run extract.sh first`, 'err');
       return;
     }
-    const { dense, missing } = await detectAll(images);
+    const { dense, missing, detected, firstReal } = await detectAll(images);
-    state.images = images; state.dense = dense;
+    state.images = images; state.dense = dense; state.detected = detected;
     state.aspect = images[0].naturalWidth / images[0].naturalHeight;
     status('measuring mouth interiors…');
     state.interior = measureAll(images, dense, opts());
@@ -1086,7 +1107,9 @@ async function runFrames() {
     status(`${images.length} frames · ${images[0].naturalWidth}x${images[0].naturalHeight} · ` +
       `${state.fps}fps · ${dur}s` +
       (state.audio ? ' · audio loaded' : ' · no audio') +
-      (missing.length ? ` · no face on ${missing.length} (held previous)` : ''),
+      (missing.length ? ` · no face on ${missing.length}` +
+        (firstReal ? ` (${firstReal} at the top back-filled from ${firstReal + 1}, rest held)`
+                   : ' (held previous)') : ''),
       missing.length ? 'warn' : 'ok');
   } catch (e) { status(e.message, 'err'); console.error(e); }
 }
@@ -1096,6 +1119,7 @@ function runSynthetic() {
   attachAudio(null);
   state.fps = 12;
   state.aspect = 1;         // synthetic landmarks are generated square
+  state.detected = null;    // no detection ran, so every frame counts as real
   state.lead = 0;
   state.interior = null;    // no pixels, so no teeth