noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Give features stable identity, eye pairs and per-feature presence

Browse source 
Step 8's data model, ahead of its controls. Nothing here is a UI.

domain/params holds every knob's definition once — default, applicable area,
value constraints and the areas a change would force to regenerate. flow/take's
literal knob map becomes a view of it, so the take's defaults and the future
parameter panel cannot drift apart.

domain/feature adds subjects, features and groups as document data the renderer
never reads. A feature ID is stable for the whole clip, across occlusion: a run
of visible frames is not a new identity. An eye pair is an explicit group of one
or two eyes of the same subject, so a profile view with one identified eye needs
no invented partner. Settings resolve area -> subject -> group -> feature, and
dropping an eye from a pair materialises its effective values first so playback
does not jump. scene/problems now validates all of it.

Presence becomes per-feature rather than per-subject. freeze's :absent predicate
takes a track as well as a frame, so one occluded eye can be absent while its
partner still has a value; a full-face miss still marks everything absent. A
manifest may annotate known gaps as one-based inclusive intervals, which ingest
expands into observation tracks before measurement. An unobserved eye then gets
no vote in the iris pairing and cannot steer the shared gaze — gaze falls back to
whichever eye is visible. Temporal filters still see a sample on every frame,
held from the last observed one, because the numbers are a rectangular buffer;
the state mask, not the buffer, is what says the frame has no value.

js/app.js gets the same occlusion lesson: leading nulls from a face that starts
occluded used to throw away the whole take, and the neutral frame could be chosen
from a held duplicate pose.

Parameter editing, scoped regeneration and a feature-level detector remain. Until
one exists, footage without annotations falls back to the full-face mask rather
than claiming occlusions it cannot see.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B87NVmiU36qQmN9gmFYnJ9
This commit is contained in:
Olive Vaughn 2026-09-27 22:44:36 -04:00
parent ccca93e233
commit 35ef150b48
18 changed files with 631 additions and 86 deletions
Download patch file Download diff file Expand all files Collapse all files

54
frontend/test/arthur/domain/feature_test.cljs Normal file
View file

@ -0,0 +1,54 @@
(ns arthur.domain.feature-test
(:require [cljs.test :refer [deftest is]]
[arthur.domain.feature :as feature]
[arthur.domain.params :as params]))
(defn- nine-eyes []
(let [people (mapv #(keyword (str "person-" %)) (range 5))
eyes (mapv #(keyword (str "eye-" %)) (range 9))
members [[:eye-0 :eye-1] [:eye-2 :eye-3] [:eye-4 :eye-5]
[:eye-6 :eye-7] [:eye-8]]]
{:subjects (into {} (map (fn [id] [id {:id id :params {}}]) people))
:features (into {} (map-indexed (fn [i id]
[id {:id id :subject (nth people (quot i 2))
:area :eye :nodes [] :params {}}]) eyes))
:groups (into {} (map-indexed (fn [i ids]
(let [id (keyword (str "pair-" i))]
[id {:id id :kind :eye-pair
:subject (nth people i)
:members ids :params {}}])) members))
:nodes {}}))
(deftest five-people-can-have-nine-identified-eyes
(let [scene (nine-eyes)]
(is (empty? (feature/problems scene)))
(is (= [:eye-8] (get-in scene [:groups :pair-4 :members])))
(is (= :person-4 (get-in scene [:features :eye-8 :subject])))))
(deftest parameter-definitions-drive-the-current-take-defaults
(is (= #{:eye} (params/affected-areas :blink-cut)))
(is (= #{:head :mouth :eye :brow :teeth}
(params/affected-areas :anchor-avg)))
(is (params/valid-settings? :eye {:blink-cut 0.2}))
(is (not (params/valid-settings? :eye {:verts 8}))))
(deftest pair-values-are-shared-but-eye-overrides-are-independent
(let [scene (-> (nine-eyes)
(assoc-in [:groups :pair-0 :params :blink-cut] 0.2)
(assoc-in [:features :eye-1 :params :blink-cut] 0.3))]
(is (= 0.2 (:blink-cut (feature/effective-params scene :eye-0))))
(is (= 0.3 (:blink-cut (feature/effective-params scene :eye-1))))
(is (= (:eye-verts params/defaults)
(:eye-verts (feature/effective-params scene :eye-0))))
(let [before (feature/effective-params scene :eye-0)
unpaired (feature/remove-from-pair scene :eye-0)]
(is (= before (feature/effective-params unpaired :eye-0)))
(is (= [:eye-1] (get-in unpaired [:groups :pair-0 :members])))
(is (empty? (feature/problems unpaired))))))
(deftest a-pair-cannot-cross-subjects-or-own-an-eye-twice
(let [scene (nine-eyes)]
(is (seq (feature/problems
(assoc-in scene [:groups :pair-0 :members] [:eye-0 :eye-2]))))
(is (seq (feature/problems
(assoc-in scene [:groups :pair-1 :members] [:eye-0 :eye-3]))))))

67
frontend/test/arthur/flow/eye_occlusion_test.cljs Normal file
View file

@ -0,0 +1,67 @@
(ns arthur.flow.eye-occlusion-test
(:require [cljs.test :refer [deftest is]]
[arthur.demo.take :as take]
[arthur.domain.channel :as ch]
[arthur.domain.landmarks :as lm]
[arthur.domain.palette :as pal]
[arthur.domain.scene :as scene]
[arthur.flow.condition.eyes :as condition-eyes]
[arthur.flow.ingest :as ingest]
[arthur.flow.measure.eyes :as eyes]
[arthur.flow.take :as flow-take]))
(deftest an-unobserved-eye-cannot-steer-the-shared-gaze
(let [frame 11
params (assoc take/params :aspect 1)
presence (ingest/feature-presence take/frames {:eye-r [[10 14]]})
inputs {:dense @take/analysis
:transforms (:transforms @take/measured)
:presence presence}
clean (eyes/measure params inputs)
iris-ids (if (= :a (:right (:iris-pair clean))) lm/IRIS-A lm/IRIS-B)
bad-ids (distinct (concat lm/EYE-R-RING iris-ids))
corrupted (mapv (fn [i source]
(if (<= 9 i 13)
(reduce (fn [points id]
(update-in points [id :x] + 0.25))
source bad-ids)
source))
(range take/frames) @take/analysis)
masked (eyes/measure params (assoc inputs :dense corrupted))
unmasked (eyes/measure params
(dissoc (assoc inputs :dense corrupted) :presence))
drawn-clean (condition-eyes/apply-defaults params clean)
drawn-masked (condition-eyes/apply-defaults params masked)]
(is (= lm/NUM-LANDMARKS (count (nth corrupted frame))))
(is (:has-iris? masked) (str "pair " (:iris-pair masked)))
(is (false? (nth (:observed-r masked) frame)))
(is (true? (nth (:observed-l masked) frame)))
(is (= (nth (:gaze clean) frame) (nth (:gaze masked) frame)))
(is (not= (nth (:gaze clean) frame) (nth (:gaze unmasked) frame)))
(is (= (nth (:lid-r drawn-clean) frame)
(nth (:lid-r drawn-masked) frame)))
(is (= (nth (:iris-l drawn-clean) frame)
(nth (:iris-l drawn-masked) frame)))))
(deftest the-full-take-hides-only-the-annotated-eye-and-returns-to-the-same-id
(let [presence (ingest/feature-presence take/frames {:eye-r [[10 14]]})
{:keys [scene store]} (flow-take/build
(assoc take/params :aspect 1 :name "observed-gap")
{:dense @take/analysis :presence presence})
sample (fn [id frame]
(ch/value-at (get-in scene [:nodes id :channels [:geom :pts]])
frame store))]
(is (empty? (scene/problems scene)))
(is (= [:eye-r :eye-l] (get-in scene [:groups :eyes-1 :members])))
(doseq [f (range 9 14)]
(is (ch/nothing? (sample :eye-r f)))
(is (not (ch/nothing? (sample :eye-l f))))
(is (not (ch/nothing? (sample :mouth f)))))
(let [drawn (into #{} (map :node)
((scene/resolver scene store pal/index-of) 11))]
(is (not (contains? drawn :eye-r)))
(is (not (contains? drawn :iris-r)))
(is (contains? drawn :eye-l))
(is (contains? drawn :mouth)))
(is (not (ch/nothing? (sample :eye-r 8))))
(is (not (ch/nothing? (sample :eye-r 14))))))

32
frontend/test/arthur/flow/freeze_test.cljs
View file

@ -381,6 +381,27 @@
;; ---------------------------------------------------------------------------
;; presence is not visibility
(deftest an-eye-can-disappear-and-return-under-the-same-identity
(let [gap (set (range 40 60))
presence {:eye-r (mapv #(not (contains? gap %)) (range take/frames))}
c (freeze/clip (assoc take/params :name "one-eye-gappy")
(assoc @take/measured :presence presence))
sc (:scene c)
at (fn [id f]
(ch/value-at (get-in sc [:nodes id :channels [:geom :pts]]) f (:store c)))]
(is (= [:eye-r :eye-l] (get-in sc [:groups :eyes-1 :members])))
(is (= :eye-r (get-in sc [:features :eye-r :id])))
(is (not (ch/nothing? (at :eye-r 39))))
(is (ch/nothing? (at :eye-r 50)))
(is (not (ch/nothing? (at :eye-r 60))))
(is (not (ch/nothing? (at :eye-l 50))))
(is (not (ch/nothing? (at :mouth 50))))
(let [drawn-nodes (into #{} (map :node) ((scene/resolver sc (:store c) pal/index-of) 50))]
(is (not (contains? drawn-nodes :eye-r)))
(is (contains? drawn-nodes :eye-l))
(is (contains? drawn-nodes :mouth)))
(is (empty? (scene/problems sc)))))
(deftest an-undetected-frame-has-no-pose-at-all
;; A subject that was not on the frame has NO VALUE, which is different from a
;; part being switched off. The mask lands on every block of the freeze, so an
@ -447,10 +468,15 @@
;; beat are not. This is the numeric half of step 5's done-criterion; the other
;; half is a picture and lives in test/browser/take.mjs.
(let [locked (freeze/head-mode {:mode :locked} @clip)
shot (fn [f] (render locked f))
shot (fn [f]
(let [r (raster/make W H)
mouth (filter #(= :mouth (:node %)) (ops-at locked f))]
(raster/clear! r (get pal/index-of :bg))
(raster/draw-ops! r mouth)
(vec (array-seq (:buf r)))))
differ (fn [a b] (count (remove true? (map = a b))))]
;; Beat 1 is wide open and beat 3 is shut. Rendered with the head LOCKED, so
;; what differs is articulation and not the head wandering across the stage.
;; Beat 1 is wide open and beat 3 is shut. Compare only the mouth outline:
;; eyes and brows added in step 7 have their own motion within the beat.
(is (> (differ (shot 10) (shot 28)) 300)
"the open and the shut mouth rasterise the same")
;; And within a beat, on the exposure grid, it holds.

15
frontend/test/arthur/flow/ingest_test.cljs Normal file
View file

@ -0,0 +1,15 @@
(ns arthur.flow.ingest-test
(:require [cljs.test :refer [deftest is]]
[arthur.flow.ingest :as ingest]))
(deftest feature-absence-uses-the-source-frame-numbers
(let [tracks (ingest/feature-presence
8 {:eye-r [[3 5]] :eye-l [[1 1] [8 8]]})]
(is (= [true true false false false true true true] (:eye-r tracks)))
(is (= [false true true true true true true false] (:eye-l tracks)))))
(deftest malformed-feature-absence-fails-before-analysis
(doseq [bad [{:eye-r [[0 2]]}
{:eye-r [[6 9]]}
{:eye-r [[5 3]]}]]
(is (thrown? ExceptionInfo (ingest/feature-presence 8 bad)))))