noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Give features stable identity, eye pairs and per-feature presence

Browse source 
Step 8's data model, ahead of its controls. Nothing here is a UI.

domain/params holds every knob's definition once — default, applicable area,
value constraints and the areas a change would force to regenerate. flow/take's
literal knob map becomes a view of it, so the take's defaults and the future
parameter panel cannot drift apart.

domain/feature adds subjects, features and groups as document data the renderer
never reads. A feature ID is stable for the whole clip, across occlusion: a run
of visible frames is not a new identity. An eye pair is an explicit group of one
or two eyes of the same subject, so a profile view with one identified eye needs
no invented partner. Settings resolve area -> subject -> group -> feature, and
dropping an eye from a pair materialises its effective values first so playback
does not jump. scene/problems now validates all of it.

Presence becomes per-feature rather than per-subject. freeze's :absent predicate
takes a track as well as a frame, so one occluded eye can be absent while its
partner still has a value; a full-face miss still marks everything absent. A
manifest may annotate known gaps as one-based inclusive intervals, which ingest
expands into observation tracks before measurement. An unobserved eye then gets
no vote in the iris pairing and cannot steer the shared gaze — gaze falls back to
whichever eye is visible. Temporal filters still see a sample on every frame,
held from the last observed one, because the numbers are a rectangular buffer;
the state mask, not the buffer, is what says the frame has no value.

js/app.js gets the same occlusion lesson: leading nulls from a face that starts
occluded used to throw away the whole take, and the neutral frame could be chosen
from a held duplicate pose.

Parameter editing, scoped regeneration and a feature-level detector remain. Until
one exists, footage without annotations falls back to the full-face mask rather
than claiming occlusions it cannot see.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B87NVmiU36qQmN9gmFYnJ9
This commit is contained in:
Olive Vaughn 2026-09-27 22:44:36 -04:00
parent ccca93e233
commit 35ef150b48
18 changed files with 631 additions and 86 deletions
Download patch file Download diff file Expand all files Collapse all files

67
frontend/test/arthur/flow/eye_occlusion_test.cljs Normal file
View file

@ -0,0 +1,67 @@
(ns arthur.flow.eye-occlusion-test
(:require [cljs.test :refer [deftest is]]
[arthur.demo.take :as take]
[arthur.domain.channel :as ch]
[arthur.domain.landmarks :as lm]
[arthur.domain.palette :as pal]
[arthur.domain.scene :as scene]
[arthur.flow.condition.eyes :as condition-eyes]
[arthur.flow.ingest :as ingest]
[arthur.flow.measure.eyes :as eyes]
[arthur.flow.take :as flow-take]))
(deftest an-unobserved-eye-cannot-steer-the-shared-gaze
(let [frame 11
params (assoc take/params :aspect 1)
presence (ingest/feature-presence take/frames {:eye-r [[10 14]]})
inputs {:dense @take/analysis
:transforms (:transforms @take/measured)
:presence presence}
clean (eyes/measure params inputs)
iris-ids (if (= :a (:right (:iris-pair clean))) lm/IRIS-A lm/IRIS-B)
bad-ids (distinct (concat lm/EYE-R-RING iris-ids))
corrupted (mapv (fn [i source]
(if (<= 9 i 13)
(reduce (fn [points id]
(update-in points [id :x] + 0.25))
source bad-ids)
source))
(range take/frames) @take/analysis)
masked (eyes/measure params (assoc inputs :dense corrupted))
unmasked (eyes/measure params
(dissoc (assoc inputs :dense corrupted) :presence))
drawn-clean (condition-eyes/apply-defaults params clean)
drawn-masked (condition-eyes/apply-defaults params masked)]
(is (= lm/NUM-LANDMARKS (count (nth corrupted frame))))
(is (:has-iris? masked) (str "pair " (:iris-pair masked)))
(is (false? (nth (:observed-r masked) frame)))
(is (true? (nth (:observed-l masked) frame)))
(is (= (nth (:gaze clean) frame) (nth (:gaze masked) frame)))
(is (not= (nth (:gaze clean) frame) (nth (:gaze unmasked) frame)))
(is (= (nth (:lid-r drawn-clean) frame)
(nth (:lid-r drawn-masked) frame)))
(is (= (nth (:iris-l drawn-clean) frame)
(nth (:iris-l drawn-masked) frame)))))
(deftest the-full-take-hides-only-the-annotated-eye-and-returns-to-the-same-id
(let [presence (ingest/feature-presence take/frames {:eye-r [[10 14]]})
{:keys [scene store]} (flow-take/build
(assoc take/params :aspect 1 :name "observed-gap")
{:dense @take/analysis :presence presence})
sample (fn [id frame]
(ch/value-at (get-in scene [:nodes id :channels [:geom :pts]])
frame store))]
(is (empty? (scene/problems scene)))
(is (= [:eye-r :eye-l] (get-in scene [:groups :eyes-1 :members])))
(doseq [f (range 9 14)]
(is (ch/nothing? (sample :eye-r f)))
(is (not (ch/nothing? (sample :eye-l f))))
(is (not (ch/nothing? (sample :mouth f)))))
(let [drawn (into #{} (map :node)
((scene/resolver scene store pal/index-of) 11))]
(is (not (contains? drawn :eye-r)))
(is (not (contains? drawn :iris-r)))
(is (contains? drawn :eye-l))
(is (contains? drawn :mouth)))
(is (not (ch/nothing? (sample :eye-r 8))))
(is (not (ch/nothing? (sample :eye-r 14))))))