942e2f38ab
`stabilize` is three things wearing one name, and it is now three functions in two stages: `flow/measure/anchor` fits the rigid transform, `flow/condition` smooths its parameters, `flow/measure/mouth` measures the lip rings through the result. Parity is on the COMPOSITION and not on the pieces -- a split that agreed function by function and not end to end would be a split rather than a port. The oracle now drives `stabilize` at three configurations and the port agrees to 1e-9 on ref, rigid, transforms, outer, inner and aperture, plus `smoothContours` at three radii. Two of the three configurations are at aspect 0.5625, a 1080x1920 phone clip, because at aspect 1 `pick` is the identity: a port that dropped the anisotropy correction outright would pass every other assertion in the suite. 148 tests, up from 134. Three decisions worth the reading time. `makeXform` is not ported, and its absence takes the face oval with it. It centres on the oval's bounding box and zooms until the face is 80% of the raster height, so every vertex it touched carried a cropping decision made once, at analysis time, from one frame's landmarks. Geometry belongs in the node's own local space with the framing as a transform on a node, so this is a deletion. The oval's only other consumer was the placeholder plate outline, which is painting. The residual is taken against the RAW fit, and the prototype took it against the smoothed one. That is the only deliberate numeric divergence here, and parity is kept by asserting `anchor/residuals` on exactly what the prototype handed it. The number's job is to say whether a section is stabilisable at all; folding the smoothing error into it makes a slider look like a property of the footage, and docs/architecture.md lists the residual under stage 3, which requires it to be knob-free. `condition/anchor` therefore replaces `:transforms` and leaves `:residual` alone. The stage order is not the strict chain the table in docs/architecture.md looks like, and that document now says so. The fit is knob-free, conditioning smooths it, and the rings are measured *through* the conditioned transform -- so `anchor avg` does re-run the ring mapping, which is a few hundred frames of twenty points. The guarantee was only ever about the part that reads a source pixel, and that part never sees a transform. Two things fall out and are asserted rather than assumed. Smoothing and subsampling commute, because both are per-slot, which is what lets `vertices` stay a stage-5 knob downstream of a stage-4 one -- and it is also why the port can smooth the full twenty slots where the prototype smooths eight and still match. And `condition/contours` is `geom/moving-average` per vertex per axis rather than its own clamped window, so "radius 2" cannot come to mean two different things at the two knobs. One dead end recorded so nobody walks it twice: the synth's head is perfectly rigid -- its jitter is a whole-head translation, which a similarity absorbs exactly -- so every frame's rigid configuration is congruent with frame zero's and the Procrustes mean IS frame zero to 1e-15, jitter or none. "The reference is the mean and not frame zero" cannot be asserted on this track and is asserted in geom-test, where the two can differ. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
85 lines
4.3 KiB
Clojure
85 lines
4.3 KiB
Clojure
(ns arthur.flow.measure.mouth-test
|
|
(:require [cljs.test :refer [deftest is testing]]
|
|
[arthur.domain.landmarks :as lm]
|
|
[arthur.domain.ring :as ring]
|
|
[arthur.flow.condition :as condition]
|
|
[arthur.flow.measure.anchor :as anchor]
|
|
[arthur.flow.measure.mouth :as mouth]
|
|
[arthur.synth :as synth]))
|
|
|
|
(def clean (delay (synth/synth-dense 72 {:rand-fn (constantly 0.5)})))
|
|
|
|
(def measured
|
|
(delay
|
|
(let [fitted (anchor/fit {:aspect 1} {:dense @clean})]
|
|
(mouth/measure {:aspect 1} {:dense @clean :transforms (:transforms fitted)}))))
|
|
|
|
(defn- spread
|
|
"Largest distance between any two of `rings` at the same slot: how much the
|
|
shape moved over the frames given."
|
|
[rings]
|
|
(reduce max (for [a rings b rings [p q] (map vector a b)]
|
|
(js/Math.hypot (- (:x p) (:x q)) (- (:y p) (:y q))))))
|
|
|
|
;; The synth holds each mouth pose for nine frames, so frames 0-8 are one pose
|
|
;; carried around by a moving, rolling, scaling head. That is the case the whole
|
|
;; stage exists for.
|
|
(def ^:private hold (range 0 9))
|
|
|
|
(deftest a-held-pose-holds-still-once-the-head-is-taken-out
|
|
(let [local (spread (map #(nth (:outer @measured) %) hold))
|
|
image (spread (map (fn [f] (anchor/pick (nth @clean f) lm/LIPS-OUTER 1)) hold))]
|
|
(is (< local 1e-9)
|
|
(str "head-local outer ring moved " local " over a held pose"))
|
|
;; And it moved plenty in the footage, or the line above is measuring nothing.
|
|
(is (> image 0.02)
|
|
(str "the ring only moved " image " in image space over the same frames"))))
|
|
|
|
(deftest the-aperture-measures-the-mouth-and-not-the-head
|
|
(let [ap (:aperture @measured)
|
|
beat (fn [b] (map #(nth ap %) (range (* b 9) (* (inc b) 9))))
|
|
;; open-amt per beat in synth.cljs is [0.004 0.05 0.022 0.0], so the
|
|
;; ordering is known in advance rather than read off the output.
|
|
peak (fn [b] (reduce max (beat b)))]
|
|
(is (> (peak 1) (peak 2) (peak 0) (peak 3))
|
|
(str "aperture peaks by beat: " (pr-str (mapv peak (range 4)))))
|
|
;; The head's scale swings 6% across the take. A held pose whose aperture
|
|
;; tracked the head would drift with it; head-local, it is one number.
|
|
(is (< (- (peak 0) (reduce min (beat 0))) 1e-9)
|
|
(str "aperture wandered by " (- (peak 0) (reduce min (beat 0)))
|
|
" over a held pose"))))
|
|
|
|
(deftest slot-identity-survives-the-stabilisation
|
|
;; Slot position IS the vertex's identity, and that is what makes temporal
|
|
;; correspondence possible at all. On a 20-slot ring the four cardinals land on
|
|
;; the four quarter slots: 0 and 10 are the two corners, 5 and 15 the lip
|
|
;; centres. So the corners are the horizontal extremes and 5 is above 15, with y
|
|
;; growing downward — and a traversal reversed, rotated or transposed anywhere
|
|
;; breaks that. The simplicity check catches the same class; this one says which
|
|
;; slot moved.
|
|
;;
|
|
;; WHICH corner is slot 0 is deliberately not asserted. Left and right in
|
|
;; MediaPipe's own naming are viewer-relative in some places and
|
|
;; subject-relative in others, so a test written from the table would be pinning
|
|
;; a coin flip; the synth puts slot 0 at larger x and that is all this knows.
|
|
(doseq [f (range 0 72 7)]
|
|
(let [r (nth (:outer @measured) f)
|
|
at (fn [s] (nth r s))]
|
|
(is (> (:x (at 0)) (:x (at 5)) (:x (at 10)))
|
|
(str "frame " f ": corners are not the horizontal extremes"))
|
|
(is (< (:y (at 5)) (:y (at 15)))
|
|
(str "frame " f ": the top centre is not above the bottom centre")))))
|
|
|
|
(deftest smoothing-and-subsampling-commute
|
|
;; The claim the stage split rests on. `contour avg` is stage 4 and `vertices` is
|
|
;; stage 5, and that ordering is only free because both operations are per-slot:
|
|
;; averaging slot s over time then picking slot s is picking it then averaging.
|
|
;; The prototype subsamples first, so this is also what keeps parity honest.
|
|
(doseq [radius [1 2 3]
|
|
verts [4 6 8 10]]
|
|
(let [slots (ring/subsample-slots (count lm/LIPS-OUTER) verts)
|
|
pick (fn [rings] (mapv (fn [r] (mapv #(nth r %) slots)) rings))
|
|
first' (pick (condition/contours {:contour-avg radius} (:outer @measured)))
|
|
then (condition/contours {:contour-avg radius} (pick (:outer @measured)))]
|
|
(is (= first' then)
|
|
(str "radius " radius ", " verts " verts: the two orders disagree")))))
|