From 02069e88f036772d32e169750c43ad88218cf830 Mon Sep 17 00:00:00 2001 From: Your Name Date: Thu, 1 Oct 2026 10:13:00 -0400 Subject: [PATCH] A protected frame is on screen, so it anchors the walk MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `domain/select`: the choosing half of a selection, pure and handed a plain vector by each site. `pose/held-frame` was already the shared reading half; nothing chose automatically at all. The greedy walk rather than the DP, deliberately — it is what the prototype shipped, and having two implementations is how the DP gets tested. Several tests pin its exact output for that comparison and will need rewriting when it lands. Protection is settled before the walk and is not unioned onto its result, because a protected frame anchors the walk like any other kept frame. The anchor is what is on screen, so measuring the next frame's drift from a frame that is no longer displayed holds a one-frame closure across two frames and shows it twice as long as it was measured. The test that says so is about the picture rather than the algorithm: the frames the mouth was measured shut on and the frames the selection shows it shut on are the same list. A strict local extremum compares against the nearest DIFFERING samples, not the immediate neighbours, and a run of equal samples is one extremum at the frame it begins. Immediate neighbours find nothing at all on a closure lasting more than one frame, which is most of them. One width for the whole signal, so `distance` is never comparing the prefix two samples happen to share; a non-finite tolerance falls back to nought, since NaN compares false against everything and would quietly collapse a performance to a single pose. THE EXTREMA HALF HAS NO CALLER under the plan as it now stands, and this is the commit to revert if it stays that way: `segments`, `turns`, the `:extrema` branch of `propose` and the multi-component refusal that only guards it, plus the three tests over them. Plate selections take no extrema by design, and the performance half gets its closures from the `[:vis]` cut `flow/freeze` already stores, so nothing asks this code for anything. It is correct and tested and speculative; docs/frame-selection.md says so beside the build order. Also corrects `ring/subsample-slots`, which claimed every even budget lands on the cardinal positions. The corners do; the lip centres survive only multiples of 4, so the aperture pair cannot be read off a subsampled ring at verts 6, 10, 14 or 18. That is why the performance signal reads a stored cut instead. Co-Authored-By: Claude Opus 5 --- frontend/src/arthur/domain/ring.cljs | 12 +- frontend/src/arthur/domain/select.cljs | 191 +++++++++++++++++ frontend/test/arthur/domain/select_test.cljs | 213 +++++++++++++++++++ 3 files changed, 413 insertions(+), 3 deletions(-) create mode 100644 frontend/src/arthur/domain/select.cljs create mode 100644 frontend/test/arthur/domain/select_test.cljs diff --git a/frontend/src/arthur/domain/ring.cljs b/frontend/src/arthur/domain/ring.cljs index 57a1a50..241b72f 100644 --- a/frontend/src/arthur/domain/ring.cljs +++ b/frontend/src/arthur/domain/ring.cljs @@ -13,9 +13,15 @@ back to positions with indexOf would silently pick the wrong slot if a table ever repeated an id. - For even n this naturally lands on the cardinal positions (corners and lip - centres) of a 20-point ring. Fixed indices, never adaptive decimation: the - vertex at slot k means the same thing on every frame of the shot." + Fixed indices, never adaptive decimation: the vertex at slot k means the same + thing on every frame of the shot. + + WHICH CARDINAL POSITIONS SURVIVE DEPENDS ON n, and not merely on n being even. + On a 20-point ring the corners at 0 and 10 are kept by every even n, but the lip + centres at 5 and 15 are kept only when n is a multiple of 4: n = 6, 10, 14 and + 18 all drop them. So nothing may assume a named slot is still in the output — + the aperture pair least of all. A signal that needs those two landmarks reads + them from a measurement, not from a subsampled ring." [len n] (mapv (fn [k] (mod (js/Math.round (/ (* k len) n)) len)) (range n))) diff --git a/frontend/src/arthur/domain/select.cljs b/frontend/src/arthur/domain/select.cljs new file mode 100644 index 0000000..7644f6f --- /dev/null +++ b/frontend/src/arthur/domain/select.cljs @@ -0,0 +1,191 @@ +(ns arthur.domain.select + "Which frames out of a dense measurement are worth keeping. + + A SELECTION IS A SET OF FRAMES CHOSEN OUT OF A DENSE MEASUREMENT, AND READ BY + HOLDING THE LATEST ONE AT OR BEFORE NOW. The holding half is `pose/held-frame` + and is already shared by tracing and by pose tracks; this is the choosing half, + which nothing did automatically before. One component, two sites: the plate + frames an artist has to draw a head on, and the frames a performance changes a + shape on. + + `domain/cadence` answers the same question with no opinion about content, and + cannot know that the one frame where a mouth is fully shut is worth more than + its neighbours. That is the gap `:protect` closes. + + THREE LAYERS, AND THE MIDDLE ONE IS DERIVED. A selection is a `:policy` (what + the proposer was asked for), a `:keep` and a `:drop` (what the hand insists on + and refuses). Only the hand's two sets are the document: the proposal is + recomputed whenever the policy or the signal changes, which is the whole reason + the hand's decisions are stored as their own sets rather than as the resulting + frame list. Re-suggesting at a new tolerance must never cost somebody their + pinned blink. + + Pure. No store access and no clip access — each site reads its own dense data + and hands over a plain vector. Proposing is a command and not a subscription: + this allocates, which is fine for a button press over a few hundred frames and + would not be fine on the per-frame path.") + +(defn- sample + "One sample as a flat vector of numbers, or nil where there is no measurement. + + A plain number is a one-component sample, and nested points are flattened: + four transformed corners and the eight numbers in them are the same signal, so + neither site has to flatten on the way in. Nil is an ABSENT measurement — a + frame where the face was not found — which says nothing about the signal and is + not the same fact as a frame being skipped." + [s] + (cond + (nil? s) nil + (number? s) [s] + :else (not-empty (vec (flatten s))))) + +(defn- distance + "How far apart two samples are: the largest absolute difference over their + components. A tolerance therefore means \"this far\" in the units the site + handed over, whether it handed over an aperture or a quad of corners, and + nobody has to say which. + + Both samples are the same width, which `propose` has already insisted on rather + than comparing whatever prefix the two happen to share." + [a b] + (reduce (fn [m i] (max m (js/Math.abs (- (nth a i) (nth b i))))) + 0 (range (count a)))) + +(defn- walk + "The greedy walk: keep frame 0 as the anchor, keep every frame in `forced`, and + keep every other frame that has moved further than `tolerance` from the anchor. + Each kept frame becomes the anchor. + + A FORCED FRAME BECOMES THE ANCHOR LIKE ANY OTHER, which is why protection is + settled before the walk rather than unioned into its result. The anchor is what + is on screen; once a protected frame is kept the viewer is looking at it, so + measuring drift from a frame that is no longer displayed is simply wrong. Doing + it the other way holds a protected one-frame closure across two frames and + shows it twice as long as it was measured, which is the perceptual error the + protection was added to prevent. + + Order-dependent and slightly suboptimal, and deliberately the first + implementation anyway — it is what the prototype shipped, and the dynamic + program that replaces it gets tested by being compared against it. Keeping the + anchor honest here leaves greed as the only difference between the two. + + An absent sample is not a change: it cannot move the anchor and it is not worth + a frame. A measurement arriving where the anchor has none is, since a signal + coming back is a frame the picture has to show something on. A forced frame + with no measurement is still kept — it was asked for — and leaves the anchor + where it was, there being nothing there to measure from." + [n sig tolerance forced] + (loop [f 1 anchor (nth sig 0) kept (transient [0])] + (if (>= f n) + (persistent! kept) + (let [s (nth sig f)] + (if (or (contains? forced f) + (and s (or (nil? anchor) (> (distance anchor s) tolerance)))) + (recur (inc f) (or s anchor) (conj! kept f)) + (recur (inc f) anchor kept)))))) + +(defn- segments + "Maximal runs of consecutive measured frames. A gap is not something to compare + across: the samples either side of an absent measurement are not neighbours." + [n sig] + (->> (range n) + (partition-by #(some? (nth sig %))) + (remove #(nil? (nth sig (first %)))))) + +(defn- turns + "Frames in one measured segment that are a strict local extremum by more than + `tolerance` — a full mouth closure, a full blink. These are exactly the frames + a zero-order hold is most wrong about, and exactly what a cadence drops. + + A run of equal samples is ONE extremum, reported at the frame it begins: the + hold reads from there, so naming any later frame of the run would show the + closure late. The ends of a segment are not extrema; they have only one side." + [sig tolerance frames] + (let [runs (vec (partition-by #(nth sig %) frames)) + at #(first (nth sig %))] + (for [i (range 1 (dec (count runs))) + :let [f (first (nth runs i)) + v (at f) + p (at (first (nth runs (dec i)))) + q (at (first (nth runs (inc i))))] + :when (and (or (and (< v p) (< v q)) (and (> v p) (> v q))) + (> (min (js/Math.abs (- v p)) (js/Math.abs (- v q))) + tolerance))] + f))) + +(defn propose + "Frames worth keeping out of `n`, given `signal`. + + `signal` is a vector of n samples, each a number or a seq of numbers in a fixed + order. FRAME REMOVAL, NOT KEY EXTRACTION: every frame is a candidate and the + question is which can be dropped, which is why this walks and holds rather than + looking for peaks. + + `:tolerance` is how far the signal may move before a frame has to be kept, in + whatever units the signal is in. `:protect` is the ONE input for frames that + have to be kept whatever the walk thought, and it carries two kinds of thing at + once so that it never has to become two parameters: + + :extrema keep the signal's own strict local extrema as well + [12 30] keep these frames, whatever the walk thought + #{:extrema 12} both + + The frames are how one selection constrains another — every kept plate frame is + a protected frame of the performance selection, so the drawing and the + performance can never cut against each other on neighbouring frames — and it is + the same mechanism that keeps a blink, which is why there is only one input. + + `:extrema` is refused on a multi-component signal. The magnitude of a vector of + landmarks has local maxima that mean nothing; an aperture's closure is a real + extremum and a head has no such thing as an extreme position worth protecting. + + Protection is settled BEFORE the walk, not unioned into its result, because a + protected frame anchors the walk like any other kept frame — see `walk`. The + result is therefore not `(walk) ∪ (protected)`: protected frames change what is + kept after them, which is the whole reason they are not applied afterwards. + + Always contains frame 0 when there is a frame at all. The result is an ascending + vector of distinct frames, which is the shape both sites already store." + [n signal {:keys [tolerance protect]}] + (if-not (pos? n) + [] + (let [;; A slider that has been cleared reads as NaN, and every comparison + ;; against NaN is false, which would propose frame 0 alone and quietly + ;; collapse a whole performance to one pose. Nought keeps every frame + ;; that changes, which is merely the cadence back again: wrong in a way + ;; somebody can see and undo. + tol (if (js/Number.isFinite tolerance) tolerance 0) + sig (mapv #(sample (nth signal % nil)) (range n)) + ;; One width for the whole signal, so `distance` is never comparing the + ;; prefix two samples happen to share. A reader that drops a component + ;; on one frame is a bug with a loud version and a silent version, and + ;; the silent version is a signal that is quietly the wrong shape. + seen (into #{} (map count) (remove nil? sig)) + width (first seen) + p (cond (nil? protect) #{} + (keyword? protect) #{protect} + :else (set protect)) + pins (into #{} (filter #(and (integer? %) (<= 0 %) (< % n))) p)] + (when (< 1 (count seen)) + (throw (ex-info "signal samples are not all the same width" + {:widths (vec (sort seen))}))) + (when (and (contains? p :extrema) width (< 1 width)) + (throw (ex-info ":protect :extrema needs a single-component signal" + {:components width}))) + (walk n sig tol (cond-> pins + (contains? p :extrema) + (into (mapcat #(turns sig tol %) (segments n sig)))))))) + +(defn effective + "`proposed` with the hand's decisions applied. Always contains 0: something has + to be on screen at the start. + + Manual precedence is absolute — a drop beats a proposal, always — and it is the + proposal it beats, not the start of the take. Hand decisions apply whether or + not the proposer is switched on, which is why they are not a mode: turning + smart picking off must not throw away the frames somebody pinned." + [proposed keep drop] + (-> (reduce disj (into (set proposed) keep) drop) + (conj 0) + sort + vec)) diff --git a/frontend/test/arthur/domain/select_test.cljs b/frontend/test/arthur/domain/select_test.cljs new file mode 100644 index 0000000..7fc1505 --- /dev/null +++ b/frontend/test/arthur/domain/select_test.cljs @@ -0,0 +1,213 @@ +(ns arthur.domain.select-test + "The one-frame closure is the feature, so it is the first test in the file. + + Both the cadence and a selection are sets of native frames read the same way — + hold the latest one at or before now — so they are compared through the same + reading function over the same native frames. Nothing here goes near a store, + a clip or an output grid: `select` is handed a vector of numbers." + (:require [cljs.test :refer [deftest is testing]] + [clojure.set :as set] + [arthur.domain.cadence :as cadence] + [arthur.domain.pose :as pose] + [arthur.domain.select :as select])) + +;; A mouth over 30 native frames that is fully shut for exactly ONE of them. +;; +;; It is already nearly shut either side of frame 13 — the aperture wobbles +;; between 0.015 and 0.03 — and at 13 it reaches 0. That is the shape of the +;; case a walk cannot see on displacement alone: the closure moves the signal by +;; 0.015, less than any tolerance worth using on the talking either side of it, +;; while being the only frame that reads as a closed mouth. +(def aperture + [0.30 0.30 0.26 0.20 0.14 0.10 0.07 0.04 0.03 0.025 0.015 0.015 + 0.03 0.0 0.03 + 0.06 0.09 0.15 0.22 0.30 0.36 0.40 0.43 0.43 0.40 0.36 0.30 0.24 0.18 0.12]) + +(def closure 13) +(def tol 0.02) + +(defn held + "The sample `signal` shows at native frame `f` when read through a selection of + native `frames`. This is the existing shared reading half, not a second one." + [signal frames f] + (nth signal (pose/held-frame (mapv #(vector % %) frames) f (first frames)))) + +(defn shut + "The frames on which `signal` read through the selection `frames` shows a fully + shut mouth. With `frames` nil it is the frames the mouth was MEASURED shut on, + which is what a selection is answerable to: the picture should show a closure + for exactly as long as it happened, and no longer." + [signal frames] + (filterv #(zero? (if frames (held signal frames %) (nth signal %))) + (range (count signal)))) + +(deftest a-one-frame-closure-survives-only-because-it-is-protected + (let [grid (mapv #(cadence/frame % 12 30) (range 12)) + walk (select/propose 30 aperture {:tolerance tol}) + kept (select/propose 30 aperture {:tolerance tol :protect :extrema})] + (testing "a uniform 12-from-30 cadence drops it" + (is (= [0 2 5 7 10 12 15 17 20 22 25 27] grid)) + (is (not (some #{closure} grid))) + ;; Not one of the twelve frames a cadence keeps is a shut mouth, and + ;; holding over them never reaches one either: at the closure it is still + ;; showing frame 12, which is open. + (is (= 0.03 (held aperture grid closure))) + (is (empty? (shut aperture grid)))) + (testing "the walk alone drops it too, because displacement is all it sees" + (is (= [0 2 3 4 5 6 7 10 15 16 17 18 19 20 21 22 24 25 26 27 28 29] walk)) + (is (not (some #{closure} walk))) + ;; It holds frame 10 straight across the closure and on to 15. + (is (= 0.015 (held aperture walk closure))) + (is (empty? (shut aperture walk)))) + (testing ":protect :extrema keeps it, for exactly as long as it happened" + (is (some #{closure} kept)) + (is (zero? (held aperture kept closure))) + (is (= [closure] (shut aperture nil) (shut aperture kept))) + ;; "At or before", so it does not show early either. + (is (= 0.015 (held aperture kept 12)))) + (testing "the protected frame anchors the walk, so the reopening is kept too" + ;; Frame 14 is the mouth coming back open. Without it the closure at 13 + ;; would hold across 14 and read twice as long as it was measured, which + ;; is why protection is settled before the walk and not unioned onto it. + (is (= #{closure 14} (set/difference (set kept) (set walk)))) + (is (= 0.03 (held aperture kept 14)))))) + +(deftest extrema-are-turns-worth-more-than-the-tolerance + ;; One shape, two depths, one tolerance. Neither dip moves the signal far + ;; enough for the walk to keep anything, so what separates them is prominence + ;; alone — the tolerance doing its job rather than an accident of the data. + (testing "a shallow wobble is not a closure" + (let [s [0.015 0.025 0.01 0.01 0.025 0.015]] + (is (= [0] (select/propose 6 s {:tolerance tol}))) + (is (= [0] (select/propose 6 s {:tolerance tol :protect :extrema}))))) + (testing "a two-frame closure is one extremum, at the frame it begins" + ;; The hold reads from the frame the closure begins on, so naming the second + ;; frame of the run instead would show it late. Frame 4 is kept because the + ;; protected frame 2 anchored the walk, and that is what makes the closure + ;; the same two frames long on screen as it was measured. + (let [s [0.015 0.03 0.0 0.0 0.03 0.015] + kept (select/propose 6 s {:tolerance tol :protect :extrema})] + (is (= [0] (select/propose 6 s {:tolerance tol}))) + (is (= [0 2 4] kept)) + (is (= [2 3] (shut s nil) (shut s kept))))) + (testing "a monotone ramp has no extrema to protect" + (is (= (select/propose 8 [0 1 2 3 4 5 6 7] {:tolerance 2}) + (select/propose 8 [0 1 2 3 4 5 6 7] {:tolerance 2 :protect :extrema})))) + (testing "the ends are not extrema; frame 0 is kept for being the start" + (is (= [0] (select/propose 3 [0.0 0.5 1.0] {:tolerance 2 :protect :extrema}))))) + +(deftest a-motionless-take-proposes-one-frame + (is (= [0] (select/propose 30 (vec (repeat 30 0.2)) {:tolerance tol}))) + (is (= [0] (select/propose 30 (vec (repeat 30 0.2)) + {:tolerance tol :protect :extrema}))) + (is (= [0] (select/propose 30 (vec (repeat 30 [1 2 3 4])) {:tolerance tol}))) + (is (= [] (select/propose 0 [] {:tolerance tol})))) + +;; A head sliding right at 0.004 stage units a frame, as four transformed +;; corners. The tolerance is a distance a person can reason about: "the head has +;; moved this far". +(def slide + (mapv (fn [f] (let [x (* 0.004 f)] + [[(- x 0.5) -0.5] [(+ x 0.5) -0.5] + [(+ x 0.5) 0.5] [(- x 0.5) 0.5]])) + (range 10))) + +(deftest distance-is-the-largest-change-over-components + (testing "landmark pairs and a bare number both answer without being declared" + (is (= [0 3 6 9] (select/propose 10 slide {:tolerance 0.01}))) + (is (= (select/propose 10 slide {:tolerance 0.01}) + (select/propose 10 (mapv flatten slide) {:tolerance 0.01})))) + (testing "one component moving is enough" + (is (= [0 2] (select/propose 3 [[0 0] [0 0.5] [0 1]] {:tolerance 0.6}))))) + +(deftest a-signal-of-two-shapes-is-a-reader-bug + ;; Comparing the prefix two samples happen to share would let a reader that + ;; drops a component on one frame read as no movement at all — a signal that is + ;; quietly the wrong shape rather than one that says so. + (is (thrown? ExceptionInfo + (select/propose 3 [[0 0] [0 0 0] [0 0]] {:tolerance 0.01}))) + (testing "an absent measurement is not a second shape" + (is (= [0] (select/propose 3 [[0 0] nil [0 0]] {:tolerance 0.01}))))) + +(deftest a-tolerance-that-is-not-a-number-fails-towards-the-cadence + ;; A slider that has been cleared reads as NaN, and every comparison against + ;; NaN is false. Taken at face value that proposes frame 0 alone and collapses + ;; a whole performance to one pose. Nought proposes every frame that changes, + ;; which is only the cadence back again: visible, and undone by moving the + ;; slider somewhere. + (is (= (select/propose 30 aperture {:tolerance 0}) + (select/propose 30 aperture {:tolerance js/NaN}) + (select/propose 30 aperture {}))) + (is (< 1 (count (select/propose 30 aperture {:tolerance js/NaN}))))) + +(deftest extrema-are-refused-on-a-multi-component-signal + ;; The magnitude of a vector of landmarks has local maxima that mean nothing, + ;; so asking for them is a mistake rather than a thing to silently ignore. + (is (thrown? ExceptionInfo + (select/propose 10 slide {:tolerance 0.01 :protect :extrema}))) + (testing "protected frames are not: a plate selection is fed in this way" + ;; 6 and 9 give way to 8: frame 5 is on screen from 5 onwards, so that is + ;; where the head's next 0.01 of travel is measured from. + (is (= [0 3 6 9] (select/propose 10 slide {:tolerance 0.01}))) + (is (= [0 3 5 8] (select/propose 10 slide {:tolerance 0.01 :protect [5]}))))) + +(deftest protect-carries-frames-and-extrema-through-one-input + ;; Named frames and the signal's own extrema arrive by the ONE input, which is + ;; what lets the plate selection constrain the performance later without + ;; `propose` growing a second parameter for it. + (let [walk (select/propose 30 aperture {:tolerance tol}) + kept (select/propose 30 aperture {:tolerance tol :protect #{:extrema 8 9}})] + (is (every? (set kept) [closure 8 9])) + (testing "and both kinds anchor the walk, so neither is a union" + ;; 10 was kept only because the walk had drifted from 7 by then. Pinning 8 + ;; and 9 re-anchors it, and 10 is within tolerance of 9. + (is (some #{10} walk)) + (is (not (some #{10} kept))))) + (testing "a frame outside the signal is not a frame" + (is (= [0 5] (select/propose 6 (vec (repeat 6 0.2)) + {:tolerance tol :protect [5 6 30 -1]}))))) + +(deftest the-hand-outranks-the-proposal-in-both-directions + (is (= [0 12 30 47] (select/effective [0 12 30] #{47} #{}))) + (is (= [0 12 47] (select/effective [0 12 30] #{47} #{30}))) + (testing "a drop beats a proposal, always" + (is (= [0] (select/effective [0 4 8] #{} #{4 8})))) + (testing "a drop of a frame nobody proposed is harmless" + (is (= [0 4] (select/effective [0 4] #{} #{9})))) + (testing "something has to be on screen at the start" + (is (= [0] (select/effective [] #{} #{}))) + (is (= [0 4] (select/effective [4] #{} #{0})))) + (testing "the result is a frame list in the shape the two sites already store" + (let [fs (select/effective [30 0 12] [47 12] #{})] + (is (vector? fs)) + (is (= fs (vec (sort (distinct fs)))))))) + +(deftest re-proposing-at-another-tolerance-costs-nobody-their-closure + ;; Step 4's test belongs with storage, but the half of it that is this + ;; namespace's is assertable now: the proposal is a value, so holding a hand + ;; decision across two of them is composition and not state. + (let [pinned #{closure} dropped #{2} + at (fn [t] (select/effective (select/propose 30 aperture {:tolerance t}) + pinned dropped))] + (doseq [t [0.01 0.02 0.05 0.2]] + (is (some #{closure} (at t)) (str "closure survived tolerance " t)) + (is (not (some #{2} (at t))) (str "drop survived tolerance " t))) + (is (not= (at 0.01) (at 0.2))))) + +(deftest an-absent-measurement-is-not-a-change + ;; A frame where the face was not found says nothing about the signal. It is + ;; neither a frame worth keeping nor a reason to move the anchor. + (let [s (assoc (vec (repeat 10 0.2)) 4 nil 5 nil)] + (is (= [0] (select/propose 10 s {:tolerance tol}))) + (is (= [0] (select/propose 10 s {:tolerance tol :protect :extrema})))) + (testing "a signal that starts absent begins where it begins" + (is (= [0 3] (select/propose 6 [nil nil nil 0.2 0.2 0.2] {:tolerance tol})))) + (testing "a measurement either side of a gap is not compared across it" + (is (= [0] (select/propose 5 [0.2 0.2 nil 0.2 0.2] + {:tolerance tol :protect :extrema})))) + (testing "a protected frame with no measurement is still kept" + ;; A plate frame is a frame somebody draws on. The face not having been found + ;; there is a different fact, and not a reason to drop the frame — but there + ;; is nothing to anchor on either, so the anchor stays where it was. + (is (= [0 2] (select/propose 5 [0.2 0.2 nil 0.2 0.2] + {:tolerance tol :protect [2]})))))