Hold a tracing photo steady, and switch it on where you would look

The photo blinked out for a frame or two and sometimes never arrived, from
three causes that each present as the same bug. The still cache emptied
itself on the frame it filled, so every still on screen had to be fetched
and decoded again — every 48 frames of a scrub, and with two traced faces a
permanent flicker, each one's still evicting the other's; it drops the least
recently used now, a still being touched on every frame it is drawn. A still
cannot decode in the animation frame that asks for it, and a face whose next
one had not arrived drew nothing, so it now keeps the still it was showing
until the new one is there. And nothing was read ahead, so continuous
playback was always a frame behind its own footage; the next few frames'
stills are asked for while the playhead is moving on its own, and only then,
because a scrub asks for a different few at every step.

Whether the footage shows is no longer the document's. It was an :underlay
on an instance, inherited down the row path, nearest wins, and it went
through edit — so showing a reference photo was an undo step that travelled
to collaborators. It is [:ui :trace] now, the faces switched on and one
opacity, like solo, and there is nothing left to inherit: a face is the same
face wherever it is placed, so one switch covers every placement of it. The
paint loop asks for its own redraw when that changes, since the resolver no
longer does it for them.

Opening a face shows its footage, because a symbol has measured footage
behind it only because it was traced from that; a take does not, because a
take is the picture. The switch is on the bar above the stage, with the tone
and the tool, and on each face's timeline row beside solo — not a section
that appeared once the right row had been found. A face open in its own tab
could not show its footage at all before, shown having walked instances, and
that is the one place tracing matters most. The inspector keeps the face's
own keyed facts, its trace keys and its origin, and says why it cannot key a
frame rather than greying out the only button in the section.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

frontend/src/arthur/db.cljs

@@ -12,6 +12,7 @@
   are entirely dense."
   (:require [arthur.demo :as demo]
             [arthur.domain.clip :as domain-clip]
+            [arthur.domain.trace :as trace]
             [arthur.demo.swarm :as swarm]
             [arthur.demo.take :as take]))
 
@@ -163,6 +164,7 @@
         :tool      nil
         :draft     []
         :knobs     {}
+        :trace     {:faces #{} :opacity trace/opacity-default}
         :expanded  #{}}})
 
 (def rates

frontend/src/arthur/domain/trace.cljs

@@ -9,13 +9,12 @@
 
       :trace {:frames [0 12 30] :origin :keys}
 
-  Whether the photo is showing, and how strongly, is a viewing aid for one
+  Whether the photo is showing, and how strongly, is not the document's at all.
-  placement. It is on an INSTANCE, not keyed, and draws nothing into the picture.
+  It is a viewing aid, like soloing a row, so it lives in the editor's own state
-  It covers every face at or below that instance, and the nearest instance that
+  — `[:ui :trace]`, a set of face symbols switched on and one opacity — and is
-  says anything decides, so a take shows its faces' footage and one face inside
+  never keyed, saved or exported. A face is the same face wherever it is placed,
-  it can still be switched off:
+  so one switch shows it in the take it is placed in AND in its own tab, which is
-
+  where it is drawn over; nothing has to be switched on twice or per placement.
-      :underlay {:on? true :opacity 0.5}
 
   THE TRACE IS THE ONE FACT about which measured frame a head reads. `:continuous`
   reads the frame it is on, `:keys` jumps to each trace frame's measured head and
@@ -29,6 +28,10 @@
 
 (def origins [:continuous :keys :start])
 
+(def ^:const opacity-default
+  "How strongly a switched-on photo draws until someone moves the slider."
+  0.5)
+
 (defn of
   "The head's trace, a head that has none being one that moves freely."
   [head]
@@ -117,31 +120,38 @@
                     (sort-by (comp str key) (get-in clip [:symbols sid :nodes]))))]
     (vec (walk sid []))))
 
-(defn underlay-at
+(defn traceable-faces
-  "The underlay in force at the instance at row path `path` from symbol `sid`:
+  "Every face that can be traced while symbol `sid` is open, each once: `sid`
-  the nearest one set on it or above it, `:own?` saying which. Nil when none is."
+  itself when it is a face — open in its own tab, to be drawn over — and the
-  [clip sid path]
+  faces placed inside it at any depth."
-  (:u (reduce (fn [{:keys [sid u]} id]
+  [clip sid]
-                (let [n (get-in clip [:symbols sid :nodes id])]
+  (into [] (distinct)
-                  {:sid (:of n)
+        (cond->> (map :face (faces clip sid))
-                   :u   (if-let [own (:underlay n)]
+          (traceable? clip sid) (cons sid))))
-                          (assoc own :own? true)
+
-                          (some-> u (assoc :own? false)))}))
+(defn showing-for
-              {:sid sid} path)))
+  "The faces showing their footage once `sid` is the open symbol, given the ones
+  `on` already showing.
+
+  OPENING A FACE IS ASKING TO DRAW OVER IT: a symbol has measured footage behind
+  it only because it was traced from that footage, so its own tab starts with the
+  footage showing rather than with a switch to be found first. A take or a scene
+  is the picture itself, and a reference drawn over one would read as part of it,
+  so nothing is switched on for those. Either way it is switched by hand
+  afterwards, from the bar above the stage or from a face's own timeline row."
+  [clip sid on]
+  (cond-> (set on) (traceable? clip sid) (conj sid)))
 
 (defn shown
-  "Every face whose footage shows from symbol `sid` down, as `{:path :face
+  "The faces whose footage is showing while symbol `sid` is open, as `faces` lists
-  :opacity}`: the row path of the face's instance, the face's symbol, and how
+  them — the row path of the face's instance from `sid`, and the face — filtered
-  strongly to draw it."
+  to the `showing` set.
-  [clip sid]
+
-  (letfn [(walk [sid path opacity]
+  The open symbol itself is in the list, at the EMPTY path, when it is a face:
-            (mapcat (fn [[id n]]
+  that is a face open in its own tab to be drawn over, and the one place tracing
-                      (when (= :instance (:kind n))
+  matters most. There is no inheritance to work out and no opacity to carry
-                        (let [path    (conj path id)
+  because the switch is the face's, not a placement's."
-                              u       (:underlay n)
+  [clip sid showing]
-                              opacity (if u (when (:on? u) (:opacity u 0.5)) opacity)]
+  (into [] (filter (comp (set showing) :face))
-                          (cond->> (walk (:of n) path opacity)
+        (cond->> (faces clip sid)
-                            (and opacity (traceable? clip (:of n)))
+          (traceable? clip sid) (cons {:path [] :in sid :face sid}))))
-                            (cons {:path path :face (:of n) :opacity opacity})))))
-                    (get-in clip [:symbols sid :nodes])))]
-    (vec (walk sid [] nil))))

frontend/src/arthur/events/playback.cljs

@@ -13,6 +13,7 @@
   (:require [arthur.audio.mix :as mix]
             [arthur.clock :as clock]
             [arthur.domain.clip :as clip]
+            [arthur.domain.trace :as trace]
             [arthur.footage.store :as footage]
             [re-frame.core :as rf]))
 
@@ -36,7 +37,13 @@
     (-> db
         (assoc :clip/current id
                :clip (select-keys entry [:fps :width :height :audio :display-fps]))
-        (update :ui merge {:open sid :tabs (if sid [sid] [])})
+        (update :ui merge
+                {:open sid :tabs (if sid [sid] [])
+                 ;; From scratch, not merged: the faces switched on were another
+                 ;; document's, and a face id means nothing in this one.
+                 :trace {:faces (trace/showing-for (:clip entry) sid #{})
+                         :opacity (or (get-in db [:ui :trace :opacity])
+                                      trace/opacity-default)}})
         (assoc-in [:playback :frame] 0)
         (assoc-in [:playback :playing?] false))))
 
@@ -188,6 +195,7 @@
        {:db (-> db
                 (update-in [:ui :tabs] #(if (some #{sid} %) % (conj (vec %) sid)))
                 (assoc-in [:ui :open] sid)
+                (update-in [:ui :trace :faces] #(trace/showing-for clip sid %))
                 (assoc-in [:playback :frame] 0)
                 (assoc-in [:playback :playing?] false))
         ::pause! nil

frontend/src/arthur/events/project.cljs

@@ -630,12 +630,6 @@
  (fn [db [_ sid value]]
    (edit/edit db #(assoc-in % [:symbols sid :nodes :head :trace] value))))
 
-;; Whether one instance shows its face's footage under it. Not keyed: it is a
-;; drawing aid, not part of the picture.
-(rf/reg-event-db
- ::set-underlay
- (fn [db [_ sid id underlay]]
-   (edit/edit db #(assoc-in % [:symbols sid :nodes id :underlay] underlay))))
 
 (rf/reg-event-db
  ::set-segment-interp

frontend/src/arthur/events/ui.cljs

@@ -49,6 +49,32 @@
                       (= on #{path})    #{}
                       :else             #{path})))))
 
+(rf/reg-event-db
+ ::trace-face
+ ;; Showing the footage under a face is a viewing aid, like solo: editor state
+ ;; rather than the document, so it is not an undo step, does not travel to a
+ ;; collaborator and cannot reach an export. Per FACE and not per placement — a
+ ;; face is the same face wherever it is placed, and it is the face being traced
+ ;; — so one switch shows it in the take and in its own tab both.
+ (fn [db [_ face]]
+   (update-in db [:ui :trace :faces]
+              #(if (contains? % face) (disj % face) (conj (set %) face)))))
+
+(rf/reg-event-db
+ ::trace-faces
+ ;; Every face the open symbol has, from the bar above the stage: switched on
+ ;; unless they all already are, which is the one gesture a person wants when
+ ;; there is exactly one face and when there are five.
+ (fn [db [_ faces]]
+   (let [faces (set faces)
+         on    (set (get-in db [:ui :trace :faces]))]
+     (assoc-in db [:ui :trace :faces]
+               (if (every? on faces) (reduce disj on faces) (into on faces))))))
+
+(rf/reg-event-db
+ ::trace-opacity
+ (fn [db [_ opacity]] (assoc-in db [:ui :trace :opacity] opacity)))
+
 (defn- where-new-goes
   "The row path, from the open symbol down, of the symbol a new thing goes into:
   INSIDE the selected instance, or BESIDE any other selected node, or at the top

frontend/src/arthur/subs/render.cljs

@@ -24,6 +24,7 @@
 (rf/reg-sub ::sliding (fn [db _] (get-in db [:ui :sliding])))
 (rf/reg-sub ::gesture (fn [db _] (get-in db [:ui :gesture])))
 (rf/reg-sub ::solo (fn [db _] (get-in db [:ui :solo (get-in db [:ui :open])])))
+(rf/reg-sub ::tracing (fn [db _] (get-in db [:ui :trace])))
 
 (rf/reg-sub
  ::clip
@@ -163,12 +164,19 @@
  :<- [::store]
  :<- [::open]
  :<- [::solo]
- (fn [[id document store open solo] _]
+ :<- [::tracing]
-   ;; What `ui/underlay` needs to paint the footage under the faces being
+ (fn [[id document store open solo {:keys [faces opacity]}] _]
-   ;; traced, besides the resolver that says where they went. A face outside
+   ;; What `ui/underlay` needs to paint the footage of the faces being traced,
-   ;; every soloed row is not on stage, so neither is its footage.
+   ;; besides the resolver that says where they went. A face outside every soloed
+   ;; row is not on stage, so neither is its footage.
    (let [solo (filter #(placed? document open %) solo)]
      {:document document :store store
       :footage-id (:footage-id (footage/entry id))
-      :traces (cond->> (when document (trace/shown document open))
+      :opacity (or opacity trace/opacity-default)
-                (seq solo) (filterv (fn [{:keys [path]}] (some #(= % (take (count %) path)) solo))))})))
+      :traces (cond->> (when document (trace/shown document open faces))
+                (seq solo) (filterv (fn [{:keys [path]}]
+                                      ;; The open symbol's own face is at the
+                                      ;; empty path: it is not under any row, so
+                                      ;; soloing a row cannot hide it.
+                                      (or (empty? path)
+                                          (some #(= % (take (count %) path)) solo)))))})))

frontend/src/arthur/ui/palette.cljs

@@ -11,9 +11,18 @@
 
   Slot 0 is the background, which is why it is shown and not selectable: a
   polygon filled with index 0 is invisible against a stage cleared to index 0, so
-  offering it as a fill is offering a shape that vanishes on creation."
+  offering it as a fill is offering a shape that vanishes on creation.
+
+  The footage switch is here too, because it is the same kind of thing as the
+  tone and the tool: something you set before you draw and leave alone, in one
+  place whatever is selected. In the inspector it was a section that appeared
+  only once the right row had been found — so the way to see the footage you are
+  tracing depended on what you had clicked, which is not a thing anyone can be
+  expected to learn."
   (:require [arthur.domain.palette :as pal]
+            [arthur.domain.trace :as trace]
             [arthur.events.ui :as ui]
+            [arthur.subs.render :as render]
             [arthur.subs.ui :as sub]
             [re-frame.core :as rf]))
 
@@ -34,6 +43,31 @@
       :disabled (not pick)
       :on-click #(rf/dispatch [::ui/set-tone slot-tone])}]))
 
+(defn- tracing
+  "The footage under the faces the open symbol has, on or off and how strongly.
+
+  ONE SWITCH FOR THE FACES THAT ARE HERE. A face's footage is the face's, not a
+  placement's, so there is nothing to inherit and nothing to set twice; with
+  several faces in a take the box says how many are showing and switches the rest
+  on, and one face alone is switched from its own timeline row."
+  []
+  (let [clip  @(rf/subscribe [::render/clip])
+        open  @(rf/subscribe [::render/open])
+        {:keys [faces opacity]} @(rf/subscribe [::render/tracing])
+        here  (trace/traceable-faces clip open)
+        on    (filterv (set faces) here)]
+    (when (seq here)
+      [:<>
+       [:label.dim {:title (str "show the footage these faces were traced from, over the "
+                               "picture · a reference, never exported")}
+        [:input {:type "checkbox" :checked (= (count on) (count here))
+                 :on-change #(rf/dispatch [::ui/trace-faces here])}]
+        (str " footage" (when (< 1 (count here)) (str " " (count on) "/" (count here))))]
+       [:input.trace-opacity
+        {:type "range" :min 0 :max 1 :step 0.05 :title "how strongly the footage draws"
+         :value (or opacity trace/opacity-default) :disabled (empty? on)
+         :on-change #(rf/dispatch [::ui/trace-opacity (js/parseFloat (.. % -target -value))])}]])))
+
 (defn bar []
   (let [tone  @(rf/subscribe [::sub/tone])
         tool  @(rf/subscribe [::sub/tool])
@@ -42,6 +76,7 @@
      [:div.swatches (doall (map #(swatch % tone) (range slots)))]
      [:span.dim (name tone)]
      [:span {:style {:flex 1}}]
+     [tracing]
      (if (= :polygon tool)
        [:<>
         [:span.dim (str (quot (count draft) 2) " points")]

frontend/src/arthur/ui/params.cljs

@@ -236,21 +236,32 @@
 ;; ---------------------------------------------------------------------------
 ;; tracing a face
 ;;
-;; Two owners in one section, and the heading says whose each is. Showing the
+;; THE FACE'S OWN FACTS ONLY, and both of them are keyed to the face rather than
-;; footage and how strongly is this INSTANCE's, a drawing aid that is not keyed,
+;; to a placement of it: which of its frames its drawings were made over, and
-;; and it covers every face at or below it — so a take shows its faces' footage.
+;; what its origin does between those frames. See `domain/trace`.
-;; The trace keys and the origin are the FACE's — its symbol's head — so they are
+;;
-;; the same in every placement of it. See `domain/trace`.
+;; Whether the footage is SHOWING is deliberately not here. It is a viewing aid
+;; and it belongs with the other things you look through rather than edit, on the
+;; bar above the stage — where it is in the same place whatever is selected,
+;; instead of appearing in the inspector only once the right row has been found.
 
-(defn- face-trace [face]
+(defn- trace-keys
+  "The face's trace keys and origin. `frame` is the frame of the FACE that the
+  playhead is over, and `seek!` goes to one of its frames — both of which depend
+  on whether the face is open in its own tab or placed in what is."
+  [face frame seek!]
   (let [t    (trace/of (get-in @(rf/subscribe [::render/clip]) [:symbols face :nodes :head]))
-        {:keys [frame time]} @(rf/subscribe [::sub/selected-local])
+        put  #(rf/dispatch [::project/set-trace face %])
-        put   #(rf/dispatch [::project/set-trace face %])]
+        key? (boolean (some #{frame} (:frames t)))]
     [:<>
      [:div.row {:style {:margin "5px 0"}}
-      [:button {:disabled (nil? frame)
+      ;; A DEAD BUTTON WITH NO REASON GIVEN is what reads as the feature not
-                :on-click #(put (trace/toggle-frame t frame))}
+      ;; working. There is no frame of this face under the playhead, so say that
-       (if (some #{frame} (:frames t)) "remove trace key" "trace key here")]]
+      ;; rather than greying out the one control in the section.
+      (if (nil? frame)
+        [:span.dim "move the playhead over this face to key it"]
+        [:button {:on-click #(put (trace/toggle-frame t frame))}
+         (if key? (str "remove trace key at " frame) (str "trace key at " frame))])]
      (when (seq (:frames t))
        [:div.row
         [:span.dim "keys"]
@@ -258,9 +269,8 @@
          (for [f (:frames t)]
            ^{:key f}
            [:button {:class (when (= frame f) "on")
-                     :disabled (nil? time)
+                     :disabled (nil? seek!)
-                     :on-click #(rf/dispatch [::pb/seek (js/Math.round
+                     :on-click #(seek! f)}
-                                                         (+ (:at time) (/ f (:rate time))))])}
             (str f)]))])
      [:div.row {:style {:margin-top "5px"}}
       [:span.dim "origin"]
@@ -268,36 +278,38 @@
        (for [[o label] (map vector trace/origins ["continuous" "at keys" "start"])]
          ^{:key o}
          [:button {:class (when (= o (:origin t)) "on")
+                   :title (case o
+                            :continuous "the head reads the frame it is on"
+                            :keys "the head jumps to each trace key and holds it"
+                            :start "the head holds frame 0 forever")
                    :on-click #(put (assoc t :origin o))}
           label]))]]))
 
-(defn- tracing-section [[sid id n] faces]
+(defn- tracing-section
+  "`face` is the face these facts belong to, `faces` the faces placed inside it
+  to offer as somewhere to go next, and `path` the row path `faces` are under."
+  [face faces path]
   (let [clip @(rf/subscribe [::render/clip])
-        open  @(rf/subscribe [::render/open])
+        own? (= face @(rf/subscribe [::render/open]))
-        [_ _ _ path] @(rf/subscribe [::sub/selection])
+        ;; The face's OWN frame, which is the playhead itself when the face is
-        path  (or path [id])
+        ;; the open symbol, and the selected placement's local frame when it is
-        {:keys [on? opacity own?] :or {opacity 0.5} :as u} (trace/underlay-at clip open path)
+        ;; placed in it. `time` maps the open symbol's frames to that
-        show  #(rf/dispatch [::project/set-underlay sid id (merge {:on? (boolean on?) :opacity opacity} %)])]
+        ;; placement's, so seeking to one of the face's frames is a conversion.
-    [section (str "tracing · " (name (:of n)))
+        {:keys [frame time]} (when-not own? @(rf/subscribe [::sub/selected-local]))
-     [:div.row
+        frame (if own? @(rf/subscribe [::playback/frame]) frame)
-      [:label.dim [:input {:type "checkbox" :checked (boolean on?)
+        seek! (cond own? #(rf/dispatch [::pb/seek %])
-                           :on-change #(show {:on? (.. % -target -checked)})}]
+                    time #(rf/dispatch [::pb/seek (js/Math.round
-       " footage"
+                                                   (+ (:at time) (/ % (:rate time))))]))]
-       (when (and u (not own?)) " · as above")]
+    [section (str "tracing · " (name face))
-      [:input {:type "range" :min 0 :max 1 :step 0.05 :value opacity
+     (when (trace/traceable? clip face) [trace-keys face frame seek!])
-               :disabled (not on?)
-               :on-focus #(rf/dispatch [::history/hold])
-               :on-blur #(rf/dispatch [::history/settle])
-               :on-change #(show {:opacity (js/parseFloat (.. % -target -value))})}]]
-     (when (trace/traceable? clip (:of n)) [face-trace (:of n)])
      (when (seq faces)
        [:div.row {:style {:margin-top "5px"}}
         [:span.dim "faces"]
         (doall
-         (for [{p :path in :in face :face} faces]
+         (for [{p :path in :in f :face} faces]
            ^{:key (str p)}
            [:button {:on-click #(rf/dispatch [::ui/select [:node in (peek p) (into path p)]])}
-            (name face)]))])]))
+            (name f)]))])]))
 
 ;; ---------------------------------------------------------------------------
 ;; a symbol
@@ -416,17 +428,29 @@
 
 (defn view []
   (let [clip      @(rf/subscribe [::render/clip])
+        open      @(rf/subscribe [::render/open])
         selection @(rf/subscribe [::sub/selection])
         node      @(rf/subscribe [::sub/selected-node])
-        tracked?  (seq (owners clip))]
+        tracked?  (seq (owners clip))
+        ;; The face the tracing section is about: the SELECTED PLACEMENT's symbol,
+        ;; or, when the selection is not a placement or there is none, the OPEN
+        ;; symbol — which is the face itself when a face is open to be drawn over.
+        ;; That last case had no section at all before, and it is the one where
+        ;; the keys are actually being set.
+        placed    (:of (peek node))
+        face      (or placed (when (trace/traceable? clip open) open))
+        faces     (when face (trace/faces clip face))
+        ;; Where that face sits, as a row path from the open symbol, so the faces
+        ;; inside it can be selected by their own rows. A selection made on the
+        ;; stage has no path and names a node directly in the open symbol; the
+        ;; open symbol itself is at no path at all.
+        path      (if placed (or (nth selection 3 nil) [(second node)]) [])]
     [:section.pane.params
      [:div.pane-head "inspector"]
      [:div {:style {:min-height 0}}
       [clip-section]
       (when node [node-section node])
-      (when-let [of (:of (peek node))]
+      (when (and face (or (trace/traceable? clip face) (seq faces)))
-        (let [faces (trace/faces clip of)]
+        [tracing-section face faces path])
-          (when (or (trace/traceable? clip of) (seq faces))
-            [tracing-section node faces])))
       (when (= :symbol (first selection)) [symbol-section (second selection)])
       (when tracked? [tracking-section])]]))

frontend/src/arthur/ui/player.cljs

@@ -101,8 +101,9 @@
   (some-> @tracker ratom/dispose!)
   (reset! tracker
           (ratom/run!
-           (let [was (:resolver @snapshot)
+           (let [{was :resolver was-u :underlay} @snapshot
-                 now @(rf/subscribe [::render/shown])]
+                 now @(rf/subscribe [::render/shown])
+                 u   @(rf/subscribe [::render/underlay])]
              (reset! snapshot
                      {:resolver now
                       :palette  @(rf/subscribe [::render/palette])
@@ -111,12 +112,23 @@
                       :frames   @(rf/subscribe [::render/frames])
                       :width    @(rf/subscribe [::sub/width])
                       :height   @(rf/subscribe [::sub/height])
-                      :underlay @(rf/subscribe [::render/underlay])
+                      :underlay u
                       :frame    @(rf/subscribe [::sub/frame])
                       :playing? @(rf/subscribe [::sub/playing?])})
              ;; A new resolver means a new scene or a new palette, and neither
              ;; moves the playhead — so nothing else would ask for a redraw.
-             (when-not (identical? was now) (repaint!))))))
+             ;;
+             ;; Switching a face's footage on wants the same redraw for the same
+             ;; reason, and it needs asking for SEPARATELY: it is a viewing aid
+             ;; in the editor's own state, so it changes what the canvases should
+             ;; show without touching the frame number OR the resolver. The
+             ;; document and the store are left out of the comparison because
+             ;; they are what a new resolver already means.
+             ;; Comparing the whole map is as cheap as picking fields out of it:
+             ;; the document and the store it carries are the same OBJECTS unless
+             ;; the resolver changed too, and that is tested first.
+             (when-not (and (identical? was now) (= was-u u))
+               (repaint!))))))
 
 (defn set-canvas! [el]
   (swap! state assoc :canvas el)
@@ -146,7 +158,7 @@
   rasterised before the next frame is asked for."
   [f]
   (let [{:keys [canvas]} @state
-        {:keys [resolver palette ramp width height underlay]} @snapshot]
+        {:keys [resolver palette ramp width height underlay playing?]} @snapshot]
     (when (and canvas resolver width height)
       ;; User Timing, so a profile in the DevTools performance panel has named
       ;; spans in the Timings track instead of a wall of anonymous frames. Three
@@ -163,7 +175,8 @@
               (raster/draw-ops! ops)))
         (js/performance.mark "arthur/blit:start")
         (canvas/blit! canvas ras ramp)
-        (underlay/paint! (assoc underlay :width width) resolver repaint!))
+        (underlay/paint! (assoc underlay :width width :playing? playing?)
+                         resolver repaint!))
       (js/performance.measure "arthur/resolve+draw" "arthur/paint:start" "arthur/blit:start")
       (js/performance.measure "arthur/paint" "arthur/paint:start")
       ;; User Timing entries otherwise accumulate forever in the browser's

frontend/src/arthur/ui/timeline.cljs

@@ -22,6 +22,7 @@
   happen and not where they happen again."
   (:require [clojure.string :as str]
             [arthur.domain.node :as node]
+            [arthur.domain.trace :as trace]
             [arthur.events.playback :as pb]
             [arthur.events.ui :as ui]
             [arthur.subs.playback :as playback]
@@ -266,7 +267,7 @@
   (memoize (fn [_selection] (fn [el] (some-> el (.scrollIntoView #js {:block "nearest"}))))))
 
 (defn- label-cell [{:keys [path depth label kind node-kind select expandable? expanded? of via]}
-                   selection over solo]
+                   selection over solo tracing]
   (let [node? (= :node kind)
         [over-path where] @over]
     [:div (cond-> {:class (str "tl-label" (when (and select (= select selection)) " on")
@@ -323,6 +324,17 @@
       (when expandable? (if expanded? "▾" "▸"))]
      [:span.name label]
      (when node? [:span.kind (if via (str "· in " via) (str "·" (name node-kind)))])
+     ;; A face's row is where its own footage is switched on, next to solo
+     ;; because the two are the same kind of thing: what this row shows, here,
+     ;; now, and nothing the picture keeps. The bar above the stage does all of
+     ;; them at once; this is how one face out of a take is singled out.
+     (when (contains? (:faces tracing) of)
+       [:button {:class    (str "tl-trace" (when (contains? (:on tracing) of) " on"))
+                 :title    "show the footage this face was traced from"
+                 :on-click (fn [^js e]
+                             (.stopPropagation e)
+                             (rf/dispatch [::ui/trace-face of]))}
+        "T"])
      (when (= :instance node-kind)
        [:button {:class    (str "tl-solo" (when (contains? solo path) " on"))
                  :title    "show only this on the stage (⇧ for more than one)"
@@ -402,6 +414,11 @@
           drop      @(rf/subscribe [::sub/drop])
           solo      (set @(rf/subscribe [::render/solo]))
           open      @(rf/subscribe [::render/open])
+          ;; Which rows offer a footage switch, and which of them are switched
+          ;; on. A set rather than a lookup per row: the answer is the same for
+          ;; every placement of a face, because the switch is the face's.
+          tracing   {:faces (set (trace/traceable-faces clip open))
+                     :on    (set (:faces @(rf/subscribe [::render/tracing])))}
           ;; Where a drag out of the pool would land, as a row of its own at the
           ;; top of its section: its own length, starting on the frame it would
           ;; start on. The stage's drop shows it too, at the playhead.
@@ -441,7 +458,7 @@
          (doall (for [row visible]
                   (with-meta (if (= :section (:kind row))
                                [:div.tl-label.tl-section (:label row)]
-                               [label-cell row selection over solo])
+                               [label-cell row selection over solo tracing])
                     {:key (str (:path row))})))]
         [:div.tl-tracks
          {:on-drag-enter (fn [^js e] (when (drag/accepts?) (.preventDefault e)))

frontend/src/arthur/ui/underlay.cljs

@@ -2,18 +2,27 @@
   "The footage a face is traced over, on its own canvas above the stage.
 
   A REFERENCE, NOT OUTPUT. The tracing still never enters the indexed raster, so
-  it cannot reach an export, and it is drawn OVER the picture at the instance's
+  it cannot reach an export, and it is drawn OVER the picture at the face's
   opacity rather than under it, because the raster clears to an opaque ground.
   The canvas is the stage's size on screen, not the raster's, so a 1280px still
   is not squeezed through a 320px stage on its way to being seen.
 
   Painted by `ui/player` straight after each frame, from the same snapshot, so
-  it moves with the face it registers to — see `domain/trace/photo-matrix`."
+  it moves with the face it registers to — see `domain/trace/photo-matrix`.
+
+  THREE THINGS KEEP IT STEADY, because a still is decoded asynchronously and the
+  frame it belongs to is already on screen by the time it arrives: the cache drops
+  the stills used longest ago rather than emptying itself, a face keeps showing
+  the still it last showed until its next one has decoded, and the stills a little
+  way ahead of the playhead are asked for before they are needed. Each is a
+  separate cause of the same symptom — the tracing blinking out for a frame or two
+  — and none of them covers the others: the hold is what survives a seek, the
+  reading ahead is what keeps playback from being a frame behind for good."
   (:require [arthur.domain.symbol :as symbol]
             [arthur.domain.trace :as trace]
             [arthur.flow.ingest :as ingest]))
 
-(defonce ^:private state (atom {:canvas nil :urls {} :images {}}))
+(defonce ^:private state (atom {:canvas nil :urls {} :images {} :order [] :last {}}))
 
 (defn set-canvas! [el] (swap! state assoc :canvas el))
 
@@ -37,45 +46,134 @@
   scrubbing a face with no trace keys asks for every frame of the take."
   48)
 
+(def ^:private ^:const ahead
+  "Frames past the playhead whose stills are asked for while the current one is
+  drawn. A still cannot be decoded in the animation frame that wants it, so
+  playing a continuously traced face would otherwise always be a frame or two
+  behind its own footage."
+  8)
+
+(defn- decoded
+  "`img` if the still has arrived and can be drawn."
+  [^js img]
+  (when (and (.-complete img) (pos? (.-naturalHeight img))) img))
+
+(defn- touch!
+  "Hold `img` under `url` as the most recently used still, dropping the ones used
+  longest ago once there are more than `kept`.
+
+  LEAST RECENTLY USED, touched when it arrives and again on every frame it is
+  DRAWN, which is what makes it safe without a list of exceptions: the still on
+  screen is the newest thing in the cache and cannot be the one dropped — a still
+  held at a trace key for a hundred frames included. Read-and-not-drawn does not
+  touch, because a still read only to be warmed was just inserted anyway, and
+  reordering the whole list ten times per face per frame in the draw loop is the
+  kind of allocation `ui/player` exists to avoid. What this replaces emptied the
+  cache on the frame it filled, so every still on screen had to be fetched and
+  decoded again: a blink every `kept` frames of a scrub, and with two faces and a
+  full cache a permanent one."
+  [url img]
+  (swap! state
+         (fn [{:keys [images order] :as s}]
+           (let [images (assoc images url img)
+                 order  (conj (into [] (remove #{url}) order) url)
+                 over   (- (count order) kept)]
+             (if (pos? over)
+               (assoc s :images (apply dissoc images (subvec order 0 over))
+                        :order  (subvec order over))
+               (assoc s :images images :order order))))))
+
 (defn- image
-  "The still at `url`, or nil until it has loaded."
+  "The still at `url`, or nil until it has loaded. Asks for it the first time it
+  is wanted; `on-ready` paints again once it is there.
+
+  A still that fails is remembered as having failed, so a broken URL is one
+  console line and one request rather than one of each per animation frame."
   [url on-ready]
-  (let [img (or (get-in @state [:images url])
+  (when url
-                (let [img (js/Image.)]
+    (let [held (get-in @state [:images url])]
+      (cond
+        (= :failed held) nil
+        held             (decoded held)
+        :else            (let [img (js/Image.)]
                            (set! (.-onload img) on-ready)
+                           (set! (.-onerror img)
+                                 (fn [_]
+                                   (swap! state assoc-in [:images url] :failed)
+                                   (js/console.error "arthur: a tracing still did not load" url)))
                            (set! (.-src img) url)
-                  (swap! state update :images
+                           (touch! url img)
-                         #(assoc (if (< (count %) kept) % {}) url img))
+                           nil)))))
-                  img))]
+
-    (when (and (.-complete img) (pos? (.-naturalHeight img))) img)))
+(defn- still
+  "The URL of the still showing the face's own frame `p`. The manifest is the
+  whole footage's and `p` is an index into the analysed range, so the range's
+  start is where the face's frame 0 was filmed."
+  [us start p]
+  (get us (+ start p)))
+
+(defn- warm!
+  "Ask for the stills the next `ahead` frames will want. Held trace frames
+  collapse to the one still, so a face traced at keys asks for almost nothing.
+
+  WHILE PLAYING ONLY, because that is the only time the next frame is the one
+  after this one. A scrub asks for a different `ahead` frames at every step, so
+  reading ahead through one would be `ahead` requests per step for stills the
+  pointer has already gone past — and a cache thrashed by its own guesses."
+  [us start t lf on-ready]
+  (doseq [p (distinct (map #(trace/photo-frame t %) (range (inc lf) (+ lf 1 ahead))))]
+    (image (still us start p) on-ready)))
 
 (defn paint!
-  "Draw every switched-on underlay on the frame `resolver` last resolved. It
+  "Draw every switched-on face's footage on the frame `resolver` last resolved. It
   says where each face's head went and which of its frames it was on, so this
   reads the frame rather than resolving it again. `on-ready` is called when a
   still or a manifest that was missing arrives, to paint again."
-  [{:keys [document store footage-id traces width]} resolver on-ready]
+  [{:keys [document store footage-id traces opacity width playing?]} resolver on-ready]
   (when-let [^js canvas (:canvas @state)]
     (let [ctx  (.getContext canvas "2d")
           zoom (/ (.-width canvas) width)]
       (.setTransform ctx 1 0 0 1 0 0)
       (.clearRect ctx 0 0 (.-width canvas) (.-height canvas))
+      ;; What each face is showing, forgotten for the faces switched off. Before
+      ;; the early exits, so switching them all off forgets all of them.
+      (swap! state update :last select-keys (map :path traces))
       (when-let [us (and (seq traces) footage-id (urls footage-id on-ready))]
-        (let [start (first (get-in document [:analysis :range] [0]))]
+        (let [start (first (get-in document [:analysis :range] [0]))
-          (doseq [{:keys [path face opacity]} traces
+              ;; Read once: each face writes only its own entry below, and what it
-                  :let [at    (conj path :head)
+              ;; was showing is what it falls back to. A face that has been off
+              ;; stage for a few frames still has the still it went away with.
+              was   (:last @state)]
+          ;; ONE OPACITY for all of them, set once: how strongly the reference
+          ;; draws is a property of looking at the stage, not of a face.
+          (set! (.-globalAlpha ctx) opacity)
+          (doseq [{:keys [path face]} traces]
+            (let [at    (conj path :head)
                   world (symbol/world-of resolver at)
                   frame (symbol/frame-of resolver at)]
-                  :when (and world (number? frame))
+              (when (and world (number? frame))
-                  :let [head (get-in document [:symbols face :nodes :head])
+                (let [head (get-in document [:symbols face :nodes :head])
-                        p    (trace/photo-frame (trace/of head) (js/Math.floor frame))
+                      t    (trace/of head)
-                        img  (some-> (get us (+ start p)) (image on-ready))
+                      lf   (js/Math.floor frame)
+                      want (trace/photo-frame t lf)
+                      ;; THE STILL IT HAS, not nothing. A frame whose still is
+                      ;; still decoding keeps the one before it — a reference a
+                      ;; frame stale, registered where that frame's face was,
+                      ;; rather than a face with its footage blinking off.
+                      [img url p] (or (let [u (still us start want)]
+                                        (when-let [i (image u on-ready)] [i u want]))
+                                      (let [{:keys [url p]} (get was path)]
+                                        (when-let [i (image url on-ready)] [i url p])))
                       m    (when img
                              (trace/photo-matrix world head store p (.-naturalHeight img)))]
-                  :when m]
+                  (when m
-            (set! (.-globalAlpha ctx) opacity)
+                    ;; Drawn, so it is the newest still in the cache and what
+                    ;; this face falls back to while its next one decodes.
+                    (touch! url img)
+                    (swap! state assoc-in [:last path] {:url url :p p})
                     (.setTransform ctx
                                    (* zoom (aget m 0)) (* zoom (aget m 1))
                                    (* zoom (aget m 2)) (* zoom (aget m 3))
                                    (* zoom (aget m 4)) (* zoom (aget m 5)))
-            (.drawImage ctx img 0 0)))))))
+                    (.drawImage ctx img 0 0))
+                  (when playing? (warm! us start t lf on-ready)))))))))))

frontend/test/arthur/domain/trace_test.cljs

@@ -70,39 +70,47 @@
         "a continuous head carries the held photo along with it")))
 
 (defn- wrapped
-  "Face-1's take placed, moved, inside a symbol :wrap, with `underlays` on the
+  "Face-1's take placed, moved, inside a symbol :wrap."
-  instances named."
+  []
-  [{:keys [outer inner]}]
+  (assoc-in @frozen [:symbols :wrap]
-  (-> @frozen
+            {:id :wrap :frames 200
-      (assoc-in [:symbols :wrap] {:id :wrap :frames 200
+             :nodes {:m {:id :m :kind :instance :of :main :z "a0"
-                                  :nodes {:m (cond-> {:id :m :kind :instance :of :main :z "a0"
+                         :channels {[:xform :pos] {:animated? false :value [30 -10]}}}}}))
-                                                      :channels {[:xform :pos] {:animated? false
-                                                                                :value [30 -10]}}}
-                                               outer (assoc :underlay outer))}})
-      (cond-> inner (assoc-in [:symbols :main :nodes :face-1 :underlay] inner))))
 
-(deftest an-underlay-covers-the-faces-below-it-and-the-nearest-decides
+(deftest a-face-switched-on-shows-wherever-it-is-placed
-  (is (= [{:path [:m :face-1] :face :face-1 :opacity 0.3}]
+  (let [c (wrapped)]
-         (trace/shown (wrapped {:outer {:on? true :opacity 0.3}}) :wrap))
+    (is (= [{:path [:m :face-1] :in :main :face :face-1}] (trace/shown c :wrap #{:face-1}))
-      "switched on at the take, its face shows")
+        "a face inside a take inside a symbol, at the row path it is at")
-  (is (= [] (trace/shown (wrapped {:outer {:on? true} :inner {:on? false}}) :wrap))
+    (is (= [] (trace/shown c :wrap #{})) "and nothing when it is switched off")
-      "and the face can still be switched off inside it")
+    (is (= [{:path [:face-1] :in :main :face :face-1}] (trace/shown c :main #{:face-1}))
-  (is (= [{:path [:m :face-1] :face :face-1 :opacity 0.8}]
+        "the same switch, one symbol down")
-         (trace/shown (wrapped {:inner {:on? true :opacity 0.8}}) :wrap)))
+    (is (= [{:path [] :in :face-1 :face :face-1}] (trace/shown c :face-1 #{:face-1}))
-  (let [c (wrapped {:outer {:on? true :opacity 0.3}})]
+        "the face open in its own tab is at no path at all — it IS the stage")
-    (is (= {:on? true :opacity 0.3 :own? false} (trace/underlay-at c :wrap [:m :face-1])))
+    (is (= [] (trace/shown c :wrap #{:main}))
-    (is (= {:on? true :opacity 0.3 :own? true} (trace/underlay-at c :wrap [:m])))
+        "a symbol that is not a face has no footage of its own to show")))
-    (is (nil? (trace/underlay-at @frozen :main [:face-1])))))
+
+(deftest the-faces-that-can-be-traced-are-listed-once-each
+  (is (= [:face-1] (trace/traceable-faces (wrapped) :wrap)))
+  (is (= [:face-1] (trace/traceable-faces @frozen :main)) "the take it was frozen into")
+  (is (= [:face-1] (trace/traceable-faces @frozen :face-1)) "itself, open to draw over"))
+
+(deftest a-face-opened-to-be-drawn-over-starts-with-its-footage-showing
+  (let [c (wrapped)]
+    (is (= #{:face-1} (trace/showing-for c :face-1 #{})) "the face's own tab")
+    (is (= #{} (trace/showing-for c :main #{}))
+        "and not the take it is placed in, which is the picture itself")
+    (is (= #{:face-1} (trace/showing-for c :main #{:face-1}))
+        "one already switched on stays on wherever you go")))
 
 (deftest a-take-lists-the-faces-in-it
   (is (= [{:path [:face-1] :in :main :face :face-1}] (trace/faces @frozen :main)))
-  (is (= [{:path [:m :face-1] :in :main :face :face-1}] (trace/faces (wrapped {}) :wrap)))
+  (is (= [{:path [:m :face-1] :in :main :face :face-1}] (trace/faces (wrapped) :wrap)))
   (is (= [] (trace/faces @frozen :face-1))))
 
 (deftest the-resolver-says-where-a-nested-head-went-on-its-last-frame
   ;; The same answer as `nest/placement`, which walks and resolves the path all
   ;; over again — the resolver has it already, from drawing the frame.
-  (let [c (wrapped {})
+  (let [c (wrapped)
         r (clip/resolver c @store pal/index-of :wrap)
         path [:m :face-1 :head]]
     (doseq [f [0 17 60]]

static/arthur/app.css

@@ -435,6 +435,9 @@ button.share-button:hover, button.share-button.on { filter: brightness(1.1); }
 .tab:hover .close, .tab.on .close { visibility: visible; }
 .tab .close:hover { background: var(--hair); color: var(--fg); }
 
+.trace-opacity { width: 64px; }
+.palette-bar label.dim { display: inline-flex; align-items: center; gap: 2px; white-space: nowrap; }
+
 .palette-bar {
   display: flex;
   align-items: center;
@@ -650,10 +653,10 @@ button.share-button:hover, button.share-button.on { filter: brightness(1.1); }
 .tl-twist:disabled { opacity: 0; cursor: default; }
 .tl-delete { margin-left: auto; padding: 0 4px; border: 0; background: none; color: var(--dim); }
 .tl-delete:hover { color: var(--fg); }
-.tl-solo { margin-left: auto; padding: 0 4px; border: 0; background: none; color: var(--dim); font-size: 10px; }
+.tl-solo, .tl-trace { margin-left: auto; padding: 0 4px; border: 0; background: none; color: var(--dim); font-size: 10px; }
-.tl-solo:hover { color: var(--fg); }
+.tl-solo:hover, .tl-trace:hover { color: var(--fg); }
-.tl-solo.on { color: var(--sel); font-weight: 600; }
+.tl-solo.on, .tl-trace.on { color: var(--sel); font-weight: 600; }
-.tl-solo + .tl-delete { margin-left: 0; }
+.tl-solo + .tl-delete, .tl-trace + .tl-solo { margin-left: 0; }
 
 /* The ruler and the corner above the labels stay at the top of the body while
    the rows scroll under them: the ruler is the scrubber, and scrolling down to