noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

grand unification of time

Browse source
This commit is contained in:
Your Name 2026-10-01 01:47:08 -04:00
parent a4ce750be2
commit 05878ca48b
40 changed files with 334 additions and 292 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.claude/worktrees/unified-time-selection Submodule

@ -0,0 +1 @@
Subproject commit e369ad9462ed1816a39b1a0f5c79055a2c013f4f

12
docs/architecture.md
View file

@ -52,10 +52,10 @@ overrides and kept-frame sets are all in clip-frame space, so a clip slides on
the timeline without a single stored number changing. Exposure and lead are
transforms *within* clip space:
Detection retains every source frame. A chosen picture fps samples the frozen
roto in clip time; it changes neither source-frame count nor the audio clock.
The set of source frames an artist uses as cel tracing references is another
selection, independent of the picture fps.
Detection retains every source frame. Symbols carry their native fps; project
fps selects the output grid without changing source data or audio speed.
See [Time selection](time.md) for boundary sampling and frame units.
Tracing references remain an independent selection.
```clojure
(defn pose-frame [clip cf]
@ -90,8 +90,8 @@ it a name is most of the work: `state` in `app.js` is a clip with its analysis
inlined and its palette global.
Cel keys select where drawings begin and how long they hold. The source frames
shown beneath a cel while tracing are chosen independently, and picture fps
only controls which analyzed pose the finished roto displays at a given time.
shown beneath a cel while tracing are chosen independently. Project fps controls
which native frames can appear on the output grid.
### Two things called "track"

36
docs/time.md Normal file
View file

@ -0,0 +1,36 @@
# Time selection
Project `:fps` is the playback and export grid. Each symbol has its own native
`:fps` and `:frames`; keys, spans, trace choices and corrections stay in that
native space. A symbol without an explicit rate inherits the document rate;
changing project fps first records that rate so its existing timing stays put.
An output frame selects the latest native frame at or before its time:
`floor(output-frame * native-fps / output-fps)`. Thus 30fps content in a 12fps
project reads source frames 0, 2, 5, 7, 10… and retains its duration. A partial
last output frame is included. Changing back to 30 restores the original grid.
Nothing rewrites or discards the dense measurements.
The same boundary selection runs when entering a placed symbol. Placement and
artistic speed are applied before selection; the stored `:time :rate` and
`:playback :speed` never contain a frame-rate conversion. The derived maps used
by timeline rows, picking and editing account for the units of each symbol.
`clip/frames` is a native length; `clip/output-frames` is a transport/export
length. Resolver frame queries return native frames for edits.
There is one fps control. The old transient picture-fps control and node
sample-fps fields are gone. Existing exposure, trace choices and per-instance
pose tracks remain available: a pose track can hold a chosen closed-mouth frame
without deleting its neighboring measurements. Those choices stay in native
frames when output fps changes. Automatic content-aware frame selection is not
implemented. An event between output frames appears on the next output frame;
it cannot create an extra frame in a 12fps output.
Audio uses continuous time through the same derived placement maps, without
picture floors or holds. Frame-rate units cancel before Web Audio playbackRate
is set, so only deliberate speed changes affect pitch and duration. Export and
playback use the same output count and resolver.
Earlier imports with frame-rate conversion baked into stored retimes must be
re-imported. There is no second reader for that representation. Source video
presentation timestamps are still future work; this model assumes constant fps.

2
docs/timing-model.md
View file

@ -1,5 +1,7 @@
# Timing model
[Time selection](time.md) defines the current frame-rate representation.
[The Lane Model](lane-model.md) defines the revised target for occurrence timing,
source playback, sampling scope, and inverse editing. It supersedes conflicting
proposals here; the sections below describe earlier implementation decisions.

8
frontend/src/arthur/audio/mix.cljs
View file

@ -113,7 +113,7 @@
(defn- render! [document sid sources store]
(let [fps (:fps document)
frames (:frames (clip/symbol document sid))
frames (clip/output-frames document sid)
tracks (tracks-of document sid)
output (js/OfflineAudioContext.
2 (js/Math.ceil (* (/ frames fps) 44100)) 44100)]
@ -121,7 +121,7 @@
(let [[start end] (or (node/placed-span track) [0 frames])
start (max 0 start)
end (min frames end)
{:keys [buffer fps] :or {fps (:fps document)}} (get sources (:source track))
{:keys [buffer fps] :or {fps (:fps track)}} (get sources (:source track))
sound (.createBufferSource output)
gain (.createGain output)
pan (.createStereoPanner output)]
@ -130,7 +130,7 @@
(set! (.-loop sound) (boolean (get-in track [:time :loop?])))
(automate! (.-playbackRate sound)
(get-in track [:channels [:audio :rate]])
start end (:fps document) (or (get-in track [:time :rate]) 1) 1 store)
start end (:fps document) (* (or (get-in track [:time :rate]) 1) (/ (:fps document) fps)) 1 store)
(automate! (.-gain gain)
(get-in track [:channels [:audio :gain]])
start end (:fps document) 1 1 store)
@ -195,6 +195,6 @@
buffer (wav-url buffer)
(and fallback-url (= sid (clip/opens-on document))) fallback-url
:else (let [rate 44100
n (max 1 (js/Math.ceil (* rate (/ (clip/frames document sid)
n (max 1 (js/Math.ceil (* rate (/ (clip/output-frames document sid)
(:fps document)))))]
(wav-url (.createBuffer (js/OfflineAudioContext. 1 1 rate) 1 n rate))))))))

5
frontend/src/arthur/clock.cljs
View file

@ -97,8 +97,3 @@
rather than only inside the scene."
[f expose]
(node/expose f expose))
(defn picture-frame
"The source pose displayed at f after picture-rate sampling and exposure."
[f source-fps picture-fps expose]
(node/expose (node/sample-frame f source-fps picture-fps) expose))

5
frontend/src/arthur/db.cljs
View file

@ -30,8 +30,7 @@
;; serves by hash — and the synthetic take needs a sound of its own so
;; that the clock has something to run against with no footage ingested.
:audio "/static/arthur/audio.wav"
:cid (name label-key)
:display-fps (:fps clip)}
:cid (name label-key)}
(select-keys clip [:fps :width :height])))
(def clips
@ -72,7 +71,7 @@
:clip (let [c (domain-clip/blank)]
{:fps (:fps c)
:width (:width c) :height (:height c)
:audio nil :display-fps (:fps c)})
:audio nil})
;; Which ingested footage to detect, and what the last load said. The list
;; comes from the server — tier 3 is the backend's since step 9 — so there is

24
frontend/src/arthur/domain/bring.cljs
View file

@ -44,7 +44,7 @@
n))
copy (fn [sid]
(-> (clip/symbol other sid)
(assoc :id (ids sid))
(assoc :id (ids sid) :fps (or (:fps (clip/symbol other sid)) (:fps other)))
(update :nodes #(into {} (map (fn [[id n]] [id (repoint n ids)])) %))))]
{:clip (reduce (fn [c sid] (assoc-in c [:symbols (ids sid)] (copy sid))) clip reach)
:ids ids}))
@ -76,21 +76,6 @@
[clip frozen label footage-id range]
(let [{c :clip ids :ids} (symbols clip frozen [:main] {:main (symbol-id label)})
sid (ids :main)
source-fps (:fps frozen)
project-fps (:fps clip)
;; The new symbol is played by the receiving project's clock. Keep its
;; wall-clock duration by giving it project-rate frames, while its root
;; maps those frames back onto the source timeline. A 30fps source in a
;; 12fps project therefore has 2.5 source frames per project frame.
source-rate (if (and (number? source-fps) (pos? source-fps)
(number? project-fps) (pos? project-fps))
(/ source-fps project-fps)
1)
output-frames (max 1 (js/Math.ceil (/ (clip/frames c sid) source-rate)))
c (-> c
(assoc-in [:symbols sid :frames] output-frames)
(update-in [:symbols sid :nodes :root :time]
#(assoc (or % {}) :mode :map :at 0 :rate source-rate)))
tracked? (and (nil? (:analysis clip))
(every? #(= % (ids %)) (keys (:subjects frozen))))]
{:sid sid
@ -103,8 +88,8 @@
;; Source frame `start` plays on the symbol's 0.
:span range
:time {:mode :map
:at (/ (- (first range)) source-rate)
:rate source-rate}}))
:at (- (first range))
:rate 1}}))
tracked? (-> (assoc :analysis (:analysis frozen))
(update :subjects merge (:subjects frozen))
(update :features merge (:features frozen))
@ -118,4 +103,5 @@
when there is none. See `clip/place-symbol`."
[entry brought store sid host frame uuid point]
(let [st (merge (:store entry) store)]
(assoc entry :store st :clip (clip/place-symbol brought st host sid frame uuid point))))
(assoc entry :store st :clip (clip/place-symbol brought st host sid
(* frame (:rate (clip/grid-time brought host))) uuid point))))

15
frontend/src/arthur/domain/cadence.cljs Normal file
View file

@ -0,0 +1,15 @@
(ns arthur.domain.cadence
"Select integer content frames across frame grids. Stored frames never change.")
(defn ratio [grid native]
(if (and grid native (pos? grid) (pos? native)) (/ native grid) 1))
(defn frame
"Latest native frame at or before reader frame f. Never sample the future."
[f grid native]
(js/Math.floor (if (and grid native) (/ (* f native) grid) f)))
(defn frames
"Reader frames covering a native length, including a partial final frame."
[n grid native]
(when n (max 1 (js/Math.ceil (/ n (ratio grid native))))))

90
frontend/src/arthur/domain/clip.cljs
View file

@ -33,12 +33,11 @@
them. `unplaced` answers the question a reserved root used to: which symbols
nothing else places, and so which ones a person opening the document wants.
WHY :fps IS HERE AND :frames IS NOT. A rate is how fast the whole clip plays
against its audio, and a nested symbol cannot have one of its own — retiming an
instance is `:rate` on its `:time` map, which is a factor and not a rate. A
frame COUNT is a property of a frame space, so every symbol has its own."
:fps is the output grid. A symbol's optional :fps names the native grid its
frames were authored or measured on; absent means the document's grid."
(:refer-clojure :exclude [symbol])
(:require [arthur.domain.channel :as ch]
(:require [arthur.domain.cadence :as cadence]
[arthur.domain.channel :as ch]
[arthur.domain.feature :as feature]
[arthur.domain.node :as node]
[arthur.domain.palette :as pal]
@ -69,6 +68,36 @@
[clip sid]
(:frames (symbol clip sid)))
(defn fps [clip sid] (or (:fps (symbol clip sid)) (:fps clip)))
(defn set-fps
"Change the output grid without rewriting any content's frames."
[clip rate]
(-> clip
(update :symbols #(into {} (map (fn [[sid sym]]
[sid (assoc sym :fps (fps clip sid))])) %))
(assoc :fps rate)))
(defn output-frames [clip sid]
(cadence/frames (frames clip sid) (:fps clip) (fps clip sid)))
(defn grid-time [clip sid]
{:at 0 :rate (cadence/ratio (:fps clip) (fps clip sid))})
(defn source-time
"Derived cel-to-content map. Frame-rate units never enter stored retimes."
[clip host n]
(when-let [t (node/source-time n)]
(let [r (cadence/ratio (fps clip host) (fps clip (node/source n)))]
(-> t (update :rate * r) (update :at / r)))))
(defn placed-frame [clip host n f]
(let [child (node/source n)
r (cadence/ratio (fps clip host) (fps clip child))]
(some-> (node/placed-frame (update-in n [:playback :speed] #(* (or % 1) r))
f (frames clip child))
(update :frame js/Math.floor))))
(defn stage
"A symbol's stage as `[width height]`: its own, or the clip's where it has none.
Absent rather than copied in at creation, so a symbol nobody has sized follows
@ -136,7 +165,7 @@
{:name "untitled"
:fps 30
:width 320 :height 200
:symbols {:main {:id :main :frames blank-frames :nodes {}}}})
:symbols {:main {:id :main :fps 30 :frames blank-frames :nodes {}}}})
(defn- transform-op
"Put a symbol's already resolved mark into its instance's parent space. Its
@ -162,23 +191,10 @@
op)))
(defn resolver
"Resolve symbol `sid` of a clip, including every symbol its instances place.
Each instance owns its own symbol resolver, so two offsets never share a
channel cursor or point buffer. The returned ops must be drawn before the next
frame, as with symbol/resolver.
Any symbol can be resolved and none is the default: the frame space is the
resolved symbol's own `:frames`, and nested instances inside it still resolve,
because this is the function that knows how to do that.
It answers `symbol/world-of` and `symbol/frame-of` for a ROW PATH — the ids
from `sid` down through instances, as a timeline row names a node — as of the
frame it last resolved: the matrix into `sid`'s coordinates, and the node's
own frame. Nil for a node that was not on that frame. It is how something
drawn beside the picture, like a tracing photo, rides a node inside it without
resolving anything a second time."
[clip sid store palette {:keys [picture-fps] :as opts}]
"Resolve an output frame, selecting native content at each symbol boundary.
Every instance owns its cursors and buffers. The IResolver queries return
native node frames and world matrices for the last rendered output frame."
[clip sid store palette opts]
(letfn [(build [sid chain pose-tracks]
(when (some #{sid} chain)
(throw (ex-info "symbol cycle" {:chain (conj chain sid)})))
@ -188,8 +204,7 @@
rank (symbol/draw-rank nodes (symbol/order nodes))
ids (sort-by rank (keys nodes))
own (symbol/resolver sym store palette
(assoc opts :pose-tracks pose-tracks
:source-fps (:fps clip)))
(assoc opts :pose-tracks pose-tracks))
;; Each cel owns its source resolver and mutable buffers.
children (into {}
(for [[id n] nodes
@ -205,7 +220,7 @@
entered (volatile! {})
step (fn [f]
(vreset! entered {})
(let [by-id (into {} (map (juxt :node identity)) (own f))]
(let [by-id (into {} (map (juxt :node identity)) (own (js/Math.floor f)))]
(into []
(mapcat
(fn [id]
@ -215,7 +230,7 @@
local (symbol/frame-of own id)
length (frames clip (node/source n))
shown (when (and m (number? local))
(node/placed-frame n local length))
(placed-frame clip sid n local))
frame (:frame shown)]
(if (and frame (<= 0 frame) (< frame length))
(do (vswap! entered assoc id (:symbol shown))
@ -240,7 +255,13 @@
(when (contains? @entered id)
(symbol/frame-of (get children [id (get @entered id)]) (vec more)))
(symbol/frame-of own id))))))]
(build sid [] nil)))
(let [r (build sid [] nil)]
(reify
IFn
(-invoke [_ f] (r (cadence/frame f (or (:grid-fps opts) (:fps clip)) (fps clip sid))))
symbol/IResolver
(world-of [_ path] (symbol/world-of r path))
(frame-of [_ path] (symbol/frame-of r path))))))
(defn center
"The middle of everything symbol `sid` draws, over all its frames, in its own
@ -254,7 +275,7 @@
Effects' anchor point are set once and left. A symbol that grows later keeps
its instances' pivots where they were, so nothing on screen moves."
[clip store sid]
(let [resolve (resolver clip sid store pal/index-of nil)
(let [resolve (resolver clip sid store pal/index-of {:grid-fps (fps clip sid)})
bounds (fn [[x0 y0 x1 y1 :as b] x y]
(if b [(min x0 x) (min y0 y) (max x1 x) (max y1 y)] [x y x y]))
[x0 y0 x1 y1]
@ -310,7 +331,7 @@
;; Lexicographic draw order, as `domain/paint` does it: an instance made
;; later sits above one made earlier, and neither has to renumber.
:z (str "z" (js/Date.now) "-" (name sid))
:span [0 (:frames target)]
:span [0 (cadence/frames (:frames target) (fps clip host) (fps clip sid))]
:time {:mode :map :at frame :rate 1}
:source {:symbol sid}
:playback {:in 0 :speed 1 :end :stop}
@ -319,11 +340,8 @@
[:xform :anchor] {:animated? false :value middle}}})))))
(defn place-sound
"An audio node playing `source` — `{:sound id}`, an uploaded file, or
`{:footage id}`, a video's own sound — inside symbol `host` from `frame` of it.
Timed as an instance is: `:at frame`, its span `length` of its OWN frames, and
`rate` of those to one of `host`'s, which is what keeps a 30fps video's sound
its own length in a 12fps project. Nothing else: a sound is not in the picture."
"Place a sound at host frame `frame`. Its span uses `source`'s fps when
supplied, otherwise the host's. `rate` is a deliberate playback speed."
[clip host source label length rate frame uuid]
(let [end (frames clip host)]
(if (or (nil? end) (nil? frame) (neg? frame) (>= frame end))
@ -354,7 +372,7 @@
(if (or (nil? end) (symbol clip sid) (nil? frame) (neg? frame) (>= frame end))
clip
(-> clip
(assoc-in [:symbols sid] {:id sid :name (name sid) :frames (- end frame) :nodes {}})
(assoc-in [:symbols sid] {:id sid :name (name sid) :fps (fps clip host) :frames (- end frame) :nodes {}})
(place-symbol nil host sid frame uuid nil)))))
(defn free-id

6
frontend/src/arthur/domain/lane.cljs
View file

@ -31,7 +31,7 @@
(reduce node/then-time {:at 0 :rate 1} (map node/time-of (reverse chain)))
(let [n (get nodes id) t (:time n)]
(when (and n (not (contains? seen id))
(not (:loop? t)) (not (:sample-fps t))
(not (:loop? t))
(<= (or (:expose t) 1) 1))
(recur (:parent n) (conj seen id) (conj chain n)))))))
@ -396,7 +396,7 @@
(when (clip/symbol clip drawing-id) "the new drawing ID is already used"))]
{:refused why}
(place (assoc-in clip [:symbols drawing-id]
{:id drawing-id :name (name drawing-id) :frames 1 :nodes {}})
{:id drawing-id :name (name drawing-id) :fps (clip/fps clip sid) :frames 1 :nodes {}})
sid lane-id id drawing-id at extent true)))
(defn reuse-drawing
@ -475,7 +475,7 @@
(if (:refused cleared)
cleared
(place (assoc-in (:clip cleared) [:symbols drawing-id]
{:id drawing-id :name (name drawing-id) :frames 1 :nodes {}})
{:id drawing-id :name (name drawing-id) :fps (clip/fps clip sid) :frames 1 :nodes {}})
sid lane-id id drawing-id at extent false))))))
(defn make-unique

9
frontend/src/arthur/domain/leaf.cljs
View file

@ -14,7 +14,7 @@
clip/<cid>/subject/<subj> a tracked subject and its params
clip/<cid>/feature/<fid> one feature: area, nodes, params
clip/<cid>/group/<gid> an eye pair and its shared params
clip/<cid>/symbol/<sid> frames, and a palette one day
clip/<cid>/symbol/<sid> native frames and fps, optional palette
clip/<cid>/symbol/<sid>/node/<nid> kind, parent, stencil, z, time
clip/<cid>/symbol/<sid>/channel/<nid>/<prop>
clip/<cid>/symbol/<sid>/measured/<nid> the channels a re-freeze owns
@ -23,9 +23,8 @@
its own nodes, so the symbol id is a path segment. No symbol has a reserved
segment: `main` in a path is an id like any other.
`:frames` MOVED OFF `timing` onto the symbol. A symbol is a frame space and a
clip is a rate, so `timing` holds `:fps` alone. Both used to be in one leaf, which
is how a nested symbol's length would have had nowhere to go.
The timing leaf holds output fps. Each symbol leaf holds its native fps
and frame count, so changing the output grid leaves content untouched.
WHY THESE BOUNDARIES. Last-writer-wins only clobbers when its unit is too big,
so the cut is chosen so that the things people do simultaneously land on
@ -145,7 +144,7 @@
;; disagree with itself about; `clip` puts it back.
(for [[sid sym] (:symbols clip)]
{(at "symbol" (segment sid))
(select-keys sym [:name :frames :width :height :palette])})
(select-keys sym [:name :frames :fps :width :height :palette])})
(for [[sid sym] (:symbols clip)
[id n] (:nodes sym)]
{(at "symbol" (segment sid) "node" (segment id))

50
frontend/src/arthur/domain/nest.cljs
View file

@ -33,7 +33,7 @@
[clip store sid frame id]
(let [sym (clip/symbol clip sid)
sym (update sym :nodes select-keys (symbol/lineage (:nodes sym) id))
r (symbol/resolver sym store pal/index-of {:source-fps (:fps clip)})]
r (symbol/resolver sym store pal/index-of nil)]
(r frame)
r))
@ -67,8 +67,7 @@
;; the hold pins, not on `local`, and inside a playing insert
;; is its animation at its own in-point and speed.
shown (when (number? local)
(node/placed-frame (get nodes id) local
(clip/frames clip (node/source (get nodes id)))))
(clip/placed-frame clip sid (get nodes id) local))
inner (:symbol shown)
lf (if shown (:frame shown) local)]
(if (and m (number? local)
@ -80,11 +79,12 @@
;; Forward sampling above works for holds too. :time is the
;; invertible edit map; source in-points and speeds belong in it.
:time (when (and time (not-any? #(get-in % [:time :loop?]) chain)
(or (not inst?) (node/source-time (get nodes id))))
(or (not inst?) (clip/source-time clip sid (get nodes id))))
(cond-> (reduce node/then-time time (map node/time-of chain))
inst? (node/then-time (node/source-time (get nodes id)))))}
inst? (node/then-time (clip/source-time clip sid (get nodes id)))))}
(reduced nil))))
{:sid sid :frame f :matrix (node/mat) :time {:at 0 :rate 1}}
{:sid sid :frame (js/Math.floor (* f (:rate (clip/grid-time clip sid))))
:matrix (node/mat) :time (clip/grid-time clip sid)}
path))
(defn placement
@ -154,7 +154,10 @@
(let [n (get nodes id)
parent (if-let [pid (:parent n)] (get acc pid)
{:time outer :bounds bounds})
m (node/then-time (:time parent) (node/time-of n))]
m (node/then-time (:time parent) (node/time-of n))
m (if (= :audio (:kind n))
(update m :rate * (/ (or (get-in n [:source :fps]) (clip/fps clip sid))
(clip/fps clip sid))) m)]
(assoc acc id {:time m :bounds (window m (:span n) (:bounds parent))})))
{} (symbol/order nodes))]
(mapcat
@ -164,6 +167,7 @@
(case (:kind n)
:audio [(-> n
(assoc :parent nil :path (conj path id) :owner sid
:fps (or (get-in n [:source :fps]) (clip/fps clip sid))
:span [(to-local m lo) (to-local m hi)]
:time (merge (:time n) {:mode :map :at (:at m) :rate (:rate m) :offset 0})
:channels (channels (:channels n) m)))]
@ -173,7 +177,10 @@
length (clip/frames clip child)]
;; A visual freeze does not emit a sustained audio sample.
(when (and child length (pos? speed))
(let [source (node/then-time m {:at (- (/ in speed)) :rate speed})
(let [source (node/then-time m (clip/source-time clip sid
(-> n
(assoc-in [:playback :end] :stop)
(update :time dissoc :loop?))))
loop? (or (= end :loop) (get-in n [:time :loop?]))
periods (if loop?
(range (js/Math.floor (/ (to-local source lo) length))
@ -185,7 +192,7 @@
periods))))
nil))))
(sort-by (comp str key) nodes))))]
(vec (walk sid {:at 0 :rate 1} [0 (clip/frames clip sid)] [] #{}))))
(vec (walk sid (clip/grid-time clip sid) [0 (clip/output-frames clip sid)] [] #{}))))
(defn- retime
"Node `n` with its own time map replaced by `m`, and nothing else touched: its
@ -306,17 +313,18 @@
sids (reductions only sid path)
;; Every node on the way, outermost first: each instance, after its
;; parents in the symbol it is in.
chain (mapcat (fn [sid id]
(let [nodes (:nodes (clip/symbol clip sid))]
(map #(get nodes %) (rseq (symbol/lineage nodes id)))))
sids path)]
{:sid (last sids)
:time (when (not-any? #(or (get-in % [:time :loop?])
(and (= :instance (:kind %)) (nil? (node/source-time %)))) chain)
(reduce node/then-time {:at 0 :rate 1}
(mapcat (fn [n] (cond-> [(node/time-of n)]
(= :instance (:kind n)) (conj (node/source-time n))))
chain)))}))
maps (mapcat (fn [sid id]
(let [nodes (:nodes (clip/symbol clip sid))
chain (map #(get nodes %) (rseq (symbol/lineage nodes id)))]
(mapcat (fn [n]
(if (get-in n [:time :loop?]) [nil]
(cond-> [(node/time-of n)]
(= :instance (:kind n)) (conj (clip/source-time clip sid n)))))
chain)))
sids path)]
{:sid (last sids)
:time (when (every? some? maps)
(reduce node/then-time (clip/grid-time clip sid) maps))}))
(defn slide
"Move the node at row path `path` along its symbol's time by `df` frames of
@ -402,7 +410,7 @@
start (js/Math.floor (max 0 (apply min (map first spans))))
end (min (second whole) (apply max (map second spans)))
made (-> clip
(assoc-in [:symbols sid] {:id sid :name (name sid)
(assoc-in [:symbols sid] {:id sid :name (name sid) :fps (clip/fps clip host)
:frames (max 1 (js/Math.ceil (- end start)))
:nodes {}})
(clip/place-symbol store host sid start uuid nil))

56
frontend/src/arthur/domain/node.cljs
View file

@ -150,30 +150,18 @@
[f n]
(if (and n (> n 1)) (* (js/Math.floor (/ f n)) n) f))
(defn sample-frame
"Pick a source frame for a lower picture rate without changing clip time.
The input is already an integer source frame from the audio clock. Its time is
f/source-fps. Quantise that time to the picture grid, then read the latest
source frame at or before it. The result is always an integer and never from
the future, including when the rates do not divide (30 source → 24 picture)."
[f source-fps picture-fps]
(if (and source-fps picture-fps
(pos? source-fps) (pos? picture-fps)
(< picture-fps source-fps))
(min f (js/Math.floor
(* (js/Math.floor (/ (* f picture-fps) source-fps))
(/ source-fps picture-fps))))
f))
(defn time-of
"A node's own time as the affine map it is: `{:at a :rate r}`, meaning a frame
`p` of its parent is frame `r·(p − a)` of its own. THE SAME FOR EVERY NODE. A
node with no time map is `{:at 0 :rate 1}`, reading its parent's frames as its
own; a mouth lead's `:offset` is folded into `:at`. Cel and picture
sampling are floors, not part of the map, and are left out: this is the map a
move preserves and a timeline row draws with, and `local-frame` is what reads
a frame, floors and the lead in their load-bearing order."
own; a mouth lead's `:offset` is folded into `:at`. Exposure is a floor, not
part of the map, and is left out: this is the map a move preserves and a
timeline row draws with, and `local-frame` is what reads a frame, the floor and
the lead in their load-bearing order.
`:rate` is a RETIME SOMEBODY CHOSE and nothing else — half speed on an insert.
Reconciling two frame rates is not a retime and does not belong here: it is a
selection, and it lives in `domain/cadence`."
[n]
(let [{:keys [mode at rate offset] :or {mode :map at 0 rate 1 offset 0}} (:time n)]
(if (= mode :map)
@ -258,31 +246,15 @@
(defn finite-number? [v] (and (number? v) (js/Number.isFinite v)))
(defn local-frame
"Apply a node's time map to the frame it was handed by its parent.
ORDER IS LOAD-BEARING: expose first, then offset. Flooring onto a grid and
shifting against the clock do not commute — shift first and the floor discards
it on most frames, so the lead slider reads as doing nothing at cels above
1, which is indistinguishable from the slider being unwired.
Composed along the parent chain, outermost first, by symbol/eval-frame. Two
rules fall out and they are different rules: cel INHERITS STRICTLY,
because a head cutting on odd frames against a mouth cutting on even ones reads
as two performances; offset is PER-NODE by design, because mouth lead applies
to performance nodes and not to the plate, which is the entire point of it."
"Apply an artistic time map within one frame space: expose, then offset.
Frame-rate selection happens at symbol boundaries in domain/clip."
[n f]
(let [{:keys [mode source-fps sample-fps] ex :expose :or {mode :map}} (:time n)]
(let [{:keys [mode at rate offset] ex :expose :or {mode :map at 0 rate 1}} (:time n)]
(if (= mode :inherit)
f
(do
(when (and sample-fps (not (and source-fps (pos? source-fps))))
(throw (ex-info "picture sampling needs a positive source fps"
{:node (:id n) :time (:time n)})))
(let [{:keys [at rate offset] :or {at 0 rate 1}} (:time n)]
(cond-> (* rate (- f at))
sample-fps (sample-frame source-fps sample-fps)
ex (expose ex)
offset (+ offset)))))))
(cond-> (* rate (- f at))
ex (expose ex)
offset (+ offset)))))
;; ---------------------------------------------------------------------------
;; the transform

10
frontend/src/arthur/domain/pick.cljs
View file

@ -92,7 +92,7 @@
symbol to BUILD and a lookup to RUN. Asking frame by frame through a fresh one
is a resolver per frame, which is what made `pivot` want a separate path for
instances rather than the one walk it is."
[document store n]
[document store sid n]
(let [grow (fn [[x0 y0 x1 y1 :as b] x y]
(if b [(min x0 x) (min y0 y) (max x1 x) (max y1 y)] [x y x y]))
at (fn [f p] (ch/value-at (get (node/channels n) p) f store))]
@ -105,10 +105,10 @@
(let [loop? (get-in n [:time :loop?])
resolvers (into {} (map (fn [child]
[child (clip/resolver document child store
pal/index-of nil)]))
pal/index-of {:grid-fps (clip/fps document child)})]))
(node/sources n))]
(fn [f0]
(let [shown (node/placed-frame n f0 (clip/frames document (node/source n)))
(let [shown (clip/placed-frame document sid n f0)
frames (when shown (clip/frames document (:symbol shown)))
f (when (number? frames)
(if loop? (mod (:frame shown) frames) (:frame shown)))
@ -151,7 +151,7 @@
ALL its frames, not the first, for the reason `clip/center` says: a mouth that
opens and travels still has its middle where the mouth is."
[document store n fs]
(let [bounds (bounds-of document store n)]
[document store sid n fs]
(let [bounds (bounds-of document store sid n)]
(when-let [[x0 y0 x1 y1] (reduce #(union %1 (bounds %2)) nil fs)]
[(/ (+ x0 x1) 2) (/ (+ y0 y1) 2)])))

19
frontend/src/arthur/domain/symbol.cljs
View file

@ -399,7 +399,7 @@
(defn- base-channel-frame
"A trace selects the measured frames its node reads; marked channels read
instance pose choices."
[choices traces nodes source-fps picture-fps id c lf]
[choices traces nodes id c lf]
(cond
(contains? traces id)
(trace/held-frame (get traces id) lf)
@ -410,9 +410,9 @@
[:node id]
(or (:pose-group (get nodes id)) id))
lf
(node/sample-frame lf source-fps picture-fps))
(js/Math.floor lf))
:else lf))
:else (js/Math.floor lf)))
(defn- prepared-traces [nodes]
(into {}
@ -465,14 +465,14 @@
the only place the space changes.
This is the definition of what a frame means. `resolver` is what plays it."
[sym f store palette {:keys [pose-tracks source-fps picture-fps]}]
[sym f store palette {:keys [pose-tracks]}]
(let [nodes (nodes-of sym)
choices (pose/prepare pose-tracks)
traces (prepared-traces nodes)
ord (order nodes)]
(eval-into {:read (fn [id path c lf]
(ch/value-at c (base-channel-frame choices traces nodes
source-fps picture-fps id c lf)
id c lf)
lf store))
:palette palette
:mat-for (fn [_id] (node/mat))
@ -531,7 +531,7 @@
because a fifth of them later is then a key rather than a nil at every one of
these call sites — which is what the arity ladder that used to be here was
standing in for."
[sym store palette {:keys [pose-tracks source-fps picture-fps]}]
[sym store palette {:keys [pose-tracks]}]
(let [nodes (nodes-of sym)
choices (pose/prepare pose-tracks)
traces (prepared-traces nodes)
@ -560,7 +560,7 @@
(when-let [cursor (get-in cursors [id path])]
(ch/sample! cursor
(base-channel-frame choices traces nodes
source-fps picture-fps id c lf)
id c lf)
lf)))
:palette palette
:mat-for (fn [id] (get mats id))
@ -596,7 +596,7 @@
`:width` and `:height` are the symbol's own stage, and are absent until someone
sets them: a symbol without them uses the clip's — see `clip/stage`."
#{:id :name :frames :width :height :nodes :palette})
#{:id :name :frames :fps :width :height :nodes :palette})
(defn problems
"Human-readable reasons this symbol will not evaluate. Empty means it will.
@ -642,6 +642,9 @@
(into (when-not (or (nil? (:frames sym)) (and (integer? (:frames sym)) (pos? (:frames sym))))
[(str ":frames is " (pr-str (:frames sym))
" — a symbol is a frame SPACE, so its length is a positive integer")]))
(into (when (and (some? (:fps sym))
(not (and (node/finite-number? (:fps sym)) (pos? (:fps sym)))))
[":fps must be a positive finite native rate"]))
(into (for [k [:width :height]
:let [v (get sym k)]
:when (and (some? v) (not (and (integer? v) (pos? v))))]

2
frontend/src/arthur/events/edit.cljs
View file

@ -59,7 +59,7 @@
[db clip]
(cond-> (update db :clip merge (select-keys clip [:width :height]))
(not= (:fps clip) (get-in db [:clip :fps]))
(update :clip merge {:fps (:fps clip) :display-fps (:fps clip)})))
(update :clip merge {:fps (:fps clip)})))
(defn history
"Apply `f` to the loaded entry's undo history, which is not an edit: nothing

4
frontend/src/arthur/events/export.cljs
View file

@ -117,8 +117,7 @@
(let [{:keys [clip]} (store/entry (:clip/current db))
{sid :symbol isolate :isolate} (target db)]
(export/plan {:clip clip :symbol sid :zoom (get-in db [:export :zoom])
:isolate isolate
:picture-fps (get-in db [:clip :display-fps])}))))
:isolate isolate}))))
(rf/reg-event-db
::set-target
@ -158,7 +157,6 @@
(:palette db) pal/index-of)
:ramp (get {:arthur/default pal/rgb} (:palette db) pal/rgb)
:zoom zoom
:picture-fps (get-in db [:clip :display-fps])
:audio-url (:audio entry)
:name (stem (:label entry)
(label-of (:clip entry) (get-in db [:ui :open])

4
frontend/src/arthur/events/footage.cljs
View file

@ -152,7 +152,7 @@
source-blocks (source/pack-subjects (:id (:analysis built)) subjects)
_ (mark! "build-clip: pack source blocks")]
(assoc (select-keys built [:fps :width :height])
:display-fps (:fps built)
:clip built :store (:store frozen)
:source-blocks source-blocks
:source-inputs (assoc source-inputs :subjects with-presence)
@ -501,7 +501,7 @@
{:keys [clip sid tracked?]}
(bring/take (:clip (store/entry (:clip/current db))) (:clip built)
name footage-id range)
imported-frames (clip/frames clip sid)
imported-frames (clip/output-frames clip sid)
source-fps (get-in built [:clip :fps])
db (edit/edit-entry
db

11
frontend/src/arthur/events/playback.cljs
View file

@ -24,7 +24,7 @@
the db beside it, because an edit can change it."
[db]
(or (some-> (footage/entry (:clip/current db)) :clip
(clip/frames (get-in db [:ui :open])))
(clip/output-frames (get-in db [:ui :open])))
1))
(defn show
@ -36,7 +36,7 @@
sid (clip/opens-on (:clip entry))]
(-> db
(assoc :clip/current id
:clip (select-keys entry [:fps :width :height :audio :display-fps]))
:clip (select-keys entry [:fps :width :height :audio]))
(update :ui merge
{:open sid :tabs (if sid [sid] [])
;; From scratch, not merged: the faces switched on were another
@ -95,13 +95,6 @@
{:db (assoc-in db [:playback :rate] r)
::rate! r}))
(rf/reg-event-db
::set-picture-fps
(fn [db [_ target]]
(if (and (number? target) (pos? target) (<= target (fps db)))
(assoc-in db [:clip :display-fps] target)
db)))
;; --- effects: every DOM touch on the audio element is one of these ---
(rf/reg-fx ::play! (fn [_] (clock/play!)))

19
frontend/src/arthur/events/project.cljs
View file

@ -120,7 +120,6 @@
;; differs from it, and a collaborator's
;; write lands on what does not.
:synced (project/tier1 (.-leaves clip-json))
:display-fps (:fps built)
:clip built :store (:store loaded)
:footage-id footage-id
:audio (if footage (.-audio footage)
@ -512,7 +511,6 @@
;; A content id of its own from the start: `::save!` addresses the clip by
;; it, and two untitled documents saved from two tabs are two documents.
:cid (str (random-uuid))
:display-fps (:fps c)
:fps (:fps c) :width (:width c) :height (:height c)}))
(rf/reg-event-fx
@ -529,7 +527,7 @@
:busy? false :status "new document"}))
;; The readout goes home with the document; so must the clock, or play
;; picks up wherever the last document's audio had got to.
::pb/seek! [(:fps entry) (clip/frames (:clip entry) (clip/opens-on (:clip entry))) 0]
::pb/seek! [(:fps entry) (clip/output-frames (:clip entry) (clip/opens-on (:clip entry))) 0]
::pb/pause! nil})))
(rf/reg-event-fx
@ -594,12 +592,15 @@
(if (or (not (#{:fps :width :height} key))
(not (and (integer? value) (pos? value))))
{}
(let [db' (-> (edit/edit db #(assoc % key value))
(assoc-in [:clip key] value)
(cond-> (= key :fps) (assoc-in [:clip :display-fps] value)))
frame (get-in db [:playback :frame])]
(let [db' (edit/edit db #(if (= key :fps) (clip/set-fps % value) (assoc % key value)))
db' (assoc-in db' [:clip key] value)
frame (min (dec (pb/frames db'))
(js/Math.floor (* (get-in db [:playback :frame])
(/ value (get-in db [:clip :fps])))))]
(cond-> {:db db'}
(= key :fps) (assoc ::pb/seek! [value (pb/frames db') frame]))))))
(= key :fps) (assoc :db (assoc-in db' [:playback :frame] frame)
::pb/seek! [value (pb/frames db') frame]
:dispatch [::pb/refresh-clock]))))))
(rf/reg-event-db
::rename-symbol
@ -742,7 +743,7 @@
:status (str "opened " name " r" seq)}))
::pb/pause! nil
::pb/seek! (let [{c :clip fps :fps} (store/entry clip-id)]
[fps (clip/frames c (clip/opens-on c)) 0])}))
[fps (clip/output-frames c (clip/opens-on c)) 0])}))
(rf/reg-event-fx
::failed

6
frontend/src/arthur/events/ui.cljs
View file

@ -130,7 +130,7 @@
[clip st open selection frame]
(let [[_ sid _ path] selection]
(if (or (= sid open) (not (seq path)))
frame
(:frame (nest/inside clip st open [] frame))
(:frame (nest/inside clip st open (pop path) frame)))))
(rf/reg-event-fx
@ -472,7 +472,7 @@
(-> db
(update :ui dissoc :drop)
(edit/edit-entry #(update % :clip clip/place-symbol (:store %)
host sid frame uuid point))
host sid (* frame (:rate (clip/grid-time (:clip %) host))) uuid point))
(assoc-in [:ui :selection] [:node host uuid [uuid]])))))
(rf/reg-event-db
@ -482,7 +482,7 @@
host (get-in db [:ui :open])]
(-> db
(update :ui dissoc :drop)
(edit/edit #(clip/place-sound % host source label length rate frame uuid))
(edit/edit #(clip/place-sound % host source label length rate (* frame (:rate (clip/grid-time % host))) uuid))
(assoc-in [:ui :selection] [:node host uuid [uuid]])))))
;; ---------------------------------------------------------------------------

15
frontend/src/arthur/export.cljs
View file

@ -158,29 +158,24 @@
Separate from `run!` so the UI can show the size and length it is about to
commit to, and so the arithmetic is assertable without a sink."
[{:keys [clip zoom picture-fps] sid :symbol isolate-id :isolate}]
[{:keys [clip zoom] sid :symbol isolate-id :isolate}]
(let [sym (some-> (clip/symbol clip sid) (isolate isolate-id))
[width height] (clip/stage clip sid)
zoom (max 1 (js/Math.floor (or zoom 1)))]
(when sym
{:frames (:frames sym)
{:frames (clip/output-frames clip sid)
:fps (:fps clip)
:zoom zoom
:width (* width zoom)
:height (* height zoom)
:seconds (/ (:frames sym) (:fps clip))
;; The unedited picture-grid count. A per-instance pose track can add or
;; remove changes, so this is only the grid's nominal count.
:poses (if (and picture-fps (< picture-fps (:fps clip)))
(js/Math.ceil (* (/ (:frames sym) (:fps clip)) picture-fps))
(:frames sym))})))
:seconds (/ (clip/output-frames clip sid) (:fps clip))})))
(defn run!
"Render symbol `sid` into `exporter`. Promise of `{:filename :blob}`.
`on-progress` is called with `[done total]` as frames complete, and is where a
UI hangs its readout."
[{:keys [clip store palette ramp zoom picture-fps name audio-url]
[{:keys [clip store palette ramp zoom name audio-url]
sid :symbol isolate-id :isolate}
exporter on-progress]
(let [sym (some-> (clip/symbol clip sid) (isolate isolate-id))]
@ -195,7 +190,7 @@
;; inside it still resolve — clip/resolver is the function that knows
;; how.
doc (assoc-in clip [:symbols sid] sym)
resolve-frame (clip/resolver doc sid store palette {:picture-fps picture-fps})
resolve-frame (clip/resolver doc sid store palette nil)
ras (raster/make width height)
bg (get palette :bg 0)]
(-> (audio! doc sid store audio-url)

6
frontend/src/arthur/flow/freeze.cljs
View file

@ -728,7 +728,7 @@
at [:symbols sid :nodes id :channels [:xform :anchor]]]
(if (or (get-in c at) (node/measured? n))
c
(if-let [p (pick/pivot c store n (range (get-in c [:symbols sid :frames])))]
(if-let [p (pick/pivot c store sid n (range (get-in c [:symbols sid :frames])))]
(assoc-in c at (ch/framed p))
c))))
clip
@ -767,7 +767,7 @@
:features (merged :features) :groups (merged :groups)
:symbols
(into {:main
{:id :main :frames nf
{:id :main :fps fps :frames nf
:nodes (into {:root {:id :root :name "clip" :kind :group :z "a1"
:time {:mode :map :expose expose}}}
(map-indexed
@ -777,7 +777,7 @@
:source {:symbol id}}]))
ordered)}}
(map (fn [[id part]]
[id (place-in (:symbol part) placement)]))
[id (assoc (place-in (:symbol part) placement) :fps fps)]))
parts)}]
(doseq [[subject inputs] ordered
[id track] (:presence inputs)]

2
frontend/src/arthur/flow/regenerate.cljs
View file

@ -151,7 +151,7 @@
(throw (ex-info "regeneration needs retained source landmarks"
{:subject subject})))
base-params (merge take/knobs
{:fps (:fps changed)
{:fps (or (get-in changed [:symbols subject :fps]) (:fps changed))
:aspect (get-in changed [:analysis :aspect])
:analysis (:analysis changed)})
;; One conditioned anchor for the whole edit, and it is the SUBJECT's.

4
frontend/src/arthur/flow/take.cljs
View file

@ -119,8 +119,8 @@
(defn footage
"A real manifest and its detected landmarks through the same measurement and
freeze path as the synthetic take. The source cadence stays in :fps; picture
sampling is a root time map applied only after this artifact exists."
freeze path as the synthetic take. Symbols keep the source fps; the reader
selects native frames on the project grid."
[manifest {:keys [dimensions detector subjects]}]
(let [[w h] dimensions
params (merge knobs

1
frontend/src/arthur/subs/playback.cljs
View file

@ -12,7 +12,6 @@
(rf/reg-sub ::loop? (fn [db _] (get-in db [:playback :loop?])))
(rf/reg-sub ::muted? (fn [db _] (get-in db [:playback :muted?])))
(rf/reg-sub ::fps (fn [db _] (get-in db [:clip :fps])))
(rf/reg-sub ::display-fps (fn [db _] (get-in db [:clip :display-fps])))
;; The open symbol can have a local stage size; otherwise it follows the project.
(rf/reg-sub ::stage-size
:<- [:arthur.subs.playback/clip-id]

13
frontend/src/arthur/subs/render.cljs
View file

@ -64,11 +64,9 @@
(rf/reg-sub
::frames
:<- [::symbol]
(fn [sym _]
;; The open symbol's length, read off it rather than copied into the db, so a
;; drop that lengthens it lengthens the transport with it.
(:frames sym)))
:<- [::clip]
:<- [::open]
(fn [[document sid] _] (clip/output-frames document sid)))
(rf/reg-sub
::exposure
@ -129,10 +127,9 @@
:<- [::open]
:<- [::store]
:<- [::palette]
:<- [::playback/display-fps]
(fn [[document sid store palette picture-fps] _]
(fn [[document sid store palette] _]
(when (and document (clip/symbol document sid))
(clip/resolver document sid store palette {:picture-fps picture-fps}))))
(clip/resolver document sid store palette nil))))
(rf/reg-sub
::shown

2
frontend/src/arthur/subs/ui.cljs
View file

@ -68,7 +68,7 @@
(when n
(let [st (:store (store/entry clip-id))]
(when-let [pl (nest/placement clip st open (or path [id]) f)]
(assoc pl :node n :bounds ((pick/bounds-of clip st n) (:frame pl))))))))
(assoc pl :node n :bounds ((pick/bounds-of clip st (:sid pl) n) (:frame pl))))))))
(rf/reg-sub
::settled-clip

2
frontend/src/arthur/ui/drag.cljs
View file

@ -37,7 +37,7 @@
(let [{document :clip st :store} (store/entry clip-id)]
(reset! carrying (merge {:kind :symbol :sid sid
:label (clip/symbol-name document sid)
:frames (clip/frames document sid)
:frames (clip/output-frames document sid)
;; A symbol cannot go inside itself or inside
;; anything it places. Refused by not ACCEPTING the
;; drop, so the pointer says so while it hovers.

16
frontend/src/arthur/ui/params.cljs
View file

@ -80,7 +80,6 @@
(defn- clip-section []
(let [clip @(rf/subscribe [::render/clip])
fps @(rf/subscribe [::playback/fps])
picture @(rf/subscribe [::playback/display-fps])
open @(rf/subscribe [::render/open])
frames @(rf/subscribe [::render/frames])
project @(rf/subscribe [::playback/project])
@ -98,18 +97,7 @@
[number-field "height" (:height clip)
#(rf/dispatch [::project/project-setting :height %]) nil busy?]
[number-field "project fps" fps
#(rf/dispatch [::project/project-setting :fps %]) nil busy?]]
;; Sampling the frozen roto at a lower rate. The source track, the duration
;; and the audio clock are untouched — a drawing is HELD, the file is never
;; short — which is why this is a picture rate and not a playback rate.
[:div.row {:style {:margin-top "5px"}}
[:span.dim "picture"]
(doall
(for [r (distinct (filter #(<= % fps) [8 12 15 24 fps]))]
^{:key r}
[:button {:class (when (= r picture) "on")
:on-click #(rf/dispatch [::pb/set-picture-fps r])}
(if (= r fps) "source" (str r))]))]]))
#(rf/dispatch [::project/project-setting :fps %]) nil busy?]]]))
;; ---------------------------------------------------------------------------
;; a node
@ -456,7 +444,7 @@
[section "symbol"
[facts
"id" (str sid)
"length" (str (:frames sym) " frames")
"length" (str (:frames sym) " frames at " (clip-domain/fps clip sid) " fps")
"stage" (let [[w h] (clip-domain/stage clip sid)] (str w "×" h))
"nodes" (str (count (:nodes sym)))]
[:div.inspector-form

4
frontend/src/arthur/ui/pool.cljs
View file

@ -294,8 +294,8 @@
[{:keys [id label duration footage? fps]} project-fps rename]
(let [frames (js/Math.ceil (* duration project-fps))
[source length rate] (if footage?
[{:footage id} (js/Math.round (* duration fps)) (/ fps project-fps)]
[{:sound id} frames 1])]
[{:footage id :fps fps} (js/Math.round (* duration fps)) 1]
[{:sound id :fps project-fps} frames 1])]
^{:key (str (when footage? "f") id)}
[row (merge {:label label
:sub (str (.toFixed duration 1) "s")

13
frontend/src/arthur/ui/timeline.cljs
View file

@ -22,6 +22,7 @@
happen and not where they happen again."
(:require [clojure.string :as str]
[arthur.domain.node :as node]
[arthur.domain.clip :as clip]
[arthur.domain.nest :as nest]
[arthur.domain.lane :as lane]
[arthur.domain.symbol :as symbol]
@ -65,8 +66,10 @@
(defn- source-frames
"How long the thing this node places is, for a placement carrying no span of
its own. 0 for a node that places nothing, or a reference with nothing there."
[clip n]
(or (get-in clip [:symbols (node/source n) :frames]) 0))
[clip sid n]
(if-let [{:keys [at rate]} (clip/source-time clip sid n)]
(+ at (/ (or (clip/frames clip (node/source n)) 0) rate))
(clip/frames clip sid)))
(defn- node-label
"What to call a node in the label column.
@ -126,7 +129,7 @@
(or (node/placed-span
(cond-> n
(and (= :instance (:kind n)) (nil? (:span n)))
(assoc :span [0 (source-frames clip n)])))
(assoc :span [0 (source-frames clip sid n)])))
[0 (:frames sym)]))
row {:path rpath
:depth depth
@ -167,13 +170,13 @@
(-> [row]
(into (channel-rows n rpath (inc depth) self span))
(into (when-let [{:keys [at rate]} (and (= :instance (:kind n))
(node/source-time n))]
(clip/source-time clip sid n))]
(when-let [child (node/source n)]
(walk child rpath (inc depth)
(comp self #(+ at (/ % rate)))))))))))))
ordered))))]
(if (get-in clip [:symbols sid])
(walk sid [] 0 identity)
(walk sid [] 0 #(/ % (:rate (clip/grid-time clip sid))))
[])))
(defn sound-rows

70
frontend/test/arthur/domain/cadence_test.cljs Normal file
View file

@ -0,0 +1,70 @@
(ns arthur.domain.cadence-test
(:require [cljs.test :refer [deftest is testing]]
[arthur.domain.bring :as bring]
[arthur.domain.cadence :as cadence]
[arthur.domain.channel :as ch]
[arthur.domain.clip :as clip]
[arthur.domain.leaf :as leaf]
[arthur.domain.nest :as nest]
[arthur.domain.node :as node]
[arthur.domain.palette :as pal]
[arthur.domain.pick :as pick]
[arthur.domain.pose :as pose]
[arthur.ui.timeline :as timeline]))
(def footage
{:fps 30 :width 100 :height 100
:symbols {:main {:id :main :fps 30 :frames 60
:nodes {:mark {:id :mark :kind :rect :z "a" :pose-group :mouth
:channels {[:geom :size] {:animated? true :pose-sampled? true
:dense {:store "sizes" :offset 0 :stride 1 :frames 60}}
[:style :color] (ch/framed :brow)}}}}}})
(def store {"sizes" {:data (js/Float64Array. (clj->js (range 1 61)))}})
(deftest selection-is-integral-and-never-early
(is (= [0 2 5 7 10 12] (mapv #(cadence/frame % 12 30) (range 6))))
(doseq [grid [8 12 24 30 60] native [12 24 30 60] f (range 90)]
(let [selected (cadence/frame f grid native)]
(is (and (integer? selected) (<= selected (/ (* f native) grid))
(< (- (/ (* f native) grid) selected) 1))))))
(deftest changing-output-fps-preserves-dense-content-and-seconds
(let [doc (clip/set-fps footage 12)
draw (clip/resolver doc :main store pal/index-of nil)]
(is (= 24 (clip/output-frames doc :main)))
(is (= (:symbols footage) (:symbols doc)))
(is (= footage (clip/set-fps doc 30)))
(is (= doc (leaf/clip "test" (leaf/leaves "test" doc))))
(is (= [1 3 6 8 11 13] (mapv #(:size (first (draw %))) (range 6))))
(is (= 7 (:frame (nest/inside doc store :main [:mark] 3))))
(is (= [0 24] (:span (first (timeline/rows doc :main #{})))))))
(deftest imported-footage-uses-selection-for-picture-and-real-speed-for-audio
(let [{doc :clip sid :sid} (bring/take (clip/set-fps (clip/blank) 12)
footage "take" "video" [15 75])
;; A host authored at 12, with a 30fps source starting half a second in.
doc (-> doc (assoc-in [:symbols :main :fps] 12)
(clip/place-symbol store :main sid 6 :insert nil))
n (get-in doc [:symbols :main :nodes :insert])
draw (clip/resolver doc :main store pal/index-of nil)
[sound] (nest/audio-tracks doc :main)]
(is (= 60 (clip/frames doc sid)))
(is (= 1 (get-in doc [:symbols sid :nodes :sound :time :rate])))
(is (= [0 24] (:span n)))
(is (empty? (draw 5)))
(is (= 8 (:size (first (draw 9)))))
(is (= 7 (:frame (nest/inside doc store :main [:insert :mark] 9))))
(is (= [-4 -4 4 4] ((pick/bounds-of doc store :main n) 3)))
(is (= [6 30] (node/placed-span sound)))
(is (= 15 (node/local-frame sound 6)))
(is (= 1 (* (get-in sound [:time :rate]) (/ (:fps doc) (:fps sound)))))
(is (= [6 30] (:span (first (timeline/rows doc :main #{[:insert]})))))
(testing "an off-grid closed mouth can win without modifying the dense source"
(let [pinned (pose/put-cut doc :main :insert :mouth 7 6)
r (clip/resolver pinned :main store pal/index-of nil)]
(is (= 7 (:size (first (r 9)))))
(is (= (get-in doc [:symbols sid]) (get-in pinned [:symbols sid])))))
(testing "a deliberate half speed still retimes audio"
(let [slow (assoc-in doc [:symbols :main :nodes :insert :playback :speed] 0.5)
[track] (nest/audio-tracks slow :main)]
(is (= 0.5 (* (get-in track [:time :rate]) (/ (:fps slow) (:fps track)))))))))

8
frontend/test/arthur/domain/gesture_test.cljs
View file

@ -98,7 +98,7 @@
[c st open path f]
(let [{:keys [sid id world frame]} (nest/placement c st open path f)
n (get-in c [:symbols sid :nodes id])
[x0 y0 x1 y1] ((pick/bounds-of c st n) frame)]
[x0 y0 x1 y1] ((pick/bounds-of c st sid n) frame)]
{:corners (mapv #(at world %) [[x0 y0] [x1 y0] [x1 y1] [x0 y1]])
:pivot (at world (:anchor (gesture/values n frame st)))}))
@ -135,7 +135,7 @@
[]
(let [c (two-down)]
(assoc-in c [:symbols :box :nodes :shape :channels [:xform :anchor]]
(ch/framed (pick/pivot c nil (get-in c [:symbols :box :nodes :shape]) [4])))))
(ch/framed (pick/pivot c nil :box (get-in c [:symbols :box :nodes :shape]) [4])))))
(deftest dragging-a-corner-away-from-the-middle-makes-it-bigger
(doseq [[what c path f]
@ -287,5 +287,5 @@
(deftest an-instances-box-is-what-its-symbol-draws
(let [c (two-down)
{:keys [frame]} (nest/placement c nil :main [u v] 16)]
(is (= [0 0 10 10] ((pick/bounds-of c nil (get-in c [:symbols :mid :nodes v])) frame)))
(is (= [0 0 10 10] ((pick/bounds-of c nil (get-in c [:symbols :box :nodes :shape])) 4)))))
(is (= [0 0 10 10] ((pick/bounds-of c nil :mid (get-in c [:symbols :mid :nodes v])) frame)))
(is (= [0 0 10 10] ((pick/bounds-of c nil :box (get-in c [:symbols :box :nodes :shape])) 4)))))

14
frontend/test/arthur/domain/instance_test.cljs
View file

@ -82,9 +82,7 @@
:sym/poses symbol}}
store {"sizes" {:data values}}
resolve (clip/resolver document :main store pal/index-of nil)
low-resolve (clip/resolver document :main store pal/index-of {:picture-fps 8})
at (fn [f] (into {} (map (fn [op] [(:node op) op])) (resolve f)))
low-at (fn [f] (into {} (map (fn [op] [(:node op) op])) (low-resolve f)))]
at (fn [f] (into {} (map (fn [op] [(:node op) op])) (resolve f)))]
(is (empty? (clip/problems document)))
(is (= document (leaf/clip "stage" (leaf/leaves "stage" document))))
(is (= 102 (:size (get (at 7) [:first :mouth]))) "eight static frames, plus its correction")
@ -97,14 +95,8 @@
(is (nil? (get (at 9) [:first :mouth]))
"generated visibility is read from the same selected pose")
(is (some? (get (at 9) [:second :mouth])))
(is (= 105 (:size (get (low-at 7) [:first :brow])))
"picture rate samples the generated base and leaves correction time alone")
(is (= 22 (:size (get (low-at 8) [:first :mouth])))
"an explicit cut occurs at its exact local frame, even off the picture grid")
(is (= 8 (:cx (get (low-at 8) [:first :eye])))
"authored position ignores the picture grid")
(let [sym (get-in document [:symbols :sym/poses])
opts {:source-fps 30 :picture-fps 8 :pose-tracks {:mouth {0 0, 8 20}}}]
opts {:pose-tracks {:mouth {0 0, 8 20}}}]
(is (= (mapv #(select-keys % [:node :cx :size])
(symbol/eval-frame sym 8 store pal/index-of opts))
(mapv #(select-keys % [:node :cx :size])
@ -249,7 +241,7 @@
made (clip/new-symbol c :outer id 20 u)]
(is (= :symbol-1 id))
(is (= :symbol-2 (clip/fresh-id made)) "the next one does not collide")
(is (= {:id :symbol-1 :name "symbol-1" :frames 180 :nodes {}}
(is (= {:id :symbol-1 :name "symbol-1" :fps 30 :frames 180 :nodes {}}
(clip/symbol made :symbol-1))
"empty, and as long as the rest of what it was placed in")
(is (= {:span [0 180] :time {:mode :map :at 20 :rate 1}}

4
frontend/test/arthur/domain/lane_test.cljs
View file

@ -165,8 +165,8 @@
(is (= 0 (:frame (nest/inside doc nil :main [:a] 3))))
(is (nil? (:time (nest/inside doc nil :main [:a] 3))))
(is (nil? (nest/inside doc nil :main [:a] 4)))
(is (= ((pick/bounds-of doc nil n) 2)
((pick/bounds-of doc nil (assoc-in n [:playback :in] 5)) 0)))))
(is (= ((pick/bounds-of doc nil :main n) 2)
((pick/bounds-of doc nil :main (assoc-in n [:playback :in] 5)) 0)))))
(deftest seeking-and-source-reuse-do-not-share-cursors
(let [doc (assoc-in (document) [:symbols :main :nodes :b :source :symbol] :drawing-a)

4
frontend/test/arthur/domain/leaf_test.cljs
View file

@ -49,8 +49,8 @@
;; A TIMELINE ID IS A SEGMENT, which is what lets a symbol's nodes be
;; addressed by the same path shape as the clip's own. `main` is the root.
(is (contains? ls "clip/c7/symbol/main"))
(is (= {:frames 229} (get ls "clip/c7/symbol/main"))
"a timeline's leaf is its frame space; :fps is the clip's")
(is (= {:frames 229 :fps 30} (get ls "clip/c7/symbol/main"))
"a symbol persists its native grid separately from the output grid")
;; Nodes are local to the face timeline; feature and group ids are clip-wide.
(is (contains? ls "clip/c7/symbol/face-1/node/mouth"))
(is (contains? ls "clip/c7/symbol/face-1/channel/mouth/geom.pts"))

50
frontend/test/arthur/export_test.cljs
View file

@ -1,19 +1,5 @@
(ns arthur.export-test
"The frame walk and the arithmetic above the sink.
THE SYNC RULE IS THE POINT OF THIS FILE. `arthur.export` states it twice — in
its own docstring and in `plan`'s comment — because it is the one failure the
export path exists to prevent: a lower picture rate must HOLD each pose across
several frames and never drop frames, so the emitted length always matches the
audio. Decimating instead gives a file that is silently short, whose sound
slides progressively out of sync, and which looks correct in every other
respect. Nothing downstream can detect that, so it is asserted here, at both
levels: `plan` reports poses separately from frames, and `run!` emits every
frame of the frame space whatever the picture rate is.
The sink is a recording fake. What the walk owes a sink is an ordering and a
count, and a fake is the only way to assert on those without also asserting on
PNG bytes — which `export.frames-test` already does."
"The output-grid frame walk and duration, using a recording sink."
(:require [cljs.test :refer [deftest is testing async]]
[arthur.domain.channel :as ch]
[arthur.domain.clip :as clip]
@ -85,21 +71,12 @@
(is (= 1 (zoom-of nil)) "an absent zoom is 1:1")
(is (= 1 (zoom-of 1.9)))))
(deftest a-lower-picture-rate-changes-the-poses-and-not-the-length
;; THE SYNC RULE, in the arithmetic. 48 frames at 24fps is two seconds; at a
;; 12fps picture rate it is still 48 frames and two seconds, holding 24 poses.
;; If :frames ever tracks :poses here, every export at a reduced picture rate
;; comes out half length with the audio sliding off it.
(let [p (export/plan {:clip (a-clip {:frames 48 :fps 24}) :symbol :main
:picture-fps 12})]
(is (= 48 (:frames p)) "the frame count does not move")
(is (= 2 (:seconds p)) "and neither does the duration")
(is (= 24 (:poses p)) "but the picture holds half as many poses"))
(testing "a picture rate at or above the clip's rate changes nothing"
(doseq [fps [24 48 nil]]
(let [p (export/plan {:clip (a-clip {:frames 48 :fps 24}) :symbol :main
:picture-fps fps})]
(is (= 48 (:poses p)) (str "picture-fps " fps))))))
(deftest a-lower-output-rate-keeps-native-duration
(let [doc (assoc-in (a-clip {:frames 48 :fps 24}) [:symbols :main :fps] 24)
p (export/plan {:clip (clip/set-fps doc 12) :symbol :main})]
(is (= 24 (:frames p)))
(is (= 12 (:fps p)))
(is (= 2 (:seconds p)))))
(deftest plan-of-a-symbol-that-is-not-there-is-nothing
(is (nil? (export/plan {:clip (a-clip {}) :symbol :nope :zoom 1}))))
@ -115,17 +92,12 @@
(done)))
(.catch (fn [e] (is false (str "threw: " e)) (done))))))
(deftest a-lower-picture-rate-still-emits-every-frame
;; THE SYNC RULE, in the walk — the assertion that matters most in this file.
;; The poses repeat; the frames do not thin out.
(deftest a-lower-output-rate-emits-the-selected-grid
(async done
(-> (run!* (a-clip {:frames 12 :fps 24}) :picture-fps 8)
(-> (run!* (clip/set-fps (a-clip {:frames 12 :fps 24}) 8))
(.then (fn [{:keys [frames spec]}]
(is (= 12 (count frames))
"a 12-frame timeline exports 12 frames at any picture rate")
(is (= (range 12) (map :i frames)))
(is (= 24 (:fps spec))
"and the file's rate is the CLIP's, not the picture rate")
(is (= (range 4) (map :i frames)))
(is (= 8 (:fps spec)))
(done)))
(.catch (fn [e] (is false (str "threw: " e)) (done))))))

4
frontend/test/arthur/flow/freeze_test.cljs
View file

@ -336,14 +336,14 @@
frames (range (get-in c [:symbols sid :frames]))
anchor (:value (get-in n [:channels [:xform :anchor]]))
usable (and (not (node/measured? n))
(some? (pick/pivot c @store n frames)))]
(some? (pick/pivot c @store sid n frames)))]
(is (= usable (some? anchor))
(str sid "/" id " has a pivot: " (some? anchor)
", but a hand can use one: " usable))
(is (= (nil? (gesture/refusal n)) (not (node/measured? n)))
(str sid "/" id ": `refusal` and `measured?` disagree"))
(when anchor
(let [bounds (pick/bounds-of c @store n)
(let [bounds (pick/bounds-of c @store sid n)
[x0 y0 x1 y1] (reduce #(let [k (bounds %2)]
(cond (nil? %1) k (nil? k) %1
:else (mapv (fn [op i] (op (nth %1 i) (nth k i)))