Play the audio mixdown instead of encoding a WAV of it
Opening the 8625 study froze the main thread for 4.7 seconds and settled at 615MB of heap. A profile put three quarters of a project open inside mix/wav-bytes, which playback had no business calling at all. Two separate causes. The peak scan built a lazy sequence of one boxed double per SAMPLE -- ten million of them for a seven-minute mix -- to compute a single maximum over data already sitting in Float32Arrays; the hand-written loop is 73x faster and agrees to the bit. The rest was structural: the WAV existed only because an <audio> element can hold a URL and nothing else, and the element existed only to be the clock. So a mixdown that was already rendered got encoded to 73MB of 16-bit PCM, on the main thread, on open, on every tab switch and on every edit to a track -- and a symbol with no sound got silence synthesized and encoded full length so the element had a duration to report. arthur.clock keeps its interface and all of its arithmetic; the position now comes from a backend behind a protocol. clock.graph plays the AudioBuffer through an AudioBufferSourceNode and derives the frame from the context's own clock, which is the audio device's position in double precision rather than whatever the media pipeline last published. clock.element is the old path, kept switchable while the new one earns trust -- BACKEND, or use-backend! -- which is also why every one of the original clock tests passes unchanged: the derivation they assert is shared, and the backends can only disagree about the position under it. 6.5s to 1.8s, 4.7s of blocking to 370ms, 615MB to 68MB. THE POSITION IS COMPENSATED FOR OUTPUT LATENCY, and piecewise because of it. currentTime is the quantum being rendered, which the speaker is tens of milliseconds behind; report the renderer and the picture leads the sound, which in a lip-sync tool is the only artefact that matters. Audio already rendered cannot be re-rated, though, so reading it back at a new rate jumped the playhead backwards by three latencies on every press of the rate button. Each play, pause, seek and rate change now records a segment and a position is read against whichever segment was in force when that audio was rendered. One duplicate fell out of this. Opening a project asked for its clock twice -- once from ::opened and once from a ::refresh-clock the shell raised because it compared symbol ids, and two different documents both open on :main. The sounds subscription carries the clip id now, so "an edit under the same symbol" means what it says. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
90b1fbe2f8
commit
5b5b9ae4c3
10 changed files with 846 additions and 98 deletions
|
|
@ -2,14 +2,19 @@
|
||||||
"Render independently placed audio tracks into one stage audio clock.
|
"Render independently placed audio tracks into one stage audio clock.
|
||||||
|
|
||||||
The mix is derived from saved audio track leaves and immutable footage blobs.
|
The mix is derived from saved audio track leaves and immutable footage blobs.
|
||||||
The transport still has one audio element, so seeking, rate changes and looping
|
The transport still has ONE clock, so seeking, rate changes and looping stay
|
||||||
stay tied to the same clock the picture reads.
|
tied to the same position the picture is drawn from.
|
||||||
|
|
||||||
THE AUDIO BUFFER IS THE PRODUCT AND THE WAV IS ONE PACKAGING OF IT. Playback
|
THE AUDIO BUFFER IS THE PRODUCT AND THE WAV IS ONE PACKAGING OF IT. `buffer!`
|
||||||
wants a URL an `<audio>` element can hold; an export wants the samples, either
|
renders and the wrappers below it package, rather than the render being spelled
|
||||||
as WAV bytes to put in an archive or as the `AudioBuffer` a muxer takes as an
|
once per consumer.
|
||||||
audio track. So `buffer!` renders and the two wrappers below it package, rather
|
|
||||||
than the render being spelled once per consumer."
|
PLAYBACK NO LONGER PACKAGES AT ALL. It takes the buffer as it is — see
|
||||||
|
`clock-source!` and `arthur.clock.graph` — because encoding a WAV so an
|
||||||
|
`<audio>` element had a URL to hold cost O(the clip's length) on the main
|
||||||
|
thread, on open, on every tab switch and on every edit to a track. The WAV is
|
||||||
|
now what an EXPORT wants: bytes for an archive, or the buffer itself for a
|
||||||
|
muxer. `clock!` below is the element backend's packaging and goes when it does."
|
||||||
(:require [arthur.domain.channel :as ch]
|
(:require [arthur.domain.channel :as ch]
|
||||||
[arthur.domain.clip :as clip]
|
[arthur.domain.clip :as clip]
|
||||||
[arthur.domain.nest :as nest]
|
[arthur.domain.nest :as nest]
|
||||||
|
|
@ -29,9 +34,22 @@
|
||||||
bytes (js/ArrayBuffer. (+ 44 (* frames channels 2)))
|
bytes (js/ArrayBuffer. (+ 44 (* frames channels 2)))
|
||||||
view (js/DataView. bytes)
|
view (js/DataView. bytes)
|
||||||
samples (mapv #(.getChannelData buffer %) (range channels))
|
samples (mapv #(.getChannelData buffer %) (range channels))
|
||||||
peak (reduce max 0
|
;; A HAND-WRITTEN LOOP over the typed arrays, not `(reduce max (for ...))`.
|
||||||
(for [channel samples i (range frames)]
|
;; The lazy sequence that read beautifully allocated one boxed double per
|
||||||
(js/Math.abs (aget channel i))))
|
;; SAMPLE — ten million of them for a three-minute mix — and spent the
|
||||||
|
;; whole of a sixteen-second project open walking them and collecting
|
||||||
|
;; them. Same arithmetic, no allocation.
|
||||||
|
peak (loop [c 0 p 0]
|
||||||
|
(if (< c channels)
|
||||||
|
(recur (inc c)
|
||||||
|
(let [^js data (nth samples c)]
|
||||||
|
(loop [i 0 p p]
|
||||||
|
(if (< i frames)
|
||||||
|
(recur (inc i)
|
||||||
|
(let [a (js/Math.abs (aget data i))]
|
||||||
|
(if (> a p) a p)))
|
||||||
|
p))))
|
||||||
|
p))
|
||||||
level (if (> peak 0.98) (/ 0.98 peak) 1)]
|
level (if (> peak 0.98) (/ 0.98 peak) 1)]
|
||||||
(doseq [[offset word] [[0 "RIFF"] [8 "WAVE"] [12 "fmt "] [36 "data"]]]
|
(doseq [[offset word] [[0 "RIFF"] [8 "WAVE"] [12 "fmt "] [36 "data"]]]
|
||||||
(dotimes [i 4] (.setUint8 view (+ offset i) (.charCodeAt word i))))
|
(dotimes [i 4] (.setUint8 view (+ offset i) (.charCodeAt word i))))
|
||||||
|
|
@ -44,11 +62,14 @@
|
||||||
(.setUint16 view 32 (* channels 2) true)
|
(.setUint16 view 32 (* channels 2) true)
|
||||||
(.setUint16 view 34 16 true)
|
(.setUint16 view 34 16 true)
|
||||||
(.setUint32 view 40 (* frames channels 2) true)
|
(.setUint32 view 40 (* frames channels 2) true)
|
||||||
(dotimes [i frames]
|
;; Channel-outer so the channel's array is looked up once rather than once
|
||||||
|
;; per frame. The byte offsets are unchanged, so the interleaving is too.
|
||||||
(dotimes [c channels]
|
(dotimes [c channels]
|
||||||
(let [sample (* level (aget (get samples c) i))]
|
(let [^js data (nth samples c)]
|
||||||
|
(dotimes [i frames]
|
||||||
|
(let [sample (* level (aget data i))]
|
||||||
(.setInt16 view (+ 44 (* (+ (* i channels) c) 2))
|
(.setInt16 view (+ 44 (* (+ (* i channels) c) 2))
|
||||||
(js/Math.round (* 32767 (max -1 (min 1 sample)))) true))))
|
(js/Math.round (* 32767 (max -1 (min 1 sample)))) true)))))
|
||||||
(js/Uint8Array. bytes)))
|
(js/Uint8Array. bytes)))
|
||||||
|
|
||||||
(defn- wav-url [^js buffer]
|
(defn- wav-url [^js buffer]
|
||||||
|
|
@ -171,17 +192,43 @@
|
||||||
(.arrayBuffer response)))
|
(.arrayBuffer response)))
|
||||||
(.then decode-bytes!)))
|
(.then decode-bytes!)))
|
||||||
|
|
||||||
(defn mix!
|
(defn clock-source!
|
||||||
"Promise of a mixed WAV URL for symbol `sid`, or the original URL when it has
|
"Promise of `{:buffer :seconds}` — the audio the transport runs its clock on
|
||||||
no audio tracks. Each track can be trimmed and faded independently of its
|
while symbol `sid` is open, and how long that clock is.
|
||||||
linked picture."
|
|
||||||
|
Same order of preference as the WAV packaging in `clock!` below: the symbol's
|
||||||
|
own placed tracks, mixed; the document's audio file, for the symbol the
|
||||||
|
document opens on and only that one; and otherwise NO BUFFER AT ALL and the
|
||||||
|
symbol's own length.
|
||||||
|
|
||||||
|
THE SILENT CASE IS WHY THE DURATION IS RETURNED BESIDE THE BUFFER rather than
|
||||||
|
read off it. A symbol with no sound still needs a clock exactly as long as it
|
||||||
|
is, and `clock!` had to synthesize that silence and then ENCODE it, full
|
||||||
|
length, so an element had a duration to report. There is nothing to decode for
|
||||||
|
a symbol with no sound: saying how long it is answers the only question the
|
||||||
|
silence was ever asked."
|
||||||
[document sid fallback-url store]
|
[document sid fallback-url store]
|
||||||
(-> (buffer! document sid store)
|
(-> (buffer! document sid store)
|
||||||
(.then (fn [buffer] (if buffer (wav-url buffer) fallback-url)))))
|
(.then (fn [^js buffer]
|
||||||
|
(cond
|
||||||
|
buffer
|
||||||
|
{:buffer buffer :seconds (.-duration buffer)}
|
||||||
|
|
||||||
|
(and fallback-url (= sid (clip/opens-on document)))
|
||||||
|
(-> (decode! fallback-url)
|
||||||
|
(.then (fn [^js b] {:buffer b :seconds (.-duration b)})))
|
||||||
|
|
||||||
|
:else
|
||||||
|
{:buffer nil
|
||||||
|
:seconds (/ (clip/output-frames document sid) (:fps document))})))))
|
||||||
|
|
||||||
(defn clock!
|
(defn clock!
|
||||||
"Promise of the URL the transport should play while symbol `sid` is open.
|
"Promise of the URL the transport should play while symbol `sid` is open.
|
||||||
|
|
||||||
|
THE ELEMENT BACKEND'S PACKAGING of `clock-source!`, kept while that backend is
|
||||||
|
— see `arthur.clock`. Every cost this namespace had on open is in the two
|
||||||
|
`wav-url` calls below.
|
||||||
|
|
||||||
The frame is derived from the audio element and from nothing else, so every
|
The frame is derived from the audio element and from nothing else, so every
|
||||||
open symbol needs a sound exactly as long as it is. In order: its own placed
|
open symbol needs a sound exactly as long as it is. In order: its own placed
|
||||||
tracks, mixed; the document's audio file, for the symbol the document opens on
|
tracks, mixed; the document's audio file, for the symbol the document opens on
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,7 @@
|
||||||
|
|
||||||
THE FRAME IS DERIVED FROM THE AUDIO, never counted:
|
THE FRAME IS DERIVED FROM THE AUDIO, never counted:
|
||||||
|
|
||||||
frame = ⌊currentTime · fps⌋
|
frame = ⌊position · fps⌋
|
||||||
|
|
||||||
A loop that counted frames and hoped to keep up would drift, and drift against
|
A loop that counted frames and hoped to keep up would drift, and drift against
|
||||||
a voice is the one artefact that cannot be fixed downstream — a lip-sync tool
|
a voice is the one artefact that cannot be fixed downstream — a lip-sync tool
|
||||||
|
|
@ -12,26 +12,79 @@
|
||||||
The failure mode becomes a visible stutter rather than an invisible slide, and
|
The failure mode becomes a visible stutter rather than an invisible slide, and
|
||||||
those are very different bugs to own.
|
those are very different bugs to own.
|
||||||
|
|
||||||
½× and ¼× are `playbackRate` and nothing else. The audio slows, `currentTime`
|
½× and ¼× are the backend's playback rate and nothing else. The audio slows,
|
||||||
advances proportionally, and the derived frame follows — so slow motion cannot
|
the position advances proportionally, and the derived frame follows — so slow
|
||||||
desync by construction. Implementing rate as a multiplier on a counted frame
|
motion cannot desync by construction. Implementing rate as a multiplier on a
|
||||||
would give the picture a rate and the sound another.
|
counted frame would give the picture a rate and the sound another.
|
||||||
|
|
||||||
It is outside app-db because the audio element is the source of truth and
|
It is outside app-db because the audio is the source of truth and copying it
|
||||||
copying it into the db every frame would make the db a lagging mirror of
|
into the db every frame would make the db a lagging mirror of something
|
||||||
something authoritative elsewhere. What DOES belong in the db is the playhead
|
authoritative elsewhere. What DOES belong in the db is the playhead as a piece
|
||||||
as a piece of document state — see events/playback — and that is written from
|
of document state — see events/playback — and that is written from here, not
|
||||||
here, not read by here."
|
read by here.
|
||||||
(:require [arthur.domain.node :as node]))
|
|
||||||
|
|
||||||
(defonce ^:private el (atom nil))
|
TWO BACKENDS, ONE ARITHMETIC. `position` comes from either a Web Audio graph
|
||||||
|
(`clock.graph`, the default) or an `<audio>` element (`clock.element`, kept
|
||||||
|
switchable while the first earns trust). Everything below the position — the
|
||||||
|
derivation, the clamp, the exposure grid — is here and is the same either way,
|
||||||
|
so the two can be compared on the same take rather than swapped on faith.
|
||||||
|
Build with `:closure-defines {arthur.clock/BACKEND \"element\"}`, or call
|
||||||
|
`use-backend!` from the console, to put the element back."
|
||||||
|
(:require [arthur.clock.element :as element]
|
||||||
|
[arthur.clock.graph :as graph]
|
||||||
|
[arthur.clock.transport :as t]
|
||||||
|
[arthur.domain.node :as node]))
|
||||||
|
|
||||||
|
(goog-define ^String BACKEND "graph")
|
||||||
|
|
||||||
|
(defonce ^:private mode (atom (keyword BACKEND)))
|
||||||
|
|
||||||
|
;; `{:source x :backend b}`. The source is kept so that re-attaching the same
|
||||||
|
;; thing can be recognised as the no-op it is — see `install!`.
|
||||||
|
(defonce ^:private current (atom nil))
|
||||||
|
|
||||||
|
(defn graph?
|
||||||
|
"Whether the app should wire up the graph backend. Read by `ui/shell`, which
|
||||||
|
renders the audio element only when this is false, and by `events/playback`,
|
||||||
|
which fetches a buffer rather than a WAV URL when it is true."
|
||||||
|
[]
|
||||||
|
(and (= :graph @mode) (graph/available?)))
|
||||||
|
|
||||||
|
(defn use-backend!
|
||||||
|
"Switch backends. Takes effect on the next thing that attaches one, which is
|
||||||
|
the next tab switch or document open — nothing is torn down under a take
|
||||||
|
that is already playing."
|
||||||
|
[m]
|
||||||
|
(reset! mode m))
|
||||||
|
|
||||||
|
(defn- install!
|
||||||
|
"Put a backend on `source`, unless `source` is already the clock's.
|
||||||
|
|
||||||
|
IDEMPOTENCE IS LOAD-BEARING HERE. The element's `:ref` is an inline closure,
|
||||||
|
so React hands it the same node again on every re-render of the shell —
|
||||||
|
rebuilding the clock there would mean a pane being dragged released whatever
|
||||||
|
was playing. The source is the identity: the element, or the buffer and its
|
||||||
|
length."
|
||||||
|
[source make]
|
||||||
|
(let [{:keys [backend] prev :source} @current]
|
||||||
|
(when-not (and backend (= prev source))
|
||||||
|
(when backend (t/-release! backend))
|
||||||
|
(reset! current {:source source
|
||||||
|
:backend (when (some? source) (make))}))))
|
||||||
|
|
||||||
(defn attach!
|
(defn attach!
|
||||||
"Hand the clock its audio element. Idempotent."
|
"Hand the clock an audio element. Idempotent. Element backend only — the
|
||||||
|
`:ref` that calls this is on a node `ui/shell` renders only in that mode."
|
||||||
[audio-el]
|
[audio-el]
|
||||||
(reset! el audio-el))
|
(install! audio-el #(element/backend audio-el)))
|
||||||
|
|
||||||
(defn element [] @el)
|
(defn attach-buffer!
|
||||||
|
"Hand the clock `seconds` of audio to run on, as an `AudioBuffer` or as nil
|
||||||
|
for silence of that length. Graph backend only."
|
||||||
|
[buffer seconds]
|
||||||
|
;; Keyed on the length as well as the buffer, because two silent clocks of
|
||||||
|
;; different lengths are two different clocks and both have a nil buffer.
|
||||||
|
(install! [buffer seconds] #(graph/backend buffer seconds)))
|
||||||
|
|
||||||
(defn- clamp [f frames]
|
(defn- clamp [f frames]
|
||||||
(-> f (max 0) (min (dec frames))))
|
(-> f (max 0) (min (dec frames))))
|
||||||
|
|
@ -39,56 +92,52 @@
|
||||||
(defn frame
|
(defn frame
|
||||||
"The clip frame the audio is currently on."
|
"The clip frame the audio is currently on."
|
||||||
[fps frames]
|
[fps frames]
|
||||||
(if-let [a @el]
|
(if-let [b (:backend @current)]
|
||||||
(clamp (js/Math.floor (* (.-currentTime a) fps)) frames)
|
(clamp (js/Math.floor (* (t/-position b) fps)) frames)
|
||||||
0))
|
0))
|
||||||
|
|
||||||
(defn playing? []
|
(defn playing? []
|
||||||
(boolean (when-let [a @el] (and (not (.-paused a)) (not (.-ended a))))))
|
(boolean (when-let [b (:backend @current)] (t/-playing? b))))
|
||||||
|
|
||||||
(defn rate []
|
(defn rate []
|
||||||
(if-let [a @el] (.-playbackRate a) 1.0))
|
(if-let [b (:backend @current)] (t/-rate b) 1.0))
|
||||||
|
|
||||||
(defn set-rate! [r]
|
(defn set-rate! [r]
|
||||||
(when-let [a @el] (set! (.-playbackRate a) r)))
|
(when-let [b (:backend @current)] (t/-set-rate! b r)))
|
||||||
|
|
||||||
(defn play! []
|
(defn play! []
|
||||||
(when-let [a @el]
|
(when-let [b (:backend @current)] (t/-play! b)))
|
||||||
;; Returns a promise that rejects if the browser has not seen a gesture yet.
|
|
||||||
;; Swallowed: the transport button IS the gesture, so this can only fire on a
|
|
||||||
;; programmatic play, where a console error is noise rather than news.
|
|
||||||
(some-> (.play a) (.catch (fn [_])))))
|
|
||||||
|
|
||||||
(defn pause! []
|
(defn pause! []
|
||||||
(when-let [a @el] (.pause a)))
|
(when-let [b (:backend @current)] (t/-pause! b)))
|
||||||
|
|
||||||
(defn seek!
|
(defn seek!
|
||||||
"Put the audio at the start of frame f. Seeking to the frame's start rather
|
"Put the audio at the start of frame f. Seeking to the frame's start rather
|
||||||
than its middle keeps `frame` idempotent: seek to f, read back f."
|
than its middle keeps `frame` idempotent: seek to f, read back f."
|
||||||
[fps frames f]
|
[fps frames f]
|
||||||
(when-let [a @el]
|
(when-let [b (:backend @current)]
|
||||||
(set! (.-currentTime a) (/ (clamp f frames) fps))))
|
(t/-seek! b (/ (clamp f frames) fps))))
|
||||||
|
|
||||||
(defn set-loop!
|
(defn set-loop!
|
||||||
"Wrap at the end instead of stopping. The frame stays derived — `currentTime`
|
"Wrap at the end instead of stopping. The frame stays derived — the position
|
||||||
simply returns to zero — so nothing about the sync changes, which is the point
|
simply returns to zero — so nothing about the sync changes, which is the point
|
||||||
of not counting frames.
|
of not counting frames.
|
||||||
|
|
||||||
It earns its place at 2x and 4x, where the whole clip is gone in under four
|
It earns its place at 2x and 4x, where the whole clip is gone in under four
|
||||||
seconds and a profile wants more than that to look at."
|
seconds and a profile wants more than that to look at."
|
||||||
[on?]
|
[on?]
|
||||||
(when-let [a @el] (set! (.-loop a) (boolean on?))))
|
(when-let [b (:backend @current)] (t/-set-loop! b on?)))
|
||||||
|
|
||||||
(defn set-muted! [on?]
|
(defn set-muted! [on?]
|
||||||
(when-let [a @el] (set! (.-muted a) (boolean on?))))
|
(when-let [b (:backend @current)] (t/-set-muted! b on?)))
|
||||||
|
|
||||||
(defn duration-frames
|
(defn duration-frames
|
||||||
"How many frames the audio actually covers, which need not be the clip's
|
"How many frames the audio actually covers, which need not be the clip's
|
||||||
length. Reported rather than assumed: a clip longer than its audio is a
|
length. Reported rather than assumed: a clip longer than its audio is a
|
||||||
legitimate thing to be told about, not a thing to silently truncate."
|
legitimate thing to be told about, not a thing to silently truncate."
|
||||||
[fps]
|
[fps]
|
||||||
(when-let [a @el]
|
(when-let [b (:backend @current)]
|
||||||
(let [d (.-duration a)]
|
(let [d (t/-duration b)]
|
||||||
(when (and d (js/isFinite d)) (js/Math.ceil (* d fps))))))
|
(when (and d (js/isFinite d)) (js/Math.ceil (* d fps))))))
|
||||||
|
|
||||||
(defn exposed-frame
|
(defn exposed-frame
|
||||||
|
|
|
||||||
45
frontend/src/arthur/clock/element.cljs
Normal file
45
frontend/src/arthur/clock/element.cljs
Normal file
|
|
@ -0,0 +1,45 @@
|
||||||
|
(ns arthur.clock.element
|
||||||
|
"The clock on an `<audio>` element. The original backend, kept switchable.
|
||||||
|
|
||||||
|
Reads `currentTime` and takes it as the position. That is the whole of it, and
|
||||||
|
it is why this backend needs a WAV: an element holds a URL, so a mixdown has
|
||||||
|
to be encoded and blobbed before it can be played — see `audio/mix`'s
|
||||||
|
`clock!`. The encode is O(the clip's length) and runs on the main thread, so a
|
||||||
|
long take costs seconds of frozen UI on open, on every tab switch and on every
|
||||||
|
edit to a track. `arthur.clock.graph` exists to not do that.
|
||||||
|
|
||||||
|
What this backend still has that the graph one has to be told: the OS media
|
||||||
|
keys and the media session, which the element gets from the browser for free.
|
||||||
|
|
||||||
|
KEPT so the two can be compared on the same document rather than swapped on
|
||||||
|
faith. When the graph backend has been trusted for a while, this namespace and
|
||||||
|
`mix/clock!`'s WAV packaging go together."
|
||||||
|
(:require [arthur.clock.transport :as t]))
|
||||||
|
|
||||||
|
(deftype Element [^js el]
|
||||||
|
t/Transport
|
||||||
|
(-position [_] (.-currentTime el))
|
||||||
|
(-duration [_] (.-duration el))
|
||||||
|
(-playing? [_] (and (not (.-paused el)) (not (.-ended el))))
|
||||||
|
(-rate [_] (.-playbackRate el))
|
||||||
|
(-set-rate! [_ r] (set! (.-playbackRate el) r))
|
||||||
|
(-play! [_]
|
||||||
|
;; Returns a promise that rejects if the browser has not seen a gesture yet.
|
||||||
|
;; Swallowed: the transport button IS the gesture, so this can only fire on a
|
||||||
|
;; programmatic play, where a console error is noise rather than news.
|
||||||
|
(some-> (.play el) (.catch (fn [_]))))
|
||||||
|
(-pause! [_] (.pause el))
|
||||||
|
(-seek! [_ seconds] (set! (.-currentTime el) seconds))
|
||||||
|
(-set-loop! [_ on?] (set! (.-loop el) (boolean on?)))
|
||||||
|
(-set-muted! [_ on?] (set! (.-muted el) (boolean on?)))
|
||||||
|
(-release! [_]
|
||||||
|
;; Nothing. React owns the element and unmounting it is what stops it; this
|
||||||
|
;; backend is a few property accesses wrapped in a type and holds no more
|
||||||
|
;; than that.
|
||||||
|
nil))
|
||||||
|
|
||||||
|
(defn backend
|
||||||
|
"Wrap an audio element — or anything that answers the same five properties,
|
||||||
|
which is what `clock-test`'s fake is."
|
||||||
|
[el]
|
||||||
|
(->Element el))
|
||||||
266
frontend/src/arthur/clock/graph.cljs
Normal file
266
frontend/src/arthur/clock/graph.cljs
Normal file
|
|
@ -0,0 +1,266 @@
|
||||||
|
(ns arthur.clock.graph
|
||||||
|
"The clock on a Web Audio graph. No encode, no blob, no element.
|
||||||
|
|
||||||
|
THE BUFFER IS PLAYED, NOT PACKAGED. `audio/mix`'s `buffer!` already renders
|
||||||
|
the mixdown; the element backend then spent O(the clip's length) encoding that
|
||||||
|
buffer to a WAV purely so a `src` attribute had something to point at. An
|
||||||
|
`AudioBufferSourceNode` takes the buffer as it is, so opening a document costs
|
||||||
|
one node instead of a hundred megabytes of 16-bit PCM.
|
||||||
|
|
||||||
|
THE FRAME IS STILL DERIVED, AND FROM A BETTER CLOCK. `AudioContext.currentTime`
|
||||||
|
is the audio device's own position in double precision, advancing every 128
|
||||||
|
samples; an element's `currentTime` is whatever the media pipeline last
|
||||||
|
published and is permitted to lag. Nothing is counted here either — position is
|
||||||
|
an anchor plus elapsed context time, and the anchor is re-set on every seek and
|
||||||
|
every rate change, so no arithmetic accumulates across either.
|
||||||
|
|
||||||
|
AND IT IS LATENCY-COMPENSATED, which is the one thing this backend must get
|
||||||
|
right. `currentTime` is the time of the quantum being RENDERED, which the
|
||||||
|
speaker is `outputLatency` behind — tens of milliseconds, far more over
|
||||||
|
Bluetooth. Report the renderer's position and the picture leads the sound by
|
||||||
|
exactly that much, which in a lip-sync tool is the only artefact that matters.
|
||||||
|
So `scheduled` is the bookkeeping and `-position` is `scheduled` read one
|
||||||
|
latency in the past. The element has the same lag underneath; the difference is
|
||||||
|
that this one is a number we can subtract rather than an error we inherit.
|
||||||
|
|
||||||
|
A SYMBOL WITH NO SOUND GETS A CLOCK ANYWAY, with no buffer at all: `seconds`
|
||||||
|
is its length and the context's clock does the rest. The element backend had to
|
||||||
|
synthesize silence and encode THAT to a WAV, full length, so the element had a
|
||||||
|
duration to report — the clearest sign that the element was driving the design
|
||||||
|
rather than serving it."
|
||||||
|
(:require [arthur.clock.transport :as t]))
|
||||||
|
|
||||||
|
(defonce ^:private shared (atom nil))
|
||||||
|
|
||||||
|
(defn available?
|
||||||
|
"Whether this backend can run at all. False under node, where the tests live."
|
||||||
|
[]
|
||||||
|
(exists? js/AudioContext))
|
||||||
|
|
||||||
|
(defn context
|
||||||
|
"The app's one `AudioContext`, made on first use.
|
||||||
|
|
||||||
|
Lazy because constructing one before anything wants to play is how a browser
|
||||||
|
decides the page is trying to autoplay, and because `available?` is false in
|
||||||
|
the test runner."
|
||||||
|
[]
|
||||||
|
(or (:ctx @shared)
|
||||||
|
(let [ctx (js/AudioContext.)
|
||||||
|
;; ONE output gain for the app, not one per backend: a tab switch
|
||||||
|
;; replaces the backend, and a gain node per backend would leave the
|
||||||
|
;; old one wired to the destination. Mute lives here for the same
|
||||||
|
;; reason — it is a property of the transport, not of whichever
|
||||||
|
;; buffer happens to be loaded.
|
||||||
|
out (.createGain ctx)]
|
||||||
|
(.connect out (.-destination ctx))
|
||||||
|
(:ctx (reset! shared {:ctx ctx :out out})))))
|
||||||
|
|
||||||
|
(defn output [] (do (context) (:out @shared)))
|
||||||
|
|
||||||
|
(defn- latency
|
||||||
|
"How far ahead of the speaker `currentTime` is, in seconds.
|
||||||
|
|
||||||
|
`outputLatency` is the whole path and the number we want. Firefox reports 0
|
||||||
|
until the graph has actually run, so `baseLatency` stands in — it is only the
|
||||||
|
graph's own buffering and therefore an underestimate, which errs towards the
|
||||||
|
picture leading slightly rather than the correction overshooting. Neither
|
||||||
|
exists everywhere; 0 is then no worse than the element."
|
||||||
|
[^js ctx]
|
||||||
|
(let [out (.-outputLatency ctx)
|
||||||
|
base (.-baseLatency ctx)]
|
||||||
|
(cond
|
||||||
|
(and (number? out) (js/isFinite out) (pos? out)) out
|
||||||
|
(and (number? base) (js/isFinite base) (pos? base)) base
|
||||||
|
:else 0)))
|
||||||
|
|
||||||
|
(defn- seg-at
|
||||||
|
"The segment governing context time `t`."
|
||||||
|
[segs t]
|
||||||
|
(or (last (filter #(<= (:from %) t) segs)) (first segs)))
|
||||||
|
|
||||||
|
(defn- raw
|
||||||
|
"Where the audio is at context time `t`, in seconds into the buffer.
|
||||||
|
|
||||||
|
PIECEWISE, and that is the whole subtlety of this namespace. Audio already
|
||||||
|
rendered cannot be re-rated: when the transport goes 1x -> 4x, the samples
|
||||||
|
still travelling to the speaker were rendered at 1x, so reading them back at
|
||||||
|
4x jumps the playhead BACKWARDS by three output latencies — about fourteen
|
||||||
|
frames on a laptop, which is a visible lurch on every rate change and was
|
||||||
|
exactly what the first version of this did. So each `play`, `pause`, `seek`
|
||||||
|
and rate change records a segment, and a position is read against whichever
|
||||||
|
segment was in force when that audio was rendered.
|
||||||
|
|
||||||
|
Evaluated before the oldest segment we kept, it clamps. That is what makes a
|
||||||
|
seek read back exactly what was seeked to: a seek has nothing in flight worth
|
||||||
|
honouring — the user has jumped — so its segment starts the timeline over."
|
||||||
|
[segs t]
|
||||||
|
(let [t (max t (:from (first segs)))
|
||||||
|
s (seg-at segs t)]
|
||||||
|
(+ (:anchor s) (* (:rate s) (- t (:from s))))))
|
||||||
|
|
||||||
|
(defn- at-renderer
|
||||||
|
"Where the graph has rendered up to: `raw` at the context's own clock."
|
||||||
|
[{:keys [^js ctx segs]}]
|
||||||
|
(raw segs (.-currentTime ctx)))
|
||||||
|
|
||||||
|
(defn- at-speaker
|
||||||
|
"Where the sound being heard is: `raw` one output latency in the past. See the
|
||||||
|
namespace docstring — this is the whole of the compensation."
|
||||||
|
[{:keys [^js ctx segs]}]
|
||||||
|
(raw segs (- (.-currentTime ctx) (latency ctx))))
|
||||||
|
|
||||||
|
(defn- running?
|
||||||
|
"Whether the renderer is advancing. Read off the segments rather than kept
|
||||||
|
beside them, so there is one answer and not two that can disagree."
|
||||||
|
[{:keys [segs]}]
|
||||||
|
(pos? (:rate (last segs))))
|
||||||
|
|
||||||
|
(defn- prune
|
||||||
|
"Drop segments no position can still need: everything before the last one that
|
||||||
|
began at or before the in-flight window. Unbounded history would otherwise
|
||||||
|
grow by one entry per rate change for the life of the document."
|
||||||
|
[segs cutoff]
|
||||||
|
(let [n (count (take-while #(<= (:from %) cutoff) segs))]
|
||||||
|
(if (<= n 1) segs (subvec segs (dec n)))))
|
||||||
|
|
||||||
|
(defn- position
|
||||||
|
"`at-speaker`, brought inside the audio. Clamped before the wrap, so the few
|
||||||
|
milliseconds of negative position right after a looped play — the first
|
||||||
|
samples are still in the output buffer — read as 0 rather than as the end."
|
||||||
|
[{:keys [seconds loop?] :as st}]
|
||||||
|
(let [p (at-speaker st)]
|
||||||
|
(cond
|
||||||
|
(not (and seconds (pos? seconds))) 0
|
||||||
|
loop? (mod (max 0 p) seconds)
|
||||||
|
:else (-> p (max 0) (min seconds)))))
|
||||||
|
|
||||||
|
(defn- done?
|
||||||
|
"Run off the end. Judged at the SPEAKER, so playback is still reported as
|
||||||
|
running while the last scheduled samples are on their way out."
|
||||||
|
[{:keys [seconds loop?] :as st}]
|
||||||
|
(and (not loop?) seconds (pos? seconds) (>= (at-speaker st) seconds)))
|
||||||
|
|
||||||
|
(defn- spin-up!
|
||||||
|
"A fresh source node playing from where the renderer now is.
|
||||||
|
`AudioBufferSourceNode`s are single-use, so a play, a seek while playing and a
|
||||||
|
resumed pause each make a new one; they are cheap, which is the point of this
|
||||||
|
backend.
|
||||||
|
|
||||||
|
Nil when there is no buffer — the silent clock runs on the context alone."
|
||||||
|
[{:keys [^js ctx ^js buffer ^js out rate loop? seconds] :as st}]
|
||||||
|
(when buffer
|
||||||
|
(let [node (.createBufferSource ctx)]
|
||||||
|
(set! (.-buffer node) buffer)
|
||||||
|
(set! (.-loop node) (boolean loop?))
|
||||||
|
(set! (.-value (.-playbackRate node)) rate)
|
||||||
|
(.connect node out)
|
||||||
|
;; `when` 0 is "as soon as the graph can"; the offset is where in the
|
||||||
|
;; buffer to begin. Clamped because starting past the end is a range
|
||||||
|
;; error rather than a no-op in some engines.
|
||||||
|
(.start node 0 (-> (at-renderer st) (max 0) (min (or seconds 0))))
|
||||||
|
node)))
|
||||||
|
|
||||||
|
(defn- spin-down! [{:keys [^js node]}]
|
||||||
|
(when node
|
||||||
|
(try (.stop node) (catch :default _ nil))
|
||||||
|
(try (.disconnect node) (catch :default _ nil))))
|
||||||
|
|
||||||
|
(defn- restart!
|
||||||
|
"Begin a new timeline at `anchor`, running at `rate` or held when it is 0.
|
||||||
|
|
||||||
|
A RESET rather than a segment, for the three cases that have nothing in flight
|
||||||
|
worth honouring: a play starts fresh, a seek means the user has jumped, and a
|
||||||
|
pause wants one stable number for the readout rather than a position that
|
||||||
|
creeps forward as the output buffer drains."
|
||||||
|
[state anchor rate]
|
||||||
|
(let [st @state]
|
||||||
|
(spin-down! st)
|
||||||
|
(swap! state assoc
|
||||||
|
:segs [{:from (.-currentTime ^js (:ctx st)) :anchor anchor :rate rate}]
|
||||||
|
:node nil)
|
||||||
|
(when (pos? rate)
|
||||||
|
(swap! state assoc :node (spin-up! @state)))))
|
||||||
|
|
||||||
|
(deftype Graph [state]
|
||||||
|
t/Transport
|
||||||
|
(-position [_] (position @state))
|
||||||
|
(-duration [_] (:seconds @state))
|
||||||
|
(-playing? [_] (let [st @state] (boolean (and (running? st) (not (done? st))))))
|
||||||
|
(-rate [_] (:rate @state))
|
||||||
|
|
||||||
|
(-set-rate! [_ r]
|
||||||
|
;; A SEGMENT, not a reset — the one case where what is already in flight has
|
||||||
|
;; to keep its old rate or the playhead lurches. See `raw`.
|
||||||
|
(let [st @state
|
||||||
|
now (.-currentTime ^js (:ctx st))]
|
||||||
|
(if (running? st)
|
||||||
|
(let [anchor (at-renderer st)]
|
||||||
|
(swap! state #(-> %
|
||||||
|
(assoc :rate r)
|
||||||
|
(update :segs (fn [segs]
|
||||||
|
(prune (conj segs {:from now :anchor anchor :rate r})
|
||||||
|
(- now (latency ^js (:ctx st))))))))
|
||||||
|
(when-let [^js node (:node st)]
|
||||||
|
(set! (.-value (.-playbackRate node)) r)))
|
||||||
|
;; Stopped: nothing is in flight and nothing is rendering, so the rate
|
||||||
|
;; is simply what the next play will run at.
|
||||||
|
(swap! state assoc :rate r))))
|
||||||
|
|
||||||
|
(-play! [_]
|
||||||
|
(let [st @state]
|
||||||
|
;; `done?` counts as not playing. It is DERIVED — no flag is cleared when
|
||||||
|
;; the take runs off its end — so testing "running" alone would make play
|
||||||
|
;; a dead button from the moment the audio finished.
|
||||||
|
(when (or (not (running? st)) (done? st))
|
||||||
|
;; The context starts suspended and only a gesture may resume it. The
|
||||||
|
;; transport button IS the gesture, so a rejection here means a
|
||||||
|
;; programmatic play and is noise rather than news — as on the element.
|
||||||
|
(some-> (.resume ^js (:ctx st)) (.catch (fn [_])))
|
||||||
|
;; Play at the end starts over, which is what the element does.
|
||||||
|
(restart! state (if (done? st) 0 (position st)) (:rate st)))))
|
||||||
|
|
||||||
|
(-pause! [_]
|
||||||
|
(let [st @state]
|
||||||
|
(when (running? st)
|
||||||
|
;; Anchored at what was HEARD, not at what was scheduled, so play after
|
||||||
|
;; pause resumes from where the sound stopped. It re-plays the last few
|
||||||
|
;; milliseconds rather than skipping them, which is the kinder of the
|
||||||
|
;; two roundings.
|
||||||
|
(restart! state (position st) 0))))
|
||||||
|
|
||||||
|
(-seek! [_ seconds]
|
||||||
|
(restart! state seconds (if (running? @state) (:rate @state) 0)))
|
||||||
|
|
||||||
|
(-set-loop! [_ on?]
|
||||||
|
(swap! state assoc :loop? (boolean on?))
|
||||||
|
;; The audio thread reads `loop` every quantum, so a live node picks this up
|
||||||
|
;; without being restarted — the same as setting `loop` on a playing element.
|
||||||
|
(when-let [^js node (:node @state)]
|
||||||
|
(set! (.-loop node) (boolean on?))))
|
||||||
|
|
||||||
|
(-set-muted! [_ on?]
|
||||||
|
(set! (.-value (.-gain ^js (:out @state))) (if on? 0 1)))
|
||||||
|
|
||||||
|
(-release! [_]
|
||||||
|
;; The source node is wired to the output the whole app shares, so a backend
|
||||||
|
;; dropped while playing would go on being heard under the one that replaced
|
||||||
|
;; it. Nothing else here needs releasing: the context and its gain outlive
|
||||||
|
;; every backend by design.
|
||||||
|
(let [st @state]
|
||||||
|
(spin-down! st)
|
||||||
|
(swap! state assoc
|
||||||
|
:segs [{:from (.-currentTime ^js (:ctx st))
|
||||||
|
:anchor (position st) :rate 0}]
|
||||||
|
:node nil))))
|
||||||
|
|
||||||
|
(defn backend
|
||||||
|
"A clock on `buffer`, `seconds` long. A nil buffer is a silent clock of that
|
||||||
|
length — see the namespace docstring.
|
||||||
|
|
||||||
|
The context and output are injected by the four-argument form so the whole of
|
||||||
|
this is assertable against a fake in node, where there is no Web Audio."
|
||||||
|
([buffer seconds] (backend (context) (output) buffer seconds))
|
||||||
|
([^js ctx ^js out buffer seconds]
|
||||||
|
(->Graph (atom {:ctx ctx :out out :buffer buffer :seconds seconds
|
||||||
|
:rate 1.0 :loop? false :node nil
|
||||||
|
:segs [{:from (.-currentTime ctx) :anchor 0 :rate 0}]}))))
|
||||||
38
frontend/src/arthur/clock/transport.cljs
Normal file
38
frontend/src/arthur/clock/transport.cljs
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
(ns arthur.clock.transport
|
||||||
|
"What the clock needs of a thing that plays sound, and nothing more.
|
||||||
|
|
||||||
|
`arthur.clock` does the arithmetic — the frame derivation, the clamping, the
|
||||||
|
exposure grid — and a backend only has to answer where the sound has got to
|
||||||
|
and do as it is told. Keeping the protocol this thin is what makes the two
|
||||||
|
implementations comparable: if the graph backend and the element backend
|
||||||
|
disagree about a take's sync, the difference is in these nine methods and not
|
||||||
|
in anything derived from them.
|
||||||
|
|
||||||
|
POSITION IS WHERE THE SPEAKER IS, not where the renderer is. A backend that
|
||||||
|
schedules audio ahead of the output owes the difference back here — see
|
||||||
|
`arthur.clock.graph` — because the picture is drawn against this number and a
|
||||||
|
lip-sync tool that draws against the scheduler leads the sound it is matching.")
|
||||||
|
|
||||||
|
(defprotocol Transport
|
||||||
|
(-position [this]
|
||||||
|
"Seconds into the audio, as heard. Never negative, never past `-duration`,
|
||||||
|
and wrapped rather than clamped while looping.")
|
||||||
|
(-duration [this]
|
||||||
|
"Seconds of audio, or nil when the backend has not been told yet.")
|
||||||
|
(-playing? [this]
|
||||||
|
"Running AND not finished. A backend that has reached its end reports false
|
||||||
|
even if nothing told it to stop, because the end of the sound is the
|
||||||
|
authority on playback having stopped — see `ui/player`'s loop.")
|
||||||
|
(-rate [this])
|
||||||
|
(-set-rate! [this r])
|
||||||
|
(-play! [this])
|
||||||
|
(-pause! [this])
|
||||||
|
(-seek! [this seconds]
|
||||||
|
"Put the audio at `seconds`. Reading `-position` back must give the same
|
||||||
|
number, which is what makes a scrub idempotent.")
|
||||||
|
(-set-loop! [this on?])
|
||||||
|
(-set-muted! [this on?])
|
||||||
|
(-release! [this]
|
||||||
|
"Give up whatever this backend holds, because something else is about to be
|
||||||
|
the clock. NOT a pause: a backend that owns nothing has nothing to do here,
|
||||||
|
and the position it was last at is no longer anybody's business."))
|
||||||
|
|
@ -139,7 +139,8 @@
|
||||||
:status "built-in example · not a saved project"}))]
|
:status "built-in example · not a saved project"}))]
|
||||||
{:db db
|
{:db db
|
||||||
::pause! nil
|
::pause! nil
|
||||||
::seek! [(fps db) (frames db) 0]})))
|
::seek! [(fps db) (frames db) 0]
|
||||||
|
::clock! {:id id :sid (get-in db [:ui :open])}})))
|
||||||
|
|
||||||
;; ---------------------------------------------------------------------------
|
;; ---------------------------------------------------------------------------
|
||||||
;; tabs
|
;; tabs
|
||||||
|
|
@ -153,16 +154,39 @@
|
||||||
|
|
||||||
(rf/reg-fx
|
(rf/reg-fx
|
||||||
::clock!
|
::clock!
|
||||||
|
;; WHAT THE OPEN SYMBOL SOUNDS LIKE, fetched once per thing that can change it:
|
||||||
|
;; a document opening, a tab switch, an edit to a track. The graph backend is
|
||||||
|
;; handed the buffer; the element backend needs a URL, which means a WAV, which
|
||||||
|
;; is the whole of what this used to cost.
|
||||||
(fn [{:keys [id sid]}]
|
(fn [{:keys [id sid]}]
|
||||||
(let [{:keys [clip audio store]} (footage/entry id)]
|
(let [{:keys [clip audio store]} (footage/entry id)]
|
||||||
(-> (mix/clock! clip sid audio store)
|
(-> (if (clock/graph?)
|
||||||
|
(mix/clock-source! clip sid audio store)
|
||||||
|
(.then (mix/clock! clip sid audio store) (fn [u] {:url u})))
|
||||||
(.then #(rf/dispatch [::clock-ready id sid %]))
|
(.then #(rf/dispatch [::clock-ready id sid %]))
|
||||||
(.catch #(js/console.error %))))))
|
(.catch #(js/console.error %))))))
|
||||||
|
|
||||||
(rf/reg-event-fx
|
(rf/reg-event-fx
|
||||||
::clock-ready
|
::clock-ready
|
||||||
(fn [{:keys [db]} [_ id sid url]]
|
(fn [{:keys [db]} [_ id sid {:keys [url buffer seconds]}]]
|
||||||
(if (and (= id (:clip/current db)) (= sid (get-in db [:ui :open])))
|
(if-not (and (= id (:clip/current db)) (= sid (get-in db [:ui :open])))
|
||||||
|
{}
|
||||||
|
(if (clock/graph?)
|
||||||
|
(do
|
||||||
|
(clock/attach-buffer! buffer seconds)
|
||||||
|
;; RATE, LOOP AND MUTE ARE RE-APPLIED. They are transport state and
|
||||||
|
;; outlive the clock they were set on, but a fresh graph starts at its
|
||||||
|
;; defaults — so without this a tab switch would quietly drop the take
|
||||||
|
;; back to 1x, unlooped and unmuted. The element kept them because they
|
||||||
|
;; were properties of a node that survived its own `src` changing.
|
||||||
|
{:fx [[::rate! (get-in db [:playback :rate] 1.0)]
|
||||||
|
[::loop! (boolean (get-in db [:playback :loop?]))]
|
||||||
|
[::mute! (boolean (get-in db [:playback :muted?]))]
|
||||||
|
[::seek! [(fps db) (frames db) (get-in db [:playback :frame] 0)]]
|
||||||
|
;; An edit under a playing take resumes it. The element stopped
|
||||||
|
;; dead here, which made adjusting a fade while listening to it
|
||||||
|
;; a thing you could only do once.
|
||||||
|
(when (get-in db [:playback :playing?]) [::play! nil])]})
|
||||||
(do
|
(do
|
||||||
;; A blob URL made for the last tab is released when the next one lands,
|
;; A blob URL made for the last tab is released when the next one lands,
|
||||||
;; and never the document's own file.
|
;; and never the document's own file.
|
||||||
|
|
@ -171,8 +195,7 @@
|
||||||
(reset! clock-url (when (and (not= url (:audio (footage/entry id)))
|
(reset! clock-url (when (and (not= url (:audio (footage/entry id)))
|
||||||
(.startsWith url "blob:"))
|
(.startsWith url "blob:"))
|
||||||
url))
|
url))
|
||||||
{:db (assoc-in db [:clip :audio] url)})
|
{:db (assoc-in db [:clip :audio] url)})))))
|
||||||
{})))
|
|
||||||
|
|
||||||
(rf/reg-event-db
|
(rf/reg-event-db
|
||||||
::paint-failed
|
::paint-failed
|
||||||
|
|
|
||||||
|
|
@ -30,7 +30,6 @@
|
||||||
[arthur.domain.node :as node]
|
[arthur.domain.node :as node]
|
||||||
[arthur.domain.palette :as pal]
|
[arthur.domain.palette :as pal]
|
||||||
[arthur.events.edit :as edit]
|
[arthur.events.edit :as edit]
|
||||||
[arthur.audio.mix :as mix]
|
|
||||||
[arthur.demo.stage :as stage]
|
[arthur.demo.stage :as stage]
|
||||||
[arthur.domain.feature :as feature]
|
[arthur.domain.feature :as feature]
|
||||||
[arthur.domain.project :as project]
|
[arthur.domain.project :as project]
|
||||||
|
|
@ -124,9 +123,15 @@
|
||||||
;; write lands on what does not.
|
;; write lands on what does not.
|
||||||
:synced (project/tier1 (.-leaves clip-json))
|
:synced (project/tier1 (.-leaves clip-json))
|
||||||
:clip built :store (:store loaded)
|
:clip built :store (:store loaded)
|
||||||
|
;; The document's OWN file, unmixed. What
|
||||||
|
;; the symbol actually sounds like is the
|
||||||
|
;; clock's business and is fetched once,
|
||||||
|
;; by `::pb/clock!`, when it goes on
|
||||||
|
;; screen — this used to mix it here as
|
||||||
|
;; well, and the two encodes of the same
|
||||||
|
;; audio were most of a project open.
|
||||||
:audio "/static/arthur/audio.wav"})]
|
:audio "/static/arthur/audio.wav"})]
|
||||||
(-> (mix/mix! built (clip/opens-on built) (:audio entry) (:store entry))
|
entry))))))
|
||||||
(.then (fn [audio] (assoc entry :audio audio))))))))))
|
|
||||||
|
|
||||||
(defn- saved-clip!
|
(defn- saved-clip!
|
||||||
"Promise of clip `cid` of saved project `pid`, as `{:clip :store}`: its
|
"Promise of clip `cid` of saved project `pid`, as `{:clip :store}`: its
|
||||||
|
|
@ -411,11 +416,7 @@
|
||||||
entry (assoc entry :clip built :label (:name built)
|
entry (assoc entry :clip built :label (:name built)
|
||||||
:cid "stage-8625"
|
:cid "stage-8625"
|
||||||
:width (:width built) :height (:height built))]
|
:width (:width built) :height (:height built))]
|
||||||
(-> (mix/mix! built (clip/opens-on built) (:audio entry) (:store entry))
|
(rf/dispatch [::stage-opened (store/install! entry "stage")]))))
|
||||||
(.then (fn [audio]
|
|
||||||
(rf/dispatch
|
|
||||||
[::stage-opened
|
|
||||||
(store/install! (assoc entry :audio audio) "stage")])))))))
|
|
||||||
(.catch (fn [error]
|
(.catch (fn [error]
|
||||||
(js/console.error error)
|
(js/console.error error)
|
||||||
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))
|
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))
|
||||||
|
|
@ -557,12 +558,13 @@
|
||||||
;; events
|
;; events
|
||||||
|
|
||||||
(def blank-audio
|
(def blank-audio
|
||||||
"A new document still needs a clock.
|
"The audio a new document opens on, until it gets one of its own.
|
||||||
|
|
||||||
The frame is derived from an audio element and from nothing else — see
|
It is no longer load-bearing. The frame is derived from a position, and the
|
||||||
`arthur.clock` — so a stage with no sound has no time and `play` is a button
|
graph backend can hold one for a symbol with no sound at all — see
|
||||||
that cannot work. The synthetic take borrows this same asset for exactly this
|
`mix/clock-source!` — so a silent stage has time and `play` works. This stays
|
||||||
reason. When a document gets audio of its own, it replaces this."
|
because a new document borrowing the synthetic take's soundtrack is a
|
||||||
|
convenience worth keeping, not because the clock would stop without it."
|
||||||
"/static/arthur/audio.wav")
|
"/static/arthur/audio.wav")
|
||||||
|
|
||||||
(defn blank-entry
|
(defn blank-entry
|
||||||
|
|
@ -591,7 +593,8 @@
|
||||||
;; The readout goes home with the document; so must the clock, or play
|
;; The readout goes home with the document; so must the clock, or play
|
||||||
;; picks up wherever the last document's audio had got to.
|
;; picks up wherever the last document's audio had got to.
|
||||||
::pb/seek! [(:fps entry) (clip/output-frames (:clip entry) (clip/opens-on (:clip entry))) 0]
|
::pb/seek! [(:fps entry) (clip/output-frames (:clip entry) (clip/opens-on (:clip entry))) 0]
|
||||||
::pb/pause! nil})))
|
::pb/pause! nil
|
||||||
|
::pb/clock! {:id id :sid (clip/opens-on (:clip entry))}})))
|
||||||
|
|
||||||
(rf/reg-event-fx
|
(rf/reg-event-fx
|
||||||
::save
|
::save
|
||||||
|
|
@ -848,7 +851,8 @@
|
||||||
(assoc :project {:id nil :cid nil :name nil :seq nil
|
(assoc :project {:id nil :cid nil :name nil :seq nil
|
||||||
:busy? false :status "loaded 8625 stage study"}))]
|
:busy? false :status "loaded 8625 stage study"}))]
|
||||||
{:db db
|
{:db db
|
||||||
::pb/seek! [(get-in db [:clip :fps]) (pb/frames db) 0]})))
|
::pb/seek! [(get-in db [:clip :fps]) (pb/frames db) 0]
|
||||||
|
::pb/clock! {:id clip-id :sid (get-in db [:ui :open])}})))
|
||||||
|
|
||||||
(rf/reg-event-fx
|
(rf/reg-event-fx
|
||||||
::saved
|
::saved
|
||||||
|
|
@ -898,7 +902,9 @@
|
||||||
:status (str "opened " name " r" seq)}))
|
:status (str "opened " name " r" seq)}))
|
||||||
::pb/pause! nil
|
::pb/pause! nil
|
||||||
::pb/seek! (let [{c :clip fps :fps} (store/entry clip-id)]
|
::pb/seek! (let [{c :clip fps :fps} (store/entry clip-id)]
|
||||||
[fps (clip/output-frames c (clip/opens-on c)) 0])}))
|
[fps (clip/output-frames c (clip/opens-on c)) 0])
|
||||||
|
::pb/clock! {:id clip-id
|
||||||
|
:sid (clip/opens-on (:clip (store/entry clip-id)))}}))
|
||||||
|
|
||||||
(rf/reg-event-fx
|
(rf/reg-event-fx
|
||||||
::failed
|
::failed
|
||||||
|
|
|
||||||
|
|
@ -72,8 +72,14 @@
|
||||||
(fn [[id _ sid] _]
|
(fn [[id _ sid] _]
|
||||||
;; What the open symbol plays, off the document as saved rather than `::clip`,
|
;; What the open symbol plays, off the document as saved rather than `::clip`,
|
||||||
;; so a bar being slid does not re-mix on every frame of the drag.
|
;; so a bar being slid does not re-mix on every frame of the drag.
|
||||||
|
;;
|
||||||
|
;; THE DOCUMENT IS PART OF THE IDENTITY, not just the symbol. `ui/shell`
|
||||||
|
;; re-mixes when this changes under the SAME symbol, and two documents both
|
||||||
|
;; opening on `:main` would otherwise look like one symbol whose tracks had
|
||||||
|
;; been edited — so opening a project re-mixed it a second time, on top of
|
||||||
|
;; the mix the open itself had already asked for.
|
||||||
(let [c (:clip (footage/entry id))]
|
(let [c (:clip (footage/entry id))]
|
||||||
[sid (when (clip/symbol c sid) (nest/audio-tracks c sid))])))
|
[id sid (when (clip/symbol c sid) (nest/audio-tracks c sid))])))
|
||||||
|
|
||||||
(rf/reg-sub
|
(rf/reg-sub
|
||||||
::symbol
|
::symbol
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
(ns arthur.ui.shell
|
(ns arthur.ui.shell
|
||||||
"The window: one grid, five panes, a location bar, and the audio element.
|
"The window: one grid, five panes, a location bar, and the clock's audio
|
||||||
|
element on the backend that needs one.
|
||||||
|
|
||||||
Nothing else. Each pane owns its own subscriptions, so this component re-renders
|
Nothing else. Each pane owns its own subscriptions, so this component re-renders
|
||||||
only when the grid itself would change — which is a pane being resized, shut or
|
only when the grid itself would change — which is a pane being resized, shut or
|
||||||
|
|
@ -29,11 +30,24 @@
|
||||||
;; only a change under the same one counts.
|
;; only a change under the same one counts.
|
||||||
(r/with-let [heard (atom nil)
|
(r/with-let [heard (atom nil)
|
||||||
remix (r/track! (fn []
|
remix (r/track! (fn []
|
||||||
(let [[sid :as now] @(rf/subscribe [::render/sounds])
|
(let [[id sid :as now] @(rf/subscribe [::render/sounds])
|
||||||
[was :as before] @heard]
|
[was-id was-sid :as before] @heard]
|
||||||
(reset! heard now)
|
(reset! heard now)
|
||||||
(when (and before (= sid was) (not= now before))
|
;; The same symbol OF THE SAME DOCUMENT. A
|
||||||
|
;; different document, or a different symbol, has
|
||||||
|
;; already asked for its own clock — see
|
||||||
|
;; `::pb/clock!`'s callers — and re-mixing here as
|
||||||
|
;; well is a second render of minutes of audio.
|
||||||
|
(when (and before
|
||||||
|
(= [id sid] [was-id was-sid])
|
||||||
|
(not= now before))
|
||||||
(rf/dispatch [::pb/refresh-clock])))))]
|
(rf/dispatch [::pb/refresh-clock])))))]
|
||||||
|
;; NOTHING TO RENDER under the graph backend: it plays the mixdown buffer
|
||||||
|
;; directly and takes its position from the audio context, so there is no
|
||||||
|
;; element and no WAV for one to hold. The `remix` track above still runs
|
||||||
|
;; either way — it is what notices an edit to a track, and `with-let` builds
|
||||||
|
;; it whatever this body returns.
|
||||||
|
(when-not (clock/graph?)
|
||||||
[:audio
|
[:audio
|
||||||
{:ref #(when % (clock/attach! %))
|
{:ref #(when % (clock/attach! %))
|
||||||
:src @(rf/subscribe [::playback/audio])
|
:src @(rf/subscribe [::playback/audio])
|
||||||
|
|
@ -43,7 +57,7 @@
|
||||||
;; OS media keys, a browser autoplay block — has to be able to correct the
|
;; OS media keys, a browser autoplay block — has to be able to correct the
|
||||||
;; document rather than be contradicted by it.
|
;; document rather than be contradicted by it.
|
||||||
:on-play #(rf/dispatch [::pb/play])
|
:on-play #(rf/dispatch [::pb/play])
|
||||||
:on-pause #(rf/dispatch [::pb/pause])}]
|
:on-pause #(rf/dispatch [::pb/pause])}])
|
||||||
(finally (r/dispose! remix))))
|
(finally (r/dispose! remix))))
|
||||||
|
|
||||||
(defn view []
|
(defn view []
|
||||||
|
|
|
||||||
254
frontend/test/arthur/clock/graph_test.cljs
Normal file
254
frontend/test/arthur/clock/graph_test.cljs
Normal file
|
|
@ -0,0 +1,254 @@
|
||||||
|
(ns arthur.clock.graph-test
|
||||||
|
"The graph clock, asserted against a hand-moved context clock.
|
||||||
|
|
||||||
|
The context's `currentTime` is set by these tests rather than advanced by a
|
||||||
|
device, which is exactly the coupling worth asserting: if any of this counted
|
||||||
|
time instead of deriving it, moving the clock by hand would not move the
|
||||||
|
position. A drift bug is invisible for the first second and unmistakable by
|
||||||
|
the tenth, which is the worst possible shape for a bug to have."
|
||||||
|
(:require [cljs.test :refer [deftest is testing]]
|
||||||
|
[arthur.clock.element :as element]
|
||||||
|
[arthur.clock.graph :as graph]
|
||||||
|
[arthur.clock.transport :as t]))
|
||||||
|
|
||||||
|
(defn- fake-node [log]
|
||||||
|
(let [n #js {:loop false :playbackRate #js {:value 1.0}}]
|
||||||
|
(set! (.-connect n) (fn [_] nil))
|
||||||
|
(set! (.-disconnect n) (fn [] nil))
|
||||||
|
(set! (.-start n) (fn [_when offset] (swap! log conj [:start offset]) nil))
|
||||||
|
(set! (.-stop n) (fn [] (swap! log conj [:stop]) nil))
|
||||||
|
n))
|
||||||
|
|
||||||
|
(defn- fake-ctx
|
||||||
|
"Stands in for an `AudioContext`. `currentTime` is a plain writable property
|
||||||
|
so a test can move it; the rest is the three methods the backend calls."
|
||||||
|
[latency log]
|
||||||
|
(let [ctx #js {:currentTime 0.0 :outputLatency latency :baseLatency 0}]
|
||||||
|
(set! (.-resume ctx) (fn [] nil))
|
||||||
|
(set! (.-createBufferSource ctx) (fn [] (fake-node log)))
|
||||||
|
ctx))
|
||||||
|
|
||||||
|
(defn- rig
|
||||||
|
"A backend, its context, and the log of what it did to its source nodes."
|
||||||
|
[& {:keys [seconds latency buffer?] :or {seconds 10.0 latency 0 buffer? true}}]
|
||||||
|
(let [log (atom [])
|
||||||
|
ctx (fake-ctx latency log)
|
||||||
|
out #js {:gain #js {:value 1}}]
|
||||||
|
{:ctx ctx :log log :out out
|
||||||
|
:g (graph/backend ctx out (when buffer? #js {}) seconds)}))
|
||||||
|
|
||||||
|
(defn- at! [{:keys [^js ctx]} t] (set! (.-currentTime ctx) t))
|
||||||
|
|
||||||
|
(defn- close?
|
||||||
|
"Within a frame's worth of a microsecond. Positions compound a multiply and a
|
||||||
|
subtract, so the last bit of a double is not something to assert on."
|
||||||
|
[a b]
|
||||||
|
(< (js/Math.abs (- a b)) 1e-9))
|
||||||
|
|
||||||
|
(deftest the-position-is-derived-from-the-context-clock-not-counted
|
||||||
|
;; Nothing here is ticked. The context moves and the position follows, which
|
||||||
|
;; is the whole property: a stalled paint loop cannot lose the clock, because
|
||||||
|
;; the clock was never in the paint loop.
|
||||||
|
(let [{:keys [g] :as r} (rig)]
|
||||||
|
(t/-play! g)
|
||||||
|
(doseq [[now want] [[0.0 0.0] [0.5 0.5] [2.0 2.0] [9.25 9.25]]]
|
||||||
|
(at! r now)
|
||||||
|
(is (= want (t/-position g)) (str "context at " now "s")))))
|
||||||
|
|
||||||
|
(deftest a-stall-lands-where-the-audio-already-is
|
||||||
|
;; THE property the derivation buys, same as the element's. A loop that
|
||||||
|
;; stalled for a third of a second resumes at the position the audio reached,
|
||||||
|
;; not a third of a second behind it.
|
||||||
|
(let [{:keys [g] :as r} (rig)]
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 1.0)
|
||||||
|
(is (= 1.0 (t/-position g)))
|
||||||
|
(at! r 1.5) ; fifteen frames' worth of stall
|
||||||
|
(is (= 1.5 (t/-position g)) "and nothing was counted in between")))
|
||||||
|
|
||||||
|
(deftest the-position-is-the-speaker-not-the-renderer
|
||||||
|
;; `currentTime` is the quantum being RENDERED; the speaker is one output
|
||||||
|
;; latency behind it. Report the renderer and the picture leads the sound by
|
||||||
|
;; that much, which in a lip-sync tool is the only artefact that matters.
|
||||||
|
(let [{:keys [g] :as r} (rig :latency 0.05)]
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 2.0)
|
||||||
|
(is (= 1.95 (t/-position g)) "50ms of output latency, subtracted")
|
||||||
|
(testing "and scaled by the rate, because a latency is wall time"
|
||||||
|
(t/-set-rate! g 4.0)
|
||||||
|
(at! r 3.0)
|
||||||
|
;; Banked at 1.95 + 0.05 = 2.0, then four seconds of buffer per second of
|
||||||
|
;; wall clock, less the 4 x 50ms still in the output buffer.
|
||||||
|
(is (close? 5.8 (t/-position g))))))
|
||||||
|
|
||||||
|
(deftest a-rate-change-does-not-lurch-the-playhead
|
||||||
|
;; THE bug the segments exist for. Audio already rendered cannot be re-rated:
|
||||||
|
;; read the samples still travelling to the speaker at the NEW rate and the
|
||||||
|
;; position jumps backwards by three output latencies the instant 1x becomes
|
||||||
|
;; 4x — fourteen frames at 30fps with a laptop's 150ms, which is a visible
|
||||||
|
;; lurch on every press of the rate button.
|
||||||
|
(let [{:keys [g] :as r} (rig :seconds 100.0 :latency 0.15)]
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 2.0)
|
||||||
|
(let [before (t/-position g)]
|
||||||
|
(t/-set-rate! g 4.0)
|
||||||
|
(is (close? before (t/-position g))
|
||||||
|
"the same instant, read either side of the change")
|
||||||
|
(testing "and it goes FORWARD from there, at the new rate"
|
||||||
|
(at! r 2.5)
|
||||||
|
(is (> (t/-position g) before))
|
||||||
|
(at! r 3.0)
|
||||||
|
;; At 3.0s of context the speaker is hearing what was rendered at
|
||||||
|
;; 2.85s. The rate changed when the renderer was at buffer position
|
||||||
|
;; 2.0, which is 0.85s of wall clock earlier — so 4 x 0.85 of buffer
|
||||||
|
;; since, and nothing before that boundary re-rated.
|
||||||
|
(is (close? (+ 2.0 (* 4 0.85)) (t/-position g)))))
|
||||||
|
(testing "and back to 1x is continuous too"
|
||||||
|
(let [before (t/-position g)]
|
||||||
|
(t/-set-rate! g 1.0)
|
||||||
|
(is (close? before (t/-position g)))))))
|
||||||
|
|
||||||
|
(deftest segments-do-not-pile-up
|
||||||
|
;; One per rate change, pruned to the in-flight window. Unbounded history
|
||||||
|
;; would grow for the life of the document.
|
||||||
|
(let [{:keys [g] :as r} (rig :seconds 1000.0 :latency 0.05)]
|
||||||
|
(t/-play! g)
|
||||||
|
(dotimes [i 200]
|
||||||
|
(at! r (+ 1.0 i))
|
||||||
|
(t/-set-rate! g (if (even? i) 4.0 1.0)))
|
||||||
|
(at! r 500.0)
|
||||||
|
(is (number? (t/-position g)))
|
||||||
|
(is (<= (count (:segs @(.-state g))) 3)
|
||||||
|
"the one in force, plus whatever is still in flight")))
|
||||||
|
|
||||||
|
(deftest the-first-samples-are-not-the-last-ones
|
||||||
|
;; Right after a looped play the audible position is a few milliseconds
|
||||||
|
;; NEGATIVE — the first samples are still in the output buffer. Wrapping
|
||||||
|
;; before clamping would read that as the very end of the take and jump the
|
||||||
|
;; playhead there on every play.
|
||||||
|
(let [{:keys [g] :as r} (rig :seconds 2.0 :latency 0.05)]
|
||||||
|
(t/-set-loop! g true)
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 0.0)
|
||||||
|
(is (= 0 (t/-position g)))))
|
||||||
|
|
||||||
|
(deftest a-rate-change-is-banked-so-nothing-accumulates
|
||||||
|
;; 1x -> 4x -> 1x across a take. The anchor moves to where the graph had got
|
||||||
|
;; to and the new rate applies from there, so switching rates mid-take cannot
|
||||||
|
;; leave an offset behind — which is what a multiplier on a counted frame
|
||||||
|
;; would do.
|
||||||
|
(let [{:keys [g] :as r} (rig :seconds 100.0)]
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 2.0)
|
||||||
|
(is (= 2.0 (t/-position g)))
|
||||||
|
(t/-set-rate! g 4.0)
|
||||||
|
(at! r 3.0)
|
||||||
|
(is (= 6.0 (t/-position g)) "one second of wall clock, four of buffer")
|
||||||
|
(t/-set-rate! g 1.0)
|
||||||
|
(at! r 4.0)
|
||||||
|
(is (= 7.0 (t/-position g)) "and back to one for one, from six")
|
||||||
|
(is (= 1.0 (t/-rate g)))))
|
||||||
|
|
||||||
|
(deftest a-seek-round-trips
|
||||||
|
;; Reading back exactly what was seeked to is what makes a scrub idempotent.
|
||||||
|
(let [{:keys [g]} (rig)]
|
||||||
|
(doseq [t [0.0 0.5 3.25 9.0]]
|
||||||
|
(t/-seek! g t)
|
||||||
|
(is (= t (t/-position g)) (str "seek to " t "s")))))
|
||||||
|
|
||||||
|
(deftest a-seek-while-playing-restarts-the-source-at-the-new-offset
|
||||||
|
;; Source nodes are single-use, so a seek under a playing take has to stop the
|
||||||
|
;; old one and start a new one at the offset. Checked through the log because
|
||||||
|
;; an offset that silently stayed at zero would still read back correctly from
|
||||||
|
;; the arithmetic while playing the wrong audio.
|
||||||
|
(let [{:keys [g log] :as r} (rig)]
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 1.0)
|
||||||
|
(reset! log [])
|
||||||
|
(t/-seek! g 4.0)
|
||||||
|
(is (= [[:stop] [:start 4.0]] @log))
|
||||||
|
(is (= 4.0 (t/-position g)))))
|
||||||
|
|
||||||
|
(deftest looping-wraps-rather-than-stopping
|
||||||
|
(let [{:keys [g] :as r} (rig :seconds 2.0)]
|
||||||
|
(t/-set-loop! g true)
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 5.0)
|
||||||
|
(is (= 1.0 (t/-position g)) "five seconds into a two-second loop")
|
||||||
|
(is (t/-playing? g) "and a loop never ends")))
|
||||||
|
|
||||||
|
(deftest running-off-the-end-stops-being-playing
|
||||||
|
;; The end of the sound is the authority on playback having stopped — nothing
|
||||||
|
;; counts frames to notice it. `ui/player`'s loop reads this to correct the
|
||||||
|
;; document.
|
||||||
|
(let [{:keys [g] :as r} (rig :seconds 2.0)]
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 1.9)
|
||||||
|
(is (t/-playing? g))
|
||||||
|
(at! r 2.5)
|
||||||
|
(is (not (t/-playing? g)))
|
||||||
|
(is (= 2.0 (t/-position g)) "and the position stops at the end, not past it")
|
||||||
|
(testing "and playing again starts over, as the element does"
|
||||||
|
(t/-play! g)
|
||||||
|
(is (= 0.0 (t/-position g))))))
|
||||||
|
|
||||||
|
(deftest a-symbol-with-no-sound-still-keeps-time
|
||||||
|
;; NO BUFFER AT ALL. The element backend had to synthesize silence and encode
|
||||||
|
;; it to a WAV, full length, so that it had a duration to report; here the
|
||||||
|
;; length is simply told to the clock and the context does the rest.
|
||||||
|
(let [{:keys [g log] :as r} (rig :buffer? false :seconds 3.0)]
|
||||||
|
(t/-play! g)
|
||||||
|
(is (= [] @log) "and no source node was made for silence")
|
||||||
|
(at! r 1.0)
|
||||||
|
(is (= 1.0 (t/-position g)))
|
||||||
|
(is (t/-playing? g))
|
||||||
|
(is (= 3.0 (t/-duration g)))
|
||||||
|
(at! r 3.5)
|
||||||
|
(is (not (t/-playing? g)) "and silence ends when the symbol does")))
|
||||||
|
|
||||||
|
(deftest pause-anchors-at-what-was-heard
|
||||||
|
;; Resuming replays the last few milliseconds rather than skipping them, which
|
||||||
|
;; is the kinder of the two roundings.
|
||||||
|
(let [{:keys [g] :as r} (rig :latency 0.05)]
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 2.0)
|
||||||
|
(t/-pause! g)
|
||||||
|
(is (= 1.95 (t/-position g)))
|
||||||
|
(is (not (t/-playing? g)))
|
||||||
|
(testing "and the position holds while paused, however the context moves"
|
||||||
|
(at! r 8.0)
|
||||||
|
(is (= 1.95 (t/-position g))))))
|
||||||
|
|
||||||
|
(deftest releasing-stops-the-source
|
||||||
|
;; A backend dropped while playing would go on being heard under the one that
|
||||||
|
;; replaced it: the source is wired to the output the whole app shares.
|
||||||
|
(let [{:keys [g log] :as r} (rig)]
|
||||||
|
(t/-play! g)
|
||||||
|
(at! r 1.0)
|
||||||
|
(reset! log [])
|
||||||
|
(t/-release! g)
|
||||||
|
(is (= [[:stop]] @log))
|
||||||
|
(is (not (t/-playing? g)))))
|
||||||
|
|
||||||
|
(deftest mute-is-the-shared-output-and-not-the-buffer
|
||||||
|
(let [{:keys [g ^js out]} (rig)]
|
||||||
|
(t/-set-muted! g true)
|
||||||
|
(is (= 0 (.-value (.-gain out))))
|
||||||
|
(t/-set-muted! g false)
|
||||||
|
(is (= 1 (.-value (.-gain out))))))
|
||||||
|
|
||||||
|
(deftest the-two-backends-report-the-same-position
|
||||||
|
;; The point of keeping both switchable. Everything BELOW the position — the
|
||||||
|
;; frame derivation, the clamp, the exposure grid — is in `arthur.clock` and
|
||||||
|
;; shared, so the two backends can only disagree about the position itself. At
|
||||||
|
;; zero output latency they must not; `clock-test` asserts what is built on
|
||||||
|
;; top of it, once, for both.
|
||||||
|
(let [{:keys [g] :as r} (rig :seconds 7.633)]
|
||||||
|
(t/-play! g)
|
||||||
|
(doseq [now [0.0 0.034 1.0 1.999 2.0 7.0]]
|
||||||
|
(at! r now)
|
||||||
|
(let [e (element/backend #js {:currentTime now :paused false :ended false
|
||||||
|
:playbackRate 1.0 :duration 7.633})]
|
||||||
|
(is (= (t/-position e) (t/-position g)) (str "at " now "s"))
|
||||||
|
(is (= (t/-duration e) (t/-duration g)))
|
||||||
|
(is (= (t/-playing? e) (t/-playing? g)))))))
|
||||||
Loading…
Add table
Add a link
Reference in a new issue