Play the audio mixdown instead of encoding a WAV of it

Opening the 8625 study froze the main thread for 4.7 seconds and settled at
615MB of heap. A profile put three quarters of a project open inside
mix/wav-bytes, which playback had no business calling at all.

Two separate causes. The peak scan built a lazy sequence of one boxed double
per SAMPLE -- ten million of them for a seven-minute mix -- to compute a
single maximum over data already sitting in Float32Arrays; the hand-written
loop is 73x faster and agrees to the bit. The rest was structural: the WAV
existed only because an <audio> element can hold a URL and nothing else, and
the element existed only to be the clock. So a mixdown that was already
rendered got encoded to 73MB of 16-bit PCM, on the main thread, on open, on
every tab switch and on every edit to a track -- and a symbol with no sound
got silence synthesized and encoded full length so the element had a duration
to report.

arthur.clock keeps its interface and all of its arithmetic; the position now
comes from a backend behind a protocol. clock.graph plays the AudioBuffer
through an AudioBufferSourceNode and derives the frame from the context's own
clock, which is the audio device's position in double precision rather than
whatever the media pipeline last published. clock.element is the old path,
kept switchable while the new one earns trust -- BACKEND, or use-backend! --
which is also why every one of the original clock tests passes unchanged: the
derivation they assert is shared, and the backends can only disagree about the
position under it. 6.5s to 1.8s, 4.7s of blocking to 370ms, 615MB to 68MB.

THE POSITION IS COMPENSATED FOR OUTPUT LATENCY, and piecewise because of it.
currentTime is the quantum being rendered, which the speaker is tens of
milliseconds behind; report the renderer and the picture leads the sound,
which in a lip-sync tool is the only artefact that matters. Audio already
rendered cannot be re-rated, though, so reading it back at a new rate jumped
the playhead backwards by three latencies on every press of the rate button.
Each play, pause, seek and rate change now records a segment and a position is
read against whichever segment was in force when that audio was rendered.

One duplicate fell out of this. Opening a project asked for its clock twice --
once from ::opened and once from a ::refresh-clock the shell raised because it
compared symbol ids, and two different documents both open on :main. The
sounds subscription carries the clip id now, so "an edit under the same
symbol" means what it says.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Your Name 2026-10-03 04:47:35 -04:00
parent 90b1fbe2f8
commit 5b5b9ae4c3
10 changed files with 846 additions and 98 deletions

View file

@ -2,14 +2,19 @@
"Render independently placed audio tracks into one stage audio clock. "Render independently placed audio tracks into one stage audio clock.
The mix is derived from saved audio track leaves and immutable footage blobs. The mix is derived from saved audio track leaves and immutable footage blobs.
The transport still has one audio element, so seeking, rate changes and looping The transport still has ONE clock, so seeking, rate changes and looping stay
stay tied to the same clock the picture reads. tied to the same position the picture is drawn from.
THE AUDIO BUFFER IS THE PRODUCT AND THE WAV IS ONE PACKAGING OF IT. Playback THE AUDIO BUFFER IS THE PRODUCT AND THE WAV IS ONE PACKAGING OF IT. `buffer!`
wants a URL an `<audio>` element can hold; an export wants the samples, either renders and the wrappers below it package, rather than the render being spelled
as WAV bytes to put in an archive or as the `AudioBuffer` a muxer takes as an once per consumer.
audio track. So `buffer!` renders and the two wrappers below it package, rather
than the render being spelled once per consumer." PLAYBACK NO LONGER PACKAGES AT ALL. It takes the buffer as it is — see
`clock-source!` and `arthur.clock.graph` — because encoding a WAV so an
`<audio>` element had a URL to hold cost O(the clip's length) on the main
thread, on open, on every tab switch and on every edit to a track. The WAV is
now what an EXPORT wants: bytes for an archive, or the buffer itself for a
muxer. `clock!` below is the element backend's packaging and goes when it does."
(:require [arthur.domain.channel :as ch] (:require [arthur.domain.channel :as ch]
[arthur.domain.clip :as clip] [arthur.domain.clip :as clip]
[arthur.domain.nest :as nest] [arthur.domain.nest :as nest]
@ -29,9 +34,22 @@
bytes (js/ArrayBuffer. (+ 44 (* frames channels 2))) bytes (js/ArrayBuffer. (+ 44 (* frames channels 2)))
view (js/DataView. bytes) view (js/DataView. bytes)
samples (mapv #(.getChannelData buffer %) (range channels)) samples (mapv #(.getChannelData buffer %) (range channels))
peak (reduce max 0 ;; A HAND-WRITTEN LOOP over the typed arrays, not `(reduce max (for ...))`.
(for [channel samples i (range frames)] ;; The lazy sequence that read beautifully allocated one boxed double per
(js/Math.abs (aget channel i)))) ;; SAMPLE — ten million of them for a three-minute mix — and spent the
;; whole of a sixteen-second project open walking them and collecting
;; them. Same arithmetic, no allocation.
peak (loop [c 0 p 0]
(if (< c channels)
(recur (inc c)
(let [^js data (nth samples c)]
(loop [i 0 p p]
(if (< i frames)
(recur (inc i)
(let [a (js/Math.abs (aget data i))]
(if (> a p) a p)))
p))))
p))
level (if (> peak 0.98) (/ 0.98 peak) 1)] level (if (> peak 0.98) (/ 0.98 peak) 1)]
(doseq [[offset word] [[0 "RIFF"] [8 "WAVE"] [12 "fmt "] [36 "data"]]] (doseq [[offset word] [[0 "RIFF"] [8 "WAVE"] [12 "fmt "] [36 "data"]]]
(dotimes [i 4] (.setUint8 view (+ offset i) (.charCodeAt word i)))) (dotimes [i 4] (.setUint8 view (+ offset i) (.charCodeAt word i))))
@ -44,11 +62,14 @@
(.setUint16 view 32 (* channels 2) true) (.setUint16 view 32 (* channels 2) true)
(.setUint16 view 34 16 true) (.setUint16 view 34 16 true)
(.setUint32 view 40 (* frames channels 2) true) (.setUint32 view 40 (* frames channels 2) true)
(dotimes [i frames] ;; Channel-outer so the channel's array is looked up once rather than once
;; per frame. The byte offsets are unchanged, so the interleaving is too.
(dotimes [c channels] (dotimes [c channels]
(let [sample (* level (aget (get samples c) i))] (let [^js data (nth samples c)]
(dotimes [i frames]
(let [sample (* level (aget data i))]
(.setInt16 view (+ 44 (* (+ (* i channels) c) 2)) (.setInt16 view (+ 44 (* (+ (* i channels) c) 2))
(js/Math.round (* 32767 (max -1 (min 1 sample)))) true)))) (js/Math.round (* 32767 (max -1 (min 1 sample)))) true)))))
(js/Uint8Array. bytes))) (js/Uint8Array. bytes)))
(defn- wav-url [^js buffer] (defn- wav-url [^js buffer]
@ -171,17 +192,43 @@
(.arrayBuffer response))) (.arrayBuffer response)))
(.then decode-bytes!))) (.then decode-bytes!)))
(defn mix! (defn clock-source!
"Promise of a mixed WAV URL for symbol `sid`, or the original URL when it has "Promise of `{:buffer :seconds}` — the audio the transport runs its clock on
no audio tracks. Each track can be trimmed and faded independently of its while symbol `sid` is open, and how long that clock is.
linked picture."
Same order of preference as the WAV packaging in `clock!` below: the symbol's
own placed tracks, mixed; the document's audio file, for the symbol the
document opens on and only that one; and otherwise NO BUFFER AT ALL and the
symbol's own length.
THE SILENT CASE IS WHY THE DURATION IS RETURNED BESIDE THE BUFFER rather than
read off it. A symbol with no sound still needs a clock exactly as long as it
is, and `clock!` had to synthesize that silence and then ENCODE it, full
length, so an element had a duration to report. There is nothing to decode for
a symbol with no sound: saying how long it is answers the only question the
silence was ever asked."
[document sid fallback-url store] [document sid fallback-url store]
(-> (buffer! document sid store) (-> (buffer! document sid store)
(.then (fn [buffer] (if buffer (wav-url buffer) fallback-url))))) (.then (fn [^js buffer]
(cond
buffer
{:buffer buffer :seconds (.-duration buffer)}
(and fallback-url (= sid (clip/opens-on document)))
(-> (decode! fallback-url)
(.then (fn [^js b] {:buffer b :seconds (.-duration b)})))
:else
{:buffer nil
:seconds (/ (clip/output-frames document sid) (:fps document))})))))
(defn clock! (defn clock!
"Promise of the URL the transport should play while symbol `sid` is open. "Promise of the URL the transport should play while symbol `sid` is open.
THE ELEMENT BACKEND'S PACKAGING of `clock-source!`, kept while that backend is
— see `arthur.clock`. Every cost this namespace had on open is in the two
`wav-url` calls below.
The frame is derived from the audio element and from nothing else, so every The frame is derived from the audio element and from nothing else, so every
open symbol needs a sound exactly as long as it is. In order: its own placed open symbol needs a sound exactly as long as it is. In order: its own placed
tracks, mixed; the document's audio file, for the symbol the document opens on tracks, mixed; the document's audio file, for the symbol the document opens on

View file

@ -3,7 +3,7 @@
THE FRAME IS DERIVED FROM THE AUDIO, never counted: THE FRAME IS DERIVED FROM THE AUDIO, never counted:
frame = ⌊currentTime · fps⌋ frame = ⌊position · fps⌋
A loop that counted frames and hoped to keep up would drift, and drift against A loop that counted frames and hoped to keep up would drift, and drift against
a voice is the one artefact that cannot be fixed downstream — a lip-sync tool a voice is the one artefact that cannot be fixed downstream — a lip-sync tool
@ -12,26 +12,79 @@
The failure mode becomes a visible stutter rather than an invisible slide, and The failure mode becomes a visible stutter rather than an invisible slide, and
those are very different bugs to own. those are very different bugs to own.
½× and ¼× are `playbackRate` and nothing else. The audio slows, `currentTime` ½× and ¼× are the backend's playback rate and nothing else. The audio slows,
advances proportionally, and the derived frame follows — so slow motion cannot the position advances proportionally, and the derived frame follows — so slow
desync by construction. Implementing rate as a multiplier on a counted frame motion cannot desync by construction. Implementing rate as a multiplier on a
would give the picture a rate and the sound another. counted frame would give the picture a rate and the sound another.
It is outside app-db because the audio element is the source of truth and It is outside app-db because the audio is the source of truth and copying it
copying it into the db every frame would make the db a lagging mirror of into the db every frame would make the db a lagging mirror of something
something authoritative elsewhere. What DOES belong in the db is the playhead authoritative elsewhere. What DOES belong in the db is the playhead as a piece
as a piece of document state — see events/playback — and that is written from of document state — see events/playback — and that is written from here, not
here, not read by here." read by here.
(:require [arthur.domain.node :as node]))
(defonce ^:private el (atom nil)) TWO BACKENDS, ONE ARITHMETIC. `position` comes from either a Web Audio graph
(`clock.graph`, the default) or an `<audio>` element (`clock.element`, kept
switchable while the first earns trust). Everything below the position — the
derivation, the clamp, the exposure grid — is here and is the same either way,
so the two can be compared on the same take rather than swapped on faith.
Build with `:closure-defines {arthur.clock/BACKEND \"element\"}`, or call
`use-backend!` from the console, to put the element back."
(:require [arthur.clock.element :as element]
[arthur.clock.graph :as graph]
[arthur.clock.transport :as t]
[arthur.domain.node :as node]))
(goog-define ^String BACKEND "graph")
(defonce ^:private mode (atom (keyword BACKEND)))
;; `{:source x :backend b}`. The source is kept so that re-attaching the same
;; thing can be recognised as the no-op it is — see `install!`.
(defonce ^:private current (atom nil))
(defn graph?
"Whether the app should wire up the graph backend. Read by `ui/shell`, which
renders the audio element only when this is false, and by `events/playback`,
which fetches a buffer rather than a WAV URL when it is true."
[]
(and (= :graph @mode) (graph/available?)))
(defn use-backend!
"Switch backends. Takes effect on the next thing that attaches one, which is
the next tab switch or document open — nothing is torn down under a take
that is already playing."
[m]
(reset! mode m))
(defn- install!
"Put a backend on `source`, unless `source` is already the clock's.
IDEMPOTENCE IS LOAD-BEARING HERE. The element's `:ref` is an inline closure,
so React hands it the same node again on every re-render of the shell —
rebuilding the clock there would mean a pane being dragged released whatever
was playing. The source is the identity: the element, or the buffer and its
length."
[source make]
(let [{:keys [backend] prev :source} @current]
(when-not (and backend (= prev source))
(when backend (t/-release! backend))
(reset! current {:source source
:backend (when (some? source) (make))}))))
(defn attach! (defn attach!
"Hand the clock its audio element. Idempotent." "Hand the clock an audio element. Idempotent. Element backend only — the
`:ref` that calls this is on a node `ui/shell` renders only in that mode."
[audio-el] [audio-el]
(reset! el audio-el)) (install! audio-el #(element/backend audio-el)))
(defn element [] @el) (defn attach-buffer!
"Hand the clock `seconds` of audio to run on, as an `AudioBuffer` or as nil
for silence of that length. Graph backend only."
[buffer seconds]
;; Keyed on the length as well as the buffer, because two silent clocks of
;; different lengths are two different clocks and both have a nil buffer.
(install! [buffer seconds] #(graph/backend buffer seconds)))
(defn- clamp [f frames] (defn- clamp [f frames]
(-> f (max 0) (min (dec frames)))) (-> f (max 0) (min (dec frames))))
@ -39,56 +92,52 @@
(defn frame (defn frame
"The clip frame the audio is currently on." "The clip frame the audio is currently on."
[fps frames] [fps frames]
(if-let [a @el] (if-let [b (:backend @current)]
(clamp (js/Math.floor (* (.-currentTime a) fps)) frames) (clamp (js/Math.floor (* (t/-position b) fps)) frames)
0)) 0))
(defn playing? [] (defn playing? []
(boolean (when-let [a @el] (and (not (.-paused a)) (not (.-ended a)))))) (boolean (when-let [b (:backend @current)] (t/-playing? b))))
(defn rate [] (defn rate []
(if-let [a @el] (.-playbackRate a) 1.0)) (if-let [b (:backend @current)] (t/-rate b) 1.0))
(defn set-rate! [r] (defn set-rate! [r]
(when-let [a @el] (set! (.-playbackRate a) r))) (when-let [b (:backend @current)] (t/-set-rate! b r)))
(defn play! [] (defn play! []
(when-let [a @el] (when-let [b (:backend @current)] (t/-play! b)))
;; Returns a promise that rejects if the browser has not seen a gesture yet.
;; Swallowed: the transport button IS the gesture, so this can only fire on a
;; programmatic play, where a console error is noise rather than news.
(some-> (.play a) (.catch (fn [_])))))
(defn pause! [] (defn pause! []
(when-let [a @el] (.pause a))) (when-let [b (:backend @current)] (t/-pause! b)))
(defn seek! (defn seek!
"Put the audio at the start of frame f. Seeking to the frame's start rather "Put the audio at the start of frame f. Seeking to the frame's start rather
than its middle keeps `frame` idempotent: seek to f, read back f." than its middle keeps `frame` idempotent: seek to f, read back f."
[fps frames f] [fps frames f]
(when-let [a @el] (when-let [b (:backend @current)]
(set! (.-currentTime a) (/ (clamp f frames) fps)))) (t/-seek! b (/ (clamp f frames) fps))))
(defn set-loop! (defn set-loop!
"Wrap at the end instead of stopping. The frame stays derived — `currentTime` "Wrap at the end instead of stopping. The frame stays derived — the position
simply returns to zero — so nothing about the sync changes, which is the point simply returns to zero — so nothing about the sync changes, which is the point
of not counting frames. of not counting frames.
It earns its place at 2x and 4x, where the whole clip is gone in under four It earns its place at 2x and 4x, where the whole clip is gone in under four
seconds and a profile wants more than that to look at." seconds and a profile wants more than that to look at."
[on?] [on?]
(when-let [a @el] (set! (.-loop a) (boolean on?)))) (when-let [b (:backend @current)] (t/-set-loop! b on?)))
(defn set-muted! [on?] (defn set-muted! [on?]
(when-let [a @el] (set! (.-muted a) (boolean on?)))) (when-let [b (:backend @current)] (t/-set-muted! b on?)))
(defn duration-frames (defn duration-frames
"How many frames the audio actually covers, which need not be the clip's "How many frames the audio actually covers, which need not be the clip's
length. Reported rather than assumed: a clip longer than its audio is a length. Reported rather than assumed: a clip longer than its audio is a
legitimate thing to be told about, not a thing to silently truncate." legitimate thing to be told about, not a thing to silently truncate."
[fps] [fps]
(when-let [a @el] (when-let [b (:backend @current)]
(let [d (.-duration a)] (let [d (t/-duration b)]
(when (and d (js/isFinite d)) (js/Math.ceil (* d fps)))))) (when (and d (js/isFinite d)) (js/Math.ceil (* d fps))))))
(defn exposed-frame (defn exposed-frame

View file

@ -0,0 +1,45 @@
(ns arthur.clock.element
"The clock on an `<audio>` element. The original backend, kept switchable.
Reads `currentTime` and takes it as the position. That is the whole of it, and
it is why this backend needs a WAV: an element holds a URL, so a mixdown has
to be encoded and blobbed before it can be played — see `audio/mix`'s
`clock!`. The encode is O(the clip's length) and runs on the main thread, so a
long take costs seconds of frozen UI on open, on every tab switch and on every
edit to a track. `arthur.clock.graph` exists to not do that.
What this backend still has that the graph one has to be told: the OS media
keys and the media session, which the element gets from the browser for free.
KEPT so the two can be compared on the same document rather than swapped on
faith. When the graph backend has been trusted for a while, this namespace and
`mix/clock!`'s WAV packaging go together."
(:require [arthur.clock.transport :as t]))
(deftype Element [^js el]
t/Transport
(-position [_] (.-currentTime el))
(-duration [_] (.-duration el))
(-playing? [_] (and (not (.-paused el)) (not (.-ended el))))
(-rate [_] (.-playbackRate el))
(-set-rate! [_ r] (set! (.-playbackRate el) r))
(-play! [_]
;; Returns a promise that rejects if the browser has not seen a gesture yet.
;; Swallowed: the transport button IS the gesture, so this can only fire on a
;; programmatic play, where a console error is noise rather than news.
(some-> (.play el) (.catch (fn [_]))))
(-pause! [_] (.pause el))
(-seek! [_ seconds] (set! (.-currentTime el) seconds))
(-set-loop! [_ on?] (set! (.-loop el) (boolean on?)))
(-set-muted! [_ on?] (set! (.-muted el) (boolean on?)))
(-release! [_]
;; Nothing. React owns the element and unmounting it is what stops it; this
;; backend is a few property accesses wrapped in a type and holds no more
;; than that.
nil))
(defn backend
"Wrap an audio element — or anything that answers the same five properties,
which is what `clock-test`'s fake is."
[el]
(->Element el))

View file

@ -0,0 +1,266 @@
(ns arthur.clock.graph
"The clock on a Web Audio graph. No encode, no blob, no element.
THE BUFFER IS PLAYED, NOT PACKAGED. `audio/mix`'s `buffer!` already renders
the mixdown; the element backend then spent O(the clip's length) encoding that
buffer to a WAV purely so a `src` attribute had something to point at. An
`AudioBufferSourceNode` takes the buffer as it is, so opening a document costs
one node instead of a hundred megabytes of 16-bit PCM.
THE FRAME IS STILL DERIVED, AND FROM A BETTER CLOCK. `AudioContext.currentTime`
is the audio device's own position in double precision, advancing every 128
samples; an element's `currentTime` is whatever the media pipeline last
published and is permitted to lag. Nothing is counted here either — position is
an anchor plus elapsed context time, and the anchor is re-set on every seek and
every rate change, so no arithmetic accumulates across either.
AND IT IS LATENCY-COMPENSATED, which is the one thing this backend must get
right. `currentTime` is the time of the quantum being RENDERED, which the
speaker is `outputLatency` behind — tens of milliseconds, far more over
Bluetooth. Report the renderer's position and the picture leads the sound by
exactly that much, which in a lip-sync tool is the only artefact that matters.
So `scheduled` is the bookkeeping and `-position` is `scheduled` read one
latency in the past. The element has the same lag underneath; the difference is
that this one is a number we can subtract rather than an error we inherit.
A SYMBOL WITH NO SOUND GETS A CLOCK ANYWAY, with no buffer at all: `seconds`
is its length and the context's clock does the rest. The element backend had to
synthesize silence and encode THAT to a WAV, full length, so the element had a
duration to report — the clearest sign that the element was driving the design
rather than serving it."
(:require [arthur.clock.transport :as t]))
(defonce ^:private shared (atom nil))
(defn available?
"Whether this backend can run at all. False under node, where the tests live."
[]
(exists? js/AudioContext))
(defn context
"The app's one `AudioContext`, made on first use.
Lazy because constructing one before anything wants to play is how a browser
decides the page is trying to autoplay, and because `available?` is false in
the test runner."
[]
(or (:ctx @shared)
(let [ctx (js/AudioContext.)
;; ONE output gain for the app, not one per backend: a tab switch
;; replaces the backend, and a gain node per backend would leave the
;; old one wired to the destination. Mute lives here for the same
;; reason — it is a property of the transport, not of whichever
;; buffer happens to be loaded.
out (.createGain ctx)]
(.connect out (.-destination ctx))
(:ctx (reset! shared {:ctx ctx :out out})))))
(defn output [] (do (context) (:out @shared)))
(defn- latency
"How far ahead of the speaker `currentTime` is, in seconds.
`outputLatency` is the whole path and the number we want. Firefox reports 0
until the graph has actually run, so `baseLatency` stands in — it is only the
graph's own buffering and therefore an underestimate, which errs towards the
picture leading slightly rather than the correction overshooting. Neither
exists everywhere; 0 is then no worse than the element."
[^js ctx]
(let [out (.-outputLatency ctx)
base (.-baseLatency ctx)]
(cond
(and (number? out) (js/isFinite out) (pos? out)) out
(and (number? base) (js/isFinite base) (pos? base)) base
:else 0)))
(defn- seg-at
"The segment governing context time `t`."
[segs t]
(or (last (filter #(<= (:from %) t) segs)) (first segs)))
(defn- raw
"Where the audio is at context time `t`, in seconds into the buffer.
PIECEWISE, and that is the whole subtlety of this namespace. Audio already
rendered cannot be re-rated: when the transport goes 1x -> 4x, the samples
still travelling to the speaker were rendered at 1x, so reading them back at
4x jumps the playhead BACKWARDS by three output latencies — about fourteen
frames on a laptop, which is a visible lurch on every rate change and was
exactly what the first version of this did. So each `play`, `pause`, `seek`
and rate change records a segment, and a position is read against whichever
segment was in force when that audio was rendered.
Evaluated before the oldest segment we kept, it clamps. That is what makes a
seek read back exactly what was seeked to: a seek has nothing in flight worth
honouring — the user has jumped — so its segment starts the timeline over."
[segs t]
(let [t (max t (:from (first segs)))
s (seg-at segs t)]
(+ (:anchor s) (* (:rate s) (- t (:from s))))))
(defn- at-renderer
"Where the graph has rendered up to: `raw` at the context's own clock."
[{:keys [^js ctx segs]}]
(raw segs (.-currentTime ctx)))
(defn- at-speaker
"Where the sound being heard is: `raw` one output latency in the past. See the
namespace docstring — this is the whole of the compensation."
[{:keys [^js ctx segs]}]
(raw segs (- (.-currentTime ctx) (latency ctx))))
(defn- running?
"Whether the renderer is advancing. Read off the segments rather than kept
beside them, so there is one answer and not two that can disagree."
[{:keys [segs]}]
(pos? (:rate (last segs))))
(defn- prune
"Drop segments no position can still need: everything before the last one that
began at or before the in-flight window. Unbounded history would otherwise
grow by one entry per rate change for the life of the document."
[segs cutoff]
(let [n (count (take-while #(<= (:from %) cutoff) segs))]
(if (<= n 1) segs (subvec segs (dec n)))))
(defn- position
"`at-speaker`, brought inside the audio. Clamped before the wrap, so the few
milliseconds of negative position right after a looped play — the first
samples are still in the output buffer — read as 0 rather than as the end."
[{:keys [seconds loop?] :as st}]
(let [p (at-speaker st)]
(cond
(not (and seconds (pos? seconds))) 0
loop? (mod (max 0 p) seconds)
:else (-> p (max 0) (min seconds)))))
(defn- done?
"Run off the end. Judged at the SPEAKER, so playback is still reported as
running while the last scheduled samples are on their way out."
[{:keys [seconds loop?] :as st}]
(and (not loop?) seconds (pos? seconds) (>= (at-speaker st) seconds)))
(defn- spin-up!
"A fresh source node playing from where the renderer now is.
`AudioBufferSourceNode`s are single-use, so a play, a seek while playing and a
resumed pause each make a new one; they are cheap, which is the point of this
backend.
Nil when there is no buffer — the silent clock runs on the context alone."
[{:keys [^js ctx ^js buffer ^js out rate loop? seconds] :as st}]
(when buffer
(let [node (.createBufferSource ctx)]
(set! (.-buffer node) buffer)
(set! (.-loop node) (boolean loop?))
(set! (.-value (.-playbackRate node)) rate)
(.connect node out)
;; `when` 0 is "as soon as the graph can"; the offset is where in the
;; buffer to begin. Clamped because starting past the end is a range
;; error rather than a no-op in some engines.
(.start node 0 (-> (at-renderer st) (max 0) (min (or seconds 0))))
node)))
(defn- spin-down! [{:keys [^js node]}]
(when node
(try (.stop node) (catch :default _ nil))
(try (.disconnect node) (catch :default _ nil))))
(defn- restart!
"Begin a new timeline at `anchor`, running at `rate` or held when it is 0.
A RESET rather than a segment, for the three cases that have nothing in flight
worth honouring: a play starts fresh, a seek means the user has jumped, and a
pause wants one stable number for the readout rather than a position that
creeps forward as the output buffer drains."
[state anchor rate]
(let [st @state]
(spin-down! st)
(swap! state assoc
:segs [{:from (.-currentTime ^js (:ctx st)) :anchor anchor :rate rate}]
:node nil)
(when (pos? rate)
(swap! state assoc :node (spin-up! @state)))))
(deftype Graph [state]
t/Transport
(-position [_] (position @state))
(-duration [_] (:seconds @state))
(-playing? [_] (let [st @state] (boolean (and (running? st) (not (done? st))))))
(-rate [_] (:rate @state))
(-set-rate! [_ r]
;; A SEGMENT, not a reset — the one case where what is already in flight has
;; to keep its old rate or the playhead lurches. See `raw`.
(let [st @state
now (.-currentTime ^js (:ctx st))]
(if (running? st)
(let [anchor (at-renderer st)]
(swap! state #(-> %
(assoc :rate r)
(update :segs (fn [segs]
(prune (conj segs {:from now :anchor anchor :rate r})
(- now (latency ^js (:ctx st))))))))
(when-let [^js node (:node st)]
(set! (.-value (.-playbackRate node)) r)))
;; Stopped: nothing is in flight and nothing is rendering, so the rate
;; is simply what the next play will run at.
(swap! state assoc :rate r))))
(-play! [_]
(let [st @state]
;; `done?` counts as not playing. It is DERIVED — no flag is cleared when
;; the take runs off its end — so testing "running" alone would make play
;; a dead button from the moment the audio finished.
(when (or (not (running? st)) (done? st))
;; The context starts suspended and only a gesture may resume it. The
;; transport button IS the gesture, so a rejection here means a
;; programmatic play and is noise rather than news — as on the element.
(some-> (.resume ^js (:ctx st)) (.catch (fn [_])))
;; Play at the end starts over, which is what the element does.
(restart! state (if (done? st) 0 (position st)) (:rate st)))))
(-pause! [_]
(let [st @state]
(when (running? st)
;; Anchored at what was HEARD, not at what was scheduled, so play after
;; pause resumes from where the sound stopped. It re-plays the last few
;; milliseconds rather than skipping them, which is the kinder of the
;; two roundings.
(restart! state (position st) 0))))
(-seek! [_ seconds]
(restart! state seconds (if (running? @state) (:rate @state) 0)))
(-set-loop! [_ on?]
(swap! state assoc :loop? (boolean on?))
;; The audio thread reads `loop` every quantum, so a live node picks this up
;; without being restarted — the same as setting `loop` on a playing element.
(when-let [^js node (:node @state)]
(set! (.-loop node) (boolean on?))))
(-set-muted! [_ on?]
(set! (.-value (.-gain ^js (:out @state))) (if on? 0 1)))
(-release! [_]
;; The source node is wired to the output the whole app shares, so a backend
;; dropped while playing would go on being heard under the one that replaced
;; it. Nothing else here needs releasing: the context and its gain outlive
;; every backend by design.
(let [st @state]
(spin-down! st)
(swap! state assoc
:segs [{:from (.-currentTime ^js (:ctx st))
:anchor (position st) :rate 0}]
:node nil))))
(defn backend
"A clock on `buffer`, `seconds` long. A nil buffer is a silent clock of that
length — see the namespace docstring.
The context and output are injected by the four-argument form so the whole of
this is assertable against a fake in node, where there is no Web Audio."
([buffer seconds] (backend (context) (output) buffer seconds))
([^js ctx ^js out buffer seconds]
(->Graph (atom {:ctx ctx :out out :buffer buffer :seconds seconds
:rate 1.0 :loop? false :node nil
:segs [{:from (.-currentTime ctx) :anchor 0 :rate 0}]}))))

View file

@ -0,0 +1,38 @@
(ns arthur.clock.transport
"What the clock needs of a thing that plays sound, and nothing more.
`arthur.clock` does the arithmetic — the frame derivation, the clamping, the
exposure grid — and a backend only has to answer where the sound has got to
and do as it is told. Keeping the protocol this thin is what makes the two
implementations comparable: if the graph backend and the element backend
disagree about a take's sync, the difference is in these nine methods and not
in anything derived from them.
POSITION IS WHERE THE SPEAKER IS, not where the renderer is. A backend that
schedules audio ahead of the output owes the difference back here — see
`arthur.clock.graph` — because the picture is drawn against this number and a
lip-sync tool that draws against the scheduler leads the sound it is matching.")
(defprotocol Transport
(-position [this]
"Seconds into the audio, as heard. Never negative, never past `-duration`,
and wrapped rather than clamped while looping.")
(-duration [this]
"Seconds of audio, or nil when the backend has not been told yet.")
(-playing? [this]
"Running AND not finished. A backend that has reached its end reports false
even if nothing told it to stop, because the end of the sound is the
authority on playback having stopped — see `ui/player`'s loop.")
(-rate [this])
(-set-rate! [this r])
(-play! [this])
(-pause! [this])
(-seek! [this seconds]
"Put the audio at `seconds`. Reading `-position` back must give the same
number, which is what makes a scrub idempotent.")
(-set-loop! [this on?])
(-set-muted! [this on?])
(-release! [this]
"Give up whatever this backend holds, because something else is about to be
the clock. NOT a pause: a backend that owns nothing has nothing to do here,
and the position it was last at is no longer anybody's business."))

View file

@ -139,7 +139,8 @@
:status "built-in example · not a saved project"}))] :status "built-in example · not a saved project"}))]
{:db db {:db db
::pause! nil ::pause! nil
::seek! [(fps db) (frames db) 0]}))) ::seek! [(fps db) (frames db) 0]
::clock! {:id id :sid (get-in db [:ui :open])}})))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; tabs ;; tabs
@ -153,16 +154,39 @@
(rf/reg-fx (rf/reg-fx
::clock! ::clock!
;; WHAT THE OPEN SYMBOL SOUNDS LIKE, fetched once per thing that can change it:
;; a document opening, a tab switch, an edit to a track. The graph backend is
;; handed the buffer; the element backend needs a URL, which means a WAV, which
;; is the whole of what this used to cost.
(fn [{:keys [id sid]}] (fn [{:keys [id sid]}]
(let [{:keys [clip audio store]} (footage/entry id)] (let [{:keys [clip audio store]} (footage/entry id)]
(-> (mix/clock! clip sid audio store) (-> (if (clock/graph?)
(mix/clock-source! clip sid audio store)
(.then (mix/clock! clip sid audio store) (fn [u] {:url u})))
(.then #(rf/dispatch [::clock-ready id sid %])) (.then #(rf/dispatch [::clock-ready id sid %]))
(.catch #(js/console.error %)))))) (.catch #(js/console.error %))))))
(rf/reg-event-fx (rf/reg-event-fx
::clock-ready ::clock-ready
(fn [{:keys [db]} [_ id sid url]] (fn [{:keys [db]} [_ id sid {:keys [url buffer seconds]}]]
(if (and (= id (:clip/current db)) (= sid (get-in db [:ui :open]))) (if-not (and (= id (:clip/current db)) (= sid (get-in db [:ui :open])))
{}
(if (clock/graph?)
(do
(clock/attach-buffer! buffer seconds)
;; RATE, LOOP AND MUTE ARE RE-APPLIED. They are transport state and
;; outlive the clock they were set on, but a fresh graph starts at its
;; defaults — so without this a tab switch would quietly drop the take
;; back to 1x, unlooped and unmuted. The element kept them because they
;; were properties of a node that survived its own `src` changing.
{:fx [[::rate! (get-in db [:playback :rate] 1.0)]
[::loop! (boolean (get-in db [:playback :loop?]))]
[::mute! (boolean (get-in db [:playback :muted?]))]
[::seek! [(fps db) (frames db) (get-in db [:playback :frame] 0)]]
;; An edit under a playing take resumes it. The element stopped
;; dead here, which made adjusting a fade while listening to it
;; a thing you could only do once.
(when (get-in db [:playback :playing?]) [::play! nil])]})
(do (do
;; A blob URL made for the last tab is released when the next one lands, ;; A blob URL made for the last tab is released when the next one lands,
;; and never the document's own file. ;; and never the document's own file.
@ -171,8 +195,7 @@
(reset! clock-url (when (and (not= url (:audio (footage/entry id))) (reset! clock-url (when (and (not= url (:audio (footage/entry id)))
(.startsWith url "blob:")) (.startsWith url "blob:"))
url)) url))
{:db (assoc-in db [:clip :audio] url)}) {:db (assoc-in db [:clip :audio] url)})))))
{})))
(rf/reg-event-db (rf/reg-event-db
::paint-failed ::paint-failed

View file

@ -30,7 +30,6 @@
[arthur.domain.node :as node] [arthur.domain.node :as node]
[arthur.domain.palette :as pal] [arthur.domain.palette :as pal]
[arthur.events.edit :as edit] [arthur.events.edit :as edit]
[arthur.audio.mix :as mix]
[arthur.demo.stage :as stage] [arthur.demo.stage :as stage]
[arthur.domain.feature :as feature] [arthur.domain.feature :as feature]
[arthur.domain.project :as project] [arthur.domain.project :as project]
@ -124,9 +123,15 @@
;; write lands on what does not. ;; write lands on what does not.
:synced (project/tier1 (.-leaves clip-json)) :synced (project/tier1 (.-leaves clip-json))
:clip built :store (:store loaded) :clip built :store (:store loaded)
;; The document's OWN file, unmixed. What
;; the symbol actually sounds like is the
;; clock's business and is fetched once,
;; by `::pb/clock!`, when it goes on
;; screen — this used to mix it here as
;; well, and the two encodes of the same
;; audio were most of a project open.
:audio "/static/arthur/audio.wav"})] :audio "/static/arthur/audio.wav"})]
(-> (mix/mix! built (clip/opens-on built) (:audio entry) (:store entry)) entry))))))
(.then (fn [audio] (assoc entry :audio audio))))))))))
(defn- saved-clip! (defn- saved-clip!
"Promise of clip `cid` of saved project `pid`, as `{:clip :store}`: its "Promise of clip `cid` of saved project `pid`, as `{:clip :store}`: its
@ -411,11 +416,7 @@
entry (assoc entry :clip built :label (:name built) entry (assoc entry :clip built :label (:name built)
:cid "stage-8625" :cid "stage-8625"
:width (:width built) :height (:height built))] :width (:width built) :height (:height built))]
(-> (mix/mix! built (clip/opens-on built) (:audio entry) (:store entry)) (rf/dispatch [::stage-opened (store/install! entry "stage")]))))
(.then (fn [audio]
(rf/dispatch
[::stage-opened
(store/install! (assoc entry :audio audio) "stage")])))))))
(.catch (fn [error] (.catch (fn [error]
(js/console.error error) (js/console.error error)
(rf/dispatch [::failed (or (ex-message error) (str error))])))))) (rf/dispatch [::failed (or (ex-message error) (str error))]))))))
@ -557,12 +558,13 @@
;; events ;; events
(def blank-audio (def blank-audio
"A new document still needs a clock. "The audio a new document opens on, until it gets one of its own.
The frame is derived from an audio element and from nothing else — see It is no longer load-bearing. The frame is derived from a position, and the
`arthur.clock` — so a stage with no sound has no time and `play` is a button graph backend can hold one for a symbol with no sound at all — see
that cannot work. The synthetic take borrows this same asset for exactly this `mix/clock-source!` — so a silent stage has time and `play` works. This stays
reason. When a document gets audio of its own, it replaces this." because a new document borrowing the synthetic take's soundtrack is a
convenience worth keeping, not because the clock would stop without it."
"/static/arthur/audio.wav") "/static/arthur/audio.wav")
(defn blank-entry (defn blank-entry
@ -591,7 +593,8 @@
;; The readout goes home with the document; so must the clock, or play ;; The readout goes home with the document; so must the clock, or play
;; picks up wherever the last document's audio had got to. ;; picks up wherever the last document's audio had got to.
::pb/seek! [(:fps entry) (clip/output-frames (:clip entry) (clip/opens-on (:clip entry))) 0] ::pb/seek! [(:fps entry) (clip/output-frames (:clip entry) (clip/opens-on (:clip entry))) 0]
::pb/pause! nil}))) ::pb/pause! nil
::pb/clock! {:id id :sid (clip/opens-on (:clip entry))}})))
(rf/reg-event-fx (rf/reg-event-fx
::save ::save
@ -848,7 +851,8 @@
(assoc :project {:id nil :cid nil :name nil :seq nil (assoc :project {:id nil :cid nil :name nil :seq nil
:busy? false :status "loaded 8625 stage study"}))] :busy? false :status "loaded 8625 stage study"}))]
{:db db {:db db
::pb/seek! [(get-in db [:clip :fps]) (pb/frames db) 0]}))) ::pb/seek! [(get-in db [:clip :fps]) (pb/frames db) 0]
::pb/clock! {:id clip-id :sid (get-in db [:ui :open])}})))
(rf/reg-event-fx (rf/reg-event-fx
::saved ::saved
@ -898,7 +902,9 @@
:status (str "opened " name " r" seq)})) :status (str "opened " name " r" seq)}))
::pb/pause! nil ::pb/pause! nil
::pb/seek! (let [{c :clip fps :fps} (store/entry clip-id)] ::pb/seek! (let [{c :clip fps :fps} (store/entry clip-id)]
[fps (clip/output-frames c (clip/opens-on c)) 0])})) [fps (clip/output-frames c (clip/opens-on c)) 0])
::pb/clock! {:id clip-id
:sid (clip/opens-on (:clip (store/entry clip-id)))}}))
(rf/reg-event-fx (rf/reg-event-fx
::failed ::failed

View file

@ -72,8 +72,14 @@
(fn [[id _ sid] _] (fn [[id _ sid] _]
;; What the open symbol plays, off the document as saved rather than `::clip`, ;; What the open symbol plays, off the document as saved rather than `::clip`,
;; so a bar being slid does not re-mix on every frame of the drag. ;; so a bar being slid does not re-mix on every frame of the drag.
;;
;; THE DOCUMENT IS PART OF THE IDENTITY, not just the symbol. `ui/shell`
;; re-mixes when this changes under the SAME symbol, and two documents both
;; opening on `:main` would otherwise look like one symbol whose tracks had
;; been edited — so opening a project re-mixed it a second time, on top of
;; the mix the open itself had already asked for.
(let [c (:clip (footage/entry id))] (let [c (:clip (footage/entry id))]
[sid (when (clip/symbol c sid) (nest/audio-tracks c sid))]))) [id sid (when (clip/symbol c sid) (nest/audio-tracks c sid))])))
(rf/reg-sub (rf/reg-sub
::symbol ::symbol

View file

@ -1,5 +1,6 @@
(ns arthur.ui.shell (ns arthur.ui.shell
"The window: one grid, five panes, a location bar, and the audio element. "The window: one grid, five panes, a location bar, and the clock's audio
element on the backend that needs one.
Nothing else. Each pane owns its own subscriptions, so this component re-renders Nothing else. Each pane owns its own subscriptions, so this component re-renders
only when the grid itself would change — which is a pane being resized, shut or only when the grid itself would change — which is a pane being resized, shut or
@ -29,11 +30,24 @@
;; only a change under the same one counts. ;; only a change under the same one counts.
(r/with-let [heard (atom nil) (r/with-let [heard (atom nil)
remix (r/track! (fn [] remix (r/track! (fn []
(let [[sid :as now] @(rf/subscribe [::render/sounds]) (let [[id sid :as now] @(rf/subscribe [::render/sounds])
[was :as before] @heard] [was-id was-sid :as before] @heard]
(reset! heard now) (reset! heard now)
(when (and before (= sid was) (not= now before)) ;; The same symbol OF THE SAME DOCUMENT. A
;; different document, or a different symbol, has
;; already asked for its own clock — see
;; `::pb/clock!`'s callers — and re-mixing here as
;; well is a second render of minutes of audio.
(when (and before
(= [id sid] [was-id was-sid])
(not= now before))
(rf/dispatch [::pb/refresh-clock])))))] (rf/dispatch [::pb/refresh-clock])))))]
;; NOTHING TO RENDER under the graph backend: it plays the mixdown buffer
;; directly and takes its position from the audio context, so there is no
;; element and no WAV for one to hold. The `remix` track above still runs
;; either way — it is what notices an edit to a track, and `with-let` builds
;; it whatever this body returns.
(when-not (clock/graph?)
[:audio [:audio
{:ref #(when % (clock/attach! %)) {:ref #(when % (clock/attach! %))
:src @(rf/subscribe [::playback/audio]) :src @(rf/subscribe [::playback/audio])
@ -43,7 +57,7 @@
;; OS media keys, a browser autoplay block — has to be able to correct the ;; OS media keys, a browser autoplay block — has to be able to correct the
;; document rather than be contradicted by it. ;; document rather than be contradicted by it.
:on-play #(rf/dispatch [::pb/play]) :on-play #(rf/dispatch [::pb/play])
:on-pause #(rf/dispatch [::pb/pause])}] :on-pause #(rf/dispatch [::pb/pause])}])
(finally (r/dispose! remix)))) (finally (r/dispose! remix))))
(defn view [] (defn view []

View file

@ -0,0 +1,254 @@
(ns arthur.clock.graph-test
"The graph clock, asserted against a hand-moved context clock.
The context's `currentTime` is set by these tests rather than advanced by a
device, which is exactly the coupling worth asserting: if any of this counted
time instead of deriving it, moving the clock by hand would not move the
position. A drift bug is invisible for the first second and unmistakable by
the tenth, which is the worst possible shape for a bug to have."
(:require [cljs.test :refer [deftest is testing]]
[arthur.clock.element :as element]
[arthur.clock.graph :as graph]
[arthur.clock.transport :as t]))
(defn- fake-node [log]
(let [n #js {:loop false :playbackRate #js {:value 1.0}}]
(set! (.-connect n) (fn [_] nil))
(set! (.-disconnect n) (fn [] nil))
(set! (.-start n) (fn [_when offset] (swap! log conj [:start offset]) nil))
(set! (.-stop n) (fn [] (swap! log conj [:stop]) nil))
n))
(defn- fake-ctx
"Stands in for an `AudioContext`. `currentTime` is a plain writable property
so a test can move it; the rest is the three methods the backend calls."
[latency log]
(let [ctx #js {:currentTime 0.0 :outputLatency latency :baseLatency 0}]
(set! (.-resume ctx) (fn [] nil))
(set! (.-createBufferSource ctx) (fn [] (fake-node log)))
ctx))
(defn- rig
"A backend, its context, and the log of what it did to its source nodes."
[& {:keys [seconds latency buffer?] :or {seconds 10.0 latency 0 buffer? true}}]
(let [log (atom [])
ctx (fake-ctx latency log)
out #js {:gain #js {:value 1}}]
{:ctx ctx :log log :out out
:g (graph/backend ctx out (when buffer? #js {}) seconds)}))
(defn- at! [{:keys [^js ctx]} t] (set! (.-currentTime ctx) t))
(defn- close?
"Within a frame's worth of a microsecond. Positions compound a multiply and a
subtract, so the last bit of a double is not something to assert on."
[a b]
(< (js/Math.abs (- a b)) 1e-9))
(deftest the-position-is-derived-from-the-context-clock-not-counted
;; Nothing here is ticked. The context moves and the position follows, which
;; is the whole property: a stalled paint loop cannot lose the clock, because
;; the clock was never in the paint loop.
(let [{:keys [g] :as r} (rig)]
(t/-play! g)
(doseq [[now want] [[0.0 0.0] [0.5 0.5] [2.0 2.0] [9.25 9.25]]]
(at! r now)
(is (= want (t/-position g)) (str "context at " now "s")))))
(deftest a-stall-lands-where-the-audio-already-is
;; THE property the derivation buys, same as the element's. A loop that
;; stalled for a third of a second resumes at the position the audio reached,
;; not a third of a second behind it.
(let [{:keys [g] :as r} (rig)]
(t/-play! g)
(at! r 1.0)
(is (= 1.0 (t/-position g)))
(at! r 1.5) ; fifteen frames' worth of stall
(is (= 1.5 (t/-position g)) "and nothing was counted in between")))
(deftest the-position-is-the-speaker-not-the-renderer
;; `currentTime` is the quantum being RENDERED; the speaker is one output
;; latency behind it. Report the renderer and the picture leads the sound by
;; that much, which in a lip-sync tool is the only artefact that matters.
(let [{:keys [g] :as r} (rig :latency 0.05)]
(t/-play! g)
(at! r 2.0)
(is (= 1.95 (t/-position g)) "50ms of output latency, subtracted")
(testing "and scaled by the rate, because a latency is wall time"
(t/-set-rate! g 4.0)
(at! r 3.0)
;; Banked at 1.95 + 0.05 = 2.0, then four seconds of buffer per second of
;; wall clock, less the 4 x 50ms still in the output buffer.
(is (close? 5.8 (t/-position g))))))
(deftest a-rate-change-does-not-lurch-the-playhead
;; THE bug the segments exist for. Audio already rendered cannot be re-rated:
;; read the samples still travelling to the speaker at the NEW rate and the
;; position jumps backwards by three output latencies the instant 1x becomes
;; 4x — fourteen frames at 30fps with a laptop's 150ms, which is a visible
;; lurch on every press of the rate button.
(let [{:keys [g] :as r} (rig :seconds 100.0 :latency 0.15)]
(t/-play! g)
(at! r 2.0)
(let [before (t/-position g)]
(t/-set-rate! g 4.0)
(is (close? before (t/-position g))
"the same instant, read either side of the change")
(testing "and it goes FORWARD from there, at the new rate"
(at! r 2.5)
(is (> (t/-position g) before))
(at! r 3.0)
;; At 3.0s of context the speaker is hearing what was rendered at
;; 2.85s. The rate changed when the renderer was at buffer position
;; 2.0, which is 0.85s of wall clock earlier — so 4 x 0.85 of buffer
;; since, and nothing before that boundary re-rated.
(is (close? (+ 2.0 (* 4 0.85)) (t/-position g)))))
(testing "and back to 1x is continuous too"
(let [before (t/-position g)]
(t/-set-rate! g 1.0)
(is (close? before (t/-position g)))))))
(deftest segments-do-not-pile-up
;; One per rate change, pruned to the in-flight window. Unbounded history
;; would grow for the life of the document.
(let [{:keys [g] :as r} (rig :seconds 1000.0 :latency 0.05)]
(t/-play! g)
(dotimes [i 200]
(at! r (+ 1.0 i))
(t/-set-rate! g (if (even? i) 4.0 1.0)))
(at! r 500.0)
(is (number? (t/-position g)))
(is (<= (count (:segs @(.-state g))) 3)
"the one in force, plus whatever is still in flight")))
(deftest the-first-samples-are-not-the-last-ones
;; Right after a looped play the audible position is a few milliseconds
;; NEGATIVE — the first samples are still in the output buffer. Wrapping
;; before clamping would read that as the very end of the take and jump the
;; playhead there on every play.
(let [{:keys [g] :as r} (rig :seconds 2.0 :latency 0.05)]
(t/-set-loop! g true)
(t/-play! g)
(at! r 0.0)
(is (= 0 (t/-position g)))))
(deftest a-rate-change-is-banked-so-nothing-accumulates
;; 1x -> 4x -> 1x across a take. The anchor moves to where the graph had got
;; to and the new rate applies from there, so switching rates mid-take cannot
;; leave an offset behind — which is what a multiplier on a counted frame
;; would do.
(let [{:keys [g] :as r} (rig :seconds 100.0)]
(t/-play! g)
(at! r 2.0)
(is (= 2.0 (t/-position g)))
(t/-set-rate! g 4.0)
(at! r 3.0)
(is (= 6.0 (t/-position g)) "one second of wall clock, four of buffer")
(t/-set-rate! g 1.0)
(at! r 4.0)
(is (= 7.0 (t/-position g)) "and back to one for one, from six")
(is (= 1.0 (t/-rate g)))))
(deftest a-seek-round-trips
;; Reading back exactly what was seeked to is what makes a scrub idempotent.
(let [{:keys [g]} (rig)]
(doseq [t [0.0 0.5 3.25 9.0]]
(t/-seek! g t)
(is (= t (t/-position g)) (str "seek to " t "s")))))
(deftest a-seek-while-playing-restarts-the-source-at-the-new-offset
;; Source nodes are single-use, so a seek under a playing take has to stop the
;; old one and start a new one at the offset. Checked through the log because
;; an offset that silently stayed at zero would still read back correctly from
;; the arithmetic while playing the wrong audio.
(let [{:keys [g log] :as r} (rig)]
(t/-play! g)
(at! r 1.0)
(reset! log [])
(t/-seek! g 4.0)
(is (= [[:stop] [:start 4.0]] @log))
(is (= 4.0 (t/-position g)))))
(deftest looping-wraps-rather-than-stopping
(let [{:keys [g] :as r} (rig :seconds 2.0)]
(t/-set-loop! g true)
(t/-play! g)
(at! r 5.0)
(is (= 1.0 (t/-position g)) "five seconds into a two-second loop")
(is (t/-playing? g) "and a loop never ends")))
(deftest running-off-the-end-stops-being-playing
;; The end of the sound is the authority on playback having stopped — nothing
;; counts frames to notice it. `ui/player`'s loop reads this to correct the
;; document.
(let [{:keys [g] :as r} (rig :seconds 2.0)]
(t/-play! g)
(at! r 1.9)
(is (t/-playing? g))
(at! r 2.5)
(is (not (t/-playing? g)))
(is (= 2.0 (t/-position g)) "and the position stops at the end, not past it")
(testing "and playing again starts over, as the element does"
(t/-play! g)
(is (= 0.0 (t/-position g))))))
(deftest a-symbol-with-no-sound-still-keeps-time
;; NO BUFFER AT ALL. The element backend had to synthesize silence and encode
;; it to a WAV, full length, so that it had a duration to report; here the
;; length is simply told to the clock and the context does the rest.
(let [{:keys [g log] :as r} (rig :buffer? false :seconds 3.0)]
(t/-play! g)
(is (= [] @log) "and no source node was made for silence")
(at! r 1.0)
(is (= 1.0 (t/-position g)))
(is (t/-playing? g))
(is (= 3.0 (t/-duration g)))
(at! r 3.5)
(is (not (t/-playing? g)) "and silence ends when the symbol does")))
(deftest pause-anchors-at-what-was-heard
;; Resuming replays the last few milliseconds rather than skipping them, which
;; is the kinder of the two roundings.
(let [{:keys [g] :as r} (rig :latency 0.05)]
(t/-play! g)
(at! r 2.0)
(t/-pause! g)
(is (= 1.95 (t/-position g)))
(is (not (t/-playing? g)))
(testing "and the position holds while paused, however the context moves"
(at! r 8.0)
(is (= 1.95 (t/-position g))))))
(deftest releasing-stops-the-source
;; A backend dropped while playing would go on being heard under the one that
;; replaced it: the source is wired to the output the whole app shares.
(let [{:keys [g log] :as r} (rig)]
(t/-play! g)
(at! r 1.0)
(reset! log [])
(t/-release! g)
(is (= [[:stop]] @log))
(is (not (t/-playing? g)))))
(deftest mute-is-the-shared-output-and-not-the-buffer
(let [{:keys [g ^js out]} (rig)]
(t/-set-muted! g true)
(is (= 0 (.-value (.-gain out))))
(t/-set-muted! g false)
(is (= 1 (.-value (.-gain out))))))
(deftest the-two-backends-report-the-same-position
;; The point of keeping both switchable. Everything BELOW the position — the
;; frame derivation, the clamp, the exposure grid — is in `arthur.clock` and
;; shared, so the two backends can only disagree about the position itself. At
;; zero output latency they must not; `clock-test` asserts what is built on
;; top of it, once, for both.
(let [{:keys [g] :as r} (rig :seconds 7.633)]
(t/-play! g)
(doseq [now [0.0 0.034 1.0 1.999 2.0 7.0]]
(at! r now)
(let [e (element/backend #js {:currentTime now :paused false :ended false
:playbackRate 1.0 :duration 7.633})]
(is (= (t/-position e) (t/-position g)) (str "at " now "s"))
(is (= (t/-duration e) (t/-duration g)))
(is (= (t/-playing? e) (t/-playing? g)))))))