From 179770d7d4ef63bbe81b8e5f6ac24b8d88927449 Mon Sep 17 00:00:00 2001 From: Olive Vaughn Date: Tue, 29 Sep 2026 12:25:46 -0400 Subject: [PATCH 01/26] Split the shell into topbar, pool, stage, timeline and params panes Co-Authored-By: Claude Opus 5.5 --- README.md | 4 +- clips/templates/clips/index.html | 75 +-- frontend/README.md | 94 +++- frontend/src/arthur/core.cljs | 8 +- frontend/src/arthur/db.cljs | 53 ++- frontend/src/arthur/domain/clip.cljs | 53 +++ frontend/src/arthur/events/edit.cljs | 26 ++ frontend/src/arthur/events/footage.cljs | 9 +- frontend/src/arthur/events/paint.cljs | 21 +- frontend/src/arthur/events/playback.cljs | 7 + frontend/src/arthur/events/project.cljs | 74 ++- frontend/src/arthur/events/ui.cljs | 79 ++++ frontend/src/arthur/subs/ui.cljs | 28 ++ frontend/src/arthur/ui/openmenu.cljs | 78 ++++ frontend/src/arthur/ui/paint.cljs | 149 ------ frontend/src/arthur/ui/palette.cljs | 51 ++ frontend/src/arthur/ui/params.cljs | 249 ++++++++++ frontend/src/arthur/ui/pool.cljs | 130 ++++++ frontend/src/arthur/ui/shell.cljs | 322 ++----------- frontend/src/arthur/ui/stage.cljs | 139 ++++++ frontend/src/arthur/ui/timeline.cljs | 266 +++++++++++ frontend/src/arthur/ui/topbar.cljs | 60 +++ frontend/test/browser/take.mjs | 209 ++++++--- static/arthur/app.css | 565 +++++++++++++++++++++++ 24 files changed, 2138 insertions(+), 611 deletions(-) create mode 100644 frontend/src/arthur/events/edit.cljs create mode 100644 frontend/src/arthur/events/ui.cljs create mode 100644 frontend/src/arthur/subs/ui.cljs create mode 100644 frontend/src/arthur/ui/openmenu.cljs delete mode 100644 frontend/src/arthur/ui/paint.cljs create mode 100644 frontend/src/arthur/ui/palette.cljs create mode 100644 frontend/src/arthur/ui/params.cljs create mode 100644 frontend/src/arthur/ui/pool.cljs create mode 100644 frontend/src/arthur/ui/stage.cljs create mode 100644 frontend/src/arthur/ui/timeline.cljs create mode 100644 frontend/src/arthur/ui/topbar.cljs create mode 100644 static/arthur/app.css diff --git a/README.md b/README.md index 6f178d2..7b70ce8 100644 --- a/README.md +++ b/README.md @@ -30,8 +30,8 @@ mise exec -- python manage.py migrate ./do start # Django + frontend watcher ``` -In the app, upload a video, choose its footage, click **load frames**, then -**save**. Opening that project on another client reuses its saved landmarks and +In the app, drop a video on the media pool — it uploads, extracts and runs +detection — then **save**. Opening that project on another client reuses its saved landmarks and mouth crops without detecting source frames again. The upload path derives its footage response from database records; it does not create or consume a `manifest.json` file. See [frontend/README.md](frontend/README.md) for details. diff --git a/clips/templates/clips/index.html b/clips/templates/clips/index.html index e08393d..4fe4e5e 100644 --- a/clips/templates/clips/index.html +++ b/clips/templates/clips/index.html @@ -2,11 +2,14 @@ {% comment %} The host page, served by Django since port-plan step 9. -It was `frontend/public/index.html`, served by shadow-cljs's `:dev-http`, and that -key is gone. The bundle is unchanged: shadow-cljs writes it into -`static/arthur/js` and staticfiles serves it from there, so `manage.py runserver` -and `shadow-cljs watch app` are the whole dev loop with nothing copying files -between them. +It carries no styles of its own any more. They are `static/arthur/app.css`, which +staticfiles serves from the same tree as the bundle — the page grew a five-pane +application chrome and "the styles" stopped being a thing you read in passing on +the way to the markup. + +The bundle is unchanged: shadow-cljs writes it into `static/arthur/js` and +staticfiles serves it from there, so `manage.py runserver` and `shadow-cljs watch +app` are the whole dev loop with nothing copying files between them. The CSRF token is rendered so that Django sets its cookie, which is what `arthur.fx.http` reads to write the `X-CSRFToken` header. Saves are ordinary POSTs @@ -17,67 +20,7 @@ and PUTs with ordinary CSRF protection — no endpoint in this app is exempt. arthur - + {% csrf_token %} diff --git a/frontend/README.md b/frontend/README.md index 62a7946..fff8e0f 100644 --- a/frontend/README.md +++ b/frontend/README.md @@ -88,18 +88,60 @@ cd frontend && mise exec -- npx shadow-cljs watch app ``` Then open ****. Django serves the page from -`clips/templates/clips/index.html`, and staticfiles serves the bundle out of -`static/arthur/js`, where `shadow-cljs` already writes it — so nothing copies files -between the two. +`clips/templates/clips/index.html`, its styles from `static/arthur/app.css`, and +the bundle out of `static/arthur/js`, where `shadow-cljs` already writes it — so +nothing copies files between the two. + +### The window + +One screen, five panes, no scrolling page. `src/arthur/ui/shell.cljs` is the grid +and nothing else; each pane owns its own subscriptions. + +``` + top the document: its name and last status, export, new / open / save + left media pool — the open document's symbols, and footage on the server + centre the palette strip (16 slots) above the stage + right inspector — the clip, the selected node, the tracked objects + bottom timeline — transport, ruler, a row per node +``` + +**It opens on a blank document**, and **new** makes another one. Nothing is +loaded until it is asked for. + +**Whole documents live under `open ▾`, not in the media pool**, and the split is +load-bearing rather than tidy. Opening a project REPLACES the stage; everything +in the pool is a thing to put ON it. Listing documents beside the symbols inside +one of them makes them read as two kinds of the same thing. The menu lists the +projects the server holds; the built-in scenes are under their own heading, +italic, and are not projects — they are compiled into the bundle and the server +has never heard of them. + +Selection lives in app-db under `:ui`, as `[:node ]`, +`[:timeline ]` or `[:subject|:feature|:group ]` — four panes ask what is +selected, and a ratom private to one of them can only be shared by making the +other three require it. + +**Drop a video on the media pool** and it uploads, extracts and goes straight on +into detection. Dragging a symbol out of the pool onto the stage places an +instance of it at the playhead. + +The timeline's rows are the open clip's nodes, front-most first, with a dot per +keyframe and a bar over the frames the node exists on; a dense channel is hatched +rather than ticked, because one value per frame is a solid block that says less +than the bar does. Opening a row shows its channels; opening a **symbol** row +shows the timeline it instances, with every frame number mapped back into the +stage's own frame space — see the namespace docstring in `ui/timeline.cljs`, which +is where that mapping is argued. ### Paint sketch -Click **new polygon**, place at least three vertices on the stage, then click -**finish shape**. Select a shape to drag its vertices. Scrub to another frame and -click **new drawing key** to copy the visible outline there; the previous drawing -holds until that key. The numbered drawing-key buttons jump to editable keys. -The transition control between two drawing keys can switch that gap between a -hold and linear vertex tweening. Other gaps keep their own timing. Tweening works +Pick a tone from the palette strip, click **polygon**, place at least three +vertices on the stage, then click **finish**. Select a shape — on the stage, or by +its timeline row — to drag its vertices. Scrub to another frame and click +**drawing key here** in the inspector to copy the visible outline there; the +previous drawing holds until that key. The numbered key buttons jump to editable +keys. The transition control between two drawing keys can switch that gap between +a hold and linear vertex tweening. Other gaps keep their own timing. Tweening works best when the same vertex keeps the same meaning in every drawing. Paint shapes use the timeline clock directly, so the roto exposure grid does not delay a drawing key or step its @@ -109,7 +151,7 @@ tween. Use the project **save** button to persist the drawings. has used since step 5, when shadow-cljs's `:dev-http` did no directory-index resolution and the suite learned to ask for the file. -Four built-in clips, on buttons in the transport: +Four built-in clips, under **built-in examples** in the open menu: | | | | --- | --- | @@ -125,14 +167,14 @@ The demo scene itself is `src/arthur/demo/scene.edn`. Both the synthetic take and real footage use `src/arthur/flow/take.cljs` for the measurement order and `src/arthur/flow/freeze.cljs` for the landmark-to-channel conversion. -**stage 8625** loads the locally saved `IMG_8625.MOV` project and places its +**8625 stage study**, in the open menu, loads the locally saved `IMG_8625.MOV` project and places its post-processed timeline twice. The stage layout is `src/arthur/demo/stage_8625.edn`: the right picture and sound start at frame 48, and the two pictures overlap slightly in stage space. Audio has its own timeline nodes, linked to the picture instances but with independent spans and gain channels. The right sound swells and pans across the stage, then fades out at frame 260 while its picture continues to -frame 280. The button needs that saved 8625 project in the local server database. +frame 280. The row needs that saved 8625 project in the local server database. ### Projects and the EDN fixtures @@ -148,16 +190,17 @@ the same ClojureScript clip. The intended editor creates and changes that in-memory clip directly: a project browser and **new stage** action, timeline instance placement, node and channel editors, then the existing save path. EDN remains useful for checked-in examples -and reproducible studies. The current UI has save and open, but no project -browser, blank-stage action, or authoring controls yet; open chooses the most -recent project. +and reproducible studies. The UI now has the blank-stage action (**new**), a +project browser (`open ▾`), and placement by dragging a symbol out of the media +pool; node and channel editors are still to come — the inspector reports a +channel's shape but has nowhere to change its values. ### Real footage -Choose a video in the **footage** file input. The server probes it, re-encodes it +Drop a video on the media pool, or use its **+** button. The server probes it, re-encodes it to an H.264 proxy and a raw stream of the same coded frames, pulls WAV audio and one tracing JPEG per frame, -then makes the resulting footage selectable. Click **load frames** to detect and -freeze it. Extraction progress is currently read from `/api/extractions/`; a +then makes the resulting footage selectable and runs detection on it. **roto**, in +the pool's header, does the same for footage that is already there. Extraction progress is currently read from `/api/extractions/`; a future WebSocket can push the same job state. The uploaded bytes, extraction job, and decoded footage have separate records, so the same uploaded video can be reopened without decoding it again. @@ -216,14 +259,14 @@ the root — all of it is extraction output, and tier 3 does not belong in the r Loading detects one face per frame, measures the mouth, eyes and brows from landmarks and the teeth from source pixels, then freezes them into channels, -and adds a button for the footage clip. Detection happens once when you load; +and opens the footage clip. Detection happens once when you load; playback only resolves channels and paints. Frames without a detection remain marked absent even though their neighbouring poses are used to condition the track. The scene now records stable subject and feature IDs and explicit eye pairs; dense channels can mark one feature absent while another is observed. Current MediaPipe loading supplies only the full-face detection mask. The stage stays 320×200 regardless of the footage dimensions. Real -footage starts at the source picture rate. The **picture fps** buttons sample the +footage starts at the source picture rate. The **picture** buttons in the inspector sample the frozen roto at lower rates while the source track, duration and audio clock stay unchanged. Picking frames to trace into cels is a separate future editing step. **save** also stores the detection mask, dense landmarks and raw RGBA mouth crops @@ -252,7 +295,7 @@ runs the old JS tool on 8777, and the two are meant to run side by side. ## Saving -**save** and **open** in the transport. A save has three ordered stages: +**new**, **open** and **save** in the top bar. A save has three ordered stages: is the tier split: 1. the **analysis** record, so every block stored afterwards can name the detector @@ -269,10 +312,11 @@ unchanged document says `0 leaves · 0 blocks`, which is both halves of the addressing working at once — an unchanged leaf keeps its version, and a content-addressed block is already there. -Two things are deliberately visible as failures. Saving `swarm` is refused, -because its blocks have hand-written names and a document may only name content -addresses. And **open** takes the most recently updated project and shows its first -clip: there is no project browser, and the store holds one clip at a time. +Saving `swarm` is deliberately visible as a failure: its blocks have +hand-written names and a document may only name content addresses. + +`open ▾` lists every project the server holds, newest first, and shows the first +clip of whichever one is picked — the store holds one clip at a time. ## The oracle, which is finished diff --git a/frontend/src/arthur/core.cljs b/frontend/src/arthur/core.cljs index e359902..90848b4 100644 --- a/frontend/src/arthur/core.cljs +++ b/frontend/src/arthur/core.cljs @@ -7,9 +7,11 @@ [arthur.events.footage :as footage] [arthur.events.playback] [arthur.events.paint] - [arthur.events.project] + [arthur.events.project :as project] + [arthur.events.ui] [arthur.subs.playback] [arthur.subs.render] + [arthur.subs.ui] [arthur.ui.player :as player] [arthur.ui.shell :as shell] [re-frame.core :as rf] @@ -28,6 +30,10 @@ (defn init [] (rf/dispatch-sync [::init]) + ;; A blank document, before the first render. Synchronous for the same reason + ;; `::init` is: the shell reads the clip's dimensions, and mounting against a + ;; db that has no clip in it yet is a frame of nothing for no reason. + (rf/dispatch-sync [::project/new]) ;; What the server already holds, asked for once. The list is small — a row per ;; ingested take — and having it before the first click is what lets the footage ;; picker be a picker rather than a path to type. diff --git a/frontend/src/arthur/db.cljs b/frontend/src/arthur/db.cljs index a116505..5ad50aa 100644 --- a/frontend/src/arthur/db.cljs +++ b/frontend/src/arthur/db.cljs @@ -54,7 +54,14 @@ (def default {;; --- the document --- - :clip/current :take + ;; + ;; NOTHING IS LOADED. `core/init` dispatches `::project/new` before the first + ;; render, so the app opens on a blank stage rather than on whichever built-in + ;; scene happened to be convenient — the demos, the swarm and the two takes are + ;; rows in the media pool like anything else, and reference material is not a + ;; default. The values below are what a blank document is; they are replaced by + ;; that dispatch and exist so this map is a valid db on its own. + :clip/current nil :paint/revision 0 :palette :arthur/default ; a NAME; the ramp itself is project data @@ -64,7 +71,10 @@ ;; footage's. That is what deleting `makeXform` buys — the framing became a ;; transform on a node, so nothing downstream of the freeze knows the frame ;; size — and it is why ui/player no longer hardcodes 320x200. - :clip (select-keys (clip-entry :take) [:fps :frames :width :height :audio :display-fps]) + :clip (let [c (domain-clip/blank)] + {:fps (:fps c) :frames (domain-clip/frames c) + :width (:width c) :height (:height c) + :audio nil :display-fps (:fps c)}) ;; Which ingested footage to detect, and what the last load said. The list ;; comes from the server — tier 3 is the backend's since step 9 — so there is @@ -77,6 +87,10 @@ ;; is what will make staleness self-healing once there is a broadcast to miss. :project {:id nil :cid nil :name nil :seq nil :busy? false :status nil} + ;; What the server holds, for the open menu. A list of rows and nothing more — + ;; opening one fetches the document itself. + :projects {:items [] :loading? false} + ;; --- transport --- ;; ;; The playhead is in app-db like everything else. An earlier draft of @@ -105,7 +119,40 @@ ;; Both for profiling: loop so a run at 4x lasts longer than the ;; clip, mute so sitting in one does not require enduring it. :loop? false - :muted? false}}) + :muted? false} + + ;; --- the editor's own state --- + ;; + ;; IN app-db, not in ratoms beside the components that read it. What is + ;; selected is asked by four panes at once — the params pane renders it, the + ;; timeline highlights its row, the stage draws its handles, the palette says + ;; which tone a new shape gets — and a `defonce` atom private to one namespace + ;; can only be shared by making the other three require that namespace for its + ;; state. It is also small and authored, which is the bar `arthur.db` sets. + ;; + ;; `:selection` is a vector whose first element says what kind of thing it + ;; names, so a pane dispatches on it rather than on which of several + ;; "selected-x" keys happens to be non-nil: + ;; + ;; [:node ] a shape or a placement + ;; [:timeline ] a timeline, root or library + ;; [:subject ] [:feature ] [:group ] a tracked object + ;; + ;; `:draft` is the polygon being clicked out, flat [x y x y …] as geometry is + ;; stored everywhere. `:expanded` holds timeline row PATHS — a path and not a + ;; node id, because one symbol placed twice is two rows that open separately. + ;; + ;; `:knobs` holds a generated setting's value WHILE THE REGENERATION IS IN + ;; FLIGHT, keyed by [scope id knob]. Moving a slider dispatches a preview that + ;; re-freezes blocks asynchronously, so until it lands the clip still reports + ;; the old value — and a slider reading from the clip would spring back under + ;; the user's finger on every frame of the drag. + :ui {:selection nil + :tone :skin-base + :tool nil + :draft [] + :knobs {} + :expanded #{}}}) (def rates "The transport's rates — all of them `playbackRate` on the audio element, so diff --git a/frontend/src/arthur/domain/clip.cljs b/frontend/src/arthur/domain/clip.cljs index 1a80d6d..16a6a6e 100644 --- a/frontend/src/arthur/domain/clip.cljs +++ b/frontend/src/arthur/domain/clip.cljs @@ -83,6 +83,59 @@ [clip] (:nodes (root clip))) +(def ^:const blank-frames + "How long a new document is before anything says otherwise. Four seconds at 30, + which is long enough to key something into and short enough to scrub by hand." + 120) + +(defn blank + "A new, empty document. + + `:nodes` is empty rather than seeded with a layer, because an empty timeline is + a true statement and a layer nobody asked for is one more thing to delete. The + tracking maps are present and empty for the same reason `clip-keys` exists: a + field that is sometimes absent is a field every reader needs a fallback for." + [] + {:name "untitled" + :fps 30 + :width 320 :height 200 + :subjects {} :features {} :groups {} + :timelines {root-id {:id root-id :frames blank-frames :nodes {}}}}) + +(defn place-symbol + "An instance of library timeline `tid`, on the root timeline, at `frame`. + + THE UUID IS AN ARGUMENT. A placement's identity is the key it has in the node + map — it is what `:linked-to`, an export target and a saved leaf all name — so + generating one in here would make this function's result depend on when it was + called, and this namespace is the pure one. `demo/stage_8625.edn` authors its + placements' uuids by hand for the same reason, in more words. + + The instance's own time starts where it was dropped: `:at frame` with `:in 0` + means local frame 0 of the symbol plays on `frame` of the stage, which is what + dragging something onto a playhead is asking for. `:span` runs to the end of + the root's frame space rather than to the symbol's length, because a symbol + shorter than the space it is placed in should hold its last frame rather than + disappear." + [clip tid frame uuid [x y]] + (let [target (timeline clip tid) + end (frames clip)] + (if (or (nil? target) (= root-id tid) (nil? frame) (neg? frame) (>= frame end)) + clip + (update-root + clip assoc-in [:nodes uuid] + {:id uuid + :name (name tid) + :kind :symbol + :of tid + :parent nil + ;; Lexicographic draw order, as `domain/paint` does it: a placement made + ;; later sits above one made earlier, and neither has to renumber. + :z (str "z" (js/Date.now) "-" (name tid)) + :span [frame end] + :time {:mode :map :at frame :in 0 :rate 1} + :channels {[:xform :pos] {:animated? false :value [x y]}}})))) + (defn- transform-op "Put a symbol's already resolved mark into its instance's parent space." [op m path] diff --git a/frontend/src/arthur/events/edit.cljs b/frontend/src/arthur/events/edit.cljs new file mode 100644 index 0000000..42ce07d --- /dev/null +++ b/frontend/src/arthur/events/edit.cljs @@ -0,0 +1,26 @@ +(ns arthur.events.edit + "The one way an event changes the loaded document. + + Three things have to happen together and the bug is any one of them being + forgotten: the clip in `footage/store` is edited, the id app-db refers to it by + is updated — `edit-clip!` may INSTALL A COPY, because a built-in clip is a + delayed value that must stay reusable — and `:paint/revision` is bumped so the + layer-3 subs downstream of `::render/clip` recompute. The revision exists + because the clip itself is behind a handle: app-db holds an id, the id does not + change when the document does, and a sub keyed only on the id would never see + the edit. + + It started life private inside `events/paint`, which was right while polygons + were the only thing anyone could edit. They are not." + (:require [arthur.footage.store :as store])) + +(defn edit + "Apply `f` to the loaded clip and return the new db." + [db f] + (let [id (store/edit-clip! (:clip/current db) f)] + (if id + (-> db + (assoc :clip/current id) + (update :paint/revision (fnil inc 0)) + (update :project merge {:status "edited · unsaved"})) + db))) diff --git a/frontend/src/arthur/events/footage.cljs b/frontend/src/arthur/events/footage.cljs index 2218446..cb793c1 100644 --- a/frontend/src/arthur/events/footage.cljs +++ b/frontend/src/arthur/events/footage.cljs @@ -337,9 +337,14 @@ (rf/reg-event-fx ::uploaded (fn [{:keys [db]} [_ footage-id]] + ;; STRAIGHT INTO DETECTION. Dropping a video on the media pool is one + ;; intention — get this footage into the tool — and stopping after extraction + ;; to wait for a second click only means finding the button that does the + ;; obvious next thing. `::load` reads `:chosen`, which this event has just + ;; written, because `:dispatch` runs after the db update. {:db (update db :footage merge {:loading? false :chosen footage-id - :status "video extracted — load frames to analyze"}) - :dispatch [::refresh]})) + :status "video extracted"}) + :dispatch-n [[::refresh] [::load]]})) (rf/reg-event-fx ::refresh diff --git a/frontend/src/arthur/events/paint.cljs b/frontend/src/arthur/events/paint.cljs index 4b92003..9442d4b 100644 --- a/frontend/src/arthur/events/paint.cljs +++ b/frontend/src/arthur/events/paint.cljs @@ -1,33 +1,26 @@ (ns arthur.events.paint + "Polygon edits. Every one of them is `edit/edit` plus a pure `domain/paint` + function, which is the shape every document edit in this app should have." (:require [arthur.domain.paint :as paint] - [arthur.footage.store :as store] + [arthur.events.edit :as edit] [re-frame.core :as rf])) -(defn- edit [db f] - (let [id (store/edit-clip! (:clip/current db) f)] - (if id - (-> db - (assoc :clip/current id) - (update :paint/revision (fnil inc 0)) - (update :project merge {:status "paint edited · unsaved"})) - db))) - (rf/reg-event-db ::new-shape (fn [db [_ id points color]] - (edit db #(paint/new-shape % id (get-in db [:playback :frame]) points color)))) + (edit/edit db #(paint/new-shape % id (get-in db [:playback :frame]) points color)))) (rf/reg-event-db ::add-key (fn [db [_ id]] - (edit db #(paint/add-key % id (get-in db [:playback :frame]))))) + (edit/edit db #(paint/add-key % id (get-in db [:playback :frame]))))) (rf/reg-event-db ::set-vertex (fn [db [_ id key-frame vertex point]] - (edit db #(paint/set-vertex % id key-frame vertex point)))) + (edit/edit db #(paint/set-vertex % id key-frame vertex point)))) (rf/reg-event-db ::set-segment-interp (fn [db [_ id key-frame interp]] - (edit db #(paint/set-segment-interp % id key-frame interp)))) + (edit/edit db #(paint/set-segment-interp % id key-frame interp)))) diff --git a/frontend/src/arthur/events/playback.cljs b/frontend/src/arthur/events/playback.cljs index b0afca2..14b3106 100644 --- a/frontend/src/arthur/events/playback.cljs +++ b/frontend/src/arthur/events/playback.cljs @@ -106,6 +106,13 @@ ;; sizes, and the raster the loop paints into is the clip's, not ;; the app's. (assoc :clip (select-keys clip [:fps :frames :width :height :audio :display-fps])) + ;; The document's identity goes with it. A built-in scene has no + ;; project on the server, so this CLEARS the id rather than keeping + ;; the last one — saving a fixture must create a document of its + ;; own, not overwrite whatever happened to be open before it. + (assoc :project {:id nil :cid (:cid clip) :name (:label clip) + :seq nil :busy? false + :status "built-in example · not a saved project"}) (assoc-in [:playback :frame] 0) (assoc-in [:playback :playing?] false)) ::pause! nil diff --git a/frontend/src/arthur/events/project.cljs b/frontend/src/arthur/events/project.cljs index 49e9a03..f2ca40a 100644 --- a/frontend/src/arthur/events/project.cljs +++ b/frontend/src/arthur/events/project.cljs @@ -21,7 +21,8 @@ Nothing here touches app-db except through events. The promise chain lives in an fx, which is the only thing in this namespace that is not pure." - (:require [arthur.domain.clip :as clip] + (:require [arthur.db :as db] + [arthur.domain.clip :as clip] [arthur.audio.mix :as mix] [arthur.demo.stage :as stage] [arthur.domain.feature :as feature] @@ -160,6 +161,19 @@ (js/console.error error) (rf/dispatch [::failed (or (ex-message error) (str error))]))))))) +(rf/reg-fx + ::list! + (fn [_] + (-> (http/GET "/api/projects") + (.then (fn [^js listed] + (rf/dispatch [::listed + (mapv (fn [^js row] + {:id (.-id row) :name (.-name row) + :seq (.-seq row) :updated (.-updated row)}) + (array-seq (.-projects listed)))]))) + (.catch (fn [error] + (rf/dispatch [::failed (or (ex-message error) (str error))])))))) + (rf/reg-fx ::open! (fn [id] @@ -330,6 +344,7 @@ :request request}}))))) (rf/reg-sub ::regeneration (fn [db _] (:regeneration db))) +(rf/reg-sub ::listing (fn [db _] (:projects db))) (rf/reg-event-fx ::settings-previewed @@ -345,6 +360,46 @@ ;; --------------------------------------------------------------------------- ;; events +(def blank-audio + "A new document still needs a clock. + + The frame is derived from an audio element and from nothing else — see + `arthur.clock` — so a stage with no sound has no time and `play` is a button + that cannot work. The synthetic take borrows this same asset for exactly this + reason. When a document gets audio of its own, it replaces this." + "/static/arthur/audio.wav") + +(defn blank-entry + "A blank clip, in the shape `footage/store` and the transport expect." + [] + (let [c (clip/blank)] + {:label "untitled" :clip c :store nil + :audio blank-audio + ;; A content id of its own from the start: `::save!` addresses the clip by + ;; it, and two untitled documents saved from two tabs are two documents. + :cid (str (random-uuid)) + :display-fps (:fps c) + :frames (clip/frames c) + :fps (:fps c) :width (:width c) :height (:height c)})) + +(rf/reg-event-fx + ::new + (fn [{:keys [db]} _] + ;; A document with no id on the server, so the next `save` creates one. This + ;; is also what the app opens on: nothing is loaded until something is asked + ;; for, and the built-in scenes are rows in the media pool like anything else. + (let [entry (blank-entry) + id (store/install! entry "new")] + {:db (-> db + (assoc :clip/current id + :clip (select-keys entry [:fps :frames :width :height :audio :display-fps])) + (assoc :project {:id nil :cid (:cid entry) :name nil :seq nil + :busy? false :status "new document"}) + (assoc :ui (:ui db/default)) + (assoc-in [:playback :frame] 0) + (assoc-in [:playback :playing?] false)) + ::pb/pause! nil}))) + (rf/reg-event-fx ::save (fn [{:keys [db]} _] @@ -359,13 +414,26 @@ :clip clip}})))) (rf/reg-event-fx - ::open + ::list (fn [{:keys [db]} _] + {:db (assoc-in db [:projects :loading?] true) + ::list! nil})) + +(rf/reg-event-db + ::listed + (fn [db [_ rows]] (assoc db :projects {:items rows :loading? false}))) + +(rf/reg-event-fx + ::open + (fn [{:keys [db]} [_ id]] + ;; `id` names which project. Without one it is the open document's own id, and + ;; without that the most recently updated — which is what "open" meant when + ;; there was no list to pick from. (if (:busy? (:project db)) {} {:db (update db :project merge {:busy? true :status "opening…"}) ::pb/pause! nil - ::open! (:id (:project db))}))) + ::open! (or id (:id (:project db)))}))) (rf/reg-event-fx ::load-stage diff --git a/frontend/src/arthur/events/ui.cljs b/frontend/src/arthur/events/ui.cljs new file mode 100644 index 0000000..14dd657 --- /dev/null +++ b/frontend/src/arthur/events/ui.cljs @@ -0,0 +1,79 @@ +(ns arthur.events.ui + "Selection, the active tone, the polygon being drawn, and which timeline rows + are open. + + All of it is `assoc-in` under `:ui`. There is no effect in this namespace and + there should not be one: an editor's own state is the cheapest thing in the + app to change and the most expensive to have two copies of." + (:require [arthur.domain.clip :as clip] + [arthur.events.edit :as edit] + [arthur.events.paint :as paint] + [re-frame.core :as rf])) + +(rf/reg-event-db + ::select + (fn [db [_ selection]] (assoc-in db [:ui :selection] selection))) + +(rf/reg-event-db + ::set-tone + (fn [db [_ tone]] (assoc-in db [:ui :tone] tone))) + +(rf/reg-event-db + ::toggle-row + (fn [db [_ path]] + (update-in db [:ui :expanded] #(if (contains? % path) (disj % path) (conj % path))))) + +;; --------------------------------------------------------------------------- +;; drawing a polygon +;; +;; Three events and a vector of numbers. The draft is in app-db rather than in a +;; ratom because the stage draws it, the palette colours it and the params pane +;; reports its vertex count — and because a half-drawn shape surviving a hot +;; reload is worth more than the handful of dispatches it costs. Clicks are rare; +;; this is not the drag path. + +(rf/reg-event-db + ::begin-polygon + (fn [db _] (update db :ui merge {:tool :polygon :draft [] :selection nil}))) + +(rf/reg-event-db + ::cancel-polygon + (fn [db _] (update db :ui merge {:tool nil :draft []}))) + +(rf/reg-event-db + ::add-draft-point + (fn [db [_ x y]] + (if (= :polygon (get-in db [:ui :tool])) + (update-in db [:ui :draft] into [x y]) + db))) + +(rf/reg-event-fx + ::finish-polygon + (fn [{:keys [db]} _] + (let [draft (get-in db [:ui :draft])] + (if (< (count draft) 6) + {} + ;; `random-uuid` is the one impurity in this namespace, and it is here + ;; rather than in `domain/paint` for the reason `clip/place-symbol` spells + ;; out: a node's id is its identity in the saved document, so the pure + ;; layer must be handed one rather than invent one. If replaying the event + ;; log ever has to reproduce a document exactly, this becomes a cofx. + (let [id (keyword (str "paint-" (random-uuid)))] + {:db (update db :ui merge {:tool nil :draft [] :selection [:node :main id]}) + :dispatch [::paint/new-shape id draft (get-in db [:ui :tone])]}))))) + +(rf/reg-event-db + ::set-knob + (fn [db [_ scope id knob value]] + (assoc-in db [:ui :knobs [scope id knob]] value))) + +;; --------------------------------------------------------------------------- +;; the pool, onto the stage + +(rf/reg-event-db + ::place-symbol + (fn [db [_ tid [x y]]] + (let [uuid (random-uuid)] + (-> db + (edit/edit #(clip/place-symbol % tid (get-in db [:playback :frame]) uuid [x y])) + (assoc-in [:ui :selection] [:node :main uuid]))))) diff --git a/frontend/src/arthur/subs/ui.cljs b/frontend/src/arthur/subs/ui.cljs new file mode 100644 index 0000000..4941f2c --- /dev/null +++ b/frontend/src/arthur/subs/ui.cljs @@ -0,0 +1,28 @@ +(ns arthur.subs.ui + "Layer-2 extractors over the editor's own state, plus the one layer-3 that + resolves a selection to the thing it names. + + Cheap by construction, like `subs/playback`: each reads a path and returns a + value, so clicking a swatch notifies the swatches and nothing else." + (:require [arthur.subs.render :as render] + [re-frame.core :as rf])) + +(rf/reg-sub ::selection (fn [db _] (get-in db [:ui :selection]))) +(rf/reg-sub ::tone (fn [db _] (get-in db [:ui :tone]))) +(rf/reg-sub ::tool (fn [db _] (get-in db [:ui :tool]))) +(rf/reg-sub ::draft (fn [db _] (get-in db [:ui :draft]))) +(rf/reg-sub ::expanded (fn [db _] (get-in db [:ui :expanded]))) +(rf/reg-sub ::knobs (fn [db _] (get-in db [:ui :knobs]))) + +(rf/reg-sub + ::selected-node + :<- [::selection] + :<- [::render/clip] + (fn [[selection clip] _] + ;; `[timeline-id node-id node]`, or nil. Returned as a triple rather than as + ;; the node alone because every caller that wants the node also wants to know + ;; where it lives — an edit names the timeline, and a bare node has forgotten. + (when (and clip (= :node (first selection))) + (let [[_ tid id] selection] + (when-let [n (get-in clip [:timelines tid :nodes id])] + [tid id n]))))) diff --git a/frontend/src/arthur/ui/openmenu.cljs b/frontend/src/arthur/ui/openmenu.cljs new file mode 100644 index 0000000..5f10bd3 --- /dev/null +++ b/frontend/src/arthur/ui/openmenu.cljs @@ -0,0 +1,78 @@ +(ns arthur.ui.openmenu + "File → Open. The projects the server holds, and nothing else. + + IT IS NOT THE MEDIA POOL. A project is a whole document — opening one replaces + the stage — and the pool is the open document's own library. Listing documents + beside the symbols inside one of them makes them look like two kinds of the + same thing, which is the confusion this split exists to end. + + The built-in scenes are under their own heading and are NOT projects: they are + compiled into the bundle, the server has never heard of them, and `demo/scene` + and the swarm exist to exercise and to profile the model rather than to be + worked on. They are here because there is nowhere else for a fixture to live, + and they are labelled so that nothing about them reads as a saved document." + (:require [arthur.db :as db] + [arthur.events.playback :as pb] + [arthur.events.project :as project] + [arthur.subs.playback :as playback] + [re-frame.core :as rf] + [reagent.core :as r])) + +(defn- when-said + "The ISO stamp the server sends, as a date. Truncated rather than formatted: + a list of documents wants to be sorted and scanned, not read to the second." + [iso] + (when iso (subs (str iso) 0 10))) + +(defn- item + "One row. `example?` is not decoration: the server is full of documents called + \"take\", so \"the row labelled take\" does not identify one thing — the class is + what lets a reader, and `test/browser`, tell a saved project from the fixture + that shares its name." + [{:keys [label sub on-click disabled? example?]}] + [:button {:class (str "menu-item" (when example? " example")) + :on-click on-click :disabled (boolean disabled?) :title label} + label + (when sub [:span.sub sub])]) + +(defn view [] + (r/with-let [open? (r/atom false)] + (let [{:keys [items loading?]} @(rf/subscribe [::project/listing]) + busy? (:busy? @(rf/subscribe [::playback/project])) + choose! (fn [event] (reset! open? false) (rf/dispatch event))] + [:div.menu-wrap + [:button {:disabled busy? + :class (when @open? "on") + :on-click (fn [] + (when-not @open? (rf/dispatch [::project/list])) + (swap! open? not))} + "open ▾"] + (when @open? + [:<> + ;; A full-page catcher behind the panel, so clicking anywhere else + ;; dismisses it. Cheaper and more predictable than a document-level + ;; listener that has to be added, removed and told to ignore the click + ;; that opened the menu. + [:div.menu-scrim {:on-click #(reset! open? false)}] + [:div.menu + [:h2 "projects"] + (cond + loading? [:div.dim "…"] + (empty? items) [:div.dim "none saved yet"] + :else + (doall + (for [{:keys [id name seq updated]} items] + ^{:key id} + [item {:label (or name "untitled") + :sub (str "r" seq " · " (when-said updated)) + :on-click #(choose! [::project/open id])}]))) + [:h2 "built-in examples"] + [:div.dim "compiled in, not saved documents"] + (doall + (for [[id {:keys [label]}] db/clips] + ^{:key id} + [item {:label label :example? true + :on-click #(choose! [::pb/select-clip id])}])) + [item {:label "8625 stage study" :example? true + :sub "composed from a locally saved project" + :on-click #(choose! [::project/load-stage])}]]])]))) diff --git a/frontend/src/arthur/ui/paint.cljs b/frontend/src/arthur/ui/paint.cljs deleted file mode 100644 index a2c2ebc..0000000 --- a/frontend/src/arthur/ui/paint.cljs +++ /dev/null @@ -1,149 +0,0 @@ -(ns arthur.ui.paint - "Polygon authoring overlay. The raster canvas remains the playback sink; SVG - supplies only editor handles and an unfinished outline." - (:require [arthur.domain.channel :as channel] - [arthur.domain.paint :as paint] - [arthur.domain.palette :as palette] - [arthur.events.paint :as events] - [arthur.events.playback :as pb] - [arthur.subs.playback :as playback] - [arthur.subs.render :as render] - [re-frame.core :as rf] - [reagent.core :as r])) - -(defonce ^:private selected (r/atom nil)) -(defonce ^:private drawing? (r/atom false)) -(defonce ^:private draft (r/atom [])) -(defonce ^:private tone (r/atom :skin-base)) -(defonce ^:private dragging (atom nil)) - -(defn- point [event w h] - (let [box (.getBoundingClientRect (.-currentTarget event))] - [(-> (/ (* (- (.-clientX event) (.-left box)) w) (.-width box)) - js/Math.round (max 0) (min (dec w))) - (-> (/ (* (- (.-clientY event) (.-top box)) h) (.-height box)) - js/Math.round (max 0) (min (dec h)))])) - -(defn- pairs [pts] - (mapv vec (partition 2 pts))) - -(defn- points-text [pts] - (apply str (interpose " " (map (fn [[x y]] (str x "," y)) (pairs pts))))) - -(defn- begin! [] - (reset! selected nil) - (reset! draft []) - (reset! drawing? true)) - -(defn- finish! [] - (when (>= (count @draft) 6) - (let [id (keyword (str "paint-" (random-uuid)))] - (rf/dispatch [::events/new-shape id @draft @tone]) - (reset! selected id) - (reset! draft []) - (reset! drawing? false)))) - -(defn toolbar [] - (let [clip @(rf/subscribe [::render/clip]) - frame @(rf/subscribe [::playback/frame]) - shapes (paint/shapes clip) - ids (set (map first shapes)) - id (when (contains? ids @selected) @selected) - shape (get-in clip [:timelines :main :nodes id]) - geom (get-in shape [:channels paint/geometry]) - active (when geom (paint/active-frame geom frame)) - keys (when geom (sort (keys (:keys geom)))) - next-key (first (filter #(> % active) keys)) - tween? (= :linear (channel/segment-interp geom active))] - [:div.paint-tools - [:div.row - [:strong "paint"] - [:button {:class (when @drawing? "on") :on-click begin!} "new polygon"] - (when @drawing? - [:button {:disabled (< (count @draft) 6) :on-click finish!} "finish shape"]) - (when @drawing? - [:button {:on-click #(do (reset! drawing? false) (reset! draft []))} "cancel"]) - [:label "colour " - [:select {:value (name @tone) - :on-change #(reset! tone (keyword (.. % -target -value)))} - (for [{:keys [name hex]} (rest palette/entries)] - ^{:key name} [:option {:value (clojure.core/name name)} - (str (clojure.core/name name) " " hex)])]]] - [:div.row - [:label "shape " - [:select {:value (if id (name id) "") - :on-change #(reset! selected (when-not (= "" (.. % -target -value)) - (keyword (.. % -target -value))))} - [:option {:value ""} "select"] - (for [[shape-id node] shapes] - ^{:key shape-id} [:option {:value (name shape-id)} (:name node)])]] - (when id - [:button {:disabled (or (< frame (first (:span shape))) - (contains? (:keys geom) frame)) - :on-click #(rf/dispatch [::events/add-key id])} - "new drawing key"]) - (when (and id next-key) - [:label (str "key " active " → " next-key " ") - [:select {:value (name (or (channel/segment-interp geom active) :hold)) - :on-change #(rf/dispatch [::events/set-segment-interp id active - (keyword (.. % -target -value))])} - [:option {:value "hold"} "hold"] - [:option {:value "linear"} "tween shape"]]])] - (when (seq keys) - [:div.row - [:span "drawing keys "] - (for [f keys] - ^{:key f} - [:button {:class (when (= frame f) "on") - :on-click #(rf/dispatch [::pb/seek f])} - (str f)])]) - [:div.hint - (cond - @drawing? (str "Click vertices on the stage, then Finish shape. " - (quot (count @draft) 2) " points") - (and id tween? (not (contains? (:keys geom) frame))) - "Tweening between drawings. Add a drawing key here, or jump to a key to edit its vertices." - id (str "Drag vertices to edit drawing key " active - ". New drawing key copies the visible shape at this frame. The transition control changes only the selected gap.") - :else "Create a polygon, or select one to edit its drawing keys.")]])) - -(defn overlay [w h zoom] - (let [clip @(rf/subscribe [::render/clip]) - frame @(rf/subscribe [::playback/frame]) - shape (get-in clip [:timelines :main :nodes @selected]) - geom (get-in shape [:channels paint/geometry]) - active (when geom (paint/active-frame geom frame)) - visible? (and shape (let [[start end] (:span shape)] (<= start frame) (< frame end))) - pts (when visible? (channel/value-at geom frame)) - editable? (or (not= :linear (channel/segment-interp geom active)) - (contains? (:keys geom) frame))] - [:svg.paint-overlay - {:width (* zoom w) :height (* zoom h) - :view-box (str "0 0 " w " " h) - :on-pointer-down (fn [event] - (when @drawing? - (let [[x y] (point event w h)] - (swap! draft into [x y])))) - :on-pointer-move (fn [event] - (when-let [[id key-frame vertex] @dragging] - (rf/dispatch [::events/set-vertex id key-frame vertex - (point event w h)]))) - :on-pointer-up (fn [_] (reset! dragging nil)) - :on-pointer-cancel (fn [_] (reset! dragging nil))} - (when (seq @draft) - [:polyline {:points (points-text @draft) :fill "none" - :stroke "#d0ba86" :stroke-width 1}]) - (when (and visible? (not @drawing?) pts) - [:g - [:polygon {:points (points-text pts) :fill "none" - :stroke "#e6ca8b" :stroke-width 1}] - (for [[i [x y]] (map-indexed vector (pairs (when editable? pts)))] - ^{:key i} - [:circle {:cx x :cy y :r 2.6 :fill "#fff1be" - :stroke "#161820" :stroke-width 0.7 - :on-pointer-down (fn [event] - (.stopPropagation event) - (.preventDefault event) - (.setPointerCapture (.-currentTarget event) - (.-pointerId event)) - (reset! dragging [@selected active i]))}])])])) diff --git a/frontend/src/arthur/ui/palette.cljs b/frontend/src/arthur/ui/palette.cljs new file mode 100644 index 0000000..dcd35b3 --- /dev/null +++ b/frontend/src/arthur/ui/palette.cljs @@ -0,0 +1,51 @@ +(ns arthur.ui.palette + "The bar above the stage: the tone a new shape gets, and the tool that makes + one. + + SIXTEEN SLOTS, and the palette supplies nine of them. The count is the format's + and not the data's — an indexed 320x200 picture in the Animator Pro idiom this + tool inherits has a fixed-size table, and a strip that grew and shrank as tones + were added would make the palette look like a list of colours rather than like + a table with room in it. So the empty slots are drawn, hatched, and refuse the + click. + + Slot 0 is the background, which is why it is shown and not selectable: a + polygon filled with index 0 is invisible against a stage cleared to index 0, so + offering it as a fill is offering a shape that vanishes on creation." + (:require [arthur.domain.palette :as pal] + [arthur.events.ui :as ui] + [arthur.subs.ui :as sub] + [re-frame.core :as rf])) + +(def ^:const slots 16) + +(defn- swatch [i tone] + (let [{slot-tone :name :keys [hex]} (get pal/entries i) + bg? (zero? i) + pick (and slot-tone (not bg?))] + [:button + {:key i + :class (str "swatch" + (when-not slot-tone " empty") + (when bg? " bg") + (when (and pick (= slot-tone tone)) " on")) + :style (when hex {:background hex}) + :title (if slot-tone (str i " · " (name slot-tone) " " hex) (str i " · empty")) + :disabled (not pick) + :on-click #(rf/dispatch [::ui/set-tone slot-tone])}])) + +(defn bar [] + (let [tone @(rf/subscribe [::sub/tone]) + tool @(rf/subscribe [::sub/tool]) + draft @(rf/subscribe [::sub/draft])] + [:div.palette-bar + [:div.swatches (doall (map #(swatch % tone) (range slots)))] + [:span.dim (name tone)] + [:span {:style {:flex 1}}] + (if (= :polygon tool) + [:<> + [:span.dim (str (quot (count draft) 2) " points")] + [:button {:disabled (< (count draft) 6) + :on-click #(rf/dispatch [::ui/finish-polygon])} "finish"] + [:button {:on-click #(rf/dispatch [::ui/cancel-polygon])} "cancel"]] + [:button {:on-click #(rf/dispatch [::ui/begin-polygon])} "polygon"])])) diff --git a/frontend/src/arthur/ui/params.cljs b/frontend/src/arthur/ui/params.cljs new file mode 100644 index 0000000..3a582ad --- /dev/null +++ b/frontend/src/arthur/ui/params.cljs @@ -0,0 +1,249 @@ +(ns arthur.ui.params + "The right pane: what the selection is, and what can be changed about it. + + Sections rather than a mode switch. The clip's facts are always true, so the + clip section is always there; the node and timeline sections appear when + something of that kind is selected; the tracking section appears when the clip + has analysis in it. Nothing here computes — every control dispatches an intent + and every readout comes off a subscription." + (:require [clojure.string :as str] + [arthur.domain.channel :as channel] + [arthur.domain.feature :as feature] + [arthur.domain.node :as node] + [arthur.domain.paint :as paint] + [arthur.domain.params :as params] + [arthur.events.paint :as paint-events] + [arthur.events.playback :as pb] + [arthur.events.project :as project] + [arthur.events.ui :as ui] + [arthur.subs.playback :as playback] + [arthur.subs.render :as render] + [arthur.subs.ui :as sub] + [re-frame.core :as rf])) + +(defn- brief + "A value that fits the column. The `facts` list puts the whole thing in a + `title`, so nothing is lost — but a node id is a uuid and a `:z` is a + timestamp joined to one, and either wrapped over three lines pushes every + parameter below it off the pane." + [v] + (let [s (str v)] + (if (> (count s) 22) (str (subs s 0 20) "…") s))) + +(defn- facts [& pairs] + (into [:dl.facts] + (mapcat (fn [[k v]] (when v [[:dt k] [:dd {:title (str v)} v]]))) + (partition 2 pairs))) + +(defn- section [title & body] + (into [:section.section [:h2 title]] body)) + +;; --------------------------------------------------------------------------- +;; the clip + +(defn- clip-section [] + (let [clip @(rf/subscribe [::render/clip]) + fps @(rf/subscribe [::playback/fps]) + picture @(rf/subscribe [::playback/display-fps]) + frames @(rf/subscribe [::playback/frames]) + expose @(rf/subscribe [::render/exposure])] + [section "clip" + [facts + "name" (:name clip) + "stage" (str (:width clip) "×" (:height clip)) + "length" (str frames " frames") + "source" (str fps " fps") + "expose" (str "on " expose "s")] + ;; Sampling the frozen roto at a lower rate. The source track, the duration + ;; and the audio clock are untouched — a drawing is HELD, the file is never + ;; short — which is why this is a picture rate and not a playback rate. + [:div.row {:style {:margin-top "5px"}} + [:span.dim "picture"] + (doall + (for [r (distinct (filter #(<= % fps) [8 12 15 24 fps]))] + ^{:key r} + [:button {:class (when (= r picture) "on") + :on-click #(rf/dispatch [::pb/set-picture-fps r])} + (if (= r fps) "source" (str r))]))]])) + +;; --------------------------------------------------------------------------- +;; a node + +(defn- channel-state + "One line saying how this channel is animated, which is the only thing about it + this pane can say without an editor for its values." + [ch] + (cond + (:dense ch) (str "dense · " (get-in ch [:dense :frames]) " frames") + (:keys ch) (str (count (:keys ch)) " keys · " (name (or (:interp ch) :hold))) + :else (let [v (:value ch)] + (str "framed · " (if (channel/nothing? v) "absent" (pr-str v)))))) + +(defn- drawing-keys + "The polygon controls: jump to a drawing key, add one here, and choose what the + gap after the current one does. Lifted out of the old stage toolbar unchanged — + a drawing key is a parameter of the shape, and this is where the shape's + parameters are." + [id n frame] + (let [geom (get-in n [:channels paint/geometry]) + active (when geom (paint/active-frame geom frame)) + ks (when geom (sort (keys (:keys geom)))) + next-k (first (filter #(> % active) ks)) + [start end] (:span n)] + [:<> + [:div.row {:style {:margin "5px 0"}} + [:button {:disabled (or (< frame start) (>= frame end) + (contains? (:keys geom) frame)) + :on-click #(rf/dispatch [::paint-events/add-key id])} + "drawing key here"]] + (when (seq ks) + [:div.row + [:span.dim "keys"] + (doall + (for [f ks] + ^{:key f} + [:button {:class (when (= frame f) "on") + :on-click #(rf/dispatch [::pb/seek f])} + (str f)]))]) + (when next-k + [:div.row {:style {:margin-top "5px"}} + [:label.dim (str "key " active " → " next-k " ") + [:select {:value (name (or (channel/segment-interp geom active) :hold)) + :on-change #(rf/dispatch [::paint-events/set-segment-interp + id active (keyword (.. % -target -value))])} + [:option {:value "hold"} "hold"] + [:option {:value "linear"} "tween"]]]])])) + +(defn- node-section [[tid id n]] + (let [frame @(rf/subscribe [::playback/frame]) + [start end] (:span n)] + [section (str (name (:kind n)) (when (not= :main tid) (str " · " (name tid)))) + [facts + "name" (or (:name n) (brief id)) + "id" (brief id) + ;; Which library timeline a placement plays. The one fact that makes a + ;; symbol instance legible as an instance rather than as a node. + "of" (when (= :symbol (:kind n)) (str (:of n))) + "span" (when start (str start " … " end))] + (when (:paint? n) [drawing-keys id n frame]) + [:div.row {:style {:margin-top "6px"}} [:span.dim "channels"]] + [:dl.facts + (doall + (for [[path ch] (sort-by (comp str key) (node/channels n))] + ^{:key (str path)} + [:<> + [:dt (str/join " " (map name path))] + [:dd (channel-state ch)]]))]])) + +;; --------------------------------------------------------------------------- +;; a timeline + +(defn- timeline-section [tid] + (let [tl @(rf/subscribe [::render/clip]) + tl (get-in tl [:timelines tid])] + [section "timeline" + [facts + "id" (str tid) + "length" (str (:frames tl) " frames") + "nodes" (str (count (:nodes tl)))]])) + +;; --------------------------------------------------------------------------- +;; tracked objects +;; +;; The generated settings. Unlike everything above, moving one of these does real +;; work: `::project/preview-settings` re-freezes whatever blocks the knob +;; invalidates, which is why the slider's value is held in `:ui :knobs` until the +;; regeneration comes back. + +(defn- owners [clip] + (vec (for [[scope objects] [[:subject (:subjects clip)] + [:feature (:features clip)] + [:group (:groups clip)]] + id (sort-by str (keys objects))] + [scope id]))) + +(defn- bounds [knob {:keys [type default min even?] upper :max}] + {:min (or min (if (= knob :blob-grow) -3 0)) + :max (or upper (get {:verts 20 :eye-verts 16 :brow-verts 10 + :teeth-verts 20 :blob-grow 3 :top-bias 2 + :gaze-gain 4} knob) + (max 1 (* 2 default))) + :step (cond even? 2 (= type :integer) 1 :else 0.01)}) + +(defn- tracking-section [] + (let [clip @(rf/subscribe [::render/clip]) + selection @(rf/subscribe [::sub/selection]) + knobs @(rf/subscribe [::sub/knobs]) + busy? (:busy? @(rf/subscribe [::playback/project])) + report @(rf/subscribe [::project/regeneration]) + all (owners clip) + [scope id :as owner] (if (some #{selection} all) selection (first all)) + area (case scope + :subject :subject + :feature (get-in clip [:features id :area]) + :group :eye + nil) + values (case scope + :subject (merge (params/for-area :subject) + (get-in clip [:subjects id :params])) + :feature (when id (feature/effective-params clip id)) + :group (merge (params/for-area :eye) + (get-in clip [:groups id :params])) + nil)] + [section "tracking" + ;; The option VALUE is the INDEX, not the owner. A `[scope id]` pair written + ;; into the DOM comes back as a string that has to be parsed back into a + ;; keyword pair, and `subs`/`keyword` on a namespaced id loses its namespace + ;; — the same trap `events/export/target-value` exists to avoid. An index + ;; into a vector both ends agree on cannot be misread. + [:select {:value (or (first (keep-indexed #(when (= %2 owner) %1) all)) 0) + :on-change #(rf/dispatch + [::ui/select (nth all (js/parseInt (.. % -target -value) 10))])} + (doall + (for [[i [kind object-id]] (map-indexed vector all)] + ^{:key i} + [:option {:value i} (str (name kind) " · " (subs (str object-id) 1))]))] + (when area + [:div {:style {:margin-top "6px"}} + (doall + (for [[knob spec] (sort-by (comp str key) params/definitions) + :when (= area (:area spec))] + (let [k (get knobs [scope id knob] (get values knob)) + {:keys [min max step]} (bounds knob spec)] + ^{:key knob} + [:label.knob + [:span.top-line [:span.name (name knob)] [:span (str k)]] + [:input {:type "range" :min min :max max :step step :value k + :disabled busy? + :on-change + (fn [^js event] + (let [s (.. event -target -value) + v (if (= :integer (:type spec)) + (js/parseInt s 10) + (js/parseFloat s))] + (rf/dispatch [::ui/set-knob scope id knob v]) + (rf/dispatch [::project/preview-settings + {:scope scope :id id :knob knob :value v}])))}]])))]) + (when report + ;; `regeneration-debug` is a hook rather than a style: `test/browser` reads + ;; this line to assert which feature a knob dirtied and which tier it + ;; reached, and that is the one thing on the page that says so. + [:div.dim.regeneration-debug {:style {:margin-top "6px"}} + (str "dirty: " (pr-str (:features report)) + " · blocks: " (if (seq (:roles report)) + (pr-str (sort (:roles report))) "tier 1 only"))])])) + +;; --------------------------------------------------------------------------- + +(defn view [] + (let [clip @(rf/subscribe [::render/clip]) + selection @(rf/subscribe [::sub/selection]) + node @(rf/subscribe [::sub/selected-node]) + tracked? (seq (owners clip))] + [:section.pane.params + [:div.pane-head "inspector"] + [:div {:style {:min-height 0}} + [clip-section] + (when node [node-section node]) + (when (= :timeline (first selection)) [timeline-section (second selection)]) + (when tracked? [tracking-section])]])) diff --git a/frontend/src/arthur/ui/pool.cljs b/frontend/src/arthur/ui/pool.cljs new file mode 100644 index 0000000..beac85f --- /dev/null +++ b/frontend/src/arthur/ui/pool.cljs @@ -0,0 +1,130 @@ +(ns arthur.ui.pool + "The media pool: the open document's own contents. + + TWO GROUPS, AND NEITHER IS A DOCUMENT. Whole projects are `ui/openmenu`'s, and + the split is the point: opening a project REPLACES the stage, where everything + in here is a thing to put ON it. Listing documents beside the symbols inside + one of them made them look like two kinds of the same thing, which is exactly + the confusion this pane should be ending. + + A SYMBOL is one timeline out of the open document's library — the thing a + `:kind :symbol` node is an instance of. Dragging one onto the stage places an + instance. Open a different project and this is a different list, because a + library belongs to the document that holds it. + + FOOTAGE is source media on the server: uploads that have been probed, + transcoded and cut into frames. Running one through detection produces a + document whose roto IS a symbol — `flow/freeze/clip` gives each tracked subject + its own library timeline and places it on `:main` with one instance — so + footage becomes a row in the group above it by way of being detected. + + THE WHOLE PANE IS THE DROP TARGET. A video dropped anywhere in it uploads and + then goes straight into detection, because \"upload\" and \"roto\" are one + intention and splitting them into two clicks only means finding the second + button. The uploaded bytes, the extraction job and the decoded footage stay + separate records on the server, so dropping the same file twice does not decode + it twice." + (:require [arthur.events.footage :as footage] + [arthur.events.ui :as ui] + [arthur.subs.playback :as playback] + [arthur.subs.render :as render] + [arthur.subs.ui :as sub] + [re-frame.core :as rf] + [reagent.core :as r])) + +(defn- item + "One row. `opts` is merged last so a caller can add `:draggable` and its + handlers without this function growing a parameter per affordance." + [{:keys [label sub on? disabled?] :as opts}] + [:button (merge {:class (str "pool-item" (when on? " on")) + :disabled (boolean disabled?) + :title label} + (dissoc opts :label :sub :on? :disabled?)) + label + (when sub [:span.sub sub])]) + +(defn- group + "A titled group of rows. The title doubles as the group's class, so `.clips` + and `.footage` are addressable — `test/browser` needs to ask which CLIP is open + without the chosen footage row, which is also marked, answering instead." + [title & children] + (into [:div.pool-group {:class title} [:h2 title]] children)) + +(defn- symbols [] + (let [clip @(rf/subscribe [::render/clip]) + selection @(rf/subscribe [::sub/selection]) + library (sort-by str (remove #{:main} (keys (:timelines clip))))] + (group "symbols" + [:div.dim "timelines in this document"] + (if (empty? library) + [:div.dim "none yet"] + (doall + (for [tid library + :let [tl (get-in clip [:timelines tid])]] + ^{:key tid} + [item {:label (name tid) + :sub (str (:frames tl) "f · " (count (:nodes tl)) " nodes") + :on? (= selection [:timeline tid]) + :draggable true + :on-drag-start + (fn [^js event] + (.setData (.-dataTransfer event) "text/plain" + (str "symbol:" (subs (str tid) 1))) + (set! (.. event -dataTransfer -effectAllowed) "copy")) + :on-click #(rf/dispatch [::ui/select [:timeline tid]])}]))) + (when (seq library) + [:div.dim "drag onto the stage to place"])))) + +(defn- footage [] + (let [{:keys [loading? available chosen]} @(rf/subscribe [::playback/footage])] + (group "footage" + [:div.dim "source media on the server"] + (if (empty? available) + [:div.dim "drop a video here"] + (doall + (for [{:keys [id label frames fps]} available] + ^{:key id} + [item {:label label + :sub (str frames "f @ " fps) + :on? (= id chosen) + :disabled? loading? + :on-click #(rf/dispatch [::footage/choose id])}])))))) + +(defn view [] + (r/with-let [;; Counted, not a boolean. `dragenter`/`dragleave` fire for every + ;; child element the pointer crosses, so a flag set on enter and + ;; cleared on leave flickers off the moment the drag passes over a + ;; row — the depth counter is what makes the outline steady. + depth (r/atom 0)] + (let [{:keys [loading? chosen status]} @(rf/subscribe [::playback/footage])] + [:section.pane.pool + {:class (when (pos? @depth) "dropping") + :on-drag-enter (fn [^js e] (.preventDefault e) (swap! depth inc)) + :on-drag-leave (fn [_] (swap! depth #(max 0 (dec %)))) + :on-drag-over (fn [^js e] (.preventDefault e)) + :on-drop (fn [^js e] + (.preventDefault e) + (reset! depth 0) + (when-let [file (aget (.. e -dataTransfer -files) 0)] + (rf/dispatch [::footage/upload file])))} + [:div.pane-head + "media pool" + [:span.spacer] + [:button {:disabled (or loading? (nil? chosen)) + :title "detect and freeze the chosen footage" + :on-click #(rf/dispatch [::footage/load])} + (if loading? "…" "roto")] + [:button {:title "add a video" + :disabled loading? + :on-click #(.click (js/document.getElementById "pool-file"))} + "+"]] + [:input {:id "pool-file" :type "file" :accept "video/*" + :style {:display "none"} + :on-change (fn [^js event] + (when-let [file (aget (.. event -target -files) 0)] + (rf/dispatch [::footage/upload file]) + (set! (.. event -target -value) "")))}] + [:div.pane-body + [symbols] + [footage] + (when status [:div.dim status])]]))) diff --git a/frontend/src/arthur/ui/shell.cljs b/frontend/src/arthur/ui/shell.cljs index 999f4f0..e19b8c8 100644 --- a/frontend/src/arthur/ui/shell.cljs +++ b/frontend/src/arthur/ui/shell.cljs @@ -1,300 +1,40 @@ (ns arthur.ui.shell - "The page. Transport, canvas, readouts. + "The window: one grid, five panes, and the audio element. - Nothing here computes anything about a frame: it dispatches intents and reads - extractors. The picture is put on the canvas by ui/player's loop, not by this - component re-rendering — which is why the canvas has no reactive content and - why scrubbing at speed does not re-render the page." + Nothing else. Each pane owns its own subscriptions, so this component re-renders + only when the grid itself would change — which is never. The picture is put on + the canvas by `ui/player`'s loop rather than by anything here re-rendering, + which is why scrubbing at speed does not touch React at all." (:require [arthur.clock :as clock] - [arthur.db :as db] - [arthur.domain.feature :as feature] - [arthur.domain.params :as params] - [arthur.events.export :as export] - [arthur.events.footage :as footage] [arthur.events.playback :as pb] - [arthur.events.project :as project] - [arthur.subs.playback :as sub] - [arthur.subs.render :as render] - [arthur.ui.player :as player] - [arthur.ui.paint :as paint] - [re-frame.core :as rf] - [reagent.core :as r])) - -(def ^:private zoom 2) -(defonce ^:private selected-owner (r/atom nil)) -(defonce ^:private drafts (r/atom {})) - -(defn- setting-owners [clip] - (vec (for [[scope objects] [[:subject (:subjects clip)] - [:feature (:features clip)] - [:group (:groups clip)]] - id (sort-by str (keys objects))] - [scope id]))) - -(defn- slider-bounds [knob {:keys [type default min even?] upper :max}] - {:min (or min (if (= knob :blob-grow) -3 0)) - :max (or upper (get {:verts 20 :eye-verts 16 :brow-verts 10 - :teeth-verts 20 :blob-grow 3 :top-bias 2 - :gaze-gain 4} knob) - (max 1 (* 2 default))) - :step (cond even? 2 (= type :integer) 1 :else 0.01)}) - -(defn- controls [] - (let [clip @(rf/subscribe [::render/clip]) - busy? (:busy? @(rf/subscribe [::sub/project])) - report @(rf/subscribe [::project/regeneration]) - owners (setting-owners clip) - [scope id :as owner] (if (some #{(deref selected-owner)} owners) - @selected-owner (first owners)) - area (case scope - :subject :subject - :feature (get-in clip [:features id :area]) - :group :eye - nil) - values (case scope - :subject (merge (params/for-area :subject) - (get-in clip [:subjects id :params])) - :feature (when id (feature/effective-params clip id)) - :group (merge (params/for-area :eye) - (get-in clip [:groups id :params])) - nil)] - [:section.controls - [:label "active object " - [:select {:value (or (first (keep-indexed - (fn [i option] (when (= option owner) i)) owners)) 0) - :on-change #(reset! selected-owner - (nth owners (js/parseInt (.. % -target -value) 10)))} - (if (seq owners) - (doall (for [[i [kind object-id]] (map-indexed vector owners)] - ^{:key i} [:option {:value i} - (str (name kind) " · " (subs (str object-id) 1))])) - [:option {:value 0} "no tracked objects"])] ] - (when area - [:div.control-list - (doall - (for [[knob spec] (sort-by (comp str key) params/definitions) - :when (= area (:area spec))] - (let [draft-key [scope id knob] - value (get @drafts draft-key (get values knob)) - {:keys [min max step]} (slider-bounds knob spec)] - ^{:key (str draft-key)} - [:label.control-row - [:span (name knob)] - [:input {:type "range" :min min :max max :step step :value value - :disabled busy? - :on-change (fn [event] - (let [s (.. event -target -value) - v (if (= :integer (:type spec)) - (js/parseInt s 10) - (js/parseFloat s))] - (swap! drafts assoc draft-key v) - (rf/dispatch [::project/preview-settings - {:scope scope :id id :knob knob - :value v}]))) }] - [:output (str value)]])))]) - (when report - [:pre.regeneration-debug - (str "dirty features: " (pr-str (:features report)) "\n" - "new block roles: " (if (seq (:roles report)) - (pr-str (sort (:roles report))) "none (tier 1 only)") - "\n" (get-in @(rf/subscribe [::sub/project]) [:status]))])])) + [arthur.subs.playback :as playback] + [arthur.ui.palette :as palette] + [arthur.ui.params :as params] + [arthur.ui.pool :as pool] + [arthur.ui.stage :as stage] + [arthur.ui.timeline :as timeline] + [arthur.ui.topbar :as topbar] + [re-frame.core :as rf])) (defn- audio [] - (let [src @(rf/subscribe [::sub/audio])] - [:audio - {:ref #(when % (clock/attach! %)) - :src src - :preload "auto" - ;; Transport state follows the ELEMENT, not the other way round: the audio - ;; is the clock, so anything that can change its state — the end of the - ;; file, the OS media keys, a browser autoplay block — has to be able to - ;; correct the document rather than be contradicted by it. + [:audio + {:ref #(when % (clock/attach! %)) + :src @(rf/subscribe [::playback/audio]) + :preload "auto" + ;; Transport state follows the ELEMENT, not the other way round: the audio is + ;; the clock, so anything that can change its state — the end of the file, the + ;; OS media keys, a browser autoplay block — has to be able to correct the + ;; document rather than be contradicted by it. :on-play #(rf/dispatch [::pb/play]) - :on-pause #(rf/dispatch [::pb/pause])}])) - -(defn- transport [] - (let [playing? @(rf/subscribe [::sub/playing?]) - rate @(rf/subscribe [::sub/rate]) - frame @(rf/subscribe [::sub/frame]) - frames @(rf/subscribe [::sub/frames]) - fps @(rf/subscribe [::sub/fps]) - picture-fps @(rf/subscribe [::sub/display-fps]) - current @(rf/subscribe [::render/clip-id]) - expose @(rf/subscribe [::render/exposure]) - {:keys [id label loading? status available chosen]} @(rf/subscribe [::sub/footage]) - {project-name :name :keys [busy?] project-status :status - project-seq :seq} @(rf/subscribe [::sub/project])] - [:div.transport - [:div.row - [:button {:on-click #(rf/dispatch [::pb/toggle])} - (if playing? "pause" "play")] - [:button {:on-click #(rf/dispatch [::pb/seek 0])} "|<"] - [:button {:on-click #(rf/dispatch [::pb/step -1])} "-1"] - [:button {:on-click #(rf/dispatch [::pb/step 1])} "+1"] - [:button {:class (when @(rf/subscribe [::sub/loop?]) "on") - :on-click #(rf/dispatch [::pb/toggle-loop])} "loop"] - [:button {:class (when @(rf/subscribe [::sub/muted?]) "on") - :on-click #(rf/dispatch [::pb/toggle-mute])} "mute"] - [:span.gap] - (doall - (for [[id {:keys [label]}] db/clips] - ^{:key id} - [:button {:class (when (= id @(rf/subscribe [::render/clip-id])) "on") - :on-click #(rf/dispatch [::pb/select-clip id])} - label])) - (when id - [:button {:class (when (= id @(rf/subscribe [::render/clip-id])) "on") - :on-click #(rf/dispatch [::pb/select-clip id])} - (or label "footage")]) - [:button {:disabled (or loading? (nil? chosen)) - :on-click #(rf/dispatch [::footage/load])} - (if loading? "loading…" "load frames")] - [:span.gap] - ;; The document, over HTTP. Two buttons, because the round trip is the proof - ;; the model serialises and a proof nobody can run is not one. - [:button {:disabled busy? :on-click #(rf/dispatch [::project/save])} "save"] - [:button {:disabled busy? :on-click #(rf/dispatch [::project/open])} "open"] - [:button {:disabled busy? :on-click #(rf/dispatch [::project/load-stage])} - "stage 8625"] - [:span.gap] - (doall - (for [r db/rates] - ^{:key r} - [:button {:class (when (== r rate) "on") - ;; playbackRate and nothing else: the audio slows, currentTime - ;; advances proportionally, and the derived frame follows. Slow - ;; motion cannot desync by construction. - :on-click #(rf/dispatch [::pb/set-rate r])} - (case r 1.0 "1x" 0.5 "1/2x" 0.25 "1/4x" 2.0 "2x" 4.0 "4x" (str r))]))] - [:input.scrub - {:type "range" :min 0 :max (dec frames) :step 1 :value frame - :on-change #(rf/dispatch [::pb/seek (js/parseInt (.. % -target -value) 10)])}] - [:div.readout - [:span (str "frame " frame " / " frames)] - [:span (str "source " fps " fps")] - [:span (str "picture " picture-fps " fps")] - [:span (str "pose " (clock/picture-frame frame fps picture-fps expose))] - [:span (str (js/Math.round (* 100 rate)) "%")] - ;; Measured in the loop, not derived from the clock: the whole question - ;; while profiling is whether the painting keeps up with the clock, so a - ;; number computed FROM the clock would answer itself. - (let [{:keys [fps drop]} @player/meter] - [:span {:class (when (and drop (> drop 1.35)) "warn")} - (str (.toFixed (or fps 0) 1) " paint/s" - (when (and drop (pos? drop)) - (str " · " (.toFixed drop 2) " frames/paint")))])] - (when (= id current) - [:div.picture-rate - [:span "picture fps "] - (doall - (for [r (distinct (filter #(<= % fps) [8 12 15 24 fps]))] - ^{:key r} - [:button {:class (when (= r picture-fps) "on") - :on-click #(rf/dispatch [::pb/set-picture-fps r])} - (if (= r fps) "source" (str r))]))]) - ;; Upload a video or choose existing server footage. Both routes produce the - ;; same immutable frame and audio manifest. - [:label.source-path "footage " - [:input {:type "file" :accept "video/*" :disabled loading? - :on-change (fn [event] - (when-let [file (aget (.. event -target -files) 0)] - (rf/dispatch [::footage/upload file]) - (set! (.. event -target -value) "")))}] - [:select {:value (or chosen "") :disabled loading? - :on-change #(rf/dispatch [::footage/choose (.. % -target -value)])} - (if (seq available) - (doall (for [{:keys [id label frames fps]} available] - ^{:key id} - [:option {:value id} - (str label " · " frames "f @" fps)])) - [:option {:value ""} "no footage yet"])] - [:button {:disabled loading? - :on-click #(rf/dispatch [::footage/refresh])} "refresh"]] - (when status [:div.load-status status]) - (when (or project-name project-status) - [:div.load-status - (when project-name (str "project " project-name - (when project-seq (str " r" project-seq)) " · ")) - project-status])])) - -(defn- exporter [] - (let [{:keys [timeline zoom isolate busy? done total status]} @(rf/subscribe [::export/state]) - targets @(rf/subscribe [::export/targets]) - {:keys [width height frames fps seconds poses]} @(rf/subscribe [::export/plan])] - [:section.export - [:div.row - [:label "export " - ;; ONE SELECT over everything exportable: the clip, each symbol in its - ;; library, and each placement on the stage. They are one list because they - ;; are one kind of request — render this, alone — and a mode switch beside a - ;; picker would only make the same choice twice. - ;; - ;; The option VALUE is `events/export/target-value`, not `(name id)`: a - ;; symbol timeline is `:sym/face-8625` and a placement is a uuid, and - ;; writing either through `name` loses what identifies it. That is the bug - ;; where the id read back as `:face-8625`, matched no timeline, and the - ;; export died inside re-frame's `:do-fx` with the button stuck on - ;; "rendering…". - [:select {:value (export/target-value {:timeline (or timeline :main) - :isolate isolate}) - :disabled busy? - :on-change #(rf/dispatch [::export/set-target - (export/target-id (.. % -target -value))])} - (doall - (for [{:keys [label] :as target} targets] - ^{:key (export/target-value target)} - [:option {:value (export/target-value target)} - ;; A placement is indented under the library above it, so that "the - ;; drawing" and "that one on the stage" read as different things. - (str (when (:isolate target) "· ") label)]))]] - [:span.gap] - (doall - (for [z export/zooms] - ^{:key z} - [:button {:class (when (= z zoom) "on") :disabled busy? - :on-click #(rf/dispatch [::export/set-zoom z])} - (str z "x")])) - [:button {:disabled busy? :on-click #(rf/dispatch [::export/start])} - (if busy? "rendering…" "frames + wav")]] - (when (and width frames) - [:div.readout - [:span (str width "x" height)] - [:span (str frames " frames @ " fps)] - [:span (str (.toFixed seconds 2) "s")] - ;; Poses and frames are different numbers at a lower picture rate, and - ;; showing both is how "exposure holds a drawing" stops being invisible: - ;; the file is never short, the drawing is just held. - (when (not= poses frames) [:span (str poses " poses")])]) - (when busy? - [:div.load-status (str "frame " done " / " total)]) - (when (and status (not busy?)) [:div.load-status status]) - [:p.note - "A lossless PNG sequence at an integer zoom, with the mixed audio, in one " - "zip. Import the sequence and the WAV as separate tracks."]])) - -(defn- stage [] - ;; The canvas is the STAGE's size, and the stage is the clip's — not a constant - ;; and not the footage's. Reactive, so selecting a clip of another size resizes - ;; it; ui/canvas guards the width assignment, which reallocates the backing - ;; store, so this being a re-render costs nothing per frame. - (let [w @(rf/subscribe [::sub/width]) - h @(rf/subscribe [::sub/height])] - [:div.stage-wrap - [:canvas.stage - {:ref #(player/set-canvas! %) - :width w :height h - :style {:width (str (* zoom w) "px") :height (str (* zoom h) "px")}}] - [paint/overlay w h zoom]])) + :on-pause #(rf/dispatch [::pb/pause])}]) (defn view [] - [:main - [:h1 "arthur"] - [stage] - [paint/toolbar] - [audio] - [transport] - [exporter] - [controls] - [:p.note - "Upload a video, choose its footage, then load frames. Save the project to " - "share the analyzed take without detecting frames again."]]) + [:div.app + [topbar/view] + [pool/view] + [:section.view + [palette/bar] + [stage/view]] + [params/view] + [timeline/view] + [audio]]) diff --git a/frontend/src/arthur/ui/stage.cljs b/frontend/src/arthur/ui/stage.cljs new file mode 100644 index 0000000..50533d7 --- /dev/null +++ b/frontend/src/arthur/ui/stage.cljs @@ -0,0 +1,139 @@ +(ns arthur.ui.stage + "The stage: the raster canvas, and the SVG editor over it. + + They are one widget and live in one namespace. The canvas is the playback sink + — `ui/player`'s loop writes it from an animation frame, so it has no reactive + content and scrubbing at speed does not re-render this component — and the SVG + is the only thing on the page that draws a shape a human can grab. Splitting + them would mean two namespaces sharing one coordinate transform. + + THE CANVAS IS THE RASTER'S OWN SIZE, scaled by CSS. See `ui/canvas` for why + that is load-bearing rather than convenient." + (:require [arthur.domain.channel :as channel] + [arthur.domain.paint :as paint] + [arthur.events.paint :as paint-events] + [arthur.events.ui :as ui] + [arthur.subs.playback :as playback] + [arthur.subs.render :as render] + [arthur.subs.ui :as sub] + [arthur.ui.player :as player] + [re-frame.core :as rf])) + +(def ^:const zoom + "Integer, and the browser suite reads the canvas's own pixels rather than a + screenshot because of it." + 2) + +(defn stage-point + "Where a pointer event landed, in stage pixels. Shared by the vertex editor and + by a drop out of the media pool, which is the whole reason it is public." + [event w h] + (let [box (.getBoundingClientRect (.-currentTarget event))] + [(-> (/ (* (- (.-clientX event) (.-left box)) w) (.-width box)) + js/Math.round (max 0) (min (dec w))) + (-> (/ (* (- (.-clientY event) (.-top box)) h) (.-height box)) + js/Math.round (max 0) (min (dec h)))])) + +(defn- pairs [pts] (mapv vec (partition 2 pts))) + +(defn- points-text [pts] + (apply str (interpose " " (map (fn [[x y]] (str x "," y)) (pairs pts))))) + +;; Which vertex the pointer has captured. A PLAIN atom, not app-db and not a +;; ratom: it is pointer bookkeeping that lives for the length of one drag, it is +;; written on every pointermove, and nothing renders from it — the moves it +;; produces go straight out as `::set-vertex`, which is where the document +;; changes and where re-frame belongs. +(defonce ^:private dragging (atom nil)) + +(defn- editing + "The selected node when it is a polygon on the root timeline, as + `[id node geom active-key editable?]`. Nothing else is vertex-editable yet." + [clip frame] + (let [[tid id n] @(rf/subscribe [::sub/selected-node])] + (when (and (= :main tid) (:paint? n)) + (let [geom (get-in n [:channels paint/geometry]) + active (when geom (paint/active-frame geom frame)) + [start end] (:span n)] + [id n geom active + ;; A frame between two drawing keys with a tween running has no vertices + ;; of its own to move: what is on screen there is interpolated, and + ;; dragging it would silently edit the key behind it instead. + (and (<= start frame) (< frame end) + (or (not= :linear (channel/segment-interp geom active)) + (contains? (:keys geom) frame)))])))) + +(defn- overlay [w h] + (let [clip @(rf/subscribe [::render/clip]) + frame @(rf/subscribe [::playback/frame]) + tool @(rf/subscribe [::sub/tool]) + draft @(rf/subscribe [::sub/draft]) + drawing? (= :polygon tool) + [id _ geom active editable?] (editing clip frame) + pts (when geom (channel/value-at geom frame))] + [:svg {:class (str "paint-overlay" (when drawing? " drawing")) + :width (* zoom w) :height (* zoom h) + :view-box (str "0 0 " w " " h) + :on-pointer-down (fn [event] + (when drawing? + (let [[x y] (stage-point event w h)] + (rf/dispatch [::ui/add-draft-point x y])))) + :on-pointer-move (fn [event] + (when-let [[node key-frame vertex] @dragging] + (rf/dispatch [::paint-events/set-vertex + node key-frame vertex + (stage-point event w h)]))) + :on-pointer-up (fn [_] (reset! dragging nil)) + :on-pointer-cancel (fn [_] (reset! dragging nil))} + (when (seq draft) + [:polyline {:points (points-text draft) :fill "none" + :stroke "#d0ba86" :stroke-width 1}]) + (when (and id pts (not drawing?) (not (channel/nothing? pts))) + [:g + [:polygon {:points (points-text pts) :fill "none" + :stroke "#e6ca8b" :stroke-width 1}] + (when editable? + (doall + (for [[i [x y]] (map-indexed vector (pairs pts))] + ^{:key i} + [:circle {:cx x :cy y :r 2.6 :fill "#fff1be" + :stroke "#161820" :stroke-width 0.7 + :on-pointer-down + (fn [event] + (.stopPropagation event) + (.preventDefault event) + (.setPointerCapture (.-currentTarget event) + (.-pointerId event)) + (reset! dragging [id active i]))}])))])])) + +(defn- dropped-symbol + "The library timeline a drag out of the media pool is carrying, or nil. + + `text/plain` with a prefix rather than a custom MIME type: the payload is one + short string, every browser agrees about this type, and a drag that arrives + from somewhere else simply fails the prefix test." + [^js event] + (let [data (.getData (.-dataTransfer event) "text/plain")] + (when (and data (.startsWith data "symbol:")) + (keyword (subs data (count "symbol:")))))) + +(defn view [] + ;; Reactive on the clip's dimensions, so selecting a clip of another size + ;; resizes the canvas. `ui/canvas` guards the width assignment — which + ;; reallocates the backing store — so this re-rendering costs nothing per frame. + (let [w @(rf/subscribe [::playback/width]) + h @(rf/subscribe [::playback/height])] + [:div.stage-area + [:div.stage-wrap + {:on-drag-over (fn [^js event] + (when (.-types (.-dataTransfer event)) + (.preventDefault event))) + :on-drop (fn [^js event] + (.preventDefault event) + (when-let [tid (dropped-symbol event)] + (rf/dispatch [::ui/place-symbol tid (stage-point event w h)])))} + [:canvas.stage {:ref #(player/set-canvas! %) + :width w :height h + :style {:width (str (* zoom w) "px") + :height (str (* zoom h) "px")}}] + [overlay w h]]])) diff --git a/frontend/src/arthur/ui/timeline.cljs b/frontend/src/arthur/ui/timeline.cljs new file mode 100644 index 0000000..ea5a223 --- /dev/null +++ b/frontend/src/arthur/ui/timeline.cljs @@ -0,0 +1,266 @@ +(ns arthur.ui.timeline + "The bottom pane: the transport, a ruler, and a row per node. + + `rows` is the whole of the interesting part and it is a PURE function of the + clip and the set of open paths. It flattens the document's two axes of nesting + — parent/child within a timeline, and instance into a library timeline — into + one depth-tagged list, which is what lets the labels column and the tracks + column render from the same vector and therefore stay aligned without measuring + anything. + + EVERY FRAME NUMBER A ROW CARRIES IS IN ROOT FRAME SPACE. A symbol's keys are + its own timeline's, and drawing them against the stage's ruler unmapped would + put a key under the wrong frame — silently, and most convincingly when the + instance starts at 0. So the walk carries a `->root` function and composes one + more mapping into it at each instance. The mapping is the inverse of + `node/local-frame`: `local = in + rate·(parent - at)`, so + `parent = at + (local - in)/rate`. + + WHAT IT DOES NOT DO YET: a looping instance repeats its timeline, and only the + first pass is drawn. An expanded loop therefore shows keys where they first + happen and not where they happen again." + (:require [clojure.string :as str] + [arthur.domain.node :as node] + [arthur.events.playback :as pb] + [arthur.events.ui :as ui] + [arthur.subs.playback :as playback] + [arthur.subs.render :as render] + [arthur.subs.ui :as sub] + [arthur.ui.player :as player] + [re-frame.core :as rf] + [reagent.core :as r])) + +;; --------------------------------------------------------------------------- +;; the rows + +(defn- local->parent + "The inverse of `node/local-frame`: where a frame of this node's OWN time sits + on the timeline it lives in. + + Two frame spaces meet at every node and mixing them up is the bug this exists + to prevent. A node's `:span` is checked against the frame its PARENT hands it, + but its channels are read at `local-frame` of that — so a placement `:at 48` + whose scale is keyed at 0 has that key on stage frame 48, and drawing it at 0 + puts every placement's keys in the same place however staggered they are. + + Exposure is not inverted, because it is a floor and has no inverse: a key on a + frame the exposure grid never samples is still authored on that frame, and that + is where the row should show it." + [n] + (let [{:keys [mode offset at in rate] :or {mode :inherit at 0 in 0 rate 1}} (:time n)] + (if (= mode :inherit) + identity + (fn [f] + (let [f (- f (or offset 0))] + (if (and (#{:symbol :audio} (:kind n)) (not (zero? rate))) + (js/Math.round (+ at (/ (- f in) rate))) + f)))))) + +(defn- keyed-frames [ch] (some-> (:keys ch) keys sort)) + +(defn- node-label + "What to call a node in the label column. + + A placement's id is a uuid and an authored node's is a keyword, and neither + reads as a name: `(str id)` gives `:face-1` with the colon still on it, or + thirty-six characters of hex that push the column open. `:name` when there is + one, and a legible stand-in when there is not." + [id n] + (or (:name n) + (if (keyword? id) (subs (str id) 1) (subs (str id) 0 8)))) + +(defn- channel-rows [n path depth ->root span] + (for [[cpath ch] (sort-by (comp str key) (node/channels n))] + {:path (conj path cpath) + :depth depth + :label (str/join " " (map name cpath)) + :kind :channel + :select nil + :span (when (or (:dense ch) (seq (:keys ch))) span) + :keys (mapv ->root (keyed-frames ch)) + :dense? (boolean (:dense ch))})) + +(defn rows + "The visible rows, outermost first. `expanded` is a set of row paths." + [clip expanded] + (letfn [(walk [tid path depth ->root] + (let [tl (get-in clip [:timelines tid]) + ordered (->> (:nodes tl) + ;; Front-most at the top, as a layer list is drawn + ;; everywhere. `:z` is the lexicographic draw key; + ;; the id breaks ties so the order is stable. + (sort-by (fn [[id n]] [(or (:z n) "") (str id)])) + reverse)] + (into [] + (mapcat + (fn [[id n]] + (let [rpath (conj path id) + open? (contains? expanded rpath) + channels (node/channels n) + ;; The span is in the PARENT's space and the channels + ;; are in the node's own, so they take different + ;; mappings. `self` is also what the symbol's target + ;; timeline is resolved in — `clip/resolver` roots the + ;; child at this node's local frame — so the nested + ;; walk carries it down unchanged. + self (comp ->root (local->parent n)) + span (mapv ->root (or (:span n) [0 (:frames tl)])) + row {:path rpath + :depth depth + :label (node-label id n) + :kind :node + :node-kind (:kind n) + :select [:node tid id] + :expandable? true + :expanded? open? + :span span + :keys (into [] (comp (mapcat keyed-frames) + (map self) + (distinct)) + (vals channels)) + :dense? (boolean (some :dense (vals channels)))}] + (if-not open? + [row] + (-> [row] + (into (channel-rows n rpath (inc depth) self span)) + (into (when (= :symbol (:kind n)) + (walk (:of n) rpath (inc depth) self))))))) + ordered))))] + (if (get-in clip [:timelines :main]) + (walk :main [] 0 identity) + []))) + +;; --------------------------------------------------------------------------- +;; geometry +;; +;; Percentages, so nothing has to measure the track column. A frame f occupies +;; [f/frames, (f+1)/frames), and a mark that names one frame sits at its centre. + +(defn- at% [f frames] (str (* 100 (/ (+ f 0.5) (max 1 frames))) "%")) +(defn- edge% [f frames] (str (* 100 (/ f (max 1 frames))) "%")) + +(defn- frame-at + "Which frame the pointer is over." + [^js event frames] + (let [box (.getBoundingClientRect (.-currentTarget event)) + x (- (.-clientX event) (.-left box))] + (-> (/ (* x frames) (.-width box)) js/Math.floor (max 0) (min (dec frames))))) + +;; --------------------------------------------------------------------------- +;; the panes + +(defn- transport [] + (let [playing? @(rf/subscribe [::playback/playing?]) + rate @(rf/subscribe [::playback/rate]) + frame @(rf/subscribe [::playback/frame]) + frames @(rf/subscribe [::playback/frames]) + {:keys [fps drop]} @player/meter] + [:div.pane-head + [:button {:on-click #(rf/dispatch [::pb/toggle])} (if playing? "pause" "play")] + [:button {:on-click #(rf/dispatch [::pb/seek 0])} "|<"] + [:button {:on-click #(rf/dispatch [::pb/step -1])} "-1"] + [:button {:on-click #(rf/dispatch [::pb/step 1])} "+1"] + [:button {:class (when @(rf/subscribe [::playback/loop?]) "on") + :on-click #(rf/dispatch [::pb/toggle-loop])} "loop"] + [:button {:class (when @(rf/subscribe [::playback/muted?]) "on") + :on-click #(rf/dispatch [::pb/toggle-mute])} "mute"] + (doall + (for [r [0.25 0.5 1.0 2.0 4.0]] + ^{:key r} + ;; playbackRate on the audio element and nothing else: the sound slows, + ;; currentTime advances proportionally, and the derived frame follows. Slow + ;; motion cannot desync by construction. + [:button {:class (when (== r rate) "on") + :on-click #(rf/dispatch [::pb/set-rate r])} + (case r 1.0 "1x" 0.5 "½" 0.25 "¼" 2.0 "2x" 4.0 "4x" (str r))])) + [:span.spacer] + [:span.dim (str frame " / " frames)] + ;; Measured in the loop, not derived from the clock — the whole question + ;; while profiling is whether the painting keeps up with the clock, and a + ;; number computed FROM the clock would answer itself. + [:span {:class (if (and drop (> drop 1.35)) "warn" "dim")} + (str (.toFixed (or fps 0) 1) " paint/s" + (when (and drop (pos? drop)) (str " · " (.toFixed drop 2) " f/paint")))]])) + +(defn- label-cell [{:keys [path depth label kind node-kind select expandable? expanded?]} + selection] + [:div {:class (str "tl-label" (when (and select (= select selection)) " on")) + :style {:padding-left (str (+ 4 (* 11 depth)) "px")} + :title label + :on-click #(when select (rf/dispatch [::ui/select select]))} + [:button.tl-twist + {:disabled (not expandable?) + :on-click (fn [^js e] + (.stopPropagation e) + (rf/dispatch [::ui/toggle-row path]))} + (when expandable? (if expanded? "▾" "▸"))] + [:span.name label] + (when (= :node kind) [:span.kind (str "·" (name node-kind))])]) + +(defn- track-cell [{:keys [span keys dense?]} frames] + [:div.tl-track + (when span + [:div {:class (str "tl-span" (when dense? " dense")) + :style {:left (edge% (first span) frames) + :width (str (* 100 (/ (- (second span) (first span)) + (max 1 frames))) "%")}}]) + ;; A dense channel has a value on every frame, so ticking each one is a solid + ;; block that says less than the bar behind it already does. + (when-not dense? + (doall + (for [f keys :when (and (<= 0 f) (< f frames))] + ^{:key f} [:div.tl-key {:style {:left (at% f frames)}}])))]) + +(defn view [] + (r/with-let [scrubbing (r/atom false)] + (let [clip @(rf/subscribe [::render/clip]) + frames (max 1 (or @(rf/subscribe [::playback/frames]) 1)) + frame @(rf/subscribe [::playback/frame]) + selection @(rf/subscribe [::sub/selection]) + expanded @(rf/subscribe [::sub/expanded]) + visible (rows clip expanded) + ;; Roughly ten labels, on a round number of frames. + step (* 10 (js/Math.ceil (/ frames 100)))] + [:section.pane.time + [transport] + [:div.tl-body + [:div.tl-labels + [:div {:style {:height "var(--ruler)" + :border-bottom "1px solid var(--line)" + :background "var(--chrome)"}}] + (doall (for [row visible] + ^{:key (str (:path row))} [label-cell row selection]))] + [:div.tl-tracks + ;; Five frames as a percentage of the whole span, handed to the + ;; stylesheet so the frame grid can be a repeating background instead of + ;; a div per frame. A 900-frame take is 900 elements nobody needs. + {:style {"--tick" (str (* 100 (/ 5 frames)) "%")}} + [:div.tl-ruler + {:on-pointer-down (fn [^js e] + (rf/dispatch [::pb/seek (frame-at e frames)]) + (reset! scrubbing true) + ;; Capture is what keeps a drag scrubbing once it + ;; leaves the ruler, and it is an ENHANCEMENT: it + ;; throws on a pointer id the browser does not have + ;; an active pointer for, which is every event + ;; `test/browser` synthesises. Seeking already + ;; happened, so the catch loses the drag and nothing + ;; else — where letting it throw would put an + ;; uncaught error on the console that the suite + ;; rightly fails on. + (try + (.setPointerCapture (.-currentTarget e) (.-pointerId e)) + (catch :default _ nil))) + :on-pointer-move (fn [^js e] + (when @scrubbing + (rf/dispatch [::pb/seek (frame-at e frames)]))) + :on-pointer-up (fn [_] (reset! scrubbing false)) + :on-pointer-cancel (fn [_] (reset! scrubbing false))} + (doall + (for [f (range 0 frames step)] + ^{:key f} [:div.tick {:style {:left (edge% f frames)}} f]))] + (if (seq visible) + (doall (for [row visible] + ^{:key (str (:path row))} [track-cell row frames])) + [:div.tl-empty "nothing on this timeline"]) + [:div.tl-playhead {:style {:left (at% frame frames)}}]]]]))) diff --git a/frontend/src/arthur/ui/topbar.cljs b/frontend/src/arthur/ui/topbar.cljs new file mode 100644 index 0000000..12bf3ed --- /dev/null +++ b/frontend/src/arthur/ui/topbar.cljs @@ -0,0 +1,60 @@ +(ns arthur.ui.topbar + "The strip across the top: what document this is, and the three things you can + do to the whole of it. + + Save, open and export are here rather than in a pane because none of them is a + property of a selection — they act on the document, and the document is the + window." + (:require [arthur.events.export :as export] + [arthur.events.project :as project] + [arthur.subs.playback :as playback] + [arthur.ui.openmenu :as openmenu] + [re-frame.core :as rf])) + +(defn- exporter [] + (let [{:keys [timeline zoom isolate busy? done total]} @(rf/subscribe [::export/state]) + targets @(rf/subscribe [::export/targets])] + [:<> + ;; ONE SELECT over everything exportable: the clip, each symbol in its + ;; library, and each placement on the stage. They are one list because they + ;; are one kind of request — render this, alone. + ;; + ;; The option VALUE goes through `export/target-value`, never `name`: a + ;; symbol timeline is `:sym/face-8625` and a placement is a uuid, and writing + ;; either through `name` loses what identifies it. + [:select {:value (export/target-value {:timeline (or timeline :main) :isolate isolate}) + :disabled busy? + :title "what to render" + :on-change #(rf/dispatch [::export/set-target + (export/target-id (.. % -target -value))])} + (doall + (for [{:keys [label] :as target} targets] + ^{:key (export/target-value target)} + [:option {:value (export/target-value target)} + (str (when (:isolate target) "· ") label)]))] + [:select {:value zoom :disabled busy? + :title "integer zoom" + :on-change #(rf/dispatch [::export/set-zoom + (js/parseInt (.. % -target -value) 10)])} + (doall (for [z export/zooms] ^{:key z} [:option {:value z} (str z "x")]))] + [:button {:disabled busy? :on-click #(rf/dispatch [::export/start]) + :title "a lossless PNG sequence and the mixed audio, in one zip"} + (if busy? (str "rendering " done "/" total) "export")]])) + +(defn view [] + (let [{project-name :name :keys [busy? seq] project-status :status} + @(rf/subscribe [::playback/project]) + {footage-status :status} @(rf/subscribe [::playback/footage]) + {export-status :status} @(rf/subscribe [::export/state])] + [:header.top + [:span.brand "arthur"] + [:span.status + (str (or project-name "untitled") (when seq (str " r" seq)) + ;; One line, and the most recent thing to have happened wins it. A + ;; strip with three status slots is three empty boxes most of the time. + (when-let [said (or export-status project-status footage-status)] + (str " · " said)))] + [exporter] + [:button {:disabled busy? :on-click #(rf/dispatch [::project/new])} "new"] + [openmenu/view] + [:button {:disabled busy? :on-click #(rf/dispatch [::project/save])} "save"]])) diff --git a/frontend/test/browser/take.mjs b/frontend/test/browser/take.mjs index 090f928..6e5d40f 100644 --- a/frontend/test/browser/take.mjs +++ b/frontend/test/browser/take.mjs @@ -185,33 +185,90 @@ const PROBE = `(() => { w: c.width, h: c.height, drawn, tones: [...tones].length, toneSet: [...tones], cx: drawn ? cx / drawn : null, cy: drawn ? cy / drawn : null, hash: h >>> 0, - frame: document.querySelector('.readout span')?.textContent ?? '', - selectedClip: [...document.querySelectorAll('.transport .row button')] - .filter((b) => b.classList.contains('on')).map((b) => b.textContent), + frame: document.querySelector('.time .pane-head .dim')?.textContent ?? '', + // What the top bar says about the document. The media pool holds the OPEN + // document's library now and no longer names documents at all, so the status + // line is the page's own answer to "what am I looking at". + doc: document.querySelector('.top .status')?.textContent ?? '', }; })()`; +// The scrubber is the timeline's ruler — there is no range input any more — so a +// seek is a pointer event at the frame's own column. The ruler maps x to +// floor(x / width * frames), and a mark that names one frame sits at its centre, +// so (f + 0.5) lands inside f and nowhere near its neighbours. +// +// The frame count comes off the readout rather than being passed in: the caller +// knows which frame it wants, not how long the clip it is looking at happens to +// be, and reading it here is one place instead of every call site. const SEEK = (f) => `(() => { - const el = document.querySelector('input.scrub'); - // React installs its own value setter on the element, so assigning .value - // directly updates the DOM and not React's idea of it, and onChange never - // fires. The prototype-level setter plus a bubbling 'input' event is what - // React's synthetic onChange actually listens for. - const set = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value').set; - set.call(el, '${f}'); - el.dispatchEvent(new Event('input', { bubbles: true })); - return el.value; + const read = document.querySelector('.time .pane-head .dim').textContent; + // Split, not a regex: this is inside a template literal, where an escaped + // slash collapses to a bare one and the two together open a line comment that + // eats the rest of the statement. The readout is "12 / 229" and nothing else. + const frames = Number(read.split('/')[1]); + if (!frames) return 'no readout'; + const ruler = document.querySelector('.tl-ruler'); + const box = ruler.getBoundingClientRect(); + ruler.dispatchEvent(new PointerEvent('pointerdown', { + bubbles: true, pointerId: 1, + clientX: box.left + (${f} + 0.5) * box.width / frames, + clientY: box.top + box.height / 2, + })); + ruler.dispatchEvent(new PointerEvent('pointerup', { bubbles: true, pointerId: 1 })); + return document.querySelector('.time .pane-head .dim').textContent; })()`; // Everything the page has to say about loading, saving and opening. Read off the // page rather than out of app-db, for the same reason the playhead is: what the // page SHOWS is what a person would check. -const STATUS = `[...document.querySelectorAll('.load-status')].map((d) => d.textContent).join(' | ')`; +const STATUS = `[...document.querySelectorAll('.top .status, .pool .pane-body > .dim')] + .map((d) => d.textContent).join(' | ')`; +// Any button on the page, by its own label. The controls are spread across five +// panes now — the transport is in the timeline's header, save and open are in the +// top bar, the clips are rows in the media pool — and a helper that knew which +// pane each one lived in would be a second copy of the layout. +// +// `firstChild` is the label: a media-pool row has a second line in a child span, +// so matching on textContent would never find "take". const CLICK = (label) => `(() => { - const b = [...document.querySelectorAll('.transport button')] - .find((b) => b.textContent.trim() === ${JSON.stringify(label)}); - if (!b) return false; + const b = [...document.querySelectorAll('button')] + .find((b) => (b.firstChild?.textContent ?? '').trim() === ${JSON.stringify(label)}); + if (!b || b.disabled) return false; + b.click(); + return true; +})()`; + +// Documents are behind File -> Open now, not in the media pool: the pool holds +// the OPEN document's library, and a whole project is not a thing you put on a +// stage. So reaching a built-in scene or a saved project is two clicks with a +// turn of the event loop between them, as it is for anybody using the app. +const MENU_OPEN = `(() => { + if (document.querySelector('.menu')) return true; + const b = [...document.querySelectorAll('button')] + .find((b) => b.textContent.trim() === 'open \u25be'); + if (!b || b.disabled) return false; + b.click(); + return true; +})()`; + +// Scoped to `.example`, and it has to be: the local server accumulates a saved +// project called "take" on every run of this suite, so an unscoped match by +// label picks whichever of those sorted first and the fixture is never reached. +const MENU_PICK = (label) => `(() => { + const b = [...document.querySelectorAll('.menu .menu-item.example')] + .find((b) => (b.firstChild?.textContent ?? '').trim() === ${JSON.stringify(label)}); + if (!b || b.disabled) return false; + b.click(); + return true; +})()`; + +// The first row under PROJECTS, which the server orders by -updated: this is +// what "open" meant before there was a list to pick from. +const MENU_PICK_NEWEST = `(() => { + const b = document.querySelector('.menu .menu-item:not(.example)'); + if (!b || b.disabled) return false; b.click(); return true; })()`; @@ -223,24 +280,44 @@ async function main() { mkdirSync(OUT, { recursive: true }); const page = await connect(); + const fromMenu = async (what) => { + if (!(await page.eval(MENU_OPEN))) return false; + await sleep(150); + return page.eval(what); + }; try { - // Mounted, and painting. Polled rather than waited on a fixed delay: the - // canvas :ref fires after the loop starts, so there genuinely is a window in - // which the page is up and the canvas is blank. + // Mounted. Polled rather than waited on a fixed delay: the canvas :ref fires + // after the loop starts, so there genuinely is a window in which the page is + // up and there is nothing to measure. + // + // It is NOT polled for pixels any more. The app opens on a blank document — + // no clip is loaded until one is asked for — so "the canvas has drawn + // something" is now a thing this suite makes happen rather than a thing it + // waits for. let probe = null; for (let i = 0; i < 100; i++) { probe = await page.eval(PROBE); - if (probe && probe.drawn > 0) break; + if (probe && probe.w) break; await sleep(100); } if (!probe) throw new Error('no canvas.stage on the page — ' + (page.logs.slice(0, 3).join(' | ') || 'is `shadow-cljs watch app` running?')); - console.log(`\ncanvas ${probe.w}x${probe.h}, clip ${JSON.stringify(probe.selectedClip)}`); + console.log(`\ncanvas ${probe.w}x${probe.h}, ${probe.doc}`); - check(probe.selectedClip.includes('take'), 'the take is the clip that opens'); + check(probe.drawn === 0 && /new document/.test(probe.doc), + 'the app opens on a blank document', + `${probe.drawn} px drawn — ${probe.doc}`); check(probe.w === 320 && probe.h === 200, 'the canvas is the stage size', `${probe.w}x${probe.h}`); + + // Everything below is about the frozen take, so open it out of the pool. + check(await fromMenu(MENU_PICK('take')), 'the take opens from the open menu'); + for (let i = 0; i < 100; i++) { + probe = await page.eval(PROBE); + if (probe.drawn > 0) break; + await sleep(100); + } check(probe.drawn > 200, 'the first frame is not blank', `${probe.drawn} px drawn`); // --- it is a mouth: two tones, one inside the other --- @@ -283,7 +360,7 @@ async function main() { 'as filmed, the head carries the mouth across the stage', `centroid x spans ${(Math.max(...xs) - Math.min(...xs)).toFixed(1)} px`); - check(await page.eval(CLICK('locked')), 'the locked take is selectable'); + check(await fromMenu(MENU_PICK('locked')), 'the locked take is selectable'); await sleep(200); const locked = []; for (const f of [0, 40, 80, 120, 160, 200]) { @@ -304,7 +381,7 @@ async function main() { 'and it is still a performance, not a still frame'); // --- it PLAYS, against the audio clock --- - check(await page.eval(CLICK('take')), 'back to the take'); + check(await fromMenu(MENU_PICK('take')), 'back to the take'); await sleep(150); await page.eval(SEEK(0)); await sleep(150); @@ -312,7 +389,7 @@ async function main() { const during = []; for (let i = 0; i < 8; i++) { await sleep(180); during.push(await page.eval(PROBE)); } await page.eval(CLICK('pause')); - // The readout is "frame 12 / 229", so the first run of digits is the + // The readout is "12 / 229", so the first run of digits is the // playhead. Parsed rather than reached for in app-db on purpose: what the // page SHOWS is what a person would check, and the readout agreeing with the // picture is half of what the transport is for. @@ -351,7 +428,7 @@ async function main() { } const FRAMES = [0, 10, 28, 80, 160]; - check(await page.eval(CLICK('take')), 'back to the take, for the round trip'); + check(await fromMenu(MENU_PICK('take')), 'back to the take, for the round trip'); await sleep(150); const sent = await sample(FRAMES); @@ -377,7 +454,7 @@ async function main() { check(resaved !== null, 'saving an unchanged document writes nothing', resaved ?? (await page.eval(STATUS))); - check(await page.eval(CLICK('open')), 'open is clickable'); + check(await fromMenu(MENU_PICK_NEWEST), 'open is clickable'); const opened = await statusMatching(/opened /); check(opened !== null, 'the project opens', opened ?? (await page.eval(STATUS))); @@ -392,8 +469,8 @@ async function main() { 'and the open mouth still has an interior on the far side'); // The reopened clip is not one of the built-ins: this is the document that came // back from the server, not the one that was in the page all along. - check(!back[0].selectedClip.includes('take'), 'the picture is the reopened document', - JSON.stringify(back[0].selectedClip)); + check(/opened /.test(back[0].doc), 'the picture is the reopened document', + back[0].doc); // Paint through the visible controls. This exercises the SVG pointer path, // the authored node, the raster preview and the document round trip. @@ -404,10 +481,20 @@ async function main() { const d = c.getContext('2d').getImageData(25, 20, 1, 1).data; return [...d]; })()`); - const painted = await page.eval(`(() => { - const button = [...document.querySelectorAll('.paint-tools button')] - .find(b => b.textContent === 'new polygon'); + // Arming the tool and clicking the vertices are two evals with a turn of the + // event loop between them, and they have to be: picking the tool dispatches a + // re-frame event, which is queued rather than applied, so a pointerdown in the + // same synchronous block arrives while the tool is still unset and is dropped. + // A person cannot click twice inside one microtask; a test should not either. + const armed = await page.eval(`(() => { + const button = [...document.querySelectorAll('.palette-bar button')] + .find(b => b.textContent === 'polygon'); + if (!button) return false; button.click(); + return true; + })()`); + await sleep(100); + const painted = armed && await page.eval(`(() => { const svg = document.querySelector('.paint-overlay'); const box = svg.getBoundingClientRect(); for (const [x, y] of [[10, 10], [40, 10], [25, 40]]) { @@ -420,8 +507,8 @@ async function main() { })()`); await sleep(100); const finished = await page.eval(`(() => { - const button = [...document.querySelectorAll('.paint-tools button')] - .find(b => b.textContent === 'finish shape'); + const button = [...document.querySelectorAll('.palette-bar button')] + .find(b => b.textContent === 'finish'); if (!button || button.disabled) return false; button.click(); return true; @@ -438,8 +525,8 @@ async function main() { await page.eval(SEEK(f)); await sleep(100); check(await page.eval(`(() => { - const button = [...document.querySelectorAll('.paint-tools button')] - .find(b => b.textContent === 'new drawing key'); + const button = [...document.querySelectorAll('.params button')] + .find(b => b.textContent === 'drawing key here'); if (!button || button.disabled) return false; button.click(); return true; @@ -449,7 +536,7 @@ async function main() { await page.eval(SEEK(8)); await sleep(100); const tweenSelected = await page.eval(`(() => { - const label = [...document.querySelectorAll('.paint-tools label')] + const label = [...document.querySelectorAll('.params label')] .find(el => el.textContent.startsWith('key 8 → 16')); const select = label?.querySelector('select'); if (!select) return false; @@ -459,14 +546,14 @@ async function main() { })()`); await sleep(100); check(tweenSelected && await page.eval(`(() => { - const label = [...document.querySelectorAll('.paint-tools label')] + const label = [...document.querySelectorAll('.params label')] .find(el => el.textContent.startsWith('key 8 → 16')); return label?.querySelector('select')?.value === 'linear'; })()`), 'the second drawing gap can be set to tween'); check(await page.eval(CLICK('save')), 'the painted document can be saved'); check((await statusMatching(/saved r\d+ · \d+ leaves/)) !== null, 'the painted shape is saved', await page.eval(STATUS)); - check(await page.eval(CLICK('open')), 'the painted document can be reopened'); + check(await fromMenu(MENU_PICK_NEWEST), 'the painted document can be reopened'); check((await statusMatching(/opened /)) !== null, 'the painted shape is reopened', await page.eval(STATUS)); await sleep(150); @@ -479,7 +566,7 @@ async function main() { await page.eval(SEEK(8)); await sleep(100); check(await page.eval(`(() => { - const label = [...document.querySelectorAll('.paint-tools label')] + const label = [...document.querySelectorAll('.params label')] .find(el => el.textContent.startsWith('key 8 → 16')); return label?.querySelector('select')?.value === 'linear'; })()`), 'the per-gap tween setting survives the project round trip'); @@ -489,11 +576,11 @@ async function main() { const stageSource = await page.eval(`fetch('/api/projects/4379f900-bdd2-409b-acf6-32081f8ce01f') .then(r => r.ok)`); if (stageSource) { - check(await page.eval(CLICK('stage 8625')), 'the 8625 stage loads'); + check(await fromMenu(MENU_PICK('8625 stage study')), 'the 8625 stage loads'); const stageLoaded = await statusMatching(/loaded 8625 stage study/, 160); check(stageLoaded !== null, 'the 8625 stage is ready', stageLoaded ?? (await page.eval(STATUS))); const eyeSelected = await page.eval(`(() => { - const select = document.querySelector('.controls select'); + const select = document.querySelector('.params .section:last-child select'); // Settings belong to tracked features shared by the stage placements. const option = [...select.options] .find(o => o.textContent.trim().endsWith('feature · face-1/eye-r')); @@ -505,8 +592,8 @@ async function main() { check(eyeSelected, 'a stage instance exposes its tracked right eye'); await sleep(100); const irisSlider = await page.eval(`(() => { - const row = [...document.querySelectorAll('.control-row')] - .find(row => row.querySelector('span')?.textContent === 'iris-size'); + const row = [...document.querySelectorAll('.params .knob')] + .find(row => row.querySelector('.name')?.textContent === 'iris-size'); if (!row) return null; // Scrolled into view FIRST, because the click below is dispatched at // viewport coordinates: a control panel that has grown past the fold @@ -520,7 +607,7 @@ async function main() { check(irisSlider !== null, 'the stage eye has an iris-size slider'); if (irisSlider) { check(await page.eval(CLICK('play')), 'the stage starts playing'); - const startFrame = await page.eval(`Number(document.querySelector('.readout span').textContent.match(/\\d+/)[0])`); + const startFrame = await page.eval(`Number(document.querySelector('.time .pane-head .dim').textContent.match(/\\d+/)[0])`); await page.send('Input.dispatchMouseEvent', { type: 'mousePressed', x: irisSlider.x, y: irisSlider.y, button: 'left', clickCount: 1, }); @@ -532,12 +619,12 @@ async function main() { check(preview !== null, 'the slider updates the stage preview', preview ?? debug); check(debug.includes(':face-1/eye-r') && debug.includes('tier 1 only'), 'the panel reports the affected feature and tier', debug); - const endFrame = await page.eval(`Number(document.querySelector('.readout span').textContent.match(/\\d+/)[0])`); + const endFrame = await page.eval(`Number(document.querySelector('.time .pane-head .dim').textContent.match(/\\d+/)[0])`); check(endFrame > startFrame, 'playback continues during tuning', `${startFrame} -> ${endFrame}`); await page.eval(CLICK('pause')); } const subjectSelected = await page.eval(`(() => { - const select = document.querySelector('.controls select'); + const select = document.querySelector('.params .section:last-child select'); const option = [...select.options] .find(o => o.textContent.includes('subject · face-1')); if (!option) return false; @@ -549,8 +636,8 @@ async function main() { if (subjectSelected) { await sleep(100); const anchorSlider = await page.eval(`(() => { - const row = [...document.querySelectorAll('.control-row')] - .find(row => row.querySelector('span')?.textContent === 'anchor-avg'); + const row = [...document.querySelectorAll('.params .knob')] + .find(row => row.querySelector('.name')?.textContent === 'anchor-avg'); if (!row) return null; // Scrolled into view FIRST, because the click below is dispatched at // viewport coordinates: a control panel that has grown past the fold @@ -586,7 +673,7 @@ async function main() { } } const teethSelected = await page.eval(`(() => { - const select = document.querySelector('.controls select'); + const select = document.querySelector('.params .section:last-child select'); const option = [...select.options] .find(o => o.textContent.trim().endsWith('feature · face-1/teeth')); if (!option) return false; @@ -598,8 +685,8 @@ async function main() { if (teethSelected) { await sleep(100); const slider = await page.eval(`(() => { - const row = [...document.querySelectorAll('.control-row')] - .find(row => row.querySelector('span')?.textContent === 'cavity-erode'); + const row = [...document.querySelectorAll('.params .knob')] + .find(row => row.querySelector('.name')?.textContent === 'cavity-erode'); if (!row) return null; // Scrolled into view FIRST, because the click below is dispatched at // viewport coordinates: a control panel that has grown past the fold @@ -623,10 +710,10 @@ async function main() { let debug = ''; for (let i = 0; i < 160; i++) { debug = await page.eval(`document.querySelector('.regeneration-debug')?.textContent ?? ''`); - if (debug.includes('dirty features: [:face-1/teeth]')) break; + if (debug.includes('dirty: [:face-1/teeth]')) break; await sleep(250); } - check(debug.includes('dirty features: [:face-1/teeth]'), + check(debug.includes('dirty: [:face-1/teeth]'), 'the pixel setting invalidates only teeth', debug); const preview = await statusMatching(/preview · unsaved/, 160); check(preview !== null, 'the teeth edit finishes previewing', @@ -650,7 +737,19 @@ async function main() { nodeId: root.nodeId, selector: 'input[type=file]', }); await page.send('DOM.setFileInputFiles', { files: [video], nodeId }); - const extracted = await statusMatching(/video extracted/, 160); + // Waited for in the MEDIA POOL rather than in the status line. Dropping or + // choosing a video now runs straight on into detection, so "video + // extracted" is a state the app passes through in one turn of the event + // loop and not one a poller can be relied on to catch. The row appearing in + // the pool is the durable evidence, and it is what a person would look at. + let extracted = null; + for (let i = 0; i < 160 && extracted === null; i++) { + await sleep(250); + if (await page.eval(`[...document.querySelectorAll('.pool-item')] + .some((b) => (b.firstChild?.textContent ?? '').trim() === 'browser-upload.mp4')`)) { + extracted = 'listed in the media pool'; + } + } check(extracted !== null, 'the page uploads and extracts a video', extracted ?? (await page.eval(STATUS))); const uploadedFootage = await page.eval(`(async () => { diff --git a/static/arthur/app.css b/static/arthur/app.css new file mode 100644 index 0000000..90b8bb3 --- /dev/null +++ b/static/arthur/app.css @@ -0,0 +1,565 @@ +/* The application chrome: one screen, five panes, no scrolling page. + * + * Macromedia-era, deliberately. Light grey panels, hairline rules, small type, + * and a timeline whose keyframes are DOTS on a frame grid — Flash's vocabulary, + * because this tool's model is Flash's model (a library of timelines, instances + * placed on a stage, a layer list against a ruler) and borrowing the look is the + * cheapest way to make that legible to anyone who has used one. What is not + * borrowed is the chrome of the period: no bevels, no gradients on buttons, no + * inset wells. Flat, clean, one accent. + * + * Raw CSS on purpose. This file styles a fixed grid of panes whose class names + * are the domain's own — `.tl-row`, `.pool-item`, `.swatch` — so a utility + * framework would buy naming we want anyway, at the cost of a third process in + * a dev loop the repo keeps at two. Tokens are custom properties; everything + * else is a handful of grids. + * + * It lives under static/ rather than inline in the template because it is now + * long enough that "the styles" and "the page" are two things. */ + +:root { + color-scheme: light; + + /* The stage's background is the PALETTE's index 0, not a theme colour. The + two agreeing is what makes the preview honest, so it is written here as the + same value arthur.domain.palette declares and nowhere else in this file. + It is the one dark surface on the page, and it should be: the picture is + 320x200 of indexed colour and the UI is not allowed to compete with it. */ + --stage: #12141c; + + --desk: #8e8e8e; /* the work area the stage floats on */ + --pane: #f2f2f0; + --chrome: #e4e4e1; + --sunk: #ebebe8; + --line: #b0b0ac; + --hair: #d2d2ce; + --fg: #1f1f1f; + --dim: #6e6e6a; + + /* One accent, used for selection and for nothing else. */ + --sel: #2f6fc0; + --sel-bg: #cfe0f5; + + /* A keyframe is a dot and a dot is ink. Flash draws them black and so does + this; the frames a node exists over are a pale tint behind them. */ + --key: #1f1f1f; + --span: #dfe8f4; + --span-line:#a9bdd8; + --grid: #dcdcd8; + --grid-5: #c6c6c2; + --playhead: #c8322b; + --warn: #b4502a; + + --row: 21px; + --ruler: 19px; + --label: 210px; +} + +* { box-sizing: border-box; } + +html, body { margin: 0; height: 100%; overflow: hidden; } + +body { + background: var(--desk); + color: var(--fg); + font: 11px/1.45 "Lucida Grande", "Segoe UI", system-ui, sans-serif; +} + +#app { height: 100%; } + +/* The preview is nearest-neighbour everywhere. A browser that smoothed the + upscale would misrepresent the exact look the tool exists to judge, so this + rule is load-bearing rather than cosmetic. */ +canvas { image-rendering: pixelated; } + +audio { display: none; } + +/* -------------------------------------------------------------------------- + the frame */ + +.app { + display: grid; + height: 100%; + grid-template-columns: var(--label) minmax(0, 1fr) 250px; + grid-template-rows: 30px minmax(0, 1fr) 232px; + grid-template-areas: + "top top top" + "pool view params" + "time time time"; + gap: 1px; + background: var(--line); +} + +.top { grid-area: top; } +.pool { grid-area: pool; } +.view { grid-area: view; } +.params { grid-area: params; } +.time { grid-area: time; } + +/* Every pane is its own scroll container. `min-height: 0` is what lets a grid + row shrink below its content instead of pushing the layout taller than the + viewport — the single line that separates "a page that scrolls" from "an + application window". */ +.pane { + background: var(--pane); + min-height: 0; + min-width: 0; + overflow: auto; + display: flex; + flex-direction: column; +} + +.pane-head { + position: sticky; + top: 0; + z-index: 2; + display: flex; + align-items: center; + gap: 5px; + padding: 0 7px; + height: 21px; + flex: 0 0 21px; + background: var(--chrome); + border-bottom: 1px solid var(--line); + color: var(--dim); + letter-spacing: .03em; +} + +.pane-head .spacer { flex: 1; } +.pane-body { padding: 5px; } + +/* -------------------------------------------------------------------------- + controls */ + +button { + font: inherit; + color: var(--fg); + background: var(--pane); + border: 1px solid var(--line); + border-radius: 2px; + padding: 1px 7px; + cursor: pointer; +} + +button:hover:not(:disabled) { background: #fff; } +button:active:not(:disabled) { background: var(--sunk); } +button:disabled { opacity: .38; cursor: default; } + +button.on { + background: var(--sel-bg); + border-color: var(--sel); + color: #14395f; +} + +select, input[type="text"] { + font: inherit; + color: var(--fg); + background: #fff; + border: 1px solid var(--line); + border-radius: 2px; + padding: 1px 3px; + max-width: 100%; +} + +input[type="range"] { width: 100%; accent-color: var(--sel); } + +.row { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: 4px; +} + +.dim { color: var(--dim); } +.warn { color: var(--warn); } + +/* -------------------------------------------------------------------------- + top bar */ + +.top { + display: flex; + align-items: center; + gap: 7px; + padding: 0 8px; + background: var(--chrome); +} + +.top .brand { color: var(--dim); letter-spacing: .14em; text-transform: uppercase; } +.top .status { + flex: 1; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; + color: var(--dim); +} + +/* -------------------------------------------------------------------------- + the open menu */ + +.menu-wrap { position: relative; } + +/* A full-page catcher behind the panel, so a click anywhere else dismisses it. + More predictable than a document listener that has to be added, removed, and + told to ignore the click that opened the menu. */ +.menu-scrim { position: fixed; inset: 0; z-index: 40; } + +.menu { + position: absolute; + top: calc(100% + 3px); + right: 0; + z-index: 41; + min-width: 250px; + max-height: 72vh; + overflow: auto; + padding: 5px; + background: var(--pane); + border: 1px solid var(--line); + border-radius: 2px; + box-shadow: 0 3px 10px rgba(0, 0, 0, .3); +} + +.menu > h2 { + margin: 6px 0 3px; + font: inherit; + font-weight: 600; + color: var(--dim); + text-transform: uppercase; + letter-spacing: .07em; +} + +.menu > h2:first-child { margin-top: 0; } +.menu > h2 + .dim { margin-bottom: 4px; } + +.menu-item { + display: block; + width: 100%; + text-align: left; + padding: 2px 6px; + border: 1px solid transparent; + border-radius: 2px; + background: none; + cursor: pointer; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.menu-item:hover:not(:disabled) { background: var(--sel-bg); } + +/* A fixture is not a document. Italic is the whole of the distinction, and it is + enough: the heading above already says what these are. */ +.menu-item.example { font-style: italic; } +.menu-item .sub { display: block; color: var(--dim); } + +/* -------------------------------------------------------------------------- + media pool */ + +.pool-group { margin-bottom: 9px; } + +/* The one-line gloss under a group heading. Small and quiet: it answers "what + is this list" once, for someone who has not read the model. */ +.pool-group > h2 + .dim { margin-bottom: 4px; } + +.pool-group > h2 { + margin: 0 0 3px; + font: inherit; + font-weight: 600; + color: var(--dim); + text-transform: uppercase; + letter-spacing: .07em; +} + +.pool-item { + display: block; + width: 100%; + text-align: left; + padding: 2px 6px; + border: 1px solid transparent; + border-radius: 2px; + background: none; + cursor: grab; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.pool-item:hover:not(:disabled) { background: #fff; } +.pool-item.on { background: var(--sel-bg); border-color: var(--sel); } +.pool-item .sub { display: block; color: var(--dim); } + +/* The whole pane is the drop target, so the cue has to be the pane and not a + zone inside it — a drop that lands two pixels outside a dashed rectangle and + silently does nothing is the failure this avoids. */ +.pool.dropping { + outline: 2px dashed var(--sel); + outline-offset: -3px; + background: var(--sel-bg); +} + +/* -------------------------------------------------------------------------- + the viewer: palette above, stage below */ + +.view { + display: flex; + flex-direction: column; + background: var(--desk); + min-height: 0; + overflow: hidden; +} + +.palette-bar { + display: flex; + align-items: center; + gap: 9px; + padding: 4px 8px; + background: var(--chrome); + border-bottom: 1px solid var(--line); +} + +.swatches { display: flex; gap: 3px; } + +/* Circles. A palette entry is one indivisible tone, not an area of coverage, and + a row of dots says that where a row of tiles says "swatch book". */ +.swatch { + width: 15px; + height: 15px; + padding: 0; + border: 1px solid var(--line); + border-radius: 50%; + cursor: pointer; +} + +.swatch:hover:not(:disabled) { border-color: var(--fg); } + +/* A swatch that cannot be picked still has to SHOW ITS COLOUR: the generic + disabled rule fades it towards the pane, which turned the background slot — + the darkest tone in the palette — into pale grey, i.e. into a lie about what + index 0 is. Slot 0 says what it is with a ring instead. */ +.swatch:disabled { opacity: 1; } +.swatch.bg { cursor: default; box-shadow: inset 0 0 0 2px var(--pane); } + +.swatch.on { + border-color: var(--fg); + box-shadow: 0 0 0 2px var(--pane), 0 0 0 3px var(--sel); +} + +/* An empty slot has to read as "no colour here" rather than as a pale colour, + which a flat fill at this size cannot do. */ +.swatch.empty { + cursor: default; + background: #fff; + border-style: dashed; + border-color: var(--hair); +} + +.stage-area { + flex: 1; + min-height: 0; + display: grid; + place-items: center; + overflow: auto; + padding: 14px; +} + +/* The stage sits ON the work area rather than filling it, as it does in every + tool of this shape: the picture has edges and they are part of what you are + judging. */ +.stage-wrap { + position: relative; + line-height: 0; + box-shadow: 0 1px 5px rgba(0, 0, 0, .35); + outline: 1px solid rgba(0, 0, 0, .45); +} + +.stage { display: block; background: var(--stage); } + +.paint-overlay { position: absolute; inset: 0; touch-action: none; } +.paint-overlay.drawing { cursor: crosshair; } +.paint-overlay circle { cursor: grab; } + +/* -------------------------------------------------------------------------- + params */ + +.section { border-bottom: 1px solid var(--hair); padding: 6px; } +.section:last-child { border-bottom: 0; } + +.section > h2 { + margin: 0 0 5px; + font: inherit; + font-weight: 600; + color: var(--dim); + text-transform: uppercase; + letter-spacing: .07em; +} + +/* Label / value, once, for every read-only fact in the pane. */ +.facts { display: grid; grid-template-columns: auto 1fr; gap: 2px 8px; margin: 0; } +.facts dt { color: var(--dim); } +.facts dd { margin: 0; overflow: hidden; text-overflow: ellipsis; } + +.knob { display: grid; gap: 1px; margin-bottom: 6px; } +.knob .top-line { display: flex; justify-content: space-between; gap: 8px; } +.knob .top-line .name { color: var(--dim); } + +.regeneration-debug { + padding: 4px 5px; + background: var(--sunk); + border: 1px solid var(--hair); + border-radius: 2px; +} + +/* -------------------------------------------------------------------------- + timeline */ + +.time { overflow: hidden; } + +.tl-body { + flex: 1; + min-height: 0; + display: flex; + overflow: auto; +} + +.tl-labels { + flex: 0 0 var(--label); + /* A flex item's `min-width` defaults to `auto`, which floors its used size at + its MIN-CONTENT width — so a row labelled with a uuid widens the whole + column and pushes the tracks out of alignment with the ruler above them. + `0` is what makes the declared basis the actual width. */ + min-width: 0; + position: sticky; + left: 0; + z-index: 1; + background: var(--pane); + border-right: 1px solid var(--line); +} + +.tl-tracks { + flex: 1; + min-width: 340px; + position: relative; + background: #fff; + /* The frame grid, five frames to a division, as a background rather than as + an element per frame: a 900-frame take is 900 divs nobody needs in the DOM. + `--tick` is five frames as a percentage of the span, set from the component + because only it knows how long the clip is. */ + background-image: + repeating-linear-gradient(90deg, + var(--grid-5) 0 1px, transparent 1px var(--tick, 10%)); +} + +.tl-label, +.tl-track { + height: var(--row); + border-bottom: 1px solid var(--hair); +} + +/* The row is the containing block for its own span and keys. Without this they + resolve against `.tl-tracks` instead and every row's marks land stacked on the + first one, full height — which reads as "the timeline draws nothing" rather + than as a positioning bug. */ +.tl-track { position: relative; } + +.tl-label { + display: flex; + align-items: center; + gap: 3px; + padding-right: 6px; + cursor: default; + white-space: nowrap; + overflow: hidden; +} + +.tl-label.on { background: var(--sel-bg); } +.tl-label:hover:not(.on) { background: #fff; } +.tl-label .name { min-width: 0; overflow: hidden; text-overflow: ellipsis; } +.tl-label .kind { color: var(--dim); } + +/* A fixed-width cell whether or not there is a triangle in it, so names at the + same depth line up down the column. */ +.tl-twist { + flex: 0 0 13px; + padding: 0; + border: 0; + border-radius: 0; + background: none; + color: var(--dim); + cursor: pointer; + text-align: center; +} + +.tl-twist:hover:not(:disabled) { background: none; color: var(--fg); } +.tl-twist:disabled { opacity: 0; cursor: default; } + +.tl-ruler { + position: relative; + height: var(--ruler); + border-bottom: 1px solid var(--line); + background: var(--chrome); + cursor: pointer; + user-select: none; +} + +.tl-ruler .tick { + position: absolute; + top: 0; + bottom: 0; + border-left: 1px solid var(--line); + padding-left: 3px; + color: var(--dim); + font-size: 10px; + line-height: var(--ruler); +} + +/* The span the node exists over. Drawn under the keys so a dot on the first + frame of a span is not half-hidden by its own bar. */ +.tl-span { + position: absolute; + top: 3px; + bottom: 3px; + background: var(--span); + border: 1px solid var(--span-line); + border-radius: 1px; +} + +.tl-span.dense { + /* Generated, one value per frame: hatched, because ticking every frame would + be a solid block that says less than the bar behind it does. */ + background: repeating-linear-gradient( + -45deg, var(--span) 0 3px, #c3d4ea 3px 6px); +} + +/* Flash's keyframe: a filled dot. */ +.tl-key { + position: absolute; + top: 50%; + width: 7px; + height: 7px; + margin: -3.5px 0 0 -3.5px; + background: var(--key); + border-radius: 50%; +} + +.tl-playhead { + position: absolute; + top: 0; + bottom: 0; + width: 1px; + background: var(--playhead); + pointer-events: none; + z-index: 3; +} + +.tl-playhead::before { + content: ""; + position: absolute; + top: 1px; + left: -4px; + width: 9px; + height: 9px; + border-radius: 50%; + background: var(--playhead); +} + +.tl-empty { padding: 9px; color: var(--dim); } From 5dff4901624d0db11a76837218747bb64f402417 Mon Sep 17 00:00:00 2001 From: Olive Vaughn Date: Tue, 29 Sep 2026 12:46:42 -0400 Subject: [PATCH 02/26] Symbols, not timelines; no symbol is special Everything that holds nodes is a symbol (domain/timeline -> domain/symbol, :timelines -> :symbols) and a node that places one is :kind :instance. The reserved :main root is gone: which symbol is on screen is editor state ([:ui :open]), every domain function that needs a symbol is told which, and a document opens on the longest symbol nothing else places. Saved projects move to schema 2 through migration 0007, which rewrites leaf paths, instance kinds and the feature :symbol key; the client refuses a schema it does not read. Co-Authored-By: Claude Opus 5.5 --- .../migrations/0007_symbols_not_timelines.py | 75 +++ clips/models.py | 2 +- clips/tests/test_api.py | 24 +- docs/animation-model.md | 15 +- docs/architecture.md | 20 +- docs/multi-face-representation.md | 6 +- docs/timing-handoff.md | 2 +- frontend/README.md | 28 +- frontend/src/arthur/audio/mix.cljs | 39 +- frontend/src/arthur/db.cljs | 32 +- frontend/src/arthur/demo.cljs | 14 +- frontend/src/arthur/demo/scene.edn | 2 +- frontend/src/arthur/demo/stage.cljs | 6 +- frontend/src/arthur/demo/swarm.cljs | 2 +- frontend/src/arthur/demo/take.cljs | 2 +- frontend/src/arthur/domain/clip.cljs | 257 ++++---- frontend/src/arthur/domain/feature.cljs | 16 +- frontend/src/arthur/domain/leaf.cljs | 88 +-- frontend/src/arthur/domain/node.cljs | 18 +- frontend/src/arthur/domain/paint.cljs | 31 +- frontend/src/arthur/domain/png.cljs | 2 +- frontend/src/arthur/domain/pose.cljs | 23 +- frontend/src/arthur/domain/project.cljs | 10 +- frontend/src/arthur/domain/raster.cljs | 2 +- .../domain/{timeline.cljs => symbol.cljs} | 130 ++-- frontend/src/arthur/events/export.cljs | 99 +-- frontend/src/arthur/events/footage.cljs | 11 +- frontend/src/arthur/events/paint.cljs | 16 +- frontend/src/arthur/events/playback.cljs | 52 +- frontend/src/arthur/events/project.cljs | 56 +- frontend/src/arthur/events/ui.cljs | 16 +- frontend/src/arthur/export.cljs | 86 +-- frontend/src/arthur/flow/freeze.cljs | 34 +- frontend/src/arthur/flow/regenerate.cljs | 4 +- frontend/src/arthur/fx/http.cljs | 4 +- frontend/src/arthur/subs/playback.cljs | 1 - frontend/src/arthur/subs/render.cljs | 35 +- frontend/src/arthur/subs/ui.cljs | 6 +- frontend/src/arthur/ui/params.cljs | 41 +- frontend/src/arthur/ui/player.cljs | 2 +- frontend/src/arthur/ui/pool.cljs | 24 +- frontend/src/arthur/ui/stage.cljs | 28 +- frontend/src/arthur/ui/timeline.cljs | 59 +- frontend/src/arthur/ui/topbar.cljs | 10 +- frontend/test/arthur/bench_test.cljs | 6 +- frontend/test/arthur/domain/feature_test.cljs | 4 +- .../test/arthur/domain/instance_test.cljs | 236 +++++++ frontend/test/arthur/domain/leaf_test.cljs | 50 +- frontend/test/arthur/domain/node_test.cljs | 2 +- frontend/test/arthur/domain/paint_test.cljs | 22 +- frontend/test/arthur/domain/project_test.cljs | 20 +- frontend/test/arthur/domain/symbol_test.cljs | 576 ++++++++++++------ .../test/arthur/domain/timeline_test.cljs | 397 ------------ frontend/test/arthur/events/export_test.cljs | 46 +- frontend/test/arthur/export/frames_test.cljs | 2 +- frontend/test/arthur/export_test.cljs | 62 +- .../test/arthur/flow/eye_occlusion_test.cljs | 6 +- frontend/test/arthur/flow/freeze_test.cljs | 62 +- .../test/arthur/flow/multi_face_test.cljs | 43 +- .../test/arthur/flow/regenerate_test.cljs | 36 +- frontend/test/arthur/support/ops.cljs | 18 +- 61 files changed, 1587 insertions(+), 1431 deletions(-) create mode 100644 clips/migrations/0007_symbols_not_timelines.py rename frontend/src/arthur/domain/{timeline.cljs => symbol.cljs} (84%) create mode 100644 frontend/test/arthur/domain/instance_test.cljs delete mode 100644 frontend/test/arthur/domain/timeline_test.cljs diff --git a/clips/migrations/0007_symbols_not_timelines.py b/clips/migrations/0007_symbols_not_timelines.py new file mode 100644 index 0000000..1f726c8 --- /dev/null +++ b/clips/migrations/0007_symbols_not_timelines.py @@ -0,0 +1,75 @@ +"""Schema 2: a document holds symbols, not timelines, and no symbol is reserved. + +Three renames, each in the stored transit and nowhere else: + + clip//timeline/... -> clip//symbol/... + a node leaf's :kind :symbol -> :kind :instance + a feature leaf's :timeline key -> :symbol + +A leaf value is transit's map form, ["^ ", k1, v1, k2, v2, ...]. Only TOP-LEVEL +pairs are rewritten, and only literal ones: transit caches a repeated keyword as +"^N", and a rename that met a cache reference where it expected the keyword would +be guessing. Every saved leaf at the time of writing had these as literals; if one +does not, the migration stops rather than writing a document that decodes to +something else. + +Renaming a cached keyword in place is safe because the cache is positional: the +literal keeps its slot, so any later "^N" that referred to it now refers to the +new name, which is what it meant. +""" + +import re + +from django.db import migrations, models + +PATH = re.compile(r"^(clip/[^/]+/)timeline(/|$)") + + +def _rename_pair(value, key, old, new, path): + if not (isinstance(value, list) and value[:1] == ["^ "]): + return value + out = list(value) + for i in range(1, len(out) - 1, 2): + if out[i] != key: + continue + if old is None: + out[i] = new + elif out[i + 1] == old: + out[i + 1] = new + elif isinstance(out[i + 1], str) and out[i + 1].startswith("^") and out[i + 1] != "^ ": + raise RuntimeError(f"leaf {path!r} has a cached {key} value; migrate it by hand") + return out + + +def forwards(apps, schema_editor): + Leaf = apps.get_model("clips", "Leaf") + Project = apps.get_model("clips", "Project") + for leaf in Leaf.objects.all(): + path = PATH.sub(r"\1symbol\2", leaf.path) + value = leaf.value + parts = path.split("/") + if len(parts) == 6 and parts[2] == "symbol" and parts[4] == "node": + value = _rename_pair(value, "~:kind", "~:symbol", "~:instance", leaf.path) + if len(parts) == 4 and parts[2] == "feature": + value = _rename_pair(value, "~:timeline", None, "~:symbol", leaf.path) + if path != leaf.path or value != leaf.value: + leaf.path = path + leaf.value = value + leaf.version += 1 + leaf.save(update_fields=["path", "value", "version"]) + Project.objects.update(schema_version=2) + + +class Migration(migrations.Migration): + dependencies = [ + ("clips", "0006_project_schema_version"), + ] + + operations = [ + migrations.AlterField( + model_name="project", + name="schema_version", + field=models.PositiveIntegerField(default=2), + ), + migrations.RunPython(forwards, migrations.RunPython.noop), + ] diff --git a/clips/models.py b/clips/models.py index b120873..c561a03 100644 --- a/clips/models.py +++ b/clips/models.py @@ -206,7 +206,7 @@ class Project(models.Model): id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) name = models.CharField(max_length=200, default="untitled") - schema_version = models.PositiveIntegerField(default=1) + schema_version = models.PositiveIntegerField(default=2) seq = models.PositiveBigIntegerField(default=0) palette = models.CharField(max_length=64, default="arthur/default") created = models.DateTimeField(auto_now_add=True) diff --git a/clips/tests/test_api.py b/clips/tests/test_api.py index 08554ad..6d67bd5 100644 --- a/clips/tests/test_api.py +++ b/clips/tests/test_api.py @@ -382,13 +382,13 @@ class DocumentTests(TestCase): # cache marker, keyword keys, and a frame-keyed inner map. return { "clip/c1/timing": ["^ ", "~:fps", 30], - "clip/c1/timeline/main": ["^ ", "~:frames", 48], - "clip/c1/timeline/main/node/mouth": ["^ ", "~:id", "~:mouth", "~:z", "a1"], - "clip/c1/timeline/main/channel/mouth/geom.pts": [ + "clip/c1/symbol/main": ["^ ", "~:frames", 48], + "clip/c1/symbol/main/node/mouth": ["^ ", "~:id", "~:mouth", "~:z", "a1"], + "clip/c1/symbol/main/channel/mouth/geom.pts": [ "^ ", "~:animated?", True, "~:dense", ["^ ", "~:store", self.block, "~:offset", 0, "~:stride", 16], ], - "clip/c1/timeline/main/channel/mouth-in/vis": [ + "clip/c1/symbol/main/channel/mouth-in/vis": [ "^ ", "~:animated?", True, "~:keys", ["^ ", "~i0", True, "~i12", False], ], } @@ -407,7 +407,7 @@ class DocumentTests(TestCase): self.assertEqual(5, len(response.json()["written"])) loaded = self.client.get(f"/api/projects/{self.project.id}").json() - self.assertEqual(1, loaded["schema_version"]) + self.assertEqual(2, loaded["schema_version"]) self.assertEqual(1, len(loaded["clips"])) clip = loaded["clips"][0] self.assertEqual("c1", clip["cid"]) @@ -424,22 +424,22 @@ class DocumentTests(TestCase): self.save() first = {leaf.path: leaf.version for leaf in Leaf.objects.all()} moved = self.leaves() - moved["clip/c1/timeline/main/channel/mouth-in/vis"] = [ + moved["clip/c1/symbol/main/channel/mouth-in/vis"] = [ "^ ", "~:animated?", True, "~:keys", ["^ ", "~i0", False], ] response = self.save(moved) - self.assertEqual(["clip/c1/timeline/main/channel/mouth-in/vis"], response.json()["written"]) + self.assertEqual(["clip/c1/symbol/main/channel/mouth-in/vis"], response.json()["written"]) self.assertEqual(4, response.json()["unchanged"]) after = {leaf.path: leaf.version for leaf in Leaf.objects.all()} - self.assertEqual(2, after["clip/c1/timeline/main/channel/mouth-in/vis"]) + self.assertEqual(2, after["clip/c1/symbol/main/channel/mouth-in/vis"]) self.assertEqual(first["clip/c1/timing"], after["clip/c1/timing"]) def test_a_removed_node_removes_its_leaf(self): self.save() fewer = {k: v for k, v in self.leaves().items() - if k != "clip/c1/timeline/main/node/mouth"} + if k != "clip/c1/symbol/main/node/mouth"} response = self.save(fewer) - self.assertEqual(["clip/c1/timeline/main/node/mouth"], response.json()["removed"]) + self.assertEqual(["clip/c1/symbol/main/node/mouth"], response.json()["removed"]) self.assertEqual(4, Leaf.objects.count()) def test_a_save_does_not_disturb_another_clip(self): @@ -472,7 +472,7 @@ class DocumentTests(TestCase): def test_a_leaf_write_carries_an_etag(self): self.save() - url = f"/api/projects/{self.project.id}/leaves/clip/c1/timeline/main/node/mouth" + url = f"/api/projects/{self.project.id}/leaves/clip/c1/symbol/main/node/mouth" got = self.client.get(url) self.assertEqual('"1"', got["ETag"]) @@ -488,7 +488,7 @@ class DocumentTests(TestCase): # take-theirs. A PUT that replaced unconditionally is the bug where the # loser's work disappears silently. self.save() - url = f"/api/projects/{self.project.id}/leaves/clip/c1/timeline/main/node/mouth" + url = f"/api/projects/{self.project.id}/leaves/clip/c1/symbol/main/node/mouth" self.put(url, {"value": ["^ ", "~:z", "a2"]}, HTTP_IF_MATCH='"1"') stale = self.put(url, {"value": ["^ ", "~:z", "a3"]}, HTTP_IF_MATCH='"1"') self.assertEqual(409, stale.status_code) diff --git a/docs/animation-model.md b/docs/animation-model.md index 4c1a2bc..a8393cf 100644 --- a/docs/animation-model.md +++ b/docs/animation-model.md @@ -415,9 +415,11 @@ different rules: *not* to the plate, which is the whole point of it — so the offset genuinely belongs at the node, not the clip. -## Timelines, and why a scene is one +## Symbols, and why a scene is one -A **timeline** is an ordered bag of nodes in its own frame space: +A **symbol** is an ordered bag of nodes in its own frame space. (Earlier drafts +and code called this a *timeline*; that word now means only the UI pane that +shows one.) ```clojure {:frames 91 @@ -427,9 +429,10 @@ A **timeline** is an ordered bag of nodes in its own frame space: That is the whole type, and **everything that holds nodes is one of these**: -- a clip's **scene** is its root timeline, -- a **symbol** in the library is a timeline, -- a node with `:kind :symbol` is an **instance** of one. +- what a document opens on is a symbol, and **no symbol is reserved** — a new + document's is called `main` only because it has to be called something, +- anything placed inside another symbol is a symbol, +- a node with `:kind :instance` is an **instance** of one. An earlier draft of this document had a scene and a `:kind :timeline` symbol as two structures with the same fields and never said they were the same thing. @@ -474,7 +477,7 @@ for all three is the same — **their own**: ### Instances -A node with `:kind :symbol` and `:of :sym/blink` places one. Its own channels +A node with `:kind :instance` and `:of :sym/blink` places one. Its own channels compose *over* the symbol's, so one definition is placed many times and tinted, offset or retimed at each placement — that is how a three-frame blink is reused at frames 40, 88 and 200 without copying it. diff --git a/docs/architecture.md b/docs/architecture.md index 90d6536..09bc757 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -639,10 +639,10 @@ is what step 9 implemented, for the subset that exists: clip//name clip//subject/ clip//timing clip//feature/ clip//stage clip//group/ -clip//source clip//timeline/ -clip//timeline//node/ -clip//timeline//measured/ -clip//timeline//channel// +clip//source clip//symbol/ +clip//symbol//node/ +clip//symbol//measured/ +clip//symbol//channel// ``` Settings live on subject, feature and group leaves. Each feature has one area, so @@ -785,12 +785,12 @@ clip/:cid/timing clip rate clip/:cid/subject/:sid tracked subject and settings clip/:cid/feature/:fid tracked feature and settings clip/:cid/group/:gid shared settings for an eye pair -clip/:cid/timeline/:tid frame count, palette -clip/:cid/timeline/:tid/node/:nid one node: parent, stencil, z, time -clip/:cid/timeline/:tid/channel/:nid/:prop -clip/:cid/timeline/:tid/measured/:nid -clip/:cid/timeline/:tid/cel/:nid/:frame -clip/:cid/timeline/:tid/overrides/:nid/:prop +clip/:cid/symbol/:sid frame count, palette +clip/:cid/symbol/:sid/node/:nid one node: parent, stencil, z, time +clip/:cid/symbol/:sid/channel/:nid/:prop +clip/:cid/symbol/:sid/measured/:nid +clip/:cid/symbol/:sid/cel/:nid/:frame +clip/:cid/symbol/:sid/overrides/:nid/:prop ``` Each feature and node has its own leaf, so tuning separate features and adding diff --git a/docs/multi-face-representation.md b/docs/multi-face-representation.md index 65584c3..51ae686 100644 --- a/docs/multi-face-representation.md +++ b/docs/multi-face-representation.md @@ -8,11 +8,11 @@ symbol instance. Timelines already provide local node names, independent playbac and persistence. No new kind of scene container is needed. ```clojure -:timelines +:symbols {:main {:nodes {:root {:time {:mode :map :expose 2}} :face {:parent :root :channels } - :face-1 {:kind :symbol :of :face-1 :parent :face :z "a0"} - :face-2 {:kind :symbol :of :face-2 :parent :face :z "a1"}}} + :face-1 {:kind :instance :of :face-1 :parent :face :z "a0"} + :face-2 {:kind :instance :of :face-2 :parent :face :z "a1"}}} :face-1 {:nodes {:head {...} :mouth {:parent :head ...} ...}} :face-2 {:nodes {:head {...} :mouth {:parent :head ...} ...}}} diff --git a/docs/timing-handoff.md b/docs/timing-handoff.md index 30e15f7..b9fa406 100644 --- a/docs/timing-handoff.md +++ b/docs/timing-handoff.md @@ -70,7 +70,7 @@ handling and the relevant key whitelist if its storage location requires it. - `freeze/performance-nodes` marks generated animated channels with `:pose-sampled?` and local `:pose-group` names. This includes keyed visibility as well as dense geometry. `:generated` remains provenance for regeneration. -- `timeline/channel-frame` already applies explicit pose choices and default +- `symbol/channel-frame` already applies explicit pose choices and default picture sampling to marked channels. Playback and export both use `clip/resolver` with `:picture-fps`; there is no need for a second sampling implementation. Export's pose count is still a rate-based estimate. diff --git a/frontend/README.md b/frontend/README.md index fff8e0f..42dc4ba 100644 --- a/frontend/README.md +++ b/frontend/README.md @@ -116,8 +116,14 @@ projects the server holds; the built-in scenes are under their own heading, italic, and are not projects — they are compiled into the bundle and the server has never heard of them. -Selection lives in app-db under `:ui`, as `[:node ]`, -`[:timeline ]` or `[:subject|:feature|:group ]` — four panes ask what is +Everything that holds nodes is a **symbol**, and none is special: a new document +has one called `main` because it has to be called something. Which symbol is on +screen is editor state, `[:ui :open]`, not a fact about the document — the stage +draws it, the timeline lists it, the transport plays it and a new shape goes into +it. A document opens on the longest symbol nothing else places. + +Selection lives in app-db under `:ui`, as `[:node ]`, +`[:symbol ]` or `[:subject|:feature|:group ]` — four panes ask what is selected, and a ratom private to one of them can only be shared by making the other three require it. @@ -125,12 +131,12 @@ other three require it. into detection. Dragging a symbol out of the pool onto the stage places an instance of it at the playhead. -The timeline's rows are the open clip's nodes, front-most first, with a dot per +The timeline's rows are the open symbol's nodes, front-most first, with a dot per keyframe and a bar over the frames the node exists on; a dense channel is hatched rather than ticked, because one value per frame is a solid block that says less -than the bar does. Opening a row shows its channels; opening a **symbol** row -shows the timeline it instances, with every frame number mapped back into the -stage's own frame space — see the namespace docstring in `ui/timeline.cljs`, which +than the bar does. Opening a row shows its channels; opening an **instance** row +shows the symbol it places, with every frame number mapped back into the open +symbol's frame space — see the namespace docstring in `ui/timeline.cljs`, which is where that mapping is argued. ### Paint sketch @@ -168,9 +174,9 @@ and real footage use `src/arthur/flow/take.cljs` for the measurement order and `src/arthur/flow/freeze.cljs` for the landmark-to-channel conversion. **8625 stage study**, in the open menu, loads the locally saved `IMG_8625.MOV` project and places its -post-processed timeline twice. The stage layout is +post-processed face symbol twice. The stage layout is `src/arthur/demo/stage_8625.edn`: the right picture and sound start at frame 48, -and the two pictures overlap slightly in stage space. Audio has its own timeline +and the two pictures overlap slightly in stage space. Audio has its own nodes, linked to the picture instances but with independent spans and gain channels. The right sound swells and pans across the stage, then fades out at frame 260 while its picture continues to @@ -357,12 +363,12 @@ them is `clips/templates/clips/index.html`. ## Two evaluators, on purpose -`domain/timeline` has both `eval-frame` and `resolver`, and they are not +`domain/symbol` has both `eval-frame` and `resolver`, and they are not alternatives: -- **`(eval-frame timeline f store)`** is the specification. Allocating, order-free, +- **`(eval-frame symbol f store)`** is the specification. Allocating, order-free, obviously correct. Tests and one-off renders use it. -- **`(resolver timeline store)` -> `(fn [f] ops)`** is what playback uses. It caches +- **`(resolver symbol store)` -> `(fn [f] ops)`** is what playback uses. It caches the topological order and the z paths, holds a cursor per channel and reuses one point buffer per node, so a frame allocates the op maps and nothing else. diff --git a/frontend/src/arthur/audio/mix.cljs b/frontend/src/arthur/audio/mix.cljs index cec1e6f..98dea64 100644 --- a/frontend/src/arthur/audio/mix.cljs +++ b/frontend/src/arthur/audio/mix.cljs @@ -91,18 +91,15 @@ (.setValueAtTime param (* factor v) (/ f fps))))))) (defn tracks-of - "The audio nodes of one of the clip's timelines. + "The audio nodes of one of the clip's symbols. Any symbol may carry its own + sound, and playback mixes the open one's." + [document sid] + (filter #(= :audio (:kind %)) (vals (:nodes (clip/symbol document sid))))) - A timeline parameter rather than always the root, because a symbol is a - timeline and may carry its own sound. `:main` is the clip's own, which is what - playback mixes." - [document tid] - (filter #(= :audio (:kind %)) (vals (:nodes (clip/timeline document tid))))) - -(defn- render! [document tid sources store] +(defn- render! [document sid sources store] (let [fps (:fps document) - frames (:frames (clip/timeline document tid)) - tracks (tracks-of document tid) + frames (:frames (clip/symbol document sid)) + tracks (tracks-of document sid) output (js/OfflineAudioContext. 2 (js/Math.ceil (* (/ frames fps) 44100)) 44100)] (doseq [track tracks] @@ -133,20 +130,20 @@ (.startRendering output))) (defn buffer! - "Promise of the `AudioBuffer` one timeline's audio tracks mix down to, or nil + "Promise of the `AudioBuffer` one symbol's audio tracks mix down to, or nil when it has none. The raw product. `mix!` packages it as a WAV URL for the transport and `export/frames` packages it as WAV bytes in an archive; a muxer would take it as it is, which is why this is the function the others are written in terms of." - ([document tid] (buffer! document tid nil)) - ([document tid store] - (let [tracks (tracks-of document tid)] + ([document sid] (buffer! document sid nil)) + ([document sid store] + (let [tracks (tracks-of document sid)] (if (empty? tracks) (js/Promise.resolve nil) (-> (js/Promise.all (into-array (map source! (distinct (map #(get-in % [:source :footage]) tracks))))) - (.then (fn [pairs] (render! document tid (into {} (array-seq pairs)) store)))))))) + (.then (fn [pairs] (render! document sid (into {} (array-seq pairs)) store)))))))) (defn decode! "Promise of the `AudioBuffer` behind a URL. What a clip whose audio is a plain @@ -162,9 +159,9 @@ (.decodeAudioData (js/OfflineAudioContext. 1 1 44100) bytes))))) (defn mix! - "Promise of a mixed WAV URL, or the original URL for a clip without audio - tracks. Each track can be trimmed and faded independently of its linked picture." - ([document fallback-url] (mix! document fallback-url nil)) - ([document fallback-url store] - (-> (buffer! document clip/root-id store) - (.then (fn [buffer] (if buffer (wav-url buffer) fallback-url)))))) + "Promise of a mixed WAV URL for symbol `sid`, or the original URL when it has + no audio tracks. Each track can be trimmed and faded independently of its + linked picture." + [document sid fallback-url store] + (-> (buffer! document sid store) + (.then (fn [buffer] (if buffer (wav-url buffer) fallback-url))))) diff --git a/frontend/src/arthur/db.cljs b/frontend/src/arthur/db.cljs index 5ad50aa..740faca 100644 --- a/frontend/src/arthur/db.cljs +++ b/frontend/src/arthur/db.cljs @@ -20,10 +20,8 @@ Read OFF the clip rather than written again beside it: copying a number by hand into this table is how it comes to disagree with the document it describes. - `:frames` comes from the ROOT TIMELINE and `:fps` from the clip, which is the - split `arthur.domain.clip` exists to make — a timeline is a frame space, a clip - is a rate — and an earlier version of this docstring noted that they sat on one - map \"only because there is one clip per scene today\". They do not any more." + There is no `:frames` here, because a length belongs to a symbol and which + symbol is open is the editor's state — see `events/playback/frames`." [label-key label clip store] (merge {:label label :clip clip :store store ;; A static asset since step 9, and not the repo root's `audio.wav`. @@ -32,8 +30,7 @@ ;; that the clock has something to run against with no footage ingested. :audio "/static/arthur/audio.wav" :cid (name label-key) - :display-fps (:fps clip) - :frames (domain-clip/frames clip)} + :display-fps (:fps clip)} (select-keys clip [:fps :width :height]))) (def clips @@ -72,7 +69,7 @@ ;; transform on a node, so nothing downstream of the freeze knows the frame ;; size — and it is why ui/player no longer hardcodes 320x200. :clip (let [c (domain-clip/blank)] - {:fps (:fps c) :frames (domain-clip/frames c) + {:fps (:fps c) :width (:width c) :height (:height c) :audio nil :display-fps (:fps c)}) @@ -106,11 +103,12 @@ ;; machinery that would share it. ;; --- export --- ;; - ;; The REQUEST and its progress, never the frames. Which timeline to write and + ;; The REQUEST and its progress, never the frames. Which symbol to write and ;; at what integer zoom is authored state like anything else; the megabytes the ;; render produces are handed straight to a download and never enter the db. - ;; `:isolate` is the placement to render alone, or nil for the whole timeline. - :export {:timeline :main :isolate nil :zoom 4 :busy? false :done 0 :total 0 + ;; `:isolate` is the placement to render alone, or nil for the whole symbol; + ;; `:symbol` nil means whichever symbol is open. + :export {:symbol nil :isolate nil :zoom 4 :busy? false :done 0 :total 0 :status nil} :playback {:frame 0 @@ -134,20 +132,28 @@ ;; names, so a pane dispatches on it rather than on which of several ;; "selected-x" keys happens to be non-nil: ;; - ;; [:node ] a shape or a placement - ;; [:timeline ] a timeline, root or library + ;; [:node ] a shape or an instance + ;; [:symbol ] a symbol ;; [:subject ] [:feature ] [:group ] a tracked object ;; ;; `:draft` is the polygon being clicked out, flat [x y x y …] as geometry is ;; stored everywhere. `:expanded` holds timeline row PATHS — a path and not a ;; node id, because one symbol placed twice is two rows that open separately. ;; + ;; `:open` is the symbol on screen — the one the stage draws, the timeline + ;; lists, the transport plays and a new shape goes into — and `:tabs` the + ;; symbols open beside it. Editor state and not the document's, because no + ;; symbol is special to the document: which one you are looking at is a fact + ;; about you. + ;; ;; `:knobs` holds a generated setting's value WHILE THE REGENERATION IS IN ;; FLIGHT, keyed by [scope id knob]. Moving a slider dispatches a preview that ;; re-freezes blocks asynchronously, so until it lands the clip still reports ;; the old value — and a slider reading from the clip would spring back under ;; the user's finger on every frame of the drag. - :ui {:selection nil + :ui {:open nil + :tabs [] + :selection nil :tone :skin-base :tool nil :draft [] diff --git a/frontend/src/arthur/demo.cljs b/frontend/src/arthur/demo.cljs index 3d89a1e..46b527f 100644 --- a/frontend/src/arthur/demo.cljs +++ b/frontend/src/arthur/demo.cljs @@ -6,7 +6,7 @@ validates would not be the one that renders, and the model would be validated against a scene nobody ever looked at." (:require [arthur.domain.clip :as domain-clip] - [arthur.domain.timeline :as timeline] + [arthur.domain.symbol :as symbol] [cljs.reader :as reader] [shadow.resource :as rc])) @@ -14,15 +14,15 @@ (def clip (reader/read-string source)) -(def timeline - "The clip's root timeline: what an evaluator takes. `clip` is the document." - (domain-clip/root clip)) +(def main + "The scene's one symbol: what an evaluator takes. `clip` is the document." + (domain-clip/symbol clip :main)) (def fps (:fps clip)) -(def frames (domain-clip/frames clip)) +(def frames (domain-clip/frames clip :main)) (defn ops-at "Draw ops for one frame, via the specification path. The page uses - `timeline/resolver` instead; this is here for the REPL." + `symbol/resolver` instead; this is here for the REPL." [f] - (timeline/eval-frame timeline f)) + (symbol/eval-frame main f)) diff --git a/frontend/src/arthur/demo/scene.edn b/frontend/src/arthur/demo/scene.edn index f445462..19a6d08 100644 --- a/frontend/src/arthur/demo/scene.edn +++ b/frontend/src/arthur/demo/scene.edn @@ -32,7 +32,7 @@ :width 320 :height 200 - :timelines + :symbols {:main {:id :main :frames 229 diff --git a/frontend/src/arthur/demo/stage.cljs b/frontend/src/arthur/demo/stage.cljs index 9119ed3..b0645f8 100644 --- a/frontend/src/arthur/demo/stage.cljs +++ b/frontend/src/arthur/demo/stage.cljs @@ -35,7 +35,7 @@ (let [{:keys [name width height frames symbol instances audio scale]} layout default-anchor (or (:anchor layout) [(/ (:width source) 2) (/ (:height source) 2)]) - original (get-in source [:timelines :main]) + original (get-in source [:symbols :main]) ;; Authored id -> uuid, so the `:linked-to` in the EDN resolves to the ;; identity the document uses. Built before either pass because the audio ;; nodes refer to the instances. @@ -49,7 +49,7 @@ {:root {:id :root :name "stage" :kind :group :z "a1"}} (map (fn [{:keys [uuid name z span at in center anchor drift phase]}] (let [anchor (or anchor default-anchor)] - [uuid {:id uuid :name name :kind :symbol :of symbol + [uuid {:id uuid :name name :kind :instance :of symbol :parent :root :z z :span span :time {:mode :map :at at :in in :rate 1} :channels {[:xform :pos] (if drift @@ -68,6 +68,6 @@ pan (assoc [:audio :pan] pan))}]) audio))] (assoc source :name name :width width :height height - :timelines (assoc (:timelines source) + :symbols (assoc (:symbols source) :main {:id :main :frames frames :nodes nodes} symbol (assoc original :id symbol))))) diff --git a/frontend/src/arthur/demo/swarm.cljs b/frontend/src/arthur/demo/swarm.cljs index 0ef0010..e03f103 100644 --- a/frontend/src/arthur/demo/swarm.cljs +++ b/frontend/src/arthur/demo/swarm.cljs @@ -153,7 +153,7 @@ :fps fps :width 320 :height 200 - :timelines + :symbols {:main {:id :main :frames frames diff --git a/frontend/src/arthur/demo/take.cljs b/frontend/src/arthur/demo/take.cljs index 9f5b25a..dc3e737 100644 --- a/frontend/src/arthur/demo/take.cljs +++ b/frontend/src/arthur/demo/take.cljs @@ -12,7 +12,7 @@ │ FREEZE ──▶ channels on nodes │ - timeline/resolver ──▶ raster + symbol/resolver ──▶ raster — and the order of that diagram is the whole argument for the stage split. The anchor fit is knob-free. Conditioning smooths its four parameters. The rings are diff --git a/frontend/src/arthur/domain/clip.cljs b/frontend/src/arthur/domain/clip.cljs index 16a6a6e..ffa6d1c 100644 --- a/frontend/src/arthur/domain/clip.cljs +++ b/frontend/src/arthur/domain/clip.cljs @@ -1,47 +1,43 @@ (ns arthur.domain.clip - "A CLIP: the unit of work, and a library of timelines. + "A CLIP: the unit of work, and a library of symbols. {:name \"take\" :fps 30 :width 320 :height 200 :analysis {...} :subjects {...} :features {...} :groups {...} - :timelines {:main {:id :main :frames 229 :nodes {...}}}} + :symbols {:main {:id :main :frames 229 :nodes {...}}}} Every field here is a fact about the clip and NOT about a bag of nodes, which is the cut this namespace exists to make. Before it, one map carried both: `:fps`, the stage dimensions, the analysis record and the tracking identities sat beside - `:nodes`, and `arthur.db` said of it — correctly — that they \"sit on the scene - map only because there is one clip per scene today\". The cost of leaving them - together was not untidiness. It was that a SYMBOL had nowhere to live: a library - timeline is a bag of nodes with a frame space and nothing else, so under the old - shape it would have had to be a clip with seven meaningless fields, or a second - structure with the same `:nodes` key that every walk had to be taught about. + `:nodes`. The cost of leaving them together was not untidiness. It was that a + SYMBOL had nowhere to live: a symbol is a bag of nodes with a frame space and + nothing else, so under the old shape it would have had to be a clip with seven + meaningless fields. - Now there is one node-holding type — `arthur.domain.timeline` — and a clip holds - a MAP of them. A `:kind :symbol` instance names a timeline in `:timelines`, - and the clip resolver gives each placement its own reading heads. + Now there is one node-holding type — `arthur.domain.symbol` — and a clip holds + a MAP of them. A `:kind :instance` node places one symbol inside another, and + the clip resolver gives each instance its own reading heads. - THE ROOT TIMELINE HAS A RESERVED ID, `:main`, rather than the clip carrying a - pointer to it. A pointer is a field that can be wrong — it can name a timeline - that is not there, and then every reader needs a fallback — where a reserved name - can only be absent, which `problems` reports once. Flash reserves `_root` the - same way and for the same reason. Nothing else about `:main` is special: it is an - ordinary entry in the map, and a symbol is another one. + NO SYMBOL IS SPECIAL. There is no reserved root and no pointer to one: which + symbol is on screen is the editor's state, not the document's, and every + function here that needs a symbol is told which. A new document has one symbol + called `:main` because it has to be called something, and that is all the name + means — it can be renamed, placed inside another symbol or deleted like any of + them. `unplaced` answers the question a reserved root used to: which symbols + nothing else places, and so which ones a person opening the document wants. WHY :fps IS HERE AND :frames IS NOT. A rate is how fast the whole clip plays - against its audio, and a nested timeline cannot have one of its own — retiming an + against its audio, and a nested symbol cannot have one of its own — retiming an instance is `:rate` on its `:time` map, which is a factor and not a rate. A - frame COUNT is a property of a frame space, so every timeline has its own." + frame COUNT is a property of a frame space, so every symbol has its own." + (:refer-clojure :exclude [symbol]) (:require [arthur.domain.feature :as feature] [arthur.domain.node :as node] [arthur.domain.palette :as pal] [arthur.domain.pose :as pose] - [arthur.domain.timeline :as timeline])) - -(def ^:const root-id - "The reserved id of the timeline a clip plays. See the namespace docstring." - :main) + [arthur.domain.symbol :as symbol])) (def clip-keys "Every top-level field of a clip, and the reason `arthur.domain.leaf` refuses @@ -50,38 +46,56 @@ that loses something on every round trip, which is the one bug a persistence layer must not be able to have. Add the field here and to `leaf/leaves` and `leaf/clip` in the same commit." - #{:name :fps :analysis :subjects :features :groups :width :height :timelines}) + #{:name :fps :analysis :subjects :features :groups :width :height :symbols}) -(defn timeline - "One of the clip's timelines, by id." - [clip id] - (get-in clip [:timelines id])) - -(defn root - "The timeline the clip plays." - [clip] - (timeline clip root-id)) +(defn symbol + "One of the clip's symbols, by id." + [clip sid] + (get-in clip [:symbols sid])) (defn frames - "The clip's length, which is its root timeline's frame space and is not written - down twice. Reading it off the root is what stops the two from disagreeing." + "A symbol's length. Read off the symbol, never copied beside it." + [clip sid] + (:frames (symbol clip sid))) + +(defn update-symbol + "Apply f to one symbol in place." + [clip sid f & args] + (apply update-in clip [:symbols sid] f args)) + +(defn places + "The ids of the symbols `sid` places, directly." + [clip sid] + (into #{} (keep (fn [n] (when (= :instance (:kind n)) (:of n)))) + (vals (:nodes (symbol clip sid))))) + +(defn contains-symbol? + "Whether `inner` is `outer` or is placed anywhere inside it. Placing `outer` + into `inner` when this is true is a cycle." + [clip outer inner] + (let [seen (volatile! #{})] + (letfn [(walk [sid] + (or (= sid inner) + (when-not (@seen sid) + (vswap! seen conj sid) + (some walk (places clip sid)))))] + (boolean (walk outer))))) + +(defn unplaced + "The symbols no other symbol places, sorted by id. What to open when a + document is opened." [clip] - (:frames (root clip))) + (let [placed (into #{} (mapcat #(places clip %)) (keys (:symbols clip)))] + (vec (sort-by str (remove placed (keys (:symbols clip))))))) -(defn update-timeline - "Apply f to one timeline in place." - [clip id f & args] - (apply update-in clip [:timelines id] f args)) - -(defn update-root [clip f & args] - (apply update-timeline clip root-id f args)) - -(defn nodes - "The root timeline's nodes. A convenience for the many callers that mean the - root and would otherwise spell it out; anything that could mean a symbol says - which timeline instead." +(defn opens-on + "The symbol a document opens on: the longest one nothing else places, ties + broken by id. The symbol that contains everything else is the longest of the + unplaced ones in every document made so far, and a reserved name is what this + replaces." [clip] - (:nodes (root clip))) + (first (sort-by (fn [sid] [(- (or (frames clip sid) 0)) (str sid)]) + (unplaced clip)))) (def ^:const blank-frames "How long a new document is before anything says otherwise. Four seconds at 30, @@ -89,9 +103,9 @@ 120) (defn blank - "A new, empty document. + "A new, empty document: one empty symbol. - `:nodes` is empty rather than seeded with a layer, because an empty timeline is + `:nodes` is empty rather than seeded with a layer, because an empty symbol is a true statement and a layer nobody asked for is one more thing to delete. The tracking maps are present and empty for the same reason `clip-keys` exists: a field that is sometimes absent is a field every reader needs a fallback for." @@ -100,39 +114,39 @@ :fps 30 :width 320 :height 200 :subjects {} :features {} :groups {} - :timelines {root-id {:id root-id :frames blank-frames :nodes {}}}}) + :symbols {:main {:id :main :frames blank-frames :nodes {}}}}) (defn place-symbol - "An instance of library timeline `tid`, on the root timeline, at `frame`. + "An instance of symbol `sid`, inside symbol `into`, at `frame` of `into`. THE UUID IS AN ARGUMENT. A placement's identity is the key it has in the node map — it is what `:linked-to`, an export target and a saved leaf all name — so generating one in here would make this function's result depend on when it was - called, and this namespace is the pure one. `demo/stage_8625.edn` authors its - placements' uuids by hand for the same reason, in more words. + called, and this namespace is the pure one. The instance's own time starts where it was dropped: `:at frame` with `:in 0` - means local frame 0 of the symbol plays on `frame` of the stage, which is what - dragging something onto a playhead is asking for. `:span` runs to the end of - the root's frame space rather than to the symbol's length, because a symbol - shorter than the space it is placed in should hold its last frame rather than - disappear." - [clip tid frame uuid [x y]] - (let [target (timeline clip tid) - end (frames clip)] - (if (or (nil? target) (= root-id tid) (nil? frame) (neg? frame) (>= frame end)) + means local frame 0 of the symbol plays on `frame` of `into`, which is what + dragging something onto a playhead is asking for. + + Refused, returning the clip unchanged, when it would make a cycle: a symbol + cannot be placed inside itself or inside anything it places." + [clip into sid frame uuid [x y]] + (let [target (symbol clip sid) + end (frames clip into)] + (if (or (nil? target) (nil? end) (nil? frame) (neg? frame) (>= frame end) + (contains-symbol? clip sid into)) clip - (update-root - clip assoc-in [:nodes uuid] + (update-symbol + clip into assoc-in [:nodes uuid] {:id uuid - :name (name tid) - :kind :symbol - :of tid + :name (name sid) + :kind :instance + :of sid :parent nil ;; Lexicographic draw order, as `domain/paint` does it: a placement made ;; later sits above one made earlier, and neither has to renumber. - :z (str "z" (js/Date.now) "-" (name tid)) - :span [frame end] + :z (str "z" (js/Date.now) "-" (name sid)) + :span [frame (min end (+ frame (:frames target)))] :time {:mode :map :at frame :in 0 :rate 1} :channels {[:xform :pos] {:animated? false :value [x y]}}})))) @@ -158,35 +172,30 @@ op))) (defn resolver - "Resolve a clip, including each library timeline placed by a symbol instance. + "Resolve symbol `sid` of a clip, including every symbol its instances place. - Each instance owns its own timeline resolver, so two offsets never share a + Each instance owns its own symbol resolver, so two offsets never share a channel cursor or point buffer. The returned ops must be drawn before the next - frame, as with timeline/resolver. + frame, as with symbol/resolver. - `root` is which timeline to resolve AS the root, and it defaults to the clip's. - Passing a symbol's id is the whole of \"render that symbol\": a library timeline - and the clip's own are the same type, so a symbol resolves by being rooted - rather than by a second code path — which is the return on collapsing the two - into `domain/timeline`. Its frame space is its own `:frames`, and nested symbols - inside it still resolve, because this is the function that knows how to do that." - ([clip store] (resolver clip store pal/index-of root-id)) - ([clip store palette] (resolver clip store palette root-id)) - ([clip store palette root] (resolver clip store palette root nil)) - ([clip store palette root {:keys [picture-fps] :as opts}] - (letfn [(build [tid chain pose-tracks] - (when (some #{tid} chain) - (throw (ex-info "symbol timeline cycle" {:chain (conj chain tid)}))) - (let [tl (or (timeline clip tid) - (throw (ex-info "symbol names a missing timeline" {:timeline tid}))) - nodes (:nodes tl) - rank (timeline/draw-rank nodes (timeline/order nodes)) + Any symbol can be resolved and none is the default: the frame space is the + resolved symbol's own `:frames`, and nested instances inside it still resolve, + because this is the function that knows how to do that." + ([clip store palette sid] (resolver clip store palette sid nil)) + ([clip store palette sid {:keys [picture-fps] :as opts}] + (letfn [(build [sid chain pose-tracks] + (when (some #{sid} chain) + (throw (ex-info "symbol cycle" {:chain (conj chain sid)}))) + (let [sym (or (symbol clip sid) + (throw (ex-info "an instance names a missing symbol" {:symbol sid}))) + nodes (:nodes sym) + rank (symbol/draw-rank nodes (symbol/order nodes)) ids (sort-by rank (keys nodes)) - own (timeline/resolver tl store palette pose-tracks + own (symbol/resolver sym store palette pose-tracks (assoc opts :source-fps (:fps clip))) children (into {} - (for [[id n] nodes :when (= :symbol (:kind n))] - [id (build (:of n) (conj chain tid) + (for [[id n] nodes :when (= :instance (:kind n))] + [id (build (:of n) (conj chain sid) (get-in n [:playback :tracks]))]))] (fn [f] (let [by-id (into {} (map (juxt :node identity)) (own f))] @@ -194,10 +203,10 @@ (mapcat (fn [id] (let [n (get nodes id)] - (if (= :symbol (:kind n)) - (let [m (timeline/world-of own id) - local (timeline/frame-of own id) - target (timeline clip (:of n)) + (if (= :instance (:kind n)) + (let [m (symbol/world-of own id) + local (symbol/frame-of own id) + target (symbol clip (:of n)) length (:frames target) frame (when (and m (number? local)) (if (get-in n [:time :loop?]) @@ -208,7 +217,7 @@ [])) (when-let [op (get by-id id)] [op])))) ids))))))] - (build root [] nil)))) + (build sid [] nil)))) (defn problems "Human-readable reasons this clip will not evaluate or save." @@ -217,28 +226,26 @@ (concat (for [k (remove clip-keys (keys clip))] (str "clip has a field with no leaf to save it in: " (pr-str k))) - (when-not (map? (:timelines clip)) - [":timelines must be a map of id -> timeline"]) - (when (and (map? (:timelines clip)) (nil? (root clip))) - [(str "no " (pr-str root-id) " timeline — a clip plays the one with the reserved id")]) + (when-not (map? (:symbols clip)) + [":symbols must be a map of id -> symbol"]) (when-not (or (nil? (:fps clip)) (and (number? (:fps clip)) (pos? (:fps clip)))) [(str ":fps is " (pr-str (:fps clip)) " — a rate is a positive number")]) - (for [[id tl] (:timelines clip) - :when (not= id (:id tl))] - (str "timeline under key " (pr-str id) " has :id " (pr-str (:id tl)))) - (for [[id tl] (:timelines clip) - p (timeline/problems tl)] - (str "timeline " (pr-str id) ": " p)) - (for [[tid tl] (:timelines clip) - [id n] (:nodes tl) - :when (and (= :symbol (:kind n)) - (not (contains? (:timelines clip) (:of n))))] - (str "timeline " (pr-str tid) " symbol " (pr-str id) - " names missing timeline " (pr-str (:of n)))) - (for [[tid tl] (:timelines clip) - [id n] (:nodes tl) - :when (= :symbol (:kind n)) - :let [target (get-in clip [:timelines (:of n)]) + (for [[id sym] (:symbols clip) + :when (not= id (:id sym))] + (str "symbol under key " (pr-str id) " has :id " (pr-str (:id sym)))) + (for [[id sym] (:symbols clip) + p (symbol/problems sym)] + (str "symbol " (pr-str id) ": " p)) + (for [[sid sym] (:symbols clip) + [id n] (:nodes sym) + :when (and (= :instance (:kind n)) + (not (contains? (:symbols clip) (:of n))))] + (str "symbol " (pr-str sid) " instance " (pr-str id) + " names missing symbol " (pr-str (:of n)))) + (for [[sid sym] (:symbols clip) + [id n] (:nodes sym) + :when (= :instance (:kind n)) + :let [target (get-in clip [:symbols (:of n)]) active (filter (fn [node] (some :pose-sampled? (vals (:channels node)))) (vals (:nodes target))) @@ -247,11 +254,11 @@ (map #(vector :node (:id %)) active)))] p (pose/problems (get-in n [:playback :tracks]) (:frames target) groups)] - (str "timeline " (pr-str tid) " symbol " (pr-str id) ": " p)) - (for [[tid tl] (:timelines clip) - [id n] (:nodes tl) + (str "symbol " (pr-str sid) " instance " (pr-str id) ": " p)) + (for [[sid sym] (:symbols clip) + [id n] (:nodes sym) :when (and (= :audio (:kind n)) (:linked-to n) - (not (contains? (:nodes tl) (:linked-to n))))] - (str "timeline " (pr-str tid) " audio " (pr-str id) + (not (contains? (:nodes sym) (:linked-to n))))] + (str "symbol " (pr-str sid) " audio " (pr-str id) " links to missing node " (pr-str (:linked-to n)))) (feature/problems clip)))) diff --git a/frontend/src/arthur/domain/feature.cljs b/frontend/src/arthur/domain/feature.cljs index 69839e2..13a3d4b 100644 --- a/frontend/src/arthur/domain/feature.cljs +++ b/frontend/src/arthur/domain/feature.cljs @@ -1,6 +1,6 @@ (ns arthur.domain.feature "Tracked subjects, feature ownership, and eye-pair settings. - Features name their timeline explicitly; node ids are local to that timeline." + Features name their symbol explicitly; node ids are local to that symbol." (:require [arthur.domain.params :as params])) (defn owned @@ -44,14 +44,14 @@ clip)) (defn problems - "Check tracked identities and timeline-local node ownership." + "Check tracked identities and symbol-local node ownership." [clip] (let [subjects (:subjects clip) features (:features clip) groups (:groups clip) memberships (mapcat (comp :members val) groups) node-owners (for [[_ f] features n (:nodes f)] - [(:timeline f) n])] + [(:symbol f) n])] (vec (concat (for [[id s] subjects :when (not= id (:id s))] @@ -60,8 +60,8 @@ :when (not (params/valid-settings? :subject (or (:params s) {})))] (str "subject " (pr-str id) " has invalid settings")) (for [[id _] subjects - :when (not (seq (get-in clip [:timelines id :nodes :head :measured])))] - (str "subject " (pr-str id) " has no measured head in its timeline")) + :when (not (seq (get-in clip [:symbols id :nodes :head :measured])))] + (str "subject " (pr-str id) " has no measured head in its symbol")) (for [[id f] features :when (not= id (:id f))] (str "feature " (pr-str id) " has a different :id")) (for [[id f] features :when (not (contains? subjects (:subject f)))] @@ -72,10 +72,10 @@ :when (not (params/valid-settings? (:area f) (or (:params f) {})))] (str "feature " (pr-str id) " has invalid settings for " (pr-str (:area f)))) (for [[id f] features - :when (not (contains? (:timelines clip) (:timeline f)))] - (str "feature " (pr-str id) " names a missing timeline")) + :when (not (contains? (:symbols clip) (:symbol f)))] + (str "feature " (pr-str id) " names a missing symbol")) (for [[id f] features node-id (:nodes f) - :let [owned-nodes (get-in clip [:timelines (:timeline f) :nodes])] + :let [owned-nodes (get-in clip [:symbols (:symbol f) :nodes])] :when (not (contains? owned-nodes node-id))] (str "feature " (pr-str id) " refers to missing node " (pr-str node-id))) (for [[id n] (frequencies node-owners) :when (> n 1)] diff --git a/frontend/src/arthur/domain/leaf.cljs b/frontend/src/arthur/domain/leaf.cljs index d5fbc6d..f318ef0 100644 --- a/frontend/src/arthur/domain/leaf.cljs +++ b/frontend/src/arthur/domain/leaf.cljs @@ -11,21 +11,21 @@ clip//timing fps clip//stage width, height clip//source the analysis record this came out of - clip//subject/ a tracked subject and its params + clip//subject/ a tracked subject and its params clip//feature/ one feature: area, nodes, params clip//group/ an eye pair and its shared params - clip//timeline/ frames, and a palette one day - clip//timeline//node/ kind, parent, stencil, z, time - clip//timeline//channel// - clip//timeline//measured/ the channels a re-freeze owns + clip//symbol/ frames, and a palette one day + clip//symbol//node/ kind, parent, stencil, z, time + clip//symbol//channel// + clip//symbol//measured/ the channels a re-freeze owns - WHY NODES SIT UNDER A TIMELINE. A clip holds a library of timelines. Its root - and each symbol have their own nodes, so the timeline id is a path segment. - The root is `main`, and a symbol's nodes use the same path shape. + WHY NODES SIT UNDER A SYMBOL. A clip holds a library of symbols and each has + its own nodes, so the symbol id is a path segment. No symbol has a reserved + segment: `main` in a path is an id like any other. - `:frames` MOVED OFF `timing` onto the timeline. A timeline is a frame space and a + `:frames` MOVED OFF `timing` onto the symbol. A symbol is a frame space and a clip is a rate, so `timing` holds `:fps` alone. Both used to be in one leaf, which - is how a nested timeline's length would have had nowhere to go. + is how a nested symbol's length would have had nowhere to go. WHY THESE BOUNDARIES. Last-writer-wins only clobbers when its unit is too big, so the cut is chosen so that the things people do simultaneously land on @@ -56,7 +56,7 @@ it is one character rather than a scheme." (:require [arthur.domain.clip :as clip] [arthur.domain.sha256 :as sha] - [arthur.domain.timeline :as timeline] + [arthur.domain.symbol :as symbol] [clojure.string :as str])) ;; --------------------------------------------------------------------------- @@ -124,11 +124,11 @@ (when (seq unknown) (throw (ex-info "the clip has a field with no leaf to save it in; see arthur.domain.clip/clip-keys" {:unknown (vec (sort-by str unknown))})))) - (doseq [[id tl] (:timelines clip)] - (let [unknown (remove timeline/timeline-keys (keys tl))] + (doseq [[id sym] (:symbols clip)] + (let [unknown (remove symbol/symbol-keys (keys sym))] (when (seq unknown) - (throw (ex-info "a timeline has a field with no leaf to save it in; see arthur.domain.timeline/timeline-keys" - {:timeline id :unknown (vec (sort-by str unknown))}))))) + (throw (ex-info "a symbol has a field with no leaf to save it in; see arthur.domain.symbol/symbol-keys" + {:symbol id :unknown (vec (sort-by str unknown))}))))) (let [at (fn [& parts] (str/join "/" (into ["clip" (segment cid)] parts))) some-leaf (fn [path v] (when (seq v) {path v}))] (apply merge @@ -140,30 +140,30 @@ (for [[id v] (:subjects clip)] {(at "subject" (segment id)) v}) (for [[id v] (:features clip)] {(at "feature" (segment id)) v}) (for [[id v] (:groups clip)] {(at "group" (segment id)) v}) - ;; The timeline's own facts. `:id` is the path segment, so writing it + ;; The symbol's own facts. `:id` is the path segment, so writing it ;; into the value as well would be the one field a rename could ;; disagree with itself about; `clip` puts it back. - (for [[tid tl] (:timelines clip)] - {(at "timeline" (segment tid)) - (select-keys tl [:frames :palette])}) - (for [[tid tl] (:timelines clip) - [id n] (:nodes tl)] - {(at "timeline" (segment tid) "node" (segment id)) + (for [[sid sym] (:symbols clip)] + {(at "symbol" (segment sid)) + (select-keys sym [:frames :palette])}) + (for [[sid sym] (:symbols clip) + [id n] (:nodes sym)] + {(at "symbol" (segment sid) "node" (segment id)) (apply dissoc n node-channel-keys)}) - (for [[tid tl] (:timelines clip) - [id n] (:nodes tl) + (for [[sid sym] (:symbols clip) + [id n] (:nodes sym) :when (seq (:measured n))] - {(at "timeline" (segment tid) "measured" (segment id)) (:measured n)}) - (for [[tid tl] (:timelines clip) - [id n] (:nodes tl) + {(at "symbol" (segment sid) "measured" (segment id)) (:measured n)}) + (for [[sid sym] (:symbols clip) + [id n] (:nodes sym) [prop ch] (:channels n)] - {(at "timeline" (segment tid) "channel" (segment id) (prop->path prop)) ch}))))) + {(at "symbol" (segment sid) "channel" (segment id) (prop->path prop)) ch}))))) (defn clip "The inverse of `leaves`, for one clip. Paths belonging to another clip are ignored, so a project's whole leaf map can be handed straight in. - A timeline's `:id` is restored from its path segment rather than read out of the + A symbol's `:id` is restored from its path segment rather than read out of the value, which is why `leaves` does not write it: a segment and a field that both claim to be the id are two places for one fact." [cid leaves] @@ -173,14 +173,14 @@ (let [[_ found kind a b c] (str/split path #"/")] (if-not (= want found) acc - (if (= "timeline" kind) - (let [tid (unsegment a) - acc (assoc-in acc [:timelines tid :id] tid)] + (if (= "symbol" kind) + (let [sid (unsegment a) + acc (assoc-in acc [:symbols sid :id] sid)] (case b - nil (update-in acc [:timelines tid] merge v) - "node" (update-in acc [:timelines tid :nodes (unsegment c)] merge v) - "measured" (assoc-in acc [:timelines tid :nodes (unsegment c) :measured] v) - "channel" (assoc-in acc [:timelines tid :nodes (unsegment c) + nil (update-in acc [:symbols sid] merge v) + "node" (update-in acc [:symbols sid :nodes (unsegment c)] merge v) + "measured" (assoc-in acc [:symbols sid :nodes (unsegment c) :measured] v) + "channel" (assoc-in acc [:symbols sid :nodes (unsegment c) :channels (path->prop (nth (str/split path #"/") 6))] v) (throw (ex-info "not a leaf path" {:path path})))) @@ -210,20 +210,20 @@ content-addressed is that it does not have to travel with tier 1 to be found." [leaves] (let [parts (into {} (map (juxt identity #(vec (str/split % #"/")))) (keys leaves)) - ;; A node leaf, by (clip, timeline, node). Under a timeline id, because a - ;; symbol and the root may both hold a `:mouth` and a channel of one is not - ;; a channel of the other. + ;; A node leaf, by (clip, symbol, node). Under a symbol id, because two + ;; symbols may both hold a `:mouth` and a channel of one is not a channel + ;; of the other. nodes (into #{} (keep (fn [[_ p]] - (when (and (= 6 (count p)) (= "timeline" (nth p 2)) + (when (and (= 6 (count p)) (= "symbol" (nth p 2)) (= "node" (nth p 4))) [(nth p 1) (nth p 3) (nth p 5)]))) parts) ;; Which segment index holds the kind, and what shapes are legal. legal? (fn [p] (and (= "clip" (first p)) (second p) - (if (= "timeline" (nth p 2 nil)) + (if (= "symbol" (nth p 2 nil)) (case (count p) - 4 true ; the timeline itself + 4 true ; the symbol itself 6 (#{"node" "measured"} (nth p 4)) 7 (= "channel" (nth p 4)) false) @@ -238,13 +238,13 @@ :when (not (legal? p))] (str (pr-str path) " is not a leaf path")) (for [[path p] (sort-by key parts) - :when (and (legal? p) (= "timeline" (nth p 2 nil)) (>= (count p) 6) + :when (and (legal? p) (= "symbol" (nth p 2 nil)) (>= (count p) 6) (#{"channel" "measured"} (nth p 4)) (not (contains? nodes [(nth p 1) (nth p 3) (nth p 5)])))] (str (pr-str path) " addresses a node with no node leaf")) (for [[path p] (sort-by key parts) :let [v (get leaves path)] - :when (and (legal? p) (= "timeline" (nth p 2 nil)) (= 7 (count p)) + :when (and (legal? p) (= "symbol" (nth p 2 nil)) (= 7 (count p)) (:dense v) (not (sha/key? (:store (:dense v)))))] (str (pr-str path) " names tier 2 as " (pr-str (:store (:dense v))) " — a dense channel in a saved document names a content address")))))) diff --git a/frontend/src/arthur/domain/node.cljs b/frontend/src/arthur/domain/node.cljs index bc2d432..a4ffb07 100644 --- a/frontend/src/arthur/domain/node.cljs +++ b/frontend/src/arthur/domain/node.cljs @@ -21,9 +21,9 @@ "`:bitmap` is in the vocabulary and not implemented; it is here so that a scene that names one fails as \"not implemented\" rather than as \"not a kind\"." - #{:poly :disc :rect :group :bitmap :symbol :audio}) + #{:poly :disc :rect :group :bitmap :instance :audio}) -(def implemented-kinds #{:poly :disc :rect :group :symbol :audio}) +(def implemented-kinds #{:poly :disc :rect :group :instance :audio}) (def xform-paths "In composition order, which is also the order they have to be sampled in. @@ -45,7 +45,7 @@ change to this spec silently change what gets drawn." (let [base (into #{[:vis]} xform-paths)] {:group base - :symbol base + :instance base :audio (into base [[:audio :gain] [:audio :pan] [:audio :rate]]) :poly (into base [[:geom :pts] [:style :color]]) ;; A disc's radius is framed in practice — iris size is a knob, not a @@ -111,7 +111,7 @@ it on most frames, so the lead slider reads as doing nothing at exposures above 1, which is indistinguishable from the slider being unwired. - Composed along the parent chain, outermost first, by timeline/eval-frame. Two + Composed along the parent chain, outermost first, by symbol/eval-frame. Two rules fall out and they are different rules: exposure INHERITS STRICTLY, because a head cutting on odd frames against a mouth cutting on even ones reads as two performances; offset is PER-NODE by design, because mouth lead applies @@ -122,13 +122,13 @@ (if (= mode :inherit) f (do - (when (and (not (#{:symbol :audio} (:kind n))) rate (not= rate 1.0) (not= rate 1)) - (throw (ex-info "time map :rate belongs to a symbol or audio instance" + (when (and (not (#{:instance :audio} (:kind n))) rate (not= rate 1.0) (not= rate 1)) + (throw (ex-info "time map :rate belongs to an instance or an audio node" {:node (:id n) :time (:time n)}))) (when (and sample-fps (not (and source-fps (pos? source-fps)))) (throw (ex-info "picture sampling needs a positive source fps" {:node (:id n) :time (:time n)}))) - (cond-> (if (#{:symbol :audio} (:kind n)) + (cond-> (if (#{:instance :audio} (:kind n)) (+ (or in 0) (* (or rate 1) (- f (or at 0)))) f) sample-fps (sample-frame source-fps sample-fps) @@ -265,10 +265,10 @@ (not (contains? implemented-kinds k))) (conj (str ":kind " k " is in the vocabulary but not implemented")) - (and (= k :symbol) (nil? (:of n))) (conj "a symbol instance needs :of") + (and (= k :instance) (nil? (:of n))) (conj "an instance needs :of") (and (= k :audio) (nil? (get-in n [:source :footage]))) (conj "an audio instance needs :source :footage") - (and (#{:symbol :audio} k) (some? (get-in n [:time :rate])) + (and (#{:instance :audio} k) (some? (get-in n [:time :rate])) (not (pos? (get-in n [:time :rate])))) (conj "an instance's :rate must be positive") (nil? (:z n)) (conj "no :z — draw order is authored per scene, not implied by the tree") diff --git a/frontend/src/arthur/domain/paint.cljs b/frontend/src/arthur/domain/paint.cljs index 6e943a2..53f1fe4 100644 --- a/frontend/src/arthur/domain/paint.cljs +++ b/frontend/src/arthur/domain/paint.cljs @@ -1,12 +1,13 @@ (ns arthur.domain.paint - "Small authored polygon operations. Paint nodes read timeline frames directly; - the roto root's exposure and picture sampling must not quantise a hand edit." + "Small authored polygon operations, each on a named symbol. Paint nodes read + their symbol's frames directly; a roto instance's exposure and picture sampling + must not quantise a hand edit." (:require [arthur.domain.channel :as channel])) (def geometry [:geom :pts]) -(defn shapes [clip] - (->> (get-in clip [:timelines :main :nodes]) +(defn shapes [clip sid] + (->> (get-in clip [:symbols sid :nodes]) (filter (fn [[_ node]] (:paint? node))) (sort-by (comp :z val)) vec)) @@ -15,21 +16,21 @@ (let [frames (sort (keys (:keys ch)))] (or (last (take-while #(<= % frame) frames)) (first frames)))) -(defn new-shape [clip id frame points color] - (let [end (get-in clip [:timelines :main :frames]) +(defn new-shape [clip sid id frame points color] + (let [end (get-in clip [:symbols sid :frames]) z (str "z" (js/Date.now) "-" (name id))] (if (and (<= 0 frame) (< frame end) (>= (count points) 6) (even? (count points))) - (assoc-in clip [:timelines :main :nodes id] - {:id id :name (str "shape " (inc (count (shapes clip)))) + (assoc-in clip [:symbols sid :nodes id] + {:id id :name (str "shape " (inc (count (shapes clip sid)))) :kind :poly :paint? true :parent nil :z z :span [frame end] :channels {geometry (channel/keyed {frame points}) [:style :color] (channel/framed color)}}) clip))) -(defn add-key [clip id frame] - (let [path [:timelines :main :nodes id] +(defn add-key [clip sid id frame] + (let [path [:symbols sid :nodes id] node (get-in clip path) ch (get-in node [:channels geometry]) [start end] (:span node)] @@ -38,20 +39,20 @@ (vec (channel/value-at ch frame))) clip))) -(defn set-vertex [clip id key-frame vertex [x y]] - (let [path [:timelines :main :nodes id :channels geometry :keys key-frame] +(defn set-vertex [clip sid id key-frame vertex [x y]] + (let [path [:symbols sid :nodes id :channels geometry :keys key-frame] points (get-in clip path) i (* 2 vertex)] (if (and points (< (inc i) (count points))) (assoc-in clip path (-> points (assoc i x) (assoc (inc i) y))) clip))) -(defn set-segment-interp [clip id key-frame interp] - (let [node (get-in clip [:timelines :main :nodes id]) +(defn set-segment-interp [clip sid id key-frame interp] + (let [node (get-in clip [:symbols sid :nodes id]) keys (get-in node [:channels geometry :keys])] (if (and (:paint? node) (contains? keys key-frame) (some #(< key-frame %) (clojure.core/keys keys)) (#{:hold :linear} interp)) - (assoc-in clip [:timelines :main :nodes id :channels geometry + (assoc-in clip [:symbols sid :nodes id :channels geometry :segments key-frame] interp) clip))) diff --git a/frontend/src/arthur/domain/png.cljs b/frontend/src/arthur/domain/png.cljs index 7958255..97b8c4c 100644 --- a/frontend/src/arthur/domain/png.cljs +++ b/frontend/src/arthur/domain/png.cljs @@ -88,7 +88,7 @@ (defn encoder "(fn [raster ramp] -> promise of PNG bytes), for one stage size and one zoom. - Built once per export rather than per frame, in the shape `timeline/resolver` + Built once per export rather than per frame, in the shape `symbol/resolver` already uses: everything that does not change frame to frame is held here. What that buys is the scanline scratch, which at zoom 6 is seven megabytes — a per-frame allocation of that size is the one thing that would make a long export diff --git a/frontend/src/arthur/domain/pose.cljs b/frontend/src/arthur/domain/pose.cljs index c309a22..d305fd6 100644 --- a/frontend/src/arthur/domain/pose.cljs +++ b/frontend/src/arthur/domain/pose.cljs @@ -32,16 +32,17 @@ default-frame)) (defn put-cut - "Set one held pose on a symbol instance. Earlier motion stays untouched." - [clip instance group at source] - (let [node (get-in clip [:timelines :main :nodes instance]) - symbol (get-in clip [:timelines (:of node)]) - length (:frames symbol) + "Set one held pose on an instance inside symbol `sid`. Earlier motion stays + untouched." + [clip sid instance group at source] + (let [node (get-in clip [:symbols sid :nodes instance]) + placed (get-in clip [:symbols (:of node)]) + length (:frames placed) active (filter (fn [n] (some :pose-sampled? (vals (:channels n)))) - (vals (:nodes symbol))) + (vals (:nodes placed))) groups (set (map #(or (:pose-group %) (:id %)) active)) ids (set (map :id active))] - (when-not (and (= :symbol (:kind node)) + (when-not (and (= :instance (:kind node)) (or (contains? groups group) (and (vector? group) (= 2 (count group)) (= :node (first group)) @@ -50,17 +51,17 @@ (integer? source) (<= 0 source) (< source length)) (throw (ex-info "invalid stage pose cut" {:instance instance :group group :at at :source source}))) - (update-in clip [:timelines :main :nodes instance :playback :tracks group] + (update-in clip [:symbols sid :nodes instance :playback :tracks group] #(assoc (or % {}) at source)))) (defn remove-cut "Remove a cut; an empty track again follows the normal generated motion." - [clip instance group at] - (let [path [:timelines :main :nodes instance :playback :tracks group]] + [clip sid instance group at] + (let [path [:symbols sid :nodes instance :playback :tracks group]] (if-let [entries (get-in clip path)] (if-let [remaining (not-empty (dissoc entries at))] (assoc-in clip path remaining) - (update-in clip [:timelines :main :nodes instance :playback :tracks] + (update-in clip [:symbols sid :nodes instance :playback :tracks] dissoc group)) clip))) diff --git a/frontend/src/arthur/domain/project.cljs b/frontend/src/arthur/domain/project.cljs index 41163fe..3f68019 100644 --- a/frontend/src/arthur/domain/project.cljs +++ b/frontend/src/arthur/domain/project.cljs @@ -29,6 +29,12 @@ (:require [arthur.domain.leaf :as leaf] [arthur.domain.wire :as wire])) +(def schema-version + "The stored document format this client reads and writes. 2 is symbols: leaf + paths say `symbol`, a placing node is `:kind :instance`, and no symbol id is + reserved. `clips/migrations/0007` moved every saved project from 1." + 2) + (defn block-keys "Every tier-2 key a leaf map names, in a stable order." [leaves] @@ -53,8 +59,8 @@ round-trip a clip through `JSON.parse(JSON.stringify(...))` and be running the same conversion the network runs, rather than a CLJS-shaped rehearsal of it. The one thing a keywordising `js->clj` would quietly break is the leaf paths — - `:clip/c1/timeline/main/node/mouth` is a keyword whose `name` is - \"c1/timeline/main/node/mouth\", so the + `:clip/c1/symbol/main/node/mouth` is a keyword whose `name` is + \"c1/symbol/main/node/mouth\", so the \"clip/\" would be lost on the way back in. Refuses a document `domain/leaf` calls unaddressable, which is where a hand-made diff --git a/frontend/src/arthur/domain/raster.cljs b/frontend/src/arthur/domain/raster.cljs index 22023ed..e279cd0 100644 --- a/frontend/src/arthur/domain/raster.cljs +++ b/frontend/src/arthur/domain/raster.cljs @@ -30,7 +30,7 @@ edge landing exactly on a pixel boundary resolves consistently. Flat and preallocated because this is the per-frame path: fixed topology means - a node's vertex count is known at freeze time, so timeline/resolver hands the same + a node's vertex count is known at freeze time, so symbol/resolver hands the same buffer back every frame and a frame allocates nothing. At 30fps per-frame allocation is the only thing that will make this stutter. diff --git a/frontend/src/arthur/domain/timeline.cljs b/frontend/src/arthur/domain/symbol.cljs similarity index 84% rename from frontend/src/arthur/domain/timeline.cljs rename to frontend/src/arthur/domain/symbol.cljs index 09017ae..5c7335c 100644 --- a/frontend/src/arthur/domain/timeline.cljs +++ b/frontend/src/arthur/domain/symbol.cljs @@ -1,38 +1,36 @@ -(ns arthur.domain.timeline - "A TIMELINE: an ordered bag of nodes in its own frame space, and the two ways to +(ns arthur.domain.symbol + "A SYMBOL: an ordered bag of nodes in its own frame space, and the two ways to evaluate it at a frame. {:id :main :frames 229 :nodes {id -> node} :palette nil} - That is the whole type, and EVERYTHING THAT HOLDS NODES IS ONE OF THESE. A - clip's root timeline is one; a symbol in the library is one; a `:kind :symbol` - node is an INSTANCE of one. An earlier arrangement had the clip's node tree and - a library symbol as two structures with the same fields and never said they were - the same thing — the clip map carried `:fps`, `:width`, `:height`, `:analysis` - and the tracking identities alongside `:nodes`, so a symbol had nowhere to live - that was not a clip with seven meaningless fields. Flash's `_root` is a - MovieClip and After Effects' pre-comp is just a layer; collapsing them is what - makes nesting arbitrary and free rather than a feature to be added. + That is the whole type, and EVERYTHING THAT HOLDS NODES IS ONE OF THESE. What + a document opens on is a symbol; what a `:kind :instance` node places is a + symbol; there is no second structure. An earlier arrangement had a root node + tree and a library entry as two structures with the same fields and never said + they were the same thing. Flash's `_root` is a MovieClip and After Effects' + pre-comp is just a layer; collapsing them is what makes nesting arbitrary and + free rather than a feature to be added. - The clip-level facts are in `arthur.domain.clip`. A timeline has a FRAME SPACE, + The clip-level facts are in `arthur.domain.clip`. A symbol has a FRAME SPACE, not a rate and not a size: `:fps` is the clip's, because a rate is a fact about - how fast the whole thing plays, and a nested timeline cannot have its own. + how fast the whole thing plays, and a nested symbol cannot have its own. TWO AXES OF NESTING, and conflating them is why \"nested\" and \"flat with parent pointers\" sound contradictory when they are not. Parent/child is transform - composition WITHIN one timeline and is stored flat with pointers. Instance is a - timeline inside another timeline and is stored by reference into the library. - Each timeline is flat; timelines nest. Every argument for flat storage — + composition WITHIN one symbol and is stored flat with pointers. Instance is a + symbol inside another symbol and is stored by reference into the library. + Each symbol is flat; symbols nest. Every argument for flat storage — addressability, one-field reparenting, structural sharing, per-node sync leaves — is about the first axis and is untouched by the second. Two ways to evaluate one at a frame: - (eval-frame tl f store) THE SPECIFICATION. Allocating, order-free, + (eval-frame sym f store) THE SPECIFICATION. Allocating, order-free, obviously correct. Use it in tests and for a one-off render. - (resolver tl store) -> (fn [f] ops). What playback uses. Caches the + (resolver sym store) -> (fn [f] ops). What playback uses. Caches the topological order and the z paths, holds one CURSOR per channel and one PREALLOCATED point buffer per node, so a frame allocates the op @@ -42,7 +40,7 @@ read and where points are written. That is deliberate: two independent implementations of frame evaluation would drift, and the drift would look like a rendering bug rather than like two functions disagreeing. What differs - between them is exactly the part that can be wrong, and timeline-test asserts + between them is exactly the part that can be wrong, and symbol-test asserts they agree frame for frame in forward, backward and random order. The output is a list of DRAW OPS, and it is the boundary with the rasteriser: @@ -76,12 +74,12 @@ (when-let [p (:parent (get nodes i))] (if (contains? nodes p) p - (throw (ex-info "node's :parent is not in the timeline" + (throw (ex-info "node's :parent is not in the symbol" {:node i :parent p}))))) chain (into [] (comp (take-while some?) (take (inc (count nodes)))) (iterate up id))] (when (> (count chain) (count nodes)) - (throw (ex-info "parent cycle in timeline" {:node id :chain chain}))) + (throw (ex-info "parent cycle in symbol" {:node id :chain chain}))) chain)) (defn depth @@ -129,9 +127,9 @@ "id -> its position in draw order. Computed ONCE. Draw order is a function of the z paths, which are structural — - they change when the timeline changes and never because the playhead moved — so + they change when the symbol changes and never because the playhead moved — so sorting ops by z on every frame was re-deriving a constant thirty times a - second. Here it is derived when the timeline is, and a frame sorts small integers. + second. Here it is derived when the symbol is, and a frame sorts small integers. `sort-by` is stable and `ord` is topological, so nodes sharing a z path keep parent-before-child order without a tiebreak field on every op." @@ -146,9 +144,9 @@ "Tone keyword -> the index the raster writes, in a given palette. `palette` is a map of tone -> index. It is a PARAMETER, not a global: a tone - names which mark this is, and which ramp it is read in belongs to the timeline + names which mark this is, and which ramp it is read in belongs to the symbol the node sits in, so resolution cannot reach for one ambient answer. Today - there is one palette and it is passed in anyway; when timelines carry a + there is one palette and it is passed in anyway; when symbols carry a `:palette` channel, the walk carries the palette in scope exactly as it already carries the parent transform and the local frame. @@ -229,7 +227,7 @@ flow/freeze writes it KEYED, because a threshold crossing is a handful of transitions and hold is the default, and because a human has to be able to fix one frame of it. When something does want a dense one it will land here loudly - instead of blanking the timeline. + instead of blanking the symbol. Absence is not a boolean and is not an error: a subject that is not on the frame has nothing to show." @@ -271,13 +269,13 @@ :rd rd}))))))) (defn- emit - "Emit geometry in the timeline's space. Rect sizes stay fractional until + "Emit geometry in the symbol's space. Rect sizes stay fractional until rasterization, so enclosing symbol transforms can still scale them." [{:keys [palette buf-for]} n {:keys [m rd]} base] (let [colour #(colour-index palette (rd [:style :color]))] (case (:kind n) :group nil - :symbol nil + :instance nil :audio nil :poly @@ -311,20 +309,20 @@ {:node (:id n) :kind (:kind n)}))))) (defn- nodes-of - "The timeline's node map, REFUSING a map that has none. + "The symbol's node map, REFUSING a map that has none. - A clip and a timeline both have an `:id` and both are maps, so handing a CLIP to + A clip and a symbol both have an `:id` and both are maps, so handing a CLIP to an evaluator is the one mistake this type split makes easy — and the result is not an error, it is `(:nodes clip)` being nil and a frame resolving to no ops at all. That reads as a black stage, or, in a benchmark, as \"0 nodes\" and a flattering number. It happened once while the split was being made, which is why this is a guard and not a comment." - [tl] - (let [nodes (:nodes tl)] + [sym] + (let [nodes (:nodes sym)] (when-not (map? nodes) - (throw (ex-info (str "not a timeline: :nodes is " (pr-str nodes) - " — a clip is not a timeline, its `:timelines` hold them") - {:keys (vec (sort-by str (keys tl)))}))) + (throw (ex-info (str "not a symbol: :nodes is " (pr-str nodes) + " — a clip is not a symbol, its `:symbols` hold them") + {:keys (vec (sort-by str (keys sym)))}))) nodes)) (defn- channel-frame @@ -386,18 +384,18 @@ ;; the specification (defn eval-frame - "Timeline at frame f -> draw ops in z order. Pure, and allocates freely. + "Symbol at frame f -> draw ops in z order. Pure, and allocates freely. - `f` is in THIS timeline's frame space. At the clip's root that is clip frames; - inside an instance it is the instance's own space, and the instance boundary is + `f` is in THIS symbol's frame space. For the symbol on screen that is the + transport's frame; inside an instance it is the instance's own space, and the instance boundary is the only place the space changes. This is the definition of what a frame means. `resolver` is what plays it." - ([tl f] (eval-frame tl f nil pal/index-of)) - ([tl f store] (eval-frame tl f store pal/index-of)) - ([tl f store palette] (eval-frame tl f store palette nil nil)) - ([tl f store palette pose-tracks opts] - (let [nodes (nodes-of tl) + ([sym f] (eval-frame sym f nil pal/index-of)) + ([sym f store] (eval-frame sym f store pal/index-of)) + ([sym f store palette] (eval-frame sym f store palette nil nil)) + ([sym f store palette pose-tracks opts] + (let [nodes (nodes-of sym) choices (pose/prepare pose-tracks) anchors (prepared-anchors nodes) {:keys [source-fps picture-fps]} opts @@ -456,12 +454,12 @@ The op maps themselves are allocated fresh, and deliberately: there are a dozen of them per frame against hundreds of points, so pooling them would buy nothing and cost the ability to hand an op list around as plain data." - ([tl] (resolver tl nil pal/index-of nil nil)) - ([tl store] (resolver tl store pal/index-of nil nil)) - ([tl store palette] (resolver tl store palette nil nil)) - ([tl store palette pose-tracks] (resolver tl store palette pose-tracks nil)) - ([tl store palette pose-tracks {:keys [source-fps picture-fps]}] - (let [nodes (nodes-of tl) + ([sym] (resolver sym nil pal/index-of nil nil)) + ([sym store] (resolver sym store pal/index-of nil nil)) + ([sym store palette] (resolver sym store palette nil nil)) + ([sym store palette pose-tracks] (resolver sym store palette pose-tracks nil)) + ([sym store palette pose-tracks {:keys [source-fps picture-fps]}] + (let [nodes (nodes-of sym) choices (pose/prepare pose-tracks) anchors (prepared-anchors nodes) ord (order nodes) @@ -507,20 +505,20 @@ ;; --------------------------------------------------------------------------- -(def timeline-keys - "Every field a timeline may carry, and the reason `arthur.domain.leaf` refuses +(def symbol-keys + "Every field a symbol may carry, and the reason `arthur.domain.leaf` refuses one it does not know: a field added without a leaf to save it in is a field that saves silently and comes back missing. - `:palette` is in the vocabulary and nothing writes one yet. A timeline is where - a ramp belongs — `domain/timeline` takes the palette as a PARAMETER rather than - reaching for a global precisely so that a nested timeline can carry its own — + `:palette` is in the vocabulary and nothing writes one yet. A symbol is where + a ramp belongs — `domain/symbol` takes the palette as a PARAMETER rather than + reaching for a global precisely so that a nested symbol can carry its own — and leaving the field out would make the first one a migration instead of a write." #{:id :frames :nodes :palette}) (defn problems - "Human-readable reasons this timeline will not evaluate. Empty means it will. + "Human-readable reasons this symbol will not evaluate. Empty means it will. Node structure only. The tracking identities — subjects, features, groups — are the CLIP's and are checked by `arthur.domain.clip/problems`, which is not a @@ -530,8 +528,8 @@ Total by construction — it reports a cycle rather than looping on one — because its whole job is to be safe to run over authored data before that data is trusted." - [tl] - (let [nodes (:nodes tl)] + [sym] + (let [nodes (:nodes sym)] (if-not (map? nodes) [":nodes must be a map of id -> node"] (-> [] @@ -541,11 +539,11 @@ (into (for [[id n] nodes :when (and (:parent n) (not (contains? nodes (:parent n))))] (str "node " (pr-str id) " has :parent " (pr-str (:parent n)) - " which is not in the timeline"))) + " which is not in the symbol"))) (into (for [[id n] nodes :when (and (:stencil n) (not (contains? nodes (:stencil n))))] (str "node " (pr-str id) " has :stencil " (pr-str (:stencil n)) - " which is not in the timeline"))) + " which is not in the symbol"))) (into (for [[id n] nodes p (node/problems n)] (str "node " (pr-str id) ": " p))) @@ -554,19 +552,19 @@ :let [anchors (:anchors n)] :when (some? anchors) :when (not (and (map? anchors) (contains? anchors 0) - (integer? (:frames tl)) + (integer? (:frames sym)) (every? #(and (integer? %) (<= 0 %) - (< % (:frames tl))) + (< % (:frames sym))) (concat (keys anchors) (vals anchors))) (seq (:measured n)) (= (:channels n) (:measured n))))] (str "node " (pr-str id) ": :anchors must start at frame 0, name valid measured frames, and read that node's own measured channels"))) - (into (for [k (remove timeline-keys (keys tl))] - (str "timeline has a field with no leaf to save it in: " (pr-str k)))) - (into (when-not (or (nil? (:frames tl)) (and (integer? (:frames tl)) (pos? (:frames tl)))) - [(str ":frames is " (pr-str (:frames tl)) - " — a timeline is a frame SPACE, so its length is a positive integer")])) + (into (for [k (remove symbol-keys (keys sym))] + (str "symbol has a field with no leaf to save it in: " (pr-str k)))) + (into (when-not (or (nil? (:frames sym)) (and (integer? (:frames sym)) (pos? (:frames sym)))) + [(str ":frames is " (pr-str (:frames sym)) + " — a symbol is a frame SPACE, so its length is a positive integer")])) (into (try (doall (map #(depth nodes %) (keys nodes))) nil diff --git a/frontend/src/arthur/events/export.cljs b/frontend/src/arthur/events/export.cljs index 1e202c1..999ba5d 100644 --- a/frontend/src/arthur/events/export.cljs +++ b/frontend/src/arthur/events/export.cljs @@ -2,7 +2,7 @@ "Export, as intents and one effect. The walk is not an event and must not become one: it is a promise chain that - runs for as long as the timeline is long, and re-frame events are the wrong unit + runs for as long as the symbol is long, and re-frame events are the wrong unit for something with a middle. So `::start` collects what the render needs out of the db and hands it to an fx, and the fx dispatches progress back — the same arrangement `events/project`'s save uses, and for the same reason. @@ -44,84 +44,88 @@ (defn target-value "An export target as a `