diff --git a/docs/lane-handoff.md b/docs/lane-handoff.md index 21a409a..b164ed0 100644 --- a/docs/lane-handoff.md +++ b/docs/lane-handoff.md @@ -86,15 +86,40 @@ decision, not a cleanup. `{:refused why}`, never a half-applied edit. Where the model needs a choice nobody has made, refusing and saying why is the behaviour, not a placeholder. - **A clip is not a row.** Rows, expansion and selection are editor state. The - document has never known about rows and must not learn. + document has never known about rows and must not learn — which is what let + the row model change three times in one sitting (blocks, then a + selected-clip portal, then sound lanes under the audio heading) without + touching a single document. - **A lane is generic.** Drawing creation, library placement, and adopting an existing root instance all produce the same child instance shape. The only difference is playback policy: a new empty drawing holds source frame zero; a dropped library symbol plays at speed one. +- **Every symbol is born with a lane,** `clip/lane-node`, id `:lane`. A symbol + with none had nowhere to drop a thing, which made the first drop into any + symbol a special case. An unaimed drop fills an EMPTY lane that is already + there and otherwise makes a new one; it never takes an occupied lane nobody + pointed at, because placement claims time and would trim or delete what was + in it. - **Placement claims time.** Lanes never store overlaps. A new or extended clip trims, removes, or splits whatever previously owned the claimed interval. Real compositing overlap uses another lane, where ordering remains explicit. +## The timeline opens the whole document + +Expanding a lane opens exactly one clip — the selected one — and that portal +opens the lanes and nodes of the symbol it places, recursively, mapped into +the open symbol's ruler. The portal follows the LINEAGE of the selection, so +working on something nested keeps the rows that revealed it open. A held clip +opens too, with its rows marked `:unmapped?`: shown across the hold, with no +keys and no draggable edges, because a frozen clock gives its frames no place +on this ruler. `docs/lane-nesting-notes.md` has the reasoning and what is +still missing. + +Double-clicking a clip opens the symbol it places as a tab, the same as +double-clicking that symbol in the pool. Shift while dragging a clip body +turns the temporal move into a structural one — see the nesting notes for why +that is mostly refused today. + ## Current timeline interaction - Creating a symbol inside an aimed lane creates a one-frame held clip at the @@ -169,10 +194,16 @@ The implemented correction slice and its remaining UI limits are recorded in ## Known gaps and traps -- **Audio remains outside visual lanes.** `symbol/lane-problems` deliberately - requires symbol instances. Audio is still an independent root node that can - link to picture; making audio itself lane-based would need an explicit lane - capability rather than a mixed child rule. +- **Audio is a clip in a lane too, and a lane holds one kind.** A sound placed + from the pool lands in a lane and is moved and trimmed by the same commands + as picture. The capability the earlier note asked for is the homogeneity + rule rather than a field: `symbol/lane-problems` refuses a lane holding both + kinds, and `lane/place-symbol` and `lane/adopt` refuse BEFORE claiming time, + because placement claims time and would otherwise have deleted the sound to + make room for the picture and left a valid document behind. Audio nested + inside a placed symbol — a take's own sound — is still shown flattened by + `nest/audio-tracks`; what is in a lane of the open symbol is drawn as a lane + and not flattened twice. - **`:z` is required on cels and means nothing there.** A lane never has two cels on one frame, so draw order between them cannot matter. `node/problems` requires `:z` on every node uniformly, which is its own kind of simplicity — diff --git a/docs/lane-nesting-notes.md b/docs/lane-nesting-notes.md index 7adee9a..73d6ec0 100644 --- a/docs/lane-nesting-notes.md +++ b/docs/lane-nesting-notes.md @@ -105,12 +105,57 @@ derived portal, not another ownership edge and not a duplicated node. ## Implementation order -1. Render instance-level key ticks within lane clips. -2. Add selected-clip portal expansion to `timeline/rows` using the existing - recursive walk and root-time mapping. -3. Add the explicit structural grab affordance and clip/portal drop targets. -4. Route structural drops through `nest/move-node`; add browser coverage for a - root drawing moved into a clip in another lane without a visual jump. +1. ~~Render instance-level key ticks within lane clips.~~ Done: a clip's keys + are on its block, drawn after the blocks so they land on the one they + belong to. +2. ~~Add selected-clip portal expansion to `timeline/rows`.~~ Done, with two + additions the note did not anticipate: + - The portal is chosen by the whole LINEAGE of the selection, not the + selected id. Selecting a shape inside the clip, or the end of its span, + is still working inside that clip, and matching the id alone closed the + portal the moment anything under it was touched. + - A HELD clip opens too. `clip/source-time` is nil for a hold, so the walk + used to stop there and the inside of every drawing was unreachable from + the root timeline. Its rows are now shown across the hold and marked + `:unmapped?`: no keys, and no draggable edges, because no frame inside it + has a place on this ruler. +3. ~~Add the explicit structural affordance.~~ Done as SHIFT on a clip-body + drag rather than a separate grab handle: shift turns a temporal move into a + structural one, the target clip takes an inset highlight, and a label by the + pointer says which of the two is about to happen. +4. Route structural drops through `nest/move-node`. **Wired, and blocked in + the domain.** The gesture asks `nest/move-refusal` on the way past, so the + label says before the drop what the command would say after it. Two + refusals stand in the way of ordinary use: + - *both have to be on screen at this frame.* Inherent, and worth keeping: + the move preserves the world transform and there is no common frame to + preserve it at otherwise. It does mean nesting one clip into another in + the SAME lane can never work — a lane never overlaps itself — so this is + a between-lanes gesture with the playhead somewhere both are showing. + - *a held or looping clip has no clock to move through.* `nest/inside` + returns no `:time` for a hold, and a held one-frame drawing is the most + common thing in a document, so today nesting into one is refused — which + is most of what anybody would try. 5. After the transplant, expand the destination portal and reveal/select the - moved row. + moved row. `::ui/move-node` already selects the moved node and opens the + rows down to it; the portal follows from the lineage rule in 2. + +## The held destination, unresolved + +A held cel shows ONE source frame for its whole span, so there is no +invertible map from the lane's frames to the drawing's and `move-node` +refuses. But the refusal is stronger than the facts require. Inside a frozen +destination only one frame is ever observed, so: + +- the RATE of any map into it is unobservable — every rate shows frame `in`; +- what IS observable is that the moved node should show, at that one frame, + what it shows now at the current root frame. + +That pins a unique sensible answer — rate 1, aligned so the current frame maps +to the shown frame — and nothing else about the mapping can be seen. If that +argument holds, it is a rule rather than a guess, and it is the difference +between structural nesting working for drawings and not working at all. It +needs its own proof: a drawing authored at the root, nested into a held cel in +another lane, sampled before and after to show the same picture, in the style +of `drawn` in `lane_test`. diff --git a/frontend/src/arthur/domain/clip.cljs b/frontend/src/arthur/domain/clip.cljs index e4233a9..eeeafd8 100644 --- a/frontend/src/arthur/domain/clip.cljs +++ b/frontend/src/arthur/domain/clip.cljs @@ -149,11 +149,20 @@ which is long enough to key something into and short enough to scrub by hand." 120) -(defn blank - "A new, empty document: one empty symbol. +(def lane-node + "The lane every symbol is born with. - `:nodes` is empty rather than seeded with a layer, because an empty symbol is - a true statement and a layer nobody asked for is one more thing to delete. + EVERY SYMBOL HAS AT LEAST ONE LANE, because a lane is the only place + temporal content goes and a symbol with none has nowhere to drop a thing — + which made the first drop into any symbol a special case that had to invent + a lane before it could do what the second drop does. An empty lane is a true + statement about a symbol nobody has put anything in yet. The id is a keyword + rather than a uuid because this namespace is pure, and `:lane` reads in a + path; commands that add FURTHER lanes bring their own uuids." + {:id :lane :name "lane" :kind :group :layout :sequence :z "z-lane"}) + +(defn blank + "A new, empty document: one symbol, holding one empty lane. The tracking maps are ABSENT rather than empty, because `leaf/leaves` writes no leaf for an empty one and so cannot bring it back: a blank document that opened @@ -165,7 +174,8 @@ {:name "untitled" :fps 30 :width 320 :height 200 - :symbols {:main {:id :main :fps 30 :frames blank-frames :nodes {}}}}) + :symbols {:main {:id :main :fps 30 :frames blank-frames + :nodes {:lane lane-node}}}}) (defn- transform-op "Put a symbol's already resolved mark into its instance's parent space. Its @@ -410,7 +420,8 @@ (if (or (nil? end) (symbol clip sid) (nil? frame) (neg? frame) (>= frame end)) clip (-> clip - (assoc-in [:symbols sid] {:id sid :name (name sid) :fps (fps clip host) :frames (- end frame) :nodes {}}) + (assoc-in [:symbols sid] {:id sid :name (name sid) :fps (fps clip host) + :frames (- end frame) :nodes {:lane lane-node}}) (place-symbol nil host sid frame uuid nil))))) (defn free-id diff --git a/frontend/src/arthur/domain/lane.cljs b/frontend/src/arthur/domain/lane.cljs index d42134c..9a3366a 100644 --- a/frontend/src/arthur/domain/lane.cljs +++ b/frontend/src/arthur/domain/lane.cljs @@ -214,6 +214,17 @@ :z (symbol/z-between front nil)}) :selection id}))) +(defn- holds-other? + "Whether `lane-id` already holds clips that are not of `kind`. + + A lane holds picture or sound and not both, and the check belongs HERE + rather than only in validation: placement claims time, so a picture dropped + on a lane of sound would not be caught as a mixture — `blank` would have + deleted the sound to make room for it first, and the document would be + valid and the sound gone." + [nodes lane-id kind] + (boolean (some #(not= kind (:kind %)) (symbol/lane-clips nodes lane-id)))) + (defn place-symbol "Place arbitrary symbol `source-id` as a naturally playing clip in a lane. @@ -232,6 +243,8 @@ (first (node/placed-span n))))] (cond (not (node/lane? lane)) {:refused "select a lane"} + (holds-other? nodes lane-id :instance) + {:refused "that lane holds sound; a lane holds picture or sound, not both"} (contains? nodes id) {:refused "the new clip ID is already used"} (not (and (integer? at) (not (neg? at)))) {:refused "a position is a nonnegative whole lane frame"} @@ -252,10 +265,11 @@ (span/finish (:clip cleared) sid nodes id extent))))))) (defn adopt - "Move an existing visual instance into `lane-id` at lane frame `at`. - Its source, span, transforms, corrections, and identity come with it; the - destination interval is claimed with the same overwrite trimming as a pool - drop." + "Move an existing clip — a symbol instance or a sound — into `lane-id` at + lane frame `at`. Its source, span, transforms, corrections, and identity come + with it; the destination interval is claimed with the same overwrite trimming + as a pool drop. What a lane may not do is mix the two kinds, which + `symbol/lane-problems` is the judge of and `finish` enforces." [clip sid lane-id id at {:keys [extent remainder-id] :or {extent :keep}}] (let [nodes (get-in clip [:symbols sid :nodes]) lane (get nodes lane-id) @@ -264,7 +278,10 @@ duration (when (and lo hi) (- hi lo))] (cond (not (node/lane? lane)) {:refused "select a lane"} - (not= :instance (:kind n)) {:refused "only a symbol clip goes in a lane"} + (not (contains? #{:instance :audio} (:kind n))) + {:refused "only a symbol or sound clip goes in a lane"} + (holds-other? nodes lane-id (:kind n)) + {:refused "a lane holds picture or sound, not both"} (= lane-id (:parent n)) {:refused "this clip is already in that lane"} (not (and (integer? at) (not (neg? at)))) {:refused "a position is a nonnegative whole lane frame"} diff --git a/frontend/src/arthur/domain/nest.cljs b/frontend/src/arthur/domain/nest.cljs index 2363e51..7999ce0 100644 --- a/frontend/src/arthur/domain/nest.cljs +++ b/frontend/src/arthur/domain/nest.cljs @@ -273,6 +273,39 @@ (clip/update-symbol target update :nodes (fnil into {}) (map (juxt :id identity)) moved))})))) +(defn move-refusal + "Why `move-node` would refuse to move `from` into `to` at frame `f`, or nil. + + SAID BEFORE THE DROP, not after it. A drag that reparents has to tell the + person what it would do while they can still change their mind, and the only + honest source for that is the check the command itself makes. Hence one + function, asked by the gesture on the way past and by `move-node` on the way + in. + + The one that surprises: BOTH have to be on screen at this frame, because the + move keeps the picture and there is no common frame to keep it at otherwise. + Two clips in one lane never overlap, so nesting one into another there can + never be done — it is a thing to do between lanes, with the playhead + somewhere both of them are showing. + + The deeper refusals — generated parts, a stencil parted from what it clips — + belong to the transplant and are only known when it runs." + [clip store open from to f] + (let [here (inside clip store open (pop from) f) + there (inside clip store open to f) + n (get-in clip [:symbols (:sid here) :nodes (peek from)])] + (cond + (nil? n) "nothing to move" + (or (nil? here) (nil? there)) "both have to be on screen at this frame" + (nil? (:sid there)) "only a symbol can take it" + (not (and (:time here) (:time there))) + "a held or looping clip has no clock to move through" + (nil? (some-> there :matrix node/invert)) "the target is scaled to nothing" + (= (:sid here) (:sid there)) "it is already there" + (and (= :instance (:kind n)) + (some #(clip/contains-symbol? clip % (:sid there)) (node/sources n))) + "a symbol cannot go inside itself"))) + (defn move-node "Move the node at row path `from` — its last id is the node, the rest the instances down to where it lives — into the symbol placed by the instance at @@ -286,16 +319,11 @@ a (:time here) b (:time there) inv (some-> there :matrix node/invert)] - (cond - (nil? (get-in clip [:symbols (:sid here) :nodes (peek from)])) - {:refused "nothing to move"} - (or (nil? here) (nil? there)) {:refused "both have to be on screen at this frame"} - (nil? (:sid there)) {:refused "only a symbol can take it"} - (not (and a b)) {:refused "a looping instance is in the way"} - (nil? inv) {:refused "the target is scaled to nothing"} - :else (transplant clip store (:sid here) (:frame here) (peek from) (:sid there) - (node/mul! (node/mat) inv (:matrix here)) - (node/then-time (node/invert-time b) a))))) + (if-let [why (move-refusal clip store open from to f)] + {:refused why} + (transplant clip store (:sid here) (:frame here) (peek from) (:sid there) + (node/mul! (node/mat) inv (:matrix here)) + (node/then-time (node/invert-time b) a))))) (defn- down "Walk row path `path` down from symbol `sid` by structure alone: `{:sid diff --git a/frontend/src/arthur/domain/symbol.cljs b/frontend/src/arthur/domain/symbol.cljs index 1e909da..bd8783b 100644 --- a/frontend/src/arthur/domain/symbol.cljs +++ b/frontend/src/arthur/domain/symbol.cljs @@ -71,7 +71,13 @@ repeat cannot be longer than the number of nodes, so one step past that is proof of a loop and needs no bookkeeping. Caught rather than hung — a cycle is reachable from one bad `:node/set-parent`, and a hung tab is a far worse - diagnostic than a stack trace naming the nodes." + diagnostic than a stack trace naming the nodes. + + THE PROOF ONLY HOLDS FOR A NODE THIS SYMBOL HAS. An id that is not in `nodes` + contributes a step the count knows nothing about, and its lineage is just + itself; in an EMPTY symbol that one step used to be read as a loop, so asking + where a node of another symbol sits threw `parent cycle` instead of answering + that it sits nowhere." [nodes id] (let [up (fn [i] (when-let [p (:parent (get nodes i))] @@ -81,7 +87,7 @@ {:node i :parent p}))))) chain (into [] (comp (take-while some?) (take (inc (count nodes)))) (iterate up id))] - (when (> (count chain) (count nodes)) + (when (and (contains? nodes id) (> (count chain) (count nodes))) (throw (ex-info "parent cycle in symbol" {:node id :chain chain}))) chain)) @@ -147,10 +153,18 @@ the parent and stencil references, rather than wherever a command happens to build one. - Clips must be visual, finite and non-overlapping. An accidental overlap + Clips must be finite and non-overlapping. An accidental overlap is refused rather than resolved by draw order: two drawings exposed on one frame of one lane is a document nobody meant to write, and picking a winner - would hide it. Empty lanes are valid — a lane is made before it is filled." + would hide it. Empty lanes are valid — a lane is made before it is filled. + + A SOUND IS A CLIP TOO. A lane is the one temporal container, so audio sits in + one on the same terms as picture: its own frames in `:span`, where they land + in `:time`, and no overlap with its neighbours. What a lane may NOT hold is a + mixture, and that is the explicit capability the model wanted rather than a + per-frame guess: all picture or all sound, so what the lane does with the + frame it owns is answered by the lane and not by the clip that happens to be + under the playhead." [nodes] (vec (mapcat @@ -158,14 +172,17 @@ (when (node/lane? lane) (let [children (filter #(= id (:parent %)) (vals nodes)) valid? (fn [n] - (and (= :instance (:kind n)) + (and (contains? #{:instance :audio} (:kind n)) (empty? (node/problems n)) (:span n) (every? node/finite-number? (node/placed-span n)))) - intervals (sort-by first (map node/placed-span (filter valid? children)))] + kept (filter valid? children) + intervals (sort-by first (map node/placed-span kept))] (concat (for [n children :when (not (valid? n))] - (str "sequence " id " needs finite visual symbol clips: " (:id n))) + (str "sequence " id " needs finite symbol or sound clips: " (:id n))) + (when (< 1 (count (into #{} (map :kind) kept))) + [(str "lane " id " holds picture or sound, not both")]) (when (some (fn [[[_ b] [c _]]] (> b c)) (partition 2 1 intervals)) [(str "lane " id " has overlapping clips")]))))) nodes))) diff --git a/frontend/src/arthur/events/footage.cljs b/frontend/src/arthur/events/footage.cljs index 2966576..374b1bb 100644 --- a/frontend/src/arthur/events/footage.cljs +++ b/frontend/src/arthur/events/footage.cljs @@ -6,7 +6,9 @@ address of every block this produces." (:require [arthur.domain.bring :as bring] [arthur.domain.clip :as clip] + [arthur.domain.lane :as lane] [arthur.events.edit :as edit] + [arthur.events.ui :as ui] [arthur.events.playback :as pb] [arthur.flow.detect :as detect] [arthur.flow.ingest :as ingest] @@ -466,12 +468,13 @@ (rf/reg-event-db ::ask-convert - (fn [db [_ {:keys [frames label] :as footage} frame point]] + (fn [db [_ {:keys [frames label] :as footage} frame point target]] (assoc-in db [:ui :convert] (merge (select-keys footage [:id :label :frames :fps :video]) {:range [0 frames] :name (string/replace (str label) #"\.[^.]*$" "") - :host (get-in db [:ui :open]) :frame frame :point point})))) + :host (get-in db [:ui :open]) :frame frame :point point + :target target})))) (rf/reg-event-db ::convert-set @@ -494,28 +497,47 @@ (rf/reg-event-fx ::converted - (fn [{:keys [db]} [_ {{:keys [name host frame point range]} :request footage-id :footage-id} + ;; A TAKE IS A CLIP IN A LANE, like everything else that enters the timeline. + ;; It used to be placed straight into the open symbol as a row of its own, + ;; which was the one way to get temporal content that no lane owned. + (fn [{:keys [db]} [_ {{:keys [name frame point range target]} :request footage-id :footage-id} built]] - (let [uuid (random-uuid) + (let [entry (store/entry (:clip/current db)) + uuid (random-uuid) fps (get-in db [:clip :fps]) {:keys [clip sid tracked?]} - (bring/take (:clip (store/entry (:clip/current db))) (:clip built) - name footage-id range) + (bring/take (:clip entry) (:clip built) name footage-id range) + st (merge (:store entry) (:store built)) imported-frames (clip/output-frames clip sid) source-fps (get-in built [:clip :fps]) - db (edit/edit-entry - db - #(cond-> (bring/placed % clip (:store built) sid host frame uuid point) - tracked? (merge (select-keys built [:footage-id :source-blocks - :source-inputs]))))] - {:db (-> db - (update :ui dissoc :convert) - (assoc-in [:ui :selection] [:node host uuid [uuid]]) - (update :footage merge - {:loading? false - :status (str "made " name " · " imported-frames " frames at " fps " fps" - (when (not= fps source-fps) - (str " · sampled from " source-fps " fps")) - (when-not tracked? - " · as drawings: this project already tracks other footage"))})) - :dispatch [::pb/refresh-clock]}))) + where (ui/lane-destination db clip st frame target :picture) + result (if (:refused where) + where + (lane/place-symbol (:clip where) st (:sid where) (:lane-id where) + uuid sid (:at where) + {:extent :grow-symbol :point point + :remainder-id (random-uuid)}))] + (if-let [why (or (:refused where) (:refused result))] + {:db (-> db + (update :ui dissoc :convert) + (update :footage merge {:loading? false :status why}))} + (let [db (edit/edit-entry + db + #(cond-> (assoc % :clip (:clip result) :store st) + tracked? (merge (select-keys built [:footage-id :source-blocks + :source-inputs]))))] + {:db (-> db + (update :ui dissoc :convert) + (cond-> (:made? where) + (assoc-in [:ui :target] {:sid (:sid where) :id (:lane-id where) + :path [(:lane-id where)]})) + (assoc-in [:ui :selection] + [:node (:sid where) uuid (conj (vec (:path where)) uuid)]) + (update :footage merge + {:loading? false + :status (str "made " name " · " imported-frames " frames at " fps " fps" + (when (not= fps source-fps) + (str " · sampled from " source-fps " fps")) + (when-not tracked? + " · as drawings: this project already tracks other footage"))})) + :dispatch [::pb/refresh-clock]}))))) diff --git a/frontend/src/arthur/events/playback.cljs b/frontend/src/arthur/events/playback.cljs index c79e5e2..ce74ca5 100644 --- a/frontend/src/arthur/events/playback.cljs +++ b/frontend/src/arthur/events/playback.cljs @@ -190,6 +190,18 @@ {:db (-> db (update-in [:ui :tabs] #(if (some #{sid} %) % (conj (vec %) sid))) (assoc-in [:ui :open] sid) + ;; WHAT WAS SELECTED IS NOT IN HERE. A node selection and an + ;; aimed target are paths in the symbol being left, and every + ;; bar that reads one — the breadcrumb, the inspector, where a + ;; new symbol would land — reads it against the open one. So + ;; opening a symbol arrives with nothing selected, which is the + ;; state the root crumb already means. A `[:symbol _]` + ;; selection names a symbol rather than a place inside one and + ;; survives. + (update :ui (fn [ui] + (cond-> (dissoc ui :target) + (= :node (first (:selection ui))) + (dissoc :selection)))) (update-in [:ui :trace :faces] #(trace/showing-for clip sid %)) (assoc-in [:playback :frame] 0) (assoc-in [:playback :playing?] false)) diff --git a/frontend/src/arthur/events/project.cljs b/frontend/src/arthur/events/project.cljs index ab6b630..9bab3fd 100644 --- a/frontend/src/arthur/events/project.cljs +++ b/frontend/src/arthur/events/project.cljs @@ -34,6 +34,8 @@ [arthur.domain.wire :as wire] [arthur.events.footage :as footage] [arthur.events.playback :as pb] + [arthur.events.ui :as ui] + [arthur.domain.lane :as lane] [arthur.footage.store :as store] [arthur.flow.address :as address] [arthur.flow.ingest :as ingest] @@ -153,26 +155,41 @@ ::import ;; A symbol out of another saved project, dropped at `frame` of the open symbol ;; and, from the stage, with its middle on `point`. - (fn [{:keys [db]} [_ carried frame point]] + (fn [{:keys [db]} [_ carried frame point target]] {:db (update db :project merge {:status (str "fetching " (:label carried) "…")}) ::import! (assoc (select-keys carried [:project :cid :symbol :label]) - :host (get-in db [:ui :open]) :frame frame :point point)})) + :host (get-in db [:ui :open]) :frame frame :point point + :target target)})) (rf/reg-event-fx ::imported ;; As drawing: its tracking stays with the analysis that measured it. See ;; `arthur.domain.bring`. - (fn [{:keys [db]} [_ {:keys [symbol host frame point label]} other]] - (let [sid (leaf/unsegment symbol) + (fn [{:keys [db]} [_ {:keys [symbol frame point label target]} other]] + (let [entry (store/entry (:clip/current db)) + sid (leaf/unsegment symbol) uuid (random-uuid) - {:keys [clip ids]} (bring/symbols (:clip (store/entry (:clip/current db))) - (:clip other) [sid] {}) - db (edit/edit-entry db #(bring/placed % clip (:store other) (ids sid) - host frame uuid point))] - {:db (-> db - (assoc-in [:ui :selection] [:node host uuid [uuid]]) - (update :project merge {:status (str "brought in " label)})) - :dispatch [::pb/refresh-clock]}))) + {:keys [clip ids]} (bring/symbols (:clip entry) (:clip other) [sid] {}) + st (merge (:store entry) (:store other)) + ;; A symbol from another project arrives as a clip in a lane, the same + ;; as one from this project's pool. + where (ui/lane-destination db clip st frame target :picture) + result (if (:refused where) + where + (lane/place-symbol (:clip where) st (:sid where) (:lane-id where) + uuid (ids sid) (:at where) + {:extent :grow-symbol :point point + :remainder-id (random-uuid)}))] + (if-let [why (or (:refused where) (:refused result))] + {:db (update db :project merge {:status why})} + {:db (-> (edit/edit-entry db #(assoc % :clip (:clip result) :store st)) + (cond-> (:made? where) + (assoc-in [:ui :target] {:sid (:sid where) :id (:lane-id where) + :path [(:lane-id where)]})) + (assoc-in [:ui :selection] + [:node (:sid where) uuid (conj (vec (:path where)) uuid)]) + (update :project merge {:status (str "brought in " label)})) + :dispatch [::pb/refresh-clock]})))) (defn- clip-payload "One clip of a save. With `base` — the seq the open document last caught up diff --git a/frontend/src/arthur/events/ui.cljs b/frontend/src/arthur/events/ui.cljs index 36fc239..fc44f4e 100644 --- a/frontend/src/arthur/events/ui.cljs +++ b/frontend/src/arthur/events/ui.cljs @@ -108,6 +108,20 @@ n))) +(defn empty-lane + "The id of a lane of `sid` that has nothing in it, or nil. + + EVERY SYMBOL IS BORN WITH A LANE, so the first thing put into one goes + there instead of beside it. An OCCUPIED lane is never chosen this way: + placing claims time, so taking a lane nobody pointed at would trim or delete + what was already in it. That is a fine thing to ask for and not a fine thing + to assume." + [document sid] + (let [nodes (get-in document [:symbols sid :nodes])] + (first (keep (fn [lane] + (when (empty? (symbol/lane-clips nodes (:id lane))) (:id lane))) + (symbol/lanes nodes))))) + (defn apply-lane-command "Commit a successful domain command as one history step. A refused command leaves the document and history untouched; an overflow offers an explicit retry." @@ -608,7 +622,9 @@ (if-not host (update db :project merge {:status "what you are adding to is not on screen at this frame"}) - (let [prepared (lane/add-lane clip host lane-id) + (let [free (empty-lane clip host) + lane-id (or free lane-id) + prepared (if free {:clip clip} (lane/add-lane clip host lane-id)) result (if (:refused prepared) prepared (lane/overwrite-drawing (:clip prepared) host lane-id cel-id drawing-id frame @@ -643,6 +659,60 @@ ::drop-clear (fn [db _] (update db :ui dissoc :drop))) +(defn lane-destination + "Where a drop lands: the lane it was aimed at, the selected one, or a new one. + + EVERYTHING IN THE TIMELINE IS A LANE, so this is the one rule and every drop + asks it — a symbol from the pool, a sound, and a video brought in as a take + alike. It answers `{:clip :sid :lane-id :at :made?}` with the lane already + created in `:clip` when it had to make one, or `{:refused why}`. + + `kind` is what is about to go in, `:picture` or `:sound`: a lane holds one or + the other, so an aimed lane that holds the other kind is not the destination + and a new one is made beside it." + [db document st frame target kind] + (let [open (get-in db [:ui :open]) + selected (get-in db [:ui :selection]) + [_ sel-sid sel-id] selected + holds (fn [[_ sid id]] + (let [nodes (get-in document [:symbols sid :nodes])] + (when (node/lane? (get nodes id)) + (let [kinds (into #{} (map :kind) (symbol/lane-clips nodes id))] + (or (empty? kinds) + (= kinds #{(if (= :sound kind) :audio :instance)})))))) + ;; With nothing aimed, an empty lane that is already there — see + ;; `empty-lane` — and only then a new one. + free (when-let [id (empty-lane document open)] + (when (holds [:node open id]) [:node open id [id]])) + aimed (or (when (and target (holds target)) target) + (when (and (= :node (first selected)) (holds selected)) selected) + free) + [_ aimed-sid aimed-id aimed-path] aimed + sid (if aimed aimed-sid open) + lane-id (if aimed aimed-id (random-uuid)) + prepared (if aimed {:clip document} (lane/add-lane document sid lane-id)) + lane-sel [:node sid lane-id (if aimed aimed-path [lane-id])] + owner (when-not (:refused prepared) + (selection-frame (:clip prepared) st open lane-sel frame)) + at (when (number? owner) + (lane/lane-frame (:clip prepared) sid lane-id owner))] + (cond + (:refused prepared) prepared + (not (integer? at)) {:refused "the drop is not on one frame of this lane"} + :else {:clip (:clip prepared) :sid sid :lane-id lane-id :at at + :path (vec (butlast (nth lane-sel 3))) :made? (nil? aimed)}))) + +(defn landed + "`db` after a drop that produced `result`, with `uuid` selected in `lane`." + [db {:keys [sid lane-id path made?]} uuid result] + (if-let [why (:refused result)] + (-> db (update :ui dissoc :drop) (update :project merge {:status why})) + (cond-> (-> db + (update :ui dissoc :drop) + (edit/transaction (constantly (:clip result))) + (assoc-in [:ui :selection] [:node sid uuid (conj (vec path) uuid)])) + made? (assoc-in [:ui :target] {:sid sid :id lane-id :path [lane-id]})))) + (rf/reg-event-db ::drop-symbol ;; A symbol dropped on a lane becomes a naturally playing clip in that lane. @@ -650,47 +720,35 @@ ;; never invents another permanent row-per-symbol track. (fn [db [_ source-id frame point target]] (let [{document :clip st :store} (store/entry (:clip/current db)) - open (get-in db [:ui :open]) - selected (get-in db [:ui :selection]) - [_ selected-sid selected-id] selected - selected-node (get-in document [:symbols selected-sid :nodes selected-id]) - target (or target (when (node/lane? selected-node) selected)) - [_ target-sid target-id target-path] target - target-node (get-in document [:symbols target-sid :nodes target-id]) - existing? (and (= :node (first target)) (node/lane? target-node)) - lane-id (if existing? target-id (random-uuid)) - sid (if existing? target-sid open) - prepared (if existing? {:clip document} - (lane/add-lane document sid lane-id)) - lane-selection [:node sid lane-id (if existing? target-path [lane-id])] - owner-frame (selection-frame (:clip prepared) st open lane-selection frame) - at (when (number? owner-frame) - (lane/lane-frame (:clip prepared) sid lane-id owner-frame)) - uuid (random-uuid) - result (if (integer? at) - (lane/place-symbol (:clip prepared) st sid lane-id uuid source-id at - {:extent :grow-symbol :point point - :remainder-id (random-uuid)}) - {:refused "the drop is not on one frame of this lane"}) - path (conj (vec (butlast (nth lane-selection 3))) uuid)] - (if-let [why (or (:refused prepared) (:refused result))] - (-> db (update :ui dissoc :drop) (update :project merge {:status why})) - (cond-> (-> db - (update :ui dissoc :drop) - (edit/transaction (constantly (:clip result))) - (assoc-in [:ui :selection] [:node sid uuid path])) - (not existing?) - (assoc-in [:ui :target] {:sid sid :id lane-id :path [lane-id]})))))) + where (lane-destination db document st frame target :picture) + uuid (random-uuid)] + (if (:refused where) + (-> db (update :ui dissoc :drop) (update :project merge {:status (:refused where)})) + (landed db where uuid + (lane/place-symbol (:clip where) st (:sid where) (:lane-id where) + uuid source-id (:at where) + {:extent :grow-symbol :point point + :remainder-id (random-uuid)})))))) (rf/reg-event-db ::drop-sound - (fn [db [_ {:keys [source label length rate]} frame]] - (let [uuid (random-uuid) - host (get-in db [:ui :open])] - (-> db - (update :ui dissoc :drop) - (edit/edit #(clip/place-sound % host source label length rate (* frame (:rate (clip/grid-time % host))) uuid)) - (assoc-in [:ui :selection] [:node host uuid [uuid]]))))) + ;; A SOUND IS A CLIP IN A LANE TOO. It is placed and then adopted rather than + ;; written straight into the lane, so one command owns where a sound's frames + ;; are — `clip/place-sound` — and one owns what claiming lane time means. + (fn [db [_ {:keys [source label length rate]} frame target]] + (let [{document :clip st :store} (store/entry (:clip/current db)) + where (lane-destination db document st frame target :sound) + uuid (random-uuid)] + (if (:refused where) + (-> db (update :ui dissoc :drop) (update :project merge {:status (:refused where)})) + (let [sid (:sid where) + seeded (clip/place-sound (:clip where) sid source label length rate + (* (:at where) + (:rate (clip/grid-time (:clip where) sid))) + uuid)] + (landed db where uuid + (lane/adopt seeded sid (:lane-id where) uuid (:at where) + {:extent :grow-symbol :remainder-id (random-uuid)}))))))) (rf/reg-event-db ::adopt-in-lane diff --git a/frontend/src/arthur/ui/drag.cljs b/frontend/src/arthur/ui/drag.cljs index 82a174b..3161c0f 100644 --- a/frontend/src/arthur/ui/drag.cljs +++ b/frontend/src/arthur/ui/drag.cljs @@ -112,10 +112,11 @@ (case kind :symbol (rf/dispatch [::ui/drop-symbol sid frame point target]) ;; Video is asked about before anything happens: which frames, and what - ;; the symbol they become is called. - :footage (rf/dispatch [::footage/ask-convert c frame point]) - :import (rf/dispatch [::project/import c frame point]) + ;; the symbol they become is called. The lane it was aimed at travels + ;; with the question, so the answer lands where the drop pointed. + :footage (rf/dispatch [::footage/ask-convert c frame point target]) + :import (rf/dispatch [::project/import c frame point target]) ;; Where it is dropped in time; a sound has no place in space. - :sound (rf/dispatch [::ui/drop-sound c frame]) + :sound (rf/dispatch [::ui/drop-sound c frame target]) nil)) (done!))) diff --git a/frontend/src/arthur/ui/timeline.cljs b/frontend/src/arthur/ui/timeline.cljs index b7b8a0e..4e9d114 100644 --- a/frontend/src/arthur/ui/timeline.cljs +++ b/frontend/src/arthur/ui/timeline.cljs @@ -94,9 +94,82 @@ (defn rows "The visible rows of symbol `sid`, outermost first. `expanded` is a set of row - paths." - [clip sid expanded] - (letfn [(walk [sid path depth ->open] + paths, and `chosen` is the selected row's PATH, which is what an expanded + lane opens as its portal. + + AN EXPANDED LANE OPENS ONE CLIP, NOT ALL OF THEM. A lane of twelve clips that + grew twelve branches when it opened is the vertical growth the single-row + lane exists to prevent, so expanding a lane reveals exactly the clip that is + selected in it: its own keys, and — expanded in turn — the lanes and nodes of + the symbol it places, all the way down and all mapped into this ruler. + Selecting another clip swaps the portal in place rather than adding to it, so + the whole document stays editable from the root timeline at constant cost. + See `docs/lane-nesting-notes.md`. + + THE WHOLE LINEAGE CHOOSES THE PORTAL, not the selected node alone. Selecting + something nested — a shape inside the drawing the clip places, or the end of + its span — is still working inside that clip, so the portal that revealed it + must stay open. A lane therefore opens the clip whose path the selection is + under, which for a clip selected directly is the clip itself." + ([clip sid expanded] (rows clip sid expanded nil)) + ([clip sid expanded chosen] + (let [under? (fn [path] + (and chosen + (<= (count path) (count chosen)) + (= path (subvec (vec chosen) 0 (count path)))))] + (letfn [(inside-rows [sid n path depth self span] + ;; The rows of the symbol a clip places, mapped into this ruler. + ;; + ;; A HELD CLIP HAS NO INVERTIBLE CLOCK. `clip/source-time` is nil + ;; for a hold, a loop and an endpoint policy: one source frame is + ;; shown for the whole span, so no frame inside it has a place on + ;; this ruler. That used to mean the contents were not shown at + ;; all, which hid the inside of every drawing — the most ordinary + ;; thing in the document. So the structure is shown and stays + ;; selectable, and what is withheld is only what cannot be known: + ;; the key positions, and the bars that would imply them. Refuse + ;; rather than guess, without refusing the whole subtree. + (when-let [source (and (= :instance (:kind n)) (node/source n))] + (if-let [{:keys [at rate]} (clip/source-time clip sid n)] + (walk source path depth (comp self #(+ at (/ % rate)))) + (mapv (fn [row] + (-> row + (assoc :keys [] :unmapped? true) + (assoc :span (when (= :node (:kind row)) span)))) + (walk source path depth (constantly (first span))))))) + (portal [sid path depth self child] + ;; `self` maps the LANE's frames into the open symbol's; `child` is + ;; the clip selected in that lane, whose own frames are one more + ;; step in. Its row carries the clip's path — the same one its + ;; block in the lane selects with — so expanding here and selecting + ;; there are the same place. + (let [cpath (conj path (:id child)) + open? (contains? expanded cpath) + channels (node/channels child) + cself (comp self (local->parent child)) + cspan (mapv self (node/placed-span child)) + row {:path cpath + :depth depth + :label (node-label (:id child) child) + :kind :node + :node-kind (:kind child) + :of (node/source child) + :portal? true + :select [:node sid (:id child) cpath] + :expandable? true + :expanded? open? + :span cspan + :keys (into [] (comp (mapcat keyed-frames) + (map cself) + (distinct)) + (vals channels)) + :dense? (boolean (some :dense (vals channels)))}] + (if-not open? + [row] + (-> [row] + (into (channel-rows child cpath (inc depth) cself cspan)) + (into (inside-rows sid child cpath (inc depth) cself cspan)))))) + (walk [sid path depth ->open] (let [sym (get-in clip [:symbols sid]) ordered (->> (:nodes sym) ;; Front-most at the top, as a layer list is drawn @@ -153,7 +226,16 @@ ;; is shared. (if (node/lane? (get-in sym [:nodes (:parent n)])) [] - (let [row (cond-> row + (let [clips (when (node/lane? n) + (symbol/lane-clips (:nodes sym) id)) + ;; A lane of sounds is a lane like any other — + ;; same blocks, same edges, same portal — and + ;; is listed under the audio heading because + ;; that is where somebody looks for a sound, + ;; not because it is a different kind of row. + sound-lane? (and (seq clips) + (every? #(= :audio (:kind %)) clips)) + row (cond-> row (node/lane? n) (assoc :cels (mapv (fn [child] @@ -162,27 +244,54 @@ (some-> (node/source child) name)) :source (node/source child) :span (mapv self (node/placed-span child)) + ;; The clip's own keys, on the + ;; block, so a collapsed lane + ;; still says where it changes. + :keys (into [] + (comp (mapcat keyed-frames) + (map (comp self (local->parent child))) + (distinct)) + (vals (node/channels child))) :select [:node sid (:id child) (conj path (:id child))]}) - (symbol/lane-clips (:nodes sym) id))))] - (if-not open? - [row] - (-> [row] - (into (channel-rows n rpath (inc depth) self span)) - (into (when-let [{:keys [at rate]} (and (= :instance (:kind n)) - (clip/source-time clip sid n))] - (when-let [child (node/source n)] - (walk child rpath (inc depth) - (comp self #(+ at (/ % rate))))))))))))) + clips)))] + (cond->> (if-not open? + [row] + (-> [row] + (into (channel-rows n rpath (inc depth) self span)) + ;; The one clip an expanded lane opens. + (into (when (node/lane? n) + (if-let [child (first (filter #(under? (conj path (:id %))) clips))] + (portal sid path (inc depth) self child) + [{:path (conj rpath ::portal) + :depth (inc depth) + :kind :hint + :label (if (seq clips) + "select a clip to inspect" + "empty lane")}]))) + (into (inside-rows sid n rpath (inc depth) self span)))) + sound-lane? (mapv #(assoc % :sound? true))))))) ordered))))] (if (get-in clip [:symbols sid]) (walk sid [] 0 #(/ % (:rate (clip/grid-time clip sid)))) - []))) + []))))) (defn sound-rows "Audio rows use the same flattened intervals as the mixer, including source - in-points, cel speeds, parent timing, and silence beneath visual holds." + in-points, cel speeds, parent timing, and silence beneath visual holds. + + WHAT IS IN A LANE OF THIS SYMBOL IS NOT FLATTENED HERE. A sound in a lane is + a clip somebody placed and can move, trim and open, and `rows` draws it as + one; flattening it as well would show the same sound on two rows, only one of + which could be edited. What remains is what this view is for: audio nested + inside the symbols this one places, mapped into this ruler." [clip sid expanded] (if-not (get-in clip [:symbols sid]) [] + (let [nodes (get-in clip [:symbols sid :nodes]) + in-lane (into #{} (keep (fn [[id n]] + (when (and (= :audio (:kind n)) + (node/lane? (get nodes (:parent n)))) + id))) + nodes)] (vec (mapcat (fn [[path tracks]] @@ -203,7 +312,11 @@ :span (node/placed-span track) :select select}) (range) tracks))) (when open? (channel-rows n path 1 identity span))))) - (sort-by (comp str key) (group-by :path (nest/audio-tracks clip sid))))))) + (sort-by (comp str key) + (group-by :path + (remove #(and (= 1 (count (:path %))) + (contains? in-lane (first (:path %)))) + (nest/audio-tracks clip sid))))))))) ;; --------------------------------------------------------------------------- ;; geometry @@ -235,6 +348,19 @@ [track selection]))) (array-seq (.elementsFromPoint js/document (.-clientX event) (.-clientY event))))) +(defn- clip-under + "What the clip block under the pointer places, or nil where there is none. + + The track holds the pointer while a bar slides, and a captured pointer takes + the CLICK with it: press a clip block and the click and double-click that + follow are delivered to the track, never to the block. So the track resolves + them itself, by asking what is geometrically under the pointer." + [^js event] + (some (fn [^js el] + (when-let [cel (.closest el ".tl-cel")] + (aget cel "arthurCel"))) + (array-seq (.elementsFromPoint js/document (.-clientX event) (.-clientY event))))) + ;; --------------------------------------------------------------------------- ;; the panes @@ -496,24 +622,64 @@ "`sliding` is the pointer's side of a bar being dragged, `{:path :x :width :df}`. What it looks like mid-drag is `[:ui :sliding]`, which the clip every row and the stage are drawn from already has in it." - [{:keys [path span keys dense? kind node-kind select slides cels lane?]} frames sliding] + [{:keys [path span keys dense? kind node-kind select slides cels lane? of unmapped?]} + frames sliding hint {:keys [clip store open frame]}] (let [active-row (:row @sliding) slide (fn [^js e] (let [{from :row x0 :x width :width} @sliding] (when (= path from) (let [df (js/Math.round (/ (* frames (- (.-clientX e) x0)) (max 1 width))) drag (:drag @sliding) - [target-el target] (when drag (lane-under e)) + ;; TWO INTENTIONS, SAID BY A MODIFIER. An ordinary + ;; body drag moves a clip in TIME, within its lane + ;; or into another. Holding shift means something + ;; else entirely: put this node INSIDE the symbol + ;; the clip under the pointer places, keeping where + ;; it looks and when it happens. Overlap cannot say + ;; which is meant — dropping on occupied time + ;; already means claiming it — so the person says. + shift? (.-shiftKey e) + under (when (and drag shift?) (clip-under e)) + ;; Asked of the command itself, so the hint cannot + ;; promise what the drop would refuse. + why (when (and under (not= (:select under) (:selection drag))) + (nest/move-refusal clip store open + (nth (:selection drag) 3) + (nth (:select under) 3) + frame)) + nest (when (and under (nil? why) + (not= (:select under) (:selection drag))) + under) + [target-el target] (when (and drag (not shift?)) (lane-under e)) crossing? (and target (not= target (:source-lane drag))) target-frame (when crossing? (max 0 (- (frame-at-element e frames target-el) (:grab drag))))] - (if crossing? + (when (and drag hint) + (reset! hint + {:x (.-clientX e) :y (.-clientY e) + :nest? (boolean nest) + :no? (boolean why) + :text (cond + nest (str "nest into " (:label nest)) + why (str "can't nest here · " why) + shift? "shift: nest into a clip" + :else "drop to place · shift to nest")})) + (cond + nest (do - (swap! sliding assoc :target-lane target :target-frame target-frame) + (swap! sliding #(-> % (assoc :nest nest) + (dissoc :target-lane :target-frame))) (rf/dispatch [::ui/sliding nil])) + crossing? (do - (swap! sliding dissoc :target-lane :target-frame) + (swap! sliding #(-> % (assoc :target-lane target + :target-frame target-frame) + (dissoc :nest))) + (rf/dispatch [::ui/sliding nil])) + :else + (do + (swap! sliding dissoc :target-lane :target-frame :nest) (when (not= df (:df @sliding)) (swap! sliding assoc :df df) (rf/dispatch [::ui/sliding (:path @sliding) df @@ -521,21 +687,42 @@ (:other @sliding)])))))))) done (fn [commit?] (when (= path (:row @sliding)) - (let [{:keys [path df kind ripple? other target-lane target-frame drag]} @sliding] + (let [{:keys [path df kind ripple? other target-lane target-frame + drag nest on-click]} @sliding] (reset! sliding nil) + (when hint (reset! hint nil)) (cond + (and commit? nest drag) + (do (rf/dispatch [::ui/sliding nil]) + ;; `nest/move-node`, which is what keeps the world + ;; transform and the root timing across the move. + (rf/dispatch [::ui/move-node (nth (:selection drag) 3) + (nth (:select nest) 3)])) (and commit? target-lane drag) (do (rf/dispatch [::ui/sliding nil]) (rf/dispatch [::ui/adopt-in-lane (:selection drag) target-lane target-frame])) - commit? (rf/dispatch [::ui/slide path df kind ripple? other]) + commit? + ;; A press that moved nothing is a click, and a drag + ;; that only moved in time leaves what it moved + ;; selected. The two structural cases above select what + ;; they landed, so neither needs this. + (do (when on-click (rf/dispatch [::ui/select on-click])) + (rf/dispatch [::ui/slide path df kind ripple? other])) :else (rf/dispatch [::ui/sliding nil]))))) begin! (fn [^js e actual-path gesture-kind actual-select other drag] (let [track (.closest (.-currentTarget e) ".tl-track")] (.stopPropagation e) - (when actual-select (rf/dispatch [::ui/select actual-select])) + ;; SELECTING WAITS FOR THE RELEASE. Selecting on the press + ;; changed what the timeline was showing before the gesture + ;; had said anything: an expanded lane follows the + ;; selection, so pressing a clip to drag it somewhere shut + ;; the portal holding the lane being dragged INTO, out from + ;; under the pointer. The press now only starts the + ;; gesture; what it meant is known on release. (reset! sliding {:row path :path actual-path :kind gesture-kind :other other + :on-click actual-select :drag (when drag (assoc drag :source-lane select @@ -552,6 +739,15 @@ {:on-pointer-move slide :on-pointer-up (fn [e] (slide e) (done true)) :on-pointer-cancel (fn [_] (done false)) + ;; THE CLICK ARRIVES HERE, not on the block it started on, because the + ;; track captured the pointer. Opening what was double-clicked is + ;; therefore the track's job: the block under the pointer if there is + ;; one, else this row's own instance. It opens as a tab, which is what + ;; double-clicking the same symbol in the pool does. + :on-double-click (fn [^js e] + (when-let [source (or (:source (clip-under e)) of)] + (.stopPropagation e) + (rf/dispatch [::pb/open-symbol source]))) :ref (when lane? (fn [el] (when el (aset el "arthurLane" select)))) :on-drag-enter (fn [^js e] (when (and lane? (or (drag/accepts?) (drag/row))) @@ -580,15 +776,26 @@ ;; symbol still plays its own frames from 0, it is just cut off at the end. (when-let [[in out] (when (and span (nil? cels)) [(max 0 (first span)) (min frames (second span))])] (when (< in out) + ;; AN UNMAPPED BAR IS NOT DRAGGABLE. Inside a held clip a nested row + ;; is shown across the whole hold because that is when it is on + ;; screen, not because its frames are this ruler's: there is no + ;; mapping to edit through, so the bar selects and does not slide. [:div {:class (str "tl-span" (when dense? " dense") (when (= :ghost kind) " ghost") (when (= :audio node-kind) " sound") - (when select " movable") (when (= path active-row) " sliding")) + (when unmapped? " unmapped") + (when (and select (not unmapped?)) " movable") + (when (= path active-row) " sliding")) + :title (when unmapped? + "inside a held clip · its own frames have no place on this ruler") :style {:left (edge% in frames) :width (str (* 100 (/ (- out in) (max 1 frames))) "%")} + :on-click (when (and select unmapped?) + (fn [^js e] (.stopPropagation e) + (rf/dispatch [::ui/select select]))) :on-pointer-down - (when select + (when (and select (not unmapped?)) #(begin! % (or slides path) :slide select nil nil))} - (when select + (when (and select (not unmapped?)) [:span.tl-edge.out {:title "Drag endpoint · Shift-drag ripples later clips" :on-pointer-down #(begin! % path :out select nil nil)}])])) (doall @@ -607,22 +814,26 @@ :when (< in out)] ^{:key (str id)} [:button.tl-cel - {:title (str label " · select clip; double-click to edit its symbol") - :class (when ghost? "ghost") + {:title (str label " · select clip; double-click to edit its symbol" + " · shift-drag another clip onto it to nest that clip inside") + :class (str (when ghost? "ghost") + (when (and select (= select (get-in @sliding [:nest :select]))) + " nest-target")) :style {:position "absolute" :left (edge% in frames) :width (str (* 100 (/ (- out in) (max 1 frames))) "%") :top "2px" :bottom "2px" :overflow "visible" :padding "0 3px"} + ;; What this block is, for the track to read back: the click that + ;; selects it and the double-click that opens it are both delivered + ;; to the track, which holds the pointer. See `clip-under`. + :ref (when select + (fn [^js el] + (when el (aset el "arthurCel" {:select select :source source + :label label + :in (js/Math.floor in)})))) :on-pointer-down (when select #(begin! % (nth select 3) :slide select nil {:selection select :label label :in in - :duration (- out in)})) - :on-click (when select - (fn [e] (.stopPropagation e) - (rf/dispatch [::ui/select select]) - (rf/dispatch [::pb/seek (js/Math.floor in)]))) - :on-double-click (when select - (fn [e] (.stopPropagation e) - (when source (rf/dispatch [::pb/open-symbol source]))))} + :duration (- out in)}))} [:span.tl-cel-label label] (when select (if joined? @@ -645,22 +856,40 @@ :on-pointer-down #(begin! % (nth select 3) :out select nil nil)}])])) ;; A dense channel has a value on every frame, so ticking each one is a solid ;; block that says less than the bar behind it already does. + ;; The lane's own keys, and those of the clips on it drawn after the + ;; blocks so they land ON the block they belong to: a collapsed lane still + ;; says where the thing in it changes, without opening anything. (when-not dense? (doall - (for [f keys :when (and (<= 0 f) (< f frames))] + (for [f (distinct (concat keys (mapcat :keys cels))) + :when (and (<= 0 f) (< f frames))] ^{:key f} [:div.tl-key {:style {:left (at% f frames)}}])))])) +(defn- cursor-hint + "What the drag in flight would do, beside the pointer. + + ITS OWN COMPONENT, deref'ing its own atom: the pointer moves many times a + second and every row of the timeline reads `sliding`, so putting the pointer + position in there would repaint the whole pane to move a label two pixels." + [hint] + (when-let [{:keys [x y text nest? no?]} @hint] + [:div.tl-hint {:class (str (when nest? "nesting") (when no? "refusing")) + :style {:left (str (+ x 16) "px") :top (str (+ y 18) "px")}} + text])) + (defn- timeline-view [] (r/with-let [scrubbing (r/atom false) ;; The row a carried row is over and which part of it, for the ;; highlight. over (r/atom nil) sliding (r/atom nil) + hint (r/atom nil) renaming (r/atom nil) draft (r/atom "")] (let [clip @(rf/subscribe [::render/clip]) frames (max 1 (or @(rf/subscribe [::render/frames]) 1)) frame @(rf/subscribe [::playback/frame]) + store @(rf/subscribe [::render/store]) selection @(rf/subscribe [::sub/selection]) target @(rf/subscribe [::sub/target]) expanded @(rf/subscribe [::sub/expanded]) @@ -684,15 +913,18 @@ lane-ghost (when (and drop drop-lane (not (:sound? drop))) {:id ::drop :label (str "+ " (:label drop)) :ghost? true :span [(:frame drop) (+ (:frame drop) (or (:frames drop) 1))]}) - picture (cond->> (cond->> (rows clip open expanded) + chosen (when (= :node (first selection)) (nth selection 3)) + picture (cond->> (cond->> (rows clip open expanded chosen) lane-ghost (mapv (fn [row] (if (= drop-lane (:select row)) (update row :cels (fnil conj []) lane-ghost) row)))) (and ghost (not (:sound? drop))) (cons ghost)) - sounds (cond->> (sound-rows clip open expanded) + sounds (cond->> (into (vec (filter :sound? picture)) + (sound-rows clip open expanded)) (and ghost (:sound? drop)) (cons ghost)) + picture (remove :sound? picture) ;; The audio section's heading is a row like the others, so the two ;; columns stay aligned without measuring anything. visible (cond-> (vec picture) @@ -778,10 +1010,12 @@ (doall (for [row visible] (with-meta (if (= :section (:kind row)) [:div.tl-track.tl-section] - [track-cell row frames sliding]) + [track-cell row frames sliding hint + {:clip clip :store store :open open :frame frame}]) {:key (str (:path row))}))) [:div.tl-empty "nothing in this symbol"]) - [:div.tl-playhead {:style {:left (at% frame frames)}}]]]]))) + [:div.tl-playhead {:style {:left (at% frame frames)}}]]] + [cursor-hint hint]]))) (defn view [] [timeline-view]) diff --git a/frontend/test/arthur/domain/bring_test.cljs b/frontend/test/arthur/domain/bring_test.cljs index d0ddbcc..50cb809 100644 --- a/frontend/test/arthur/domain/bring_test.cljs +++ b/frontend/test/arthur/domain/bring_test.cljs @@ -22,6 +22,7 @@ (is (= {:main :take :inner :inner-2} ids) "the root gets the name asked for; a taken id gets the next free one") (is (= 10 (clip/frames clip :inner)) "what was already here is untouched") - (is (= #{:inner-2} (node/sources (first (vals (get-in clip [:symbols :take :nodes]))))) + (is (= #{:inner-2} (node/sources (first (filter #(= :instance (:kind %)) + (vals (get-in clip [:symbols :take :nodes])))))) "and the copy's instance follows its renamed symbol") (is (empty? (clip/problems clip))))) diff --git a/frontend/test/arthur/domain/instance_test.cljs b/frontend/test/arthur/domain/instance_test.cljs index 5f7572e..833d3c5 100644 --- a/frontend/test/arthur/domain/instance_test.cljs +++ b/frontend/test/arthur/domain/instance_test.cljs @@ -241,9 +241,11 @@ made (clip/new-symbol c :outer id 20 u)] (is (= :symbol-1 id)) (is (= :symbol-2 (clip/fresh-id made)) "the next one does not collide") - (is (= {:id :symbol-1 :name "symbol-1" :fps 30 :frames 180 :nodes {}} + (is (= {:id :symbol-1 :name "symbol-1" :fps 30 :frames 180 + :nodes {:lane clip/lane-node}} (clip/symbol made :symbol-1)) - "empty, and as long as the rest of what it was placed in") + "empty but for the lane every symbol is born with, and as long as the + rest of what it was placed in") (is (= {:span [0 180] :time {:mode :map :at 20 :rate 1}} (select-keys (get-in made [:symbols :outer :nodes u]) [:span :time]))) (is (= #{:symbol-1} (node/sources (get-in made [:symbols :outer :nodes u]))) diff --git a/frontend/test/arthur/domain/lane_test.cljs b/frontend/test/arthur/domain/lane_test.cljs index 5ee9f0a..36ecb9a 100644 --- a/frontend/test/arthur/domain/lane_test.cljs +++ b/frontend/test/arthur/domain/lane_test.cljs @@ -601,3 +601,50 @@ (is (= 12 (get-in (:clip (span/trim doc :main :insert :out 9)) [:symbols :main :frames])) "and trimming the last cel leaves the window where it was"))) + +(deftest a-take-placed-in-a-lane-is-still-heard + ;; `bring/take` puts a take's sound INSIDE the symbol it makes, so that + ;; "wherever the symbol is placed it is heard". A lane is one of the places it + ;; can be placed, and must not be the one place that goes silent. + (let [doc (assoc-in (document) [:symbols :take] + {:id :take :frames 10 :fps 24 + :nodes {:pic {:id :pic :kind :instance :z "a" + :source {:symbol :wave} :span [0 10] + :time {:mode :map :at 0 :rate 1} + :playback {:in 0 :speed 1 :end :stop}} + :sound {:id :sound :name "sound" :kind :audio + :parent nil :z "z-sound" + :source {:footage "f1"} :span [0 10] + :time {:mode :map :at 0 :rate 1}}}}) + at-root (clip/place-symbol doc nil :main :take 0 :root nil) + in-lane (:clip (lane/place-symbol doc nil :main :girl :drop :take 0 + {:extent :grow-symbol :remainder-id :tail}))] + (is (= 1 (count (nest/audio-tracks at-root :main))) + "a take placed at the root is heard") + (is (some? in-lane) "the take goes into the lane") + (is (= 1 (count (nest/audio-tracks in-lane :main))) + "and is still heard from inside a lane"))) + +(deftest a-sound-is-a-clip-in-a-lane-like-any-other + ;; Everything in the timeline is a lane, audio included: a sound claims lane + ;; time by the same rule, and what a lane will not do is hold both kinds. + (let [made (lane/add-lane (document) :main :track) + seeded (clip/place-sound (:clip made) :main {:sound "s1"} "voice" 6 1 2 :vo) + result (lane/adopt seeded :main :track :vo 2 {:extent :grow-symbol}) + after (:clip result) + n (get-in after [:symbols :main :nodes :vo])] + (is (nil? (:refused result)) (str (:refused result))) + (is (= :track (:parent n))) + (is (= [2 8] (node/placed-span n))) + (is (empty? (clip/problems after))) + (is (= 1 (count (nest/audio-tracks after :main))) + "a sound in a lane is still heard") + (is (= [:vo] (mapv :id (symbol/lane-clips (get-in after [:symbols :main :nodes]) :track)))) + ;; The one thing a lane refuses: being half picture and half sound, which + ;; is the explicit capability rather than a guess per frame. + (let [mixed (lane/place-symbol after nil :main :track :also :wave 2 + {:extent :grow-symbol :remainder-id :rest})] + (is (:refused mixed)) + (is (re-find #"picture or sound" (str (:refused mixed)))) + (is (= [:vo] (mapv :id (symbol/lane-clips (get-in after [:symbols :main :nodes]) :track))) + "and the sound it would have had to delete to make room is still there")))) diff --git a/frontend/test/arthur/domain/nest_test.cljs b/frontend/test/arthur/domain/nest_test.cljs index 52c27d8..4dd04d9 100644 --- a/frontend/test/arthur/domain/nest_test.cljs +++ b/frontend/test/arthur/domain/nest_test.cljs @@ -170,7 +170,8 @@ inst (get-in grouped [:symbols :main :nodes b-uuid])] (is (nil? (:refused r)) (:refused r)) (is (= #{:tri a-uuid} (set (keys (get-in grouped [:symbols :group-1 :nodes]))))) - (is (= [b-uuid] (keys (get-in grouped [:symbols :main :nodes]))) "one instance where they were") + (is (= [b-uuid] (keys (dissoc (get-in grouped [:symbols :main :nodes]) :lane))) + "one instance where they were") (is (= 4 (get-in inst [:time :at])) "starting where the earliest of them starts") (is (= 56 (clip/frames grouped :group-1)) "and lasting until the last one ends") (is (= (picture c :main fs) (picture grouped :main fs))) diff --git a/frontend/test/arthur/events/lane_test.cljs b/frontend/test/arthur/events/lane_test.cljs index 5008a1a..5f7feda 100644 --- a/frontend/test/arthur/events/lane_test.cljs +++ b/frontend/test/arthur/events/lane_test.cljs @@ -107,7 +107,10 @@ saved (:clip (store/entry (:clip/current after))) lanes (symbol/lanes (get-in saved [:symbols :main :nodes]))] (is (= :polygon (get-in after [:ui :tool]))) - (is (empty? lanes)) + (is (= [:lane] (mapv :id lanes)) + "the lane the symbol was born with, and no second one invented here") + (is (empty? (symbol/lane-clips (get-in saved [:symbols :main :nodes]) :lane)) + "and nothing put in it") (is (nil? (get-in after [:ui :target]))) (is (nil? (get-in (store/entry (:clip/current after)) [:history :done]))) (is (empty? (clip/problems saved))))) @@ -153,3 +156,67 @@ (let [refused (ui/apply-correction-command after {:refused "nope"})] (is (= "nope" (get-in refused [:project :status]))) (is (= 1 (count (get-in (store/entry id) [:history :done]))))))) + +(deftest an-expanded-lane-opens-the-selected-clip-and-everything-under-it + ;; The whole document is editable from the root timeline: a lane opens one + ;; portal — the clip selected in it — and that portal opens the lanes and + ;; nodes of the symbol it places, mapped into this ruler. + (let [doc (fixture/document) + open #{[:girl] [:insert]} + shut (timeline/rows doc :main open [:insert]) + of (fn [rows] (mapv (juxt :label :depth) rows)) + lane-row (first (filter :cels (timeline/rows doc :main open [:insert])))] + (is (= 1 (count (filter :portal? shut))) + "exactly one clip is opened, not one branch per clip in the lane") + (is (= [:insert] (:path (first (filter :portal? shut)))) + "and it is the selected one") + (is (some #{["mark" 2]} (of shut)) + "the clip's own symbol appears under it, at its depth") + (is (= [:node :main :insert [:insert]] + (:select (first (filter :portal? shut)))) + "the portal addresses the same clip its block in the lane does") + (is (= (:keys (second (:cels lane-row))) + (:keys (first (filter #(= [:b] (:path %)) (timeline/rows doc :main open [:b]))))) + "a clip's keys are on its block whether or not its portal is open"))) + +(deftest a-nested-selection-keeps-the-portal-that-revealed-it-open + ;; Clicking a shape inside the clip — or the end of its span — is still + ;; working inside that clip. Matching the selected id alone would close the + ;; portal the moment anything under it was touched. + (let [doc (fixture/document) + open #{[:girl] [:insert]} + deep (timeline/rows doc :main open [:insert :mark]) + none (timeline/rows doc :main open [:plate])] + (is (= [:insert] (:path (first (filter :portal? deep)))) + "a selection under the clip keeps that clip's portal") + (is (empty? (filter :portal? none))) + (is (some #{"select a clip to inspect"} (map :label none)) + "with nothing selected in it, an open lane says what it is waiting for"))) + +(deftest a-held-clip-shows-its-contents-without-inventing-frames-for-them + ;; `clip/source-time` is nil for a hold, so nested keys have no place on this + ;; ruler — but the drawing's own nodes must still be reachable from here. + (let [doc (fixture/document) + rows (timeline/rows doc :main #{[:girl] [:a]} [:a]) + inside (filter :unmapped? rows)] + (is (seq inside) "a held drawing opens") + (is (some #{"mark"} (map :label inside))) + (is (every? (comp empty? :keys) inside) + "no key is placed where the hold cannot say it belongs") + (is (= [[0 4]] (distinct (keep :span (filter #(= :node (:kind %)) inside)))) + "its rows span the hold, which is when it is on screen"))) + +(deftest a-lane-of-sounds-is-drawn-as-a-lane-and-not-flattened-twice + (let [made (lane/add-lane (fixture/document) :main :track) + seeded (clip/place-sound (:clip made) :main {:sound "s1"} "voice" 6 1 2 :vo) + doc (:clip (lane/adopt seeded :main :track :vo 2 {:extent :grow-symbol})) + picture (remove :sound? (timeline/rows doc :main #{} nil)) + sound-lanes (filter :sound? (timeline/rows doc :main #{} nil)) + flattened (timeline/sound-rows doc :main #{})] + (is (= 1 (count sound-lanes)) "the sound's lane is one row, like any lane") + (is (= [:vo] (mapv :id (:cels (first sound-lanes)))) + "with the sound on it as a block that can be moved and trimmed") + (is (empty? (filter #(= [:track] (:path %)) picture)) + "and it is not also listed among the picture rows") + (is (empty? flattened) + "nor flattened into a second, parallel audio row"))) diff --git a/frontend/test/browser/lane.mjs b/frontend/test/browser/lane.mjs index c9ba043..24e3038 100644 --- a/frontend/test/browser/lane.mjs +++ b/frontend/test/browser/lane.mjs @@ -160,6 +160,31 @@ try { }); await sleep(250); }; + const tabs = () => evaluate(`(() => { + const k = cljs.core.keyword, db = cljs.core.deref(re_frame.db.app_db); + return {tabs: cljs.core.clj__GT_js(cljs.core.get_in(db, [k('ui'), k('tabs')])).map(String), + open: String(cljs.core.clj__GT_js(cljs.core.get_in(db, [k('ui'), k('open')])))}; + })()`); + // A real two-press double-click, not `.dispatchEvent`: what broke here was + // where the browser decides to deliver the click, which a synthetic event + // cannot show. + const doubleClick = async selector => { + const p = await evaluate(`(() => { + const el = document.querySelector(${JSON.stringify(selector)}); + if (!el) return null; + const r = el.getBoundingClientRect(); + return {x: r.left + r.width / 2, y: r.top + r.height / 2}; + })()`); + assert(p, `something to double-click: ${selector}`); + for (const clickCount of [1, 2]) { + await send('Input.dispatchMouseEvent', {type: 'mousePressed', x: p.x, y: p.y, + button: 'left', buttons: 1, clickCount}); + await send('Input.dispatchMouseEvent', {type: 'mouseReleased', x: p.x, y: p.y, + button: 'left', buttons: 0, clickCount}); + await sleep(60); + } + await sleep(280); + }; const dropPoolSymbol = async frame => { const points = await evaluate(`(() => { const source = document.querySelector('.pool-row:not(.main) .pool-item[draggable="true"]'); @@ -301,8 +326,66 @@ try { const lanes = Object.values(s.clip.symbols.main.nodes).filter(n => n.layout === 'sequence'); assert.deepEqual(lanes.map(l => instances(s).filter(n => n.parent === l.id).length).sort(), [1, 3], 'a clip body can move from one lane to another'); + + // EXPANDING A LANE OPENS THE SELECTED CLIP. Its own keys, and under it the + // lanes and nodes of the symbol it places, all on this ruler — which is what + // makes the whole document editable from the root timeline. + const rowLabels = () => evaluate( + `[...document.querySelectorAll('.tl-labels > .tl-label')].map(e => e.textContent.trim())`); + const twist = async i => { + assert(await evaluate(`(() => { + const t = document.querySelectorAll('.tl-labels > .tl-label .tl-twist')[${i}]; + if (!t || t.disabled) return false; + t.click(); return true; + })()`), `an expander at row ${i}`); + await sleep(220); + }; + await evaluate(`(() => { document.querySelector('.tl-track .tl-cel').click(); return true })()`); + await sleep(200); + const collapsed = await rowLabels(); + await twist(0); + const opened = await rowLabels(); + assert(opened.length > collapsed.length, 'the lane opens'); + assert.equal(opened.filter(l => l.includes('instance')).length, 1, + `one clip portal, not one branch per clip: ${JSON.stringify(opened)}`); + const portalAt = opened.findIndex(l => l.includes('instance')); + await twist(portalAt); + const deep = await rowLabels(); + assert(deep.length > opened.length, + `the portal opens the symbol the clip places: ${JSON.stringify(deep)}`); + // Selecting something nested must not close the portal that revealed it. + await evaluate(`(() => { + const k = cljs.core.keyword, db = cljs.core.deref(re_frame.db.app_db); + const sel = cljs.core.get_in(db, [k('ui'), k('selection')]); + const path = cljs.core.nth(sel, 3); + re_frame.core.dispatch(cljs.core.vector( + k('arthur.events.ui/select'), + cljs.core.vector(k('node'), cljs.core.nth(sel, 1), cljs.core.nth(sel, 2), + cljs.core.conj(path, k('made-up-child'))))); + return true; + })()`); + await sleep(220); + assert.equal((await rowLabels()).filter(l => l.includes('instance')).length, 1, + 'a selection under the clip keeps its portal open'); + await twist(portalAt); + await twist(0); + + const before = await tabs(); + const tabChips = () => evaluate('document.querySelectorAll(".tabs .tab").length'); + const chipsBefore = await tabChips(); + await doubleClick('.tl-track .tl-cel'); + const after = await tabs(); + assert.equal(after.tabs.length, before.tabs.length + 1, + `double-clicking a clip opens the symbol it places, as the pool row does: ${JSON.stringify(after)}`); + assert(!before.tabs.includes(after.open) && after.tabs.includes(after.open), + `the opened symbol is the one in front: ${JSON.stringify(after)}`); + assert.equal(await tabChips(), chipsBefore + 1, + 'the opened symbol is drawn as one more tab'); + assert.equal(await evaluate('document.querySelectorAll("#app > *").length'), 1, + 'opening from the timeline leaves the editor standing: a stale node selection ' + + 'pointing into the symbol just left used to throw and unmount it'); assert.equal(errors.length, 0, JSON.stringify(errors)); - console.log('PASS: generic lanes preview, rename, move, place, trim, roll, and ripple clips'); + console.log('PASS: generic lanes preview, rename, move, place, open, trim, roll, and ripple clips'); } finally { if (ws?.readyState === WebSocket.OPEN) { ws.send(JSON.stringify({ id: 999999, method: 'Browser.close' })); diff --git a/static/arthur/app.css b/static/arthur/app.css index e608014..05db3d5 100644 --- a/static/arthur/app.css +++ b/static/arthur/app.css @@ -1271,3 +1271,28 @@ button.share-button:hover, button.share-button.on { filter: brightness(1.1); } letter-spacing: 0.04px; } .paint-overlay:focus { outline: none; } + +/* What a drag in flight would do, beside the pointer: a clip body moves in + time, and with shift it goes inside the clip under the pointer. Said next to + the cursor because that is where the eye already is mid-gesture. */ +.tl-hint { + position: fixed; + z-index: 60; + padding: 2px 6px; + border-radius: 3px; + background: var(--fg); + color: var(--bg); + font-size: 11px; + white-space: nowrap; + pointer-events: none; +} +.tl-hint.nesting { background: var(--sel); color: #fff; } + +/* The clip a shift-drag would nest into. Inset, so it reads as "into this" + rather than as a boundary between rows. */ +.tl-cel.nest-target { outline: 2px solid var(--sel); outline-offset: -2px; } + +/* A row inside a held clip: shown because the node is on screen for the whole + hold, dimmed because its own frames have no place on this ruler. */ +.tl-span.unmapped { opacity: 0.45; cursor: pointer; } +.tl-hint.refusing { background: var(--bad, #b00); color: #fff; }