Compare commits

..

128 commits

Author SHA1 Message Date
Your Name
ddef5c6bfd A node has a pivot
Rotation and scale are composed about `[:xform :pivot]`, a point in the
node's own coordinates:

    local = T(pos) · T(piv) · R · K · S · T(-piv)

Schema 7 deleted this field, on the argument that an anchor is a peg. The
algebra was right and the conclusion was not. The identity holds between a
pivot and a peg THAT ALREADY EXISTS; it says nothing about what a node
turns about when nobody has made one, and that default is what a person
meets. With no pivot in the composition, a turn about anything but the
node's own origin has to be paid for by solving `pos` per frame —
`gesture/about` — and that solution is an arc in the angle while `pos`
tweens along the chord. Right on the frame it is written, wrong on every
frame between two keys.

A drawing escaped it: `paint/centred` puts a shape's origin on the middle
of what it draws. A symbol instance cannot — its origin is its symbol's,
and a symbol is drawn on the stage, so its origin is the top-left corner
of the stage. Off the document this was reported on: a symbol's content
centred 161 px from its own origin, and one instance of it keyed rot 0→60
put the drawing where it was put on both keys and at (-88, 121) halfway
between, a stage and a half away. The advice on offer was "make a peg
first", for wanting to spin a drawing.

So a turn now writes `rot` and nothing else, always, and the pivot is held
exactly between two keys because the matrix is built about it on every
frame. The default, and the way back to it, are the parts the old anchor
was missing:

  - a node nobody has pivoted turns about the middle of what it draws,
    `pick/bounds-of` — the same bounds the selection box comes from
  - the first turn or scale writes that middle down, in the same edit,
    with the `pos` that holds the picture still (`gesture/with-pivot`)
  - `clip/place-symbol` stores the middle of what a symbol draws as the
    instance's pivot, so a drop spins in place from the start
  - ⌃/⌘-drag the cross on the stage to put the pivot anywhere, moving
    nothing — on any node now, not pegs alone
  - ⌖ beside the pivot row in the inspector puts it back on the middle of
    what the node draws NOW (`gesture/centred`)

A pivot is a CHOICE and does not follow the drawing: once it is the node's
own, adding a shape inside a symbol cannot re-aim a keyed spin of any
instance of it. `instance-test` has asserted both answers to that now, and
the stored one is right.

A peg stays a peg, for the three things a node's own pivot is not: a pivot
SHARED between nodes, a SECOND transform on one node, and a hand transform
over a measured one. `nest/repivot` is gone — a pivot inside the node's own
transform has nothing to correct in anybody else's `:pinv`, so the gesture
works on every node and is no longer refused on an animated one. A measured
node's pivot is authored like any other, so a traced mouth can be told
where to turn without a peg.

Schema 8, and the first version that converts rather than refusing: an
absent pivot reads as [0 0] and T(pos)·T(0)·M·T(-0) is T(pos)·M to the
bit, so every stored document composes to exactly the matrices it did and
the migration only restamps the version.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-06 14:33:35 -04:00
Your Name
e7f5f82845 A drawing's origin is the middle of what it draws
A shape keyed from the bottom left to the top centre with a 360° turn on the way
left the stage completely in the middle of the spin and came back. The keys were
right and every frame between them was wrong, which is the signature of a wrong
pivot and a full turn: 0° and 360° are the only two frames where a wrong pivot
cannot be seen at all.

`paint/new-shape` stored a stroke EXACTLY AS DRAWN, in the containing symbol's
coordinates, and wrote no `pos`. So a drawing's origin was the SYMBOL's origin —
on the stage, its top-left corner. `node/local!` turns and scales about the
node's own origin and nothing else, deliberately, since `[:xform :anchor]` was
deleted in 925c12f. A node whose origin is nowhere near its content therefore
turns about nowhere near its content: the reported shape orbited at a radius of
126 px on a 320x200 stage.

It could not be seen while a drag was the only way to turn something, because
`gesture/about` solves for the `pos` that holds the chosen pivot still and
`turn` wrote it alongside the rotation — exactly right on the frame of the drag.
But that solution is `p' = c + R(θ)(p − c)`, an ARC, and `pos` interpolates along
the CHORD. Right on a drag, right on a key, wrong on every frame between two.

So `paint/centred` splits a stroke into a ring about its own middle and the `pos`
that puts it back, and `new-shape` is the one place every drawing is born — the
pen, the brush, and each piece the eraser leaves. The pivot rule is unchanged,
the middle of what the node draws; for a drawing that point is now its ORIGIN, so
`gesture/at-origin?` holds, `turn` writes `rot` alone, `scale` writes `scale`
alone, and a keyed turn is right on every frame. `pos` goes back to being the
motion path it reads as. Hand-authored scenes were always written this way:
`demo/scene.edn`'s card is `[-44 -30 44 -30 44 30 -44 30]` with its place in
`pos`.

NOT the universal rule, and `a-face-part-scales-about-its-own-middle` is why. A
measured part's points and position are dense tier-2 geometry in the footage's
space and cannot be re-originated, so its pivot is not its origin and `about` is
the only thing that will hold it; the same is true of an instance, whose origin
IS its symbol's coordinate system. Both still drag correctly about their middle,
both are inexact if that drag is keyed, and for both a pivot that has to persist
or be keyed is a peg — which is a node, so its pivot is its own origin, so it
collapses again one level up.

`cut/erase` took EVERY leftover piece back through `world⁻¹`, the cut shape's own
coordinates, and handed the offcuts to `new-shape`, which gives them a fresh
identity transform. Those two spaces coincide only while a shape has `pos [0 0]`,
which was every shape, so erasing anything that had been moved already scattered
its offcuts, silently. The kept piece comes back through `world⁻¹` and the new
ones through `parent⁻¹`, the space a node's `pos` lives in.

And `::adjust-last` re-traces the same stroke from stage pixels, so it goes
through `paint/place-points` rather than writing symbol-space points into a node
that now has a position of its own.

No schema change: the same fields, better values. An existing document keeps
evaluating exactly as it does now.

`a-keyed-turn-holds-its-pivot-between-its-keys` checks all 31 frames of the
tween. Checking the keys is what let this through. 604 CLJS tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-06 10:37:31 -04:00
Your Name
2e021a13eb Give a remap shape something to drag it by
A remap draws light rather than colour, so `pick/hit-op` let a click go THROUGH
it to what it lights, and `op-bounds` left it out of a marquee's reach. Both of
those rules are about getting PAST a shape, and neither has anything to say
about the shape you have in your hands — but the stage's move gesture IS that
hit-test, while `handles` hangs its box and corners off `pick/bounds-of`, which
reads the node's points and knows nothing about colour.

So a remap shape got a full set of handles over a shape that answered no point
on the stage at all. The first press on it started a marquee, the marquee
selected nothing, and the selection a timeline row had just given it was thrown
away: selectable in the timeline, and then neither movable nor resizable, with
the handles sitting right there. A knockout was the same bug for the same
reason, and both of them read as handles that do not work.

`hit-op` now takes what is selected, and what is selected is never
click-through. That is `choose`'s own rule — a click inside what is selected
keeps it, so a deep selection can be dragged — reaching one step further down;
clicking a remap you have NOT selected still goes through to what it lights,
which is the point of it. `op-bounds` keeps a hole's and a light's bounds like
anything else, so a marquee reaches one without going to the timeline for it.

`browser/remap.mjs` asserts it as `peg.mjs` does, with real pointer events
through the handles, and for the same reason: every assertion about the maths
passed throughout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-05 22:22:52 -04:00
Your Name
39c436584b Place a peg's pivot with ⌃/⌘ rather than ⌥
⌥-drag does not survive the trip. Most Linux window managers grab Alt-drag to
move the window, so the page never sees the pointer and the gesture is simply
absent — on the machine it is absent from, with no error and nothing to find.
Reported from one.

⌃ (⌘ on a Mac) now places the pivot, which is the modifier the rest of this
stage already reaches for. ⌥ keeps working for anyone whose desktop leaves it
alone; ⇧ is deliberately not it, since it means CONSTRAIN everywhere else here —
uniform scale, 15° turn steps — and is what a snap to the child's corners will
want when this drag grows one.

The menu row says so, because a modifier nothing mentions is a modifier nobody
finds: "add peg · ⌃/⌘-drag its cross to place the pivot".

`peg.mjs` now asserts each modifier through real pointer events instead of
reading the handler, which is the only way this class of bug shows up — all
three place the pivot with a child drift of 0, and an unmodified drag still
translates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HkinzDz1VtahZVsujAGRBD
2026-10-05 21:48:59 -04:00
Your Name
357ff1f3b9 Move a peg's pivot without moving what hangs off it
Two bugs in the handles, and the second one meant a peg still had no movable
pivot — which is the complaint pegs exist to answer.

`.peg-grab` was drawn LAST, on top of the four scale corners, and at r=7 against
corners spanning 5.2 to 8.8 from the middle it swallowed the inner half of each
one. It is drawn first now, at r=5, so it stops short of them.

And dragging a peg's cross is a TRANSLATE, not a new pivot. It writes the peg's
`pos`, and a peg is a parent, so everything under it comes along — `peg.mjs` said
so all along: "dragging the peg cross moved its child — dx=18.000". That is the
right behaviour for the drag and no way to say "put the pivot here", so a peg
could be made with its pivot wherever `nest/peg` happened to put it and never
moved again.

⌥-drag now relocates it, which is After Effects' pan-behind split. `nest/repivot`
moves the peg and solves

    pinv(child)' = local'⁻¹ · local · pinv(child)

for each child, because only `local(peg) · pinv(child)` reaches a child, so
preserving that product holds the picture exactly still — verified as a drift of
0, both in `nest-test` and through real ⌥-pointer events in `peg.mjs`. The
child's own channels are never touched, so this works over a measured child,
which is the case the peg exists for.

Refused on a peg whose position is animated, rather than quietly wrong: the
compensation depends on the peg's own transform, so a keyed position wants a
different `pinv` on every frame and one stored matrix is not it. The message says
to put a peg over it instead.

One edit at the end of the drag, not per pointermove: a repivot moves nothing on
screen by construction, so only the cross needs to follow the pointer.

603 CLJS tests, and peg.mjs and onion.mjs pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HkinzDz1VtahZVsujAGRBD
2026-10-05 19:25:04 -04:00
Your Name
00b8ed34ee Give a peg handles on the stage
A peg was reachable and not usable. It is a `:group`, so it draws nothing, and
both of the stage's ways in key off a DRAWN op: `pick/choose` hit-tests the ops,
so a click never found it, and `handles` hangs its box, corners and turn knob off
`pick/bounds-of`, so it got only the pivot cross — which is `pointer-events:
none`. So the pivot `gesture/refusal` tells you to make could be made, and then
only typed at in the inspector, and reselected only from a timeline row.

Every assertion about the maths passed throughout, which is the point: nothing
under `domain/` can see this.

So the handles fall back to a fixed-size rosette about the peg's own origin —
cross to move, knob to turn, four corners to scale. Fixed size, in stage pixels,
because there is no drawing for it to be proportional to. The cross gets a
transparent `.peg-grab` disc rather than taking the handler itself, since
`.pivot` must stay `pointer-events: none` everywhere else: it is a mark on the
picture and must not eat a click meant for the shape under it.

`test/browser/peg.mjs` asserts the handles as DOM and drags through them with
real pointer events, because that is the half a unit test cannot reach: a peg
appears with no drift at all, is selected, renders one knob, four corners and no
box, and dragging its cross moves its child by exactly the drag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HkinzDz1VtahZVsujAGRBD
2026-10-05 19:17:59 -04:00
Your Name
925c12fc77 An anchor is a peg
`[:xform :anchor]` is deleted. `T(a)·M·T(-a)` is a transform conjugated by a
translation — "do M in a frame shifted by a" — and a parent already IS a shifted
frame, so an anchor was a peg written inline: one that could not be selected,
keyed, shared between nodes, or placed above a measured channel. Same expressive
content, strictly less reach. `node-test` asserts the two produce the same matrix.

Its two jobs split, and neither is a field on a node any more.

A pivot nobody chose is DERIVED PER DRAG and stored nowhere. `gesture/pivot` is
the middle of what the node draws — `pick/bounds-of`, the same call the stage
draws the selection box from, on the same frame — or the node's own origin when it
draws nothing. `gesture/about` solves for the position that holds that point
still, so a turn now writes `pos` as well as `rot`. Nothing is cached, so nothing
goes stale: the stored anchor was that same middle captured once at creation while
the box beside it was recomputed every render, so on anything edited since it was
made the cross and the box disagreed and the pivot was wrong. A symbol with more
than one node diverged on its first edit.

A pivot somebody chose is a PEG — `nest/peg`, an ordinary `:group` parent sitting
on the derived pivot with `:pinv` captured so nothing moves. It is the answer to
the three things a derived pivot cannot do: a pivot that persists (an arm about
its shoulder), a pivot that travels (a keyed `pos`), and a hand transform over a
measured one. The last was impossible before — `local`'s translation is
`pos − M·a`, so under a measured `M` writing an anchor moves the thing it was
meant to leave alone. `flow/freeze`'s `pivoted` pass knew this and skipped every
`node/measured?` node, which is exactly why the traced mouth pivoted about
(-234, -395) on a 320x200 stage: the top-left corner of the footage. That pass is
gone; there is no node a derived pivot can be missing from.

`demo/stage` is the one place the anchor did work a static `pos` cannot: `:scale`
is keyed, and the source's middle has to stay on its authored centre throughout.
It is now seven pegs, identical to the pixel.

Also: `events/ui`'s `fitted` rescales a dropped tracing right after placement, and
the anchor had been silently keeping the picture centred through that; it solves
for the middle explicitly now.

Schema 7. Nothing is converted, as in 6: every project is marked 7 and one still
carrying an anchor is refused by name, with what to do about it. Dropping an
anchor is pixel-exact wherever rotation and scale are the identity — everywhere a
freeze or a drop wrote one — but not on anything since turned by hand, and not at
all where `pos` is dense, so a conversion would be silent and wrong for exactly
the nodes somebody had placed themselves.

601 CLJS tests, 68 Django tests, and the onion and take browser suites pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HkinzDz1VtahZVsujAGRBD
2026-10-05 19:12:16 -04:00
Your Name
a6b6c116c6 Count the upload in the status line
The transfer is the longest part of an import on anything but a local
server, and it was the only part with no number on it: `uploading video…`
sat unchanged from the first byte to the last, however many there were, and
only the extraction that followed it ever counted. Most of what "slow"
means to whoever is waiting was a spinner of unknown length.

`fetch` cannot report this. A Request built from a FormData gives no way to
observe its own upload — the promise settles when the response arrives — so
`POST-form` gained an XMLHttpRequest arity, which is the one thing XHR can
still do that fetch cannot. `fail` became `failure`, building the ex-info
rather than throwing it, because the two transports raise it differently:
fetch throws inside a `.then` and XHR has to reject by hand.

`sending` dispatches only when the whole percentage moves, since the
browser fires progress as often as it pleases and each dispatch re-renders
the pane. Video, sound and image uploads all report.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-04 23:22:58 -04:00
Your Name
e26ad723fa Measure the frame rate rather than believing the container
`probe` took `r_frame_rate` whenever it was at or under the cap, on the
grounds that it is the rate that keeps every distinct source frame. It is
not a claim about frames at all: ordinary iPhone footage declares 120 over
a stream whose timestamps are 1/30s apart, and resampling it up turned an
11-second clip into 1293 proxy frames instead of 323 — four times the
encode, four times the tracing stills (91MB against 23MB), four times the
blobs and the rows, for 970 frames that are copies of their neighbours.

So `_measured_rate` reads the timestamps and `_choose_rate` keeps whichever
declared rate they bear out. Two details carry it: the times are sorted
before differencing, because an HEVC stream arrives in decode order and
differencing that measures the reordering delay instead of the rate; and
the statistic is the MEDIAN interval, which is what keeps the property the
nominal rate was being taken for — a take held on one frame still reports
the rate of the parts that move, so no distinct frame is dropped. A genuine
120fps capture still extracts at 120, and there is a test on that
specifically.

`Source.probe` also stopped being the place a reading goes to be preserved.
The facts are a pure function of bytes that are the row's own identity, so
a re-upload re-reads them: otherwise every already-uploaded source would
have gone on resampling to four times the frames with no way to correct it
short of deleting the row.

Already-extracted footage is untouched — `extraction_key` still says
scheme 3, so those jobs stay done and reachable. Bumping it re-extracts
everything at the corrected rate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-04 23:22:50 -04:00
Olive Vaughn
8d20097e61 Repaint the stage after edits that move the selection; onion any symbol
The player's tracker derefed its inputs one by one inside `ratom/run!`. An
edit that also changed the selection ran it from the selection's change: it
read `::render/shown` while still stale, then pulled the dirty
`::render/clip`, which re-ran it nested with the new resolver — and the outer
run, finishing last, wrote the old one back. A deleted layer stayed on the
stage until something else ran the tracker. The snapshot is now one sub, so
the tracker derefs a single clean input and is never re-entered.

Onion skinning only ghosted cels in `:display :lane` symbols, so a drawing
made of keyed shapes showed nothing. A ghost is now just the picture the
resolver draws n frames back and ahead, limited to the selection when there
is one. The scope setting is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:35:04 -04:00
Olive Vaughn
78fb120edd ikd mn 2026-10-04 18:14:05 -04:00
Olive Vaughn
208dddee07 Put the top bar back to words
The icon run in the top bar read worse than what it replaced. Restore the
earlier top bar — pane words, new, open ▾, undo ▾ redo, the edit menu,
snapshots ▾, title, status, export… — and its CSS. The stage bar's overlay
switches, the palette chooser and `menu/popover` stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 03:16:09 -04:00
Olive Vaughn
f238ff4f62 Put the top bar on the surface as icons, and give palettes a real chooser
Top bar: every document command is an icon button in a run — panes, new/open,
undo/history/redo, cut/copy/paste/duplicate, snapshots — with the title and
status in the middle and export, share and account on the right. Pane toggles
are pictures of the window with that pane filled in.

Stage bar: tracing, onion and passepartout are all the same toggle-plus-settings
split button, followed by one zoom group, with fit inside it.

Popovers: one `menu/popover`, measured against its button and dismissed by a
scrim, replaces the <details> popouts that never closed on an outside click.

Palette: the strip ends in a palette button whose popover lists every palette
with its swatch strip, renames the open one in place, and marks the ones on
the stage. An info badge says when the palette being edited is not one the
stage draws in at the playhead; both ends of a palette-lane blend count.
`clip/palette-at` is the resolver's root palette lookup, pulled out so the
strip asks the same question the renderer answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 03:08:21 -04:00
Olive Vaughn
eca1a96b82 correction things 2026-10-04 02:47:29 -04:00
Olive Vaughn
064a3d7c19 Cover large timeline keyframe selections in the browser suite 2026-10-04 02:14:02 -04:00
Olive Vaughn
6de71c4c66 Align stage multi-selection modifiers with the timeline 2026-10-04 02:12:06 -04:00
Olive Vaughn
9bc406f413 Show project settings only when the root is selected 2026-10-04 02:12:06 -04:00
Olive Vaughn
1c520e6a68 Add instance playback controls to the inspector 2026-10-04 02:12:06 -04:00
Olive Vaughn
52f25e0bb1 Fix inspector automation clock for looped symbol instances 2026-10-04 02:11:17 -04:00
Olive Vaughn
4d441ae606 Keep overview keyframes passive and edit expanded automation lanes 2026-10-04 02:05:22 -04:00
Olive Vaughn
606055382b Add per-symbol onion skinning controls and cel ghosts 2026-10-04 02:04:00 -04:00
Olive Vaughn
f2fc261221 Fix stage zoom anchoring and continuous passepartout shading 2026-10-04 00:59:08 -04:00
Olive Vaughn
e5b61bfcdd Fix playback stopping when fallback audio ends 2026-10-04 00:58:51 -04:00
Olive Vaughn
ee603a351b Simplify stage passepartout control 2026-10-04 00:49:28 -04:00
Olive Vaughn
981bef98f9 Start playback at the playhead 2026-10-04 00:48:43 -04:00
Olive Vaughn
6df315b73d Pan with trackpad scrolling and zoom with pinch gestures 2026-10-04 00:44:47 -04:00
Olive Vaughn
484b4f1698 Move palette controls to sidebar and expand stage navigation 2026-10-04 00:40:20 -04:00
Olive Vaughn
1ae49a4015 Group tracing layers in the pool and fit new drops to the stage 2026-10-04 00:16:03 -04:00
Olive Vaughn
f4dd047642 Merge master drawing tools with tracing layers 2026-10-04 00:05:16 -04:00
Olive Vaughn
17e1b4f403 Represent tracing media as symbols and add pool thumbnails 2026-10-04 00:02:12 -04:00
Olive Vaughn
353cb6e050 Add remap ink, eraser cuts, and improved brush geometry 2026-10-04 00:00:09 -04:00
Olive Vaughn
1f0b4d9918 Draw with tools: a pen, a brush that makes polygons, and knockouts
One tool at a time, picked from a strip down the left of the stage or by its
letter, as in Photoshop, Illustrator and Flash: V selects and transforms, P is
the pen, B the brush, E the eraser. The options bar above the stage holds the
tool's settings, and the palette is a grid under the tools with the colour a
new shape gets above it, as Deluxe Paint's was.

The pen's draft is filled into the picture as it is drawn, so the edge on
screen is the pixels the shape will be. Click the first point, Enter, Esc or
leaving the pen closes it; the pen stays the tool. Editing a shape's points is
the pen ON that shape — Figma's vector edit mode — so the separate points mode
is gone: click an edge to add a point, ⌥-click one to delete it, on every key
at once so tweens keep meaning something.

The brush paints a mask, and what it will be is shown while painting: the
stroke is traced round its pixel edges, holes and all, and simplified to a
point every so many pixels of outline by the same function the saved shapes
come from. A hole is bridged into the one ring along a whole-pixel row, which
the fill never samples. Blender's Adjust Last Operation re-traces the last
stroke from what it was made of.

A shape coloured CLEAR is a knockout: its symbol is drawn into a layer of its
own and the knockout clears it, every colour or one. The eraser makes one in
the symbol it starts on, of the colour it starts on, or of every colour with
⌥. A click goes through a knockout to what shows. Previews are drawn in the
stacking context of what they will land in.

The polygon's points no longer stay behind when the shape is moved: they were
read off the saved document while the box handles read the one mid-drag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 22:43:28 -04:00
Your Name
551d572347 idk man something hopefully helpful 2026-10-03 04:48:03 -04:00
Your Name
5b5b9ae4c3 Play the audio mixdown instead of encoding a WAV of it
Opening the 8625 study froze the main thread for 4.7 seconds and settled at
615MB of heap. A profile put three quarters of a project open inside
mix/wav-bytes, which playback had no business calling at all.

Two separate causes. The peak scan built a lazy sequence of one boxed double
per SAMPLE -- ten million of them for a seven-minute mix -- to compute a
single maximum over data already sitting in Float32Arrays; the hand-written
loop is 73x faster and agrees to the bit. The rest was structural: the WAV
existed only because an <audio> element can hold a URL and nothing else, and
the element existed only to be the clock. So a mixdown that was already
rendered got encoded to 73MB of 16-bit PCM, on the main thread, on open, on
every tab switch and on every edit to a track -- and a symbol with no sound
got silence synthesized and encoded full length so the element had a duration
to report.

arthur.clock keeps its interface and all of its arithmetic; the position now
comes from a backend behind a protocol. clock.graph plays the AudioBuffer
through an AudioBufferSourceNode and derives the frame from the context's own
clock, which is the audio device's position in double precision rather than
whatever the media pipeline last published. clock.element is the old path,
kept switchable while the new one earns trust -- BACKEND, or use-backend! --
which is also why every one of the original clock tests passes unchanged: the
derivation they assert is shared, and the backends can only disagree about the
position under it. 6.5s to 1.8s, 4.7s of blocking to 370ms, 615MB to 68MB.

THE POSITION IS COMPENSATED FOR OUTPUT LATENCY, and piecewise because of it.
currentTime is the quantum being rendered, which the speaker is tens of
milliseconds behind; report the renderer and the picture leads the sound,
which in a lip-sync tool is the only artefact that matters. Audio already
rendered cannot be re-rated, though, so reading it back at a new rate jumped
the playhead backwards by three latencies on every press of the rate button.
Each play, pause, seek and rate change now records a segment and a position is
read against whichever segment was in force when that audio was rendered.

One duplicate fell out of this. Opening a project asked for its clock twice --
once from ::opened and once from a ::refresh-clock the shell raised because it
compared symbol ids, and two different documents both open on :main. The
sounds subscription carries the clip id now, so "an edit under the same
symbol" means what it says.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-03 04:47:35 -04:00
Your Name
90b1fbe2f8 Create in a lane at the frame double-clicked, palette lanes included
The gesture was handed the pointer's frame and threw it away, creating at
the playhead instead; and the palette row carried :lane? only once its
track existed, so the first double-click on a palette lane -- the one that
has to make the track -- dispatched nothing at all.

Both are one rule now. ::new-symbol-at uses the frame it is given and
resolves every row through drop-destination, with the destination deciding
what is created: a clip of a palette track is a palette symbol, a clip of
any other lane is blank. A row with no path of its own resolves to the
symbol it names, which is what lets a palette track -- hanging off its
owner by :palette-track rather than placed in it -- be reached without a
special case; it also stops a palette cel's slide resolving against the
open symbol and looking like a transfer out of the track. The one thing
left that knows about palettes is materializing the lane a palette row
names before anything asks where the row leads.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-03 03:52:39 -04:00
Your Name
09b74de162 Duplicate a palette from the palette bar
Making a variant meant 16 colour pickers from black: the only button was
"+", which seeds an untitled palette from the built-in defaults. The copy
is selected on creation, so the next colour edit lands on it rather than
on the palette it came from.

`pal/palettes` is the source, so the implicit default — a project that
has never had a palette asset of its own — duplicates like any other.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-03 03:51:17 -04:00
Your Name
2460dce0a5 Refuse a symbol placed inside itself, and label instances by what they place
A paste put an instance of "bg" inside "bg" itself. It saved, loaded, and then
threw "symbol cycle in audio" out of nest/audio-tracks, because a container that
contains itself has no finite expansion.

place-symbol and ui/drag already refused that, each by asking
clip/contains-symbol?. Paste asks clip/problems instead, and problems did not
encode the invariant at all -- it checked missing symbols, pose tracks and audio
links, but never the placement graph. So the rule goes where every command is
already checked: paste, cut, duplicate, correction and the span ops all gate on
problems, so one rule covers them all.

Why it looked like a reasonable thing to do is the other half. place-symbol
copied the symbol's name onto the instance it made, and that copy went stale on
the next rename: the symbol read "bg" in its tab while an instance of it still
read "symbol-18", which is its id from before it was named. One object under two
names, with nothing on screen to connect them.

So instances are no longer given a name at creation, and clip/node-label reads
the symbol's name through on every render. :name on an instance now means only
what a person typed, which is what tells two instances of one symbol apart --
"8625 left" and "8625 right" of one "face" -- so an authored name still wins and
read-through is the fallback. The label logic was duplicated across four call
sites with three different fallback orders; location.cljs already read through
and the others did not. They now share one function.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-03 03:43:07 -04:00
Your Name
0d49db793b fix some stuff 2026-10-03 02:23:54 -04:00
Your Name
a90e0cfb61 fixed creating in 2026-10-03 02:15:27 -04:00
Your Name
2697401aad Scope inspector controls and key shape colors 2026-10-03 01:48:26 -04:00
Your Name
a834ccb1e2 Unify lane symbol creation and palette automation 2026-10-03 01:35:33 -04:00
Your Name
15deaea19e Support multiple regeneratable footage analyses 2026-10-03 01:24:45 -04:00
Your Name
5bcf22e458 perfect target area thing 2026-10-03 00:39:07 -04:00
Your Name
664252e0fc idk 2026-10-02 17:44:15 -04:00
Your Name
edca82cd5d Add multi-selection clipboard commands 2026-10-02 16:13:03 -04:00
Your Name
987e289f89 Enable browser source maps 2026-10-02 16:11:32 -04:00
Your Name
7c49c08bc7 Fix palette swap backgrounds and timeline drop jitter 2026-10-02 10:24:23 -04:00
Your Name
26fab9392e Model palette tracks as typed symbol lanes 2026-10-02 10:06:05 -04:00
Your Name
dff23d7994 Add multi-object stage selection and transforms 2026-10-02 09:09:18 -04:00
Your Name
b41180db08 Add project palette assets and overrides 2026-10-02 09:08:53 -04:00
Your Name
8f09b7b47f Unify lane overlap and nested timing 2026-10-02 09:06:28 -04:00
Your Name
1b2b4ad3d2 Unify root timing and persistent lane targets 2026-10-02 01:11:55 -04:00
Your Name
f5a39aee39 Keep footage audio with generated face symbols 2026-10-01 23:38:16 -04:00
Your Name
0cb9d0ebf6 Unify timeline placement and gesture geometry 2026-10-01 23:35:22 -04:00
Your Name
4e5e02c856 The window's shape is editor state, not a constant
The grid was five panes at three fixed widths and one fixed height, written as
lengths in the stylesheet. That is a claim about every screen the tool is ever
opened on, and it was wrong on two of them: a phone, where 210 + 250 pixels of
side pane leave nothing for a 320x200 stage, and a large display, where the
inspector is the width somebody once typed rather than the width their work
wants.

So the tracks are custom properties and `ui/layout` owns the numbers. A drag of
an edge is one `assoc-in` and one property on `.app`; no pane's contents
re-render, which is the same reason the picture is painted by `ui/player`'s loop
rather than by anything reactive. Sizes stay in PIXELS and not fractions —
three of these panes hold fixed-width rows, so what you drag an edge to is the
width you meant at any window size.

State, events and widgets are one namespace, which is not this repo's shape
anywhere else. They are the same six integers: `:pool` is a width, a grid track,
a drag's clamp and a button's label at once, and splitting six integers across
`db`, `events/` and `ui/` is more wiring than state.

Three things follow from making it state at all:

SHUTTING A PANE. Pool and inspector unmount; the timeline collapses to its
transport strip instead, because play, pause and the frame readout live in that
strip and a window with nowhere to press play is broken rather than small. The
toggles are in the top bar because a shut pane has no head left to carry its
own handle.

A NARROW WINDOW. Below 760px the grid is one column and the side panes are
drawers over the stage rather than columns beside it. The breakpoint is read
once, at load, and only decides what the layout OPENS at: throwing away the
sizes somebody dragged because they turned a tablet sideways is worse than a
layout that is briefly the wrong shape.

ZOOM. The stage's was already a constant 2 in `ui/stage`; it is now an integer
in [1 8], still CSS over a canvas that is the raster's own size, so the browser
suite still reads 320x200 of real pixels off `canvas.stage`. Whole pixels
only — a fractional scale under `image-rendering: pixelated` draws some rows of
the raster thicker than others, which misrepresents the one thing the preview
exists to judge. The timeline's is the stylesheet's entirely: every mark in the
tracks is positioned as a percentage of their width, so one multiplier on that
width spreads the grid, the spans, the keys, the ruler and the playhead apart
together. Both readouts are the button back to normal, accented while there is
something to return from, because the quick way back belongs in the one place
you already look to find out where you are.

One trap, paid for and then found: the top strip has to scroll sideways on a
narrow screen, and `overflow-x: auto` makes an element a clipping container on
BOTH axes. Every top-bar menu — open, snapshots, share, sign in — was then cut
off by a 30px-tall bar and drawn at whatever x the strip happened to be
scrolled to, which reads as the button doing nothing at all. They are fixed to
the viewport there, as `.menu-drop` already was one pane down for the same
reason. The same shape of bug ate the polygon tool and both zoom readouts: a
flex row that does not fit shrinks every item and pushes the last ones off the
end, so nothing in a strip shrinks now and the strip scrolls as a strip.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 21:50:05 -04:00
Your Name
93f5112bb3 Unify clip placement across symbol modes 2026-10-01 20:09:35 -04:00
Your Name
e459307a4a Make lanes explicit symbol views 2026-10-01 19:40:53 -04:00
Your Name
fb38990090 Loop is two things, and the one that matters is unreachable
Loop playback — the transport repeating the open symbol — is editor state and
the document has never heard of it. A looping INSTANCE is a node repeating the
symbol it places, which is the four-frame tire turning for the hundred and
twenty frames it is on screen, and it lives in the document as
`:playback {:end :loop}`. Same word, two scopes; named apart here before
anything is built on either.

The status of the second one is the surprise: it already works and cannot be
asked for. `node/placed-frame` does the modulo, `node/problems` already admits
`:end :loop`, `audio-tracks` already expands the periods — and no control sets
it anywhere in the UI. So the tire is three pieces of work, not one: somewhere
to edit an instance's playback, a block that draws its repeats rather than
only its first pass as the timeline's own docstring admits it does, and the
audio period guard.

This also pins down when the held cel can be torn out. Turning every drawing
into a one-frame looping instance is only safe once a looping instance can be
seen and edited; otherwise every drawing in the document quietly acquires a
property with no control on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 16:32:02 -04:00
Your Name
d029908f0b A held cel is a one-frame symbol that loops
Holding a drawing and looping a one-frame symbol are the same picture, and the
second is a case the model already carries, so the first is a special case
kept for nothing. Speed 0 goes; a cel becomes a one-frame symbol with
`:end :loop` and a span. Looping and span are already properties of the
instance and `:frames` already belongs to the symbol, so nothing moves — a
mode is deleted. Two spellings of looping collapse to one, `:time :loop?`
giving way to `:playback :end :loop`, and `extend-hold` collapses into
`resize-out`, since it exists only to refuse anything that is not frozen
before editing a span.

What it buys is one rule where there were three refusals. `nest/inside` has no
invertible clock for a hold, for `:end :hold`, or for a loop, so
shift-to-reparent refuses all three — which is most of what anybody would drag
onto. Resolving the move with the destination's map at the CURRENT frame
covers every one: a loop is affine within the period the frame falls in, a
one-frame loop is that rule with a period of one — which lands exactly where
the nesting notes argued it should from first principles — and `:end :hold` is
affine in the played part and frozen in the tail.

The trap is written down beside it, because it would be found the hard way:
`audio-tracks` expands a loop into one walk per period, and is saved today
only by the `(pos? speed)` guard that a held cel fails. Make every drawing a
loop and a drawing held for 120 frames becomes 120 walks emitting any nested
sound 120 times. An audibility precheck has to land in the same change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 16:31:06 -04:00
Your Name
6e7827e03f Non-overlap is an invariant, not a report
The plan asked what to draw when a symbol in lane mode holds overlapping
children, and offered to report it as a decision waiting for a person. Wrong
question: they never overlap. Placement claims time — anything placed, moved
or grown over occupied time trims, removes or splits what it lands on — so the
operation that could have made an overlap did not, and `span/finish`, which is
already the single commit path and already refuses rather than half-applying,
is where that is enforced. An overlap is then a bug in a command and not a
state to design around.

The check stays, named `symbol/overlaps` and used three ways: the commit path
refuses one, a property test asserts no command can produce one, and a
document that somehow holds one still LOADS and is drawn visibly wrong with
the status saying so. Not `problems`, which stops a document loading, and not
`conflicts`, which means somebody has a decision to make — a display hint must
never be able to keep a document from opening.

The one place a person can ask for the impossible is toggling lane mode on
over children that already overlap. That refuses and offers to trim them into
a sequence, through the `:required-frames` retry the model already uses.

Also written down, because it is the pair the modifier exists to separate:
a plain body drag is temporal and replaces, trimming extents as needed; shift
is structural and goes through `nest/move-node` into the symbol under the
pointer, which has to keep working for symbols held in a lane.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 16:22:33 -04:00
Your Name
10b96761fa Plan: a lane is a view, not a thing in the document
The lane model put a second container in the node map — a group with
`:layout :sequence`, with its own membership, its own validation and its own
fourteen commands — and every part of the editor then had to ask which kind of
container it was looking at. The decision recorded here is to delete all of
it: a lane becomes a way of DRAWING a symbol whose children are sequential and
non-overlapping, the display goes back to a row per symbol, and the word
survives only in the timeline and in the drag handling that re-spans a
symbol's children while it is drawn that way.

The commands are not the part being thrown away. `extend-hold`, `resize-out`,
`roll`, `blank` and the rest are what endpoint dragging IS, and their
arithmetic is right; what changes is their subject, from "the children of lane
L in symbol S" to "the children of symbol S". They belong in `span.cljs`,
which already owns re-spanning and `finish`.

The plan takes a position on the one question that decides whether this is a
simplification or a circle: lane mode is a saved hint on the symbol rather
than unsaved view state, because the drag rules follow the mode, and a toggle
the document does not record would make one gesture do two different things
to it. Nothing outside the timeline may read the hint.

It also lists what must not be lost on the way, all of which broke at least
once today: a held clip's contents reachable with no keys and no draggable
edges, double-click to open surviving the selection it leaves behind,
selection waiting for pointer-up, no drop silently deleting what it lands on,
and a sound drawn once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 16:16:30 -04:00
Your Name
2dc5735ded The timeline opens the whole document
Making a lane one row cost the thing a row was for. A clip stopped being a row,
so there was no longer any way to open a clip and see what was inside it, and
the inside of a drawing — the most ordinary thing in the document — became
reachable only by opening it as its own tab. This is that capability back,
from the root timeline, down as far as it goes.

An expanded lane opens exactly ONE clip: the selected one. Its own keys, then
the lanes and nodes of the symbol it places, then theirs, each mapped into this
ruler by the recursive walk that was already there. Twelve clips in a lane
still cost one row, and inspection costs one branch rather than twelve.

Two things that only showed up once it ran. The portal is chosen by the whole
LINEAGE of the selection and not by the selected id: selecting a shape inside
the clip — or the end of its span — is still working inside that clip, and
matching the id alone shut the portal the instant anything under it was
touched. And selecting now waits for the pointer to come UP, because selecting
on the way down re-drew the timeline before the gesture had said anything: it
shut the portal holding the lane being dragged INTO, out from under the
pointer.

A HELD clip opens too, which the old row walk never did either. `source-time`
is nil for a hold, so the walk stopped there and the contents of every drawing
were invisible from here. Its rows are shown across the hold — which is when
the node is on screen — and marked `:unmapped?`: no keys, and no draggable
edges, because a frozen clock gives no frame inside it a place on this ruler.
Refusing to place the keys is the honest half; refusing to show the rows was
not.

Double-clicking a clip opens the symbol it places as a tab, as double-clicking
the same symbol in the pool does. That was already written and had never once
run: the track captures the pointer for a slide, so the click and double-click
that follow are delivered to the track and never to the block. The track now
resolves them itself. Fixing the delivery exposed two more: `symbol/lineage`
reported a `parent cycle` for any id in a symbol with NO nodes, because a
one-element chain is longer than zero nodes — and opening a symbol left the
selection pointing into the symbol being left, which the breadcrumb and the
inspector then tried to resolve. The editor unmounted. Both are fixed where
they were wrong, and the browser test asserts the editor is still standing
afterwards.

Audio is a clip in a lane like everything else. A dropped sound lands in one
and is trimmed and moved by the same commands; a lane holds picture or sound
and not both, which is the explicit capability the model asked for rather than
a guess per frame. The refusal lives in the commands and not only in
validation, because placement claims time: `blank` would have deleted the
sound to make room for the picture and left a perfectly valid document behind.
What is in a lane of the open symbol is drawn as a lane; what is nested inside
a placed symbol is still flattened by `audio-tracks`, so no sound is on two
rows.

Everything that enters the timeline now enters a lane: a converted take, a
symbol brought in from another project, a sound. One rule answers where —
`lane-destination` — and every symbol is born with a lane for it to answer
with. An unaimed drop fills an EMPTY lane rather than taking an occupied one
nobody pointed at, because the alternative is trimming away what was there to
make room for what was dropped.

Shift during a clip-body drag means the other intention: put this node INSIDE
the symbol the clip under the pointer places, through `nest/move-node`, which
is what keeps the world transform and the root timing. Overlap cannot say
which of the two is meant — dropping on occupied time already means claiming
it — so the person says, and a label by the pointer says it back. The label
asks `nest/move-refusal`, the same check the command makes, so it cannot
promise what the drop would refuse. Today it refuses more than it allows:
both clips have to be on screen at one frame, which two clips in one lane
never are, and a held destination has no clock to move through at all.
`docs/lane-nesting-notes.md` argues that the second refusal is stronger than
the facts require and says what would settle it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 16:13:02 -04:00
Your Name
95451798d2 Nesting is a gesture of its own, not an overlap
Now that a lane is generic, the obvious next move is to read a clip dropped on
another clip as "put it inside that symbol". It cannot mean that: dropping a
clip on occupied lane time already means it claims that time and trims the
incumbent. Structural nesting therefore needs an explicit affordance -- a grab
handle with `grab`/`grabbing` cursors, distinct from the body's temporal move
and the edges' trims -- and its drop must route through `nest/move-node`, which
preserves world transform and root timing, rather than through a weaker
`:parent` assignment that would make a drawing jump when it is rehoused.

The second half of the note is what expansion should be. One permanently
expanded row per lane clip is the vertical growth the one-row lane exists to
avoid, so an expanded lane shows exactly one portal: the currently selected
clip, swapped in place when the selection changes, not following the playhead.
The portal header is the structural drop target, sub-expanding it walks the
source symbol's lanes through the existing recursive root-time mapping, and
collapsed clips carry their instance-level keys as ticks. The cost of
inspection stays constant.

No lane-as-symbol type and no second ownership edge: the hierarchy is still
symbol, lane, clip, source symbol, its nodes. Lane membership owns time; a
symbol instance owns composition. Written before the interaction is built,
because the capability it protects is easy to lose by accident.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 15:13:18 -04:00
Your Name
5ebe776ce4 A lane is a generic row of symbol clips
A lane was a drawing lane: the only thing that could go in one was a one-frame
held cel, and every other symbol instance stayed a permanent root row of its
own. Those are not two kinds of timing, they are one kind with two creation
policies. `lane/place-symbol` drops any library symbol in as a clip that plays
naturally at speed one, `lane/adopt` moves an instance that is already in the
document into a lane keeping its source, span, playback and corrections, and
`append-drawing`/`overwrite-drawing` keep being the policy that makes a new
empty symbol a one-frame hold. The child shape they produce is the same.

Both new commands claim their interval through `blank` before they write, so
the partition rule is unchanged and unduplicated: placing into occupied lane
time trims, removes or splits the incumbents, and a lane still never stores an
overlap. Real compositing overlap is another lane, where the order is explicit.

Creating a symbol with nothing aimed now makes a lane and a clip in it instead
of a loose root instance, and a pool drop prefers an explicitly targeted lane,
then the selected one, and makes a lane only when there is neither. That is
what stops the row-per-symbol growth coming back in through the drop path, and
it is why `add-lane` now takes a z in front of the existing root nodes and
calls what it makes a "lane" rather than "drawings".

The timeline learned the two gestures that a generic lane needs. A clip body
dragged over another lane's track previews there as a dashed block and lands
through `::adopt-in-lane`; the track is found with `elementsFromPoint` and its
selection read back off the element, because a pointer capture does not
retarget. A pool drop over an existing lane previews as a dashed clip inside
that lane instead of a temporary new row that appears and then vanishes --
which also needed the drag-leave check to be geometric, since inserting the
preview changes the element under the pointer and Chromium then reports a leave
with no related target. Lanes are renameable from their label, by double-click,
F2, or the pencil, through `::rename-node`.

`symbol/lane-cels` is `symbol/lane-clips`, and the vocabulary table in the
handoff now separates the two words it had merged: a clip is an instance in a
lane, and a cel is specifically the one-frame held source that drawing creation
makes. Keeping `cel` for the policy is what lets the lane stop being about
drawings at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 15:13:07 -04:00
Your Name
26ada03591 Unify cel editing in the timeline 2026-10-01 14:37:59 -04:00
Your Name
4ddd6a8d1d Suspend parameter automation during live control 2026-10-01 12:40:50 -04:00
Your Name
0a53157b7e Buffer auto-key performance takes in memory 2026-10-01 12:36:03 -04:00
Your Name
abefa1c452 Sample auto-key drags once per playback frame 2026-10-01 12:30:24 -04:00
Your Name
6e9409b0de Write auto-keys during transform drags 2026-10-01 12:25:41 -04:00
Your Name
34f62ab007 Add auto-key mode to main toolbar 2026-10-01 12:22:30 -04:00
Your Name
3879d76d57 The cel sheet creates the place a polygon needs 2026-10-01 12:17:35 -04:00
Your Name
6443366748 The snap reads back into the slot's own gap, per pose group
Step 2 of docs/frame-selection.md, which f7e16e5 planned and left unbuilt: the
preserve-snap, and nothing of the plate side. `pose/snapped-frame` is the whole
rule — the latest mark in `(lo, hi]`, and `hi` when there is none — where `hi`
is the native frame the slot defaults to and `lo` is the one the slot before it
defaulted to. So the interval is exactly the frames this slot is the first to
cover, which are exactly the ones the grid shows to nobody: it recovers a
dropped frame out of its own gap, can never read a frame another slot already
showed, and cannot reach past `hi`.

Backward only. A closure at native 13 in a 12-from-30 output is recovered by the
slot whose default is 15, reading 13 — not by the slot at 12 reaching forward,
which would show the mouth shut 17ms before it did and break the no-lead
invariant `cadence_test` asserts over every grid and native pair. That test now
covers the snap too.

Seated as the DEFAULT pose that `pose/source-frame` reaches, so an explicit hand
cut beats a snap with nothing having to say so, and per pose group rather than
at the slot: snapping where the grid becomes native is one frame for the whole
picture, so a head would go two frames stale to fix one mouth. Groups exist only
in `symbol/base-channel-frame`, which is why the interval is threaded that far
down — `(:pre parent)` carried beside `(:f parent)` through the same time maps,
so an ancestor's exposure fold or retime is already in it.

The marks are the `[:vis]` cuts `flow/freeze` already stores, with their
thresholds and hysteresis already decided: no new signal, no new stored field.
A whitelist of `:roto/mouth-aperture` and `:roto/blink` and not a test for
`:generated`, because a skipped frame, a hidden feature and an absent
measurement are three different facts — snapping onto the frames a teeth contour
happened to be missing on is the cadence being dragged about by an absence.

Off is the default and needs no second code path: an opts map that says nothing
gets the behaviour it got before the snap existed. `:snap` is asked about the
SYMBOL, because the cuts are the face's own nodes' and every placement of one
face has the same ones.

The switch is `performance · <face>` in the inspector, and the readout says it
is the stage only. The document setting docs/frame-selection.md specifies wants
a leaf and a round trip of its own; until then this is `[:ui :smart]`, not
undoable, not synced, and unable to reach an export.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 11:34:43 -04:00
Your Name
340a8dbbd6 The footage switch is the inspector's, and the open symbol's
Moves the tracing toggle and its opacity slider off the palette bar. The bar is
what you set before you draw — the tone and the tool — and whether a reference
photo is showing is not that; it is a viewing aid, like solo, and it belongs
with the other things the inspector says about what is on the stage.

Its own `footage` section rather than part of `tracing · <face>`, which is the
whole reason it could come back to the inspector at all. That section needs a
face to be about, so it is absent when a take is open and nothing is selected —
the commonest case there is — and a switch that appears only once the right row
has been found makes the way to see the footage you are tracing depend on what
you clicked, which is what put the thing on the bar in the first place. So the
section keys off `trace/traceable-faces` of the OPEN symbol: present whenever
the picture has any footage behind it, wherever the selection happens to be.

No change to the state or the events. `[:ui :trace]` is still a set of faces and
one opacity, `::trace-faces` still switches every face the open symbol has on
unless they all already are, and a face's own timeline row still singles it out.
The label loses the word "footage" because the section heading now says it.

`.trace-opacity` was 64px to survive a crowded flex bar and is `flex: 1` on a
pane row instead; `.palette-bar label.dim` had no remaining user.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 11:33:03 -04:00
Your Name
f7e16e5ef4 The snap is per group, and the slot interval has to reach it
Pulls the preserve-snap forward to step 2: it is the half with no UI and nothing
proposing today, and it needs nothing from the plate side but a fold that can
land last.

Makes the rule exact, because the direction is easy to get backwards and the
draft was vague about it. Slot k reads the latest preserved frame in
(d(k-1), d(k)], else d(k). The closure at native 13 in a 12-from-30 output is
recovered by slot 6, whose default is 15, reading 13 — NOT by slot 5 reaching
forward from 12, which would show it 17ms before the mouth shut and break the
invariant cadence_test already asserts.

Records where it goes, which is the part that was understated as "about thirty
lines". The slot interval exists only at clip.cljs:261 and group identity exists
only at symbol.cljs:399, so the interval has to be threaded down: two internal
signatures, not a drop-in. The rule's seat is the `(js/Math.floor lf)` default
`base-channel-frame` hands `pose/source-frame`, which leaves an explicit hand cut
beating a snap, as manual precedence requires.

And records the shortcut not to take: snapping at clip.cljs:261 needs no
threading and is wrong, because one native frame per output frame means the whole
picture reads 13 instead of 15 — a 67ms stale head to fix the mouth, fighting the
trace selection's own opinion about which head frame to show.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 10:21:35 -04:00
Your Name
cc42155ffa The plate side chooses frames; the performance side nudges the grid
Corrects this plan's central claim. It said one component used twice, because
the prototype's `suggestPlateFrames` and timing-handoff's performance poses
looked like the same function. They are not.

A plate selection has to be NON-UNIFORM — that is the entire reason it exists. A
head still for sixty frames and then whipping across in ten wants two drawings
and then eight, and no nudging of a uniform grid produces that distribution,
because the spacing itself is the answer. A tolerance belongs here: the artist is
buying drawings.

A performance selection is not choosing sparseness at all; the output rate or the
exposure setting already did. What is left is which native frame each decided
slot reads, so nudging the grid is the right size of answer and a tolerance would
be a knob with nothing to control.

And the harder reason, which settles it: a selection cannot put a frame on screen
that the output grid never samples. At 12fps out of 30 a closure at native 13
falls between output frames 5 and 6, so keeping 13 in a set makes it available
and never shows it — exactly what time.md says about an event between output
frames. Only moving what output frame 6 reads can show it.

So the performance half is a backward-only snap of the grid's own pick toward a
preserved frame, about thirty lines plus a prepare step, and its marks come from
cuts `flow/freeze` already stores: the mouth's aperture threshold and the eyes'
`resolve-blink` with its hysteresis. No aperture signal, no new dense track, no
threshold decided twice. The nesting survives unchanged, because a kept plate
frame is just another mark in the same pile.

Records why the aperture signal is impossible as drafted — positions 5 and 15 of
LIPS-INNER survive `freeze/rings->flat` only at verts divisible by four — and
names commit 02069e8 as the one holding the now-uncalled extrema detection, with
the two conditions under which it would be wanted again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 10:14:54 -04:00
Your Name
02069e88f0 A protected frame is on screen, so it anchors the walk
`domain/select`: the choosing half of a selection, pure and handed a plain
vector by each site. `pose/held-frame` was already the shared reading half;
nothing chose automatically at all.

The greedy walk rather than the DP, deliberately — it is what the prototype
shipped, and having two implementations is how the DP gets tested. Several tests
pin its exact output for that comparison and will need rewriting when it lands.

Protection is settled before the walk and is not unioned onto its result,
because a protected frame anchors the walk like any other kept frame. The anchor
is what is on screen, so measuring the next frame's drift from a frame that is
no longer displayed holds a one-frame closure across two frames and shows it
twice as long as it was measured. The test that says so is about the picture
rather than the algorithm: the frames the mouth was measured shut on and the
frames the selection shows it shut on are the same list.

A strict local extremum compares against the nearest DIFFERING samples, not the
immediate neighbours, and a run of equal samples is one extremum at the frame it
begins. Immediate neighbours find nothing at all on a closure lasting more than
one frame, which is most of them.

One width for the whole signal, so `distance` is never comparing the prefix two
samples happen to share; a non-finite tolerance falls back to nought, since NaN
compares false against everything and would quietly collapse a performance to a
single pose.

THE EXTREMA HALF HAS NO CALLER under the plan as it now stands, and this is the
commit to revert if it stays that way: `segments`, `turns`, the `:extrema`
branch of `propose` and the multi-component refusal that only guards it, plus
the three tests over them. Plate selections take no extrema by design, and the
performance half gets its closures from the `[:vis]` cut `flow/freeze` already
stores, so nothing asks this code for anything. It is correct and tested and
speculative; docs/frame-selection.md says so beside the build order.

Also corrects `ring/subsample-slots`, which claimed every even budget lands on
the cardinal positions. The corners do; the lip centres survive only multiples
of 4, so the aperture pair cannot be read off a subsampled ring at verts 6, 10,
14 or 18. That is why the performance signal reads a stored cut instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 10:13:00 -04:00
Your Name
14673385d4 Remove completed timing worktree 2026-10-01 09:11:57 -04:00
Your Name
95cf2598ba dockerignore 2026-10-01 01:53:04 -04:00
Your Name
05878ca48b grand unification of time 2026-10-01 01:47:08 -04:00
Your Name
a4ce750be2 The face owns its placement, not the take that holds it
The source-to-stage mapping moves off :main's :face group and onto each face's
own :place, above its head. `face-placement` computes exactly what it computed
before, over every subject together, so two faces filmed side by side keep
their filmed relation — it is written into each face instead of onto a group
above them all. Same transform, same subtree, one level lower, and the
composite is identical to the pixel: a digest over every op :main emits across
the whole take is unchanged either way.

THE OWNER IS THE POINT. A face carrying its own mapping is the right size
wherever it is put — dropped into another symbol, or opened in its own tab to
be drawn over — and the take that holds it needs to know nothing. On a group
above the instances the scale belonged to the take, so a face taken out of it
had no size at all and drew at a fraction of a pixel.

The pool's thumbnails drop the workaround that knew about this: a symbol is
rendered rooted at itself again, because a face now carries the placement that
makes that honest, so the pool needs to know nothing about where a symbol
happens to be used. `domain/node` and `arthur.export` leave its requires with it.

The tests here were reading the placement off :main. The photo registration
test changes shape rather than location: its premise was that face-1's head is
its own root, so a photo sitting where it was filmed was image pixels over
image height and nothing else. The head still cancels — that is what the test
is about — but it now cancels against the face's own placement, which is why
the photo comes with the face into its own tab instead of sitting at a
fraction of a pixel beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 01:25:37 -04:00
Your Name
e6d0ededb1 spreadsheet ui 2026-10-01 01:25:26 -04:00
Your Name
7e34d0c704 Make the media pool a list you can actually read
Every row is one line now — a thumbnail at the stage's own 16:10, the name,
and the one number you need before dragging it somewhere — at the timeline's
own row height. Two always-open folders cost eight headings before the first
row in a pane 210px wide; they are two tabs, and a search counts its hits in
the scope you are not looking at so nothing can hide behind the one you did
not pick.

Symbols draw their own first frame, through the resolver and rasteriser the
stage uses. A PLACED symbol is drawn where it is placed, with the rest of its
host isolated away: rooting at a symbol renders its DRAWING, and a rotoscoped
face is head-local in units of one image height, so the source-to-stage scale
that makes it pixels lives on the :face group of whatever places it. Rendered
rooted at itself a face is correct and under a pixel across. Thumbnails are
smoothly downscaled for the same kind of reason the preview is not: at a tenth
of the stage's size nearest neighbour samples one pixel in a hundred, and the
silhouette is the whole of what makes a thumbnail recognisable.

Names are editable, and that is two operations behind one pencil. A symbol's
name is a field of this document, so it is an undoable edit and blank gives it
back its id. Footage and sounds live beside projects rather than inside one,
so theirs is a server write shared by every project using the row — PATCH on
the existing Footage.label, and a new Sound.label kept separate from the
filename, which is a fact about the upload and not somebody's name for it.

No TIMELINES section beside a SYMBOLS one: every symbol here IS a timeline, so
that pair named one thing twice. What is true is that exactly one of them is
where the work happens, and clip/opens-on already answers which — it leads the
pane as PROJECT, drawn at a size you can read a pose off, with the library
under it. Still not :main being special; rename it or place it inside
something else and the pool follows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 00:56:39 -04:00
Your Name
815ce449ea Author corrections without baking them into motion
Constant, ramp, and return offsets now append ordinary channel layers to a lane or cel in explicit owner frames. The inspector exposes the commands as one undoable transaction, shows conflicts, and offers removal or retry while preserving generated bases through regeneration.

Ordered-stack compatibility is shared by validation, conflict reporting, and regeneration, including adjacent replacement coverage. Cel-sheet gaps and headers select their lane, so commands cannot fall through to another column's stale selection.

437 tests, 5,804 assertions; both browser flows; 56 Django tests; optimized frontend build.
2026-10-01 00:29:59 -04:00
Your Name
3dbbe285fc bread crumbs 2026-10-01 00:27:01 -04:00
Your Name
5fb04f6a7d better toolbar 2026-10-01 00:11:16 -04:00
Your Name
2f1c9b9c02 Turn the lane sideways without changing what it means
The cel sheet is a second projection of the same lane rows: frames run down, lanes run across, and each occupied cell carries the timeline cel's exact selection address. The shared action strip proves the point in the browser test by selecting a cell and issuing the existing hold command.

Before exposing that second entrance, fix the boundary mistakes it revealed. Nested commands now convert the open playhead through their enclosing instance path. Overwrite composes blanking with non-rippling placement as one transaction. Picture-rate and pose sampling select only the generated base frame while hand corrections retain the node's authored frame. Stack validation follows covering replacement layers so a document accepted by the validator cannot throw solely because a later offset sees a different shape.

429 tests, 5,767 assertions; both browser flows; 56 Django tests; optimized frontend build.
2026-09-30 19:59:38 -04:00
Your Name
7a54bfca56 Write down what the next person needs
`docs/lane-handoff.md`, after `timing-handoff.md`'s shape, because the next
thread starts cold and the expensive part of that is not the code — it is the
decisions that were argued out and would otherwise be argued again.

So the section that matters most is the one listing what NOT to re-litigate:
the shot length is authored, placing ripples and overwrite is blank-then-place,
a position inside a cel refuses and names split, a correction has no time space
of its own, a layer's values are a channel, a conflict is not a problem, and a
command refuses rather than guesses. Each of those is a paragraph here and a
commit message in full.

Then the vocabulary, since it was settled one commit ago and the old words are
still in this repository's history: instance, cel, lane, drawing, and placement
for where a node sits only. With the warning that `exposure` still means the
`:expose` grid and always did.

Then the mechanisms that keep paying out — `:span` in the node's own frames
above all, which is why split, trim and blank cost almost nothing — the known
gaps, and how to run the suites, including that a release build clobbers the dev
bundle the browser tests need.

Recommended next piece is the cel sheet: it needs no new model, and it is the
first real evidence the document is not shaped by the timeline that grew up
with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 19:45:28 -04:00
Your Name
598c186c4f One word for one thing: it is a cel
Four words had accumulated for a node that puts a symbol inside another symbol.
`instance` was the document's, from the model. `placement` was the stage and
export work's. `occurrence` came in with the lane model. `exposure` came in with
me, because it is what an animator would say. Three bodies of work each brought
a word and none of them retired anybody else's, which is how you get a codebase
that reads like three people describing the same object over each other.

It is a CEL. One drawing, held for some duration. `cel` was already the view's
word — `.tl-cel`, the cel strip — so choosing it was also the smallest change,
and the app already says "drawing" for the content, which is what frees the word
up: historically a cel IS the celluloid with the drawing on it, and that sense
has somewhere else to live here.

  instance   the `:kind`. The general thing, anywhere in a document.
  cel        an instance in a lane. UI labels, command names, prose.
  lane       the group with `:layout :sequence`.
  drawing    the content a cel names.
  placement  kept ONLY for where a node sits — `nest/placement` and the
             transform that puts a face on the stage. Retired as a noun for the
             node itself.
  occurrence gone.

AND IT SETTLES A COLLISION I SHOULD HAVE SEEN EARLIER. `:time :expose` already
existed and means something else entirely: how many frames each step of a
subtree lasts, which is what shooting on twos is. Had the block been called an
exposure too, `node/expose`, `clock/exposed-frame` and `subs/render ::exposure`
would have been permanently confusable with it. Choosing `cel` lets the word
`exposure` keep the thing it actually names, and every remaining use of it in
`src` is now that one.

`:layout :sequence` stays as the field, and it is the one place two words are
kept deliberately: the layout names the RULE — children follow one another and
may not overlap — and a group carrying it is called a lane. `node/lane?` says so
where the two meet.

The second view is traditionally the exposure sheet. It will be the CEL SHEET,
for one vocabulary.

Renamed with a script and then read, because a blind pass does real damage: it
produced "an cel" thirty times, renamed the `::exposure` sub that is about the
`:expose` grid, and turned an "exposure grid" into a "cel grid" in two
docstrings. All three classes are fixed. `arthur.domain.sequence` is now
`arthur.domain.lane`, which is what its test file was already called.

424 tests, 5,749 assertions, and both browser flows — `test/browser/lane.mjs`,
renamed too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 19:43:18 -04:00
Your Name
76106d36ee The shot is as long as somebody said it was
Trim, move and blank, and the decision they all three walked into: is the shot's
length authored, or derived from what is in it?

AUTHORED. `:frames` is the symbol's window — how long the shot IS — and the
occupied extent of its lanes is a different fact, read off the occurrences. A
command grows the window when the caller says `:grow-symbol` and NEVER shrinks
it, so blanking the end of a shot leaves a shot with empty frames at the end.
That is a true statement about what somebody authored, and the alternative is
deleting the last drawing and quietly shortening the film. `finish` had the
right behaviour by accident — `(apply max (:frames sym) ...)` — and now says
which number is which: `needed` is where the occurrences reach, `:frames` is
what was authored, and the only thing that makes the second follow the first is
a caller asking.

The three commands turned out to be one piece of geometry, which is `split`'s.
A `:span` is in the occurrence's OWN frames and `:time` says where those land in
the lane, so moving an edge of an exposure is ONE WRITE to `:span` and `:time`
and `:playback` are never touched. `local` and `edged` are the whole of it, and
split now goes through them too.

  trim   narrows one edge and moves nothing else. Lengthening is `extend-hold`,
         which carries a ripple policy and a shot-length policy because it needs
         them; letting trim grow as well would give one gesture two sets of
         rules and a way to overlap its neighbour.
  move   one write to `:time :at`, and a destination that would overlap is
         REFUSED rather than rippled. Moving a drawing and re-timing the ones
         around it are different intentions, and a move that pushed the rest
         would be the second wearing the first one's name. Clear the room first.
  blank  leaves a gap and does not close it. Wholly inside the range goes,
         overlapping an end is trimmed to it, spanning the range is split — the
         one case that needs an ID, and it asks for one instead of inventing it.

Because the source clock is untouched, trimming the front of a playing insert
starts it LATER INTO its animation rather than restarting it, which is the
difference between trimming and slipping and the reason they stay two commands.
The test samples the frames it kept and asserts they show what they showed.

Blanking leaves the drawings in the library. A lane does not own its content,
and a drawing whose last exposure is gone is still a drawing somebody made.

Overwrite is now `blank` then `place` and needs no policy argument of its own,
which is why it still is not one.

424 tests, 5,749 assertions. The browser flow trims an exposure at the playhead,
moves it into the gap that made, blanks it, and checks the shot is still as long
as it was authored.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 16:32:59 -04:00
Your Name
72b57e3786 Regenerate the base, keep the hand work, and say when you cannot
The loop the layer design exists for, tested for the first time: correct a
generated channel by hand, turn the generator's knob, and get the new base with
the correction still on it. `replace-feature` already carried `:over` across —
somebody anticipated this — so the feature path needed a test and not a fix.
The head path needed a fix, and there was a second fault of my own making.

`regenerate-head` leaves the head's authored channels alone once somebody has
placed it by hand, and decided that by `(= (:channels old) (:measured old))`.
Sound, until a correction exists: an `:over` layer makes those unequal, so the
FIRST correction anyone made would have stopped the head following
re-measurement for good — the exact opposite of what a layer is for. It compares
the channels without their layers now. The test fails against the old guard,
which is how I know the bug was real and not a story about one.

The other fault was mine, from the commit before this one. An `:offset` whose
shape does not match its base threw, which is right for authored data — the
validator catches it — but WRONG for the case the model actually names: turn the
mouth's `:verts` knob and the re-freeze gives it a different number of points,
so a correction that was correct when it was made stops fitting through nobody's
error, and a throw in the read path takes the stage down.

So a base that has outgrown a correction is a CONFLICT, and a conflict is the
third thing beside applied and discarded. The regeneration records `:conflict`
on the layer; the layer stays exactly where it is; `over-at` skips it, so the
picture is the base meanwhile; and `clip/conflicts` lists them for a view to
offer. A later regeneration that restores the shape clears the mark, so
resolving one can be as simple as putting the knob back.

Deliberately NOT `problems`. A document with a conflict loads, evaluates and
saves — it contains a decision nobody has made yet, and refusing to open it
would be the persistence layer taking a side in an editing question. The
distinction in the validator is one line: a shape mismatch nobody has recorded
is an authoring bug, and one a regeneration recorded is a conflict.

`channel/conflict-with` is the single rule for "can this layer apply to this
base", used by the validator, by `conflicts`, and by the regeneration that marks
them. Only `:offset` can conflict, since `:replace` states a whole value and has
nothing to agree with; a shape that cannot be read yet — an empty key map — is
not a disagreement. `value-shape` answers it without sampling anything.

414 tests, 5,696 assertions, and `:verts` in the test is a real topology change
rather than a synthetic one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 16:22:02 -04:00
Your Name
94c0a21de1 A correction is a layer, and a layer's values are a channel
`:over` was specified in animation-model.md, refused in two places, and
produced by nothing: `check-unimplemented!` threw on read and `channel/problems`
reported it. It reads now. This is the part of the model the rotoscoping half
depends on — generate motion, correct it by hand, turn the knob, keep the
correction — and it was the last thing in the design that had never been tried.

The shape that made it small: A LAYER'S VALUES ARE A CHANNEL.

  {:id :nudge :support [88 98] :op :offset
   :values {:animated? true :interp :linear :keys {88 [2 0], 96 [0 0]}}}

So the three commands the lane model asks for over a selected range — a
constant adjustment, a ramp, a return motion — are one mechanism and not three:
framed values say the same thing on every frame they cover, keyed values move,
and neither needs a new way to say what a value is over time. A layer reads
through `value-at` and `cursor` like any channel, which is also what stopped
blending from becoming two implementations: `over-at` is shared, and the
specification and the playback path differ only in how they READ a layer —
recursively through `value-at`, or through a reading head of its own. One level
deep; a layer's values may not carry layers, which the stack already orders.

That was the risk worth spiking for. A cursor that drifts produces the wrong
pose rather than an error, and a stack means several reading heads per channel
where there was one. The agreement test that holds the cursor to the
specification in forward, backward and random frame order now covers stacked
channels too — including a layer whose head is asked for nothing across the long
stretches outside its support and then asked again, which is where drift would
hide.

`:support` is half-open and explicit. Outside it the base evaluates exactly as
it did before, which is the whole difference between a bounded correction and
inserting boundary keys: the latter alters the neighbouring segments, and the
lane model says so.

A LAYER HAS NO TIME SPACE OF ITS OWN, and this is the design question the doc
left open. Its support and its values' keys are in the frames the base channel's
keys are in — the node's. A correction on a lane is therefore in lane frames and
reaches across the drawings exposed beneath it; one on a single occurrence is in
that occurrence's frames and travels with it when the exposure moves. Ownership
had already answered it, so there is no field to disagree with, and both halves
are under test at lane level.

Two things cost nothing, which is worth recording. A channel is ONE LEAF, so a
correction persists inside it with no codec change at all. And `node/problems`
already reports every channel's problems, so a malformed layer surfaces at the
document level and in the sequence commands' post-check without plumbing.

What is still missing is a command that MAKES one, and with it the question of
how a view offers a constant, a ramp and a return over a selected range. The
evaluator no longer has an opinion about that, which was the point.

`offset` adds component-wise and never writes into a dense value, which is a
view onto the block itself; a shape mismatch throws rather than being dropped,
since a correction that silently does not take is the failure this design exists
to prevent. `replace` can supply a value over an absent base and `offset`
cannot, as animation-model.md required.

408 tests, 5,655 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 16:07:38 -04:00
Your Name
26517af2fd A position is an argument, not another command
Everything could only be added to the end, because `append` computed its own
position — the max end of the lane — and so had no opinion to state. Insert is
not a new command; it is the argument that function was missing. `:at` takes a
lane frame or `:end`, `:end` is the position where nothing has to move, and
appending stops being a separate operation from inserting. New, reused and
duplicated drawings all take it, because there was only ever one placement rule.

Placing ripples: occurrences at or after the position move later by the new
exposure's duration, and `:keep` against `:grow-symbol` still decides what
happens at the shot's end. OVERWRITE is deliberately not a policy argument yet.
Taking frames away from the occurrence already there is TRIMMING, and an
argument whose second value is unimplemented is worse than an argument that is
not there. A position strictly inside an existing exposure refuses and names
`split`, rather than splitting on the quiet: one command performing two is how
a command stops being predictable.

Then split, which turned out to cost almost nothing, and that is the
interesting part. The two pieces keep ONE `:time` and differ only in `:span`.
The right piece's own frames therefore carry on exactly where the left's
stopped, so its source clock, its keys and its corrections go on meaning what
they meant: a held drawing holds the same frame either side of the cut, and a
playing insert plays through it without a seam. There is no arithmetic on
in-points to get wrong, and no shot-length question, since the pieces occupy
the frames the one exposure occupied. The test samples every frame before and
after and asserts the picture is identical — for a hold, for an exposure with a
correction of its own, and for a playing insert.

That is not a clever split. It is `:span` being in the node's OWN coordinates,
which was decided long before there were lanes, paying for something it was not
designed for. The same property is why extending a hold leaves lane keys alone.

Both new commands act at the playhead, which needed `lane-frame` — the symbol's
frame as a frame of the lane's own time, nil through a stepped or looping lane
where one is not the other. Nil refuses; it does not snap to a nearby frame.

Two smaller things found while doing it. `placeable` promised "a whole lane
frame" in its refusal and then accepted 2.5, so both it and `split` now require
an integer, as `extend-hold` already did for its delta. And `lane-end` is
private: `:end` is the only way to ask for it.

401 tests, 5,612 assertions. The browser flow now splits an exposure at the
playhead and puts a drawing in the gap, and checks that six exposures are still
one row.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 15:56:26 -04:00
Your Name
9446829774 Reuse, duplicate and make unique: deciding what is shared
The model's whole claim is that content and its occurrences are different
things, and until now nothing in the editor could tell them apart: you could
make a drawing and time it, but not expose one drawing twice, and so never find
out whether an edit arrives in two places. That is the first proof obligation in
the lane model and it was the one the commands could not reach.

Three commands, and the distinctions between them are the point:

  reuse       another occurrence of the same drawing. A decision to share,
              made on purpose, because sharing discovered later — when an edit
              turns up somewhere you did not expect — is the bad version.
  duplicate   a copy of the drawing, appended, for when what is on screen is
              the starting point for the next one.
  make unique this occurrence gets a private copy; the others keep sharing.
              The undo of reuse, and refused when nothing else uses the
              drawing: a copy nobody asked for is a second identical symbol in
              the library for no reason a person could see.

Duplicate copies the CONTENT and not the exposure. Its new occurrence is a plain
one-frame hold, not a copy of the source occurrence's transform or corrections,
because those belong to that use of the drawing — carrying them over would make
duplicating a drawing quietly duplicate the treatment of one exposure of it.

A copy is SHALLOW by default and keeps its references to other symbols, so a
head built out of reusable eyes still uses those eyes. `:deep? true` copies
everything it places with new ids throughout. The lane model asks for both and
says why: never promise decoupling while leaving the edited object shared, and
only the deep copy can keep that promise. `bring/symbols` already did the
reachability walk and the id remapping, so the deep copy is that function
pointed at its own clip.

`node/sources` was still being read as a SET at five call sites, each with a
comment about a lane that cuts between several drawings — the keyed source that
no longer exists. An occurrence names one symbol, so they now ask `node/source`,
and `placed-frame` answers with `:symbol` rather than `:of`, which was the last
echo of the retired field name.

To let the commands use `clip/free-id` and the copy machinery, the lane's own
validation moved from `domain/sequence` to `domain/symbol`, which is where it
belonged anyway: a sequence is the one composition rule a node map carries, and
it now sits beside the parent and stencil checks rather than in the namespace
that happens to build lanes. That also breaks the cycle — sequence can require
clip and bring, and nothing below it requires sequence. Preconditions still
check only the LANE's shape: refusing an exposure edit over an unrelated defect
elsewhere in the symbol would be this command answering for a part of the
document it never touches.

The cel strip gains reuse, duplicate and make unique, the last shown only where
the selected exposure actually shares its drawing. Drawing on twos is also now
under test: exposure length is the cadence, the lane's transform has its own
clock, and it still moves on every frame — stepping it would be the cel cadence
leaking into continuous motion.

397 tests, 5,561 assertions. `test/browser/sequence.mjs` drives the three new
commands through the real editor and checks that three exposures are still one
row.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 15:30:59 -04:00
Your Name
3d3c1bbca0 An occurrence is a node, with a clock of its own
A lane's drawings were going to be one instance whose source was a KEYED
channel: frame 0 says `:drawing-a`, frame 4 says `:drawing-b`, and the cels of
a row are that channel's keys. Two things followed from it, and both were
wrong.

The first is that playback meant whichever shape the channel happened to have.
A framed source played its symbol; a keyed source froze the selected frame.
So `node/placed-at` read animation out of storage, and adding an ordinary key
to a still turned it into an animation — the last-key bug, which was not a bug
in the code so much as the rule working as written. But WHICH drawing is used
and HOW time runs inside it are independent questions, and all four combinations
are ordinary: hold one drawing, play one animation, cut between held drawings,
cut between playing ones.

So an occurrence names one symbol in `:source {:symbol ...}` and says how its
source time advances in `:playback {:in :speed :end}` — `source = in + speed *
f`, a hold being speed 0, with `:stop`, `:hold` or `:loop` at the end named
rather than guessed. `node/placed-frame` samples it forwards, which works for
holds too, and `node/source-time` is the separate, invertible edit map, nil
where inversion is meaningless. The two were one function before, and a hold
had to lie about one of them.

The second is that a keyed source only looked necessary because an occurrence
was assumed to need a ROW. It does not. A lane is a group with `:layout
:sequence`, its occurrences are ordinary instances in the same flat node map,
and `timeline/rows` draws them as cel blocks on the lane's own row: twelve
exposures, one row, each cel still separately selectable and addressable. The
vertical growth that justified the keyed source is a presentation question, and
it is answered in the view.

`arthur.domain.sequence` holds the first commands over that shape — add lane,
append drawing, extend hold — each one history step, each refusing rather than
half-applying. Extending a hold leaves the lane's keys at their authored times,
because you are adjusting drawings underneath timed motion; a correction owned
by an occurrence travels with it. Ownership does that work, so no key needs a
flag saying what it follows. Ripple past the symbol's end is refused with the
frame count it would need, and `:extent :grow-symbol` is the caller saying yes.

`clip/blank` no longer carries `:subjects {} :features {} :groups {}`. Empty
maps write no leaf, so a blank document could not survive its own round trip —
`leaf/leaves` promises exactness and was the only honest side of that.

Documents are schema 3. A version 2 document is not read; nothing here converts
one. `docs/lane-model.md` is the design, and says which of its parts are built.

392 tests, 5,525 assertions, and `test/browser/sequence.mjs` drives the editor
through create, hold, explicit overflow and undo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 15:20:23 -04:00
Your Name
624242b407 An options map for the resolvers, and sid beside the clip it is in
Finishing the last commit, which traded a simpler definition for noisier call
sites: deleting the arity ladders left `(symbol/resolver sym st pal/index-of
nil nil)` at twenty-odd places, and two trailing nils tell a reader nothing
except to go and count positions.

The ladder was a symptom. The disease is five positional parameters, and the
split that matters is which of them are OPTIONAL:

  store, palette    positional, because neither is optional. A dense channel
                    cannot be read without the store it names — that is the
                    crash two commits ago — and every op carries a colour.
  pose-tracks       one call site, in `clip/resolver`'s own recursion
  source/picture-fps  two call sites

So the last three become one `opts` map, and the common call loses a nil. The
point is not the nil: it is that the sixth option, whenever it arrives, is a
key at one call site rather than a nil at fifty.

`clip/resolver` also had `sid` FOURTH, behind two arguments that say nothing
about which symbol is being resolved. It is second now, beside the clip it is
in: `(clip/resolver c :main store pal/index-of nil)`.

62 call sites rewritten by parsing the forms rather than by regex, because
`clip/resolver`'s arguments move past each other and a regex cannot see that.
An earlier attempt at this dropped `palette` on the floor and still compiled
at 62 sites — it only surfaced as an arity error, so if that had been a
same-arity mistake the tests would have been the last line of defence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 12:21:12 -04:00
Your Name
ee66680a0c Make every caller say what it means: no defaulted arities
Pre-alpha. Nothing here is owed a call shape it used to have.

Twelve convenience arities deleted, and the only reason to single any of
them out is that one of them was a live bug: `channel/value-at`'s `([ch f])`
filled in a nil tier-2 store, so a caller could omit it, read correctly for
every channel that happened not to be dense, and throw the first time one
was. That is the iris crash, and threading the store through `gesture/values`
last commit fixed the symptom while leaving the trapdoor open. Deleting the
arity found `node/toggle-key` standing on it too — the inspector's stopwatch
on a measured channel, the same throw, never reported.

Gone, and what the compiler then made explicit at each site:

  channel/value-at, cursor, dense-at   the store, and `nil` where a caller
                                       genuinely has none and means it
  channel/keyed                        `:hold`, which is a cut rather than a
                                       tween and not a thing to leave implied
  symbol/resolver (4), eval-frame (3)  store, palette, pose-tracks, opts
  clip/resolver                        opts
  mix/buffer!, store/install!          dead: no caller used the short form

`pick/local-bounds` goes the same way — it was `bounds-of` with the closure
thrown away, so callers build the closure and call it.

Every site was found by shadow-cljs `:fn-arity` rather than by grep, which is
the argument for the change: 90-odd call sites, and the compiler listed all of
them. BUILD BOTH TARGETS — the last three only appear in `:app`, since `:test`
compiles what the tests reach and the inspector, the pool drag and the vertex
overlay are not that.

Left alone, because an argument with a default is not the same thing as a
shim: genuine optionality like `fx/http`'s body, `geom`'s iteration count,
`zip`'s injected clock.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 12:10:22 -04:00
Your Name
11093079de Scale a part about the middle of what it draws, from the document as it is
Three faults, one gesture. None of them was in `gesture/scale`, whose
`s' = s·b/a` about the pivot was right all along.

THE PIVOT. `clip/place-symbol` writes an instance's anchor, `paint/new-shape`
a drawing's, `nest/group` a new symbol's and `face-placement` the source
placement's — and `flow/freeze` wrote none for the parts underneath, so a
traced mouth turned and scaled about its own coordinate ORIGIN, which for
head-local geometry is the top-left corner of the footage. On a 320x200 stage
that put the mouth's pivot at (-234, -395), so dragging a corner outward slid
the shape about and shrank it. `pick/pivot` is `clip/center`'s rule for a node
rather than a symbol; `freeze/pivoted` applies it to every node the freeze
makes, skipping `node/measured?` — the predicate `gesture/refusal` already
refuses a hand edit by, so a pivot is written exactly where a hand can use
one. That also keeps it off `:head`, whose scale is not 1 and where an anchor
would NOT cancel out of `local!`; skipping it for drawing nothing would have
been true only by accident. Asserted in pixels: the pass moves nothing.

THE JUMP. `ui/stage`'s overlay dereferenced the document while RENDERING and
used it when the pointer went down. The store is a mutable handle behind an id
that does not change when the document does — `:paint/revision` says that, and
the overlay subscribes to neither it nor `::render/clip` — so after any edit
the next drag began from the transform the node had before the last one: still
under the press, jumping on the first pointermove. `ctx` now carries the id and
`loaded` reads at pointer-down.

THE CRASH. `gesture/values` read its channels without the tier-2 store, which
is fine until a selection lands on a dense transform — an iris follows the
gaze, a brow the raise, a head the similarity — and then `dense-at` throws and
takes the stage down, in `begin!` and again in `handles`. It takes the store
now. Kept in this commit because it is the same two functions.

`pick/bounds-of` yields a closure, as `clip/resolver` does, so an instance's
resolver is built once for a walk instead of once per frame — which is what
let `pivot` be the one walk it is rather than a separate path for instances.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 11:53:50 -04:00
Your Name
ed88c5e674 Hold a tracing photo steady, and switch it on where you would look
The photo blinked out for a frame or two and sometimes never arrived, from
three causes that each present as the same bug. The still cache emptied
itself on the frame it filled, so every still on screen had to be fetched
and decoded again — every 48 frames of a scrub, and with two traced faces a
permanent flicker, each one's still evicting the other's; it drops the least
recently used now, a still being touched on every frame it is drawn. A still
cannot decode in the animation frame that asks for it, and a face whose next
one had not arrived drew nothing, so it now keeps the still it was showing
until the new one is there. And nothing was read ahead, so continuous
playback was always a frame behind its own footage; the next few frames'
stills are asked for while the playhead is moving on its own, and only then,
because a scrub asks for a different few at every step.

Whether the footage shows is no longer the document's. It was an :underlay
on an instance, inherited down the row path, nearest wins, and it went
through edit — so showing a reference photo was an undo step that travelled
to collaborators. It is [:ui :trace] now, the faces switched on and one
opacity, like solo, and there is nothing left to inherit: a face is the same
face wherever it is placed, so one switch covers every placement of it. The
paint loop asks for its own redraw when that changes, since the resolver no
longer does it for them.

Opening a face shows its footage, because a symbol has measured footage
behind it only because it was traced from that; a take does not, because a
take is the picture. The switch is on the bar above the stage, with the tone
and the tool, and on each face's timeline row beside solo — not a section
that appeared once the right row had been found. A face open in its own tab
could not show its footage at all before, shown having walked instances, and
that is the one place tracing matters most. The inspector keeps the face's
own keyed facts, its trace keys and its origin, and says why it cannot key a
frame rather than greying out the only button in the section.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-30 10:42:39 -04:00
Olive Vaughn
ae03b61dca sound stuff 2026-09-30 03:31:19 -04:00
Olive Vaughn
309c47e0a7 Trace a face over its footage, and choose where its origin goes
A face's :head carries :trace {:frames :origin}: the frames its photo holds
on, and whether the head reads every frame, jumps to the trace frames, or
holds frame 0. It replaces :anchors, so which measured frame a head reads is
one stored fact. An instance's :underlay shows the tracing stills over every
face at or below it, registered through each face's own head, at an opacity,
unkeyed. The clip resolver answers where a row path went on its last frame,
so the paint loop reads the photo's matrix instead of resolving again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:49:06 -04:00
Olive Vaughn
550cfe91e5 Select, move, turn and scale on the stage, at any depth
A click selects the thing in the open symbol, a double-click goes one
level in, ⌘-click goes to the shape itself and Esc comes back out —
Figma's rule — and a click inside the selection keeps it, so a shape
several symbols down can be dragged. The selection is the one a
timeline row makes, so the inspector shows it and its row opens and
scrolls into view.

A drag writes what the inspector writes: a key on the node's own frame
where the channel has keys, its one value where it has none. It is
previewed like a bar being slid and let go as one edit, so one undo
step. Measured transforms refuse. A shape's points are edited by
double-clicking it, and new shapes turn about their middle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:12:31 -04:00
Olive Vaughn
2a0426707a Hold or tween any keyed channel's gaps, as a drawing's
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:51:30 -04:00
Olive Vaughn
da7e293814 Solo an instance from its timeline row, shift-click for more than one
A nested mark is now named by the flat path its row has, [a b mark] rather
than [a [b mark]], so a soloed row is a prefix of what it shows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:44:24 -04:00
Olive Vaughn
7aaf0a15bd Inspector number boxes edit as you type, and are one undo step per focus
The boxes wrote nothing until blur, so a spinner click showed nothing on
screen. Now every keystroke and step is an edit, and history holds the
step open from focus to blur, so typing 4 then 5 is seen as 4, then 45,
and undone once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:35:52 -04:00
Olive Vaughn
277c0c3b63 Keep the timeline's ruler and playhead dot in view while its rows scroll
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:22:01 -04:00
Olive Vaughn
c139e4d747 Rename the project from the top bar; export from a dialog
The title is a button that becomes a field, saved on Enter or blur. Export's
target and scale move out of the bar into a dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:12:08 -04:00
Olive Vaughn
0f9ce826ef Set the project's stage and rate, and give a symbol its own stage
The inspector edits the project's width, height and fps, and a symbol's length
and, optionally, its own width and height. A symbol without them follows the
project's (`clip/stage`); the stage, export and placement all read it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:12:08 -04:00
Olive Vaughn
1ac21fdcab Brought-in footage plays at the project's rate
The new symbol gets project-rate frames covering the same wall-clock length,
and its root and sound map them back onto the source's frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:11:32 -04:00
Olive Vaughn
4b8123d5e2 Delete a timeline row with everything in it
Delete or Backspace, or the × on the selected row, takes the node out of its
symbol along with every node hanging off it. One edit, so undo brings it back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:11:16 -04:00
Olive Vaughn
7246534505 Key a node's transform and visibility from the inspector
Each of vis, anchor, pos, rot, scale and skew is a row: ◆ keys the channel on
the node's own frame, or takes the key there off, and a typed value is written
on blur or Enter — a key on a keyed channel, the one value on one that is not.
Rotation reads in degrees. Dense and other channels keep their readout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 01:11:16 -04:00
Olive Vaughn
38850a29ba Slide rows along time and restack them, at any depth
A node's bar drags along the timeline: one write to its :at, for every node
alike, carried down through the instances above it. The stage and rows show
the slide live through ::render/clip, and it lands as one edit on release.

A row dropped on another's top or bottom edge goes in front of it or behind:
one write to :z, between its new neighbours (symbol/z-between). Onto the edge
of a row in another symbol, it moves there first. Neither needs anything on
screen — nest/down walks a row path by structure, without a frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 00:51:34 -04:00
Olive Vaughn
1b8bbc7372 Edit shapes nested in other symbols, from any symbol above them
`nest/inside` walked a row path only through instances. Every node has
the same two maps to its parent, so the walk now steps into any node:
inside an instance is the symbol it places, inside a shape is where its
points and keys are. The selected shape at any depth is `inside` over
its full path, which gives the stage editor its handles (through the
matrix, drags back through the inverse) and the inspector the shape's
own frame to key at, and the time map back for jumping to a key.

`inside` resolves only the node's lineage: where a node is depends on
its parents and nothing else, and resolving the whole symbol cost more
than a stage frame (7.9ms against 3ms on the swarm; now 0.09ms).
Checked equal to the whole-symbol resolve on every node and frame of
the swarm, two instances deep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 00:12:21 -04:00
Olive Vaughn
6a53adb5e0 Projects live at URLs, have owners, and are edited together live
A project is only ever at /p/<id>/<slug>; / is the index of the projects
you own or edit. Every project has an owner, who can name editors;
anyone with the link can view. Every edit saves itself, one request in
flight at a time, as a patch of the leaves that changed, and a websocket
(channels + daphne) carries presence and each committed write to
everyone else in the project. The first write to a leaf wins, and the
loser is told.

Undo is per person: a step undoes only if the leaves it touched still
hold what it left, so it never takes a collaborator's work with it.
Named snapshots replace saving, and restore as an ordinary write.

An empty symbol now survives the leaf round trip with `:nodes {}`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:04:03 -04:00
Olive Vaughn
c17ee138f2 Split domain/clip along the data: clip, nest, bring
domain/clip is the document and what only needs the document: its symbol
table, placing, the resolver, problems. domain/nest is how nested symbols
relate — one walk down a row path gives the frame, the matrix and the time
map, which merges inside and time-down — and moving and grouping between
them, and nested sound. domain/bring is copying symbols in from another
clip, a take from footage, and one placed that merges the store and places
the instance, which conversion and import both now call instead of each
doing it in their own words. Tests follow the same split.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 14:11:41 -04:00
Olive Vaughn
7bb80d315d Bring in symbols from other projects; new and open send the clock home
Dropping a symbol out of the pool's all-assets folder fetches that project's
clip and the blocks it names, copies the symbol and everything it places in
through clip/adopt — renamed where ids collide — and places it where it was
dropped. It comes in as drawing: its tracking stays with the analysis that
measured it. New and open now seek the audio clock to 0 with the readout,
where play used to pick up wherever the last document's audio had got to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:51:39 -04:00
Olive Vaughn
490460bf45 Drag timeline rows into symbols, into groups, and back out
A node's row can be dragged onto an instance's row, to go inside the symbol
it places; onto any other node's row, to be grouped with it into a new
symbol; or onto empty label space, to come back to the top of the open
symbol. All three keep the picture and the timing as they are, and a refused
move says why in the status line. Pool drop targets now only take things out
of the pool, and row drags set a move effect so the browser delivers the drop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:50:34 -04:00
Olive Vaughn
eafbe6c4d2 One time map for every node; move and group nodes between symbols
Every node now has the same time map into its parent, local = rate·(parent
− at), with its span and keys in its own frames: what instances had, made the
rule. A shape without a time map reads as it always did, so no data changes.
The per-kind branches, the span-start term and the rate refusal are gone;
node/time-of, then-time and invert-time compose it like the matrix.

clip/move-node puts a node into another symbol without changing the picture
or the timing — its matrix becomes a :pinv, its time a new :at and :rate, and
its channels, keys and span are untouched — and clip/group makes a new symbol
around side-by-side nodes. Generated parts, split stencils, cycles and
looping instances are refused with the reason. Nested sounds use the same
map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:45:34 -04:00
Olive Vaughn
6bec121108 Draw into the selected symbol
With an instance selected, a finished polygon goes into the symbol it places,
its points and frame carried in through clip/inside — which resolves each
level, so the frame and matrix are the ones the stage draws with — so the
shape lands exactly where it was drawn however the instance is moved, turned,
scaled or retimed. Beside any other selected node, or at the top of the open
symbol with nothing selected. clip/inside also replaces frame-inside for new
symbols, so there is one account of 'which frame is it in there'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:35:06 -04:00
Olive Vaughn
1a42481575 An instance pivots about the middle of what its symbol draws
place-symbol sets a new instance's anchor to clip/center — the middle of the
bounds of everything the symbol draws over all its frames, or the stage's
middle for a symbol that draws nothing — and a stage drop puts that middle
under the pointer. The anchor is set once and never follows the symbol, as
Flash's transformation point and After Effects' anchor point do, so a symbol
that grows later moves nothing on screen. The drop preview marks the pivot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:32:52 -04:00
Olive Vaughn
41b4bdf110 Make a symbol from a dropped video
Dropping footage on the stage or the timeline asks which frames and what name:
a dialog plays the video with start and end handles that seek it. Detection
runs on that range only (decoding stops at the end, frames before the start
are skipped) and the range is part of the analysis address, so a partial
analysis is never served as a whole one. The frozen take comes in as one
named symbol, carrying its sound as an audio node, placed where it was
dropped; tracking and tuning come with it when the document has no analysis
of its own.

clip/adopt copies symbols between documents, renaming ids that collide, and
clip/audio-tracks carries sounds out of nested instances so a placed symbol
is heard where it is placed.

Fixes the stage going blank after a conversion: ::store recomputed only when
the clip id changed, so blocks merged into the loaded entry were invisible to
the resolver. And the paint loop now schedules its next frame before
painting and reports a frame it cannot draw instead of stopping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:21:08 -04:00
Olive Vaughn
270c5a4369 Version static URLs by mtime; flatter tabs
The stylesheet and bundle were served with Last-Modified only, so a browser
could pair new JavaScript with a stale app.css and render the tab strip and
thumbnails unstyled. Their URLs now carry the file's mtime. Tabs are flat on
the chrome with the open one underlined; the close button shows on hover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:08:38 -04:00
Olive Vaughn
f590ff19cf Drag symbols onto the stage or the timeline, with previews
A symbol dragged out of the pool shows where it would land: on the stage, a
dashed outline of its first frame centred on the pointer, and on the timeline
a preview row of its own length. A stage drop places it at the playhead under
the pointer; a timeline drop places it at the frame under the pointer, in its
own coordinates. A drop that would make a cycle is refused while hovering.
Pool thumbnails are capped at 40x30.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:06:38 -04:00
Olive Vaughn
d5f044c6da Media pool: this project and all assets
This project lists every symbol in the document and the video it uses or was
given this session, each with a frame from its proxy as a thumbnail. All
assets lists every upload on the server and every other saved project's
symbols, from a new /api/symbols that reads symbol leaf paths. Every row is a
drag source (symbol:, import:, footage:). Uploading no longer runs straight
into detection; it lands in the project's media.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:59:17 -04:00
Olive Vaughn
c81f91c442 Tabs: open any symbol, and the clock follows it
Double-clicking a symbol in the pool, or an instance's row in the timeline,
opens it as a tab above the stage; the stage, the rows, the transport and new
shapes follow the open tab. Each tab gets a clock as long as it is: its own
mixed tracks, the document's audio for the symbol it opens on, or silence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:54:58 -04:00
Olive Vaughn
6b41c6db94 An instance's span is in its own frames
A shape's :span stays in its parent's frames, but an instance's or a sound's
is now in its own: dropping a symbol at frame 97 gives it span 0 … length and
:at 97, so moving it along its parent is one write to :at. :time :in is gone
(it was the span's start written twice); node/placed-span maps an own-time
span out to the parent for playback, the mixer and the timeline rows, and
node/problems reports a stale :in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:53:21 -04:00
Olive Vaughn
2d2eb0fc9f New empty symbol, nested in the selected instance or the open symbol
'+ symbol' in the timeline makes an empty symbol and places it at the
playhead: inside the selected instance, beside any other selected node, or in
the open symbol. Timeline selections carry their row path so a symbol placed
twice nests into the row that was clicked. Symbols gain a :name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:49:46 -04:00
Olive Vaughn
5dff490162 Symbols, not timelines; no symbol is special
Everything that holds nodes is a symbol (domain/timeline -> domain/symbol,
:timelines -> :symbols) and a node that places one is :kind :instance. The
reserved :main root is gone: which symbol is on screen is editor state
([:ui :open]), every domain function that needs a symbol is told which, and
a document opens on the longest symbol nothing else places.

Saved projects move to schema 2 through migration 0007, which rewrites leaf
paths, instance kinds and the feature :symbol key; the client refuses a
schema it does not read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:46:42 -04:00
Olive Vaughn
179770d7d4 Split the shell into topbar, pool, stage, timeline and params panes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:25:46 -04:00
183 changed files with 32480 additions and 3617 deletions

@ -0,0 +1 @@
Subproject commit f4dd04764204506fc275180364d0366d693036f4

View file

@ -14,3 +14,5 @@ audio.wav
manifest.json manifest.json
*.take *.take
*.tflite *.tflite
.claude
.venv*

View file

@ -40,4 +40,4 @@ RUN python manage.py collectstatic --noinput \
USER app USER app
EXPOSE 8000 EXPOSE 8000
CMD ["sh", "-c", "python manage.py migrate --noinput && exec gunicorn server.wsgi:application --bind 0.0.0.0:8000 --workers 1 --threads 4 --timeout 120"] CMD ["sh", "-c", "python manage.py migrate --noinput && exec daphne --bind 0.0.0.0 --port 8000 server.asgi:application"]

View file

@ -30,8 +30,8 @@ mise exec -- python manage.py migrate
./do start # Django + frontend watcher ./do start # Django + frontend watcher
``` ```
In the app, upload a video, choose its footage, click **load frames**, then In the app, drop a video on the media pool — it uploads, extracts and runs
**save**. Opening that project on another client reuses its saved landmarks and detection — then **save**. Opening that project on another client reuses its saved landmarks and
mouth crops without detecting source frames again. The upload path derives its mouth crops without detecting source frames again. The upload path derives its
footage response from database records; it does not create or consume a footage response from database records; it does not create or consume a
`manifest.json` file. See [frontend/README.md](frontend/README.md) for details. `manifest.json` file. See [frontend/README.md](frontend/README.md) for details.

View file

@ -18,7 +18,7 @@ class ProjectAdmin(admin.ModelAdmin):
@admin.register(Clip) @admin.register(Clip)
class ClipAdmin(admin.ModelAdmin): class ClipAdmin(admin.ModelAdmin):
list_display = ("cid", "project", "name", "footage", "analysis") list_display = ("cid", "project", "name")
list_filter = ("project",) list_filter = ("project",)

87
clips/consumers.py Normal file
View file

@ -0,0 +1,87 @@
"""One socket per open project, and it is tl's, nearly line for line.
Two things ride it. DELTAS, which the server sends after a write commits — the
socket is read-only for the document, and a dropped socket cannot lose a write.
PRESENCE, which peers gossip between themselves: all the server does is hand out
a connection id and stamp the sender's identity onto every message, so nobody can
post as somebody else.
"""
import json
import uuid
from asgiref.sync import async_to_sync
from channels.generic.websocket import AsyncWebsocketConsumer
from channels.layers import get_channel_layer
# Who is connected, per project: {group: {cid: presence}}. A cache of what has
# already been relayed, so a joiner gets the room in one message. Process-local,
# like the in-memory channel layer this runs on.
ROOMS = {}
def group(project_id):
return f"project_{project_id}"
def broadcast(project_id, delta, kind="delta"):
"""Send a committed write to everyone in the project's room. `access` says
only that who may write has changed, and each client asks for itself."""
async_to_sync(get_channel_layer().group_send)(
group(project_id), {"type": "project.delta", "delta": {"kind": kind, **delta}},
)
class ProjectConsumer(AsyncWebsocketConsumer):
RELAYED = ("state",)
@property
def room(self):
return ROOMS.setdefault(self.group, {})
async def connect(self):
self.group = group(self.scope["url_route"]["kwargs"]["project_id"])
self.cid = uuid.uuid4().hex[:12]
user = self.scope.get("user")
self.username = user.get_username() if user and user.is_authenticated else None
await self.channel_layer.group_add(self.group, self.channel_name)
await self.accept()
me = {"cid": self.cid, "user": self.username}
others = list(self.room.values())
self.room[self.cid] = me
await self.send(text_data=json.dumps({"kind": "welcome", **me}))
await self.send(text_data=json.dumps({"kind": "roster", "peers": others}))
await self._relay({"kind": "join"})
async def disconnect(self, code):
if hasattr(self, "cid"):
self.room.pop(self.cid, None)
if not self.room:
ROOMS.pop(self.group, None)
await self._relay({"kind": "leave"})
await self.channel_layer.group_discard(self.group, self.channel_name)
async def receive(self, text_data=None, bytes_data=None):
try:
msg = json.loads(text_data or "{}")
except ValueError:
return
if not isinstance(msg, dict) or msg.get("kind") not in self.RELAYED:
return
if self.cid in self.room:
self.room[self.cid].update(
{k: v for k, v in msg.items() if k not in ("kind", "cid", "user")}
)
await self._relay(msg)
async def _relay(self, msg):
await self.channel_layer.group_send(
self.group,
{"type": "peer.msg", "msg": {**msg, "cid": self.cid, "user": self.username}},
)
async def peer_msg(self, event):
await self.send(text_data=json.dumps(event["msg"]))
async def project_delta(self, event):
await self.send(text_data=json.dumps(event["delta"]))

View file

@ -159,6 +159,108 @@ def _extract_stills(job, proxy_path, frames_dir, frames, root):
MAX_RATE = 120 # a capture rate; past this the container is describing something else MAX_RATE = 120 # a capture rate; past this the container is describing something else
# How many packet timestamps `_measured_rate` reads, and the fewest intervals it
# will draw a conclusion from. 300 is a flat cost on a long take and still a
# wide enough sample for a median; below 8 intervals there is not enough of a
# stream to outvote one odd timestamp, so the metadata is left to speak.
RATE_SAMPLE = 300
RATE_MINIMUM = 8
# How far a declared rate may sit from the measured one and still be taken as
# what the stream is: 2% covers 30 against 30000/1001 and nothing like 120
# against 30.
RATE_TOLERANCE = 0.02
def probe_image(path):
"""An uploaded still's pixel size as (width, height), refusing anything that
is not one picture."""
data = json.loads(_command(["ffprobe", "-v", "error", "-show_streams",
"-of", "json", str(path)]))
video = [s for s in data.get("streams", []) if s.get("codec_type") == "video"]
if len(video) != 1 or not (video[0].get("width") and video[0].get("height")):
raise ValueError("the uploaded file is not an image")
return int(video[0]["width"]), int(video[0]["height"])
def probe_audio(path):
"""The length of an uploaded sound in seconds, refusing a file with no audio."""
data = json.loads(_command(["ffprobe", "-v", "error", "-show_streams",
"-show_format", "-of", "json", str(path)]))
if not any(s.get("codec_type") == "audio" for s in data.get("streams", [])):
raise ValueError("the uploaded file has no audio stream")
duration = float(data.get("format", {}).get("duration") or 0)
if duration <= 0:
raise ValueError("the sound's length is unknown")
return duration
def _measured_rate(path):
"""The rate the stream's own packet timestamps imply, or None.
THE CONTAINER'S SUMMARY OF ITSELF IS NOT EVIDENCE, and this is the function
that goes and looks. An iPhone's `r_frame_rate` is 120 on footage whose
timestamps are 1/30s apart, which is the difference between 323 frames and
1293 — four times the encode, four times the tracing stills, four times the
blobs, for 970 frames that are copies of their neighbours.
It reads TIMESTAMPS, not frames: `-show_entries packet=pts_time` demuxes
without decoding, so this costs a file read and no pixels. The times are
SORTED before differencing because a stream with B-frames arrives in decode
order — an HEVC clip's first packets come out 0, 0.133, 0.067, 0.033 — and
differencing that order measures the reordering rather than the rate.
THE MEDIAN INTERVAL, which is what makes this safe on genuinely variable
input. It answers "how far apart are two frames normally", so a take held on
one frame for a second still reports the rate of the parts that move, and
choosing it keeps every distinct frame — the property `probe` used to reach
for by taking the nominal rate. Only the last few intervals of the sample are
unreliable (a frame whose turn comes after the window is missing from it), and
a median does not care.
Returning None is the honest answer for a clip too short to sample, and this
also swallows a probe that fails outright: the rate the metadata declares is
the documented fallback, so an optimisation must not be able to refuse an
upload that would otherwise have been accepted.
"""
try:
text = _command(["ffprobe", "-v", "error", "-select_streams", "v:0",
"-show_entries", "packet=pts_time", "-of", "json",
"-read_intervals", f"%+#{RATE_SAMPLE}", str(path)])
packets = json.loads(text).get("packets") or []
times = sorted(float(packet["pts_time"]) for packet in packets
if (packet.get("pts_time") or "N/A") != "N/A")
except (ValueError, OSError):
return None
intervals = sorted(b - a for a, b in zip(times, times[1:]) if b > a)
if len(intervals) < RATE_MINIMUM:
return None
median = intervals[len(intervals) // 2]
return 1.0 / median if median > 0 else None
def _choose_rate(nominal, average, measured):
"""The rate to resample onto, as an exact Fraction.
A DECLARED RATE IS PREFERRED WHEN IT AGREES WITH THE TIMESTAMPS, because it is
the exact rational the stream was authored at — 30000/1001 is not a float, and
`limit_denominator` on a measured 29.97 is a guess at a number the container
already states. So the measured rate is used to CHOOSE between what the
container declares, and only stands in itself when neither declaration
describes the stream.
"""
candidates = [rate for rate in (nominal, average) if 0 < rate <= MAX_RATE]
if measured:
agreeing = [rate for rate in candidates
if abs(float(rate) - measured) <= RATE_TOLERANCE * measured]
if agreeing:
return min(agreeing, key=lambda rate: abs(float(rate) - measured))
from_timestamps = Fraction(measured).limit_denominator(1001)
if 0 < from_timestamps <= MAX_RATE:
return from_timestamps
# Nothing to go on but the metadata, and nominal first keeps the rate that
# drops no distinct frame. An unusable pair falls through to the refusal
# below, which names the rate the file claimed rather than one of these.
return nominal if 0 < nominal <= MAX_RATE else average
def probe(path): def probe(path):
@ -178,12 +280,21 @@ def probe(path):
of itself disagreed with the container's own contents, so the guard rejected of itself disagreed with the container's own contents, so the guard rejected
CFR video for being variable. CFR video for being variable.
THE RATE IS THE NOMINAL ONE. `r_frame_rate` is the rate every timestamp in the THE RATE IS MEASURED AND THE DECLARATIONS ARE VOTED ON, which is the same
stream can be expressed at, which is the rate that keeps every distinct source distrust applied to the one number that still comes from here. This used to
frame; resampling to the average would drop some. Duration is preserved either take `r_frame_rate` outright — the rate every timestamp in the stream can be
way — ffmpeg's CFR conversion is driven by timestamps, so the audio stays in expressed at, and so the rate that keeps every distinct source frame. The
sync at any rate — so this trades a possible duplicated frame against a trouble is that it is not a claim about frames at all: the file above declares
certainly lost one. 120 and holds 30, and resampling it up cost four times the encode, four times
the tracing stills and four times the blobs for 970 duplicated frames. So
`_measured_rate` reads the timestamps, `_choose_rate` keeps whichever declared
rate they bear out, and the nominal rate is believed when it is true rather
than because it is nominal.
Duration is preserved either way — ffmpeg's CFR conversion is driven by
timestamps, so the audio stays in sync at any rate — and the median interval
keeps the no-distinct-frame-dropped property that taking the nominal rate was
reaching for. See `_measured_rate`.
""" """
data = json.loads(_command(["ffprobe", "-v", "error", "-show_streams", data = json.loads(_command(["ffprobe", "-v", "error", "-show_streams",
"-show_format", "-of", "json", str(path)])) "-show_format", "-of", "json", str(path)]))
@ -194,19 +305,25 @@ def probe(path):
average = Fraction(video.get("avg_frame_rate") or "0") average = Fraction(video.get("avg_frame_rate") or "0")
if nominal <= 0 and average <= 0: if nominal <= 0 and average <= 0:
raise ValueError("the video's frame rate is unknown") raise ValueError("the video's frame rate is unknown")
rate = nominal if 0 < nominal <= MAX_RATE else average measured = _measured_rate(path)
rate = _choose_rate(nominal, average, measured)
if not 0 < rate <= MAX_RATE: if not 0 < rate <= MAX_RATE:
raise ValueError(f"the video reports a frame rate of {float(rate):g}, which is " raise ValueError(f"the video reports a frame rate of {float(rate):g}, which is "
"not a rate footage can be measured at") "not a rate footage can be measured at")
duration = float(data.get("format", {}).get("duration") or 0) duration = float(data.get("format", {}).get("duration") or 0)
if duration > 0 and duration * float(rate) > 901: # if duration > 0 and duration * float(rate) > 901:
raise ValueError("video is longer than the 900-frame footage limit") # raise ValueError("video is longer than the 900-frame footage limit")
frames = video.get("nb_frames") frames = video.get("nb_frames")
return {"fps": float(rate), return {"fps": float(rate),
# The exact rate, for ffmpeg. 30000/1001 is not a float, and handing # The exact rate, for ffmpeg. 30000/1001 is not a float, and handing
# `-r` a rounded one is how a long take drifts out of sync. # `-r` a rounded one is how a long take drifts out of sync.
"rate": f"{rate.numerator}/{rate.denominator}", "rate": f"{rate.numerator}/{rate.denominator}",
"nominal_fps": float(nominal), "average_fps": float(average), "nominal_fps": float(nominal), "average_fps": float(average),
# What the timestamps said, and null when there were too few to ask.
# Recorded because it is the input to a decision this file used not to
# make, and the one number that explains a chosen rate matching
# neither declaration.
"measured_fps": measured,
"width": int(video["width"]), "height": int(video["height"]), "width": int(video["width"]), "height": int(video["height"]),
"duration": duration, "duration": duration,
# KEPT, AND NO LONGER TRUSTED AS A COUNT. See the docstring: this is # KEPT, AND NO LONGER TRUSTED AS A COUNT. See the docstring: this is
@ -326,8 +443,8 @@ def run(key):
proxy_facts = probe(proxy_path) proxy_facts = probe(proxy_path)
_refuse_a_shifted_timeline(proxy_path) _refuse_a_shifted_timeline(proxy_path)
frames = count_frames(proxy_path) frames = count_frames(proxy_path)
if not 1 <= frames <= 900: #if not 1 <= frames <= 900:
raise ValueError(f"the proxy holds {frames} frames; the limit is 1–900") # raise ValueError(f"the proxy holds {frames} frames; the limit is 1–900")
# CHECKED AS A DURATION, not as a frame count. The page's clock is # CHECKED AS A DURATION, not as a frame count. The page's clock is
# `frame = floor(audio.currentTime * fps)`, so what must not drift is # `frame = floor(audio.currentTime * fps)`, so what must not drift is
# how long the picture lasts against how long the audio lasts — and # how long the picture lasts against how long the audio lasts — and

View file

@ -0,0 +1,75 @@
"""Schema 2: a document holds symbols, not timelines, and no symbol is reserved.
Three renames, each in the stored transit and nowhere else:
clip/<cid>/timeline/... -> clip/<cid>/symbol/...
a node leaf's :kind :symbol -> :kind :instance
a feature leaf's :timeline key -> :symbol
A leaf value is transit's map form, ["^ ", k1, v1, k2, v2, ...]. Only TOP-LEVEL
pairs are rewritten, and only literal ones: transit caches a repeated keyword as
"^N", and a rename that met a cache reference where it expected the keyword would
be guessing. Every saved leaf at the time of writing had these as literals; if one
does not, the migration stops rather than writing a document that decodes to
something else.
Renaming a cached keyword in place is safe because the cache is positional: the
literal keeps its slot, so any later "^N" that referred to it now refers to the
new name, which is what it meant.
"""
import re
from django.db import migrations, models
PATH = re.compile(r"^(clip/[^/]+/)timeline(/|$)")
def _rename_pair(value, key, old, new, path):
if not (isinstance(value, list) and value[:1] == ["^ "]):
return value
out = list(value)
for i in range(1, len(out) - 1, 2):
if out[i] != key:
continue
if old is None:
out[i] = new
elif out[i + 1] == old:
out[i + 1] = new
elif isinstance(out[i + 1], str) and out[i + 1].startswith("^") and out[i + 1] != "^ ":
raise RuntimeError(f"leaf {path!r} has a cached {key} value; migrate it by hand")
return out
def forwards(apps, schema_editor):
Leaf = apps.get_model("clips", "Leaf")
Project = apps.get_model("clips", "Project")
for leaf in Leaf.objects.all():
path = PATH.sub(r"\1symbol\2", leaf.path)
value = leaf.value
parts = path.split("/")
if len(parts) == 6 and parts[2] == "symbol" and parts[4] == "node":
value = _rename_pair(value, "~:kind", "~:symbol", "~:instance", leaf.path)
if len(parts) == 4 and parts[2] == "feature":
value = _rename_pair(value, "~:timeline", None, "~:symbol", leaf.path)
if path != leaf.path or value != leaf.value:
leaf.path = path
leaf.value = value
leaf.version += 1
leaf.save(update_fields=["path", "value", "version"])
Project.objects.update(schema_version=2)
class Migration(migrations.Migration):
dependencies = [
("clips", "0006_project_schema_version"),
]
operations = [
migrations.AlterField(
model_name="project",
name="schema_version",
field=models.PositiveIntegerField(default=2),
),
migrations.RunPython(forwards, migrations.RunPython.noop),
]

View file

@ -0,0 +1,39 @@
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
def orphans(apps, schema_editor):
# Every project has an owner, and none of the ones saved before owners did.
apps.get_model("clips", "Project").objects.all().delete()
class Migration(migrations.Migration):
dependencies = [
("clips", "0007_symbols_not_timelines"),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.RunPython(orphans, migrations.RunPython.noop),
migrations.AddField(
model_name="leaf",
name="seq",
field=models.PositiveBigIntegerField(
default=0, help_text="the project seq of the write that last changed it"),
),
migrations.AddField(
model_name="project",
name="editors",
field=models.ManyToManyField(blank=True, related_name="shared_projects",
to=settings.AUTH_USER_MODEL),
),
migrations.AddField(
model_name="project",
name="owner",
field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE,
related_name="projects", to=settings.AUTH_USER_MODEL),
preserve_default=False,
),
]

View file

@ -0,0 +1,18 @@
# Generated by Django 5.2.17 on 2026-09-30 01:54
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('clips', '0008_owners_editors_leaf_seq'),
]
operations = [
migrations.AddField(
model_name='revision',
name='blocks',
field=models.JSONField(default=dict, help_text="each clip's tier-2 block keys, by cid, so a restore can name them"),
),
]

View file

@ -0,0 +1,25 @@
# Generated by Django 5.2.17 on 2026-09-30 07:14
import django.db.models.deletion
import uuid
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('clips', '0009_revision_blocks'),
]
operations = [
migrations.CreateModel(
name='Sound',
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True, serialize=False)),
('filename', models.CharField(max_length=255)),
('duration', models.FloatField(help_text='seconds, as ffprobe reports it')),
('created', models.DateTimeField(auto_now_add=True)),
('blob', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, related_name='sound_for', to='clips.blob')),
],
),
]

View file

@ -0,0 +1,13 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [("clips", "0010_sounds")]
operations = [
migrations.AlterField(
model_name="project",
name="schema_version",
field=models.PositiveIntegerField(default=3),
),
]

View file

@ -0,0 +1,18 @@
# Generated by Django 5.2.17 on 2026-10-01 04:38
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('clips', '0011_occurrence_schema'),
]
operations = [
migrations.AddField(
model_name='sound',
name='label',
field=models.CharField(blank=True, help_text='what a person called it; the filename when empty. Separate from `filename` because the name on disk is a fact about the upload and renaming must not rewrite it', max_length=200),
),
]

View file

@ -0,0 +1,45 @@
"""Schema 4: split animated symbol palettes from the authoring palette.
Before schema 4 a symbol's ``:palette`` leaf value was always a channel. It now
names the static palette used when that symbol is the viewed root, while the
old channel is retained as ``:palette-channel`` compatibility data. New edits
use a real lane symbol referenced by ``:palette-track``.
"""
from django.db import migrations, models
def forwards(apps, schema_editor):
Leaf = apps.get_model("clips", "Leaf")
Project = apps.get_model("clips", "Project")
for leaf in Leaf.objects.filter(path__contains="/symbol/"):
parts = leaf.path.split("/")
if len(parts) != 4 or parts[2] != "symbol":
continue
value = leaf.value
if not (isinstance(value, list) and value[:1] == ["^ "]):
continue
out = list(value)
changed = False
for i in range(1, len(out) - 1, 2):
if out[i] == "~:palette" and isinstance(out[i + 1], list):
out[i] = "~:palette-channel"
changed = True
if changed:
leaf.value = out
leaf.version += 1
leaf.save(update_fields=["value", "version"])
Project.objects.update(schema_version=4)
class Migration(migrations.Migration):
dependencies = [("clips", "0012_sound_label")]
operations = [
migrations.AlterField(
model_name="project",
name="schema_version",
field=models.PositiveIntegerField(default=4),
),
migrations.RunPython(forwards, migrations.RunPython.noop),
]

View file

@ -0,0 +1,15 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [("clips", "0013_palette_track")]
operations = [
migrations.RemoveField(model_name="clip", name="analysis"),
migrations.RemoveField(model_name="clip", name="footage"),
migrations.AlterField(
model_name="project",
name="schema_version",
field=models.PositiveIntegerField(default=5),
),
]

View file

@ -0,0 +1,48 @@
"""Schema 6: tracing is a symbol.
A face's `:head :trace` is gone: its footage is a placement of a `:type :trace`
symbol under the head, the trace keys are that placement's `:time :holds`, and the
head follows them with `:reads`. Nothing is converted. Every project is marked 6,
and one that still carries a `:trace` is refused when it is opened, by name and
with what to do about it; the rest open as they did.
Images are stills to trace over, stored like sounds.
"""
import uuid
from django.db import migrations, models
import django.db.models.deletion
def forwards(apps, schema_editor):
apps.get_model("clips", "Project").objects.update(schema_version=6)
class Migration(migrations.Migration):
dependencies = [("clips", "0014_multiple_analyses")]
operations = [
migrations.CreateModel(
name="Image",
fields=[
("id", models.UUIDField(default=uuid.uuid4, editable=False,
primary_key=True, serialize=False)),
("filename", models.CharField(max_length=255)),
("label", models.CharField(
blank=True, max_length=200,
help_text="what a person called it; the filename when empty")),
("width", models.PositiveIntegerField(help_text="pixels, as ffprobe reports them")),
("height", models.PositiveIntegerField()),
("created", models.DateTimeField(auto_now_add=True)),
("blob", models.ForeignKey(on_delete=django.db.models.deletion.PROTECT,
related_name="image_for", to="clips.blob")),
],
),
migrations.AlterField(
model_name="project",
name="schema_version",
field=models.PositiveIntegerField(default=6),
),
migrations.RunPython(forwards, migrations.RunPython.noop),
]

View file

@ -0,0 +1,42 @@
"""Schema 7: an anchor is a peg.
`[:xform :anchor]` is gone from the transform. `T(a)·M·T(-a)` is a transform
conjugated by a translation — "do M in a frame shifted by a" — and a parent
already is a shifted frame, so an anchor was a peg written inline: one that could
not be selected, keyed, shared between nodes, or placed above a measured channel.
A pivot nobody chose is now derived from what the node draws, per drag, and stored
nowhere; a pivot to keep is a peg, an ordinary `:group` parent.
Nothing is converted, as in schema 6. Every project is marked 7, and one that
still carries an anchor is refused when it is opened, by name and with what to do
about it — `node/problems` in the frontend.
Not converted rather than not worth converting. Dropping an anchor is in fact
pixel-exact wherever rotation and scale are the identity, since the anchor
cancels out of the composition there — and that is everywhere a freeze, a drop or
a new drawing wrote one. It is NOT exact on anything a hand has since turned or
scaled, where the composed translation is `a + p - M·a`, and it cannot be made
exact at all where `pos` is dense, because tier 2 is content-addressed and not
rewritable here. A conversion would therefore be silent and right for most nodes
and silent and wrong for exactly the ones somebody had hand-placed, which is the
worse failure: a refusal names the document and says what to do.
"""
from django.db import migrations, models
def forwards(apps, schema_editor):
apps.get_model("clips", "Project").objects.update(schema_version=7)
class Migration(migrations.Migration):
dependencies = [("clips", "0015_tracing_images")]
operations = [
migrations.AlterField(
model_name="project",
name="schema_version",
field=models.PositiveIntegerField(default=7),
),
migrations.RunPython(forwards, migrations.RunPython.noop),
]

View file

@ -0,0 +1,45 @@
"""Schema 8: a node has a pivot.
`[:xform :pivot]` is back in the transform, as the point rotation and scale are
composed about: `local = T(pos)·T(piv)·R·K·S·T(-piv)`. Schema 7 deleted it, on
the argument that an anchor is a peg — true as algebra, and not true as a feature.
A peg is a node, and a turn about a point that is not the turning node's own
origin still has to solve for a position to hold that point still; that solution
is an arc in the angle while a position channel tweens along the chord, so it is
right on the frame it is written and wrong on every frame between two keys. A
drawing escaped it, since its origin is the middle of what it draws. A symbol
instance could not: its origin is its symbol's, which is the top-left corner of
the stage, so one keyed turn of an instance swung its drawing round that corner
on an orbit the size of the stage.
CONVERTED, unlike 6 and 7, because adding this one is exact. A schema-7 node has
no pivot; an absent pivot reads as [0 0]; and T(pos)·T(0)·M·T(-0) is T(pos)·M to
the last bit of the mantissa. Every stored document therefore composes to exactly
the matrices it composed to before, dense tier-2 transforms included, so there is
nothing to guess at and no node a conversion could silently move. The version is
restamped and nothing else is touched.
What a converted document does NOT get is a pivot somebody chose: nodes placed
before this carry none, so they still turn about their own origin until the first
turn or scale writes one — `gesture/with-pivot`, from the middle of what the node
draws at that moment — or until the cross is dragged (ctrl/cmd-drag on the stage).
"""
from django.db import migrations, models
def forwards(apps, schema_editor):
apps.get_model("clips", "Project").objects.update(schema_version=8)
class Migration(migrations.Migration):
dependencies = [("clips", "0016_an_anchor_is_a_peg")]
operations = [
migrations.AlterField(
model_name="project",
name="schema_version",
field=models.PositiveIntegerField(default=8),
),
migrations.RunPython(forwards, migrations.RunPython.noop),
]

View file

@ -24,7 +24,9 @@ without parsing its leaves: which footage, which analysis, which blocks.
""" """
import uuid import uuid
from django.conf import settings
from django.db import models from django.db import models
from django.utils import timezone
class Blob(models.Model): class Blob(models.Model):
@ -50,6 +52,39 @@ class Source(models.Model):
created = models.DateTimeField(auto_now_add=True) created = models.DateTimeField(auto_now_add=True)
class Sound(models.Model):
"""An uploaded sound file — mp3, wav, whatever the browser can decode — kept
as uploaded. Not footage: it has no frames and nothing measures it, so it
skips extraction and an audio node plays its bytes directly."""
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
blob = models.ForeignKey(Blob, on_delete=models.PROTECT, related_name="sound_for")
filename = models.CharField(max_length=255)
label = models.CharField(
max_length=200, blank=True,
help_text="what a person called it; the filename when empty. Separate "
"from `filename` because the name on disk is a fact about the "
"upload and renaming must not rewrite it",
)
duration = models.FloatField(help_text="seconds, as ffprobe reports it")
created = models.DateTimeField(auto_now_add=True)
class Image(models.Model):
"""An uploaded still — a drawing, a photo, a model sheet — kept as uploaded, to
be traced over. Never part of the picture: a document names its blob as a
tracing symbol's `:media`, and the page draws it over the stage."""
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
blob = models.ForeignKey(Blob, on_delete=models.PROTECT, related_name="image_for")
filename = models.CharField(max_length=255)
label = models.CharField(max_length=200, blank=True,
help_text="what a person called it; the filename when empty")
width = models.PositiveIntegerField(help_text="pixels, as ffprobe reports them")
height = models.PositiveIntegerField()
created = models.DateTimeField(auto_now_add=True)
class Extraction(models.Model): class Extraction(models.Model):
"""One requested decode of a source into immutable footage.""" """One requested decode of a source into immutable footage."""
@ -201,12 +236,21 @@ class Project(models.Model):
`schema_version` identifies the stored document format. `seq` counts writes `schema_version` identifies the stored document format. `seq` counts writes
to this particular project; it is not a format version. Every write bumps to this particular project; it is not a format version. Every write bumps
`seq`, and a client that sees `seq > local + 1` refetches once broadcasts exist. `seq`, and a client that sees `seq > local + 1` refetches.
ANYONE WITH THE LINK CAN VIEW; the owner and the editors can write. Every
project has an owner.
""" """
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
owner = models.ForeignKey(
settings.AUTH_USER_MODEL, on_delete=models.CASCADE, related_name="projects",
)
editors = models.ManyToManyField(
settings.AUTH_USER_MODEL, blank=True, related_name="shared_projects",
)
name = models.CharField(max_length=200, default="untitled") name = models.CharField(max_length=200, default="untitled")
schema_version = models.PositiveIntegerField(default=1) schema_version = models.PositiveIntegerField(default=8)
seq = models.PositiveBigIntegerField(default=0) seq = models.PositiveBigIntegerField(default=0)
palette = models.CharField(max_length=64, default="arthur/default") palette = models.CharField(max_length=64, default="arthur/default")
created = models.DateTimeField(auto_now_add=True) created = models.DateTimeField(auto_now_add=True)
@ -219,10 +263,19 @@ class Project(models.Model):
return f"{self.name} ({self.id})" return f"{self.name} ({self.id})"
def bump(self): def bump(self):
self.seq += 1 """The next seq, taken with an UPDATE so that inside a transaction it is
self.save(update_fields=["seq", "updated"]) also the write lock: two concurrent saves cannot both get the same one."""
Project.objects.filter(id=self.id).update(
seq=models.F("seq") + 1, updated=timezone.now()
)
self.refresh_from_db(fields=["seq", "updated"])
return self.seq return self.seq
def can_edit(self, user):
return user.is_authenticated and (
user.id == self.owner_id or self.editors.filter(id=user.id).exists()
)
class Clip(models.Model): class Clip(models.Model):
"""Tier 1: the unit of work, and the thing leaf paths are scoped by. """Tier 1: the unit of work, and the thing leaf paths are scoped by.
@ -235,12 +288,6 @@ class Clip(models.Model):
cid = models.SlugField(max_length=64) cid = models.SlugField(max_length=64)
name = models.CharField(max_length=200, blank=True) name = models.CharField(max_length=200, blank=True)
order = models.IntegerField(default=0) order = models.IntegerField(default=0)
footage = models.ForeignKey(
Footage, null=True, blank=True, on_delete=models.SET_NULL, related_name="clips"
)
analysis = models.ForeignKey(
Analysis, null=True, blank=True, on_delete=models.SET_NULL, related_name="clips"
)
blocks = models.ManyToManyField( blocks = models.ManyToManyField(
Block, blank=True, related_name="clips", Block, blank=True, related_name="clips",
help_text="the tier-2 blocks this clip's channels name", help_text="the tier-2 blocks this clip's channels name",
@ -271,6 +318,9 @@ class Leaf(models.Model):
path = models.CharField(max_length=300) path = models.CharField(max_length=300)
value = models.JSONField() value = models.JSONField()
version = models.PositiveBigIntegerField(default=1) version = models.PositiveBigIntegerField(default=1)
seq = models.PositiveBigIntegerField(
default=0, help_text="the project seq of the write that last changed it",
)
updated = models.DateTimeField(auto_now=True) updated = models.DateTimeField(auto_now=True)
class Meta: class Meta:
@ -288,7 +338,9 @@ class Leaf(models.Model):
class Revision(models.Model): class Revision(models.Model):
"""Tier 1: a snapshot of the authored layer, with a user and a summary. """Tier 1: a snapshot of the authored layer, with a user and a summary — a
named snapshot, which is how a person marks a version now that every edit
saves itself.
ON AN EXPLICIT TRIGGER, not on every save. tl snapshots a small annotation ON AN EXPLICIT TRIGGER, not on every save. tl snapshots a small annotation
layer; arthur's tier 1 will contain cel polygons, so a snapshot per save bloats layer; arthur's tier 1 will contain cel polygons, so a snapshot per save bloats
@ -302,6 +354,9 @@ class Revision(models.Model):
author = models.CharField(max_length=200, blank=True) author = models.CharField(max_length=200, blank=True)
summary = models.CharField(max_length=500, blank=True) summary = models.CharField(max_length=500, blank=True)
document = models.JSONField(help_text="every leaf of the project, by path") document = models.JSONField(help_text="every leaf of the project, by path")
blocks = models.JSONField(
default=dict, help_text="each clip's tier-2 block keys, by cid, so a restore can name them",
)
created = models.DateTimeField(auto_now_add=True) created = models.DateTimeField(auto_now_add=True)
class Meta: class Meta:

7
clips/routing.py Normal file
View file

@ -0,0 +1,7 @@
from django.urls import path
from .consumers import ProjectConsumer
websocket_urlpatterns = [
path("ws/projects/<uuid:project_id>", ProjectConsumer.as_asgi()),
]

View file

@ -2,11 +2,14 @@
{% comment %} {% comment %}
The host page, served by Django since port-plan step 9. The host page, served by Django since port-plan step 9.
It was `frontend/public/index.html`, served by shadow-cljs's `:dev-http`, and that It carries no styles of its own any more. They are `static/arthur/app.css`, which
key is gone. The bundle is unchanged: shadow-cljs writes it into staticfiles serves from the same tree as the bundle — the page grew a five-pane
`static/arthur/js` and staticfiles serves it from there, so `manage.py runserver` application chrome and "the styles" stopped being a thing you read in passing on
and `shadow-cljs watch app` are the whole dev loop with nothing copying files the way to the markup.
between them.
The bundle is unchanged: shadow-cljs writes it into `static/arthur/js` and
staticfiles serves it from there, so `manage.py runserver` and `shadow-cljs watch
app` are the whole dev loop with nothing copying files between them.
The CSRF token is rendered so that Django sets its cookie, which is what The CSRF token is rendered so that Django sets its cookie, which is what
`arthur.fx.http` reads to write the `X-CSRFToken` header. Saves are ordinary POSTs `arthur.fx.http` reads to write the `X-CSRFToken` header. Saves are ordinary POSTs
@ -17,72 +20,12 @@ and PUTs with ordinary CSRF protection — no endpoint in this app is exempt.
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title>arthur</title> <title>arthur</title>
<style> <link rel="stylesheet" href="{% static 'arthur/app.css' %}?v={{ css_version }}">
:root { color-scheme: dark; --bg: #12141c; --fg: #c9c3b4; }
html, body { margin: 0; height: 100%; background: var(--bg); color: var(--fg); }
body { font: 14px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace; }
main { padding: 24px; }
/* The preview is nearest-neighbour everywhere. A browser that smooths the
upscale would misrepresent the look the tool exists to judge. */
canvas { image-rendering: pixelated; }
h1 { font-size: 14px; font-weight: normal; opacity: .5; margin: 0 0 12px; }
.stage { display: block; background: #12141c; }
.stage-wrap { position: relative; width: fit-content; }
.paint-overlay { position: absolute; inset: 0; touch-action: none; }
.paint-overlay circle { cursor: grab; }
.paint-tools { width: 640px; margin-top: 9px; font-size: 12px; }
.paint-tools .row { display: flex; flex-wrap: wrap; align-items: center; gap: 6px; margin: 4px 0; }
.paint-tools select { color: var(--fg); background: #1c1f2b; border: 1px solid #2b3040; font: inherit; }
.paint-tools .hint { color: #d0ba86; opacity: .8; }
audio { display: none; }
.transport { margin-top: 12px; width: 640px; }
.transport .row { display: flex; flex-wrap: wrap; gap: 6px; align-items: center; }
.transport .gap { flex: 1; }
button {
font: inherit; color: var(--fg); background: #1c1f2b;
border: 1px solid #2b3040; padding: 3px 10px; cursor: pointer;
}
button:hover { background: #242836; }
button:disabled { opacity: .45; cursor: wait; }
button.on { background: #3a4258; border-color: #556080; }
.scrub { width: 100%; margin: 10px 0 6px; }
.readout { display: flex; gap: 18px; opacity: .55; font-size: 12px; }
.readout .warn { color: #d98f5a; opacity: 1; }
.picture-rate { display: flex; align-items: center; gap: 6px; margin-top: 7px;
font-size: 12px; }
.source-path { display: block; margin-top: 8px; font-size: 12px; opacity: .7; }
.source-path select { margin: 0 8px; padding: 3px 5px;
color: var(--fg); background: #1c1f2b; border: 1px solid #2b3040;
font: inherit; max-width: 360px; }
.load-status { margin-top: 6px; font-size: 12px; opacity: .75; }
.export { width: 640px; margin-top: 14px; padding-top: 12px;
border-top: 1px solid #2b3040; font-size: 12px; }
.export .row { display: flex; flex-wrap: wrap; gap: 6px; align-items: center; }
.export .gap { flex: 1; }
.export select { margin-left: 6px; padding: 3px 5px; color: var(--fg);
background: #1c1f2b; border: 1px solid #2b3040; font: inherit; }
.export .readout { margin-top: 7px; }
.export .note { margin: 7px 0 0; }
.controls { width: 640px; margin-top: 18px; padding-top: 12px;
border-top: 1px solid #2b3040; font-size: 12px; }
.controls select { margin-left: 8px; padding: 3px 5px; color: var(--fg);
background: #1c1f2b; border: 1px solid #2b3040; font: inherit; }
.shared-note { margin-top: 6px; color: #d0ba86; }
.control-list { display: grid; grid-template-columns: 1fr 1fr; gap: 6px 16px;
margin-top: 10px; }
.control-row { display: grid; grid-template-columns: 115px 1fr 42px;
align-items: center; gap: 6px; }
.control-row input { width: 100%; }
.control-row output { text-align: right; }
.regeneration-debug { padding: 8px; margin-top: 10px; background: #1c1f2b;
white-space: pre-wrap; color: #d0ba86; }
.note { opacity: .35; font-size: 12px; max-width: 640px; }
</style>
</head> </head>
<body> <body>
{% csrf_token %} {% csrf_token %}
<div id="app"></div> <div id="app"></div>
<script src="{% static 'mediapipe/vision_bundle.js' %}"></script> <script src="{% static 'mediapipe/vision_bundle.js' %}"></script>
<script src="{% static 'arthur/js/main.js' %}"></script> <script src="{% static 'arthur/js/main.js' %}?v={{ js_version }}"></script>
</body> </body>
</html> </html>

View file

@ -34,7 +34,7 @@ from django.core.management import call_command
from django.test import TestCase, override_settings from django.test import TestCase, override_settings
from clips import blobs, extraction from clips import blobs, extraction
from clips.models import Analysis, Block, Blob, Clip, Footage, Leaf, Project, Revision, Source from clips.models import Analysis, Block, Blob, Clip, Footage, Image, Leaf, Project, Revision, Sound, Source
BLOB_DIR = tempfile.mkdtemp(prefix="arthur-test-blobs-") BLOB_DIR = tempfile.mkdtemp(prefix="arthur-test-blobs-")
@ -361,7 +361,10 @@ class DocumentTests(TestCase):
"""Tier 1: load, save, and the conditional write.""" """Tier 1: load, save, and the conditional write."""
def setUp(self): def setUp(self):
self.project = Project.objects.create(name="a project") from django.contrib.auth import get_user_model
owner = get_user_model().objects.create_user("owner", password="password1")
self.client.force_login(owner)
self.project = Project.objects.create(name="a project", owner=owner)
descriptor = analysis_descriptor() descriptor = analysis_descriptor()
self.analysis = key_for(descriptor) self.analysis = key_for(descriptor)
self.client.post("/api/analyses", data=json.dumps( self.client.post("/api/analyses", data=json.dumps(
@ -382,37 +385,73 @@ class DocumentTests(TestCase):
# cache marker, keyword keys, and a frame-keyed inner map. # cache marker, keyword keys, and a frame-keyed inner map.
return { return {
"clip/c1/timing": ["^ ", "~:fps", 30], "clip/c1/timing": ["^ ", "~:fps", 30],
"clip/c1/timeline/main": ["^ ", "~:frames", 48], "clip/c1/symbol/main": ["^ ", "~:frames", 48],
"clip/c1/timeline/main/node/mouth": ["^ ", "~:id", "~:mouth", "~:z", "a1"], "clip/c1/symbol/main/node/mouth": ["^ ", "~:id", "~:mouth", "~:z", "a1"],
"clip/c1/timeline/main/channel/mouth/geom.pts": [ "clip/c1/symbol/main/channel/mouth/geom.pts": [
"^ ", "~:animated?", True, "~:dense", "^ ", "~:animated?", True, "~:dense",
["^ ", "~:store", self.block, "~:offset", 0, "~:stride", 16], ["^ ", "~:store", self.block, "~:offset", 0, "~:stride", 16],
], ],
"clip/c1/timeline/main/channel/mouth-in/vis": [ "clip/c1/symbol/main/channel/mouth-in/vis": [
"^ ", "~:animated?", True, "~:keys", ["^ ", "~i0", True, "~i12", False], "^ ", "~:animated?", True, "~:keys", ["^ ", "~i0", True, "~i12", False],
], ],
} }
def save(self, leaves=None, blocks=None): def save(self, leaves=None, blocks=None, analyses=None):
return self.put(f"/api/projects/{self.project.id}", { return self.put(f"/api/projects/{self.project.id}", {
"name": "a project", "name": "a project",
"clips": [{"cid": "c1", "name": "take", "analysis": self.analysis, "clips": [{"cid": "c1", "name": "take",
"analyses": [self.analysis] if analyses is None else analyses,
"leaves": leaves if leaves is not None else self.leaves(), "leaves": leaves if leaves is not None else self.leaves(),
"blocks": blocks if blocks is not None else [self.block]}], "blocks": blocks if blocks is not None else [self.block]}],
}) })
def test_a_clip_declares_the_registered_analyses_its_blocks_name(self):
undeclared = self.save(analyses=[])
self.assertEqual(409, undeclared.status_code)
self.assertIn("every block", undeclared.json()["error"])
unknown = "sha256:" + "f" * 64
missing = self.save(analyses=[self.analysis, unknown])
self.assertEqual(409, missing.status_code)
self.assertEqual([unknown], missing.json()["missing"])
def test_every_saved_symbol_is_listed_across_projects(self):
leaves = self.leaves()
leaves["clip/c1/symbol/sym~face"] = ["^ ", "~:name", "face", "~:frames", 12]
leaves["clip/c1/symbol/sym~face/node/mark"] = ["^ ", "~:id", "~:mark", "~:z", "a1"]
self.assertEqual(200, self.save(leaves).status_code)
rows = self.client.get("/api/symbols").json()["symbols"]
self.assertEqual(
[("face", "sym~face", 12), ("main", "main", 48)],
[(r["name"], r["symbol"], r["frames"]) for r in rows])
self.assertEqual({str(self.project.id)}, {r["project"] for r in rows})
self.assertEqual({"c1"}, {r["cid"] for r in rows})
def test_saved_palettes_are_listed_as_assets(self):
leaves = self.leaves()
leaves["clip/c1/palette/night"] = [
"^ ", "~:id", "~:night", "~:name", "Moonlit",
"~:slots", ["~#list", [["^ ", "~:hex", "#001122"]]],
]
self.assertEqual(200, self.save(leaves).status_code)
rows = self.client.get("/api/symbols").json()["palettes"]
self.assertEqual(
[("Moonlit", "night", "c1")],
[(r["name"], r["palette"], r["cid"]) for r in rows],
)
def test_a_document_comes_back_exactly(self): def test_a_document_comes_back_exactly(self):
response = self.save() response = self.save()
self.assertEqual(200, response.status_code, response.content) self.assertEqual(200, response.status_code, response.content)
self.assertEqual(5, len(response.json()["written"])) self.assertEqual(5, len(response.json()["written"]))
loaded = self.client.get(f"/api/projects/{self.project.id}").json() loaded = self.client.get(f"/api/projects/{self.project.id}").json()
self.assertEqual(1, loaded["schema_version"]) self.assertEqual(7, loaded["schema_version"])
self.assertEqual(1, len(loaded["clips"])) self.assertEqual(1, len(loaded["clips"]))
clip = loaded["clips"][0] clip = loaded["clips"][0]
self.assertEqual("c1", clip["cid"]) self.assertEqual("c1", clip["cid"])
self.assertEqual([self.block], clip["blocks"]) self.assertEqual([self.block], clip["blocks"])
self.assertEqual(self.analysis, clip["analysis"]) self.assertNotIn("analysis", clip)
# The whole point: byte-identical values, including the integer frame keys # The whole point: byte-identical values, including the integer frame keys
# transit writes as "~i0". A JSON round trip that stringified them would # transit writes as "~i0". A JSON round trip that stringified them would
# come back "0" and the part would hold its first pose forever. # come back "0" and the part would hold its first pose forever.
@ -424,22 +463,22 @@ class DocumentTests(TestCase):
self.save() self.save()
first = {leaf.path: leaf.version for leaf in Leaf.objects.all()} first = {leaf.path: leaf.version for leaf in Leaf.objects.all()}
moved = self.leaves() moved = self.leaves()
moved["clip/c1/timeline/main/channel/mouth-in/vis"] = [ moved["clip/c1/symbol/main/channel/mouth-in/vis"] = [
"^ ", "~:animated?", True, "~:keys", ["^ ", "~i0", False], "^ ", "~:animated?", True, "~:keys", ["^ ", "~i0", False],
] ]
response = self.save(moved) response = self.save(moved)
self.assertEqual(["clip/c1/timeline/main/channel/mouth-in/vis"], response.json()["written"]) self.assertEqual(["clip/c1/symbol/main/channel/mouth-in/vis"], response.json()["written"])
self.assertEqual(4, response.json()["unchanged"]) self.assertEqual(4, response.json()["unchanged"])
after = {leaf.path: leaf.version for leaf in Leaf.objects.all()} after = {leaf.path: leaf.version for leaf in Leaf.objects.all()}
self.assertEqual(2, after["clip/c1/timeline/main/channel/mouth-in/vis"]) self.assertEqual(2, after["clip/c1/symbol/main/channel/mouth-in/vis"])
self.assertEqual(first["clip/c1/timing"], after["clip/c1/timing"]) self.assertEqual(first["clip/c1/timing"], after["clip/c1/timing"])
def test_a_removed_node_removes_its_leaf(self): def test_a_removed_node_removes_its_leaf(self):
self.save() self.save()
fewer = {k: v for k, v in self.leaves().items() fewer = {k: v for k, v in self.leaves().items()
if k != "clip/c1/timeline/main/node/mouth"} if k != "clip/c1/symbol/main/node/mouth"}
response = self.save(fewer) response = self.save(fewer)
self.assertEqual(["clip/c1/timeline/main/node/mouth"], response.json()["removed"]) self.assertEqual(["clip/c1/symbol/main/node/mouth"], response.json()["removed"])
self.assertEqual(4, Leaf.objects.count()) self.assertEqual(4, Leaf.objects.count())
def test_a_save_does_not_disturb_another_clip(self): def test_a_save_does_not_disturb_another_clip(self):
@ -472,7 +511,7 @@ class DocumentTests(TestCase):
def test_a_leaf_write_carries_an_etag(self): def test_a_leaf_write_carries_an_etag(self):
self.save() self.save()
url = f"/api/projects/{self.project.id}/leaves/clip/c1/timeline/main/node/mouth" url = f"/api/projects/{self.project.id}/leaves/clip/c1/symbol/main/node/mouth"
got = self.client.get(url) got = self.client.get(url)
self.assertEqual('"1"', got["ETag"]) self.assertEqual('"1"', got["ETag"])
@ -488,7 +527,7 @@ class DocumentTests(TestCase):
# take-theirs. A PUT that replaced unconditionally is the bug where the # take-theirs. A PUT that replaced unconditionally is the bug where the
# loser's work disappears silently. # loser's work disappears silently.
self.save() self.save()
url = f"/api/projects/{self.project.id}/leaves/clip/c1/timeline/main/node/mouth" url = f"/api/projects/{self.project.id}/leaves/clip/c1/symbol/main/node/mouth"
self.put(url, {"value": ["^ ", "~:z", "a2"]}, HTTP_IF_MATCH='"1"') self.put(url, {"value": ["^ ", "~:z", "a2"]}, HTTP_IF_MATCH='"1"')
stale = self.put(url, {"value": ["^ ", "~:z", "a3"]}, HTTP_IF_MATCH='"1"') stale = self.put(url, {"value": ["^ ", "~:z", "a3"]}, HTTP_IF_MATCH='"1"')
self.assertEqual(409, stale.status_code) self.assertEqual(409, stale.status_code)
@ -511,6 +550,61 @@ class DocumentTests(TestCase):
# --- revisions --------------------------------------------------------- # --- revisions ---------------------------------------------------------
def patch(self, base, leaves, removed=()):
return self.put(f"/api/projects/{self.project.id}", {
"base": base,
"clips": [{"cid": "c1", "analyses": [self.analysis], "leaves": leaves,
"removed": list(removed), "blocks": [self.block]}],
})
def test_a_patch_leaves_what_it_does_not_name_alone(self):
seq = self.save().json()["seq"]
response = self.patch(seq, {"clip/c1/timing": ["^ ", "~:fps", 24]},
removed=["clip/c1/symbol/main/node/mouth"])
self.assertEqual(200, response.status_code, response.content)
self.assertEqual(["clip/c1/timing"], response.json()["written"])
self.assertEqual(4, Leaf.objects.count())
def test_two_people_on_different_leaves_both_land(self):
seq = self.save().json()["seq"]
self.assertEqual(200, self.patch(seq, {"clip/c1/timing": ["^ ", "~:fps", 24]}).status_code)
# The second saver has not caught up, and touched a different leaf.
response = self.patch(seq, {"clip/c1/symbol/main": ["^ ", "~:frames", 12]})
self.assertEqual(200, response.status_code, response.content)
leaves = self.client.get(f"/api/projects/{self.project.id}").json()["clips"][0]["leaves"]
self.assertEqual(["^ ", "~:fps", 24], leaves["clip/c1/timing"])
self.assertEqual(["^ ", "~:frames", 12], leaves["clip/c1/symbol/main"])
def test_two_people_on_one_leaf_is_a_conflict_that_writes_nothing(self):
seq = self.save().json()["seq"]
self.patch(seq, {"clip/c1/timing": ["^ ", "~:fps", 24]})
response = self.patch(seq, {"clip/c1/timing": ["^ ", "~:fps", 12],
"clip/c1/symbol/main": ["^ ", "~:frames", 12]})
self.assertEqual(409, response.status_code)
self.assertEqual({"clip/c1/timing": ["^ ", "~:fps", 24]}, response.json()["conflicts"])
self.assertEqual(seq + 1, Project.objects.get(id=self.project.id).seq)
self.assertEqual(["^ ", "~:frames", 48],
Leaf.objects.get(path="clip/c1/symbol/main").value)
# Caught up to their seq, the same write is ordinary.
self.assertEqual(200, self.patch(seq + 1, {"clip/c1/timing": ["^ ", "~:fps", 12]}).status_code)
def test_a_named_snapshot_restores_as_an_ordinary_write(self):
self.save()
snap = self.client.post(f"/api/projects/{self.project.id}/revisions",
data=json.dumps({"summary": "before the big change"}),
content_type="application/json").json()
moved = self.leaves()
moved["clip/c1/timing"] = ["^ ", "~:fps", 12]
del moved["clip/c1/symbol/main/node/mouth"]
self.save(moved)
listed = self.client.get(f"/api/projects/{self.project.id}/revisions").json()["revisions"]
self.assertEqual(["before the big change"], [r["summary"] for r in listed])
restored = self.client.post(
f"/api/projects/{self.project.id}/revisions/{snap['id']}/restore").json()
self.assertEqual(2, restored["changed"])
leaves = self.client.get(f"/api/projects/{self.project.id}").json()["clips"][0]["leaves"]
self.assertEqual(self.leaves(), leaves)
def test_a_revision_snapshots_the_authored_layer(self): def test_a_revision_snapshots_the_authored_layer(self):
self.save() self.save()
response = self.client.post( response = self.client.post(
@ -587,6 +681,64 @@ class FootageTests(TestCase):
with self.assertRaisesMessage(CommandError, "refusing an inaccurate footage"): with self.assertRaisesMessage(CommandError, "refusing an inaccurate footage"):
call_command("ingest_bundle", str(root), stdout=StringIO()) call_command("ingest_bundle", str(root), stdout=StringIO())
def test_footage_can_be_renamed_and_falls_back_when_cleared(self):
"""A LABEL IS THE ONE FIELD A CLIENT MAY WRITE ON FOOTAGE. The rest is a
description of bytes that are content-addressed and immutable, so a
rename that could reach `frames` or `digest` would let the pool's name
for a clip contradict the clip."""
footage = self.ingest(self.bundle())
self.assertEqual("IMG_8608.MOV", self.client.get(
f"/api/footage/{footage.id}").json()["label"])
renamed = self.client.patch(f"/api/footage/{footage.id}",
json.dumps({"label": " the long take "}),
content_type="application/json")
self.assertEqual(200, renamed.status_code, renamed.content)
self.assertEqual("the long take", renamed.json()["label"])
# On the row, so every project listing this footage sees the new name.
footage.refresh_from_db()
self.assertEqual("the long take", footage.label)
self.assertEqual("the long take",
self.client.get("/api/footage").json()["footage"][0]["label"])
# Cleared gives back the name it was ingested under rather than nothing.
cleared = self.client.patch(f"/api/footage/{footage.id}",
json.dumps({"label": ""}),
content_type="application/json")
self.assertEqual("IMG_8608.MOV", cleared.json()["label"])
self.assertEqual(3, Footage.objects.get().frames)
def test_a_rename_that_names_no_label_is_refused(self):
footage = self.ingest(self.bundle())
refused = self.client.patch(f"/api/footage/{footage.id}",
json.dumps({"frames": 900}),
content_type="application/json")
self.assertEqual(400, refused.status_code)
self.assertEqual("a rename needs a label", refused.json()["error"])
self.assertEqual(3, Footage.objects.get().frames)
def test_a_sound_is_renamed_without_losing_the_name_it_arrived_as(self):
"""`filename` is a fact about the upload and `label` is what a person
called it, which is why renaming does not write over the first one."""
digest, size = blobs.write_stream([b"RIFF....WAVEfmt "])
blob = Blob.objects.create(digest=digest, size=size, media_type="audio/wav")
sound = Sound.objects.create(blob=blob, filename="rec0012.wav", duration=2.5)
renamed = self.client.patch(f"/api/sounds/{sound.id}",
json.dumps({"label": "arthur, line 4"}),
content_type="application/json")
self.assertEqual(200, renamed.status_code, renamed.content)
self.assertEqual("arthur, line 4", renamed.json()["label"])
self.assertEqual("rec0012.wav", renamed.json()["filename"])
sound.refresh_from_db()
self.assertEqual("rec0012.wav", sound.filename)
self.assertEqual("arthur, line 4", sound.label)
self.client.patch(f"/api/sounds/{sound.id}", json.dumps({"label": " "}),
content_type="application/json")
self.assertEqual("rec0012.wav",
self.client.get(f"/api/sounds/{sound.id}").json()["label"])
def test_the_footage_list_does_not_carry_every_url(self): def test_the_footage_list_does_not_carry_every_url(self):
# A list of takes should not be a list of six hundred URLs each. # A list of takes should not be a list of six hundred URLs each.
self.ingest(self.bundle()) self.ingest(self.bundle())
@ -651,6 +803,62 @@ class UploadTests(TestCase):
self.assertEqual(27, job.progress) self.assertEqual(27, job.progress)
job.save.assert_called_once_with(update_fields=["progress", "updated"]) job.save.assert_called_once_with(update_fields=["progress", "updated"])
def test_an_uploaded_mp3_is_a_sound_and_not_footage(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "tone.mp3"
subprocess.run([
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-f", "lavfi", "-i", "sine=frequency=440:duration=1.5", str(path),
], check=True, capture_output=True)
payload = path.read_bytes()
uploaded = self.client.post("/api/sounds", {
"file": SimpleUploadedFile("tone.mp3", payload, content_type="audio/mpeg")})
self.assertEqual(201, uploaded.status_code, uploaded.content)
sound = uploaded.json()
self.assertEqual("tone.mp3", sound["label"])
self.assertAlmostEqual(1.5, sound["duration"], delta=0.1)
self.assertEqual(payload, b"".join(self.client.get(sound["audio"]).streaming_content))
self.assertEqual(sound, self.client.get(f"/api/sounds/{sound['id']}").json())
self.assertEqual([sound], self.client.get("/api/sounds").json()["sounds"])
self.assertEqual(0, Source.objects.count())
again = self.client.post("/api/sounds", {
"file": SimpleUploadedFile("again.mp3", payload, content_type="audio/mpeg")})
self.assertEqual(200, again.status_code)
self.assertEqual(1, Sound.objects.count())
def test_an_uploaded_still_is_an_image_named_by_its_bytes(self):
payload = png(17, 5)
uploaded = self.client.post("/api/images", {
"file": SimpleUploadedFile("sheet.png", payload, content_type="image/png")})
self.assertEqual(201, uploaded.status_code, uploaded.content)
image = uploaded.json()
self.assertEqual(("sheet.png", 17, 5), (image["label"], image["width"], image["height"]))
self.assertEqual(f"/blob/{image['digest']}", image["url"])
self.assertEqual(payload, b"".join(self.client.get(image["url"]).streaming_content))
self.assertEqual([image], self.client.get("/api/images").json()["images"])
again = self.client.post("/api/images", {
"file": SimpleUploadedFile("again.png", payload, content_type="image/png")})
self.assertEqual(200, again.status_code)
self.assertEqual(1, Image.objects.count())
renamed = self.client.patch(f"/api/images/{image['id']}",
json.dumps({"label": "model sheet"}),
content_type="application/json")
self.assertEqual("model sheet", renamed.json()["label"])
def test_a_file_that_is_not_a_picture_is_not_an_image(self):
refused = self.client.post("/api/images", {
"file": SimpleUploadedFile("notes.txt", b"not a picture", content_type="text/plain")})
self.assertEqual(400, refused.status_code)
def test_a_file_without_audio_is_not_a_sound(self):
refused = self.client.post("/api/sounds", {
"file": SimpleUploadedFile("notes.txt", b"not audio", content_type="text/plain")})
self.assertEqual(400, refused.status_code)
def test_uploaded_video_extracts_to_reopenable_footage(self): def test_uploaded_video_extracts_to_reopenable_footage(self):
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "four-frames.mp4" path = Path(directory) / "four-frames.mp4"
@ -700,6 +908,37 @@ class UploadTests(TestCase):
self.assertEqual("image/jpeg", still["Content-Type"]) self.assertEqual("image/jpeg", still["Content-Type"])
self.assertEqual(200, self.client.get(footage["audio"]).status_code) self.assertEqual(200, self.client.get(footage["audio"]).status_code)
def test_re_uploading_a_source_re_reads_its_facts(self):
# A SOURCE ROW HOLDS A READING, NOT A DECISION. The facts are a pure
# function of bytes that are themselves this row's identity, so the row
# cannot be the place a reading goes to be preserved: `probe` got better
# at phone footage — it stopped believing a declared 120 over timestamps
# 1/30s apart — and a stored reading that nothing can replace would have
# left every already-uploaded source resampling to four times the frames
# with no way to correct it short of deleting the row.
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "four-frames.mp4"
subprocess.run([
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-f", "lavfi", "-i", "color=c=red:s=64x48:r=4:d=1",
"-c:v", "mpeg4", str(path),
], check=True, capture_output=True)
payload = path.read_bytes()
first = self.client.post("/api/sources", {
"file": SimpleUploadedFile("four-frames.mp4", payload, content_type="video/mp4")})
self.assertEqual(201, first.status_code, first.content)
self.assertEqual(4.0, first.json()["probe"]["fps"])
better = dict(first.json()["probe"], fps=12.0, rate="12/1", measured_fps=12.0)
with patch("clips.extraction.probe", return_value=better):
again = self.client.post("/api/sources", {
"file": SimpleUploadedFile("same.mp4", payload, content_type="video/mp4")})
self.assertEqual(200, again.status_code, again.content)
self.assertFalse(again.json()["created"], "the same bytes are the same source")
self.assertEqual("12/1", again.json()["probe"]["rate"])
self.assertEqual(12.0, Source.objects.get(id=first.json()["id"]).probe["fps"])
def test_the_proxy_is_re_encoded_rather_than_the_upload_re_served(self): def test_the_proxy_is_re_encoded_rather_than_the_upload_re_served(self):
# The footage's identity is the proxy's digest, and the proxy is produced # The footage's identity is the proxy's digest, and the proxy is produced
# by one ffmpeg invocation whatever the upload was. If the upload were # by one ffmpeg invocation whatever the upload was. If the upload were
@ -747,6 +986,73 @@ class UploadTests(TestCase):
self.assertTrue(facts["vfr"], "the disagreement is still recorded, just not fatal") self.assertTrue(facts["vfr"], "the disagreement is still recorded, just not fatal")
self.assertTrue(facts["has_audio"]) self.assertTrue(facts["has_audio"])
def test_a_declared_rate_the_timestamps_do_not_bear_out_is_not_resampled_to(self):
# THE FOUR-TIMES. An iPhone container declares `r_frame_rate` 120 over a
# stream whose frames are 1/30s apart, and taking the declaration at its
# word turned an 11-second clip into 1293 proxy frames instead of 323:
# four times the encode, four times the tracing stills, four times the
# blobs and the rows, for 970 frames that are copies of their neighbours.
# The timestamps are the evidence and they say 30.
streams = json.dumps({"streams": [
{"codec_type": "video", "r_frame_rate": "120/1",
"avg_frame_rate": "96900/3233", "nb_frames": "323",
"width": 1920, "height": 1440},
{"codec_type": "audio"}],
"format": {"duration": "10.775"}})
# IN DECODE ORDER, which is how an HEVC stream really arrives — the first
# packets of the fixture this was found on come out 0, 0.133, 0.067,
# 0.033. Differencing that order unsorted measures the reordering delay
# and not the rate, so the fixture keeps the hazard in it.
shuffled = [0, 4, 2, 1, 3, 8, 6, 5, 7, 12, 10, 9, 11]
packets = json.dumps({"packets": [{"pts_time": f"{i / 30:.6f}"} for i in shuffled]})
with patch("clips.extraction._command", side_effect=[streams, packets]):
facts = extraction.probe(Path("phone.mov"))
self.assertEqual("96900/3233", facts["rate"], "resampled to the declared 120")
self.assertAlmostEqual(30.0, facts["measured_fps"], places=2)
def test_a_genuine_high_rate_capture_is_still_taken_at_its_own_rate(self):
# The other half of the same decision, and the one that would be easy to
# break: a real 120fps capture must not be dragged down to anything. Its
# declaration and its timestamps agree, so the declaration — the exact
# rational the stream was authored at — is what is used.
streams = json.dumps({"streams": [
{"codec_type": "video", "r_frame_rate": "120/1", "avg_frame_rate": "120/1",
"width": 640, "height": 480}],
"format": {"duration": "2"}})
packets = json.dumps({"packets": [{"pts_time": f"{i / 120:.6f}"} for i in range(13)]})
with patch("clips.extraction._command", side_effect=[streams, packets]):
facts = extraction.probe(Path("slowmo.mov"))
self.assertEqual("120/1", facts["rate"])
def test_too_few_timestamps_to_measure_leaves_the_declaration_standing(self):
# A clip with nine-ish frames cannot outvote one odd timestamp, so the
# measurement declines to have an opinion and the nominal rate — the one
# that drops no distinct frame — is used exactly as it was before.
streams = json.dumps({"streams": [
{"codec_type": "video", "r_frame_rate": "30/1", "avg_frame_rate": "24/1",
"width": 640, "height": 480}],
"format": {"duration": "0.1"}})
packets = json.dumps({"packets": [{"pts_time": f"{i / 30:.6f}"} for i in range(3)]})
with patch("clips.extraction._command", side_effect=[streams, packets]):
facts = extraction.probe(Path("tiny.mov"))
self.assertEqual("30/1", facts["rate"])
self.assertIsNone(facts["measured_fps"])
def test_a_rate_measurement_that_fails_outright_cannot_refuse_an_upload(self):
# The measurement is an optimisation. If ffprobe cannot read the packets
# of a file whose streams it just read happily, the upload still has to be
# accepted on its metadata — an optimisation that can reject work is worse
# than no optimisation.
streams = json.dumps({"streams": [
{"codec_type": "video", "r_frame_rate": "25/1", "avg_frame_rate": "25/1",
"width": 640, "height": 480}],
"format": {"duration": "4"}})
with patch("clips.extraction._command",
side_effect=[streams, ValueError("ffprobe fell over")]):
facts = extraction.probe(Path("awkward.mov"))
self.assertEqual("25/1", facts["rate"])
self.assertIsNone(facts["measured_fps"])
def test_the_proxy_rate_is_exact_rather_than_a_rounded_float(self): def test_the_proxy_rate_is_exact_rather_than_a_rounded_float(self):
# 30000/1001 is not a float. Handing ffmpeg's -r a rounded one is how a # 30000/1001 is not a float. Handing ffmpeg's -r a rounded one is how a
# long take drifts out of sync with its own audio. # long take drifts out of sync with its own audio.
@ -814,3 +1120,99 @@ class UploadTests(TestCase):
digest="e" * 64, fps=12, frames=3, width=8, height=6, audio=blob) digest="e" * 64, fps=12, frames=3, width=8, height=6, audio=blob)
manifest = self.client.get(f"/api/footage/{footage.id}").json() manifest = self.client.get(f"/api/footage/{footage.id}").json()
self.assertIsNone(manifest["video"]) self.assertIsNone(manifest["video"])
@override_settings(BLOB_ROOT=BLOB_DIR)
class OwnershipTests(TestCase):
"""Anyone with the link reads; the owner and the editors write."""
def setUp(self):
from django.contrib.auth import get_user_model
User = get_user_model()
self.ann = User.objects.create_user("ann", password="password1")
self.bob = User.objects.create_user("bob", password="password1")
self.project = Project.objects.create(name="ann's", owner=self.ann)
def write(self):
return self.client.put(f"/api/projects/{self.project.id}",
data=json.dumps({"name": "renamed", "clips": []}),
content_type="application/json")
def test_anyone_with_the_link_can_read_and_nobody_else_can_write(self):
loaded = self.client.get(f"/api/projects/{self.project.id}").json()
self.assertEqual(("ann", False), (loaded["owner"], loaded["can_edit"]))
self.assertEqual(403, self.write().status_code)
self.client.login(username="bob", password="password1")
self.assertEqual(403, self.write().status_code)
def test_the_owner_names_an_editor_who_can_then_write(self):
self.client.login(username="bob", password="password1")
self.assertEqual(403, self.client.post(
f"/api/projects/{self.project.id}/editors", data=json.dumps({"username": "bob"}),
content_type="application/json").status_code)
self.client.login(username="ann", password="password1")
self.assertEqual(200, self.write().status_code)
self.assertEqual(["bob"], self.client.post(
f"/api/projects/{self.project.id}/editors", data=json.dumps({"username": "bob"}),
content_type="application/json").json()["editors"])
self.client.login(username="bob", password="password1")
self.assertTrue(self.client.get(f"/api/projects/{self.project.id}").json()["can_edit"])
self.assertEqual(200, self.write().status_code)
self.client.login(username="ann", password="password1")
self.client.delete(f"/api/projects/{self.project.id}/editors/bob")
self.client.login(username="bob", password="password1")
self.assertEqual(403, self.write().status_code)
def test_a_project_is_made_by_somebody_signed_in_and_is_theirs(self):
self.assertEqual(403, self.client.post("/api/projects", data=json.dumps({"name": "x"}),
content_type="application/json").status_code)
self.client.post("/api/signup", data=json.dumps(
{"username": "cat", "password": "password1"}), content_type="application/json")
self.assertEqual("cat", self.client.get("/api/me").json()["username"])
mine = self.client.post("/api/projects", data=json.dumps({"name": "y"}),
content_type="application/json").json()
self.assertEqual(("cat", True), (mine["owner"], mine["can_edit"]))
listed = {p["name"] for p in self.client.get("/api/projects").json()["projects"]}
self.assertEqual({"y"}, listed)
self.client.logout()
self.assertEqual([], self.client.get("/api/projects").json()["projects"])
def test_a_project_has_an_address(self):
response = self.client.get(f"/p/{self.project.id}")
self.assertEqual(200, response.status_code)
# The slug is the name, for people; the id is what finds it.
self.assertEqual(200, self.client.get(f"/p/{self.project.id}/anything-at-all").status_code)
self.assertContains(response, 'id="app"')
class SocketTests(TestCase):
"""A committed write reaches everyone in the room; presence is stamped."""
def test_a_save_is_broadcast_to_the_room(self):
from asgiref.sync import async_to_sync, sync_to_async
from channels.testing import WebsocketCommunicator
from clips.consumers import broadcast
from server.asgi import application
from django.contrib.auth import get_user_model
project = Project.objects.create(
name="shared", owner=get_user_model().objects.create_user("host"))
async def scenario():
peer = WebsocketCommunicator(application, f"/ws/projects/{project.id}",
headers=[(b"origin", b"http://localhost")])
connected, _ = await peer.connect()
self.assertTrue(connected)
self.assertEqual("welcome", (await peer.receive_json_from())["kind"])
self.assertEqual([], (await peer.receive_json_from())["peers"])
self.assertEqual("join", (await peer.receive_json_from())["kind"])
# The server stamps who sent it; a claimed name is overwritten.
await peer.send_json_to({"kind": "state", "frame": 12, "user": "forged"})
state = await peer.receive_json_from()
self.assertEqual((12, None), (state["frame"], state["user"]))
await sync_to_async(broadcast)(project.id, {"seq": 1, "clips": []})
delta = await peer.receive_json_from()
self.assertEqual(("delta", 1), (delta["kind"], delta["seq"]))
await peer.disconnect()
async_to_sync(scenario)()

View file

@ -16,16 +16,28 @@ from django.urls import path
from . import views from . import views
urlpatterns = [ urlpatterns = [
path("me", views.me),
path("login", views.login),
path("signup", views.signup),
path("logout", views.logout),
path("detector", views.detector), path("detector", views.detector),
path("sources", views.sources), path("sources", views.sources),
path("sounds", views.sounds),
path("sounds/<uuid:sound_id>", views.sound_detail),
path("images", views.images),
path("images/<uuid:image_id>", views.image_detail),
path("extractions", views.extractions), path("extractions", views.extractions),
path("extractions/<str:key>", views.extraction_detail), path("extractions/<str:key>", views.extraction_detail),
path("footage", views.footage_list), path("footage", views.footage_list),
path("footage/<uuid:footage_id>", views.footage_detail), path("footage/<uuid:footage_id>", views.footage_detail),
path("projects", views.projects), path("projects", views.projects),
path("symbols", views.symbols),
path("projects/<uuid:project_id>", views.project_detail), path("projects/<uuid:project_id>", views.project_detail),
path("projects/<uuid:project_id>/leaves/<path:leaf_path>", views.leaf_detail), path("projects/<uuid:project_id>/leaves/<path:leaf_path>", views.leaf_detail),
path("projects/<uuid:project_id>/revisions", views.revisions), path("projects/<uuid:project_id>/revisions", views.revisions),
path("projects/<uuid:project_id>/revisions/<int:revision_id>/restore", views.restore),
path("projects/<uuid:project_id>/editors", views.editors),
path("projects/<uuid:project_id>/editors/<str:username>", views.editors),
path("analyses", views.analyses), path("analyses", views.analyses),
path("analyses/<str:key>", views.analysis_detail), path("analyses/<str:key>", views.analysis_detail),
path("blocks", views.blocks), path("blocks", views.blocks),

View file

@ -32,14 +32,18 @@ from pathlib import Path
from uuid import UUID from uuid import UUID
from django.conf import settings from django.conf import settings
from django.contrib.auth import authenticate, get_user_model
from django.contrib.auth import login as auth_login, logout as auth_logout
from django.core.exceptions import ValidationError from django.core.exceptions import ValidationError
from django.db import transaction from django.db import transaction
from django.db.models import Q
from django.http import FileResponse, HttpResponse, JsonResponse from django.http import FileResponse, HttpResponse, JsonResponse
from django.shortcuts import render from django.shortcuts import render
from django.views.decorators.http import require_http_methods from django.views.decorators.http import require_http_methods
from . import blobs, extraction from . import blobs, extraction
from .models import Analysis, Block, Blob, Clip, Extraction, Footage, Leaf, Project, Revision, Source from .consumers import broadcast
from .models import Analysis, Block, Blob, Clip, Extraction, Footage, Image, Leaf, Project, Revision, Sound, Source
KEY_LENGTH = 71 # "sha256:" + 64 hex KEY_LENGTH = 71 # "sha256:" + 64 hex
@ -119,10 +123,27 @@ def _crop_blob(chunks):
# the page # the page
def page(request): def _asset_version(relative):
"""A static file's modification time, for its URL.
The stylesheet and the bundle are served with `Last-Modified` and nothing
else, so a browser is free to keep a stale copy on heuristic freshness — and
new JavaScript over an old stylesheet renders a pane the stylesheet has never
heard of as bare elements. A version in the URL makes each edit a new URL."""
for root in settings.STATICFILES_DIRS:
path = Path(root) / relative
if path.exists():
return str(int(path.stat().st_mtime))
return "0"
def page(request, project_id=None, slug=None):
"""The host page. This replaced `frontend/public/index.html` at step 9, and """The host page. This replaced `frontend/public/index.html` at step 9, and
`:dev-http` in shadow-cljs.edn went away with it.""" `:dev-http` in shadow-cljs.edn went away with it."""
return render(request, "clips/index.html") return render(request, "clips/index.html", {
"css_version": _asset_version("arthur/app.css"),
"js_version": _asset_version("arthur/js/main.js"),
})
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@ -197,6 +218,17 @@ def sources(request):
"media_type": upload.content_type or "video/mp4"}) "media_type": upload.content_type or "video/mp4"})
row, created = Source.objects.get_or_create( row, created = Source.objects.get_or_create(
blob=blob, defaults={"filename": Path(upload.name).name[:255], "probe": facts}) blob=blob, defaults={"filename": Path(upload.name).name[:255], "probe": facts})
if not created and row.probe != facts:
# THE FACTS ARE RE-READ, NOT REMEMBERED. They are a pure function of
# the bytes, and the bytes are this row's identity — so a
# disagreement means the server reads the file differently now from
# whenever it first saw it, and the fresh reading is the one to keep.
# Storing the first reading forever pins a source to a rate the code
# no longer believes in, and makes it unfixable without deleting the
# row: `extraction.probe` got better at phone footage and every
# already-uploaded source would have gone on being wrong.
row.probe = facts
row.save(update_fields=["probe"])
return JsonResponse({"id": str(row.id), "digest": digest, return JsonResponse({"id": str(row.id), "digest": digest,
"filename": row.filename, "probe": row.probe, "filename": row.filename, "probe": row.probe,
"created": created}, status=201 if created else 200) "created": created}, status=201 if created else 200)
@ -204,6 +236,112 @@ def sources(request):
return JsonResponse({"error": str(exc)}, status=400) return JsonResponse({"error": str(exc)}, status=400)
def _sound_json(row):
return {"id": str(row.id), "label": row.label or row.filename,
"filename": row.filename, "duration": row.duration,
"audio": f"/blob/{row.blob_id}"}
def _relabel(request, row):
"""PATCH one asset's display name.
A LABEL IS THE ONLY FIELD EITHER ROW LETS A CLIENT WRITE, and the body is
read for that key alone. Footage is content-addressed and its frame count,
rate and digest are facts about the bytes; an endpoint that merged whatever
it was sent would let a rename quietly contradict them. Blank clears it,
which puts the row back to the name it was uploaded under rather than
leaving it nameless.
"""
data = _body(request)
if "label" not in data:
raise Bad("a rename needs a label")
label = str(data["label"] or "").strip()[:200]
if label != row.label:
row.label = label
row.save(update_fields=["label"])
return row
@require_http_methods(["GET", "POST"])
def sounds(request):
if request.method == "GET":
return JsonResponse({"sounds": [_sound_json(row)
for row in Sound.objects.order_by("-created")]})
upload = request.FILES.get("file")
if upload is None:
return JsonResponse({"error": "upload a sound as the file field"}, status=400)
try:
digest, size = blobs.write_stream(upload.chunks())
duration = extraction.probe_audio(blobs.path_for(digest))
blob, _ = Blob.objects.get_or_create(
digest=digest, defaults={"size": size,
"media_type": upload.content_type or "audio/mpeg"})
row, created = Sound.objects.get_or_create(
blob=blob, defaults={"filename": Path(upload.name).name[:255],
"duration": duration})
return JsonResponse(_sound_json(row), status=201 if created else 200)
except (ValueError, OSError) as exc:
return JsonResponse({"error": str(exc)}, status=400)
@require_http_methods(["GET", "PATCH"])
def sound_detail(request, sound_id):
try:
row = Sound.objects.get(id=sound_id)
except Sound.DoesNotExist:
return JsonResponse({"error": "no such sound"}, status=404)
try:
if request.method == "PATCH":
row = _relabel(request, row)
except Bad as exc:
return _error(exc)
return JsonResponse(_sound_json(row))
def _image_json(row):
return {"id": str(row.id), "label": row.label or row.filename,
"filename": row.filename, "width": row.width, "height": row.height,
"digest": row.blob_id, "url": f"/blob/{row.blob_id}"}
@require_http_methods(["GET", "POST"])
def images(request):
"""Stills to trace over. The document names one by its blob digest, so an
image is the same picture in every project that uses it."""
if request.method == "GET":
return JsonResponse({"images": [_image_json(row)
for row in Image.objects.order_by("-created")]})
upload = request.FILES.get("file")
if upload is None:
return JsonResponse({"error": "upload an image as the file field"}, status=400)
try:
digest, size = blobs.write_stream(upload.chunks())
width, height = extraction.probe_image(blobs.path_for(digest))
blob, _ = Blob.objects.get_or_create(
digest=digest, defaults={"size": size,
"media_type": upload.content_type or "image/png"})
row, created = Image.objects.get_or_create(
blob=blob, defaults={"filename": Path(upload.name).name[:255],
"width": width, "height": height})
return JsonResponse(_image_json(row), status=201 if created else 200)
except (ValueError, OSError) as exc:
return JsonResponse({"error": str(exc)}, status=400)
@require_http_methods(["GET", "PATCH"])
def image_detail(request, image_id):
try:
row = Image.objects.get(id=image_id)
except Image.DoesNotExist:
return JsonResponse({"error": "no such image"}, status=404)
try:
if request.method == "PATCH":
row = _relabel(request, row)
except Bad as exc:
return _error(exc)
return JsonResponse(_image_json(row))
def _extraction_json(row): def _extraction_json(row):
return {"key": row.key, "source": str(row.source_id), "state": row.state, return {"key": row.key, "source": str(row.source_id), "state": row.state,
"progress": row.progress, "error": row.error, "progress": row.progress, "error": row.error,
@ -273,12 +411,70 @@ def footage_list(request):
) )
_SYMBOL_LEAF = re.compile(r"^clip/([^/]+)/symbol/([^/]+)$")
_PALETTE_LEAF = re.compile(r"^clip/([^/]+)/palette/([^/]+)$")
def _transit_fields(value, *keys):
"""Top-level fields of a transit map leaf, by keyword name. A leaf's own
facts are a small flat map, so no key repeats and transit's cache never
stands in for one; anything else reads as absent."""
if not (isinstance(value, list) and value[:1] == ["^ "]):
return {}
pairs = dict(zip(value[1::2], value[2::2]))
return {k: pairs.get(f"~:{k}") for k in keys}
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def symbols(request):
"""Every symbol in every saved project, for the pool's all-assets folder.
Read off the leaf PATHS rather than by loading documents: a symbol's own leaf
is `clip/<cid>/symbol/<sid>`, so listing them is one query and no decoding
beyond the name and length its value carries."""
rows = []
for leaf in Leaf.objects.filter(path__contains="/symbol/").select_related("project"):
m = _SYMBOL_LEAF.match(leaf.path)
if not m:
continue
fields = _transit_fields(leaf.value, "name", "frames")
rows.append({
"project": str(leaf.project_id),
"project_name": leaf.project.name,
"cid": m.group(1),
"symbol": m.group(2),
"name": fields.get("name") or m.group(2).replace("~", "/"),
"frames": fields.get("frames"),
})
rows.sort(key=lambda r: (r["project_name"], r["project"], r["name"]))
palettes = []
for leaf in Leaf.objects.filter(path__contains="/palette/").select_related("project"):
m = _PALETTE_LEAF.match(leaf.path)
if not m:
continue
fields = _transit_fields(leaf.value, "name")
palettes.append({
"project": str(leaf.project_id),
"project_name": leaf.project.name,
"cid": m.group(1),
"palette": m.group(2),
"name": fields.get("name") or m.group(2).replace("~", "/"),
})
palettes.sort(key=lambda r: (r["project_name"], r["project"], r["name"]))
return JsonResponse({"symbols": rows, "palettes": palettes})
@require_http_methods(["GET", "PATCH"])
def footage_detail(request, footage_id): def footage_detail(request, footage_id):
try: try:
footage = Footage.objects.select_related("audio", "video", "stream").get(id=footage_id) footage = Footage.objects.select_related("audio", "video", "stream").get(id=footage_id)
except Footage.DoesNotExist: except Footage.DoesNotExist:
return JsonResponse({"error": "no such footage"}, status=404) return JsonResponse({"error": "no such footage"}, status=404)
try:
if request.method == "PATCH":
footage = _relabel(request, footage)
except Bad as exc:
return _error(exc)
return JsonResponse(_footage_json(footage)) return JsonResponse(_footage_json(footage))
@ -564,7 +760,11 @@ def block_detail(request, key):
# tier 1: projects, clips, leaves # tier 1: projects, clips, leaves
def _project_json(project: Project): def _who(user):
return {"username": user.get_username() if user.is_authenticated else None}
def _project_json(project: Project, user):
leaves = list(project.leaves.all()) leaves = list(project.leaves.all())
clips = [] clips = []
for clip in project.clips.all(): for clip in project.clips.all():
@ -573,8 +773,6 @@ def _project_json(project: Project):
{ {
"cid": clip.cid, "cid": clip.cid,
"name": clip.name, "name": clip.name,
"footage": str(clip.footage_id) if clip.footage_id else None,
"analysis": clip.analysis_id,
"blocks": sorted(clip.blocks.values_list("key", flat=True)), "blocks": sorted(clip.blocks.values_list("key", flat=True)),
"leaves": {leaf.path: leaf.value for leaf in leaves if leaf.path.startswith(prefix)}, "leaves": {leaf.path: leaf.value for leaf in leaves if leaf.path.startswith(prefix)},
} }
@ -585,27 +783,103 @@ def _project_json(project: Project):
"schema_version": project.schema_version, "schema_version": project.schema_version,
"seq": project.seq, "seq": project.seq,
"palette": project.palette, "palette": project.palette,
"owner": project.owner.get_username(),
"editors": sorted(project.editors.values_list("username", flat=True)),
"can_edit": project.can_edit(user),
"clips": clips, "clips": clips,
} }
def _project(project_id):
try:
return Project.objects.select_related("owner").get(id=project_id)
except Project.DoesNotExist:
raise Bad("no such project", status=404)
def _writable(request, project_id):
project = _project(project_id)
if not project.can_edit(request.user):
raise Bad("only the owner and the editors can change this project; "
"save a copy instead", status=403)
return project
# ---------------------------------------------------------------------------
# who you are
#
# Django's session cookie, and the page's CSRF cookie on every write. Nothing
# here that a signed-in admin does not already have; the API gains a way in that
# is not the admin's login page.
@require_http_methods(["GET"])
def me(request):
return JsonResponse(_who(request.user))
@require_http_methods(["POST"])
def login(request):
data = json.loads(request.body or b"{}")
user = authenticate(request, username=data.get("username"), password=data.get("password"))
if user is None:
return JsonResponse({"error": "wrong username or password"}, status=400)
auth_login(request, user)
return JsonResponse(_who(user))
@require_http_methods(["POST"])
def signup(request):
data = json.loads(request.body or b"{}")
username = (data.get("username") or "").strip()
password = data.get("password") or ""
if not username or len(password) < 8:
return JsonResponse({"error": "a username, and a password of 8 or more"}, status=400)
User = get_user_model()
if User.objects.filter(username__iexact=username).exists():
return JsonResponse({"error": "that username is taken"}, status=409)
user = User.objects.create_user(username=username, password=password)
auth_login(request, user)
return JsonResponse(_who(user), status=201)
@require_http_methods(["POST"])
def logout(request):
auth_logout(request)
return JsonResponse(_who(request.user))
# ---------------------------------------------------------------------------
# tier 1: projects, clips, leaves
@require_http_methods(["GET", "POST"]) @require_http_methods(["GET", "POST"])
def projects(request): def projects(request):
"""GET lists what you own and are an editor of — nothing, signed out; POST
makes one, owned by you. Every project has an owner, so making one needs you
signed in."""
if request.method == "GET": if request.method == "GET":
if not request.user.is_authenticated:
return JsonResponse({"projects": []})
visible = Q(owner=request.user) | Q(editors=request.user)
return JsonResponse( return JsonResponse(
{ {
"projects": [ "projects": [
{"id": str(p.id), "name": p.name, {"id": str(p.id), "name": p.name,
"schema_version": p.schema_version, "seq": p.seq, "schema_version": p.schema_version, "seq": p.seq,
"owner": p.owner.get_username(),
"updated": p.updated.isoformat()} "updated": p.updated.isoformat()}
for p in Project.objects.all()[:100] for p in Project.objects.filter(visible).distinct()
.select_related("owner")[:100]
] ]
} }
) )
if not request.user.is_authenticated:
return JsonResponse({"error": "sign in to make a project"}, status=403)
try: try:
data = _body(request) data = _body(request)
project = Project.objects.create(name=data.get("name") or "untitled") project = Project.objects.create(name=data.get("name") or "untitled", owner=request.user)
return JsonResponse(_project_json(project), status=201) return JsonResponse(_project_json(project, request.user), status=201)
except Bad as exc: except Bad as exc:
return _error(exc) return _error(exc)
@ -613,43 +887,74 @@ def projects(request):
@require_http_methods(["GET", "PUT"]) @require_http_methods(["GET", "PUT"])
def project_detail(request, project_id): def project_detail(request, project_id):
try: try:
project = Project.objects.get(id=project_id)
except Project.DoesNotExist:
return JsonResponse({"error": "no such project"}, status=404)
if request.method == "GET": if request.method == "GET":
return JsonResponse(_project_json(project)) return JsonResponse(_project_json(_project(project_id), request.user))
project = _writable(request, project_id)
return _save(project, _body(request), request.user)
except Bad as exc:
return _error(exc)
@require_http_methods(["POST", "DELETE"])
def editors(request, project_id, username=None):
"""The owner names who else can write. POST {username} adds; DELETE
`editors/<username>` removes."""
try: try:
return _save(project, _body(request)) project = _project(project_id)
if not (request.user.is_authenticated and request.user.id == project.owner_id):
raise Bad("only the owner can change who edits", status=403)
if request.method == "POST":
username = _body(request).get("username")
user = get_user_model().objects.filter(username__iexact=username or "").first()
if user is None:
raise Bad(f"nobody is called {username!r}", status=404)
if request.method == "POST":
project.editors.add(user)
else:
project.editors.remove(user)
broadcast(project.id, {}, kind="access")
return JsonResponse({"editors": sorted(project.editors.values_list("username", flat=True))})
except Bad as exc: except Bad as exc:
return _error(exc) return _error(exc)
@transaction.atomic @transaction.atomic
def _save(project: Project, data): def _save(project: Project, data, user):
"""A whole-document save: one clip's leaves replace that clip's leaves. """A save: one clip's leaves, written.
SCOPED BY CLIP, not by project. A payload that carries clip `a` does not SCOPED BY CLIP, not by project. A payload that carries clip `a` does not
disturb clip `b`'s leaves, because a save is not the only way the document disturb clip `b`'s leaves.
changes — a single-leaf conditional write is — and a save that cleared
everything it did not mention would be a save that undoes a collaborator. Two shapes. Without `base`, a clip's leaves REPLACE that clip's leaves — the
whole-document save. With `base`, the seq the client last caught up to, the
save is a PATCH: `leaves` are the ones it changed, `removed` the ones it
deleted, and nothing it did not mention is touched. A leaf it names that
somebody else changed after `base`, to something else, is a conflict, and the
whole save answers 409 with their values — last-writer-wins per leaf, with the
loser told rather than silently clobbered. docs/architecture.md, "Make the
merge unit small instead of clever".
A leaf whose value is unchanged keeps its VERSION. That is what makes the A leaf whose value is unchanged keeps its VERSION. That is what makes the
entity tag mean something: a save of a document where one channel moved entity tag mean something: a save of a document where one channel moved
invalidates one leaf's etag, not all four hundred. invalidates one leaf's etag, not all four hundred.
""" """
base = data.get("base")
seq = project.bump()
if data.get("name"): if data.get("name"):
project.name = data["name"] project.name = data["name"]
if data.get("palette"): if data.get("palette"):
project.palette = data["palette"] project.palette = data["palette"]
project.save(update_fields=["name", "palette"])
written, removed, unchanged = [], [], [] written, removed, unchanged, conflicts, deltas = [], [], [], {}, []
for spec in data.get("clips") or []: for spec in data.get("clips") or []:
cid = spec.get("cid") cid = spec.get("cid")
if not cid: if not cid:
raise Bad("every clip in a save names its cid") raise Bad("every clip in a save names its cid")
leaves = spec.get("leaves") or {} leaves = spec.get("leaves") or {}
gone = spec.get("removed") or [] if base is not None else []
prefix = f"clip/{cid}/" prefix = f"clip/{cid}/"
for path in leaves: for path in [*leaves, *gone]:
if not path.startswith(prefix): if not path.startswith(prefix):
raise Bad( raise Bad(
f"leaf {path!r} is not addressed to clip {cid!r}", f"leaf {path!r} is not addressed to clip {cid!r}",
@ -657,7 +962,19 @@ def _save(project: Project, data):
) )
keys = spec.get("blocks") or [] keys = spec.get("blocks") or []
have = set(Block.objects.filter(key__in=keys).values_list("key", flat=True)) analyses = spec.get("analyses") or []
if (not isinstance(analyses, list)
or not all(isinstance(key, str) for key in analyses)
or len(analyses) != len(set(analyses))):
raise Bad("a clip's analyses must be a list of distinct analysis ids")
registered = set(Analysis.objects.filter(key__in=analyses)
.values_list("key", flat=True))
if unknown := [key for key in analyses if key not in registered]:
raise Bad("this clip names analyses the server does not know; register them first",
status=409, missing=unknown)
block_rows = list(Block.objects.filter(key__in=keys))
have = {block.key for block in block_rows}
if missing := [k for k in keys if k not in have]: if missing := [k for k in keys if k not in have]:
# Referential integrity across the tiers, enforced where it can be: # Referential integrity across the tiers, enforced where it can be:
# a document that names blocks the server does not hold would load # a document that names blocks the server does not hold would load
@ -667,39 +984,60 @@ def _save(project: Project, data):
"before saving the document that points at them", "before saving the document that points at them",
status=409, missing=missing, status=409, missing=missing,
) )
if undeclared := sorted({block.analysis_id for block in block_rows} - registered):
raise Bad("every block in a clip must name one of that clip's analyses",
status=409, missing=undeclared)
existing = {leaf.path: leaf for leaf in project.leaves.filter(path__startswith=prefix)}
if base is not None:
for path in [*leaves, *gone]:
theirs = existing.get(path)
if theirs and theirs.seq > base and (
path not in leaves or theirs.value != leaves[path]):
conflicts[path] = theirs.value
if conflicts:
continue
analysis = Analysis.objects.filter(key=spec.get("analysis")).first()
footage = None
if spec.get("footage"):
footage = Footage.objects.filter(id=spec["footage"]).first()
clip, _ = Clip.objects.update_or_create( clip, _ = Clip.objects.update_or_create(
project=project, project=project,
cid=cid, cid=cid,
defaults={"name": spec.get("name") or "", "analysis": analysis, "footage": footage}, defaults={"name": spec.get("name") or ""},
) )
clip.blocks.set(Block.objects.filter(key__in=keys)) blocks = block_rows
if base is None:
clip.blocks.set(blocks)
gone = [path for path in existing if path not in leaves]
else:
clip.blocks.add(*blocks)
existing = {leaf.path: leaf for leaf in project.leaves.filter(path__startswith=prefix)} changed = {}
for path, value in leaves.items(): for path, value in leaves.items():
leaf = existing.get(path) leaf = existing.get(path)
if leaf is None: if leaf is None:
Leaf.objects.create(project=project, path=path, value=value) Leaf.objects.create(project=project, path=path, value=value, seq=seq)
written.append(path)
elif leaf.value != value: elif leaf.value != value:
leaf.value = value leaf.value, leaf.seq = value, seq
leaf.version += 1 leaf.version += 1
leaf.save(update_fields=["value", "version", "updated"]) leaf.save(update_fields=["value", "version", "seq", "updated"])
written.append(path)
else: else:
unchanged.append(path) unchanged.append(path)
for path, leaf in existing.items(): continue
if path not in leaves: changed[path] = value
leaf.delete() dropped = [path for path in gone if path in existing]
removed.append(path) project.leaves.filter(path__in=dropped).delete()
written += changed
removed += dropped
deltas.append({"cid": cid, "leaves": changed, "removed": dropped, "blocks": keys})
seq = project.seq + 1 if conflicts:
project.seq = seq raise Bad(
project.save() "somebody else changed these since you last caught up",
status=409, seq=seq - 1, conflicts=conflicts,
)
by = user.get_username() if user.is_authenticated else None
transaction.on_commit(lambda: broadcast(project.id, {
"seq": seq, "by": by, "name": project.name, "clips": deltas,
}))
return JsonResponse( return JsonResponse(
{ {
"id": str(project.id), "id": str(project.id),
@ -722,9 +1060,9 @@ def leaf_detail(request, project_id, leaf_path):
for a painted cel that is the class of bug that ends trust in a tool. for a painted cel that is the class of bug that ends trust in a tool.
""" """
try: try:
project = Project.objects.get(id=project_id) project = _project(project_id) if request.method == "GET" else _writable(request, project_id)
except Project.DoesNotExist: except Bad as exc:
return JsonResponse({"error": "no such project"}, status=404) return _error(exc)
leaf = project.leaves.filter(path=leaf_path).first() leaf = project.leaves.filter(path=leaf_path).first()
if request.method == "GET": if request.method == "GET":
@ -742,19 +1080,24 @@ def leaf_detail(request, project_id, leaf_path):
return _error(Bad("a leaf write carries a value")) return _error(Bad("a leaf write carries a value"))
match = request.headers.get("If-Match") match = request.headers.get("If-Match")
with transaction.atomic():
seq = project.bump()
leaf = project.leaves.filter(path=leaf_path).first()
if leaf is None: if leaf is None:
# ANY `If-Match` on a leaf that does not exist is a failed precondition, # ANY `If-Match` on a leaf that does not exist is a failed precondition,
# `*` included: RFC 7232 gives `*` the meaning "the resource must already # `*` included: RFC 7232 gives `*` the meaning "the resource must already
# exist", which is exactly the write a client makes when it believes it is # exist", which is exactly the write a client makes when it believes it
# editing something. Creating it instead would turn "somebody deleted this # is editing something. Creating it instead would turn "somebody deleted
# node" into a silent resurrection. # this node" into a silent resurrection.
if match: if match:
transaction.set_rollback(True)
return JsonResponse( return JsonResponse(
{"error": "no such leaf", "path": leaf_path}, status=409 {"error": "no such leaf", "path": leaf_path}, status=409
) )
leaf = Leaf.objects.create(project=project, path=leaf_path, value=data["value"]) leaf = Leaf.objects.create(project=project, path=leaf_path, value=data["value"], seq=seq)
else: else:
if match and match not in ("*", leaf.etag): if match and match not in ("*", leaf.etag):
transaction.set_rollback(True)
response = JsonResponse( response = JsonResponse(
{ {
"error": "stale write", "error": "stale write",
@ -766,11 +1109,16 @@ def leaf_detail(request, project_id, leaf_path):
) )
response["ETag"] = leaf.etag response["ETag"] = leaf.etag
return response return response
leaf.value = data["value"] leaf.value, leaf.seq = data["value"], seq
leaf.version += 1 leaf.version += 1
leaf.save(update_fields=["value", "version", "updated"]) leaf.save(update_fields=["value", "version", "seq", "updated"])
cid = leaf_path.split("/")[1] if leaf_path.startswith("clip/") else None
by = request.user.get_username() if request.user.is_authenticated else None
transaction.on_commit(lambda: broadcast(project.id, {
"seq": seq, "by": by, "name": project.name,
"clips": [{"cid": cid, "leaves": {leaf.path: leaf.value}, "removed": [], "blocks": []}],
}))
seq = project.bump()
response = JsonResponse({"path": leaf.path, "version": leaf.version, "seq": seq}) response = JsonResponse({"path": leaf.path, "version": leaf.version, "seq": seq})
response["ETag"] = leaf.etag response["ETag"] = leaf.etag
return response return response
@ -778,16 +1126,17 @@ def leaf_detail(request, project_id, leaf_path):
@require_http_methods(["GET", "POST"]) @require_http_methods(["GET", "POST"])
def revisions(request, project_id): def revisions(request, project_id):
"""Mark a version: one snapshot of the authored layer, with a summary.""" """Named snapshots: GET lists them, POST {summary} takes one of the document
as it is now."""
try: try:
project = Project.objects.get(id=project_id) project = _project(project_id) if request.method == "GET" else _writable(request, project_id)
except Project.DoesNotExist: except Bad as exc:
return JsonResponse({"error": "no such project"}, status=404) return _error(exc)
if request.method == "GET": if request.method == "GET":
return JsonResponse( return JsonResponse(
{ {
"revisions": [ "revisions": [
{"seq": r.seq, "author": r.author, "summary": r.summary, {"id": r.id, "seq": r.seq, "author": r.author, "summary": r.summary,
"created": r.created.isoformat(), "leaves": len(r.document)} "created": r.created.isoformat(), "leaves": len(r.document)}
for r in project.revisions.all()[:100] for r in project.revisions.all()[:100]
] ]
@ -797,8 +1146,57 @@ def revisions(request, project_id):
revision = Revision.objects.create( revision = Revision.objects.create(
project=project, project=project,
seq=project.seq, seq=project.seq,
author=data.get("author") or "", author=request.user.get_username(),
summary=data.get("summary") or "", summary=(data.get("summary") or "").strip()[:500],
document={leaf.path: leaf.value for leaf in project.leaves.all()}, document={leaf.path: leaf.value for leaf in project.leaves.all()},
blocks={clip.cid: sorted(clip.blocks.values_list("key", flat=True))
for clip in project.clips.all()},
) )
return JsonResponse({"seq": revision.seq, "leaves": len(revision.document)}, status=201) return JsonResponse({"id": revision.id, "seq": revision.seq,
"leaves": len(revision.document)}, status=201)
@require_http_methods(["POST"])
def restore(request, project_id, revision_id):
"""Put a snapshot back: an ordinary write of every leaf that differs, so
everybody in the room receives it the way they receive any other."""
try:
project = _writable(request, project_id)
revision = project.revisions.filter(id=revision_id).first()
if revision is None:
raise Bad("no such snapshot", status=404)
except Bad as exc:
return _error(exc)
with transaction.atomic():
seq = project.bump()
existing = {leaf.path: leaf for leaf in project.leaves.all()}
deltas = {}
def delta(path):
cid = path.split("/")[1]
return deltas.setdefault(cid, {"cid": cid, "leaves": {}, "removed": [],
"blocks": revision.blocks.get(cid, [])})
for path, value in revision.document.items():
leaf = existing.get(path)
if leaf is None:
Leaf.objects.create(project=project, path=path, value=value, seq=seq)
elif leaf.value != value:
leaf.value, leaf.seq = value, seq
leaf.version += 1
leaf.save(update_fields=["value", "version", "seq", "updated"])
else:
continue
delta(path)["leaves"][path] = value
gone = [path for path in existing if path not in revision.document]
project.leaves.filter(path__in=gone).delete()
for path in gone:
delta(path)["removed"].append(path)
for cid, keys in revision.blocks.items():
clip = project.clips.filter(cid=cid).first()
if clip:
clip.blocks.add(*Block.objects.filter(key__in=keys))
by = request.user.get_username()
transaction.on_commit(lambda: broadcast(project.id, {
"seq": seq, "by": by, "name": project.name, "clips": list(deltas.values()),
}))
return JsonResponse({"seq": seq, "changed": sum(len(d["leaves"]) + len(d["removed"])
for d in deltas.values())})

View file

@ -1,5 +1,10 @@
# arthur — the animation model # arthur — the animation model
The revised target for lanes, occurrences, source playback, shared editing, and
multi-view UX is [The Lane Model](lane-model.md). It supersedes conflicting
proposals below. Backward compatibility is not required; this document still
contains descriptions of earlier shapes and planned features.
The data that describes a moving picture: what the primitives are, how they The data that describes a moving picture: what the primitives are, how they
nest, how they change over time, and how rotoscoped and hand-authored work end nest, how they change over time, and how rotoscoped and hand-authored work end
up being the same thing with one flag between them. up being the same thing with one flag between them.
@ -79,6 +84,20 @@ this way.
over which the node exists at all. Distinct from a `[:vis]` channel, which over which the node exists at all. Distinct from a `[:vis]` channel, which
blinks an existing node on and off. blinks an existing node on and off.
**Every node has the same two maps into its parent**, whatever kind it is:
- **space** — the matrix its transform channels compose to, times a `:pinv` if
it has been moved in from elsewhere;
- **time** — `local = rate · (parent − at)`, from `:time :at` and `:rate`,
identity when absent. `:span` and every key are in the node's **own** frames.
A move keeps a node's world maps and re-expresses them under its new parent:
the matrix becomes a `:pinv`, the time becomes a new `:at` and `:rate`, and its
channels, keys and span are not touched. Both maps are affine, so any depth of
nesting is one map and every move is one inverse. `node/time-of`,
`node/then-time` and `node/placed-span` are the time half; `clip/move-node` and
`clip/group` are the move.
### Subjects and tracked features ### Subjects and tracked features
Scene nodes describe drawings, not tracking identity. A scene may also carry a Scene nodes describe drawings, not tracking identity. A scene may also carry a
@ -153,7 +172,6 @@ Every animatable property is a channel, and channels are addressed **by path**:
[:xform :rot] {:animated? false :value 0.0} [:xform :rot] {:animated? false :value 0.0}
[:xform :scale] {:animated? false :value [1.0 1.0]} [:xform :scale] {:animated? false :value [1.0 1.0]}
[:xform :skew] {:animated? false :value [0.0 0.0]} [:xform :skew] {:animated? false :value [0.0 0.0]}
[:xform :anchor]{:animated? false :value [0.0 0.0]}
[:geom :pts] {:animated? true :interp :hold :dense {...} :generated {...}} [:geom :pts] {:animated? true :interp :hold :dense {...} :generated {...}}
[:style :color] {:animated? false :value :skin-dark} [:style :color] {:animated? false :value :skin-dark}
[:vis] {:animated? true :interp :hold :keys {0 true, 37 false}}} [:vis] {:animated? true :interp :hold :keys {0 true, 37 false}}}
@ -216,10 +234,30 @@ combines:
```clojure ```clojure
{:animated? true :interp :hold {:animated? true :interp :hold
:dense {...} :generated {...} :dense {...} :generated {...}
:over [{:blend :offset :keys {88 [2 0], 96 [0 0]}} :over [{:id :nudge :support [88 98] :op :offset
{:blend :replace :keys {104 [[3 7] [4 7] …]}}]} :values {:animated? true :interp :linear :keys {88 [2 0], 96 [0 0]}}}
{:id :redraw :support [104 105] :op :replace
:values {:animated? false :value [[3 7] [4 7] …]}}]}
``` ```
A LAYER'S VALUES ARE A CHANNEL, which is what keeps a constant adjustment, a
ramp and a return motion from being three mechanisms: a framed one says the same
thing on every frame it covers, a keyed one moves. They read through `value-at`
and `cursor` like any channel, one reading head each, so the specification and
the playback path share their blending and differ only in how they read — and a
layer's values may not carry layers of their own, which the stack already
orders.
`:support` is half-open and explicit, `[in out)`. Outside it a layer is inactive
and the base evaluates exactly as it did before, which is the difference between
a bounded correction and inserting boundary keys — the latter alters the
neighbouring segments. And a layer has NO TIME SPACE of its own: its support and
its values' keys are in the frames the base channel's keys are in, the node's
own. A correction on a lane is therefore in lane frames and reaches across the
drawings exposed under it; one on a single occurrence is in that occurrence's
frames and travels with it when the exposure moves. Ownership had already
answered the question, so there is no field to disagree with.
- **`:offset`** adds a delta to the base. "Nudge the mouth two pixels right for - **`:offset`** adds a delta to the base. "Nudge the mouth two pixels right for
ten frames" survives a re-freeze at different parameters, because it was never ten frames" survives a re-freeze at different parameters, because it was never
a position — it was a correction. a position — it was a correction.
@ -229,6 +267,22 @@ This is what `docs/design.md` means by an override layer, and it is why
re-freezing is safe: the base is regenerated, the layers are untouched. It is re-freezing is safe: the base is regenerated, the layers are untouched. It is
Blender's NLA blending and AE's effect stack at one property. Blender's NLA blending and AE's effect stack at one property.
WHEN THE BASE OUTGROWS A CORRECTION it is a CONFLICT, which is neither a dropped
layer nor an applied one. Turning `:verts` gives the mouth a different number of
points, and an `:offset` is a row of components that has to match: so the
regeneration records `:conflict` on the layer, the layer stays in the document,
the picture is the base meanwhile, and `clip/conflicts` is the list a view
offers to resolve. Deliberately not `problems` — the document loads and saves
fine, it just contains a decision nobody has made yet. A later regeneration
that restores the shape clears the mark. Only `:offset` can conflict; `:replace`
states a whole value and has nothing to agree with.
A correction is NOT a hand placement. `regenerate-head` leaves the head's
authored channels alone once somebody has placed it by hand, and it compares the
channels WITHOUT their layers to decide: otherwise the first correction anyone
made would stop the head following re-measurement forever, which is the opposite
of what a layer is for.
Layers are what "set it by hand" means for anything measured, and the measured Layers are what "set it by hand" means for anything measured, and the measured
channel does not need to know. A hand-set gaze is an `:over` on channel does not need to know. A hand-set gaze is an `:over` on
`[:xform :pos]` of the iris; a hand-set mouth shape is an `:over` on `[:xform :pos]` of the iris; a hand-set mouth shape is an `:over` on
@ -269,7 +323,7 @@ to change to allow it.
## Transform: decomposed, never a matrix ## Transform: decomposed, never a matrix
```clojure ```clojure
{:pos [x y] :rot θ :scale [sx sy] :skew [kx ky] :anchor [ax ay]} {:pos [x y] :rot θ :scale [sx sy] :skew [kx ky]}
``` ```
Stored decomposed for two reasons. Each component has to be independently Stored decomposed for two reasons. Each component has to be independently
@ -279,18 +333,141 @@ entries is meaningless — a rotation tweened through its matrix shears on the w
Composition, per node: Composition, per node:
``` ```
local = T(pos) · T(anchor) · R(rot) · K(skew) · S(scale) · T(-anchor) local = T(pos) · T(piv) · R(rot) · K(skew) · S(scale) · T(-piv)
world = world(parent) · pinv · local world = world(parent) · pinv · local
``` ```
`:anchor` is Flash's registration point and Blender's origin: rotation and scale
happen about it, and getting it wrong is why hand-placed parts swing rather than
turn.
`:pinv` is Blender's `parent_inverse`, captured at the moment of parenting so the `:pinv` is Blender's `parent_inverse`, captured at the moment of parenting so the
child does not jump when it acquires a parent. Small, and its absence is the kind child does not jump when it acquires a parent. Small, and its absence is the kind
of thing that makes a parenting feature feel broken. of thing that makes a parenting feature feel broken.
### A node has a `:pivot`, and a peg is still a peg
Rotation and scale happen about the node's **pivot**, `[:xform :pivot]`, a point
in its own coordinates:
```
local = T(pos) · T(piv) · R(rot) · K(skew) · S(scale) · T(-piv)
= T(pos + piv - M·piv) · M
```
Toon Boom gives every layer and every peg a pivot, Flash gives every instance a
transformation point, After Effects calls it the anchor point. All three store
it, and the reason is one sentence: **a turn has to be a turn on every frame**,
and the only way to keep a point still through an interpolated angle is for the
angle to be composed about that point.
This was deleted in schema 7 and restored in schema 8, and the argument for
deleting it was *not wrong*, which is why it is worth writing down. It was:
```
T(pos) · T(a) · R·K·S · T(-a) ≡ peg at pos+a carrying R·K·S, child at -a
```
to the last bit of the mantissa — `node-test` asserts it, still. `T(a)·M·T(-a)`
is `M` conjugated by a translation, which is "do `M` in a frame shifted by `a`",
and a **parent already is a shifted frame**. So an anchor was a peg written
inline, and a peg can be selected, keyed, shared between nodes and put above a
measured channel. Same expressive content, strictly more reach.
**What that identity does not say is what a node turns about when nobody has
made a peg.** It is an equivalence between a pivot and a peg *that already
exists*; it is silent on the default, and the default is what a person meets.
With no pivot in the composition, a turn about any point that is not the node's
own origin has to be paid for by writing `pos` as well — `gesture/about` solves
for it:
```
q = M⁻¹(c − t) the material point under c
p' = c − M'·q
```
and that solution is an **arc** in the angle while `pos` interpolates along the
**chord**:
| | pivot = origin | pivot ≠ origin |
| --- | --- | --- |
| one drag | right | right |
| between two keys | right | **wrong**, by the sagitta of the arc |
A 360° turn is where that is unmissable: 0° and 360° are the only two frames
where a wrong pivot cannot be seen at all, so the keys look right and every
frame between them is wrong.
**A drawing escaped it. A symbol instance could not.** `paint/centred` puts a
shape's origin on the middle of what it draws the moment it is drawn, so for a
drawing the pivot *is* the origin, `about` has nothing to do, and a keyed turn is
right between its keys. An instance's origin is its **symbol's**, and a symbol is
drawn on the stage, so its origin is the stage's top-left corner. Measured from
the document this was reported on: a symbol holding six drawn shapes had its
content centred at (99, 127), 161 px from its own origin, on a 320×200 stage. One
instance of it, keyed `rot` 0 → 60 and dragged round by hand, put the drawing at
(115, 116) on frame 0 and (241, 104) on frame 60 — both where they were put — and
at (−88, 121) on frame 30, a stage and a half from either. The answer on offer
was "make a peg first", for wanting to spin a drawing.
So the pivot is back, with the default and the escape hatch spelled out, because
a stored pivot without either is the field that was deleted:
| | what | where |
| --- | --- | --- |
| **the default, for a node nobody has pivoted** | the middle of what it draws — `pick/bounds-of`, the same call the selection box comes from, so the cross starts out on the middle of the box | `gesture/pivot` |
| **choosing it, invisibly** | the first turn or scale writes that middle down, in the same edit, with the `pos` that holds the picture still | `gesture/with-pivot` |
| **choosing it, by hand** | ⌃/⌘-drag the cross on the stage: the pivot goes under the pointer and nothing moves | `gesture/repivot`, `::ui/repivot` |
| **a placement** | `clip/place-symbol` stores the middle of what the symbol draws as the instance's pivot, so an instance turns about its drawing from the moment it is dropped | `clip/place-symbol` |
| **putting it back** | ⌖ beside the pivot row in the inspector: back to the middle of what the node draws *now*, moving nothing | `gesture/centred`, `::ui/centre-pivot` |
**A pivot is a choice, and does not follow the drawing.** Once it is the node's
own, the derived middle is never consulted for it again. This is the half the
old stored anchor got right and the derived pivot got wrong: adding a shape
inside a symbol must not re-aim every keyed spin of every instance of it, and a
pivot that tracked the content did exactly that, silently, with nothing changing
on screen at the moment it happened. The cross is visible and draggable and ⌖
puts it back, which is what the anchor was missing — it was never the storing
that was wrong.
**A peg is an ordinary `:group` parent, `nest/peg`, with `:pinv` captured so
nothing moves when it appears.** It is no longer the answer to "this turns about
the wrong point", and it is still the answer to three things a node's own pivot
is not:
| want | why the node's own pivot is not it | what the peg does |
| --- | --- | --- |
| a pivot **shared** between nodes — an arm and a forearm about one shoulder | two pivots that have to agree frame for frame are not one pivot | one transform, two children hanging off it |
| a **second** transform on one node — a drawing spinning about its middle while the limb swings about the shoulder | a node has one `rot` | stack them, as Harmony does |
| a hand transform over a **measured** one | `gesture/refusal` turns a drag on a measured channel away, because the next regenerate would discard it | the peg's channels are its own, so the hand transform composes outside the measurement, which stays regenerable |
The pivot of a measured node is *not* in that table: `[:xform :pivot]` is
authored on every node alike, never dense and never regenerated, so a traced
mouth can be told to turn about its own middle without a peg and with nothing a
regenerate will throw away. That is the row that used to be impossible — writing
an anchor under a measured `M` moved the thing it was meant to leave alone,
because the old composition was `T(pos)·M·T(-a)` and `pos` was the measurement's.
The conjugated form has no such problem: `T(a)·M·T(-a)` is the identity at `a`
whatever `M` is.
`demo/stage` places its seven faces on pegs, and that is now one way of writing
something a pivot says directly: the faces' `:scale` is **keyed** — they pulse —
and the source's middle has to stay on its authored centre throughout, which a
static `pos` cannot do since `T(pos)·S(k(f))` moves that point whenever `k`
changes. `T(center)·S(k(f))·T(-origin)` does, for every `k`, and so does one
instance with its pivot on the middle. The demo is left as it is, pegs and all:
it is a hand-authored scene that renders correctly and `instance-test` asserts
its structure, and a peg carrying a keyed scale is a perfectly good thing to
have written.
`gesture/about` survives for the one gesture whose pivot belongs to no node: a
**multi-selection** scaling about the middle of its shared box, where every
member has to move to keep the arrangement. Nobody keys that.
Schema 8 is the first version that **converts** rather than refusing. A schema-7
node has no pivot, an absent pivot reads as `[0 0]`, and `T(pos)·T(0)·M·T(-0)` is
`T(pos)·M` to the bit — so every stored document composes to exactly the matrices
it did, dense tier-2 transforms included, and the migration only restamps the
version. What a converted document does not get is a pivot anybody chose; its
nodes still turn about their origins until the first turn writes one or the cross
is dragged.
**The similarity fit already produces a decomposition.** `fitSimilarity` returns **The similarity fit already produces a decomposition.** `fitSimilarity` returns
`{s θ tx ty}`, which drops straight into `[:xform :scale]`, `[:xform :rot]` and `{s θ tx ty}`, which drops straight into `[:xform :scale]`, `[:xform :rot]` and
`[:xform :pos]` with no conversion. The analysis output and the animation model `[:xform :pos]` with no conversion. The analysis output and the animation model
@ -379,7 +556,7 @@ selected head placement, so it aligns with the vectors drawn over it.
:mouth :mouth-in :teeth :lid-r :lid-l :brow-r :brow-l … :mouth :mouth-in :teeth :lid-r :lid-l :brow-r :brow-l …
``` ```
Changing anchor keys edits `:head` and never touches `:face`, so it cannot move Changing anchor keys edits `:head` and never touches `:place`, so it cannot move
something that was placed by hand. A group node is free, and keeping the authored something that was placed by hand. A group node is free, and keeping the authored
and the measured transform apart is the whole reason the transform is decomposed and the measured transform apart is the whole reason the transform is decomposed
in the first place. in the first place.
@ -415,9 +592,11 @@ different rules:
*not* to the plate, which is the whole point of it — so the offset genuinely *not* to the plate, which is the whole point of it — so the offset genuinely
belongs at the node, not the clip. belongs at the node, not the clip.
## Timelines, and why a scene is one ## Symbols, and why a scene is one
A **timeline** is an ordered bag of nodes in its own frame space: A **symbol** is an ordered bag of nodes in its own frame space. (Earlier drafts
and code called this a *timeline*; that word now means only the UI pane that
shows one.)
```clojure ```clojure
{:frames 91 {:frames 91
@ -427,9 +606,10 @@ A **timeline** is an ordered bag of nodes in its own frame space:
That is the whole type, and **everything that holds nodes is one of these**: That is the whole type, and **everything that holds nodes is one of these**:
- a clip's **scene** is its root timeline, - what a document opens on is a symbol, and **no symbol is reserved** — a new
- a **symbol** in the library is a timeline, document's is called `main` only because it has to be called something,
- a node with `:kind :symbol` is an **instance** of one. - anything placed inside another symbol is a symbol,
- a node with `:kind :instance` is an **instance** of one.
An earlier draft of this document had a scene and a `:kind :timeline` symbol as An earlier draft of this document had a scene and a `:kind :timeline` symbol as
two structures with the same fields and never said they were the same thing. two structures with the same fields and never said they were the same thing.
@ -474,7 +654,9 @@ for all three is the same — **their own**:
### Instances ### Instances
A node with `:kind :symbol` and `:of :sym/blink` places one. Its own channels A node with `:kind :instance` and `:source {:symbol :sym/blink}` places one, and
its `:playback` says how time runs inside it — which drawing is used and how it
is played are separate facts, per [the lane model](lane-model.md). Its own channels
compose *over* the symbol's, so one definition is placed many times and tinted, compose *over* the symbol's, so one definition is placed many times and tinted,
offset or retimed at each placement — that is how a three-frame blink is reused offset or retimed at each placement — that is how a three-frame blink is reused
at frames 40, 88 and 200 without copying it. at frames 40, 88 and 200 without copying it.
@ -545,28 +727,26 @@ dense geometry. A topology setting cannot be treated as a per-frame gain curve.
The op list is the boundary with stage 7 in `docs/architecture.md`: the The op list is the boundary with stage 7 in `docs/architecture.md`: the
rasteriser takes ops and knows nothing about nodes, channels or time. rasteriser takes ops and knows nothing about nodes, channels or time.
**A photographic underlay is not an op.** The registered source frame that an **A tracing layer is an op that never reaches the raster.** Footage or a still
animator traces over is a reference, not output, and it may not enter the indexed to draw over is a symbol with `:type :trace` and a `:media`, placed by an ordinary
buffer — the same rule `docs/architecture.md` already sets for handles and instance — so it is moved, scaled, trimmed, held and put in a lane like anything
vertex boxes. It is a `drawImage` at an affine on a separate canvas, which clips else — and it resolves to one `:trace` op: `{:kind :trace :node :layer :media
at the canvas edge for free, and the only thing it needs from the model is the :frame :size :m}`. The raster refuses that kind, the player hands it to a
world transform of the node it rides: `drawImage` on a separate canvas over the picture, and `clip/resolver` makes one
only when asked with `:tracing?`, which only the stage does. An export, a
symbol's centre and a thumbnail never ask, so a reference cannot reach the
picture by any path that forgets to filter it. See `docs/tracing-symbol-plan.md`.
```clojure A face's footage is one of these, placed as `:plate` under `:head` with the
(world-of resolver :head) ;; -> Float64Array[6] anchor fit itself as its measured transform — the inverse of the head's, over
``` image height. Its world is `head · fit · 1/H`, so on a frame where the head and
the plate read the same measured frame the two cancel and the photo sits where
the face was filmed; on any other frame it rides the head. Registration is the
ordinary walk, not a matrix built beside it.
Composed with image-pixels-to-local — **both axes divided by `imgH`**, never by Which frame it shows is the placement's: `:time {:holds [...]}` holds it on
their own dimension — the photo is registered with the shapes by construction, chosen frames, and a head with `:reads {:holds-of :plate}` jumps to the same
and an unregistered underlay is merely decorative. The tracing editor chooses ones. Whether it is showing at all is the editor's, `[:ui :tracing]`.
which source frame to show under a cel. That reference choice is independent of
the finished picture fps and does not change the dense analysis track. A cel can
therefore use any useful source frame as its drawing reference, even when that
frame is not one of the displayed picture poses.
A photo that has to sit *between* two drawn layers is the case that would make it
a `:bitmap` node with an op of its own. Nothing wants that yet: a reference is
either under everything or over everything at low alpha.
### Making it fast in CLJS ### Making it fast in CLJS
@ -619,9 +799,9 @@ Proof that it covers what exists, not just what is wanted:
| square pupil | node `:pupil-r`, `:kind :rect`, parent `:iris-r`, stencil `:iris-r` | | square pupil | node `:pupil-r`, `:kind :rect`, parent `:iris-r`, stencil `:iris-r` |
| brow ring + quantised raise | node `:brow-r`, `[:geom :pts]` dense (the traced ring with height removed), `[:xform :pos]` dense (the quantised raise). **The decomposition design.md insists on is two channels.** | | brow ring + quantised raise | node `:brow-r`, `[:geom :pts]` dense (the traced ring with height removed), `[:xform :pos]` dense (the quantised raise). **The decomposition design.md insists on is two channels.** |
| head plate, kept frames | node `:head`, `:symbol` per instance, keys on `[:symbol]` at kept frames | | head plate, kept frames | node `:head`, `:symbol` per instance, keys on `[:symbol]` at kept frames |
| `makeXform` face-oval crop | **gone.** Placement is `[:xform :*]` on `:face`; the stage clips | | `makeXform` face-oval crop | **gone.** Placement is `[:xform :*]` on the face's own `:place`; the stage clips |
| `stabilize` transforms | dense `[:xform :*]` on `:head`, read through its optional `:anchors` map | | `stabilize` transforms | dense `[:xform :*]` on `:head` (the inverse fit) and on its `:plate` (the fit), read where `:reads` and `:time :holds` say |
| registered underlay | not data — a UI layer riding `(world-of resolver :head)` | | registered underlay | the face's `:plate`, an instance of the footage's tracing symbol under `:head`; a `:trace` op the raster never sees |
| painted background cel | node per layer, `[:geom :pts]` **framed**, `[:style :color]` framed | | painted background cel | node per layer, `[:geom :pts]` **framed**, `[:style :color]` framed |
| `mouth lead` | `:time {:offset k}` on performance nodes only | | `mouth lead` | `:time {:offset k}` on performance nodes only |
| `exposure` | `:time {:expose n}` on the clip root, inherited | | `exposure` | `:time {:expose n}` on the clip root, inherited |
@ -658,28 +838,33 @@ scope does not define resolves to the loud magenta, like any other missing index
### The scope rule ### The scope rule
`:palette` on a timeline is a channel like any other: `:palette-track` points to an ordinary lane symbol. Its clips are instances of
restricted palette symbols: a palette symbol owns no nodes and points at exactly
one project palette.
```clojure ```clojure
{:frames 91 {:id :shot :frames 91 :palette :day :palette-track :shot-palettes :nodes {...}}
:palette {:animated? true :interp :hold :keys {0 :day, 48 :dusk, 72 :night}}
:nodes {...}} {:id :shot-palettes :type :palette-track :display :lane :frames 91
:nodes {:day-clip {:kind :instance :source {:symbol :day-palette} ...}
:dusk-clip {:kind :instance :source {:symbol :dusk-palette} ...}}}
{:id :day-palette :type :palette :palette-ref :day :frames 1 :nodes {}}
``` ```
**Absent means inherit** from the instancing context. **Present means this `:palette` is the symbol's authoring/preview palette. It seeds evaluation only
timeline's content is read in that ramp, and it travels with the timeline** — a when that symbol is the viewed root; nested symbols do not replace the root's
symbol authored against `:night` stays night wherever it is placed. That is choice merely because they were authored under another ramp. When absent, the
lexical scope, and deliberately: a character with their own palette is a project default seeds evaluation.
character, not a decoration of whichever scene they were dropped into.
Composition is the same walk as `:time` — down the instance chain, **innermost Covered clips of the viewed root's palette track override that seed. An
set palette wins**. An enclosing timeline's palette therefore applies to uncovered lane interval is a genuine gap, restoring the authoring palette or
everything inside it that does not set its own, which is adjustment-layer project default. Palette clips use the same trim, roll, slide, claim-time and
behaviour with no adjustment layer in it. It is just scope. undo commands as visual clips; palette code does not duplicate those edits.
Thus palette-track coverage, authoring preview, and project fallback are
And because it is an ordinary channel, a project switches palette over time with separate facts rather than three accidental meanings of one field. There is no
keys on the root timeline, a child timeline switches on its own, and neither second keyed palette control on symbols or instances: time-varying palette
knows about the other. changes are authored only as clips in the palette lane.
### One index space, partitioned by palette ### One index space, partitioned by palette

View file

@ -8,6 +8,9 @@ rotoscoping rather than a sketch bolted to the side.
`docs/animation-model.md` specifies the data both of them are about — nodes, `docs/animation-model.md` specifies the data both of them are about — nodes,
channels, symbols and time maps — and supersedes this document wherever the two channels, symbols and time maps — and supersedes this document wherever the two
describe the same type. describe the same type.
The newer [Lane Model](lane-model.md) takes precedence for occurrence ownership,
playback semantics, shared editing operations, and multi-view UX. It explicitly
allows replacing the current format without backward compatibility.
Nothing here revises an aesthetic decision; several things here split a Nothing here revises an aesthetic decision; several things here split a
decision that is currently made in two places at once. decision that is currently made in two places at once.
@ -49,10 +52,10 @@ overrides and kept-frame sets are all in clip-frame space, so a clip slides on
the timeline without a single stored number changing. Exposure and lead are the timeline without a single stored number changing. Exposure and lead are
transforms *within* clip space: transforms *within* clip space:
Detection retains every source frame. A chosen picture fps samples the frozen Detection retains every source frame. Symbols carry their native fps; project
roto in clip time; it changes neither source-frame count nor the audio clock. fps selects the output grid without changing source data or audio speed.
The set of source frames an artist uses as cel tracing references is another See [Time selection](time.md) for boundary sampling and frame units.
selection, independent of the picture fps. Tracing references remain an independent selection.
```clojure ```clojure
(defn pose-frame [clip cf] (defn pose-frame [clip cf]
@ -87,8 +90,8 @@ it a name is most of the work: `state` in `app.js` is a clip with its analysis
inlined and its palette global. inlined and its palette global.
Cel keys select where drawings begin and how long they hold. The source frames Cel keys select where drawings begin and how long they hold. The source frames
shown beneath a cel while tracing are chosen independently, and picture fps shown beneath a cel while tracing are chosen independently. Project fps controls
only controls which analyzed pose the finished roto displays at a given time. which native frames can appear on the output grid.
### Two things called "track" ### Two things called "track"
@ -639,10 +642,10 @@ is what step 9 implemented, for the subset that exists:
clip/<cid>/name clip/<cid>/subject/<sid> clip/<cid>/name clip/<cid>/subject/<sid>
clip/<cid>/timing clip/<cid>/feature/<fid> clip/<cid>/timing clip/<cid>/feature/<fid>
clip/<cid>/stage clip/<cid>/group/<gid> clip/<cid>/stage clip/<cid>/group/<gid>
clip/<cid>/source clip/<cid>/timeline/<tid> clip/<cid>/source clip/<cid>/symbol/<sid>
clip/<cid>/timeline/<tid>/node/<nid> clip/<cid>/symbol/<sid>/node/<nid>
clip/<cid>/timeline/<tid>/measured/<nid> clip/<cid>/symbol/<sid>/measured/<nid>
clip/<cid>/timeline/<tid>/channel/<nid>/<prop> clip/<cid>/symbol/<sid>/channel/<nid>/<prop>
``` ```
Settings live on subject, feature and group leaves. Each feature has one area, so Settings live on subject, feature and group leaves. Each feature has one area, so
@ -675,6 +678,45 @@ and `~` is then refused inside a name. That is the whole of the escaping.
That maps onto the tiers exactly — the server stores tier 1 and snapshots tier 1, That maps onto the tiers exactly — the server stores tier 1 and snapshots tier 1,
with tiers 2 and 3 as content-addressed blobs beside it. with tiers 2 and 3 as content-addressed blobs beside it.
### As built
- **Addresses.** `/` is the index of the projects you own or edit. A project
is only ever at `/p/<uuid>/<slug>`: the id finds it, the slug is its name and
follows a rename without a history entry. "new" makes the project on the
server first and opens it; a built-in example opened in the editor is saved
at once as a project of its own. There is no bare project. The address, the
title and the socket follow `[:project :id]` through one global interceptor
(`events/collab`).
- **Ownership.** Every project has an owner (`Project.owner`, not nullable) and
`editors`. Anyone with the link reads; the owner and editors write. Making a
project needs you signed in. A reader can make a copy of their own.
`/api/{me,login,signup,logout}`, `/api/projects/<id>/editors[/<username>]`.
- **Every edit saves.** No save button. The same interceptor sees
`:paint/revision` move and saves; one request is in flight at a time, and an
edit made meanwhile goes when it lands — so a drag reaches the room as fast
as the round trip allows. A save sends only the leaves that differ from what
was last synced, and a clean document sends nothing. Analyses and blocks
already put on the server are not asked about again.
- **Saves are patches.** With `base` (the seq last caught up to) a save names
only its changed and removed leaves. A named leaf somebody else changed after
`base`, to something else, fails the whole save with 409.
- **The first write wins.** A remote write to a leaf with a local change not
yet sent waits in the entry's `:behind`; the next save, or a 409, puts theirs
on screen over ours and says so. Ours stays in the undo list.
- **The socket** (`/ws/projects/<uuid>`, channels + daphne) carries presence,
the delta each committed write broadcasts, and `access` when the editor list
changes. A gap in `seq`, a welcome, or a 409 refetches the document.
- **Undo** is per person, recorded in `events/edit` as leaf befores and afters
(`domain/history`), and applied as an ordinary edit. A step undoes only if
every leaf it touched still holds what it left there: somebody else's edit
since refuses it rather than being undone with it. Edits to the same leaves
within a second, each starting where the last left off, are one step.
- **Snapshots** are named revisions (`/api/projects/<id>/revisions`), with each
clip's block keys. Restoring one is an ordinary write, broadcast like any.
Not yet: follow mode, frame/selection in presence, the advisory `:editing`
lease, the durable outbox.
### Why this model, and not a CRDT ### Why this model, and not a CRDT
The usual reason to reach for Yjs or Automerge is automatic convergence without The usual reason to reach for Yjs or Automerge is automatic convergence without
@ -780,17 +822,17 @@ collaborator's keying. The fix is addressing, not an algorithm:
``` ```
palette palette
sequence/:sid lane/:sid
clip/:cid/timing clip rate clip/:cid/timing clip rate
clip/:cid/subject/:sid tracked subject and settings clip/:cid/subject/:sid tracked subject and settings
clip/:cid/feature/:fid tracked feature and settings clip/:cid/feature/:fid tracked feature and settings
clip/:cid/group/:gid shared settings for an eye pair clip/:cid/group/:gid shared settings for an eye pair
clip/:cid/timeline/:tid frame count, palette clip/:cid/symbol/:sid frame count, palette
clip/:cid/timeline/:tid/node/:nid one node: parent, stencil, z, time clip/:cid/symbol/:sid/node/:nid one node: parent, stencil, z, time
clip/:cid/timeline/:tid/channel/:nid/:prop clip/:cid/symbol/:sid/channel/:nid/:prop
clip/:cid/timeline/:tid/measured/:nid clip/:cid/symbol/:sid/measured/:nid
clip/:cid/timeline/:tid/cel/:nid/:frame clip/:cid/symbol/:sid/cel/:nid/:frame
clip/:cid/timeline/:tid/overrides/:nid/:prop clip/:cid/symbol/:sid/overrides/:nid/:prop
``` ```
Each feature and node has its own leaf, so tuning separate features and adding Each feature and node has its own leaf, so tuning separate features and adding

135
docs/clipboard-plan.md Normal file
View file

@ -0,0 +1,135 @@
# Selection and clipboard
This is the implementation contract for multi-selection, copy, cut, paste,
duplicate, and duplicate unique. It deliberately replaces any incidental older
behavior. The document model is the authority; the stage and timeline are two
views of the same editor state.
## One selection
`[:ui :selections]` is the ordered selection set. Its last member is the primary
selection in `[:ui :selection]`, used by the inspector and single-subject tools.
Every member is an occurrence address:
```clojure
[:node owner-symbol-id node-id row-path]
```
The row path distinguishes two occurrences of shared content. Commands that
write the document canonicalize those addresses before acting:
- invalid and non-node addresses are ignored;
- the same owned node, `[owner-symbol-id node-id]`, is acted on once;
- when one selected row path is below another selected row path, only the
ancestor is a clipboard root. Its ordinary parent-pointer subtree comes with
it, and an instance already displays the symbol it references, so also
materializing the visibly nested selection would duplicate it twice.
Plain click replaces the selection. Shift-click toggles membership, on both the
stage and timeline. A stage marquee replaces, or with Shift adds to, the same
set. Timeline rows, bars, and cel blocks render membership from that same set;
the primary member gets the inspector/focus treatment.
Creation targeting is derived rather than stored. The primary (last-selected)
occurrence is the preferred row; the playhead validates it and, when necessary,
walks outward to the nearest valid containing occurrence. A target is singular
even when selection is plural. See `docs/creating-in.md` for the resolver
contract.
## Clipboard value
The clipboard is editor state, not document state and not history. Copy records
a detached snapshot of each canonical root and its complete parent-pointer
subtree. It records source occurrence paths and authored node data, but normal
copy deliberately keeps referenced symbol identities. Therefore a pasted
instance is another use of the same symbol. The clipboard survives cutting its
nodes because it contains the node snapshot, not merely their addresses.
This first implementation is the application's clipboard, not the operating
system clipboard. It is consequently project-local and has no serialization or
cross-project identity collision policy hidden inside it.
## Paste
Paste resolves one destination from the primary selection and playhead. An
ordinary occurrence is usable only while the playhead maps through every
enclosing occurrence and lies within its extent. Otherwise resolution walks
outward, with the open symbol as the total fallback. A selected lane is an
insertion surface only while all occurrences enclosing its parent symbol are
valid. Multi-selection supplies one ordered payload, not several destinations;
only its primary member anchors this resolution.
The earliest finite start among the copied roots is aligned with the playhead
in the destination. All other root starts retain their offset from it. Roots
without a finite span remain timeless; paste does not invent a span for a shape
that was authored for the whole symbol. Parent/child timing, transforms,
channels, corrections, playback, stencil links, and relative root order are
otherwise copied exactly. Every node receives a new identity, and all internal
parent and stencil references are remapped.
In an ordinary composition overlap is valid. In a lane the pasted finite spans
claim their intervals using the lane's existing overwrite rule: covered cels
are removed, crossing cels are trimmed or split, and the pasted roots do not
overlap one another. Pasting a timeless root into a lane is refused. Validation
is all-or-nothing.
After paste, the new roots are the selection, in clipboard order, and the last
one is primary.
## Cut
Cut first takes exactly the same snapshot as copy, then deletes every canonical
root and its parent-pointer subtree. The clipboard write is editor state; the
whole document deletion is one history transaction. The selection is cleared.
Undo restores the deleted document nodes. It does not roll back the clipboard,
which matches ordinary editor behavior.
## Duplicate and Duplicate Unique
Duplicate does not read the insertion target or playhead. It is a local
operation beside the selected material:
- in an ordinary composition, copies keep the originals' parent, transform,
timing, and span, and are stacked immediately in front;
- for direct children of a lane, the selected temporal envelope is repeated
immediately after itself. Relative timing and gaps inside the selected set are
preserved, and later cels ripple forward by the envelope duration. This is
the lane's useful "duplicate forward" behavior; it is not a second command.
Selections in several owners are handled per owner in one command. Thus two
lane selections repeat in their respective lanes, while a selected composition
node duplicates in place, all as one history step.
Normal Duplicate preserves symbol references, just like normal copy/paste.
Duplicate Unique performs the same placement but deep-copies the complete graph
of every referenced symbol. One shared remap table is used for the whole batch,
so two duplicated instances that shared a nested part still share one new copy
with each other, while sharing nothing mutable with the originals. Immutable
media/store blocks may remain shared.
After either duplicate command, the new roots replace the selection.
## History, refusal, and stale state
Cut, paste, duplicate, and duplicate unique each call one domain command and
commit through one `edit/transaction`; each is exactly one undo/redo step no
matter how many nodes or symbols it touches. Copy and selection do not
touch history. A refusal changes no document leaves and creates no history step.
Undo/redo filters the complete selection set against the restored document and
repairs the primary selection. Clipboard payloads remain snapshots. Paste
validates the resolved destination and all remapped references at commit time,
so a stale selection or a newly impossible symbol cycle refuses rather than
partially editing.
## Required tests
Domain tests cover canonical ancestor/descendant selection, subtree ID remaps,
normal shared references, deep unique graph remaps, multi-root relative timing,
composition overlap, lane overwrite, lane forward duplication and ripple,
mixed-owner duplication, cycle refusal, stale targets, and all-or-nothing
failure. Event tests cover copy without history, atomic cut/paste/duplicates,
resulting multi-selection, target fallback at the playhead,
and one undo plus redo of each mutation. Browser tests cover mirrored stage and
timeline selection, Shift-toggle on labels/bars/cels, singular derived creation
targeting, and the keyboard commands.

View file

@ -0,0 +1,234 @@
# Correction authoring implementation plan
Written against `2f1c9b9` (2026-09-30), following the lane handoff in
`7a54bfc`. Implemented on `codex/correction-authoring`; this now records the
scope and acceptance criteria of that implementation.
The cel-sheet targeting work described below was removed with the cel-sheet UI
on 2026-10-01; it remains here only as history of that implementation.
Read [lane-handoff.md](lane-handoff.md) and the correction section of
[lane-model.md](lane-model.md) first. Their ownership and document rules remain
the foundation. The choices below settle the first implementation's scope.
## Outcome
A person can select a lane or cel, specify a range, and apply Constant
adjustment, Ramp, or Return motion to rotation or position. The result is one
correction layer and one undo step. It works from either timing view. A lane
correction crosses drawing boundaries; a cel correction travels with its cel.
Regeneration preserves the hand work and presents incompatible layers for an
explicit decision. Frames outside the support evaluate exactly as before.
Finish this vertical slice before adding more property types or gestures.
Numeric range fields and an Apply button are sufficient for this pass. Dragging
a range or manipulating a peak on the stage can later issue the same command.
## 1. Fix sheet targeting first
In `ui/timeline.cljs`, `cel-sheet` currently drops each lane row's `:select`.
An occupied cell selects its cel; a gap only seeks, leaving the previous target
selected. Thus clicking lane B's gap after selecting lane A can send an insert
or overwrite to A.
Carry the row's complete selection address into its column and gap cells.
An occupied cell selects its cel; a gap selects its lane. Make the column header
select the lane too: this is the explicit way to author across drawings.
Preserve full paths, not just `(peek path)`, as view identity. Keep seek and
selection dispatch order deterministic.
Add a two-lane browser case: select A, click a gap in B, overwrite, assert that
only B changes, and undo once. Add a header-selection assertion. Retain the
existing occupied-cell hold test. Do not redesign sheet rendering in this step.
## 2. Make stack compatibility consistent
There is a concrete discrepancy at this HEAD:
- `channel/problems` uses `stack-conflict`, accounting for prior replacements.
- `channel/conflicts` and `flow/regenerate.cljs`'s `rebased` use
`conflict-with`, comparing an offset directly with the base.
A two-component base, a covering three-component replacement, then a
three-component offset is valid and evaluates correctly, but the latter paths
can report or mark that offset incompatible. Conversely a replacement can make
an offset incompatible even when it fits the original base.
Extract one ordered-stack compatibility operation and use it for validation,
conflict discovery, regeneration, and resolution. Keep `conflict-with` if useful
for the narrower question its name/docstring describe. Do not use it alone to
decide whether a stacked layer is applicable.
Compute compatibility against the values that can actually reach a layer over
its support. Partition at overlapping support boundaries if needed: two adjacent
replacements can jointly cover an offset even though neither covers it alone.
An empty replacement channel does not supply a value and must not erase the
possible input shape. Preserve the evaluator's absence behavior. Explicitly
marked conflicts are skipped, so later layers must be checked against the stack
that actually runs. During regeneration, recompute compatibility in order, using
each preceding layer's resulting active/conflicted state. Preserve IDs, values,
support, and order; update compatibility reasons without dropping hand work.
Use structural shape reasoning for dense data rather than requiring every block
to be sampled. If unknown shape or missing samples limit what can be proven,
retain the current absence contract and document that limit; do not claim an
unconditional proof of runtime safety from incomplete metadata.
Tests: covering replacement of a different shape; partial coverage; adjacent
covering replacements; empty replacement; inactive conflicted replacement;
regeneration changing base shape; and the same cases through cursor evaluation.
Assert that an accepted compatible stack is not listed as a conflict, and that
an incompatible regenerated layer remains persisted but is skipped.
## 3. Pure correction commands
Add `frontend/src/arthur/domain/correction.cljs`. It owns authoring and resolving
corrections; `channel.cljs` continues to own evaluation and compatibility.
Suggested API (names may follow repository conventions):
```clojure
(add clip sid node-id channel-path
{:id layer-id :support [a b] :motion :return
:start 0 :peak angle :peak-frame p})
(remove-layer clip sid node-id channel-path layer-id)
(retry-layer clip sid node-id channel-path layer-id)
```
Return `{:clip updated :selection node-id}` or `{:refused reason}`. IDs come
from the event caller (`random-uuid`), never from the pure command. Reject a nil
ID or one already used within that channel stack. Address layers by the full
symbol/node/channel/layer tuple; no global layer registry is needed.
Resolve the base via `node/channels`, which supplies defaults. A lane with no
explicit rotation channel already has a zero rotation; materialize that channel
with its new `:over`. Preserve every existing base field, generated provenance,
and previous layer. Never route this through a setter that bakes the correction
into base keys. Append to the ordered stack and validate the resulting document.
Do not run correction edits through `lane/finish`, whose extent policy belongs
to cel arrangement. Use `clip/problems` for the candidate document instead.
First authoring properties: `[:xform :rot]` (scalar radians) and `[:xform :pos]`
(two numeric components). First blend operation: `:offset`. UI labels must say
offset/delta, since a target offset of 20 degrees does not mean an absolute
rotation of 20 degrees. Keep existing `:replace` evaluation and loaded stacks;
there is no new replacement-authoring UI in this slice.
All command support endpoints are finite integer OWNER frames, `[a b)`, with
`a < b`. Do not ban negative owner frames merely because displayed shot frames
start at zero. Validate all supplied values for finite numbers and exact shape.
Refuse unknown targets, unsupported properties/motions, malformed ranges, and
incompatible stacks with useful messages. Refusal must not mutate store/history.
Motion construction uses existing channels only:
| Command | Values | Minimum samples |
| --- | --- | --- |
| Constant adjustment | `(ch/framed delta)` | 1 |
| Ramp | `(ch/keyed {a start, (dec b) end} :linear)` | 2 |
| Return motion | `(ch/keyed {a start, p peak, (dec b) start} :linear)` | 3 |
For Return, require integer `a < p < b-1`. Default the UI peak to
`a + floor((b-a-1)/2)`; on an even-length range the earlier middle sample wins.
Expose the peak frame so this is visible and adjustable. Never place an endpoint
at `b`: it is outside the selected samples. `[10 13)` with start 0 and peak 0.5
must yield offsets `0, 0.5, 0` at 10, 11, 12. Support controls the boundary;
there is no need to insert zero keys into the base before/after it.
## 4. Owner and range UI
Add a Corrections section to the right pane (`ui/params.cljs`), extracting a
`ui/corrections.cljs` component if that keeps the pane readable. Provide an
explicit target readout (symbol, lane or cel), Rotation/Position, motion choice,
From/Through fields, relevant value fields, peak frame for Return, and Apply.
Offer a selected cel's owning lane as an explicit target choice. Do not silently
promote a cel edit to a lane edit. Shared drawing content is outside this first
UI; it has different sharing consequences.
For this first pass the range fields explicitly read **owner frames**, with
inclusive From/Through converted to `[from, through+1)`. This is a deliberate
UI scope choice, not a claim that a displayed shot range and owner range are
interchangeable. It lets nested and retimed owners be addressed without an
unproven range conversion. Match the existing zero-based numbering and show the
owner beside the range. The handoff must record that displayed-range dragging
is still outstanding.
Use an explicit three-sample initial draft in owner coordinates; for a cel,
prefer its span start where it is integral. Show the range, allow adjustment,
and do not extend a cel or shot to make the correction visible. Reset the draft
when the target changes. Rotation is shown in degrees and converted to radians
at the event boundary, following the existing inspector convention. Position
uses x/y inputs in the owner's transform coordinates.
Draft inputs must not write document state, start history groups, or invoke the
existing inspector `number-input`'s hold/settle behavior. Apply dispatches one
event; success uses `edit/transaction` once and preserves the selected node's
full address. Do not use a layer ID as node selection. Validate again at Apply,
since the target/document may have changed since the draft was opened.
If adding a “use playhead” convenience, prove its mapping separately. The clock
of a cel's transform is its own node clock, not the drawing source clock selected
by `:playback`. `nest/inside` on the complete cel path enters the source and is
therefore the wrong shortcut. The existing `selection-frame` resolves only the
owning symbol; node/ancestor time conversion remains necessary. Floors, loops,
and nonintegral mappings must never silently snap an authored range. Omit this
convenience rather than expanding the first pass into a new timing system.
## 5. Conflict actions and regeneration proof
Show a document-wide list from `clip/conflicts` in the pane, including symbol,
node, property, layer ID, and reason. Keep it accessible even when a different
node is selected. Also list the selected target's layers in stack order with
support, motion values, and status; do not require a new persisted motion label.
Provide Remove correction and Retry compatibility. Remove is explicit and
undoable; Retry rechecks the complete candidate stack and clears a conflict only
when it is valid. If retry would invalidate a downstream offset, refuse and say
why. Similarly, removing a replacement that makes a later offset invalid must
refuse, rather than commit an invalid document or silently remove more layers.
A retry that changes nothing must not manufacture an undo step.
These are minimal resolution actions, not topology remapping. Automatic geometry
remapping, reordering layers, editing arbitrary stored vector values, and resolving
removed targets are separate work. Preserve all existing generated-data behavior.
Exercise the actual `flow/regenerate.cljs` entry points in integration tests:
author a correction, regenerate compatible base data, and confirm the correction
survives with its ID/support/values intact and affects the new base. Then change
topology on a geometry fixture and assert persisted actionable conflicts. The
geometry fixture can use an existing layer directly; geometry authoring is not
required to expose and resolve a conflict already present in a document.
## 6. Verification and completion
Use focused tests that establish observable promises:
- Domain: each motion's sample values; refusal on short ranges/nonfinite values;
default channel materialization; unchanged base and prior layers; duplicate ID.
- Evaluation: compare before/after on every frame outside support, including
neighboring interpolated frames. Cursor/spec agreement in nonmonotonic order.
- Ownership: a lane Return crosses a drawing boundary; a cel correction moves
with the cel and survives split/trim. Neither alters another use of its drawing.
- Sampling: picture-rate/pose selection changes the generated base frame while
the authored correction still reads owner time. Keep HEAD's regression tests.
- Events: one Apply is one undo step; undo/redo restores complete layer data;
refusal leaves clip/history unchanged; stale target refuses; selection survives.
- Persistence: leaf and Transit round trips retain layers, order, IDs, conflicts.
- Browser: both views can select a target and apply the same correction using
actual controls. Assert evaluated results and history, not only a layer count.
Include the two-lane gap-targeting case from step 1.
- Regeneration and conflict actions: use the real flow and test undoable removal,
valid retry, invalid retry, and removal that would break a downstream layer.
Run the suites documented in `lane-handoff.md`: CLJS tests, lane and take browser
flows, Django tests, and optimized frontend build. Restore the dev app bundle
after the release build. Note that `take.mjs` writes a local project. Report
actual results and any unrun checks; do not copy previous test counts as evidence.
Suggested commit sequence: sheet targeting; consistent stack compatibility;
pure correction commands; pane/events plus browser proof; updated handoff.
Keep each commit coherent and tested. No schema version bump should be needed:
the layers already have a persisted representation.
Update `lane-handoff.md` and `lane-model.md` with what shipped, the owner-frame
range UI limitation, conflict actions available, and verified test counts. Done
means a person can author and undo the correction, regenerate its base, and see
either their preserved edit or a useful conflict. A constructor without reachable
controls, or controls without that regeneration proof, does not finish this work.

155
docs/creating-in.md Normal file
View file

@ -0,0 +1,155 @@
# Creating in
Revised 2026-10-02.
Arthur's creation model is a layer list plus a playhead. The primary active row
is the preferred place for a new thing. The playhead decides whether that row is
present in the current occurrence and supplies the frame at which the thing is
created.
This is editor behavior, not document structure. A saved clip has symbols,
instances and spans; it does not have timeline rows or a creation target.
## Selection and the active row
The selection set names the objects affected by copy, delete and transform. Its
primary member is also the active row, analogous to the active layer in a paint
program. There is no second targeting gesture for a user to maintain.
- Clicking a row label, its timeline body, or the same occurrence on the stage
makes that occurrence primary.
- Shift/marquee selection may retain several objects, but only the primary row
anchors creation.
- Clicking blank stage or timeline space returns the active row to the symbol in
the current tab.
- Row disclosure is independent. Selecting or creating never implicitly expands
a row.
The selection does not change merely because the playhead moves. An off-frame
object remains available for copying, deletion and inspection.
## Resolving the active row at the playhead
Resolution starts with the structural destination implied by the primary row:
- A lane row means the lane symbol itself.
- An ordinary symbol-instance row means the symbol placed by that occurrence.
- A non-instance row means the symbol containing that node.
- No row means the symbol in the current tab.
For an ordinary symbol occurrence, the playhead must be within that occurrence's
extent in the current nested context. Extents are tested after walking all parent
time maps and use the half-open interval `[in, out)`. If the preferred occurrence
is not present, resolution walks outward to the nearest parent whose occurrence
is present. A containing lane is therefore the natural fallback from an inactive
cel. If no nested occurrence is present, the current tab is the destination.
The preferred row is retained during fallback. Scrubbing back into its extent
makes it the effective destination again.
A lane differs only in what its row means. It is an insertion surface across its
containing timeline and does not require an existing cel under the playhead. A
cel within the lane is still an ordinary symbol occurrence with an extent.
For `A -> B -> lane C -> cel D`:
- over D, with D primary, creation happens inside D;
- past D but while C is available, creation inserts a new cel in C;
- outside C's containing occurrence but inside B, creation happens inside B;
- outside every nested occurrence, creation happens in A, the current tab.
## What creation does
Once resolved, every creation command follows the destination kind.
### Lane destination
- A new or dropped symbol is instantiated directly in the lane at the playhead.
- Its interval claims that time. Existing cels under the interval are removed or
trimmed by the lane's ordinary claim-time command.
- Beginning a drawing creates a new one-frame drawing symbol in the lane and the
finished shape is a child of that drawing.
- Selecting an existing cel changes the destination from the lane to the symbol
placed by that cel; subsequent symbols and shapes become children there.
Double-clicking a lane creates an empty cel at the playhead; beginning a drawing
creates a drawing cel there. These are the same lane-creation operation with
different payloads. The pointer chooses the lane, never a second creation time.
The resulting cel is selected, so it immediately becomes the preferred target:
drawing again enters that cel's symbol instead of replacing it.
Thus no separate "new cel" versus "add inside" mode is needed. Selecting the
lane header says new cel; selecting a cel says add inside.
### Ordinary symbol destination
- A new symbol is instantiated as a child at the mapped playhead frame.
- A new shape is authored directly in the symbol at that frame.
- Stage coordinates are transformed through the occurrence into the destination
symbol's local coordinates.
### Explicit timeline drop
A timeline drop uses the row and frame under the pointer, not the stored active
row and playhead. Dropping onto a lane therefore always instantiates in that lane
and claims the pointer's interval. A stage drop uses the resolved active row and
the playhead.
Paste, imported symbols and converted footage obey the same resolver as direct
creation. They must not each reconstruct nesting or extent fallback separately.
## Multi-selection and paste
Multi-selection does not create multiple insertion targets. Copy and cut take
the canonical forest of selected roots as one ordered payload; the primary
(last-selected) occurrence alone supplies the preferred row for a later paste.
At paste time that row and the current playhead resolve one effective
destination, and every root in the payload is inserted there in one transaction.
The earliest finite root start is aligned to the destination frame. Other roots
keep their timing offsets, hierarchy, and clipboard order. In an ordinary symbol
the roots may overlap. In a lane every root must have a finite span and the
payload's root spans must not overlap one another; valid spans claim their times
and trim or remove existing cels as a batch. An invalid member refuses the whole
paste rather than inserting a partial payload.
After paste, all new roots form the selection and the final root is primary, so
it becomes the preferred row for the next creation. Pasting one payload into
several selected destinations is intentionally not implicit: that would be a
separate distribute command. Duplicate is also distinct from paste—it stays
beside each source in its original owner and does not consult the playhead or
creation target.
## Palette lanes
Palette lanes use the same row-and-playhead resolution. Their content filter and
transition command remain palette-specific: only palettes and palette
transitions can be inserted there. This is a type restriction, not a second
targeting model.
## Implementation boundary
One pure resolver returns the effective destination:
```clojure
{:kind :lane | :symbol
:sid destination-symbol
:path effective-occurrence-path
:frame destination-local-frame
:matrix destination-to-current-tab-transform}
```
Callers may add the unchanged document as `:clip` or rename `:frame` to `:at`,
but they must not reinterpret the active row. Polygon creation, symbol creation,
stage drops, paste, import and footage conversion all consume this answer.
Explicit timeline drops use the same structural row rule with the row and frame
under the pointer; unlike playhead resolution, an invalid pointer destination is
refused rather than allowed to fall outward.
The invariants are:
1. The primary row is the preferred structural destination.
2. The playhead validates occurrences and supplies creation time.
3. Inactive targets fall outward; selection does not follow them.
4. A lane row inserts a cel, while a cel row enters its symbol.
5. Stage creation uses the playhead; timeline drops use pointer time.

510
docs/frame-selection.md Normal file
View file

@ -0,0 +1,510 @@
# Frame selection
> Since `docs/tracing-symbol-plan.md`: `domain/trace` is gone. Trace keys are
> the face's `:plate` placement's `:time :holds`, the origin is the head's
> `:reads`, and the photo's registration is the plate's own measured channels.
> Where this document names `trace/measured-local` or `trace/prepare`, read the
> plate's or the head's measured channels and `node/hold`.
Two mechanisms. One vocabulary. An earlier draft of this document claimed they
were one component used twice — because `suggestPlateFrames` in the old
`js/pipeline.js` and the never-built "performance poses" of
[timing-handoff](timing-handoff.md) looked like the same function — and that claim
is wrong. They share how a selection is *read* and how the hand overrides one.
They do not share how frames get chosen, because the two are answering questions
of different shapes.
[Time selection](time.md) is the floor both stand on: an output frame reads the
latest native frame at or before its time, and nothing rewrites the dense
measurements. That is a *cadence*: an answer with no opinion about content. It
cannot know that the one frame where the eye is fully closed is worth more than
its neighbours, so at 12fps out of 30 it drops that frame two times in three.
This document is how the picture gets an opinion.
## The one idea
**A selection is a set of frames chosen out of a dense measurement, and read by
holding the latest one at or before now.**
The holding half already exists and is already shared: `pose/held-frame` is called
by `node/hold` (a placement's `:time :holds`) and by `pose/source-frame`, which is the two sites agreeing
about reading. The hand half is shared too — see *Three layers* below. Choosing is
what differs.
| | Plate drawings (tracing) | Performance poses |
| --- | --- | --- |
| the question | which frames does an artist have to draw a head on? | which frames does the picture change a shape on? |
| the cost being managed | a person drawing | a pose looking wrong |
| signal | the measured head's motion | — none; a stored cut |
| the baseline it improves on | drawing on 2s | the cadence, or the exposure grid |
| the shape of the answer | a non-uniform set out of dense | the same grid, nudged |
| lives on | the face's `:plate` `:time :holds` | the instance's `:playback :tracks` |
| hand edit today | `::project/toggle-hold` | `pose/put-cut` / `pose/remove-cut` |
| UI today | `params/layer-section` | **none** |
| proposes today | **nothing** | **nothing** |
## Why they are not one function
A plate selection has to be **non-uniform**, and that is the whole reason it
exists. A head still for sixty frames and then whipping across in ten wants two
drawings for the first stretch and eight for the second. Drawing on 2s gives
thirty-five drawings, most of them identical, and no amount of nudging a uniform
grid will produce the distribution that is wanted — the spacing itself is the
answer. That is what the prototype's walk was for, and it is why a cost knob
(`:tolerance`) belongs on this side: the artist is buying drawings.
A performance selection is **not choosing sparseness at all**. The output rate or
the exposure setting has already chosen it. The question left over is only *which*
native frame each already-decided slot reads, and the failure it fixes is narrow:
a slot landing one or two frames off the closure. Nudging the grid is the right
size of answer, and there is nothing for a tolerance to mean.
There is a second, harder reason, and it is the one that settles it:
**A selection cannot put a frame on screen that the output grid never samples.**
At 12fps out of 30, output frame 5 reads native 12 and output frame 6 reads native
15. A closure at native 13 is *between* them. Protecting frame 13 in a set of
kept frames makes it available and makes it the frame held across 13 and 14 in
native space — and at a 12fps output it still never appears, exactly as
[time.md](time.md) says: an event between output frames cannot create an extra
frame in a 12fps output. Only moving what output frame 6 reads can show it. So
the performance side has to act on the grid, not on a set beside it.
## Three layers, and the middle one is derived
The trap this is designed around is stated in
[timing-handoff](timing-handoff.md) and is worth repeating because it is the
only hard rule here:
> Store manual edits separately from generated proposals so changing the rate or
> tolerance retains hand decisions.
So a selection is:
```clojure
{:policy {:tolerance 0.02} ; what the proposer was asked for
:keep #{47} ; frames the hand insists on
:drop #{30}} ; frames the hand refuses
```
and the effective set is `(proposed ∪ keep) \ drop`, always containing frame 0.
`:keep` and `:drop` are the document. The proposal is not: it is recomputed from
`:policy` and the dense signal whenever either changes. **Re-suggesting at a new
tolerance must never cost somebody their pinned blink**, and that is the entire
reason the hand decisions are stored as their own two sets rather than as the
resulting frame list.
This layering is the part that really is shared. On the performance side there is
no `:policy` worth storing — the grid is the policy — but `:keep` and `:drop` mean
exactly what they mean on the plate side, and `select/effective` is the one
implementation for both. A preserve mark *is* a keep.
### Materialise the result, do not derive it on the render path
The effective set is written back to where each site already reads it —
the plate's `:time :holds`, or the pose track — so that every existing reader is untouched
and nothing on the per-frame path has to open a dense block. Proposing is a
command, not a subscription. `ch/value-at` allocates per call and says so; that
is fine for a button press over a few hundred frames and would not be fine at
30fps.
This means the stored frame list is redundant with `policy + keep + drop`. That
is deliberate and it is the cheap direction of the trade: a stale list is
recoverable by pressing Suggest again, and a dense read per node per frame is
not recoverable at all.
## The plate selection: a non-uniform chooser
`arthur.domain.select`, built — see *What is revertible* for the one part of it
that is not yet wanted. Pure, no store access, no clip access: the site hands it a
signal it has already read.
```clojure
(defn propose
"Frames worth keeping out of `n`, given `signal`."
[n signal {:keys [tolerance protect]}])
(defn effective
"`proposed` with the hand's decisions applied. Always contains 0."
[proposed keep drop])
```
`propose` is the prototype's walk, generalised off landmarks:
1. keep frame 0, make it the anchor;
2. settle the protected frames from `:protect` *before* walking;
3. for each later frame, keep it when it is protected, or when
`distance(anchor, f) > tolerance`. Either way it becomes the anchor.
`distance` is the max absolute difference over components, so a signal of
landmark pairs and a signal of one number both work without the caller saying
which it handed over. Every sample in a signal is the same width, and a signal of
two widths is refused: comparing the prefix two samples happen to share would let
a reader that drops a component read as no movement at all.
**A protected frame anchors the walk like any other kept frame**, which is why
protection is settled first and is not unioned onto the walk's result. The anchor
is what is on screen; once a protected frame is kept the viewer is looking at it,
so measuring the next frame's drift from a frame no longer displayed is wrong.
**An absent measurement is `nil`, and converting to that is the reader's job.** A
frame where the face was not found says nothing about the signal: it cannot move
the anchor and it is not a frame worth keeping. `trace/measured-local` already
returns nil there. A protected frame with no measurement is still kept — a plate
frame is a frame somebody draws on whether or not the detector found a face.
**A non-finite tolerance falls back to nought.** A cleared slider reads as NaN and
every comparison against NaN is false, which taken literally proposes frame 0
alone and collapses the whole take to one drawing. Nought proposes every frame
that changes, which is merely the baseline back again: wrong in a way somebody can
see and undo.
### The signal
The head's measured transform, applied to a fixed reference quad, giving
displacement in stage units — so a tolerance means "the head has moved this far"
and is a number a person can reason about. `trace/measured-local` already builds
that matrix per frame and is private; make it public rather than writing a second
one. Map it over the frames and transform four corners through `node/apply-pt!`.
The quad's size is a real parameter hiding in the word "fixed": it sets how much
rotation registers against translation. Give it a name and a comment rather than
an inline literal.
Do not reach for raw landmarks. The prototype used them because it had them lying
around; the transform is what the drawing actually follows, it is already on the
node, and it is three channels instead of a block.
### The upgrade path, and do not start here
The greedy walk is order-dependent and slightly suboptimal. The optimal version
is a dynamic program — choose `k` frames minimising held-reconstruction error,
which is textbook segmented least squares and is O(n²k), nothing at n≈300 — and
it keeps extrema *for free*, because an extremum is exactly where a zero-order
hold is most wrong.
Build the greedy one first anyway. It is proven, it shipped in the prototype, and
having two implementations to compare is how the DP gets tested. Swap it behind
`propose` afterwards, where the signature already permits it.
Most of `select_test` pins the greedy walk's exact output, deliberately, for that
comparison — so expect to rewrite those expectations when the DP lands, and keep
them as greedy-specific tests rather than deleting them. The assertion that is a
*spec* rather than a pinned vector, and should be written on this side before the
swap, is:
> reading the signal through the selection, held, never differs from the dense
> measurement by more than `tolerance`
That is what makes the tolerance number mean something to a person. It is true of
the greedy walk by construction and it is what the DP optimises, so it survives
the swap untouched.
## The performance selection: a preserve-snap on the grid
Not built. The rule is ten lines; getting the two halves of it into the same place
is the work. An earlier draft of this section said "about thirty lines" and that
was understated — see *Where it goes* below.
A grid slot already picks a native frame — `cadence/frame` for the output rate, or
the exposure fold for a deliberate hold at full rate. Write `d(k)` for the native
frame slot `k` defaults to. Slot `k` is the first slot to cover everything in
`(d(k-1), d(k)]`, and the frames strictly inside that interval are the ones the
grid shows to nobody. So:
> **Slot `k` reads the latest preserved frame in `(d(k-1), d(k)]`, and `d(k)` when
> there is none.**
That is the whole mechanism. It recovers a dropped frame out of the slot's own gap,
it can never read a frame another slot already showed, and it cannot reach past
`d(k)`.
**Snap backward only, never forward**, and note which direction that actually is,
because it is easy to get backwards. The closure at native 13 in a 12-from-30
output is recovered by slot **6** — whose default is 15 — reading 13. It is *not*
recovered by slot 5, whose default is 12, reaching forward to 13: slot 5's instant
is 5/12s = 0.4167s and native 13's is 13/30s = 0.4333s, so that would show the
closure 17ms before the mouth shut. `cadence/frame`'s contract is the latest native
frame at or before the slot's time and `cadence_test` asserts
`selected <= f*native/grid` over every grid and native pair, so reaching forward
breaks a tested invariant as well as the no-lead rule. Reading 13 at slot 6 shows
the closure two native frames late, which is the same lateness every hold already
has.
**The marks come from a cut that is already stored.** `flow/freeze` computes both
closures and keys them as `[:vis]`, with the thresholding and hysteresis already
decided:
- the mouth, from the aperture relative to the take's peak — `[:vis]` on
`:mouth-in`, provenance `:roto/mouth-aperture` (`freeze.cljs:473`);
- the eyes, from `condition/resolve-blink` with its cut, dwell and hold — `[:vis]`
keyed per eye part, provenance `:roto/blink` (`freeze.cljs:533`).
So "preserve the frames the cut says shut" reads what the document already holds.
No new signal, no new dense track, no threshold decided twice. Brows have no
closure and get no marks, which is correct: there is no extreme brow position
worth protecting.
**Precompute the mark set when the resolver is built**, the way `pose/prepare` and
`trace/prepare` already do (`symbol.cljs:420`, `:470`, `:536`). A `[:vis]` channel
is keys, not dense, so reading it per node per frame would be cheap — but the snap
also needs the marks sorted for a backward lookup, and building that per frame is
the one thing [animation-model.md](animation-model.md) and the render-path note
above both forbid.
### Where it goes, and why it is not a one-liner
The rule needs two things that currently live at opposite ends of the resolver:
- **the slot interval** `(d(k-1), d(k)]`, which needs the output slot index and the
fps ratio. Both exist at `clip.cljs:261`, the single place the grid becomes a
native frame: `(cadence/frame f (or (:grid-fps opts) (:fps clip)) (fps clip sid))`.
- **the marks**, which are per pose group, and so belong where group identity
exists: `symbol/base-channel-frame` (`symbol.cljs:399`), whose `:pose-sampled?`
branch already calls `pose/source-frame` with `(js/Math.floor lf)` as the default
pose. **That default is the snap's seat.** Replacing it leaves an explicit hand
cut winning over a snap, which is correct — manual precedence is absolute — and
costs no new plumbing on the pose side, because per-group choices are already
threaded and already prepared.
By the time control reaches `base-channel-frame` there is only `lf`, a native local
frame that placement and retime have already been through, so the slot interval
cannot be recovered there. It has to be threaded down from `clip.cljs:261`
alongside the frame. That is the actual work of this step: two namespaces' internal
signatures, not a drop-in.
**Do not take the shortcut of snapping at `clip.cljs:261` itself.** It is right
there, it needs no threading, and it is wrong: one native frame per output frame
means the *whole picture* reads 13 instead of 15, so the head goes two frames stale
for one output frame to fix the mouth. At 12fps that is a 67ms hitch on a moving
head, and it fights the trace selection, which has its own opinion about which head
frame to show. The snap is per group because the thing being recovered is one
group's closure.
### Why not an aperture signal
An earlier draft had this side read the group's aperture as a single-component
signal and hand it to `propose` with `:protect :extrema`. It cannot. The aperture
is the separation of landmarks 13 and 14, at positions 5 and 15 of the 20-slot
`LIPS-INNER` ring, and `freeze/rings->flat` subsamples the ring to the `verts`
budget: those two positions survive only when `verts` is a multiple of four. At
`verts` 6, 10, 14 and 18 — all legal, all even — they are not in the stored data
at all. `ring/subsample-slots` used to claim otherwise and has been corrected.
`flow/measure/mouth` does compute the exact scalar and `freeze` does throw it away
after thresholding, so storing it was an option. Reading the cut is strictly less
work and decides nothing twice.
## How the two interact
They are keyed in **the same frame space**: `clip/resolver` hands an instance's
`:playback :tracks` down into the child it places, and `symbol/base-channel-frame`
reads both the trace and the pose choices at `lf`, the node's local frame inside
that face. A trace frame and a pose cut are the same kind of number.
What differs is the owner, and that asymmetry is load-bearing:
- the **trace** is the face's, on its `:head` — every instance of that face shares
it, because it says how the drawings were made;
- the **pose tracks** are the instance's — two placements of one face can be
timed differently.
### The hazard, which is already written down
[animation-model.md](animation-model.md) states it for exposure and it is the
same hazard here:
> a head cutting on odd frames against a mouth cutting on even ones reads as two
> performances
**It is not a correctness problem.** The mouth is a child of `:head` and its
geometry is stored head-local, so a mouth from frame 17 composed onto a head held
at frame 12 is exactly lip-sync on a held drawing — the decomposition already
decoupled them and nothing is geometrically wrong. The problem is perceptual, and
perceptual problems want a constraint rather than a repair.
### Nest them, do not couple them
The two are not peers. One is coarse and expensive — the prototype's own comment
says it: *"The cost being managed is an artist drawing a head, which is why the
signal is head pose and not the mouth — the mouth is traced and free."* The other
is fine and cheap.
So the rule is a subset, in one direction only:
**Every kept plate frame is a preserved frame of the performance selection.**
When the drawing changes, the performance changes with it, so the two can never
cut against each other on neighbouring frames. The mouth stays free to change on
frames where the head does not, which is what shooting a held drawing with a live
mouth *is*.
This is why the preserve set is a set and not a closure predicate: plate frames
and shut-mouth frames go into the same pile, and the snap does not care which is
which. It needs no new mechanism on either side.
The reverse is a suggestion and never automatic. A mouth closure is a reasonable
place to want a new drawing, but proposing one spends somebody's afternoon. Offer
it; do not take it.
It also composes with `:origin` for free. A head on `:continuous` has opted out
of its own selection, so there are no plate frames to preserve and the constraint
is vacuous — which is correct, because a continuously moving head cannot cut
against anything.
### Between the kept frames is a third shared field
The head's `:reads` (once `:trace :origin`) is not a tracing setting. It is the answer to *what happens
between kept frames*, and the plate selection has to answer it:
- `:continuous` — ignore the selection for this purpose and read the frame you
are on,
- `:keys` — jump to each kept frame and hold it, a hold and not a tween,
- `:start` — hold the first forever.
The performance side does not need the field: a snap picks which frame a slot
reads and the grid does the holding, so `:keys` is the only behaviour there is.
Do not rename `:origin` to match anything: for a head it genuinely means where the
face's origin goes, and a saved field in a shipped UI is not worth churning for a
vocabulary tidy.
## The modes
One setting, two states, and **"manual" is not a third state.**
- **off** — the cadence alone, which is what ships today. The output frame
reads the latest native frame at or before it, and nothing has an opinion.
- **smart frame picking** — on the plate side the proposal is live, and `:policy`
holds the tolerance; on the performance side the snap is active.
Hand keeps and drops apply in **both** states, which is why they are not a mode:
turning smart picking off must not throw away the frames somebody pinned, and
pinning a frame with smart picking off is a perfectly reasonable thing to want.
Manual precedence is absolute — a drop beats a proposal, always.
The name on the toggle should be the same word in both sections. "Smart frame
picking" is fine. What it must not be is two different names for the one idea,
which is how these became two features the first time. That the two sections are
now backed by different code is an implementation fact and must not reach the UI.
## The UI
One component rendered twice, in `ui/params.cljs`:
```
smart frame picking [ off | on ]
tolerance [ ----•------- ] 0.02 (plate section only)
[ Suggest ]
frames 0 12 30 47* 61 (* = kept by hand, strikethrough = dropped)
```
The frame strip already exists in miniature — `params/trace-keys` draws the trace
keys as seek buttons (`params.cljs:386`). Lift it into a shared component and
give it three affordances: click to seek, a modifier to pin, a modifier to drop.
A pinned frame and a proposed frame must be visually distinct, because "will this
survive me moving the slider?" is the question the strip exists to answer.
Render it in two sections:
- **`tracing · <face>`** (`params.cljs:426`), beside the existing origin row,
with the tolerance slider and Suggest.
- **`performance · <group>`** — new, on an instance's inspector, one per pose
group the placed symbol has. No tolerance: the strip shows the preserved frames
and the grid slots that snapped to them.
## Order to build it
1. **`domain/select` with the greedy walk, and its tests.** **Done**, in commit
`02069e8`. Pure, no store, no clip. Note that the extrema part of it is not
wanted by anything below — see *What is revertible*.
2. **The preserve-snap**, pulled forward ahead of the plate side because it is the
half with no UI and nothing proposing today, and because it needs nothing from
the plate side except a fold that can land last. Three pieces: the mark set from
the stored `[:vis]` cuts, built in a prepare step beside `pose/prepare`; the
slot interval threaded from `clip.cljs:261`; the rule itself, seated in
`base-channel-frame`'s default pose. The test that matters is the same synthetic
case `select_test` uses — a one-frame closure at native 13 that a 12-from-30
grid drops — asserted end to end this time: the resolver shows a shut mouth on
exactly one output frame, and the head's frame does not move while it happens.
Write it first.
3. **The plate signal reader.** `trace/head-signal`: make `trace/measured-local`
public, map it over the frames, four corners through `node/apply-pt!`. Test
that it returns a vector of the right length, that a motionless take proposes
`[0]`, and that a stretch where the face was not found becomes `nil` rather
than a pose, a zero or a gap in the vector. Add the held-reconstruction
invariant from *The upgrade path* here, since this is the first place a real
signal exists to assert it over.
4. **Storage for the plate selection.** `:policy`/`:keep`/`:drop` beside
the plate's `:time :holds`. Extend `leaf/leaves` and the key whitelists in the same
commit — a field without a leaf saves silently and comes back missing, which
is the one bug persistence must not be able to have. Round-trip test.
5. **Re-suggest preserves hand decisions.** Propose at one tolerance, pin a frame,
drop a frame, propose at another, assert both survive. Settle here whether the
hand's `:keep` is also fed to `propose` as `:protect`: under the layering as
written it is not, so a pinned frame does not re-anchor the walk even though it
is on screen, which contradicts the anchor rule above. It is the one live
caller for `propose`'s `:protect` frames.
6. **Fold the plate frames into the mark set**, which is the nesting rule and is
one line once both sides exist.
7. **The shared UI component**, then its two mountings.
## What is revertible, and where it is
Commit `02069e8` contains one part that **nothing below asks for**: extrema
detection. Specifically `segments`, `turns`, the `:extrema` branch of `propose`,
the multi-component refusal that exists only to guard it, and three tests —
`a-one-frame-closure-survives-only-because-it-is-protected`,
`extrema-are-turns-worth-more-than-the-tolerance` and
`extrema-are-refused-on-a-multi-component-signal`.
It has no caller because plate selections take no extrema by design — displacement
is the whole story for a head — and the performance side reads a stored cut
instead of finding extrema in a signal. It is correct, tested and speculative.
Revert it if the shape above holds. Keep it if either of these turns out to be
wanted: a group whose extreme is not a closure and therefore has no `[:vis]` cut
to read (a mouth at its widest, a head at the top of a nod), or a take whose mouth
never shuts far enough to cross `aperture-cut`, where a peak-relative threshold
marks nothing and an extremum would still find the most closed frame. Neither is
asked for today. The DP in *The upgrade path* gets extrema for free regardless, so
reverting costs nothing that cannot be had again more cheaply.
## Traps
- **Do not let Suggest write `:keep`.** The proposal and the hand are different
layers; collapsing them is the bug this whole shape exists to avoid, and it
will look like it works right up until somebody moves the tolerance slider.
- **Do not snap a grid slot forward.** Every hold and every pick is "at or
before". Showing a closure before the mouth shut is a lead, which is a different
control for a different reason.
- **Do not quantise a plate selection onto the output grid.** A kept frame is a
native frame and lands where it lands. [time.md](time.md) is explicit that an
event between output frames appears on the next one; forcing kept frames onto
the grid would re-create the problem the selection exists to solve. The snap is
the opposite operation and is on the other side of the fence: it moves the grid's
pick, never the kept frame.
- **Do not read the aperture pair off a subsampled ring.** Positions 5 and 15 of
`LIPS-INNER` survive only at `verts` divisible by four.
- **Do not thin the plate selection with the mouth's.** Two scopes, two owners,
and the nesting runs one way only: plate frames preserve performance frames,
never the reverse.
- **Do not give the performance side a tolerance.** The grid has already chosen
the sparseness. A second knob there would be a control with nothing to control.
- **A skipped frame, a hidden feature and an absent measurement remain three
different facts.** A selection says nothing about visibility and nothing about
whether a face was found. Note that the preserve marks are *derived from* a
visibility cut, which makes this easy to blur: the cut says the interior is
hidden, the mark says the frame is worth landing on, and one is not the other.
## Not in scope
- Automatic *grouping* of related parts beyond the existing `:pose-group`.
Related parts must share one selection — a mouth outline, its interior, the
teeth and the generated visibility reading different frames is the bug that
grouping prevents — and `:pose-group` is where the grouping already lives.
- A per-instance request for a different tolerance than the symbol's. The
resolver threads no such option today and should not grow one until something
needs it.
- Variable frame rate. [time.md](time.md) assumes constant fps and so does this;
a VFR source needs presentation timestamps before any of this means anything.

247
docs/lane-handoff.md Normal file
View file

@ -0,0 +1,247 @@
# Lane and symbol-clip handoff
Status (2026-10-01): the timeline is the one timing interface. A lane is a
generic non-overlapping row of symbol clips; it is not a special drawing type.
Dropping a library symbol makes a naturally playing clip, while creating a new
empty symbol makes a one-frame held clip at the playhead. With no destination
lane, either operation creates one. Existing legacy root symbol rows can be
dragged into a lane. Blocks move by mouse; edge drags claim time by trimming
neighbors; Shift-edge drags ripple every later clip; and the center of a shared
cut composes the two edge edits into a rolling edit. Linked audio follows picture
moves while its edges remain independently trimmable.
The commits beginning at `3d3c1bb` are the argument for the model and are worth
reading before touching what they did — they are the design record, more than
this file is.
3d3c1bb An occurrence is a node, with a clock of its own
9446829 Reuse, duplicate and make unique: deciding what is shared
26517af A position is an argument, not another command
94c0a21 A correction is a layer, and a layer's values are a channel
72b57e3 Regenerate the base, keep the hand work, and say when you cannot
76106d3 The shot is as long as somebody said it was
598c186 One word for one thing: it is a cel
## Read first, in this order
1. [The Lane Model](lane-model.md) — the design, and the status note under
*Proof obligations* says what is built. It supersedes `animation-model.md`,
`timing-model.md` and `architecture.md` wherever they overlap.
2. `frontend/src/arthur/domain/lane.cljs` — every command, and the reasoning in
its docstrings.
3. `frontend/test/arthur/domain/lane_test.cljs` — what the model is asserted to
do. It is the fastest way to see the shapes.
4. `frontend/src/arthur/domain/channel.cljs`, the correction-layer section.
## Vocabulary — one word for one thing
Renamed in `598c186`, after four words had accumulated for one object. Use these
and do not reintroduce the others.
| word | means |
| --- | --- |
| instance | the `:kind`. The general thing, anywhere in a document |
| clip | an instance in a lane. It can hold one source frame or play a symbol naturally |
| cel | specifically a one-frame source held over a clip's duration; the empty-symbol/drawing creation policy |
| lane | a group with `:layout :sequence` |
| drawing | content authored into a symbol; not a different timeline node type |
| placement | ONLY where a node sits: `nest/placement`, and the transform that puts a face on the stage. Never the node itself |
`occurrence` and `exposure` are not words for a cel. **`exposure` means something
else and still does**: `:time :expose` is how many frames each step of a subtree
lasts, which is what shooting on twos is — `node/expose`, `clock/exposed-frame`,
`subs/render ::exposure`. Keeping these apart is why the block is called a cel.
`:layout :sequence` stays as the field, and is the one place two words are kept
on purpose: the layout names the RULE — children follow one another and may not
overlap — and a group carrying it is called a lane. `node/lane?` is where they
meet.
## Decisions already made — do not re-litigate
These were each argued out and are load-bearing. Changing one is a design
decision, not a cleanup.
- **The shot length is authored.** `:frames` is the symbol's window; the
occupied extent of its lanes is a different fact derived from the cels. A
command grows the window only when the caller passes `:extent :grow-symbol`,
and never shrinks it. Blanking the end of a shot leaves empty frames at the
end, because deriving the window from the extent would make deleting the last
drawing silently shorten the film. `lane/finish`.
- **Placing ripples; overwrite is `blank` then non-rippling placement.**
`lane/overwrite-drawing` composes those pieces as one transaction. Insertion
retains its ripple rule; overwrite does not move any surviving cel.
- **A position inside a cel refuses and names `split`.** One command must not
quietly perform two. The UI offers the retry.
- **A correction has no time space of its own.** Its `:support` and its values'
keys are in the frames the base channel's keys are in — the node's. A
correction on a lane is in lane frames and reaches across the drawings under
it; one on a cel travels with that cel. Ownership already answered it.
- **A layer's values are a channel.** Constant, ramp and return motion are one
mechanism. Do not add a second way to say what a value is over time.
- **A conflict is not a `problem`.** A document whose topology outgrew a
correction loads, evaluates and saves; `clip/conflicts` lists the decisions
waiting for a person. `problems` means the document will not load.
- **Refuse rather than guess.** Every command returns `{:clip :selection}` or
`{:refused why}`, never a half-applied edit. Where the model needs a choice
nobody has made, refusing and saying why is the behaviour, not a placeholder.
- **A clip is not a row.** Rows, expansion and selection are editor state. The
document has never known about rows and must not learn — which is what let
the row model change three times in one sitting (blocks, then a
selected-clip portal, then sound lanes under the audio heading) without
touching a single document.
- **A lane is generic.** Drawing creation, library placement, and adopting an
existing root instance all produce the same child instance shape. The only
difference is playback policy: a new empty drawing holds source frame zero;
a dropped library symbol plays at speed one.
- **Lanes are explicit.** A symbol may contain ordinary overlapping children
without a lane. Selecting a lane row opts creation into its claim-time
behavior; selecting a cel enters that cel's source symbol instead.
- **Placement claims time.** Lanes never store overlaps. A new or extended clip
trims, removes, or splits whatever previously owned the claimed interval.
Real compositing overlap uses another lane, where ordering remains explicit.
## The timeline opens the whole document
Expanding a lane opens exactly one clip — the selected one — and that portal
opens the lanes and nodes of the symbol it places, recursively, mapped into
the open symbol's ruler. The portal follows the LINEAGE of the selection, so
working on something nested keeps the rows that revealed it open. A held clip
opens too, with its rows marked `:unmapped?`: shown across the hold, with no
keys and no draggable edges, because a frozen clock gives its frames no place
on this ruler. `docs/lane-nesting-notes.md` has the reasoning and what is
still missing.
Double-clicking a clip opens the symbol it places as a tab, the same as
double-clicking that symbol in the pool. Shift while dragging a clip body
turns the temporal move into a structural one — see the nesting notes for why
that is mostly refused today.
## Current timeline interaction
- Creation follows the primary active row and the playhead; the complete rule is
in `docs/creating-in.md`. A lane row creates a new cel and claims its interval,
while selecting a cel creates inside the symbol that cel places.
- Drawing with a lane row active creates a new one-frame drawing cel. Dropping a
library symbol there creates a natural-duration playing cel. Explicit timeline
drops use the row and frame under the pointer.
- Dragging a clip body moves it. A linked audio node follows a picture move;
moving or trimming the audio itself remains independent.
- Dragging a right edge changes its endpoint. Growth consumes adjacent spans
instead of overlapping them. Shift-drag inserts or removes lane time by moving
every later clip by the same delta.
- At a shared boundary, the left and right hit zones trim one side. The center
is a rolling edit: right-edge resize followed by left-edge resize at one frame.
- Split, trim-in, and trim-out are direct buttons and are disabled without an
editable selected span. Movement is a mouse gesture, not a toolbar command.
## Next steps, in order
The implemented correction slice and its remaining UI limits are recorded in
[Correction authoring](correction-authoring-plan.md).
1. **Slip source and retime.** Both have real design questions open and the doc
says to refuse rather than approximate: retime needs a defined warp and
interpolation behaviour, and is not moving keys whose numbers happen to fall
inside a selection.
2. **Deleting reused content.** Reference discovery exists (`node/sources`,
`clip/places`, `clip/contains-symbol?`); the policy does not.
3. **Displayed-range correction gestures.** The first correction panel asks for
explicit owner frames. Dragging a range in a retimed/nested view still needs
a proved mapping; do not make it snap through floors or loops.
4. **Collaboration.** `lane-model.md` is explicit that one leaf per channel does
NOT solve two people editing different keys of the same channel. No conflict
policy exists for that.
## Mechanisms to reuse — these keep paying out
- **`:span` is in the node's OWN frames** and `:time` says where they land in
the lane. Moving an edge of a cel is therefore one write to `:span`, with
`:time` and `:playback` untouched. This is why split costs nothing, why the
two halves of a split go on meaning what the one cel meant, why trimming the
front of a playing insert starts it later into its animation instead of
restarting it, and why extending a hold leaves lane keys alone. `lane/local`
and `lane/edged` are the whole geometry; trim, split and blank are all it.
- **`lane/finish`** is the one commit path: it validates, applies the shot-length
policy, and returns the refusal. New commands go through it.
- **`:required-frames` plus the retry event** is the pattern for "this needs a
decision you have not made": the domain reports what it would need, the UI
offers one button. `events/ui/lane-retry`.
- **`lane/lane-frame`** converts a symbol frame to a lane frame, or returns nil
through a stepped or looping lane where there is no single answer. Nil refuses;
it never snaps.
- **`channel/conflict-with`** is the rule for whether one offset fits a base,
used by `conflicts` and regeneration. Validation additionally follows prior
replacement layers, so it cannot approve a stack that throws when read.
- **Generated sampling applies to the base, not the hand correction.** Picture
rate and pose selection may choose an earlier generated frame; correction
support and values still read the node's current authored frame.
- **Correction commands live in `domain/correction.cljs`.** IDs come from the
event caller; the pure command materializes default transform channels,
appends one layer, and validates the complete document. The inspector authors
rotation and position offsets in explicit owner frames. One Apply is one undo
step. `channel/reconcile` is the shared ordered-stack compatibility rule used
by validation, conflict reporting, and regeneration.
- **Two test patterns worth copying.** `the-cursor-agrees-with-the-specification-in-any-frame-order`
holds the optimized cursor to `value-at` in forward, backward and random order
— add a case to it for any new channel shape. And `drawn` in `lane_test`
samples every frame before and after an edit, which is how split and trim are
proved to change nothing: state a claim as "the same picture" rather than as
numbers computed by hand.
## Known gaps and traps
- **Audio is a clip in a lane too, and a lane holds one kind.** A sound placed
from the pool lands in a lane and is moved and trimmed by the same commands
as picture. The capability the earlier note asked for is the homogeneity
rule rather than a field: `symbol/lane-problems` refuses a lane holding both
kinds, and `lane/place-symbol` and `lane/adopt` refuse BEFORE claiming time,
because placement claims time and would otherwise have deleted the sound to
make room for the picture and left a valid document behind. Audio nested
inside a placed symbol — a take's own sound — is still shown flattened by
`nest/audio-tracks`; what is in a lane of the open symbol is drawn as a lane
and not flattened twice.
- **`:z` is required on cels and means nothing there.** A lane never has two
cels on one frame, so draw order between them cannot matter. `node/problems`
requires `:z` on every node uniformly, which is its own kind of simplicity —
but the field is noise on a cel.
- **`channel/offset-onto` throws** on a shape mismatch that no regeneration has
recorded as a conflict. That is deliberate — a correction that silently does
not take is the failure the design exists to prevent, and `channel/problems`
catches the authored case — but it is a throw in the read path, so any new
producer of layers must not create a mismatched one.
- **`docs/timing-handoff.md` is a separate, unreconciled thread.** Performance-
pose selection and plate drawings/tracing, instance-specific picture-rate
requests, `pose/put-cut` addressing only `:main`. It predates the lane model
and nobody has squared the two.
- **Slip and retime are still absent.** The timeline action strip now applies
split/trim uniformly to a selected root or lane clip, but source-time slip and
retime still need their own proved semantics before they become controls.
- **`shadow-cljs release app` clobbers the dev bundle.** Both builds write
`../static/arthur/js`, which Django serves, and the optimized build does not
export the `arthur` global — so after a release the browser tests fail with
`ReferenceError: arthur is not defined`. Run `npx shadow-cljs compile app` to
restore it. A running `watch app` does not notice; it rebuilds on the next
source change.
## Running it
From `frontend/`:
npx shadow-cljs compile test && node out/node-tests.js # 469 tests, 9,592 assertions
npx shadow-cljs compile app # the bundle Django serves
npx shadow-cljs release app # then `compile app` again — see above
The browser tests need the Django dev server up (`mise exec -- python manage.py
runserver 8778` from the repo root) and a compiled dev bundle:
node --experimental-websocket test/browser/lane.mjs # generic symbol-lane flow
CHROME=/usr/bin/chromium node --experimental-websocket test/browser/take.mjs
`take.mjs` defaults to a macOS Chrome path, hence `CHROME=`. It writes a real
project to the local server by design; `lane.mjs` never writes to the server.
From the repo root: `mise exec -- python manage.py test clips` — 56 tests.
Documents are schema 3. A version 2 document is not read and nothing converts
one; there is no backward compatibility to preserve anywhere in this work.

View file

@ -0,0 +1,71 @@
# Implementation notes — "A lane is a view"
Running log for `docs/lane-is-a-view-plan.md`. `[ ]` not started, `[~]` in
progress, `[x]` done with `npm test` green.
Baseline at `fb38990`: 475 tests, 9621 assertions, 0 failures.
## Order of work
The plan's seven steps, re-grouped — see *Deviation from the plan's order* below.
- [x] A. Domain: `symbol/children`, `symbol/lane?`, `symbol/overlaps`,
`lane.cljs` → `span.cljs`, the overlap check in `span/finish`
(plan steps 3, 4, and the domain half of 5)
- [x] B. Events: re-base callers and remove lane-node-specific commands; keep
explicit `::new-lane` and cross-lane adoption (plan step 5)
- [x] C. UI: row per symbol, explicit lane creation, and drag handling
(plan steps 1, 2, and the UI half of 5)
- [x] D. Audio: delete `holds-other?`, the mixed-lane refusal, the `in-lane`
filter in `sound-rows` (plan step 6)
- [x] E. Tests: `domain/sequence_test`, `events/lane_test`, `browser/lane.mjs`
- [x] F. Shift-to-reparent still works, untouched (plan step 7)
Not in this pass — see *Left for a second pass*: tearing out the held cel, the
instance-playback control, drawing a loop's repeats, the audio period guard.
## Deviation from the plan's order
The plan's steps 1 and 2 are display work that keys off "a symbol's children",
and step 3 is what MAKES the cels a symbol's children. Until then a cel's
`:parent` is the lane node, so there is nothing for the display to read: step 2
cannot draw "a symbol's children as blocks" while the children belong to a
group. So the data model moves first (A) and the display follows (C). The
content of each step is unchanged; only the order is.
The one thing this gives up is the plan's promise that every step leaves the
editor usable — between A and C the timeline draws the new shape with the old
code. `npm test` is green at each step either way.
## Decisions taken
1. **Lane mode is `:display :lane` on the SYMBOL** — plan's recommendation 2,
and open question 1 answered "the symbol, not the instance". A symbol placed
twice is drawn as a lane in both places. Added to `symbol/symbol-keys` and to
`leaf/leaves`' `select-keys` so it saves like `:frames`.
2. **A symbol's children are its parent-less nodes** that have a placed span.
The plan's step 6 settles it: "an audio node is already a parent-less child
of a symbol, which is exactly the new shape". Span-less nodes — a shape on
screen for the whole shot — are not in the sequence and are skipped, which is
also what stops the commands destructuring a nil span.
3. **A symbol holds at most one sequence.** It follows from 1 and 2: the
container is the symbol. Two lanes of picture is now two symbols placed in a
third, which is what compositing already was.
4. **The open symbol gets a row of its own in lane mode**, and only then. The
blocks have to sit on a row and the open symbol had none; expanding it turns
its children into ordinary rows. Not a row always, which would shift every
row in the pane for no gain.
5. **Open question 2** — a lane row's edge drag trims the PLACING INSTANCE's
span, via `span/resize-out`, like the handle on every other row. Rippling
the children is what the cel blocks' own edges already do, and giving one
handle two meanings is what the plan refuses elsewhere.
6. **Open question 3** — the lane work first, the held cel after. The plan says
they are independent, and the held cel is joined to a loop control that does
not exist yet; doing it second costs one more pass over `lane_test`'s
fixtures and risks nothing.
7. **Lane creation stays explicit.** A blank document and `new symbol` create
ordinary symbols. The separate `new → lane` command creates and places a
symbol with `:display :lane` in the effective creation target derived from
selection and playhead; the new lane then becomes the primary selection.
## Notes

301
docs/lane-is-a-view-plan.md Normal file
View file

@ -0,0 +1,301 @@
# A lane is a view
Plan, 2026-10-01, written at `2dc5735`. It undoes the lane model as a thing in
the document and keeps what it was for. Build on what is there and tear out
half of it.
## The decision
> A lane is a view over a symbol with sequential, non-overlapping children.
Nothing in the document is a lane. There is no lane type, no lane group, no
`:layout :sequence`, no lane commands and no lane validation. The word
survives in exactly two places: the UI, where a symbol can be DRAWN as a lane,
and the drag handling that re-spans a symbol's children while it is being
drawn that way.
The display model goes back to a row per symbol. A symbol in lane mode draws
its children as blocks on its own single row; expanded, they are rows like
anything else. Everything else is an ordinary row that expands into what it
places.
## What a lane was, and what each part becomes
| was | becomes |
| --- | --- |
| a group node with `:layout :sequence` | nothing — the symbol is the container |
| `node/lane?` | a view question: is this symbol drawn in lane mode |
| `symbol/lane-clips nodes lane-id` | the children of a symbol, sorted by `node/placed-span` |
| `symbol/lane-problems` | `symbol/overlaps`, a diagnostic the write path calls |
| `lane/lane-frame` | `clip/source-time` — one clock instead of two |
| `domain/lane.cljs` | re-based onto `domain/span.cljs`: re-spanning a symbol's children |
| `clip/lane-node`, the born-with lane | gone; a symbol is born empty again |
| `::ui/new-lane`, `::ui/adopt-in-lane`, lane renaming | gone, gone, and ordinary node renaming |
`lane.cljs`'s fourteen commands are not deleted — they are what "endpoint drag
overlap handling" means, and they already do the right arithmetic. What
changes is their subject: every one of them currently takes a host symbol AND
a lane id and asks `lane-clips nodes lane-id`; each takes a symbol and asks
for its children. `extend-hold`, `resize-out`, `resize-in`, `roll`, `blank`,
`place-symbol`, `adopt`, `append-drawing`, `reuse-drawing`,
`duplicate-drawing`, `overwrite-drawing`, `make-unique`. `span/finish` is
already the one commit path and stays exactly as it is.
Put them in `span.cljs`, which already owns "one write to one node's span" and
`finish`. The sequence operations are the same subject — re-spanning children
— and keeping them apart was a consequence of lanes existing.
## Children in lane mode never overlap
This is an invariant, not a condition to check for and report. Placement
claims time: anything placed, moved or grown over occupied time TRIMS the
extents it lands on — trimming the incumbent, removing one wholly covered, or
splitting one it lands inside — so the result has no overlap because the
operation that could have made one did not. That is `blank` followed by a
non-rippling placement, which is what `overwrite-drawing` already composes.
Enforced at the boundary, which already exists: `span/finish` is the single
commit path for every one of these commands, it validates before it returns,
and it refuses rather than half-applying. So `finish` gains the overlap check
for a symbol in lane mode, and no command can commit one. An overlap that
appears anyway is a bug in a command, not a state to design around.
Keep the check as a named diagnostic — `symbol/overlaps`, taking a symbol and
returning the pairs — used three ways:
1. `span/finish` refuses when it would commit one.
2. The test suite asserts no command can produce one: a property over the
commands in the style of `drawn` in `lane_test`, which samples rather than
computing expected numbers by hand.
3. A document that somehow arrives holding one still LOADS — a display hint
must never be able to stop a document loading — and the timeline draws it
visibly wrong with the status line saying so. Not `clip/problems`, which
means the document will not load, and not `clip/conflicts`, which means a
person has a decision to make. This is neither: it is a bug report.
Toggling lane mode ON for a symbol whose children already overlap is the one
place a person can ask for the impossible. Refuse it and say why, with the
`:required-frames` retry pattern offering to trim them into a sequence — the
domain reports what it would need, the UI offers one button.
Outside lane mode nothing is enforced, because overlapping children are what
compositing IS. An endpoint drag there is an ordinary span edit that may
overlap; the claim-time rule follows the mode.
## The two drag intentions
Unchanged from `docs/lane-nesting-notes.md`, and both kept:
- **Plain drag** of a clip body is temporal: it moves in time, within its
symbol or into another symbol drawn as a lane, and it REPLACES — trimming,
removing and splitting extents as needed so nothing overlaps.
- **Shift-drag** is structural: the dragged node goes INSIDE the symbol the
clip under the pointer places, through `nest/move-node`, which preserves the
world transform and the root timing. This must keep working for symbols
contained in a lane, which is the case it exists for.
Overlap cannot distinguish them — dropping on occupied time already means
claiming it — so the modifier says which, and the label by the pointer says it
back. `nest/move-refusal` already answers before the drop.
## Where lane mode lives
A symbol is drawn as a lane because somebody said so, not because of what its
children happen to look like at this moment. Deriving it from "the children do
not currently overlap" means a symbol stops being a lane the moment anything
overlaps, and the rules that maintain non-overlap switch off exactly when they
are needed.
Two options:
1. **Editor state**, `[:ui :lane-mode #{sid}]`. Purest reading of "a lane is a
view". But the drag rules follow the mode, so an unsaved, per-person toggle
would decide whether dropping a symbol trims its neighbour or composites
over it — the same gesture doing two different things to the document
depending on something the document does not record.
2. **A display hint on the symbol**, e.g. `:display :lane`, saved like any
other field (`clip-keys`, `leaf/leaves`, `leaf/clip` in the same commit).
Still not a type: nothing in evaluation reads it, `symbol/problems` does
not check it, and a symbol with it set behaves identically on the stage.
Recommended: 2. It is one field, it keeps editing rules reproducible between
people, and it does not make the symbol a different kind of thing. The thing
to hold the line on is that nothing outside the timeline, and the commit
path's overlap check, is allowed to read it.
## Two things are called loop
Before any of this, name them apart, in the way the vocabulary table in
`lane-handoff.md` names a cel apart from an exposure.
- **Loop playback** is the transport repeating the open symbol while it plays.
It is `[:playback :loop?]` in app-db, the ⟳ button in the strip, and it is
EDITOR STATE. The document does not know about it.
- **A looping instance** is a node repeating the symbol it places: a four-frame
tire turning for the hundred and twenty frames the instance is on screen.
It is `:playback {:end :loop}` on the node, and it is in the DOCUMENT.
The car tire is the second one, and here is its actual status: it already
works in the evaluator and cannot be asked for. `node/placed-frame` does the
modulo, `node/problems` already admits `:end` of `:stop`, `:hold` or `:loop`,
`nest/audio-tracks` already expands a loop into its periods — and nothing in
the UI sets it. It is implemented and unreachable.
So three things are missing, and they are the work:
1. **A control.** Where an instance's playback is edited: `:in`, `:speed`, and
what happens at the end. One place, three fields, rather than a loop
checkbox somewhere else.
2. **Drawing the repeats.** `ui/timeline`'s docstring already admits that only
the first pass of a looping instance is drawn, so a tire turning thirty
times shows one turn's keys and then nothing. A looping block should show
its passes — at minimum the period boundaries, so the row says how many
times round it goes.
3. **The audio period guard** below, which a loop needs whether or not holds
become loops.
## Tear out the held cel
A held cel is a 1-frame symbol shown for many frames. A 1-frame symbol with
`:end :loop`, lengthened, is the same picture by a different route — and the
second route is a case the model already has, so keeping the first one is
keeping a special case for free.
**What is already true**, so that nothing has to move: `:span` is on the node,
in its own frames; `:time` (`:at`, `:rate`) is on the node; looping is on the
node, as `:playback :end`. The SYMBOL owns only `:frames`, the authored
window. So looping and span are instance properties already, and this change
is about deleting a mode, not relocating a field.
It does mean the two changes are joined at one point: making every drawing a
looping instance is not safe until a looping instance can be seen and edited,
or every drawing in the document acquires a property with no control on it.
**What has to be decided and collapsed:**
- There are two spellings of looping — `:time :loop?` and `:playback :end
:loop` — and both are read, in `node/placed-frame` and in
`nest/audio-tracks`. Keep one. `:playback {:in :speed :end}` already says
what happens at the ends, so `:end :loop` is the one to keep and
`:time :loop?` is the one to delete.
- `:playback :speed 0` stops being produced. Make it illegal in
`node/problems` rather than legal-but-unused, so a frozen clock has exactly
one spelling: a 1-frame loop.
- `lane/extend-hold` exists only because holds were special — it refuses
anything whose speed is not 0 and then edits a span. Once a hold is a loop,
lengthening one IS `resize-out`, and the command collapses into it.
- `cel` survives as the word for the creation policy — a new empty symbol is
one frame — but stops naming a playback mode.
**What it buys, and this is the point:** one rule for nesting, which settles
the refusal that blocks shift-to-reparent today. `nest/inside` currently has
no `:time` for a hold, for `:end :hold`, or for a loop, and so refuses all
three. The general rule that covers all of them: **resolve the move with the
destination's map at the CURRENT frame — the affine piece the current frame
falls in.**
- A loop of length L is affine within the period the current frame is in.
Timing is preserved inside that period and repeats after it, which is what
looping means.
- A 1-frame loop — the ex-hold — has a period of length 1, so the map within
it is trivially invertible and lands the moved node on frame 0, aligned to
the current frame. That is exactly the answer `docs/lane-nesting-notes.md`
argues for from first principles, arrived at here as an instance of the
general rule instead of a special case.
- `:end :hold` is affine in the played part and frozen in the tail, which the
same sentence covers.
**The trap, which must be handled in the same change.** `nest/audio-tracks`
expands a loop into one walk PER PERIOD:
periods (range (floor (/ (to-local source lo) length))
(ceil (/ (to-local source hi) length)))
Today a held cel is skipped entirely — `(pos? speed)` is the guard, and the
comment says a visual freeze does not emit a sustained audio sample. Turn
every drawing into a 1-frame loop and that guard stops firing: a drawing held
for 120 frames becomes 120 recursive walks, and any sound inside it is emitted
120 times. That is both a wrong mix and a performance cliff on the most common
node in the document. Required with this change: a cheap `symbol/audible?`
precheck so a source with no audio anywhere inside it is never period-expanded,
and a cap or a different formulation for the ones that are.
Also worth knowing before the change: `ui/timeline`'s own docstring already
says a looping instance draws only its first pass. With every drawing a loop,
that sentence now describes every drawing — harmless, since one pass of a
1-frame symbol is the whole of it, but the docstring should stop sounding like
a limitation.
**Dropping into a 1-frame symbol.** The window is authored and crops what it
holds, so a 10-frame symbol dropped into a 1-frame drawing shows its frame 0
and nothing else. That is consistent — `:frames` is the shot length and
`:extent :grow-symbol` is the opt-in — but it is probably not what somebody
dragging means. Offer the growth through the `:required-frames` retry the
model already uses: the command reports what it would need, the UI offers one
button.
## Order of work
Each step compiles, passes `npm test`, and leaves the editor usable.
1. **Row per symbol.** In `ui/timeline.cljs`, delete `portal`, the `::portal`
hint row, the `under?` lineage predicate and the `chosen` argument; emit a
row for a clip whose parent is a lane instead of skipping it. Keep the
`:cels` blocks for the collapsed row, and keep `inside-rows` — including
its `:unmapped?` branch, which is what makes a held drawing's contents
reachable at all.
2. **Lane mode as a hint.** Add the field and the toggle, draw a symbol's
children as blocks when it is set and as rows when it is not, and move the
lane-row drag handling onto it. Both display paths now exist and nothing in
the domain has changed.
3. **Re-base the commands.** Move `lane.cljs` into `span.cljs`, replacing
`(lane-clips nodes lane-id)` with the symbol's children and dropping the
`lane-id` argument. `frontend/test/arthur/domain/lane_test.cljs` is the
proof: its fixtures should change and its assertions should not, and any
assertion that has to change is a behaviour change worth noticing.
4. **Move the invariant.** `symbol/lane-problems` becomes `symbol/overlaps`,
called by `span/finish` for a symbol in lane mode, plus the property test
that no command can produce an overlap.
5. **Delete the rest.** `node/lane?`, `symbol/lane-clips`, `clip/lane-node`,
`::ui/new-lane`, `::ui/adopt-in-lane`, the lane branch of
`::ui/new-symbol`, lane renaming, `aimed-lane`, and the
`:lane?`/`sound-lane?` row flags. Rename what is left so the word does not
appear outside the timeline.
6. **Audio falls out.** An audio node is already a parent-less child of a
symbol, which is exactly the new shape — so the audio-in-lane rules added
in `2dc5735` (`holds-other?`, the mixed-lane refusal, the `in-lane` filter
in `sound-rows`) delete rather than migrate. A symbol drawn as a lane whose
children are sounds is an audio lane, and that is the whole of it.
7. **Shift-to-reparent stays** as it is: `nest/move-node` and
`nest/move-refusal` never knew about lanes.
## What must not be lost
All of this was broken at some point today and is now proved; each has a test
to keep.
- A held clip's contents are reachable from the root timeline, with no keys
and no draggable edges — `source-time` is nil for a hold, and the walk used
to stop there.
- Double-clicking a clip opens its symbol as a tab, and the editor survives
it: `symbol/lineage` must not report a cycle for an id the symbol does not
hold, and opening a symbol must drop a selection pointing into the one being
left.
- Selection waits for pointer-up, so a press does not re-draw the timeline out
from under the gesture it is starting.
- A drop never silently deletes what it lands on.
- A sound is drawn once, not twice.
## Open questions
1. **Is lane mode a property of the symbol or of the instance placing it?** A
symbol placed twice would be drawn the same way in both places under the
first reading. That is probably right, and worth saying out loud.
2. **Does a lane row's edge drag trim the placing instance's span, or ripple
the children?** Same handle, two commands; the row is now an instance, so
it has a span of its own for the first time.
3. **Does tearing out the held cel come before or after the lane work?** It
is independent of it — `nest` never knew about lanes — and it is what makes
shift-to-reparent work on the thing people would actually drag onto. Doing
it first means the lane work lands on a model with one playback mode fewer;
doing it after means two changes to `lane_test`'s fixtures instead of one.

591
docs/lane-model.md Normal file
View file

@ -0,0 +1,591 @@
# The Lane Model
Revised 2026-10-01. Clip ownership, source playback, one-row generic lanes,
direct clip movement and edge editing, correction evaluation, and correction
authoring for rotation and position are implemented. The former cel-sheet
projection was removed: the timeline is the single timing interface. Sections
below that describe a cel sheet are retained as design history and are superseded
by this revision. Retiming commands are not.
See the status note under
[Proof obligations](#proof-obligations-and-implementation-order).
[Lane and cel handoff](lane-handoff.md) records what is built, the decisions
that are settled, and what to do next.
This revises the Claude artifact [The Lane Model](https://claude.ai/code/artifact/cd42981d-ed08-493f-94df-b7dd6657f0e6).
Its prose and diagram source were recovered from session
`1c603f71-84eb-498e-aeaf-4c0346f1f513`; the live artifact was not accessible for
reading or editing here. This repository document is the revised design. The
original artifact has not been updated, and edits made there outside the recorded
session may not be represented here.
For the subjects covered here, this document supersedes the original artifact
and conflicting proposals in `animation-model.md`, `timing-model.md`, and
`architecture.md`. Those documents retain useful detail about the existing system.
## Goal and compatibility policy
Arthur is one animation document with several ways to see and edit it: drawing
on the stage, arranging clips, timing cels, editing curves, and generating
motion from footage. Each view exposes relevant facts and invokes shared editing
operations. Switching views must preserve the meaning of the work.
The user explicitly requires no backward compatibility. Replace obsolete shapes,
APIs, and tests when a better model requires it. Do not retain compatibility
branches, adapters, or migrations solely to preserve the current document format.
A format marker can reject unsupported files clearly; it does not promise to
convert them. This policy does not authorize deleting existing user assets.
Simplicity means predictable composition, clear ownership, and few independent
rules. Minimizing field count is secondary to representing independent choices.
## What stays
- A symbol is the one container for authored scene nodes. A drawing can be a
one-frame symbol; an animation uses the same container over more frames.
- Nodes have stable identities and flat parent references. Shared content is
referenced rather than copied implicitly.
- Animatable properties are addressed by channel paths. Generated and authored
values participate in the same evaluation machinery.
- Authored data, generated blocks, and source media remain separate. Documents
reference immutable blocks; caches and resolver indexes remain derived.
- A pure reference evaluator specifies the result. Playback, seeking, preview,
export, and optimized cursors must agree with it.
- Existence, visibility, and missing measured data remain distinct facts.
## Content, cels, lanes, and rows
These have different identities and responsibilities:
| Concept | Owns | Example |
| --- | --- | --- |
| Content | Reusable nodes and their animation | Drawing `a2`, an animated head, or a sound asset |
| Cel | One use of content, its interval, source playback, and local treatment | `a2` exposed on frames 12–16 |
| Lane | A sequence of cels and shared properties | The girl's drawings and the girl's overall transform |
| View row or column | Presentation and editor state | Timeline row, cel-sheet column, or property curve |
Use the existing instance/node identity mechanism for cels. A cel
should not acquire a second identity system just because it is shown as a cel.
Lanes group cels; they do not introduce another node-holding content type.
The concrete candidate below uses existing group and instance nodes; its ownership
boundaries are part of the design. It is now the implemented shape, and the field
spellings below are the ones the runtime reads.
Each cel has a stable ID. Moving it, changing its hold, swapping its source,
or trimming it preserves that ID. Repeating it creates a new cel that may
reference the same content. A split retains the original ID on the left and gives
the right piece a new ID; commands return the resulting selection explicitly.
Cels are the canonical authored arrangement. A source-at-time channel or
interval index may be compiled from them for evaluation, but is not a second
editable copy of the schedule. This replaces the earlier proposal that every cel
must be represented solely as a source key. Ordinary property animation still
uses lightweight keys; it does not need cel objects.
A lane has non-overlapping half-open cel intervals `[start,end)`
in its own time space. Uncovered intervals are gaps. Empty lanes are valid.
Compositing and simultaneous sounds are represented by multiple lanes or ordinary
scene composition; an accidental overlap never silently selects a winner.
Transitions, if added, need explicit overlap and mixing semantics.
Properties can belong to content, one cel, or the lane. For example:
- Rotate the reusable drawing: all its uses change.
- Rotate one cel: only that cel changes.
- Animate the lane's rotation: whichever drawing is showing follows it.
A cel can have its own transform, gain, corrections, and source timing
while remaining a block in the same timeline row. Independent treatment never
requires a new row or an otherwise unnecessary wrapper symbol.
### Concrete candidate: a lane and ordinary instances
A lane is a group node with `:layout :sequence`. Its cels are ordinary
instance nodes whose `:parent` points to the group. All remain in their symbol's
flat node map. The sequence constraint is document semantics; which rows the UI
expands remains editor state. Ordinary groups retain unconstrained composition.
This example is a document the runtime accepts, built and evaluated by
`frontend/test/arthur/domain/lane_test.cljs`. Times here are zero-based. Channels
use the existing representation; cel source references and playback have
replaced the `[:source]` channel, which no longer exists.
```clojure
;; Within :main's :nodes; referenced drawings/animations live in :symbols.
{:girl
{:id :girl :kind :group :layout :sequence :z "b"
:channels {[:xform :rot]
{:animated? true :interp :linear :keys {0 0, 6 30, 12 0}}}}
:cel-a
{:id :cel-a :kind :instance :parent :girl :z "a"
:time {:at 0 :rate 1} :span [0 4]
:source {:symbol :drawing-a}
:playback {:in 0 :speed 0 :end :stop}}
:cel-b
{:id :cel-b :kind :instance :parent :girl :z "b"
:time {:at 4 :rate 1} :span [0 4]
:source {:symbol :drawing-b}
:playback {:in 0 :speed 0 :end :stop}
:channels {[:xform :pos]
{:animated? true :interp :hold :keys {0 [0 0], 1 [2 0]}}}}
:animated-insert
{:id :animated-insert :kind :instance :parent :girl :z "c"
:time {:at 8 :rate 1} :span [0 4]
:source {:symbol :wave}
:playback {:in 3 :speed 1 :end :stop}}}
```
The lane's rotation reads lane time. Each cel's channels read cel
time. Its content reads source time. The stills sample frame 0, while the insert
samples source frames 3, 4, 5, and 6. The group transform composes with the
cel transform and then the content's own transform.
`:span` remains in the node's own coordinates, consistent with ordinary nodes.
The interval in lane time is derived through `:time`; do not also store parent
start/end values. Sequence children require finite intervals and positive
placement rates. Ordering and overlap checks use the mapped intervals, not `:z`.
The current sequence group contains visual symbol clips. Audio remains an
independent root node (and can be linked to picture); if audio lanes are added,
their capability must be explicit rather than inferred per frame.
A source reference is fixed within a cel. The lane changes content when
another cel becomes active. This is a deliberate revision of the original
diagnosis that making `:of` a channel was necessary to avoid vertical growth:
multiple instances can occupy one row when the view presents their containing
sequence. A lane-level source schedule is therefore derived, not authored twice.
## Source selection and source playback are independent
The current implementation makes framed sources play and keyed sources hold.
Retire that rule. Channel storage shape must not determine playback behavior.
Adding or removing a key must not turn a still into an animation or vice versa.
A cel names content and describes how its source time is sampled. In the
basic case, after mapping lane time into cel time:
```text
source_time = in_point + speed × cel_time
```
A newly created cel starts at local time zero. Moving it preserves this
origin relative to its content. Trimming can narrow its local support without
resetting that origin; split pieces likewise preserve the source and property
values at the cut. Trimming, slipping, and retiming are distinct operations with
explicitly different effects on the interval and the source map.
| Intent | Source playback |
| --- | --- |
| Hold a drawing | Constant source frame, equivalently speed 0 |
| Play an animated symbol | Advancing source time, normally speed 1 |
| Cut between animations | Several cels, each with its own in-point and speed |
| Mix stills and animation in a lane | Constant and advancing maps in the same sequence |
The source reference itself is discrete and never numerically interpolated.
Interpolation belongs to properties that support it; a property registry should
declare value types, defaults, and permitted interpolation and correction modes.
Generic key toggles must consult those capabilities rather than assume every
non-boolean value can be tweened.
Define source bounds and end behavior explicitly: stop contributing outside the
source, hold an endpoint, or loop an explicit range. A still uses a valid constant
frame. A loop uses a nonempty half-open range and a defined modulo rule. Playback
never guesses these policies from whether a channel happens to have keys.
Audio shares cel arrangement, trimming, gain ownership, and clock mapping.
It does not inherit visual frame-hold semantics: holding one audio sample is not
an audio freeze effect. Validate supported playback policies by media capability.
Actual audio scheduling must follow active cels, including gaps and cuts,
rather than playing every sound reachable through a structural reference.
## Time spaces and sampling
Name the relevant space whenever an API accepts a time or range: project,
symbol/lane, cel, or source. Store authored frame coordinates exactly;
avoid cumulative rounding when moving through nested mappings. Quantize at a
declared sampling boundary, not at every traversal step. Audio also needs its
continuous clock/sample space rather than visual frame quantization.
A hold is an evaluable time map with no unique inverse. A loop can map many
displayed cels to one source time. APIs must distinguish forward sampling
from inverse editing, and expose enough context to resolve a cel or
explicitly refuse an ambiguous operation. Do not report a missing time map merely
because inversion is unavailable.
Separate invertible placement timing from source sampling. A zero source speed
can mean hold without making the cel's own edit clock non-invertible.
Reparenting through changing transforms or non-invertible timing must either
preserve the full result by an explicit bake or return a reason it cannot; a
matrix captured at one frame does not prove preservation across the animation.
The source's frame step, generated-pose sampling, and the lane's transform clock
are independent scopes. Drawing on twos must not accidentally step a smooth lane
transform. An explicit whole-subtree stepping operation can exist separately.
Share the quantization primitive where possible, but retain its units, phase,
rounding policy, and order relative to retiming and lead. The original suggestion
that cel and picture-rate sampling are simply one floor is insufficient:
noninteger grids and source-frame quantization require specified behavior.
Identity timing can be implicit; remove `:time :mode` if it only duplicates that.
A cel interval is authored. Lane content extent is derived from its
cels, including the explicit end of the last one. A separately authored
container trim/window is legitimate when it intentionally gates children. Do not
conflate that window with occupied extent or infer a final hold from the next key
when no next key exists. A range of frame numbers alone cannot encode visibility
or a missing measurement.
## Shared editing operations
Every view issues the same domain commands. A command accepts an explicit target
and edit policy, computes a valid change, and returns the change, resulting
selection, and any refusal reason. A button and a drag must not implement two
versions of cel extension.
An edit target identifies the symbol, cel path, selected entities or
properties, and the time range with its space. Navigation also distinguishes
editing shared content directly from editing it through a particular cel.
Crossing a source cut must not silently redirect an active drawing edit to a
different symbol: retain the explicit content target until navigation changes it.
Core commands include new drawing, reuse drawing, duplicate drawing, make unique,
blank range, split, trim, move, extend cel, slip source, retime, and apply a
bounded property edit. Ripple/overwrite policy and the set of affected lanes are
explicit command arguments. Preview consequences before committing a gesture.
New drawing creates fresh empty content and a cel. Blank range removes
content coverage without inventing a hidden drawing. These are different actions.
Reuse creates another cel pointing at existing content. Duplicate creates
a new content identity. Make unique rebinds the selected cel only.
Copy semantics must specify nested sharing. A normal content copy duplicates its
owned nodes and channels while preserving references to other reusable symbols.
For a fully independent drawing assembled from nested symbols, provide an
explicit deep-copy operation with ID remapping. Never promise decoupling while
leaving the relevant edited object shared. Immutable media blocks may remain shared.
Commands are atomic undo transactions, even when they touch several leaves.
Pointer movement and keyboard invocation use explicit begin/preview/commit or
cancel boundaries; a timing heuristic alone must not decide user intent.
Collaboration applies a transaction consistently, validates affected references,
and detects conflicts at the owned data being changed. One leaf per channel does
not solve simultaneous edits to different keys of that same channel; define a
conflict policy rather than claiming that granularity solves all collaboration.
### Default timing behavior: cel edits preserve lane keys
Working default from the follow-up discussion: extending a drawing's hold changes
cel timing, leaving lane animation at its authored times. The user raised
keeping keyframes in place as a possibility; this is the proposed predictable
default, not a claim that they selected every timing policy below.
Ownership supplies the remaining rule: properties attached to a cel
travel with it. Extending its end does not stretch those properties; moving it
changes where their existing local times land. No per-key attachment flag is
needed to recover ownership that the document already expresses.
For the concrete example, extend `:cel-a` by two lane frames with ripple:
| Fact | Before | After |
| --- | --- | --- |
| Cel A's lane interval | `[0,4)` | `[0,6)` |
| Cel B's lane interval | `[4,8)` | `[6,10)` |
| Animated insert's lane interval | `[8,12)` | `[10,14)` |
| Girl's rotation peak | Lane frame 6 | Lane frame 6 |
| B's position change | B frame 1, lane frame 5 | B frame 1, lane frame 7 |
| Insert's first source frame | Source frame 3 | Source frame 3 |
The rotation peak now coincides with a different point in the drawing sequence.
That is the intended consequence of changing cels underneath timed motion.
The position correction stays attached to drawing B's cel. Neither the
background's keys nor audio on another lane moves.
The command contract for this edit names the symbol and cel, a delta in
lane frames, `:ripple` behavior, and an explicit scope of cel timing. It
extends A's local support by the delta converted through A's placement rate,
and shifts subsequent cel placements by that delta in lane time. It does
not modify any channel's key map, source in-point, or playback speed. Reject a
nonpositive resulting duration. Validate and commit the entire change together.
The symbol's authored end is another explicit boundary: preview an overflow and
offer to extend the symbol or cancel. A command can request that extension as
part of its transaction; it must not silently truncate later cels or grow
other uses of a shared symbol. In the example, a 12-frame symbol needs an explicit
extension to 14 frames or the edit must be refused without partial changes.
Retime performance is a separate operation over explicitly selected cels
and channels. It applies the same time transformation to their relevant clocks,
keys, and correction supports. Stretching an interval requires a defined warp
and interpolation behavior; it is not merely moving keys whose frame numbers
happen to lie inside the selection. Until supported, refuse this operation
rather than approximating it with a cel ripple.
The initial UI should default stage transforms to the lane when drawing in a cel
workflow, so movement usually remains independent of cel timing. The
inspector names the target: lane motion, this cel, or shared drawing.
Changing that scope is explicit. It changes what the edit means, not just which
panel happens to be open.
## Three-frame rotation and correction layers
A range says where an edit applies; it does not specify the motion. Offer distinct
commands for a constant adjustment, a ramp, and a return-to-start motion. For UI
frames 10–12, the internal range contains exactly three frame samples after
conversion from the displayed numbering convention.
- Constant adjustment: the same offset throughout those three samples.
- Ramp: interpolate from the specified start value to the target over the range.
- Return motion: interpolate from the starting value to a peak and back.
For a return motion sampled on three frames, the values can be `0, angle, 0`.
Outside the selected range, the underlying animation must evaluate exactly as it
did before. A range-scoped correction layer expresses this directly; blindly
inserting boundary keys can alter neighboring segments or destroy existing motion.
Implement corrections as an ordered stack over the base channel. Each correction
has stable identity, explicit support interval, blend operation, and values in a
named time space. Outside its support it is inactive. `replace` can supply a value
over an absent base; `offset` cannot offset a nonexistent value. Blend capability
depends on property type, and geometry corrections require compatible topology.
Regeneration replaces the generated base and preserves corrections. If changed
topology or removed targets make a correction incompatible, report a resolvable
conflict instead of silently dropping or misapplying it. Provenance explains
where the base came from; explicit sampling policy determines its playback.
This is core to the workflow: generate motion, correct it by hand, adjust the
generator, and keep the corrections. It should be proven before adding many views.
## Other unifications worth keeping
Pose choices, tracing-frame choices, and ordinary held values should share the
channel evaluator and cursor infrastructure. Preserve their different ownership,
fallback behavior, and sampling scope. A pose choice must address the relevant
content/feature explicitly; switching to another symbol must not accidentally
reuse a track just because both symbols contain a node with the same local name.
Keep the two animation idioms distinct: keyed geometry modifies one mark over
time; drawing substitution selects content that may have different structure.
Linear geometry interpolation requires compatible vertex correspondence, not
merely two drawings that happen to look related.
Derived library grouping may collect drawings used by a single lane. This is a
convenience, not ownership or deletion authority. Reference discovery for cycle
validation, copying, and deletion examines all structural references, including
currently inactive cels. Authored folders, favorites, and labels remain
legitimate user data even when the UI could have suggested defaults.
## UX: location, selection, and controls
The breadcrumb sits above the timeline and states the editing location, shared
content identity, and cel context when applicable. Show local time and
its project context where a useful mapping exists. Holds and loops need an honest
description instead of a fictitious unique global frame.
Creation follows the primary active row, resolved at the playhead as specified in
[`creating-in.md`](creating-in.md). The wider selection set still names what copy,
delete and transform affect; it is not a second list of creation destinations. A
shared drawing indicates its reuse and offers Make this cel unique. Names help identify content;
linked-use indicators must rely on IDs, because different drawings can share names.
| Surface | Primary scope and controls |
| --- | --- |
| Topbar | Project name, save/open/export, project rate and stage size |
| Location bar | Breadcrumb, add lane/content, shared-content context |
| Cel action strip | New drawing, duplicate drawing, hold longer/shorter, blank range |
| Lane header | Lane selection, lock, mute/solo where applicable, onion settings, expansion |
| Stage tools | Drawing and transform modes, active target and scope |
| Inspector | Selected content/cel/lane properties and valid key controls |
Cel actions have visible contextual buttons, shortcuts, a context menu, and
command-palette entries. These are different entrances to the same commands.
Shortcut names from the original sketch (`N`, `D`, `H`, `B`, `K`) are provisional;
their meanings must match the visible labels and avoid tool conflicts.
The inspector normally edits values and the timeline normally edits timing, but
this is an organizational default. Numeric duration and in-point controls are
useful inspector edits to the same domain facts. Do not ban a convenient control
just to preserve a visual division.
Default nesting navigation enters content; expanding a lane reveals properties.
Other views may show hierarchies differently without changing the document.
Tabs can pin explicit locations. Zoom, expansion, onion preferences, and current
selection are editor state rather than animation content. Persistent workspace
preferences can be saved separately.
## A session, revised
1. In `main`, create a girl lane and a new drawing. Draw; use New drawing (`N`)
to create the next one with the previous cel ghosted behind it.
2. Use Duplicate drawing (`D`) when the current shapes are the starting point.
Use Reuse drawing for a deliberately linked cel. The UI shows the
difference before an edit can change other uses.
3. Time the performance. Hold longer (`H`) extends the selected cel and
ripples later cels in the explicitly targeted lane. A trim gesture
can use overwrite instead. The preview shows which boundaries will move.
4. Choose a two-frame default cel for newly created drawings, or run a
separate Retime cels command on a selected range. This does not quantize
lane transforms or silently retime already authored cels.
5. Place the background in a lane below. Its source holds one frame throughout
its cel. Key the lane's X position at the beginning and end and choose
linear interpolation. The background slides while the girl's drawings cut.
6. Select three frames on the girl's lane, choose Return motion, and rotate to
the desired peak. A bounded rotation correction affects the girl across any
drawing boundaries in that range. Existing motion survives outside it.
7. Insert a playing animated symbol among the girl's held drawings. Set that
cel's source playback to advance. No lane conversion is required.
The timeline shows named cel blocks with property marks and optional curve
subrows. The cel sheet shows the same cels by frame and lane. The
graph editor edits the same properties; the stage resolves the same document.
Onion skin is configurable and counts neighboring cel events, skipping gaps
by default; a long hold does not consume the budget. Repeated uses of the same
drawing remain distinct events. Deduplicating identical ghosts is a display option.
## Proof obligations and implementation order
The source-channel prototype has been removed: a cel names one symbol
and carries its own playback clock, and `node/problems` rejects the old
`[:source]` channel. What a lane IS lives in `arthur.domain.symbol` beside the
other rules about a node map; `arthur.domain.lane` holds the commands over
one — add lane, place a drawing (new, reused or duplicated), make unique, split,
trim, move, blank and extend hold. Each is one history step, and each refuses rather than
half-applying. The timeline draws a lane's cels as cel blocks on the
lane's own row, and offers Make unique only where the selected cel actually
shares its drawing.
There is ONE placement function and a position argument, so appending is not a
different operation from inserting: `:end` is a position like any other, the one
where nothing has to move. Placing ripples — cels at or after the
position move later by the new cel's duration — and `:keep` versus
`:grow-symbol` still decides what happens at the shot's end. OVERWRITE is not a
policy argument yet, deliberately: taking frames away from the cel
already there is trimming, and until `trim` exists, placement that would need it
refuses instead of approximating it. A position inside an existing cel
refuses too, and names `split` — one command does not quietly perform two.
Splitting turned out to cost almost nothing, which is evidence for the
representation rather than for the command. The two pieces keep ONE `:time` and
differ only in `:span`, so the right piece's own frames carry on where the
left's stopped and its source clock, keys and corrections go on meaning what
they meant — a held drawing holds the same frame either side, a playing insert
plays through the cut without a seam, and the test for it samples every frame
before and after and asserts the picture is identical. That falls out of `:span`
being in the node's own coordinates; it is not something split arranges.
Content copies are shallow by default and keep their references to other
symbols; `:deep? true` is the explicit copy that shares nothing, so the promise
of independence is only made where it is kept.
THE SHOT LENGTH IS AUTHORED, which is the decision the range commands forced.
`:frames` is the symbol's window — how long the shot IS — and the occupied
extent of its lanes is a different fact derived from the cels. A command
grows the window only when the caller says `:grow-symbol`, and never shrinks it:
blanking the end of a shot leaves a shot with empty frames at the end, because
that is a true statement about what somebody authored, and deriving the window
from the extent would make deleting the last drawing quietly shorten the film.
`finish` keeps the two numbers apart by name now rather than by a `max` that
read like an accident.
Trim NARROWS one edge and moves nothing else; lengthening is `extend-hold`,
which carries the ripple and shot-length policies because it needs them.
Move is one write to `:time :at` and REFUSES a destination that would overlap,
because moving a drawing and re-timing the ones around it are different
intentions — clear the room with `blank` or `trim` first, which is the
composition. Blank leaves a gap and does not close it; a cel wholly inside
the range goes, one overlapping an end is trimmed to it, and the one spanning
the range is split. Their drawings stay in the library, since a lane does not
own its content.
All three are the same geometry as `split`: a `:span` is in the cel's own
frames, so moving an edge is one write and `:time` and `:playback` are never
touched. That is why trimming the front of a playing insert starts it later into
its animation instead of restarting it — the difference between trimming and
slipping, and the reason they stay separate commands.
Correction layers EVALUATE. `channel/problems` used to refuse an `:over` stack
and `value-at`/`cursor` used to throw on one; both now read it, and the
agreement test that holds the optimized cursor to the specification covers
stacked channels in forward, backward and random frame order. A layer's values
are themselves a channel, so a constant adjustment, a ramp and a return motion
are one mechanism; `:support` is half-open and a layer is inactive outside it;
and a layer has no time space of its own, because the node its channel is on
already has one. Nothing had to change in the codec — a channel is one leaf, so
a correction persists inside it — and nothing had to change in validation
plumbing, since `node/problems` already reports every channel's problems.
Both halves of ownership are under test at lane level: a three-frame correction
on the girl's lane reaches across the drawing boundary beneath it and leaves
every frame outside its support identical, and a correction owned by one
cel travels with that cel when a hold before it grows.
Regeneration keeps them, which is the obligation the layer design exists to
meet: `rebased` replaces a base and carries its corrections across, and a
correction the new base no longer fits is MARKED rather than dropped or
misapplied — `clip/conflicts` lists those for a view to offer, separately from
`problems`, because a conflict is a decision nobody has made yet and not a
document that will not load. Turning the mouth's `:verts` knob is a real
topology change and is what the test uses. Two latent faults turned up there and
are fixed: `regenerate-head` compared authored channels to measured ones
directly, so the first correction on the head would have stopped it following
re-measurement for good; and an incompatible offset threw in the read path,
which would have taken the stage down on exactly the case the model says to
report.
Overwrite is `blank` followed by non-rippling placement, composed inside one
transaction; insertion keeps its ripple rule. Still unbuilt: slip source,
retime, and deleting reused content. A lane cannot hold AUDIO cels — `lane-problems`
requires visual ones, though this document says a lane may hold either and
should reject only a mixture.
`domain/correction.cljs` now produces Constant adjustment, Ramp, and Return
motion layers for rotation and position. The inspector exposes them on a selected
lane or cel using an explicit range in that owner's frames; this deliberately
leaves displayed-range dragging through nested or retimed owners for later. One
Apply is one undo step. Conflicted layers are listed, can be removed, and can be
retried when the complete ordered stack is compatible again. Validation,
conflict reporting, and regeneration share that ordered-stack rule, including
coverage by adjacent replacement layers. Slip source and retime are still not
implemented; a refusal is the current behavior where the model demands an
explicit choice nobody has made yet.
The cel sheet is the same projected cels and selection addresses with its axes
turned: frames down and lanes across, so commands selected there and in the
timeline have identical targets; a gap selects its column's lane rather than
retaining a stale selection from another column. The suite stands at 437 tests and 5,804
assertions, with `frontend/test/browser/lane.mjs` driving the editor through
create, hold, overflow, undo, reuse, make unique, duplicate, split, insert,
trim, move, blank, correction authoring, and two-lane sheet targeting. Rewrite tests that encode superseded
behavior rather than preserving behavior to keep them green.
Build small adversarial documents and test their domain operations before
expanding the interface:
| Scenario | Required invariant |
| --- | --- |
| Same drawing exposed twice, then one made unique | Linked edits affect both before copying and only the selected content after |
| Holds, playing inserts, nonzero in-points, and gaps on one lane | Source behavior is independent of property key count and channel encoding |
| Adjacent cels, final hold, split, trim, ripple, and overwrite | Exact boundaries, stable IDs, deterministic collision handling |
| Extend a hold under lane keys and cel-local corrections | Lane key times remain fixed; later cel corrections travel with their owners; source playback origins survive |
| Ripple beyond the symbol end | Explicit extent policy; refusal leaves the document unchanged; resizing and retiming undo together |
| Girl on twos over a moving background | Drawing cadence does not quantize either lane's continuous properties |
| Three-frame correction crossing a drawing boundary | Exact support, same result outside it, one undo step |
| Nested retiming, holds, loops, and fractional sampling | Explicit time spaces; ambiguous inverse edits cannot silently choose a target |
| Audio inside changing source cels | Only active intervals sound, with correct trim and source timing |
| Regenerate with corrections and a topology change | Compatible edits survive; incompatible ones produce actionable conflicts |
| Reference cycles and deletion of reused content | Inactive references are validated too; no dangling references |
| Save/load and command undo/redo | Identity, source maps, corrections, and evaluation round-trip |
| Timeline and cel-sheet invocation of one command | Identical document changes and selection targets |
| Random forward/backward seeks and export | Reference and optimized evaluation agree, including defaults and absence |
| Concurrent commands on overlapping and disjoint targets | Transactions remain valid; conflicts are explicit and undo preserves others' work |
Implementation order: cel ownership and playback semantics; shared
commands and validation; correction layers and time-addressing contracts; then
breadcrumb, cel strip, and a cel-sheet projection. Use those two temporal
views plus direct stage editing to prove the model before broadening the UI.
A new presentation should not require duplicate animation state. A genuinely new
authoring capability may require new domain data. The model is successful when
such additions have a clear owner and compose with existing operations, not when
it can claim that no future feature will ever need another field.

161
docs/lane-nesting-notes.md Normal file
View file

@ -0,0 +1,161 @@
# Lane nesting interaction notes
Status: design note, 2026-10-01. This records the interaction before more lane
UI is implemented.
## The capability that must not be lost
A lane owns temporal placement, but a symbol instance is still a doorway into
another symbol. A drawing accidentally authored at the root must be movable into
an instance in any lane, including another lane, without changing its visible
position or timing.
That operation already exists as `nest/move-node`. It resolves the source and
destination at the current root frame, transplants the node, and re-expresses
its transform and time under the new parent. The lane UI must expose a target
path for it; it must not replace it with a weaker `:parent` assignment.
There are therefore two different drag intentions:
1. **Temporal move:** drag a clip body onto lane space. It remains a clip in a
lane, moves in time, and claims the destination interval by trimming/removing
incumbents.
2. **Structural move:** drag from the clip's grab affordance onto another symbol
instance. The dragged node is transplanted into the target instance's source
symbol with `nest/move-node`, preserving its world transform and root timing.
These cannot be inferred from overlap alone. Dropping clip A onto time occupied
by clip B already means “A claims that time and trims B.” Structural nesting
therefore needs an explicit grab affordance/mode. Its cursor is `grab` and
`grabbing`; trim edges keep their resize cursors and the ordinary body keeps its
timeline-move behavior.
Both visible clip blocks and an expanded symbol header are structural drop
targets. This permits moving a root drawing directly into `symbol-3` even when
its lane is collapsed.
## Compact expansion: one selected-clip portal
Expanding a lane must not restore row-per-clip vertical growth. Instead, an
expanded lane reveals exactly one clip portal: the currently selected clip in
that lane.
```text
▾ foreground lane [symbol-1][symbol-2][symbol-3]
▾ symbol-3 instance/source header and drop target
▸ body lane nested rows, mapped to the root ruler
▸ face lane
position nested keyframes mapped to root time
```
- Selecting another block in the same lane swaps the portal in place.
- With no selected clip in that lane, expansion shows a compact “select a clip
to inspect” row. It must not follow the playhead during playback; that would
make the timeline restructure itself while playing.
- The portal header represents the selected instance and is the structural drop
target for moving root or sibling content into its source symbol.
- Sub-expanding the portal uses the existing recursive symbol-row walk. Nested
lanes and channels are mapped through the instance clock into the open/root
ruler, as ordinary expanded instances already are.
- The lane's own transform/channel rows remain available separately. They affect
every clip in the lane and are not properties of the selected portal.
This keeps the cost of inspection constant: an expanded lane adds one selected
symbol branch, not one branch for every temporal clip it contains.
## Keyframe visibility
Two levels should be visible without changing editors:
- The selected clip's instance-level keys (transform, visibility, corrections)
appear as ticks inside that clip block on the lane row.
- Expanding the lane opens the selected clip portal, where source-symbol and
recursively nested keys appear on their own rows, mapped to root time.
Thus the collapsed lane answers “where does this clip change?” and the expanded
portal answers “which property inside this symbol changes?” The second view is
still the root timeline; entering the symbol is not required merely to see or
edit its keys.
## Drag targets and feedback
- Grab onto lane background: move/adopt the instance into that lane.
- Grab onto a symbol clip: structurally transplant into that clip's source
symbol.
- Grab onto the expanded portal header: the same structural transplant, with a
larger and less ambiguous target.
- Grab onto itself or one of its descendants: refuse before drop to prevent a
symbol cycle.
- A structural target receives an inset highlight and the preview stays in that
target. A lane-time target receives the dashed temporal clip preview.
- Successful structural drops expand the target lane and select the moved node
beneath the target portal, so the result is immediately visible.
## Data model consequence
No lane-as-symbol type is required. The hierarchy remains:
```text
symbol -> sequence lane -> instance clip -> source symbol -> its lanes/nodes
```
Lane membership owns time partitioning. Symbol instances own composition
nesting. The UI may present the selected instance below its lane, but that is a
derived portal, not another ownership edge and not a duplicated node.
## Implementation order
1. ~~Render instance-level key ticks within lane clips.~~ Done: a clip's keys
are on its block, drawn after the blocks so they land on the one they
belong to.
2. ~~Add selected-clip portal expansion to `timeline/rows`.~~ Done, with two
additions the note did not anticipate:
- The portal is chosen by the whole LINEAGE of the selection, not the
selected id. Selecting a shape inside the clip, or the end of its span,
is still working inside that clip, and matching the id alone closed the
portal the moment anything under it was touched.
- A HELD clip opens too. `clip/source-time` is nil for a hold, so the walk
used to stop there and the inside of every drawing was unreachable from
the root timeline. Its rows are now shown across the hold and marked
`:unmapped?`: no keys, and no draggable edges, because no frame inside it
has a place on this ruler.
3. ~~Add the explicit structural affordance.~~ Done as SHIFT on a clip-body
drag rather than a separate grab handle: shift turns a temporal move into a
structural one, the target clip takes an inset highlight, and a label by the
pointer says which of the two is about to happen.
4. Route structural drops through `nest/move-node`. **Wired, and blocked in
the domain.** The gesture asks `nest/move-refusal` on the way past, so the
label says before the drop what the command would say after it. Two
refusals stand in the way of ordinary use:
- *both have to be on screen at this frame.* Inherent, and worth keeping:
the move preserves the world transform and there is no common frame to
preserve it at otherwise. It does mean nesting one clip into another in
the SAME lane can never work — a lane never overlaps itself — so this is
a between-lanes gesture with the playhead somewhere both are showing.
- *a held or looping clip has no clock to move through.* `nest/inside`
returns no `:time` for a hold, and a held one-frame drawing is the most
common thing in a document, so today nesting into one is refused — which
is most of what anybody would try.
5. After the transplant, expand the destination portal and reveal/select the
moved row. `::ui/move-node` already selects the moved node and opens the
rows down to it; the portal follows from the lineage rule in 2.
## The held destination, unresolved
A held cel shows ONE source frame for its whole span, so there is no
invertible map from the lane's frames to the drawing's and `move-node`
refuses. But the refusal is stronger than the facts require. Inside a frozen
destination only one frame is ever observed, so:
- the RATE of any map into it is unobservable — every rate shows frame `in`;
- what IS observable is that the moved node should show, at that one frame,
what it shows now at the current root frame.
That pins a unique sensible answer — rate 1, aligned so the current frame maps
to the shown frame — and nothing else about the mapping can be seen. If that
argument holds, it is a rule rather than a guess, and it is the difference
between structural nesting working for drawings and not working at all. It
needs its own proof: a drawing authored at the root, nested into a held cel in
another lane, sampled before and after to show the same picture, in the style
of `drawn` in `lane_test`.

View file

@ -8,11 +8,13 @@ symbol instance. Timelines already provide local node names, independent playbac
and persistence. No new kind of scene container is needed. and persistence. No new kind of scene container is needed.
```clojure ```clojure
:timelines :symbols
{:main {:nodes {:root {:time {:mode :map :expose 2}} {:main {:nodes {:root {:time {:mode :map :expose 2}}
:face {:parent :root :channels <source-to-stage placement>} :face {:parent :root :channels <source-to-stage placement>}
:face-1 {:kind :symbol :of :face-1 :parent :face :z "a0"} :face-1 {:kind :instance :source {:symbol :face-1}
:face-2 {:kind :symbol :of :face-2 :parent :face :z "a1"}}} :parent :face :z "a0"}
:face-2 {:kind :instance :source {:symbol :face-2}
:parent :face :z "a1"}}}
:face-1 {:nodes {:head {...} :mouth {:parent :head ...} ...}} :face-1 {:nodes {:head {...} :mouth {:parent :head ...} ...}}
:face-2 {:nodes {:head {...} :mouth {:parent :head ...} ...}}} :face-2 {:nodes {:head {...} :mouth {:parent :head ...} ...}}}

66
docs/one-grid-plan.md Normal file
View file

@ -0,0 +1,66 @@
# The editing grid is the symbol's own frames
## What was wrong
Every number a person authors — a span, a `:time :at`, a key, a cut — is in the
frame space of the symbol it lives in (`docs/time.md`, and that part is right).
The timeline, though, drew its ruler in OUTPUT frames: `clip/output-frames`, the
transport's length. In a 12fps project holding 30fps symbols those two spaces sit
at a ratio of 2.5, so:
* a clip at symbol frame 31 was drawn at ruler frame 12.4 — **no clip edge landed
on a frame mark**, because almost none of them can;
* a gesture measured in ruler frames had to be multiplied into the symbol's
frames and rounded (`nest/dragged`), so dragging one ruler frame moved the clip
2 or 3 symbol frames — **0.8 or 1.2 ruler frames, never the 1 the pointer
said**. That is the jumpiness;
* the preview drew the gesture's own number of ruler frames while the commit
wrote the rounded one, so **the ghost sat somewhere the settled clip did not**;
* before the rounding was added, the fraction went into the document and every
later edge edit on that clip was refused for ever (`span/*` refuses a
fractional edge, as it should).
One cause, four symptoms. None of them is an edge case to patch.
## The model
1. **A symbol's own frames are the only coordinate anything authored lives in.**
Unchanged.
2. **The editor edits in the open symbol's frames.** The ruler, the marks, the
playhead's position on it, every pointer→frame answer, every drop frame and
every drag delta are the open symbol's frames. No multiplication anywhere in
the gesture path, so no rounding and nothing fractional to refuse.
3. **The output grid is playback's alone** — the clock, the audio mix, export,
and the frame the stage draws. Exactly two pure functions cross between them
and nothing else does:
* `clip/shown-frame clip sid f` — which of `sid`'s frames output frame `f`
shows (`cadence/frame`: the latest at or before it).
* `clip/first-output-frame clip sid n` — the output frame that first shows
symbol frame `n`; the inverse, for seeking from the ruler.
4. `nest/inside`, `nest/placement`, `nest/spans` and the gestures take the
subject symbol's OWN frame. They used to take an output frame and multiply it
secretly, which is what made every caller's units a guess. A caller holding
the playhead converts with `clip/shown-frame`, at its own edge, visibly.
## The gestures
5. **One pointer→frame function** for the whole timeline, `frame-under`. A drag's
delta is the difference of two of its answers, never a pixel ratio rounded
separately — so the preview and the commit are the same number by
construction.
6. **The junction between two clips is one handle with one meaning**: roll. It
moves the end of the left clip and the start of the right one together, which
is `span/roll`, which is already nothing but `resize-out` then `resize-in`.
The three 4px-wide zones it used to pick between — trim-left, roll,
trim-right, inside twelve pixels — were the "it just picks one" the handle
was accused of. An edge that is not shared still has its own in/out handles.
## Audio goes with the picture it belongs to
A take's sound lived inside the take symbol, so placing the take brought it and
placing the FACE the take is made of brought nothing. A symbol now says what it
sounds like — `:audio`, a sound source — and placing one places a linked audio
node beside the clip. Detection sets it on the face it extracts, which is the
automatic link; `::ui/link-audio` sets or clears it by hand, which is the manual
one. `:linked-to` on the audio node already existed and already follows a moved
picture.

View file

@ -146,7 +146,6 @@ Full specification in `docs/animation-model.md`. The subset to build:
[:xform :rot] {:animated? false :value 0.0} [:xform :rot] {:animated? false :value 0.0}
[:xform :scale] {:animated? false :value [1.0 1.0]} [:xform :scale] {:animated? false :value [1.0 1.0]}
[:xform :skew] {:animated? false :value [0.0 0.0]} [:xform :skew] {:animated? false :value [0.0 0.0]}
[:xform :anchor] {:animated? false :value [0.0 0.0]}
[:geom :pts] {:animated? true :interp :hold [:geom :pts] {:animated? true :interp :hold
:dense {:store "sha256:…" :offset 0 :stride 40 :frames 600} :dense {:store "sha256:…" :offset 0 :stride 40 :frames 600}
:generated {:by :roto/lips-outer :analysis "sha256:…" :generated {:by :roto/lips-outer :analysis "sha256:…"
@ -170,15 +169,24 @@ uses to offer a parameter panel instead of raw keys. It lives on the *channel*,
not the node, because a node wants a rotoscoped `[:geom :pts]` and a not the node, because a node wants a rotoscoped `[:geom :pts]` and a
hand-animated `[:xform :pos]` at the same time. hand-animated `[:xform :pos]` at the same time.
`:skew`, `:span`, `:anchor` and `:over` stay in the shape even though nothing `:skew`, `:span` and `:over` stay in the shape even though nothing drives them
drives them yet: each is a component of a decomposition or of a composition yet: each is a component of a decomposition or of a composition order, and adding
order, and adding one later migrates every stored transform. one later migrates every stored transform.
`:anchor` was in this list and has since been **deleted**, which is the one place
the reasoning above came out wrong. It is not a component of the decomposition:
`T(a)·M·T(-a)` is `M` conjugated by a translation, and a parent already is a
translated frame, so an anchor is a peg written inline — one that cannot be
selected, keyed, shared, or put above a measured channel. Rotation and scale
happen about the node's own origin; a pivot nobody chose is derived per drag by
`domain/gesture` and a pivot somebody chose is a peg. See
docs/animation-model.md, "There is no `:anchor`, because an anchor is a peg".
Transform composition, per node: Transform composition, per node:
``` ```
local = T(pos) · T(anchor) · R(rot) · K(skew) · S(scale) · T(-anchor) local = T(pos) · R(rot) · K(skew) · S(scale)
world = world(parent) · local world = world(parent) · pinv · local
``` ```
## What the prototype knows that you would otherwise rediscover ## What the prototype knows that you would otherwise rediscover
@ -333,11 +341,12 @@ per-frame header.
numbers: it centres on the face oval's bbox and zooms until the face is 80% of numbers: it centres on the face oval's bbox and zooms until the face is 80% of
the raster height, so every vertex carries a cropping decision made once from one the raster height, so every vertex carries a cropping decision made once from one
frame's landmarks. Dropping it is a deletion. Placement becomes `[:xform :*]` on frame's landmarks. Dropping it is a deletion. Placement becomes `[:xform :*]` on
an authored `:face` node, the stage clips whatever hangs off, and project an authored `:place` node inside the face, the stage clips whatever hangs off,
and project
dimensions stop being tied to the footage. See "What space geometry is in" in dimensions stop being tied to the footage. See "What space geometry is in" in
`docs/animation-model.md`. `docs/animation-model.md`.
The anchor transform freezes onto `:head`, one level under `:face`, and the The anchor transform freezes onto `:head`, one level under `:place`, and the
normalise on/off/per-plate toggle is which of the three channel shapes that node normalise on/off/per-plate toggle is which of the three channel shapes that node
carries. Always measure and always store factored, whatever the toggle says: carries. Always measure and always store factored, whatever the toggle says:
smoothing and velocity-minimum key selection both require the split to exist in smoothing and velocity-minimum key selection both require the split to exist in

41
docs/time.md Normal file
View file

@ -0,0 +1,41 @@
# Time selection
Project `:fps` is the playback and export grid. Each symbol has its own native
`:fps` and `:frames`; keys, spans, trace choices and corrections stay in that
native space. A symbol without an explicit rate inherits the document rate;
changing project fps first records that rate so its existing timing stays put.
The untouched symbol in a new document is deliberately different: it has no
authored timing to preserve, so it stays on the project grid and its empty frame
extent is rescaled to keep the same duration. This makes changing fps before
authoring establish the editor's grid instead of preserving the 30fps default.
An output frame selects the latest native frame at or before its time:
`floor(output-frame * native-fps / output-fps)`. Thus 30fps content in a 12fps
project reads source frames 0, 2, 5, 7, 10… and retains its duration. A partial
last output frame is included. Changing back to 30 restores the original grid.
Nothing rewrites or discards the dense measurements.
The same boundary selection runs when entering a placed symbol. Placement and
artistic speed are applied before selection; the stored `:time :rate` and
`:playback :speed` never contain a frame-rate conversion. The derived maps used
by timeline rows, picking and editing account for the units of each symbol.
`clip/frames` is a native length; `clip/output-frames` is a transport/export
length. Resolver frame queries return native frames for edits.
There is one fps control. The old transient picture-fps control and node
sample-fps fields are gone. Existing exposure, trace choices and per-instance
pose tracks remain available: a pose track can hold a chosen closed-mouth frame
without deleting its neighboring measurements. Those choices stay in native
frames when output fps changes. Automatic content-aware frame selection is not
implemented; [frame-selection.md](frame-selection.md) is how it should be. An
event between output frames appears on the next output frame; it cannot create
an extra frame in a 12fps output.
Audio uses continuous time through the same derived placement maps, without
picture floors or holds. Frame-rate units cancel before Web Audio playbackRate
is set, so only deliberate speed changes affect pitch and duration. Export and
playback use the same output count and resolver.
Earlier imports with frame-rate conversion baked into stored retimes must be
re-imported. There is no second reader for that representation. Source video
presentation timestamps are still future work; this model assumes constant fps.

View file

@ -70,7 +70,7 @@ handling and the relevant key whitelist if its storage location requires it.
- `freeze/performance-nodes` marks generated animated channels with - `freeze/performance-nodes` marks generated animated channels with
`:pose-sampled?` and local `:pose-group` names. This includes keyed visibility `:pose-sampled?` and local `:pose-group` names. This includes keyed visibility
as well as dense geometry. `:generated` remains provenance for regeneration. as well as dense geometry. `:generated` remains provenance for regeneration.
- `timeline/channel-frame` already applies explicit pose choices and default - `symbol/channel-frame` already applies explicit pose choices and default
picture sampling to marked channels. Playback and export both use picture sampling to marked channels. Playback and export both use
`clip/resolver` with `:picture-fps`; there is no need for a second sampling `clip/resolver` with `:picture-fps`; there is no need for a second sampling
implementation. Export's pose count is still a rate-based estimate. implementation. Export's pose count is still a rate-based estimate.

View file

@ -1,5 +1,11 @@
# Timing model # Timing model
[Time selection](time.md) defines the current frame-rate representation.
[The Lane Model](lane-model.md) defines the revised target for occurrence timing,
source playback, sampling scope, and inverse editing. It supersedes conflicting
proposals here; the sections below describe earlier implementation decisions.
The source footage, authored drawings, generated face motion, and stage placement The source footage, authored drawings, generated face motion, and stage placement
have different frame decisions. They share a clock but do not share one kept-frame have different frame decisions. They share a clock but do not share one kept-frame
list. `timing-handoff.md` records earlier implementation notes. list. `timing-handoff.md` records earlier implementation notes.
@ -35,7 +41,7 @@ measurements without losing the anchor choices.
A source image used for tracing should be registered with that image's measured A source image used for tracing should be registered with that image's measured
stabilizing transform, then the selected head transform, then the authored stabilizing transform, then the selected head transform, then the authored
`:face` placement. This makes the photo and head-local vectors share the same `:place` placement the face carries. This makes the photo and head-local vectors share the same
orientation and position. Tracing-photo selection is a separate editor address; orientation and position. Tracing-photo selection is a separate editor address;
it does not choose the head anchor. it does not choose the head anchor.
@ -78,7 +84,8 @@ candidate poses so stage cuts can still select any of them.
| --- | --- | --- | | --- | --- | --- |
| Source frames and timestamps | Footage/analysis | Constant-rate frame indexing exists; variable timestamps remain future work | | Source frames and timestamps | Footage/analysis | Constant-rate frame indexing exists; variable timestamps remain future work |
| Head anchor map | `:head` node | Implemented, stored with the node | | Head anchor map | `:head` node | Implemented, stored with the node |
| Tracing cel starts and photo address | Authored cel | Separate future work | | Trace frames (photo address) and origin | The face's `:plate` `:time :holds`, and its `:head` `:reads` | Implemented, see `docs/tracing-symbol-plan.md` |
| Showing a tracing layer, and its opacity | Editor state, `[:ui :tracing]` | Implemented; a drawing aid, never saved or keyed |
| Generated picture-rate proposal and closure protection | Roto clip/symbol | Generated-only picture sampling exists; closure protection remains future work | | Generated picture-rate proposal and closure protection | Roto clip/symbol | Generated-only picture sampling exists; closure protection remains future work |
| Stage pose cuts | Symbol instance | Implemented, stored with the instance | | Stage pose cuts | Symbol instance | Implemented, stored with the instance |

411
docs/tracing-symbol-plan.md Normal file
View file

@ -0,0 +1,411 @@
# Plan: tracing is a symbol
Status: built, 2026-10-03, on branch `worktree-tracing-layers`. Where the build
differs from the plan below, the build wins:
- The head's field is `:reads` (`{:holds [...]}` or `{:holds-of :plate}`), not
`:follow`.
- A still is not one frame. It gets as many frames as remain in the symbol it is
dropped into, at that symbol's rate, and is trimmed like any clip.
- An image's `:media` is `{:image <blob sha256>}`, served at `/blob/<sha>`. The
server's `Image` row exists for the pool's list and labels, not for identity.
- Dropping media that a tracing symbol in the document already shows reuses that
symbol.
- Nothing can be created or dropped inside a tracing symbol (`creation/target`,
`drop-destination-at`, `nest/move-refusal`), and one cannot be opened in a tab.
- Schema 6. Every project is marked 6. One that still carries `:trace` on a
node is refused when opened, with what to do about it, rather than all old
projects being refused.
No backward compatibility (see `lane-model.md`, "Goal and compatibility policy").
## What is wrong today
Tracing is not a thing in the document. It is three mechanisms that each know
about faces:
- **`:trace {:frames :origin}` on a face's `:head`.** It decides which measured
frame the head reads (`symbol/base-channel-frame` → `trace/held-frame`), and
the underlay also reads it to decide which photo to show (`trace/photo-frame`).
- **`ui/underlay`**, a painter that walks `trace/shown` → `trace/faces` (a
separate instance walk), looks up each face's subject → analysis → footage,
asks the resolver where `[...path :head]` went, and builds the photo's matrix
by hand: `world(head) · M(p)⁻¹ · 1/imageH` (`trace/photo-matrix`).
- **`[:ui :trace {:faces #{} :opacity}]`**, a per-face switch in editor state,
plus the special "opening a face shows its footage, opening a take does not"
rule (`trace/showing-for`).
What you cannot do: place footage or a still as a reference where you like, move
it, scale it, turn it, trim it, hold it, put it in a lane, or trace something
that is not a tracked face. The photo is not selectable and has no row. The one
case that works (a face over its own footage) runs on code that nothing else
uses.
## The model in one paragraph
A **tracing symbol** is a symbol with `:type :trace`. It has no nodes. It names
media (a footage range or a still image) and has that media's frame count, fps and
pixel size. It is **placed by an ordinary instance**, so lanes, spans, trim,
split, move, playback (`:in :speed :end`), transform, nesting, selection, picking
and gestures all work on it with no new code. It is evaluated to a single
**`:trace` op**, which never reaches the raster or an export. A face's footage is
not a special case. It is one of these instances, placed inside the face as a
child of `:head` and carrying the measured registration transform. The trace keys
become **holds on that instance's time**. The head's "origin" becomes the head
saying **which node's frames it follows**.
This follows the precedent of `:type :palette` symbols (no nodes, they name an
asset, and they are placed by instances in a lane), so the uniformity rule holds
(see the `model-uniformity` memory). Nothing new holds nodes, and no special
instance kind is added.
## Data model
### The tracing symbol
```clojure
:footage-8625 ; a symbol id like any other
{:id :footage-8625 :name "8625.mov"
:type :trace
:media {:footage #uuid "f8ca…" :range [12 241]} ; or {:image "sha256:…"}
:frames 229 ; the range's length; 1 for a still
:fps 30 ; the footage's own rate, so cadence handles 30→12 for free
:width 1440 :height 1920 ; pixel size = the symbol's own stage
:audio {:footage #uuid "f8ca…"} ; optional, the existing "a symbol says what it sounds like"
:nodes {}}
```
- `:media` is the one new symbol key. Add it to `symbol/symbol-keys` and to the
symbol leaf's `select-keys` in `leaf/leaves`.
- The symbol's local space is **pixels of the media**: `[0 w) × [0 h)`.
`clip/center` of a node-less symbol already returns its stage middle, which here
is `[w/2 h/2]`. So `place-symbol` puts the anchor at the image centre with no
new code.
- `symbol/problems`: `:type :trace` needs `:media` with exactly one of
`:footage`/`:image`, needs `(empty? nodes)`, and for footage needs
`(= frames (- end start))`.
- **One tracing symbol per footage range, shared.** Two faces from one take
place the same symbol, and so does a hand-placed reference. Like any symbol it
is reused by reference, and `bring/symbols` copies it like any other.
### Placing it
It is an ordinary `:kind :instance` with `:source {:symbol :footage-8625}`:
- **Start and end** are `:span` (in its own frames) and `:time :at`. You get
these from the existing trim, split, move and roll.
- **Which frame shows** comes from `:playback {:in :speed :end}`. Footage plays
with speed 1, a frozen frame has speed 0, and a still is 1 frame with `:end :hold`.
- **Placement in space** is `[:xform …]`, the same as every node: gestures,
inspector and keys.
- **In a lane, or on its own**: it is a parent-less spanned node, so a `:display
:lane` symbol draws it as a block like any cel. It can also sit as a free node
or under a group. To have a reference with its own tab, wrap it in an ordinary
symbol.
**Default on creation** (in the drop event, not in `place-symbol`): scale so the
image's height fits the stage height, centred on the drop point. This is a
creation default like "use center anchor when dropping", and nothing updates it
afterwards.
### Holds: the one new time feature
`:time {:holds [0 12 30]}` floors a node's local frame to the last hold at or
before it. Before the first hold, the first hold applies. This is `:expose`
generalized from a regular grid to authored frames. It is applied in
`node/local-frame` at the same point as `:expose`, and is inherited in the same
way. It is in the node's **own** frames. On a tracing instance with
`:in 0 :speed 1`, own frames are source frames, so the hold list is the set of
traced frames.
It is general on purpose. Holding a playing symbol on chosen drawings is the
same feature. It costs about three lines in `local-frame`, one `problems` clause
(sorted, distinct, finite), and the timeline drawing hold frames as marks on the
row.
`symbol/frame-map` refuses floors (`:expose > 1`). It must also refuse a
non-empty `:holds`, for the same reason.
## The face: "a child symbol that represents the trace"
The face symbol after a freeze:
```
:place group authored source→stage mapping (image heights → stage px)
:head group measured M(p) :reads — see below
:mouth … parts generated, every frame
:plate instance of :footage-8625 ← the trace
measured channels: fit(p) = M(p)⁻¹ · S(1/imageH)
:time {:holds [0 12 30]} ← the trace keys
```
### Registration comes from the parent
The plate is a child of `:head`. Its own measured channels are the
**stabilizing fit** at frame p: the inverse of the head's measured transform,
with pixels → image heights folded into the scale (it stays a similarity, so it
decomposes into `pos`/`rot`/`scale`). Freeze already computes the fit; the
head's measured channels are its inverse (`freeze/invert`). The plate gets a
second dense block, which is three small channels.
Because holds are a **time** floor, the plate's channels and the frame its
content shows are read at the **same** held frame q. Its world transform is:
```
world(plate) = place · M(p_head) · M(q)⁻¹ · S(1/H)
```
That is exactly `trace/photo-matrix`, but now it falls out of the ordinary walk.
It is registered to whatever the head is doing, by construction:
| head reads (p_head) | plate shows (q) | result |
| --- | --- | --- |
| f (continuous) | f (no holds) | footage where filmed |
| f (continuous) | held key | held photo rides the moving head (today's behaviour) |
| held key (same as plate) | held key | `M(q)·M(q)⁻¹ = I`: photo sits where filmed |
| 0 (start) | f or held | stabilized footage under a still head |
If a frame has no measurement, the plate's dense channel has nothing there, so
`xform-at` gives nil, so the plate is not placed and no photo shows. That is what
happens today too.
### Trace keys versus origin: who owns what
The two decisions are separate and stay separate:
- **Trace keys** are which footage frames get drawn over (the "plate drawings"
in `frame-selection.md`). They belong to the **plate**, as `:time :holds`. A
hand-placed tracing layer with holds is the *same thing*: the face's plate is
an ordinary tracing placement and nothing more.
- **Origin** is how the head moves between kept frames. `frame-selection.md`
already says `:origin` "is not a tracing setting" but a performance one, so it
belongs to the **head**:
```clojure
:head {…} ; continuous — reads its own frame
:head {… :reads {:holds [0]}} ; start
:head {… :reads {:holds-of :plate}} ; at keys — reads its measured channels at
; the frames :plate's holds select
```
`{:holds [...]}` is also what a face with no footage uses for "at keys", since it
has no plate to follow.
`:reads` changes only the **head's own channel reads**, not its children's
frames. The parts must keep running every frame, which is why this cannot be a
`:time` hold on the head. It is today's `traces` branch of
`base-channel-frame` with the hold list read from the named node. A node
reference has precedent (`:stencil`, `:pose-group`). It points from the follower
to the thing followed, so there is still one stored list of frames and nothing to
keep in sync. Validate it in `symbol/problems` the way `:stencil` is validated:
the target exists in the symbol, and it is not the head itself or an ancestor of
the head.
Rejected alternatives, so they are not re-proposed:
- **Keys on the head, and the plate reads them.** This is today's direction. It
makes the plate special: a free tracing layer could not have keys that a face's
plate also understands.
- **Hold `:time` on `:head`.** Exposure inherits strictly, so the mouth and eyes
would freeze along with the head.
- **A wrapper "registered footage" symbol holding the fit.** It is correct but
adds a symbol per face. The time-floor holds already put the fit read and the
content read on the same frame, so the wrapper buys nothing.
- **Plate as a sibling of `:head` at identity.** It is only registered when the
head and the photo read the same frame, so it breaks the continuous-plus-holds
and start rows above.
## Evaluation: an op that is never rendered
- **`clip/resolver`**, in the instance branch: when the source symbol has
`:type :trace`, it does not recurse. If `(:tracing? opts)` is set, it emits one op:
```clojure
{:kind :trace :node [id] :m <copy of world> :media … :frame shown-frame :size [w h]}
```
The frame comes from the same `placed-frame` path as any instance, so playback,
holds and the fps cadence apply. Do not build a child resolver for a trace
symbol.
- **`transform-op`** gets a `:trace` case that composes the matrix. Row paths,
solo filtering and nesting at any depth then work for free.
- **The output guarantee is structural.** `:tracing?` defaults to false. Only the
stage's `::render/resolver` passes true. Export, `clip/center` (a big photo
must not pull a symbol's pivot), thumbnails and the bench never ask for trace
ops. `raster/draw-ops!` keeps throwing on unknown kinds, so a leak fails loudly.
- **`ui/player`** sends picture ops to the raster and `:trace` ops to the
painter.
- **`ui/underlay` becomes `ui/tracing`.** It paints `:trace` ops in draw order
with `drawImage` at `op.m` and the global opacity. The media URL is the footage
manifest's `urls[range-start + frame]`, or the image blob URL. The three
steadiness fixes stay: LRU cache, hold the last still per op `:node`, and read
ahead while playing. Everything that walked faces is deleted.
- **`pick`**: a `:trace` op is hit when the point, mapped through `m⁻¹`, falls in
`[0 w) × [0 h)`. Picture ops are tested first and trace ops only if nothing
drawn is under the pointer, so a full-frame photo does not steal every click.
When the global switch is off, traces are neither painted nor picked.
- **Gestures**: no change. The face's plate is measured, so `gesture/refusal`
already says "place the instance it is in". A hand-placed layer is authored and
moves, turns and scales like anything else.
## On and off
All of it is EDITOR STATE, as ed88c5e decided for the per-face switch: showing
a reference is a way of looking at the stage, so it is not an undo step, does not
travel to collaborators, and cannot reach an export.
```clojure
[:ui :tracing {:on? true :opacity 0.5 :hidden #{[sid node-id] …}}]
```
- **One layer** is an entry in `:hidden`, keyed by the symbol the tracing
instance is in and its node id. The symbol id is needed because every face's
plate is called `:plate`. Keyed this way, hiding a face's plate hides it in
every placement of that face, which is what the per-face switch did. Shown is
the default, so opening a face shows its footage with no setup.
- **How it's applied**: `clip/resolver` already knows which symbol and node a
trace op comes from, so the op carries `:layer [sid id]`. The painter and
`pick` skip ops whose layer is hidden. The resolver does not change when you
toggle a layer, so nothing is rebuilt.
- **Global**: `:on?` and `:opacity`, a toggle plus an opacity slider in
`ui/palette/bar` next to the palette controls. When it is off, traces are
neither painted nor picked.
- **Switching one layer on also switches the global setting on.** This applies
from the tracing instance's inspector, from its timeline row, and from the
face/roto section. A layer you just enabled must not stay invisible behind a
switch you forgot. Turning one layer off never touches the global switch. It is
one event (`::ui/show-trace layer on?`) that updates `:hidden` and, when
turning on, sets `:on? true` in the same handler, so the three places cannot
behave differently.
- **`[:vis]`** still works on a tracing instance like on any node: key it to
show a reference only over part of the shot. It is not the on/off switch.
- **Deleted**: `[:ui :trace :faces]`, `::trace-face`, `::trace-faces`,
`trace/showing-for`, `traceable-faces`, `shown`, `faces`, and the "a take shows
nothing by default" rule.
## UI
- **Timeline.** A tracing instance is an ordinary row or clip block, styled to
read as reference-only (hatched block, an eye icon in place of the colour
chip). Hold frames are marks on the row. The row's eye button dispatches
`::ui/show-trace`, which replaces the face row's `tl-trace` button.
- **Inspector, tracing instance.** Show the media (footage name and range, or
image), an on/off control (which goes through `::ui/show-trace`), and the hold
list ("hold here" / "remove hold" plus seek buttons, which is `trace-keys`
re-aimed at `:time :holds`). Playback, transform and span use the existing
sections.
- **Inspector, face/roto section.** The same hold controls, aimed at the face's
`:plate`. Origin buttons write `:head :reads`. The section is found by "this
face has a `:plate`", not by `trace/traceable?`.
- **Palette bar.** The global toggle and opacity slider.
- **Making one.**
- Footage: the convert dialog gets a choice between "animate faces" (today's
flow) and "tracing layer" (no detection). The tracing-layer choice makes the
`:type :trace` symbol for the chosen range and places it where the video was
dropped.
- Footage can also be dragged from the pool with a modifier or as a second
drag kind.
- A still image: drop it on the pool and it uploads; drop it on the stage or
timeline and it is placed with `:end :hold` and a span of the host's
remaining frames.
## Freeze, bring, regenerate
- **`freeze/subject-part`** emits the `:plate` instance under `:head`, with fit
measured channels and `:time {:holds []}`, and emits one `:type :trace` symbol
for the analysed footage range. `bring/take` already copies every symbol the
take reaches and rewrites `:source :symbol` references, so the tracing symbol
comes along. It gets the footage's `:audio` too, so a dropped tracing layer
can bring its sound through the existing link.
- **`head-mode`** writes `:reads` on the head and `:holds` on the plate, in
place of `:trace`. The `frame-selection.md` plate proposal materializes into the
plate's `:time :holds` in place of `:trace :frames`.
- **Regenerate** replaces both measured blocks (head and plate) and keeps
`:holds` and `:reads`. Check that `regenerate-head`'s measured/authored
comparison handles a second measured node.
## Server
- `Image` endpoints mirroring `Sound`: `POST /api/images` stores a `Blob` and
returns `{id, width, height, url}`, and `GET /api/images` lists them. The
migration is a model only. Bump `schema_version` and refuse older documents
clearly. Do not convert them.
## Deleted outright
`trace/of`, `prepare`, `held-frame`, `problems`, `toggle-frame`, `photo-frame`,
`measured-local`, `photo-matrix`, `traceable?`, `faces`, `traceable-faces`,
`showing-for`, `shown`, `opacity-default`. `domain/trace.cljs` probably goes
away entirely; if anything is left, it is a hold helper that belongs in `node`.
Also deleted: `symbol/prepared-traces` and the `traces` arm of
`base-channel-frame`, which becomes the `:reads` lookup. `:trace` on nodes (and
`symbol/problems` reports it as removed). `::project/set-trace`.
`::render/underlay` and `::render/tracing` become one `::render/tracing` that
returns `[:ui :tracing]`. The face lookup in `params/view`.
Expected effect on code size: net negative. One time-floor clause, one op kind,
one resolver branch, one pick case and a `:reads` lookup replace the face walk,
the hand-built photo matrix, per-face showing state and the underlay's face
bookkeeping. Measure the change and report the number honestly (see the
`cljs-style` memory).
## Tests
**Domain (node).**
- `:holds` in `local-frame`, with eval-frame and resolver agreeing forward,
backward and in random order.
- A trace op appears only with `:tracing?`, and `export/run!` output contains no
`:trace` op.
- `transform-op` composes the matrix through two nesting levels.
- `clip/center` ignores traces, and is `[w/2 h/2]` for a tracing symbol.
- `pick` returns picture ops before traces and inverse-maps through a rotated
layer.
- `symbol/problems` covers `:type :trace`, `:reads` targets and cycles, and
`:holds` shape.
- Registration: port `trace_test`'s photo-matrix assertions onto the walk.
For every row of the registration table, the plate's world transform equals
the expected matrix. At a held key with `:reads {:holds-of :plate}`, it equals
`world(:place) · S(1/H)`.
- Freeze emits the plate and the tracing symbol. Regeneration keeps the holds.
- The `::ui/show-trace` event sets the global switch on when turning a layer on
and leaves it alone when turning one off.
**Browser** (CDP, see the `arthur-verify-dont-guess` memory):
- Drop a still, then drag, turn and scale it.
- Toggle a layer on with the global switch off, and confirm both are on and the
photo paints.
- Export a frame and check it has no photo pixels.
- Open a face: the plate is registered at a hold with origin "at keys", and
stabilized with origin "start".
## Order of work, each step green
1. `:time :holds` in `node/local-frame`, `problems`, `frame-map`, and timeline
marks.
2. `:type :trace` symbol, `:media`, the `:trace` op behind `:tracing?`,
`transform-op`, the player split, `ui/tracing` painter and `pick`. Footage
media only, placed by hand from a REPL or test document.
3. The face: freeze and bring emit the plate and tracing symbol, `:reads`
replaces `:trace`, delete the old trace and underlay paths, and re-aim the
inspector's face section.
4. On/off: the `:hidden` set and `:layer` on trace ops, the row eye, the
`::ui/show-trace` rule, the global toggle and opacity in the palette bar, and
delete `[:ui :trace :faces]`.
5. Creation: "tracing layer" in the convert dialog, pool drag, then the image
endpoint, pool images and image drops.
6. Docs: `animation-model.md` "A photographic underlay is not an op" becomes "a
trace is an op that never reaches the raster". Update `frame-selection.md`
(where plate keys live) and `lane-model.md` (tracing clips in lanes).
## Open questions
1. **"A take shows its faces' footage."** Dropping the old "only in a face's own
tab" default means opening a take shows every face's plate if the global
switch is on. The recommendation is to accept that, since the switch is one
click.
2. **Lane-model audio rule.** `lane-problems` requires visual cels. A tracing cel
is visual for this purpose, but say so explicitly when a lane may hold both
tracing and drawing cels.

View file

@ -88,18 +88,66 @@ cd frontend && mise exec -- npx shadow-cljs watch app
``` ```
Then open **<http://localhost:8778/>**. Django serves the page from Then open **<http://localhost:8778/>**. Django serves the page from
`clips/templates/clips/index.html`, and staticfiles serves the bundle out of `clips/templates/clips/index.html`, its styles from `static/arthur/app.css`, and
`static/arthur/js`, where `shadow-cljs` already writes it — so nothing copies files the bundle out of `static/arthur/js`, where `shadow-cljs` already writes it — so
between the two. nothing copies files between the two.
### The window
One screen, five panes, no scrolling page. `src/arthur/ui/shell.cljs` is the grid
and nothing else; each pane owns its own subscriptions.
```
top the document: its name and last status, export, new / open / save
left media pool — the open document's symbols, and footage on the server
centre the palette strip (16 slots) above the stage
right inspector — the clip, the selected node, the tracked objects
bottom timeline — transport, ruler, a row per node
```
**It opens on a blank document**, and **new** makes another one. Nothing is
loaded until it is asked for.
**Whole documents live under `open ▾`, not in the media pool**, and the split is
load-bearing rather than tidy. Opening a project REPLACES the stage; everything
in the pool is a thing to put ON it. Listing documents beside the symbols inside
one of them makes them read as two kinds of the same thing. The menu lists the
projects the server holds; the built-in scenes are under their own heading,
italic, and are not projects — they are compiled into the bundle and the server
has never heard of them.
Everything that holds nodes is a **symbol**, and none is special: a new document
has one called `main` because it has to be called something. Which symbol is on
screen is editor state, `[:ui :open]`, not a fact about the document — the stage
draws it, the timeline lists it, the transport plays it and a new shape goes into
it. A document opens on the longest symbol nothing else places.
Selection lives in app-db under `:ui`, as `[:node <symbol> <node>]`,
`[:symbol <id>]` or `[:subject|:feature|:group <id>]` — four panes ask what is
selected, and a ratom private to one of them can only be shared by making the
other three require it.
**Drop a video on the media pool** and it uploads, extracts and goes straight on
into detection. Dragging a symbol out of the pool onto the stage places an
instance of it at the playhead.
The timeline's rows are the open symbol's nodes, front-most first, with a dot per
keyframe and a bar over the frames the node exists on; a dense channel is hatched
rather than ticked, because one value per frame is a solid block that says less
than the bar does. Opening a row shows its channels; opening an **instance** row
shows the symbol it places, with every frame number mapped back into the open
symbol's frame space — see the namespace docstring in `ui/timeline.cljs`, which
is where that mapping is argued.
### Paint sketch ### Paint sketch
Click **new polygon**, place at least three vertices on the stage, then click Pick a tone from the palette strip, click **polygon**, place at least three
**finish shape**. Select a shape to drag its vertices. Scrub to another frame and vertices on the stage, then click **finish**. Select a shape — on the stage, or by
click **new drawing key** to copy the visible outline there; the previous drawing its timeline row — to drag its vertices. Scrub to another frame and click
holds until that key. The numbered drawing-key buttons jump to editable keys. **drawing key here** in the inspector to copy the visible outline there; the
The transition control between two drawing keys can switch that gap between a previous drawing holds until that key. The numbered key buttons jump to editable
hold and linear vertex tweening. Other gaps keep their own timing. Tweening works keys. The transition control between two drawing keys can switch that gap between
a hold and linear vertex tweening. Other gaps keep their own timing. Tweening works
best when the same vertex best when the same vertex
keeps the same meaning in every drawing. Paint shapes use the timeline clock keeps the same meaning in every drawing. Paint shapes use the timeline clock
directly, so the roto exposure grid does not delay a drawing key or step its directly, so the roto exposure grid does not delay a drawing key or step its
@ -109,7 +157,7 @@ tween. Use the project **save** button to persist the drawings.
has used since step 5, when shadow-cljs's `:dev-http` did no directory-index has used since step 5, when shadow-cljs's `:dev-http` did no directory-index
resolution and the suite learned to ask for the file. resolution and the suite learned to ask for the file.
Four built-in clips, on buttons in the transport: Four built-in clips, under **built-in examples** in the open menu:
| | | | | |
| --- | --- | | --- | --- |
@ -125,14 +173,14 @@ The demo scene itself is `src/arthur/demo/scene.edn`. Both the synthetic take
and real footage use `src/arthur/flow/take.cljs` for the measurement order and and real footage use `src/arthur/flow/take.cljs` for the measurement order and
`src/arthur/flow/freeze.cljs` for the landmark-to-channel conversion. `src/arthur/flow/freeze.cljs` for the landmark-to-channel conversion.
**stage 8625** loads the locally saved `IMG_8625.MOV` project and places its **8625 stage study**, in the open menu, loads the locally saved `IMG_8625.MOV` project and places its
post-processed timeline twice. The stage layout is post-processed face symbol twice. The stage layout is
`src/arthur/demo/stage_8625.edn`: the right picture and sound start at frame 48, `src/arthur/demo/stage_8625.edn`: the right picture and sound start at frame 48,
and the two pictures overlap slightly in stage space. Audio has its own timeline and the two pictures overlap slightly in stage space. Audio has its own
nodes, linked to the picture instances but with independent spans and gain nodes, linked to the picture instances but with independent spans and gain
channels. The right sound swells and pans across the stage, then fades out at channels. The right sound swells and pans across the stage, then fades out at
frame 260 while its picture continues to frame 260 while its picture continues to
frame 280. The button needs that saved 8625 project in the local server database. frame 280. The row needs that saved 8625 project in the local server database.
### Projects and the EDN fixtures ### Projects and the EDN fixtures
@ -148,16 +196,17 @@ the same ClojureScript clip.
The intended editor creates and changes that in-memory clip directly: a project The intended editor creates and changes that in-memory clip directly: a project
browser and **new stage** action, timeline instance placement, node and channel browser and **new stage** action, timeline instance placement, node and channel
editors, then the existing save path. EDN remains useful for checked-in examples editors, then the existing save path. EDN remains useful for checked-in examples
and reproducible studies. The current UI has save and open, but no project and reproducible studies. The UI now has the blank-stage action (**new**), a
browser, blank-stage action, or authoring controls yet; open chooses the most project browser (`open ▾`), and placement by dragging a symbol out of the media
recent project. pool; node and channel editors are still to come — the inspector reports a
channel's shape but has nowhere to change its values.
### Real footage ### Real footage
Choose a video in the **footage** file input. The server probes it, re-encodes it Drop a video on the media pool, or use its **+** button. The server probes it, re-encodes it
to an H.264 proxy and a raw stream of the same coded frames, pulls WAV audio and one tracing JPEG per frame, to an H.264 proxy and a raw stream of the same coded frames, pulls WAV audio and one tracing JPEG per frame,
then makes the resulting footage selectable. Click **load frames** to detect and then makes the resulting footage selectable and runs detection on it. **roto**, in
freeze it. Extraction progress is currently read from `/api/extractions/<key>`; a the pool's header, does the same for footage that is already there. Extraction progress is currently read from `/api/extractions/<key>`; a
future WebSocket can push the same job state. The uploaded bytes, extraction job, future WebSocket can push the same job state. The uploaded bytes, extraction job,
and decoded footage have separate records, so the same uploaded video can be and decoded footage have separate records, so the same uploaded video can be
reopened without decoding it again. reopened without decoding it again.
@ -216,14 +265,14 @@ the root — all of it is extraction output, and tier 3 does not belong in the r
Loading detects one face per frame, measures the mouth, eyes and brows from Loading detects one face per frame, measures the mouth, eyes and brows from
landmarks and the teeth from source pixels, then freezes them into channels, landmarks and the teeth from source pixels, then freezes them into channels,
and adds a button for the footage clip. Detection happens once when you load; and opens the footage clip. Detection happens once when you load;
playback only resolves channels and paints. Frames without a detection remain playback only resolves channels and paints. Frames without a detection remain
marked absent even though their neighbouring poses are used to condition the marked absent even though their neighbouring poses are used to condition the
track. The scene now records stable subject and feature IDs and explicit eye track. The scene now records stable subject and feature IDs and explicit eye
pairs; dense channels can mark one feature absent while another is observed. pairs; dense channels can mark one feature absent while another is observed.
Current MediaPipe loading supplies only the full-face detection mask. The stage Current MediaPipe loading supplies only the full-face detection mask. The stage
stays 320×200 regardless of the footage dimensions. Real stays 320×200 regardless of the footage dimensions. Real
footage starts at the source picture rate. The **picture fps** buttons sample the footage starts at the source picture rate. The **picture** buttons in the inspector sample the
frozen roto at lower rates while the source track, duration and audio clock stay frozen roto at lower rates while the source track, duration and audio clock stay
unchanged. Picking frames to trace into cels is a separate future editing step. unchanged. Picking frames to trace into cels is a separate future editing step.
**save** also stores the detection mask, dense landmarks and raw RGBA mouth crops **save** also stores the detection mask, dense landmarks and raw RGBA mouth crops
@ -252,7 +301,7 @@ runs the old JS tool on 8777, and the two are meant to run side by side.
## Saving ## Saving
**save** and **open** in the transport. A save has three ordered stages: **new**, **open** and **save** in the top bar. A save has three ordered stages:
is the tier split: is the tier split:
1. the **analysis** record, so every block stored afterwards can name the detector 1. the **analysis** record, so every block stored afterwards can name the detector
@ -269,10 +318,11 @@ unchanged document says `0 leaves · 0 blocks`, which is both halves of the
addressing working at once — an unchanged leaf keeps its version, and a addressing working at once — an unchanged leaf keeps its version, and a
content-addressed block is already there. content-addressed block is already there.
Two things are deliberately visible as failures. Saving `swarm` is refused, Saving `swarm` is deliberately visible as a failure: its blocks have
because its blocks have hand-written names and a document may only name content hand-written names and a document may only name content addresses.
addresses. And **open** takes the most recently updated project and shows its first
clip: there is no project browser, and the store holds one clip at a time. `open ▾` lists every project the server holds, newest first, and shows the first
clip of whichever one is picked — the store holds one clip at a time.
## The oracle, which is finished ## The oracle, which is finished
@ -313,12 +363,12 @@ them is `clips/templates/clips/index.html`.
## Two evaluators, on purpose ## Two evaluators, on purpose
`domain/timeline` has both `eval-frame` and `resolver`, and they are not `domain/symbol` has both `eval-frame` and `resolver`, and they are not
alternatives: alternatives:
- **`(eval-frame timeline f store)`** is the specification. Allocating, order-free, - **`(eval-frame symbol f store)`** is the specification. Allocating, order-free,
obviously correct. Tests and one-off renders use it. obviously correct. Tests and one-off renders use it.
- **`(resolver timeline store)` -> `(fn [f] ops)`** is what playback uses. It caches - **`(resolver symbol store)` -> `(fn [f] ops)`** is what playback uses. It caches
the topological order and the z paths, holds a cursor per channel and reuses the topological order and the z paths, holds a cursor per channel and reuses
one point buffer per node, so a frame allocates the op maps and nothing else. one point buffer per node, so a frame allocates the op maps and nothing else.

View file

@ -9,6 +9,7 @@
"version": "0.0.1", "version": "0.0.1",
"dependencies": { "dependencies": {
"@mediapipe/tasks-vision": "1.0.1", "@mediapipe/tasks-vision": "1.0.1",
"polygon-clipping": "^0.15.7",
"react": "^18.3.1", "react": "^18.3.1",
"react-dom": "^18.3.1" "react-dom": "^18.3.1"
}, },
@ -1015,6 +1016,16 @@
"node": ">= 0.10" "node": ">= 0.10"
} }
}, },
"node_modules/polygon-clipping": {
"version": "0.15.7",
"resolved": "https://registry.npmjs.org/polygon-clipping/-/polygon-clipping-0.15.7.tgz",
"integrity": "sha512-nhfdr83ECBg6xtqOAJab1tbksbBAOMUltN60bU+llHVOL0e5Onm1WpAXXWXVB39L8AJFssoIhEVuy/S90MmotA==",
"license": "MIT",
"dependencies": {
"robust-predicates": "^3.0.2",
"splaytree": "^3.1.0"
}
},
"node_modules/possible-typed-array-names": { "node_modules/possible-typed-array-names": {
"version": "1.1.0", "version": "1.1.0",
"resolved": "https://registry.npmjs.org/possible-typed-array-names/-/possible-typed-array-names-1.1.0.tgz", "resolved": "https://registry.npmjs.org/possible-typed-array-names/-/possible-typed-array-names-1.1.0.tgz",
@ -1216,6 +1227,12 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/robust-predicates": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/robust-predicates/-/robust-predicates-3.0.3.tgz",
"integrity": "sha512-NS3levdsRIUOmiJ8FZWCP7LG3QpJyrs/TE0Zpf1yvZu8cAJJ6QMW92H1c7kWpdIHo8RvmLxN/o2JXTKHp74lUA==",
"license": "Unlicense"
},
"node_modules/safe-buffer": { "node_modules/safe-buffer": {
"version": "5.2.1", "version": "5.2.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
@ -1416,6 +1433,15 @@
"source-map": "^0.5.6" "source-map": "^0.5.6"
} }
}, },
"node_modules/splaytree": {
"version": "3.2.3",
"resolved": "https://registry.npmjs.org/splaytree/-/splaytree-3.2.3.tgz",
"integrity": "sha512-7OXrNWzy6CK+r7Ch9OLPBDTKfB6XlWHjX4P0RU5B3IgFuWPeYN0XtRtlexGRjgbQxpfaUve6jTAwBGWuGntz/w==",
"license": "MIT",
"engines": {
"node": ">=18.20 || >=20"
}
},
"node_modules/stream-browserify": { "node_modules/stream-browserify": {
"version": "2.0.2", "version": "2.0.2",
"resolved": "https://registry.npmjs.org/stream-browserify/-/stream-browserify-2.0.2.tgz", "resolved": "https://registry.npmjs.org/stream-browserify/-/stream-browserify-2.0.2.tgz",

View file

@ -10,6 +10,7 @@
}, },
"dependencies": { "dependencies": {
"@mediapipe/tasks-vision": "1.0.1", "@mediapipe/tasks-vision": "1.0.1",
"polygon-clipping": "^0.15.7",
"react": "^18.3.1", "react": "^18.3.1",
"react-dom": "^18.3.1" "react-dom": "^18.3.1"
}, },

View file

@ -41,6 +41,7 @@
{:app {:target :browser {:app {:target :browser
:output-dir "../static/arthur/js" :output-dir "../static/arthur/js"
:asset-path "/static/arthur/js" :asset-path "/static/arthur/js"
:compiler-options {:source-map true}
:modules {:main {:init-fn arthur.core/init}}} :modules {:main {:init-fn arthur.core/init}}}
:test {:target :node-test :test {:target :node-test

View file

@ -2,16 +2,22 @@
"Render independently placed audio tracks into one stage audio clock. "Render independently placed audio tracks into one stage audio clock.
The mix is derived from saved audio track leaves and immutable footage blobs. The mix is derived from saved audio track leaves and immutable footage blobs.
The transport still has one audio element, so seeking, rate changes and looping The transport still has ONE clock, so seeking, rate changes and looping stay
stay tied to the same clock the picture reads. tied to the same position the picture is drawn from.
THE AUDIO BUFFER IS THE PRODUCT AND THE WAV IS ONE PACKAGING OF IT. Playback THE AUDIO BUFFER IS THE PRODUCT AND THE WAV IS ONE PACKAGING OF IT. `buffer!`
wants a URL an `<audio>` element can hold; an export wants the samples, either renders and the wrappers below it package, rather than the render being spelled
as WAV bytes to put in an archive or as the `AudioBuffer` a muxer takes as an once per consumer.
audio track. So `buffer!` renders and the two wrappers below it package, rather
than the render being spelled once per consumer." PLAYBACK NO LONGER PACKAGES AT ALL. It takes the buffer as it is — see
`clock-source!` and `arthur.clock.graph` — because encoding a WAV so an
`<audio>` element had a URL to hold cost O(the clip's length) on the main
thread, on open, on every tab switch and on every edit to a track. The WAV is
now what an EXPORT wants: bytes for an archive, or the buffer itself for a
muxer. `clock!` below is the element backend's packaging and goes when it does."
(:require [arthur.domain.channel :as ch] (:require [arthur.domain.channel :as ch]
[arthur.domain.clip :as clip] [arthur.domain.clip :as clip]
[arthur.domain.nest :as nest]
[arthur.domain.node :as node])) [arthur.domain.node :as node]))
(defn wav-bytes (defn wav-bytes
@ -28,9 +34,22 @@
bytes (js/ArrayBuffer. (+ 44 (* frames channels 2))) bytes (js/ArrayBuffer. (+ 44 (* frames channels 2)))
view (js/DataView. bytes) view (js/DataView. bytes)
samples (mapv #(.getChannelData buffer %) (range channels)) samples (mapv #(.getChannelData buffer %) (range channels))
peak (reduce max 0 ;; A HAND-WRITTEN LOOP over the typed arrays, not `(reduce max (for ...))`.
(for [channel samples i (range frames)] ;; The lazy sequence that read beautifully allocated one boxed double per
(js/Math.abs (aget channel i)))) ;; SAMPLE — ten million of them for a three-minute mix — and spent the
;; whole of a sixteen-second project open walking them and collecting
;; them. Same arithmetic, no allocation.
peak (loop [c 0 p 0]
(if (< c channels)
(recur (inc c)
(let [^js data (nth samples c)]
(loop [i 0 p p]
(if (< i frames)
(recur (inc i)
(let [a (js/Math.abs (aget data i))]
(if (> a p) a p)))
p))))
p))
level (if (> peak 0.98) (/ 0.98 peak) 1)] level (if (> peak 0.98) (/ 0.98 peak) 1)]
(doseq [[offset word] [[0 "RIFF"] [8 "WAVE"] [12 "fmt "] [36 "data"]]] (doseq [[offset word] [[0 "RIFF"] [8 "WAVE"] [12 "fmt "] [36 "data"]]]
(dotimes [i 4] (.setUint8 view (+ offset i) (.charCodeAt word i)))) (dotimes [i 4] (.setUint8 view (+ offset i) (.charCodeAt word i))))
@ -43,45 +62,62 @@
(.setUint16 view 32 (* channels 2) true) (.setUint16 view 32 (* channels 2) true)
(.setUint16 view 34 16 true) (.setUint16 view 34 16 true)
(.setUint32 view 40 (* frames channels 2) true) (.setUint32 view 40 (* frames channels 2) true)
(dotimes [i frames] ;; Channel-outer so the channel's array is looked up once rather than once
;; per frame. The byte offsets are unchanged, so the interleaving is too.
(dotimes [c channels] (dotimes [c channels]
(let [sample (* level (aget (get samples c) i))] (let [^js data (nth samples c)]
(dotimes [i frames]
(let [sample (* level (aget data i))]
(.setInt16 view (+ 44 (* (+ (* i channels) c) 2)) (.setInt16 view (+ 44 (* (+ (* i channels) c) 2))
(js/Math.round (* 32767 (max -1 (min 1 sample)))) true)))) (js/Math.round (* 32767 (max -1 (min 1 sample)))) true)))))
(js/Uint8Array. bytes))) (js/Uint8Array. bytes)))
(defn- wav-url [^js buffer] (defn- wav-url [^js buffer]
(js/URL.createObjectURL (js/URL.createObjectURL
(js/Blob. #js [(wav-bytes buffer)] #js {:type "audio/wav"}))) (js/Blob. #js [(wav-bytes buffer)] #js {:type "audio/wav"})))
(defn- source! [footage-id] (defn- fetch-ok! [url what]
(-> (js/fetch (str "/api/footage/" footage-id)) (-> (js/fetch url)
(.then (fn [response] (.then (fn [response]
(when-not (.-ok response) (when-not (.-ok response)
(throw (ex-info "audio track's footage is missing" (throw (ex-info (str "audio track's " what " is missing")
{:footage footage-id :status (.-status response)}))) {:url url :status (.-status response)})))
(.json response))) response))))
(defn- decode-bytes! [bytes]
(.decodeAudioData (js/OfflineAudioContext. 1 1 44100) bytes))
(defn- source!
"Promise of `[source {:buffer :fps}]` for an audio node's `:source`. Footage
counts its frames at its own rate; a sound file has no frames of its own, so
its `:fps` is nil and it counts in the document's."
[{:keys [footage sound] :as source}]
(if sound
(-> (fetch-ok! (str "/api/sounds/" sound) "sound")
(.then #(.json %))
(.then #(fetch-ok! (.-audio %) "blob"))
(.then #(.arrayBuffer %))
(.then decode-bytes!)
(.then (fn [buffer] [source {:buffer buffer}])))
(-> (fetch-ok! (str "/api/footage/" footage) "footage")
(.then #(.json %))
(.then (fn [^js manifest] (.then (fn [^js manifest]
(-> (js/fetch (.-audio manifest)) (-> (fetch-ok! (.-audio manifest) "blob")
(.then (fn [response] (.then #(.arrayBuffer %))
(when-not (.-ok response) (.then decode-bytes!)
(throw (ex-info "audio track's blob is missing" (.then (fn [buffer] [source {:buffer buffer :fps (.-fps manifest)}]))))))))
{:footage footage-id :status (.-status response)})))
(.arrayBuffer response)))
(.then (fn [bytes]
(let [decoder (js/OfflineAudioContext. 1 1 44100)]
(-> (.decodeAudioData decoder bytes)
(.then (fn [buffer]
[footage-id {:buffer buffer
:fps (.-fps manifest)}])))))))))))
(defn- automate! [^js param channel start end fps factor default store] (defn- automate! [^js param channel start end fps factor default store]
(let [channel (or channel (ch/framed default))] (let [channel (or channel (ch/framed default))
(.setValueAtTime param (* factor (ch/value-at channel start store)) (/ start fps)) sample (fn [f] (ch/value-at channel
(if-let [{:keys [at rate]} (:sample-time channel)]
(js/Math.floor (* rate (- f at))) f)
store))]
(.setValueAtTime param (* factor (sample start)) (/ start fps))
(cond (cond
(:dense channel) (:dense channel)
(doseq [f (range (inc start) end)] (doseq [f (range (inc start) end)]
(.setValueAtTime param (* factor (ch/value-at channel f store)) (/ f fps))) (.setValueAtTime param (* factor (sample f)) (/ f fps)))
(:animated? channel) (:animated? channel)
(doseq [[f v] (sort-by key (:keys channel)) (doseq [[f v] (sort-by key (:keys channel))
@ -91,25 +127,22 @@
(.setValueAtTime param (* factor v) (/ f fps))))))) (.setValueAtTime param (* factor v) (/ f fps)))))))
(defn tracks-of (defn tracks-of
"The audio nodes of one of the clip's timelines. "The sounds symbol `sid` plays, including those inside what it places — see
`nest/audio-tracks`. Playback mixes the open symbol's."
[document sid]
(nest/audio-tracks document sid))
A timeline parameter rather than always the root, because a symbol is a (defn- render! [document sid sources store]
timeline and may carry its own sound. `:main` is the clip's own, which is what
playback mixes."
[document tid]
(filter #(= :audio (:kind %)) (vals (:nodes (clip/timeline document tid)))))
(defn- render! [document tid sources store]
(let [fps (:fps document) (let [fps (:fps document)
frames (:frames (clip/timeline document tid)) frames (clip/output-frames document sid)
tracks (tracks-of document tid) tracks (tracks-of document sid)
output (js/OfflineAudioContext. output (js/OfflineAudioContext.
2 (js/Math.ceil (* (/ frames fps) 44100)) 44100)] 2 (js/Math.ceil (* (/ frames fps) 44100)) 44100)]
(doseq [track tracks] (doseq [track tracks]
(let [[start end] (or (:span track) [0 frames]) (let [[start end] (or (node/placed-span track) [0 frames])
start (max 0 start) start (max 0 start)
end (min frames end) end (min frames end)
{:keys [buffer fps]} (get sources (get-in track [:source :footage])) {:keys [buffer fps] :or {fps (:fps track)}} (get sources (:source track))
sound (.createBufferSource output) sound (.createBufferSource output)
gain (.createGain output) gain (.createGain output)
pan (.createStereoPanner output)] pan (.createStereoPanner output)]
@ -118,7 +151,7 @@
(set! (.-loop sound) (boolean (get-in track [:time :loop?]))) (set! (.-loop sound) (boolean (get-in track [:time :loop?])))
(automate! (.-playbackRate sound) (automate! (.-playbackRate sound)
(get-in track [:channels [:audio :rate]]) (get-in track [:channels [:audio :rate]])
start end (:fps document) (or (get-in track [:time :rate]) 1) 1 store) start end (:fps document) (* (or (get-in track [:time :rate]) 1) (/ (:fps document) fps)) 1 store)
(automate! (.-gain gain) (automate! (.-gain gain)
(get-in track [:channels [:audio :gain]]) (get-in track [:channels [:audio :gain]])
start end (:fps document) 1 1 store) start end (:fps document) 1 1 store)
@ -133,20 +166,19 @@
(.startRendering output))) (.startRendering output)))
(defn buffer! (defn buffer!
"Promise of the `AudioBuffer` one timeline's audio tracks mix down to, or nil "Promise of the `AudioBuffer` one symbol's audio tracks mix down to, or nil
when it has none. when it has none.
The raw product. `mix!` packages it as a WAV URL for the transport and The raw product. `mix!` packages it as a WAV URL for the transport and
`export/frames` packages it as WAV bytes in an archive; a muxer would take it as `export/frames` packages it as WAV bytes in an archive; a muxer would take it as
it is, which is why this is the function the others are written in terms of." it is, which is why this is the function the others are written in terms of."
([document tid] (buffer! document tid nil)) [document sid store]
([document tid store] (let [tracks (tracks-of document sid)]
(let [tracks (tracks-of document tid)]
(if (empty? tracks) (if (empty? tracks)
(js/Promise.resolve nil) (js/Promise.resolve nil)
(-> (js/Promise.all (-> (js/Promise.all
(into-array (map source! (distinct (map #(get-in % [:source :footage]) tracks))))) (into-array (map source! (distinct (map :source tracks)))))
(.then (fn [pairs] (render! document tid (into {} (array-seq pairs)) store)))))))) (.then (fn [pairs] (render! document sid (into {} (array-seq pairs)) store)))))))
(defn decode! (defn decode!
"Promise of the `AudioBuffer` behind a URL. What a clip whose audio is a plain "Promise of the `AudioBuffer` behind a URL. What a clip whose audio is a plain
@ -158,13 +190,73 @@
(throw (ex-info "the clip's audio did not load" (throw (ex-info "the clip's audio did not load"
{:url url :status (.-status response)}))) {:url url :status (.-status response)})))
(.arrayBuffer response))) (.arrayBuffer response)))
(.then (fn [bytes] (.then decode-bytes!)))
(.decodeAudioData (js/OfflineAudioContext. 1 1 44100) bytes)))))
(defn mix! (defn fit-buffer
"Promise of a mixed WAV URL, or the original URL for a clip without audio "Fit fallback audio to the open timeline, padding with silence or trimming.
tracks. Each track can be trimmed and faded independently of its linked picture." The buffer and clock must share a duration so looping wraps at the timeline's
([document fallback-url] (mix! document fallback-url nil)) end rather than repeating a short soundtrack underneath a longer animation."
([document fallback-url store] [^js buffer seconds]
(-> (buffer! document clip/root-id store) (let [rate (.-sampleRate buffer)
(.then (fn [buffer] (if buffer (wav-url buffer) fallback-url)))))) frames (max 1 (js/Math.ceil (* seconds rate)))]
(if (= frames (.-length buffer))
buffer
(let [channels (.-numberOfChannels buffer)
fitted (.createBuffer (js/OfflineAudioContext. channels 1 rate)
channels frames rate)]
(dotimes [c channels]
(.set (.getChannelData fitted c)
(.subarray (.getChannelData buffer c) 0 (min frames (.-length buffer)))))
fitted))))
(defn clock-source!
"Promise of `{:buffer :seconds}` — the audio the transport runs its clock on
while symbol `sid` is open, and how long that clock is.
Same order of preference as the WAV packaging in `clock!` below: the symbol's
own placed tracks, mixed; the document's audio file, for the symbol the
document opens on and only that one; and otherwise NO BUFFER AT ALL and the
symbol's own length.
THE SILENT CASE IS WHY THE DURATION IS RETURNED BESIDE THE BUFFER rather than
read off it. A symbol with no sound still needs a clock exactly as long as it
is, and `clock!` had to synthesize that silence and then ENCODE it, full
length, so an element had a duration to report. There is nothing to decode for
a symbol with no sound: saying how long it is answers the only question the
silence was ever asked."
[document sid fallback-url store]
(-> (buffer! document sid store)
(.then (fn [^js buffer]
(cond
buffer
{:buffer buffer :seconds (.-duration buffer)}
(and fallback-url (= sid (clip/opens-on document)))
(-> (decode! fallback-url)
(.then (fn [b]
(let [seconds (/ (clip/output-frames document sid) (:fps document))]
{:buffer (fit-buffer b seconds) :seconds seconds}))))
:else
{:buffer nil
:seconds (/ (clip/output-frames document sid) (:fps document))})))))
(defn clock!
"Promise of the URL the transport should play while symbol `sid` is open.
THE ELEMENT BACKEND'S PACKAGING of `clock-source!`, kept while that backend is
— see `arthur.clock`. Every cost this namespace had on open is in the two
`wav-url` calls below.
The frame is derived from the audio element and from nothing else, so every
open symbol needs a sound exactly as long as it is. In order: its own placed
tracks, mixed; the document's audio file, for the symbol the document opens on
and only that one; and otherwise SILENCE of the symbol's length — a ten-frame
symbol played against the whole take's soundtrack would run ten frames and then
keep the clock going for minutes."
[document sid fallback-url store]
(-> (clock-source! document sid fallback-url store)
(.then (fn [{:keys [buffer seconds]}]
(wav-url (or buffer
(.createBuffer (js/OfflineAudioContext. 1 1 44100)
1 (max 1 (js/Math.ceil (* 44100 seconds))) 44100)))))))

View file

@ -3,7 +3,7 @@
THE FRAME IS DERIVED FROM THE AUDIO, never counted: THE FRAME IS DERIVED FROM THE AUDIO, never counted:
frame = ⌊currentTime · fps⌋ frame = ⌊position · fps⌋
A loop that counted frames and hoped to keep up would drift, and drift against A loop that counted frames and hoped to keep up would drift, and drift against
a voice is the one artefact that cannot be fixed downstream — a lip-sync tool a voice is the one artefact that cannot be fixed downstream — a lip-sync tool
@ -12,26 +12,79 @@
The failure mode becomes a visible stutter rather than an invisible slide, and The failure mode becomes a visible stutter rather than an invisible slide, and
those are very different bugs to own. those are very different bugs to own.
½× and ¼× are `playbackRate` and nothing else. The audio slows, `currentTime` ½× and ¼× are the backend's playback rate and nothing else. The audio slows,
advances proportionally, and the derived frame follows — so slow motion cannot the position advances proportionally, and the derived frame follows — so slow
desync by construction. Implementing rate as a multiplier on a counted frame motion cannot desync by construction. Implementing rate as a multiplier on a
would give the picture a rate and the sound another. counted frame would give the picture a rate and the sound another.
It is outside app-db because the audio element is the source of truth and It is outside app-db because the audio is the source of truth and copying it
copying it into the db every frame would make the db a lagging mirror of into the db every frame would make the db a lagging mirror of something
something authoritative elsewhere. What DOES belong in the db is the playhead authoritative elsewhere. What DOES belong in the db is the playhead as a piece
as a piece of document state — see events/playback — and that is written from of document state — see events/playback — and that is written from here, not
here, not read by here." read by here.
(:require [arthur.domain.node :as node]))
(defonce ^:private el (atom nil)) TWO BACKENDS, ONE ARITHMETIC. `position` comes from either a Web Audio graph
(`clock.graph`, the default) or an `<audio>` element (`clock.element`, kept
switchable while the first earns trust). Everything below the position — the
derivation, the clamp, the exposure grid — is here and is the same either way,
so the two can be compared on the same take rather than swapped on faith.
Build with `:closure-defines {arthur.clock/BACKEND \"element\"}`, or call
`use-backend!` from the console, to put the element back."
(:require [arthur.clock.element :as element]
[arthur.clock.graph :as graph]
[arthur.clock.transport :as t]
[arthur.domain.node :as node]))
(goog-define ^String BACKEND "graph")
(defonce ^:private mode (atom (keyword BACKEND)))
;; `{:source x :backend b}`. The source is kept so that re-attaching the same
;; thing can be recognised as the no-op it is — see `install!`.
(defonce ^:private current (atom nil))
(defn graph?
"Whether the app should wire up the graph backend. Read by `ui/shell`, which
renders the audio element only when this is false, and by `events/playback`,
which fetches a buffer rather than a WAV URL when it is true."
[]
(and (= :graph @mode) (graph/available?)))
(defn use-backend!
"Switch backends. Takes effect on the next thing that attaches one, which is
the next tab switch or document open — nothing is torn down under a take
that is already playing."
[m]
(reset! mode m))
(defn- install!
"Put a backend on `source`, unless `source` is already the clock's.
IDEMPOTENCE IS LOAD-BEARING HERE. The element's `:ref` is an inline closure,
so React hands it the same node again on every re-render of the shell —
rebuilding the clock there would mean a pane being dragged released whatever
was playing. The source is the identity: the element, or the buffer and its
length."
[source make]
(let [{:keys [backend] prev :source} @current]
(when-not (and backend (= prev source))
(when backend (t/-release! backend))
(reset! current {:source source
:backend (when (some? source) (make))}))))
(defn attach! (defn attach!
"Hand the clock its audio element. Idempotent." "Hand the clock an audio element. Idempotent. Element backend only — the
`:ref` that calls this is on a node `ui/shell` renders only in that mode."
[audio-el] [audio-el]
(reset! el audio-el)) (install! audio-el #(element/backend audio-el)))
(defn element [] @el) (defn attach-buffer!
"Hand the clock `seconds` of audio to run on, as an `AudioBuffer` or as nil
for silence of that length. Graph backend only."
[buffer seconds]
;; Keyed on the length as well as the buffer, because two silent clocks of
;; different lengths are two different clocks and both have a nil buffer.
(install! [buffer seconds] #(graph/backend buffer seconds)))
(defn- clamp [f frames] (defn- clamp [f frames]
(-> f (max 0) (min (dec frames)))) (-> f (max 0) (min (dec frames))))
@ -39,56 +92,52 @@
(defn frame (defn frame
"The clip frame the audio is currently on." "The clip frame the audio is currently on."
[fps frames] [fps frames]
(if-let [a @el] (if-let [b (:backend @current)]
(clamp (js/Math.floor (* (.-currentTime a) fps)) frames) (clamp (js/Math.floor (* (t/-position b) fps)) frames)
0)) 0))
(defn playing? [] (defn playing? []
(boolean (when-let [a @el] (and (not (.-paused a)) (not (.-ended a)))))) (boolean (when-let [b (:backend @current)] (t/-playing? b))))
(defn rate [] (defn rate []
(if-let [a @el] (.-playbackRate a) 1.0)) (if-let [b (:backend @current)] (t/-rate b) 1.0))
(defn set-rate! [r] (defn set-rate! [r]
(when-let [a @el] (set! (.-playbackRate a) r))) (when-let [b (:backend @current)] (t/-set-rate! b r)))
(defn play! [] (defn play! []
(when-let [a @el] (when-let [b (:backend @current)] (t/-play! b)))
;; Returns a promise that rejects if the browser has not seen a gesture yet.
;; Swallowed: the transport button IS the gesture, so this can only fire on a
;; programmatic play, where a console error is noise rather than news.
(some-> (.play a) (.catch (fn [_])))))
(defn pause! [] (defn pause! []
(when-let [a @el] (.pause a))) (when-let [b (:backend @current)] (t/-pause! b)))
(defn seek! (defn seek!
"Put the audio at the start of frame f. Seeking to the frame's start rather "Put the audio at the start of frame f. Seeking to the frame's start rather
than its middle keeps `frame` idempotent: seek to f, read back f." than its middle keeps `frame` idempotent: seek to f, read back f."
[fps frames f] [fps frames f]
(when-let [a @el] (when-let [b (:backend @current)]
(set! (.-currentTime a) (/ (clamp f frames) fps)))) (t/-seek! b (/ (clamp f frames) fps))))
(defn set-loop! (defn set-loop!
"Wrap at the end instead of stopping. The frame stays derived — `currentTime` "Wrap at the end instead of stopping. The frame stays derived — the position
simply returns to zero — so nothing about the sync changes, which is the point simply returns to zero — so nothing about the sync changes, which is the point
of not counting frames. of not counting frames.
It earns its place at 2x and 4x, where the whole clip is gone in under four It earns its place at 2x and 4x, where the whole clip is gone in under four
seconds and a profile wants more than that to look at." seconds and a profile wants more than that to look at."
[on?] [on?]
(when-let [a @el] (set! (.-loop a) (boolean on?)))) (when-let [b (:backend @current)] (t/-set-loop! b on?)))
(defn set-muted! [on?] (defn set-muted! [on?]
(when-let [a @el] (set! (.-muted a) (boolean on?)))) (when-let [b (:backend @current)] (t/-set-muted! b on?)))
(defn duration-frames (defn duration-frames
"How many frames the audio actually covers, which need not be the clip's "How many frames the audio actually covers, which need not be the clip's
length. Reported rather than assumed: a clip longer than its audio is a length. Reported rather than assumed: a clip longer than its audio is a
legitimate thing to be told about, not a thing to silently truncate." legitimate thing to be told about, not a thing to silently truncate."
[fps] [fps]
(when-let [a @el] (when-let [b (:backend @current)]
(let [d (.-duration a)] (let [d (t/-duration b)]
(when (and d (js/isFinite d)) (js/Math.ceil (* d fps)))))) (when (and d (js/isFinite d)) (js/Math.ceil (* d fps))))))
(defn exposed-frame (defn exposed-frame
@ -97,8 +146,3 @@
rather than only inside the scene." rather than only inside the scene."
[f expose] [f expose]
(node/expose f expose)) (node/expose f expose))
(defn picture-frame
"The source pose displayed at f after picture-rate sampling and exposure."
[f source-fps picture-fps expose]
(node/expose (node/sample-frame f source-fps picture-fps) expose))

View file

@ -0,0 +1,45 @@
(ns arthur.clock.element
"The clock on an `<audio>` element. The original backend, kept switchable.
Reads `currentTime` and takes it as the position. That is the whole of it, and
it is why this backend needs a WAV: an element holds a URL, so a mixdown has
to be encoded and blobbed before it can be played — see `audio/mix`'s
`clock!`. The encode is O(the clip's length) and runs on the main thread, so a
long take costs seconds of frozen UI on open, on every tab switch and on every
edit to a track. `arthur.clock.graph` exists to not do that.
What this backend still has that the graph one has to be told: the OS media
keys and the media session, which the element gets from the browser for free.
KEPT so the two can be compared on the same document rather than swapped on
faith. When the graph backend has been trusted for a while, this namespace and
`mix/clock!`'s WAV packaging go together."
(:require [arthur.clock.transport :as t]))
(deftype Element [^js el]
t/Transport
(-position [_] (.-currentTime el))
(-duration [_] (.-duration el))
(-playing? [_] (and (not (.-paused el)) (not (.-ended el))))
(-rate [_] (.-playbackRate el))
(-set-rate! [_ r] (set! (.-playbackRate el) r))
(-play! [_]
;; Returns a promise that rejects if the browser has not seen a gesture yet.
;; Swallowed: the transport button IS the gesture, so this can only fire on a
;; programmatic play, where a console error is noise rather than news.
(some-> (.play el) (.catch (fn [_]))))
(-pause! [_] (.pause el))
(-seek! [_ seconds] (set! (.-currentTime el) seconds))
(-set-loop! [_ on?] (set! (.-loop el) (boolean on?)))
(-set-muted! [_ on?] (set! (.-muted el) (boolean on?)))
(-release! [_]
;; Nothing. React owns the element and unmounting it is what stops it; this
;; backend is a few property accesses wrapped in a type and holds no more
;; than that.
nil))
(defn backend
"Wrap an audio element — or anything that answers the same five properties,
which is what `clock-test`'s fake is."
[el]
(->Element el))

View file

@ -0,0 +1,266 @@
(ns arthur.clock.graph
"The clock on a Web Audio graph. No encode, no blob, no element.
THE BUFFER IS PLAYED, NOT PACKAGED. `audio/mix`'s `buffer!` already renders
the mixdown; the element backend then spent O(the clip's length) encoding that
buffer to a WAV purely so a `src` attribute had something to point at. An
`AudioBufferSourceNode` takes the buffer as it is, so opening a document costs
one node instead of a hundred megabytes of 16-bit PCM.
THE FRAME IS STILL DERIVED, AND FROM A BETTER CLOCK. `AudioContext.currentTime`
is the audio device's own position in double precision, advancing every 128
samples; an element's `currentTime` is whatever the media pipeline last
published and is permitted to lag. Nothing is counted here either — position is
an anchor plus elapsed context time, and the anchor is re-set on every seek and
every rate change, so no arithmetic accumulates across either.
AND IT IS LATENCY-COMPENSATED, which is the one thing this backend must get
right. `currentTime` is the time of the quantum being RENDERED, which the
speaker is `outputLatency` behind — tens of milliseconds, far more over
Bluetooth. Report the renderer's position and the picture leads the sound by
exactly that much, which in a lip-sync tool is the only artefact that matters.
So `scheduled` is the bookkeeping and `-position` is `scheduled` read one
latency in the past. The element has the same lag underneath; the difference is
that this one is a number we can subtract rather than an error we inherit.
A SYMBOL WITH NO SOUND GETS A CLOCK ANYWAY, with no buffer at all: `seconds`
is its length and the context's clock does the rest. The element backend had to
synthesize silence and encode THAT to a WAV, full length, so the element had a
duration to report — the clearest sign that the element was driving the design
rather than serving it."
(:require [arthur.clock.transport :as t]))
(defonce ^:private shared (atom nil))
(defn available?
"Whether this backend can run at all. False under node, where the tests live."
[]
(exists? js/AudioContext))
(defn context
"The app's one `AudioContext`, made on first use.
Lazy because constructing one before anything wants to play is how a browser
decides the page is trying to autoplay, and because `available?` is false in
the test runner."
[]
(or (:ctx @shared)
(let [ctx (js/AudioContext.)
;; ONE output gain for the app, not one per backend: a tab switch
;; replaces the backend, and a gain node per backend would leave the
;; old one wired to the destination. Mute lives here for the same
;; reason — it is a property of the transport, not of whichever
;; buffer happens to be loaded.
out (.createGain ctx)]
(.connect out (.-destination ctx))
(:ctx (reset! shared {:ctx ctx :out out})))))
(defn output [] (do (context) (:out @shared)))
(defn- latency
"How far ahead of the speaker `currentTime` is, in seconds.
`outputLatency` is the whole path and the number we want. Firefox reports 0
until the graph has actually run, so `baseLatency` stands in — it is only the
graph's own buffering and therefore an underestimate, which errs towards the
picture leading slightly rather than the correction overshooting. Neither
exists everywhere; 0 is then no worse than the element."
[^js ctx]
(let [out (.-outputLatency ctx)
base (.-baseLatency ctx)]
(cond
(and (number? out) (js/isFinite out) (pos? out)) out
(and (number? base) (js/isFinite base) (pos? base)) base
:else 0)))
(defn- seg-at
"The segment governing context time `t`."
[segs t]
(or (last (filter #(<= (:from %) t) segs)) (first segs)))
(defn- raw
"Where the audio is at context time `t`, in seconds into the buffer.
PIECEWISE, and that is the whole subtlety of this namespace. Audio already
rendered cannot be re-rated: when the transport goes 1x -> 4x, the samples
still travelling to the speaker were rendered at 1x, so reading them back at
4x jumps the playhead BACKWARDS by three output latencies — about fourteen
frames on a laptop, which is a visible lurch on every rate change and was
exactly what the first version of this did. So each `play`, `pause`, `seek`
and rate change records a segment, and a position is read against whichever
segment was in force when that audio was rendered.
Evaluated before the oldest segment we kept, it clamps. That is what makes a
seek read back exactly what was seeked to: a seek has nothing in flight worth
honouring — the user has jumped — so its segment starts the timeline over."
[segs t]
(let [t (max t (:from (first segs)))
s (seg-at segs t)]
(+ (:anchor s) (* (:rate s) (- t (:from s))))))
(defn- at-renderer
"Where the graph has rendered up to: `raw` at the context's own clock."
[{:keys [^js ctx segs]}]
(raw segs (.-currentTime ctx)))
(defn- at-speaker
"Where the sound being heard is: `raw` one output latency in the past. See the
namespace docstring — this is the whole of the compensation."
[{:keys [^js ctx segs]}]
(raw segs (- (.-currentTime ctx) (latency ctx))))
(defn- running?
"Whether the renderer is advancing. Read off the segments rather than kept
beside them, so there is one answer and not two that can disagree."
[{:keys [segs]}]
(pos? (:rate (last segs))))
(defn- prune
"Drop segments no position can still need: everything before the last one that
began at or before the in-flight window. Unbounded history would otherwise
grow by one entry per rate change for the life of the document."
[segs cutoff]
(let [n (count (take-while #(<= (:from %) cutoff) segs))]
(if (<= n 1) segs (subvec segs (dec n)))))
(defn- position
"`at-speaker`, brought inside the audio. Clamped before the wrap, so the few
milliseconds of negative position right after a looped play — the first
samples are still in the output buffer — read as 0 rather than as the end."
[{:keys [seconds loop?] :as st}]
(let [p (at-speaker st)]
(cond
(not (and seconds (pos? seconds))) 0
loop? (mod (max 0 p) seconds)
:else (-> p (max 0) (min seconds)))))
(defn- done?
"Run off the end. Judged at the SPEAKER, so playback is still reported as
running while the last scheduled samples are on their way out."
[{:keys [seconds loop?] :as st}]
(and (not loop?) seconds (pos? seconds) (>= (at-speaker st) seconds)))
(defn- spin-up!
"A fresh source node playing from where the renderer now is.
`AudioBufferSourceNode`s are single-use, so a play, a seek while playing and a
resumed pause each make a new one; they are cheap, which is the point of this
backend.
Nil when there is no buffer — the silent clock runs on the context alone."
[{:keys [^js ctx ^js buffer ^js out rate loop? seconds] :as st}]
(when buffer
(let [node (.createBufferSource ctx)]
(set! (.-buffer node) buffer)
(set! (.-loop node) (boolean loop?))
(set! (.-value (.-playbackRate node)) rate)
(.connect node out)
;; `when` 0 is "as soon as the graph can"; the offset is where in the
;; buffer to begin. Clamped because starting past the end is a range
;; error rather than a no-op in some engines.
(.start node 0 (-> (at-renderer st) (max 0) (min (or seconds 0))))
node)))
(defn- spin-down! [{:keys [^js node]}]
(when node
(try (.stop node) (catch :default _ nil))
(try (.disconnect node) (catch :default _ nil))))
(defn- restart!
"Begin a new timeline at `anchor`, running at `rate` or held when it is 0.
A RESET rather than a segment, for the three cases that have nothing in flight
worth honouring: a play starts fresh, a seek means the user has jumped, and a
pause wants one stable number for the readout rather than a position that
creeps forward as the output buffer drains."
[state anchor rate]
(let [st @state]
(spin-down! st)
(swap! state assoc
:segs [{:from (.-currentTime ^js (:ctx st)) :anchor anchor :rate rate}]
:node nil)
(when (pos? rate)
(swap! state assoc :node (spin-up! @state)))))
(deftype Graph [state]
t/Transport
(-position [_] (position @state))
(-duration [_] (:seconds @state))
(-playing? [_] (let [st @state] (boolean (and (running? st) (not (done? st))))))
(-rate [_] (:rate @state))
(-set-rate! [_ r]
;; A SEGMENT, not a reset — the one case where what is already in flight has
;; to keep its old rate or the playhead lurches. See `raw`.
(let [st @state
now (.-currentTime ^js (:ctx st))]
(if (running? st)
(let [anchor (at-renderer st)]
(swap! state #(-> %
(assoc :rate r)
(update :segs (fn [segs]
(prune (conj segs {:from now :anchor anchor :rate r})
(- now (latency ^js (:ctx st))))))))
(when-let [^js node (:node st)]
(set! (.-value (.-playbackRate node)) r)))
;; Stopped: nothing is in flight and nothing is rendering, so the rate
;; is simply what the next play will run at.
(swap! state assoc :rate r))))
(-play! [_]
(let [st @state]
;; `done?` counts as not playing. It is DERIVED — no flag is cleared when
;; the take runs off its end — so testing "running" alone would make play
;; a dead button from the moment the audio finished.
(when (or (not (running? st)) (done? st))
;; The context starts suspended and only a gesture may resume it. The
;; transport button IS the gesture, so a rejection here means a
;; programmatic play and is noise rather than news — as on the element.
(some-> (.resume ^js (:ctx st)) (.catch (fn [_])))
;; Play at the end starts over, which is what the element does.
(restart! state (if (done? st) 0 (position st)) (:rate st)))))
(-pause! [_]
(let [st @state]
(when (running? st)
;; Anchored at what was HEARD, not at what was scheduled, so play after
;; pause resumes from where the sound stopped. It re-plays the last few
;; milliseconds rather than skipping them, which is the kinder of the
;; two roundings.
(restart! state (position st) 0))))
(-seek! [_ seconds]
(restart! state seconds (if (running? @state) (:rate @state) 0)))
(-set-loop! [_ on?]
(swap! state assoc :loop? (boolean on?))
;; The audio thread reads `loop` every quantum, so a live node picks this up
;; without being restarted — the same as setting `loop` on a playing element.
(when-let [^js node (:node @state)]
(set! (.-loop node) (boolean on?))))
(-set-muted! [_ on?]
(set! (.-value (.-gain ^js (:out @state))) (if on? 0 1)))
(-release! [_]
;; The source node is wired to the output the whole app shares, so a backend
;; dropped while playing would go on being heard under the one that replaced
;; it. Nothing else here needs releasing: the context and its gain outlive
;; every backend by design.
(let [st @state]
(spin-down! st)
(swap! state assoc
:segs [{:from (.-currentTime ^js (:ctx st))
:anchor (position st) :rate 0}]
:node nil))))
(defn backend
"A clock on `buffer`, `seconds` long. A nil buffer is a silent clock of that
length — see the namespace docstring.
The context and output are injected by the four-argument form so the whole of
this is assertable against a fake in node, where there is no Web Audio."
([buffer seconds] (backend (context) (output) buffer seconds))
([^js ctx ^js out buffer seconds]
(->Graph (atom {:ctx ctx :out out :buffer buffer :seconds seconds
:rate 1.0 :loop? false :node nil
:segs [{:from (.-currentTime ctx) :anchor 0 :rate 0}]}))))

View file

@ -0,0 +1,38 @@
(ns arthur.clock.transport
"What the clock needs of a thing that plays sound, and nothing more.
`arthur.clock` does the arithmetic — the frame derivation, the clamping, the
exposure grid — and a backend only has to answer where the sound has got to
and do as it is told. Keeping the protocol this thin is what makes the two
implementations comparable: if the graph backend and the element backend
disagree about a take's sync, the difference is in these nine methods and not
in anything derived from them.
POSITION IS WHERE THE SPEAKER IS, not where the renderer is. A backend that
schedules audio ahead of the output owes the difference back here — see
`arthur.clock.graph` — because the picture is drawn against this number and a
lip-sync tool that draws against the scheduler leads the sound it is matching.")
(defprotocol Transport
(-position [this]
"Seconds into the audio, as heard. Never negative, never past `-duration`,
and wrapped rather than clamped while looping.")
(-duration [this]
"Seconds of audio, or nil when the backend has not been told yet.")
(-playing? [this]
"Running AND not finished. A backend that has reached its end reports false
even if nothing told it to stop, because the end of the sound is the
authority on playback having stopped — see `ui/player`'s loop.")
(-rate [this])
(-set-rate! [this r])
(-play! [this])
(-pause! [this])
(-seek! [this seconds]
"Put the audio at `seconds`. Reading `-position` back must give the same
number, which is what makes a scrub idempotent.")
(-set-loop! [this on?])
(-set-muted! [this on?])
(-release! [this]
"Give up whatever this backend holds, because something else is about to be
the clock. NOT a pause: a backend that owns nothing has nothing to do here,
and the position it was last at is no longer anybody's business."))

View file

@ -4,14 +4,20 @@
port-plan step 3: the hand-written scene plays at 30fps against audio, scrubs, port-plan step 3: the hand-written scene plays at 30fps against audio, scrubs,
and runs at ½× and ¼×." and runs at ½× and ¼×."
(:require [arthur.db :as db] (:require [arthur.db :as db]
[arthur.events.collab :as collab]
[arthur.events.footage :as footage] [arthur.events.footage :as footage]
[arthur.events.history :as history]
[arthur.events.playback] [arthur.events.playback]
[arthur.events.paint] [arthur.events.paint]
[arthur.events.project] [arthur.events.project :as project]
[arthur.events.ui]
[arthur.subs.playback] [arthur.subs.playback]
[arthur.subs.render] [arthur.subs.render]
[arthur.subs.ui]
[arthur.ui.index :as index]
[arthur.ui.player :as player] [arthur.ui.player :as player]
[arthur.ui.shell :as shell] [arthur.ui.shell :as shell]
[arthur.ui.tools :as tools]
[re-frame.core :as rf] [re-frame.core :as rf]
[reagent.dom.client :as rdc])) [reagent.dom.client :as rdc]))
@ -24,14 +30,24 @@
;; the loop would otherwise sit on an unchanged frame number and never redraw. ;; the loop would otherwise sit on an unchanged frame number and never redraw.
(rf/clear-subscription-cache!) (rf/clear-subscription-cache!)
(player/refresh-subs!) (player/refresh-subs!)
(rdc/render @root [shell/view])) (rdc/render @root [:<> [shell/view] [index/view]]))
(defn init [] (defn init []
(rf/dispatch-sync [::init]) (rf/dispatch-sync [::init])
;; A blank document, before the first render. Synchronous for the same reason
;; `::init` is: the shell reads the clip's dimensions, and mounting against a
;; db that has no clip in it yet is a frame of nothing for no reason.
(rf/dispatch-sync [::project/new])
;; What the server already holds, asked for once. The list is small — a row per ;; What the server already holds, asked for once. The list is small — a row per
;; ingested take — and having it before the first click is what lets the footage ;; ingested take — and having it before the first click is what lets the footage
;; picker be a picker rather than a path to type. ;; picker be a picker rather than a path to type.
(rf/dispatch [::footage/refresh]) (rf/dispatch [::footage/refresh])
(rf/dispatch [::project/list-symbols])
;; After the blank document, so an address that names a project opens it over
;; the blank one, and the blank one is what a bad address leaves on screen.
(collab/start!)
(history/install-keys!)
(tools/install-keys!)
(reset! root (rdc/create-root (js/document.getElementById "app"))) (reset! root (rdc/create-root (js/document.getElementById "app")))
(mount) (mount)
(player/start!)) (player/start!))

View file

@ -20,10 +20,8 @@
Read OFF the clip rather than written again beside it: copying a number by hand Read OFF the clip rather than written again beside it: copying a number by hand
into this table is how it comes to disagree with the document it describes. into this table is how it comes to disagree with the document it describes.
`:frames` comes from the ROOT TIMELINE and `:fps` from the clip, which is the There is no `:frames` here, because a length belongs to a symbol and which
split `arthur.domain.clip` exists to make — a timeline is a frame space, a clip symbol is open is the editor's state — see `events/playback/frames`."
is a rate — and an earlier version of this docstring noted that they sat on one
map \"only because there is one clip per scene today\". They do not any more."
[label-key label clip store] [label-key label clip store]
(merge {:label label :clip clip :store store (merge {:label label :clip clip :store store
;; A static asset since step 9, and not the repo root's `audio.wav`. ;; A static asset since step 9, and not the repo root's `audio.wav`.
@ -31,9 +29,7 @@
;; serves by hash — and the synthetic take needs a sound of its own so ;; serves by hash — and the synthetic take needs a sound of its own so
;; that the clock has something to run against with no footage ingested. ;; that the clock has something to run against with no footage ingested.
:audio "/static/arthur/audio.wav" :audio "/static/arthur/audio.wav"
:cid (name label-key) :cid (name label-key)}
:display-fps (:fps clip)
:frames (domain-clip/frames clip)}
(select-keys clip [:fps :width :height]))) (select-keys clip [:fps :width :height])))
(def clips (def clips
@ -52,9 +48,29 @@
(defn clip-entry [id] (defn clip-entry [id]
(some-> (get-in clips [id :entry]) deref)) (some-> (get-in clips [id :entry]) deref))
(def tracing
"How tracing layers show on the stage: all of them or none, how strongly, and
which ones are switched off, as `[symbol-id node-id]` — the symbol a tracing
placement is in and its id, so a face's footage is one layer wherever the face
is placed.
THE EDITOR'S, NOT THE DOCUMENT'S. Showing a reference is a way of looking at
the stage, like solo and zoom: not an undo step, not sent to collaborators,
and it cannot reach an export. A layer is shown unless it is in `:hidden`, so
footage brought in shows without being found and switched on first — once
tracing itself is on, which it is not until asked for."
{:on? false :opacity 0.5 :hidden #{}})
(def default (def default
{;; --- the document --- {;; --- the document ---
:clip/current :take ;;
;; NOTHING IS LOADED. `core/init` dispatches `::project/new` before the first
;; render, so the app opens on a blank stage rather than on whichever built-in
;; scene happened to be convenient — the demos, the swarm and the two takes are
;; rows in the media pool like anything else, and reference material is not a
;; default. The values below are what a blank document is; they are replaced by
;; that dispatch and exist so this map is a valid db on its own.
:clip/current nil
:paint/revision 0 :paint/revision 0
:palette :arthur/default ; a NAME; the ramp itself is project data :palette :arthur/default ; a NAME; the ramp itself is project data
@ -64,19 +80,31 @@
;; footage's. That is what deleting `makeXform` buys — the framing became a ;; footage's. That is what deleting `makeXform` buys — the framing became a
;; transform on a node, so nothing downstream of the freeze knows the frame ;; transform on a node, so nothing downstream of the freeze knows the frame
;; size — and it is why ui/player no longer hardcodes 320x200. ;; size — and it is why ui/player no longer hardcodes 320x200.
:clip (select-keys (clip-entry :take) [:fps :frames :width :height :audio :display-fps]) :clip (let [c (domain-clip/blank)]
{:fps (:fps c)
:width (:width c) :height (:height c)
:audio nil})
;; Which ingested footage to detect, and what the last load said. The list ;; Which ingested footage to detect, and what the last load said. The list
;; comes from the server — tier 3 is the backend's since step 9 — so there is ;; comes from the server — tier 3 is the backend's since step 9 — so there is
;; no path to type any more. ;; no path to type any more.
:footage {:id nil :label nil :loading? false :status nil :footage {:id nil :label nil :loading? false :status nil
:available [] :chosen nil} :available [] :chosen nil :uploaded #{}}
;; Every symbol in every saved project, for the pool's all-assets folder. Rows
;; from `/api/symbols`, nothing loaded: a symbol from elsewhere is fetched when
;; it is dropped.
:assets {:symbols [] :palettes [] :loading? false}
;; The document's own identity on the server. `:seq` is the monotonic project ;; The document's own identity on the server. `:seq` is the monotonic project
;; version: a client that sees a delta with `seq > local + 1` refetches, which ;; version: a client that sees a delta with `seq > local + 1` refetches, which
;; is what will make staleness self-healing once there is a broadcast to miss. ;; is what will make staleness self-healing once there is a broadcast to miss.
:project {:id nil :cid nil :name nil :seq nil :busy? false :status nil} :project {:id nil :cid nil :name nil :seq nil :busy? false :status nil}
;; What the server holds, for the open menu. A list of rows and nothing more —
;; opening one fetches the document itself.
:projects {:items [] :loading? false}
;; --- transport --- ;; --- transport ---
;; ;;
;; The playhead is in app-db like everything else. An earlier draft of ;; The playhead is in app-db like everything else. An earlier draft of
@ -92,11 +120,12 @@
;; machinery that would share it. ;; machinery that would share it.
;; --- export --- ;; --- export ---
;; ;;
;; The REQUEST and its progress, never the frames. Which timeline to write and ;; The REQUEST and its progress, never the frames. Which symbol to write and
;; at what integer zoom is authored state like anything else; the megabytes the ;; at what integer zoom is authored state like anything else; the megabytes the
;; render produces are handed straight to a download and never enter the db. ;; render produces are handed straight to a download and never enter the db.
;; `:isolate` is the placement to render alone, or nil for the whole timeline. ;; `:isolate` is the placement to render alone, or nil for the whole symbol;
:export {:timeline :main :isolate nil :zoom 4 :busy? false :done 0 :total 0 ;; `:symbol` nil means whichever symbol is open.
:export {:symbol nil :isolate nil :zoom 4 :busy? false :done 0 :total 0
:status nil} :status nil}
:playback {:frame 0 :playback {:frame 0
@ -105,7 +134,52 @@
;; Both for profiling: loop so a run at 4x lasts longer than the ;; Both for profiling: loop so a run at 4x lasts longer than the
;; clip, mute so sitting in one does not require enduring it. ;; clip, mute so sitting in one does not require enduring it.
:loop? false :loop? false
:muted? false}}) :muted? false}
;; --- the editor's own state ---
;;
;; IN app-db, not in ratoms beside the components that read it. What is
;; selected is asked by four panes at once — the params pane renders it, the
;; timeline highlights its row, the stage draws its handles, the palette says
;; which tone a new shape gets — and a `defonce` atom private to one namespace
;; can only be shared by making the other three require that namespace for its
;; state. It is also small and authored, which is the bar `arthur.db` sets.
;;
;; `:selection` is a vector whose first element says what kind of thing it
;; names, so a pane dispatches on it rather than on which of several
;; "selected-x" keys happens to be non-nil:
;;
;; [:node <symbol> <node>] a shape or an instance
;; [:symbol <id>] a symbol
;; [:subject <id>] [:feature <id>] [:group <id>] a tracked object
;;
;; `:draft` is the polygon being clicked out, flat [x y x y …] as geometry is
;; stored everywhere. `:expanded` holds timeline row PATHS — a path and not a
;; node id, because one symbol placed twice is two rows that open separately.
;;
;; `:open` is the symbol on screen — the one the stage draws, the timeline
;; lists, the transport plays and a new shape goes into — and `:tabs` the
;; symbols open beside it. Editor state and not the document's, because no
;; symbol is special to the document: which one you are looking at is a fact
;; about you.
;;
;; `:knobs` holds a generated setting's value WHILE THE REGENERATION IS IN
;; FLIGHT, keyed by [scope id knob]. Moving a slider dispatches a preview that
;; re-freezes blocks asynchronously, so until it lands the clip still reports
;; the old value — and a slider reading from the clip would spring back under
;; the user's finger on every frame of the drag.
:ui {:open nil
:tabs []
:selection nil
:selections []
:tone 1
:tool :select
:brush 6
:auto-key? false
:draft []
:knobs {}
:tracing tracing
:expanded #{}}})
(def rates (def rates
"The transport's rates — all of them `playbackRate` on the audio element, so "The transport's rates — all of them `playbackRate` on the audio element, so

View file

@ -6,7 +6,8 @@
validates would not be the one that renders, and the model would be validated validates would not be the one that renders, and the model would be validated
against a scene nobody ever looked at." against a scene nobody ever looked at."
(:require [arthur.domain.clip :as domain-clip] (:require [arthur.domain.clip :as domain-clip]
[arthur.domain.timeline :as timeline] [arthur.domain.palette :as pal]
[arthur.domain.symbol :as symbol]
[cljs.reader :as reader] [cljs.reader :as reader]
[shadow.resource :as rc])) [shadow.resource :as rc]))
@ -14,15 +15,15 @@
(def clip (reader/read-string source)) (def clip (reader/read-string source))
(def timeline (def main
"The clip's root timeline: what an evaluator takes. `clip` is the document." "The scene's one symbol: what an evaluator takes. `clip` is the document."
(domain-clip/root clip)) (domain-clip/symbol clip :main))
(def fps (:fps clip)) (def fps (:fps clip))
(def frames (domain-clip/frames clip)) (def frames (domain-clip/frames clip :main))
(defn ops-at (defn ops-at
"Draw ops for one frame, via the specification path. The page uses "Draw ops for one frame, via the specification path. The page uses
`timeline/resolver` instead; this is here for the REPL." `symbol/resolver` instead; this is here for the REPL."
[f] [f]
(timeline/eval-frame timeline f)) (symbol/eval-frame main f nil pal/index-of nil))

View file

@ -32,7 +32,7 @@
:width 320 :width 320
:height 200 :height 200
:timelines :symbols
{:main {:main
{:id :main {:id :main
:frames 229 :frames 229

View file

@ -6,8 +6,12 @@
(def layout (reader/read-string (rc/inline "arthur/demo/stage_8625.edn"))) (def layout (reader/read-string (rc/inline "arthur/demo/stage_8625.edn")))
(defn- position-track [center anchor drift phase frames] (defn- position-track
(let [base (mapv - center anchor) "Where the peg sits over time: its `center` plus a slow two-axis drift. The
peg's own position, so the stage point the face is pinned to is what moves —
not an offset that has to be kept in step with a changing scale."
[center drift phase frames]
(let [base center
[dx dy] drift [dx dy] drift
wave (fn [f period] (js/Math.sin (* 2 js/Math.PI (/ (+ f phase) period)))) wave (fn [f period] (js/Math.sin (* 2 js/Math.PI (/ (+ f phase) period))))
x0 (wave 0 96) x0 (wave 0 96)
@ -22,52 +26,95 @@
(defn compose (defn compose
"The authored layout plus a source clip -> the composed stage document. "The authored layout plus a source clip -> the composed stage document.
A PLACEMENT IS KEYED BY ITS :uuid, not by the authored id. The authored id A PEG'S IDENTITY IS AUTHORED TOO, beside its instance's, for the reason the
instance's is: `compose` is a pure function of the layout, so a generated one
would make the same stage a different document on every call.
AN INSTANCE IS KEYED BY ITS :uuid, not by the authored id. The authored id
(`:left`, `:voice-right`) is a handle for reading the EDN and for the (`:left`, `:voice-right`) is a handle for reading the EDN and for the
`:linked-to` written there; it does not appear in the document this returns. `:linked-to` written there; it does not appear in the document this returns.
What replaces it is an identity that means one placement and nothing else: seven What replaces it is an identity that means one instance and nothing else: seven
instances of one symbol are seven different things to name — to export on their instances of one symbol are seven different things to name — to export on their
own, to link a voice to, to point at later — and an id like `:left` is a own, to link a voice to, to point at later — and an id like `:left` is a
description of where a thing sits, which is exactly what changes when the stage description of where a thing sits, which is exactly what changes when the stage
is re-arranged. `:name` carries the label for a human and `:of` carries the is re-arranged. `:name` carries the label for a human and `:source :symbol`
symbol, so the node still says what it is and which drawing it plays." carries the symbol, so the node still says what it is and which drawing it
plays — and `:playback` says how time runs inside it, which is a separate
question from which drawing that is."
[source] [source]
(let [{:keys [name width height frames symbol instances audio scale]} layout (let [{:keys [name width height frames symbol instances audio scale]} layout
default-anchor (or (:anchor layout) ;; Which point of the SOURCE is pinned to the stage `:center`. Its
;; middle, unless the layout says otherwise for an off-centre drawing.
default-origin (or (:origin layout)
[(/ (:width source) 2) (/ (:height source) 2)]) [(/ (:width source) 2) (/ (:height source) 2)])
original (get-in source [:timelines :main]) original (get-in source [:symbols :main])
;; Authored id -> uuid, so the `:linked-to` in the EDN resolves to the ;; Authored id -> uuid, so the `:linked-to` in the EDN resolves to the
;; identity the document uses. Built before either pass because the audio ;; identity the document uses. Built before either pass because the audio
;; nodes refer to the instances. ;; nodes refer to the instances.
by-id (into {} (map (juxt :id :uuid)) (concat instances audio)) by-id (into {} (map (juxt :id :uuid)) (concat instances audio))
uuid-of (fn [what id] uuid-of (fn [what id]
(or (get by-id id) (or (get by-id id)
(throw (ex-info "the stage layout names a placement that is not there" (throw (ex-info "the stage layout names an instance that is not there"
{:in what :id id {:in what :id id
:known (vec (sort-by str (keys by-id)))})))) :known (vec (sort-by str (keys by-id)))}))))
;; EVERY PLACEMENT IS A PEG AND AN INSTANCE, and this layout is what
;; makes the pair necessary rather than tidy. `:scale` is KEYED — the
;; faces pulse — and it has to happen about the point the face is pinned
;; to. A stored `[:xform :anchor]` used to buy that: a static position
;; and a moving scale, turning about a fixed point. No static position
;; can do it alone, because `T(pos)·S(k(f))` moves the source's middle
;; whenever `k` changes, so holding it still would mean keying `pos` in
;; lockstep with `scale` — two channels that have to agree frame for
;; frame, which is the thing channels exist to avoid.
;;
;; A peg does it with neither:
;;
;; peg pos = center (+ drift), scale = k(f)
;; └ face pos = -origin
;;
;; world = T(center)·S(k(f))·T(-origin)
;;
;; which takes `origin` to `center` for EVERY k, with nothing keyed that
;; was not keyed before. That is the same matrix the anchor produced —
;; `node-test` asserts the identity — and it is reachable, keyable and
;; selectable, which the anchor was not.
;;
;; THE PEG IS THE PLACEMENT: it carries WHERE (pos, scale) and WHEN
;; (`:at`, `:span`), and the face under it carries only which drawing and
;; the offset to its origin. The time map has to be the peg's, because
;; `:scale` is read in the placement's own frames — that is what staggers
;; the entrances' growth — and `:span` goes with it so that
;; `node/placed-span` still answers where the placement sits on the stage.
;; The face then reads its peg's frames as its own and shows whenever the
;; peg does.
nodes (into nodes (into
{:root {:id :root :name "stage" :kind :group :z "a1"}} {:root {:id :root :name "stage" :kind :group :z "a1"}}
(map (fn [{:keys [uuid name z span at in center anchor drift phase]}] (mapcat (fn [{:keys [uuid peg name z span at center origin drift phase]}]
(let [anchor (or anchor default-anchor)] (let [origin (or origin default-origin)]
[uuid {:id uuid :name name :kind :symbol :of symbol [[peg {:id peg :name name :kind :group
:parent :root :z z :span span :parent :root :z z :span span
:time {:mode :map :at at :in in :rate 1} :time {:mode :map :at at :rate 1}
:channels {[:xform :pos] (if drift :channels {[:xform :pos]
(position-track center anchor drift phase frames) (if drift
(ch/framed (mapv - center anchor))) (position-track center drift phase frames)
[:xform :anchor] {:animated? false :value anchor} (ch/framed center))
[:xform :scale] scale}}])) [:xform :scale] scale}}]
[uuid {:id uuid :name (str name " face") :kind :instance
:parent peg :z "a1"
:source {:symbol symbol}
:channels {[:xform :pos]
(ch/framed (mapv - origin))}}]]))
instances)) instances))
nodes (into nodes nodes (into nodes
(map (fn [{:keys [uuid linked-to z source span at in gain pan]}] (map (fn [{:keys [uuid linked-to z source span at gain pan]}]
[uuid {:id uuid :kind :audio :parent :root :z z [uuid {:id uuid :kind :audio :parent :root :z z
:linked-to (uuid-of uuid linked-to) :linked-to (uuid-of uuid linked-to)
:source source :span span :source source :span span
:time {:mode :map :at at :in in :rate 1} :time {:mode :map :at at :rate 1}
:channels (cond-> {[:audio :gain] gain} :channels (cond-> {[:audio :gain] gain}
pan (assoc [:audio :pan] pan))}]) pan (assoc [:audio :pan] pan))}])
audio))] audio))]
(assoc source :name name :width width :height height (assoc source :name name :width width :height height
:timelines (assoc (:timelines source) :symbols (assoc (:symbols source)
:main {:id :main :frames frames :nodes nodes} :main {:id :main :frames frames :nodes nodes}
symbol (assoc original :id symbol))))) symbol (assoc original :id symbol)))))

View file

@ -6,8 +6,11 @@
:symbol :sym/face-8625 :symbol :sym/face-8625
:name "8625 stage study" :name "8625 stage study"
:width 320 :height 200 :frames 280 :width 320 :height 200 :frames 280
;; Each :center below places the source clip's center on the stage. A symbol ;; Each :center below places the source clip's center on the stage; :origin
;; can author :anchor to override that default for an off-center drawing. ;; overrides which point of the source that is, for an off-centre drawing.
;; :peg is the identity of the transform node the placement hangs off — it
;; carries :center and :scale, the face below it carries -:origin, and that pair
;; is what makes a KEYED scale happen about the pinned point. See demo/stage.
;; Each placement reads this pulse in its own local time, so the staggered ;; Each placement reads this pulse in its own local time, so the staggered
;; entrances start their growth at different moments on the master timeline. ;; entrances start their growth at different moments on the master timeline.
:scale {:animated? true :interp :linear :scale {:animated? true :interp :linear
@ -19,16 +22,18 @@
:over []} :over []}
;; Audio placements are ordinary timeline nodes with channel parameters. ;; Audio placements are ordinary timeline nodes with channel parameters.
;; :linked-to is an editorial link; their spans and time maps are independent. ;; :linked-to is an editorial link; their spans and time maps are independent.
;; A span is in the placement's OWN frames and :at is where its frame 0 lands on
;; the stage, so every entrance below plays from its own start.
:audio :audio
[{:id :voice-left :uuid #uuid "eeaa49c3-1238-469f-bf54-44929e379f6b" [{:id :voice-left :uuid #uuid "eeaa49c3-1238-469f-bf54-44929e379f6b"
:linked-to :left :z "a3" :linked-to :left :z "a3"
:source {:footage "f8cace9e-4ad3-4796-973c-c62eeebe3d01"} :source {:footage "f8cace9e-4ad3-4796-973c-c62eeebe3d01"}
:span [0 280] :at 0 :in 0 :at 0 :span [0 280]
:gain {:animated? false :value 1.0}} :gain {:animated? false :value 1.0}}
{:id :voice-right :uuid #uuid "468239dd-0e3a-4e5c-ac6f-1858430a0355" {:id :voice-right :uuid #uuid "468239dd-0e3a-4e5c-ac6f-1858430a0355"
:linked-to :right :z "a4" :linked-to :right :z "a4"
:source {:footage "f8cace9e-4ad3-4796-973c-c62eeebe3d01"} :source {:footage "f8cace9e-4ad3-4796-973c-c62eeebe3d01"}
:span [48 260] :at 48 :in 0 :at 48 :span [0 212]
:gain {:animated? true :interp :linear :gain {:animated? true :interp :linear
:keys {48 0.0, 60 1.0, 90 0.35, 115 0.9, 145 0.45, :keys {48 0.0, 60 1.0, 90 0.35, 115 0.9, 145 0.45,
170 1.0, 195 0.4, 220 0.85, 245 1.0, 259 0.0} 170 1.0, 195 0.4, 220 0.85, 245 1.0, 259 0.0}
@ -48,30 +53,37 @@
;; uuid. Nothing downstream of `compose` sees the authored id. ;; uuid. Nothing downstream of `compose` sees the authored id.
:instances :instances
[{:id :left :uuid #uuid "ee7321c8-faf1-46d7-8029-37771898accb" [{:id :left :uuid #uuid "ee7321c8-faf1-46d7-8029-37771898accb"
:peg #uuid "b1e0a7c4-5f3d-4a8e-9c21-70e4d1a90f01"
:name "8625 left" :z "a1" :name "8625 left" :z "a1"
:span [0 280] :at 0 :in 0 :at 0 :span [0 280]
:center [40 40] :drift [3 2] :phase 0} :center [40 40] :drift [3 2] :phase 0}
{:id :right :uuid #uuid "1aa0da78-b4ed-4bb6-8d70-b09a3ec5e2c3" {:id :right :uuid #uuid "1aa0da78-b4ed-4bb6-8d70-b09a3ec5e2c3"
:peg #uuid "b1e0a7c4-5f3d-4a8e-9c21-70e4d1a90f02"
:name "8625 right" :z "a2" :name "8625 right" :z "a2"
:span [48 280] :at 48 :in 0 :at 48 :span [0 232]
:center [120 40] :drift [-3 2] :phase 17} :center [120 40] :drift [-3 2] :phase 17}
{:id :top-third :uuid #uuid "23bb697d-eba7-4af6-a86c-606c50107088" {:id :top-third :uuid #uuid "23bb697d-eba7-4af6-a86c-606c50107088"
:peg #uuid "b1e0a7c4-5f3d-4a8e-9c21-70e4d1a90f03"
:name "8625 top third" :z "a5" :name "8625 top third" :z "a5"
:span [24 280] :at 24 :in 0 :at 24 :span [0 256]
:center [200 40] :drift [2 -3] :phase 31} :center [200 40] :drift [2 -3] :phase 31}
{:id :top-fourth :uuid #uuid "f4f0241d-026e-4e50-9bea-a4ccde896d8a" {:id :top-fourth :uuid #uuid "f4f0241d-026e-4e50-9bea-a4ccde896d8a"
:peg #uuid "b1e0a7c4-5f3d-4a8e-9c21-70e4d1a90f04"
:name "8625 top fourth" :z "a6" :name "8625 top fourth" :z "a6"
:span [72 280] :at 72 :in 0 :at 72 :span [0 208]
:center [280 40] :drift [-2 -2] :phase 49} :center [280 40] :drift [-2 -2] :phase 49}
{:id :bottom-left :uuid #uuid "8f594d72-a97f-4a32-82fd-08d1670a2218" {:id :bottom-left :uuid #uuid "8f594d72-a97f-4a32-82fd-08d1670a2218"
:peg #uuid "b1e0a7c4-5f3d-4a8e-9c21-70e4d1a90f05"
:name "8625 bottom left" :z "a7" :name "8625 bottom left" :z "a7"
:span [96 280] :at 96 :in 0 :at 96 :span [0 184]
:center [70 135] :drift [3 -2] :phase 63} :center [70 135] :drift [3 -2] :phase 63}
{:id :bottom-middle :uuid #uuid "63f3fb32-9e94-4d68-a1c2-12e6de2d04b5" {:id :bottom-middle :uuid #uuid "63f3fb32-9e94-4d68-a1c2-12e6de2d04b5"
:peg #uuid "b1e0a7c4-5f3d-4a8e-9c21-70e4d1a90f06"
:name "8625 bottom middle" :z "a8" :name "8625 bottom middle" :z "a8"
:span [120 280] :at 120 :in 0 :at 120 :span [0 160]
:center [160 135] :drift [-2 3] :phase 81} :center [160 135] :drift [-2 3] :phase 81}
{:id :bottom-right :uuid #uuid "fa338701-cb21-4d45-89f1-a5e706f045ec" {:id :bottom-right :uuid #uuid "fa338701-cb21-4d45-89f1-a5e706f045ec"
:peg #uuid "b1e0a7c4-5f3d-4a8e-9c21-70e4d1a90f07"
:name "8625 bottom right" :z "a9" :name "8625 bottom right" :z "a9"
:span [144 280] :at 144 :in 0 :at 144 :span [0 136]
:center [250 135] :drift [2 2] :phase 107}]} :center [250 135] :drift [2 2] :phase 107}]}

View file

@ -153,7 +153,7 @@
:fps fps :fps fps
:width 320 :width 320
:height 200 :height 200
:timelines :symbols
{:main {:main
{:id :main {:id :main
:frames frames :frames frames

View file

@ -12,7 +12,7 @@
│ │
FREEZE ──▶ channels on nodes FREEZE ──▶ channels on nodes
│ │
timeline/resolver ──▶ raster symbol/resolver ──▶ raster
— and the order of that diagram is the whole argument for the stage split. The — and the order of that diagram is the whole argument for the stage split. The
anchor fit is knob-free. Conditioning smooths its four parameters. The rings are anchor fit is knob-free. Conditioning smooths its four parameters. The rings are
@ -95,4 +95,4 @@
(def locked (def locked
"The same blocks, with `:head` held at measured frame zero." "The same blocks, with `:head` held at measured frame zero."
(delay (freeze/head-mode {:mode :anchored :anchors {0 0}} @frozen))) (delay (freeze/head-mode {:trace {:origin :start}} @frozen)))

View file

@ -0,0 +1,151 @@
(ns arthur.domain.bring
"Bringing symbols into a clip from another: out of a saved project, or out of
a freeze of new footage.
Copied, never linked. What comes in gets ids of its own where they are taken,
and editing it here does not touch where it came from. Tracking identities and
the analysis they were measured by come along only when the receiving clip can
hold them; otherwise what comes in is drawing, which plays but does not re-tune.
Plain data in and out — documents, and in `placed` a document with its store —
so the events that fetch them are only fetching."
(:refer-clojure :exclude [take])
(:require [arthur.domain.clip :as clip]
[arthur.domain.feature :as feature]
[arthur.domain.node :as node]
[clojure.string :as string]))
(defn symbols
"Copy symbols `roots` of clip `other`, and every symbol they place, into
`clip`. Returns `{:clip :ids}`, where `:ids` maps each copied symbol's id in
`other` to its id here.
AN ID THAT IS TAKEN IS RENAMED, never merged: two symbols that happen to share
an id are two drawings, and a cel's `:source :symbol` inside the copy is
rewritten to follow. `wanted` maps a root's id in `other` to the id it should preferably get,
which is how a symbol made from footage is called what the person typed rather
than `:main`.
Only symbols travel. What else `other` holds — tracking identities, an analysis
— is the caller's decision, because whether it can come too depends on what
`clip` already has."
[clip other roots wanted]
(let [;; A tree walk is safe because placing cannot make a cycle.
reach (into #{} (mapcat #(tree-seq any? (partial clip/places other) %)) roots)
ids (reduce (fn [ids sid]
(let [taken? #(or (contains? (:symbols clip) %)
(some #{%} (vals ids)))]
(assoc ids sid (clip/free-id taken? (get wanted sid sid)))))
{} (sort-by str reach))
;; Cel identity and timing stay put; content references follow
;; the symbol IDs assigned in the destination document.
repoint (fn [n ids]
(if (node/source n)
(update-in n [:source :symbol] ids)
n))
copy (fn [sid]
(-> (clip/symbol other sid)
(assoc :id (ids sid) :fps (or (:fps (clip/symbol other sid)) (:fps other)))
(update :nodes #(into {} (map (fn [[id n]] [id (repoint n ids)])) %))))]
{:clip (reduce (fn [c sid] (assoc-in c [:symbols (ids sid)] (copy sid))) clip reach)
:ids ids}))
(defn symbol-id
"An id for a symbol a person has named: the name, lower-cased and hyphenated,
or `:symbol` when nothing of it survives."
[label]
(let [slug (-> (str label) string/lower-case
(string/replace #"[^a-z0-9]+" "-")
(string/replace #"^-+|-+$" ""))]
(keyword (if (seq slug) slug "symbol"))))
(defn take
"Put `frozen`, a take, into `clip` as ONE symbol called `label`. Returns the
changed clip, the imported symbol id and the old-to-scoped subject ids.
`frozen` is what `flow/freeze/clip` makes: a `:main` that places one symbol per
tracked face. `:main` becomes the named symbol — it is what holds the faces in
stage pixels, so it is the thing worth placing. Each generated face symbol gets
an automatic association with the take's SOUND, source frames `range` of
footage `footage-id`. Thus the take is heard through the faces it places, and
a face subsequently placed by itself still brings its sound. This is the same
shape a future manual symbol/sound association can write; dropping a face is
not a special operation.
A multi-face take consequently reaches the same recording through several
symbols. `nest/audio-tracks` collapses simultaneous copies carrying the same
`:media-link`; placing those faces at different times still schedules each one.
The unique imported symbol id scopes every subject id. Thus two takes may both
arrive with detector subject `:face-1` without colliding in the document."
[clip frozen label footage-id range]
(let [scope (fn [root subject] (keyword (str (name root) "." (name subject))))
root (clip/free-id
(fn [candidate]
(or (contains? (:symbols clip) candidate)
(some #(contains? (:symbols clip) (scope candidate %))
(keys (:subjects frozen)))))
(symbol-id label))
scoped (partial scope root)
wanted (into {:main root :footage (scoped :footage)}
(map (fn [subject] [subject (scoped subject)]))
(keys (:subjects frozen)))
{c :clip ids :ids} (symbols clip frozen [:main] wanted)
sid (ids :main)
faces (mapv ids (sort-by str (keys (:subjects frozen))))
sound {:id :sound :name "sound" :kind :audio :parent nil
:z "z-sound" :source {:footage footage-id}
;; All automatic copies name one recording. The audio walk
;; uses this identity only to avoid mixing that recording once
;; per detected face when the complete take is played.
:media-link [:footage footage-id range]
;; Source frame `start` plays on the symbol's 0.
:span range
:time {:mode :map :at (- (first range)) :rate 1}}
feature-ids (into {}
(map (fn [[id f]]
[id (feature/owned (ids (:subject f)) (keyword (name id)))]))
(:features frozen))
subjects (into {}
(map (fn [[id subject]]
(let [new-id (ids id)]
[new-id (assoc subject :id new-id
:footage footage-id)])))
(:subjects frozen))
features (into {}
(map (fn [[id f]]
(let [new-id (feature-ids id)]
[new-id (-> f
(assoc :id new-id
:subject (ids (:subject f))
:symbol (ids (:symbol f))))])))
(:features frozen))
groups (into {}
(map (fn [[id g]]
(let [new-subject (ids (:subject g))
new-id (feature/owned new-subject (keyword (name id)))]
[new-id (-> g
(assoc :id new-id :subject new-subject)
(update :members #(mapv feature-ids %)))])))
(:groups frozen))]
{:sid sid
:subject-ids (select-keys ids (keys (:subjects frozen)))
:clip (-> (reduce (fn [document face]
(assoc-in document [:symbols face :nodes :sound] sound))
c faces)
(assoc-in [:symbols sid :name] (str label))
(update :analyses merge (:analyses frozen))
(update :subjects merge subjects)
(update :features merge features)
(update :groups merge groups))}))
(defn placed
"`entry` — a document and its store — once `brought` holds the symbols brought
in and `store` their blocks: the stores merged and an instance of `sid` placed
in `host` at `frame`, its middle on stage pixel `point` or where it was drawn
when there is none. See `clip/place-symbol`."
[entry brought store sid host frame uuid point]
(let [st (merge (:store entry) store)]
(assoc entry :store st :clip (clip/place-symbol brought st host sid
(* frame (:rate (clip/grid-time brought host))) uuid point))))

View file

@ -0,0 +1,25 @@
(ns arthur.domain.cadence
"Select integer content frames across frame grids. Stored frames never change.")
(defn ratio [grid native]
(if (and grid native (pos? grid) (pos? native)) (/ native grid) 1))
(defn frame
"Latest native frame at or before reader frame f. Never sample the future."
[f grid native]
(js/Math.floor (if (and grid native) (/ (* f native) grid) f)))
(defn frames
"Reader frames covering a native length, including a partial final frame."
[n grid native]
(when n (max 1 (js/Math.ceil (/ n (ratio grid native))))))
(defn reader-frame
"The earliest reader frame whose `frame` is at or after native frame n — the
inverse of `frame`, as far as it has one.
`frame` is a floor, so several reader frames can show one native frame and a
native frame between two of them is shown by none: this answers with the
reader frame that first reaches it, which is what seeking to a mark means."
[n grid native]
(js/Math.ceil (if (and grid native) (/ (* n grid) native) n)))

View file

@ -68,8 +68,11 @@
(defn framed [v] {:animated? false :value v}) (defn framed [v] {:animated? false :value v})
(defn keyed (defn keyed
([ks] (keyed ks :hold)) "A channel of keys, and how each one leads to the next. `interp` is an
([ks interp] {:animated? true :interp interp :keys ks :over []})) argument, never a default: `:hold` and `:linear` are the difference between a
cut and a tween, which is the whole content of the channel."
[ks interp]
{:animated? true :interp interp :keys ks :over []})
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
@ -91,17 +94,203 @@
(when-let [ks (:keys ch)] (when-let [ks (:keys ch)]
(vec (sort (keys ks))))) (vec (sort (keys ks)))))
(defn- check-unimplemented! ;; ---------------------------------------------------------------------------
"An override layer must fail LOUDLY rather than be ignored. ;; correction layers
;;
;; `:over` is an ORDERED STACK on top of whatever the channel already says.
;; Generated motion stays the base; a hand correction is a layer above it, so
;; regenerating replaces the base and the corrections survive. That is the whole
;; reason the stack exists rather than the hand edit being written into the keys.
;;
;; A LAYER'S VALUES ARE A CHANNEL. A constant adjustment is a framed one, a ramp
;; or a return motion is a keyed one, and neither needs a second way of saying
;; what a value is over time: layers read through `value-at` and `cursor` like
;; anything else, which is also what stops the fast path and the specification
;; from being two implementations of blending.
;;
;; A LAYER HAS NO TIME SPACE OF ITS OWN. Its `:support` and its values' keys are
;; in the frames the base channel's keys are in — the node's own. A correction on
;; a lane is therefore in lane frames and crosses the drawing boundaries under
;; it; a correction on one cel is in that cel's frames and travels
;; with it when it moves. Ownership already answered the question, so there is no
;; field to get wrong.
Silently dropping an :over layer would present as a hand (defn layer
correction that did not take — a correction the user made once, watched fail, "One correction: `values` applied to the base wherever `support` covers the
and has no reason to trust again. Nothing can produce one yet, so this can frame. `op` is `:offset` or `:replace`."
only fire on a data shape that has run ahead of the code." [id support op values]
{:id id :support support :op op :values values})
(defn- covers?
"Half-open, as a span is: a correction over frames 10 to 12 is `[10 13)`."
[[in out] f]
(and (<= in f) (< f out)))
(defn- width
"Components in a value, or nil for a number. A dense value is a typed-array
view, an authored one a vector, and a correction has to add to either."
[v]
(cond (number? v) nil (vector? v) (count v) :else (.-length v)))
(defn- shape
"What kind of value this is, for asking whether one can be added to another:
`:scalar`, a component count, or `:opaque` for a value that is neither — a
`[:vis]` boolean is opaque, and can be replaced but not offset."
[v]
(cond
(number? v) :scalar
(vector? v) (count v)
(and (some? v) (number? (.-length v))) (.-length v)
:else :opaque))
(defn value-shape
"The shape of the values a channel yields, without sampling it, or nil where
there is nothing to read it off — an empty key map says nothing about what its
values would have been, and nil must not be taken for a scalar."
[ch] [ch]
(when (seq (:over ch)) (cond
(throw (ex-info "channel has :over layers and the override layer is not built (port-plan step 2 scope)" (not (:animated? ch)) (when (some? (:value ch)) (shape (:value ch)))
{:over (:over ch) :channel (dissoc ch :dense)})))) (:dense ch) (if (= 1 (:stride (:dense ch))) :scalar (:stride (:dense ch)))
(seq (:keys ch)) (shape (val (first (:keys ch))))
:else nil))
(defn- shape-conflict [base-shape correction-shape]
(cond
(or (nil? base-shape) (nil? correction-shape)) nil
(= :opaque base-shape) "the base is not a number or a row of components"
(= :opaque correction-shape) "the correction is not a number or a row of components"
(not= base-shape correction-shape)
(str "the base has " (pr-str base-shape) " and the correction "
(pr-str correction-shape) " — a correction cannot offset a value of"
" a different shape")))
(defn conflict-with
"Why correction `l` cannot apply to base channel `base`, or nil.
ONLY `:offset` can conflict. It adds component by component, so it needs the
base to have the components it has — which is what a topology change takes
away when a re-freeze gives a mouth a different number of points. `:replace`
states a whole value and so has nothing to agree with.
Shapes that cannot be read yet do not conflict: an empty key map is not a
disagreement, it is a channel with nothing in it."
[base l]
(when (= :offset (:op l))
(shape-conflict (value-shape base) (value-shape (:values l)))))
(defn- support-of [l]
(let [s (:support l)]
(when (and (vector? s) (= 2 (count s))
(every? number? s) (< (first s) (second s)))
s)))
(defn stack-conflict
"Why layer `i` can encounter a value of the wrong shape after the active
layers before it, or nil.
Replacement coverage is considered at every interval boundary. This matters
when adjacent replacements jointly cover an offset: neither covers its whole
support, but the base can never reach it. A conflicted replacement is skipped,
exactly as the evaluator skips it."
[ch i]
(let [l (nth (:over ch) i nil)]
(when (and (= :offset (:op l)) (support-of l))
(let [[a b] (support-of l)
prior (take i (:over ch))
cuts (->> prior
(keep support-of)
(mapcat identity)
(filter #(< a % b))
(into [a b])
distinct sort)
;; Shape at a point is the last active, nonempty replacement's
;; shape, or the base shape when no replacement supplies a value.
at (fn [f]
(or (last (keep (fn [p]
(let [s (support-of p)
v (value-shape (:values p))]
(when (and (= :replace (:op p))
(not (:conflict p)) v s
(covers? s f))
v)))
prior))
(value-shape ch)))
shapes (into #{} (map (fn [[x y]] (at (/ (+ x y) 2))))
(partition 2 1 cuts))
v (value-shape (:values l))]
(some #(shape-conflict % v) shapes)))))
(defn reconcile
"Recheck an ordered layer stack against this channel's base.
Old conflict marks are findings from an earlier base, so they are cleared and
recomputed in order. A newly conflicted replacement is then invisible to the
layers after it, matching evaluation. Nothing is dropped or reordered."
[ch]
(let [layers (mapv #(dissoc % :conflict) (:over ch))]
(reduce (fn [out l]
(let [candidate (assoc ch :over (conj out l))
why (stack-conflict candidate (count out))]
(conj out (cond-> l why (assoc :conflict why)))))
[] layers)))
(defn conflicts
"Corrections on `ch` that cannot apply to its base, as `[{:id :why}]`.
NOT `problems`. A conflict is a legitimate state for a document to be in: a
regeneration changed the topology under a correction that was right when it was
made, and resolving it is a person's decision, not a reason the document will
not load. `flow/regenerate` records one on the layer, a conflicted layer is not
applied, and this is how a view finds them to offer."
[ch]
(vec (for [[i l] (map-indexed vector (:over ch))
:let [why (or (:conflict l) (stack-conflict ch i))]
:when why]
{:id (:id l) :why why})))
(defn- offset-onto
"`base` plus `v`, component-wise. A vector, never a write into `base`, which
for a dense channel is a view onto the block itself."
[base v ch]
(let [wb (width base) wv (width v)]
(cond
(and (nil? wb) (nil? wv)) (+ base v)
(and wb wv (= wb wv))
(mapv (fn [i] (+ (component base i) (component v i))) (range wb))
:else
(throw (ex-info "a correction cannot offset a value of a different shape"
{:base wb :correction wv :channel (dissoc ch :dense)})))))
(defn- eye-opening-onto [points amount]
(let [n (width points)
ys (map #(component points %) (range 1 n 2))
center (/ (+ (reduce min ys) (reduce max ys)) 2)]
(mapv (fn [i] (let [v (component points i)]
(if (odd? i) (+ center (* amount (- v center))) v)))
(range n))))
(defn- over-at
"Fold `ch`'s layers onto `base` at frame f. `read` samples one layer's values
and is the only thing that differs between the specification and the cursor."
[ch f base read]
(reduce-kv
(fn [v i {:keys [support op values conflict]}]
;; A conflicted correction is neither applied nor forgotten: it stays in
;; the document, `conflicts` reports it, and a person decides. Applying it
;; would misapply it; removing it would throw away hand work.
(if (or conflict (not (covers? support f)))
v
(let [x (read i values f)]
(cond
(nothing? x) v
(= :replace op) x
;; `replace` can supply a value over an absent base; `offset` has
;; nothing to add to and says so rather than inventing a pose.
(nothing? v) absent
(= :eye-opening op) (eye-opening-onto v x)
:else (offset-onto v x ch)))))
base
(vec (:over ch))))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; dense blocks ;; dense blocks
@ -147,9 +336,9 @@
Decoding costs the view. `out` is a stride-sized destination the caller owns — Decoding costs the view. `out` is a stride-sized destination the caller owns —
`cursor` allocates one per channel — because a copy per node per frame is the `cursor` allocates one per channel — because a copy per node per frame is the
allocation this whole model is arranged to avoid; passing nil allocates, which allocation this whole model is arranged to avoid; passing nil allocates, which
is what `value-at`, the specification, does." is what `value-at`, the specification, does — and it says so by passing nil,
([blk f st] (dense-at blk f st nil)) because there is no arity here that decides it for a caller."
([{:keys [store offset stride scale] nf :frames} f st out] [{:keys [store offset stride scale] nf :frames} f st out]
(let [{:keys [data state]} (get st store)] (let [{:keys [data state]} (get st store)]
(when (nil? data) (when (nil? data)
(throw (ex-info "dense channel's store key is not in the store" (throw (ex-info "dense channel's store key is not in the store"
@ -164,7 +353,7 @@
:else (let [dst (or out (js/Float64Array. stride))] :else (let [dst (or out (js/Float64Array. stride))]
(dotimes [k stride] (dotimes [k stride]
(aset dst k (/ (aget data (+ o k)) scale))) (aset dst k (/ (aget data (+ o k)) scale)))
dst)))))))) dst)))))))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; the specification ;; the specification
@ -180,9 +369,19 @@
(if (and (= :linear (segment-interp ch left)) right (>= f left) (> right left)) (if (and (= :linear (segment-interp ch left)) right (>= f left) (> right left))
(let [b (get (:keys ch) right) (let [b (get (:keys ch) right)
t (/ (- f left) (- right left))] t (/ (- f left) (- right left))]
(if (vector? a) (cond
;; Palette-choice channels interpolate identities into a blend
;; descriptor. The renderer keeps indexed geometry in the left
;; palette's bank and blends that bank's ramp toward the right one.
;; Palette ids are opaque identities. Built-ins happen to use
;; keywords, while palettes made in the editor use UUIDs; treating
;; the latter as numbers produces NaN and therefore the renderer's
;; pink bad-data sentinel.
(and (= :palette (:semantic ch)) (some? a) (some? b))
{:from a :to b :t t}
(vector? a)
(mapv (fn [x y] (+ x (* t (- y x)))) a b) (mapv (fn [x y] (+ x (* t (- y x)))) a b)
(+ a (* t (- b a))))) :else (+ a (* t (- b a)))))
a))) a)))
(defn- keyed-at (defn- keyed-at
@ -199,19 +398,39 @@
right (first (drop-while #(<= % f) fr))] right (first (drop-while #(<= % f) fr))]
(interpolate ch f left right))) (interpolate ch f left right)))
(defn repair-frame
"Map a damaged frame to a donor in the current base. Latest interval wins;
donors are sampled directly, never recursively through other repairs."
[ch f]
(reduce (fn [frame {:keys [from through donor]}]
(if (<= from f through) donor frame))
f (:repairs ch)))
(defn value-at (defn value-at
"Sample a channel at frame f. THE SPECIFICATION — correct, allocating, and "Sample a channel at frame f. THE SPECIFICATION — correct, allocating, and
O(n) in the keys. `cursor`/`sample!` is what playback uses." O(n) in the keys. `cursor`/`sample!` is what playback uses.
([ch f] (value-at ch f nil))
([ch f store] `store` IS AN ARGUMENT, NEVER A DEFAULT. A dense channel cannot be read
(check-unimplemented! ch) without the tier-2 store it names, and an arity that filled in nil let a
(cond caller omit it, read correctly for every channel that happened not to be
dense, and throw the first time a selection landed on one that was. That is
how `gesture/values` took the stage down on an iris. A caller with no store
says `nil` and means it."
([ch f store] (value-at ch f f store))
([ch base-f correction-f store]
(let [base-f (repair-frame ch base-f)
base (cond
(not (:animated? ch)) (:value ch) (not (:animated? ch)) (:value ch)
(:dense ch) (dense-at (:dense ch) f store) (:dense ch) (dense-at (:dense ch) base-f store nil)
(:keys ch) (let [ks (:keys ch)] (:keys ch) (let [ks (:keys ch)]
(if (empty? ks) absent (keyed-at ch f))) (if (empty? ks) absent (keyed-at ch base-f)))
:else :else
(throw (ex-info "animated channel has neither :keys nor :dense" {:channel ch}))))) (throw (ex-info "animated channel has neither :keys nor :dense"
{:channel ch})))]
(if (seq (:over ch))
(over-at ch correction-f base
(fn [_ values f] (value-at values f store)))
base))))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; the playback path ;; the playback path
@ -233,7 +452,7 @@
(recur (inc mid) hi mid) (recur (inc mid) hi mid)
(recur lo (dec mid) best)))))) (recur lo (dec mid) best))))))
(deftype Cursor [ch ks store buf ^:mutable i] (deftype Cursor [ch ks store buf overs ^:mutable i]
Object Object
(toString [_] (str "#Cursor{" (pr-str (if ks :keyed (if (:dense ch) :dense :framed))) " i=" i "}"))) (toString [_] (str "#Cursor{" (pr-str (if ks :keyed (if (:dense ch) :dense :framed))) " i=" i "}")))
@ -244,26 +463,28 @@
needs, for the same reason the resolver owns one point buffer per node. needs, for the same reason the resolver owns one point buffer per node.
Only a wide fixed-point block gets a buffer: a stride-1 block decodes to a Only a wide fixed-point block gets a buffer: a stride-1 block decodes to a
number and a block with no `:scale` is handed back as a view." number and a block with no `:scale` is handed back as a view.
([ch] (cursor ch nil))
([ch store] A correction layer gets a reading head of its own, because its values are a
(check-unimplemented! ch) channel and this is how a channel is read fast. One level deep: a layer's
values may not themselves carry layers, which `problems` refuses.
`store` is an argument for the reason it is one on `value-at`."
[ch store]
(let [d (:dense ch)] (let [d (:dense ch)]
(->Cursor ch (->Cursor ch
(when (and (:animated? ch) (not d) (seq (:keys ch))) (frames ch)) (when (and (:animated? ch) (not d) (seq (:keys ch))) (frames ch))
store store
(when (and d (:scale d) (> (:stride d) 1)) (when (and d (:scale d) (> (:stride d) 1))
(js/Float64Array. (:stride d))) (js/Float64Array. (:stride d)))
0)))) (mapv #(cursor (:values %) store) (:over ch))
0)))
(defn sample! (defn- base-sample!
"Value of the cursor's channel at f. O(1) when f is at or one key past where "What the cursor's channel says at f BEFORE its corrections. Advancing the
the cursor already sits — the playback case — and O(log n) otherwise, which is reading head is this function's whole job, and it is separate from blending so
a seek. Advancing and seeking are deliberately different costs: a scrub can that a layer cannot accidentally be read through the base's index."
afford a binary search and a frame cannot." [^Cursor cur ch ks f]
[^Cursor cur f]
(let [ch (.-ch cur)
ks (.-ks cur)]
(cond (cond
(not (:animated? ch)) (:value ch) (not (:animated? ch)) (:value ch)
(:dense ch) (dense-at (:dense ch) f (.-store cur) (.-buf cur)) (:dense ch) (dense-at (:dense ch) f (.-store cur) (.-buf cur))
@ -285,7 +506,25 @@
:else (bsearch ks f))] :else (bsearch ks f))]
(set! (.-i cur) i') (set! (.-i cur) i')
(interpolate ch f (nth ks i') (when (< i' last) (nth ks (inc i')))))))) (interpolate ch f (nth ks i') (when (< i' last) (nth ks (inc i')))))))
(defn sample!
"Value of the cursor's channel at f. O(1) when f is at or one key past where
the cursor already sits — the playback case — and O(log n) otherwise, which is
a seek. Advancing and seeking are deliberately different costs: a scrub can
afford a binary search and a frame cannot.
A correction layer is sampled through its OWN cursor, so a stacked channel is
still one reading head per key map and `value-at` stays the specification for
the blending as well as for the base."
([cur f] (sample! cur f f))
([^Cursor cur base-f correction-f]
(let [ch (.-ch cur)
base (base-sample! cur ch (.-ks cur) (repair-frame ch base-f))]
(if (seq (:over ch))
(over-at ch correction-f base
(fn [i _ f] (sample! (nth (.-overs cur) i) f)))
base))))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
@ -303,6 +542,7 @@
(let [values (when (map? (:keys ch)) (vals (:keys ch))) (let [values (when (map? (:keys ch)) (vals (:keys ch)))
first-value (first values) first-value (first values)
linear-values? (or (every? number? values) linear-values? (or (every? number? values)
(and (= :palette (:semantic ch)) (every? some? values))
(and (vector? first-value) (and (vector? first-value)
(pos? (count first-value)) (pos? (count first-value))
(every? (fn [v] (and (vector? v) (every? (fn [v] (and (vector? v)
@ -312,6 +552,14 @@
linear? (or (= :linear (:interp ch)) linear? (or (= :linear (:interp ch))
(some #{:linear} (vals (:segments ch))))] (some #{:linear} (vals (:segments ch))))]
(cond-> [] (cond-> []
(and (contains? ch :repairs)
(not (and (vector? (:repairs ch))
(every? (fn [{:keys [id from through donor]}]
(and id (every? integer? [from through donor])
(<= 0 from through) (<= 0 donor)))
(:repairs ch)))))
(conj "repairs require an ID and nonnegative whole donor and interval frames")
(not (map? ch)) (not (map? ch))
(conj "not a map") (conj "not a map")
@ -345,8 +593,45 @@
(or (:dense ch) (not linear-values?))) (or (:dense ch) (not linear-values?)))
(conj ":linear interpolation needs numeric keys of one shape") (conj ":linear interpolation needs numeric keys of one shape")
(and (map? ch) (seq (:over ch))) (and (map? ch) (contains? ch :over) (not (vector? (:over ch))))
(conj ":over layers are not implemented (port-plan step 2 scope)") (conj ":over is an ORDERED stack, so it is a vector")
(and (map? ch) (vector? (:over ch)))
(into (for [{:keys [id support op values]} (:over ch)
p (cond-> []
(nil? id)
(conj "needs an :id — a correction has an identity a regeneration can keep")
(not (and (vector? support) (= 2 (count support))
(every? #(and (number? %) (js/Number.isFinite %)) support)
(< (first support) (second support))))
(conj (str ":support " (pr-str support)
" must be a finite, increasing [in out)"))
(not (#{:offset :replace :eye-opening} op))
(conj (str ":op " (pr-str op) " is not :offset, :replace or :eye-opening"))
;; One level. A layer over a layer is an ordering mechanism
;; the stack already is, and it would make the read
;; unbounded in depth for nothing.
(seq (:over values))
(conj "a layer's values cannot carry layers of their own")
(seq (problems (dissoc values :over)))
(conj (str "values are not a channel: "
(first (problems (dissoc values :over))))))]
(str "correction " (pr-str id) " " p)))
;; A shape mismatch NOBODY HAS RECORDED is an authoring bug; one a
;; regeneration recorded is a conflict awaiting a person, and `conflicts`
;; reports those. The distinction is what keeps a topology change from
;; making a document that will not load.
(and (map? ch) (vector? (:over ch)))
(into (for [[i l] (map-indexed vector (:over ch))
:when (not (:conflict l))
:let [why (stack-conflict ch i)]
:when why]
(str "correction " (pr-str (:id l)) " " why)))
;; A scale of zero divides every value in the block by zero, and a negative ;; A scale of zero divides every value in the block by zero, and a negative
;; one mirrors the geometry. Both are authored-data bugs that present as a ;; one mirrors the geometry. Both are authored-data bugs that present as a

View file

@ -1,47 +1,49 @@
(ns arthur.domain.clip (ns arthur.domain.clip
"A CLIP: the unit of work, and a library of timelines. "A CLIP: the unit of work, and a library of symbols.
{:name \"take\" {:name \"take\"
:fps 30 :fps 30
:width 320 :height 200 :width 320 :height 200
:analysis {...} :analyses {analysis-id {...}}
:subjects {...} :features {...} :groups {...} :subjects {...} :features {...} :groups {...}
:timelines {:main {:id :main :frames 229 :nodes {...}}}} :symbols {:main {:id :main :frames 229 :nodes {...}}}}
Every field here is a fact about the clip and NOT about a bag of nodes, which is Every field here is a fact about the clip and NOT about a bag of nodes, which is
the cut this namespace exists to make. Before it, one map carried both: `:fps`, the cut this namespace exists to make. Before it, one map carried both: `:fps`,
the stage dimensions, the analysis record and the tracking identities sat beside the stage dimensions, the analysis record and the tracking identities sat beside
`:nodes`, and `arthur.db` said of it — correctly — that they \"sit on the scene `:nodes`. The cost of leaving them together was not untidiness. It was that a
map only because there is one clip per scene today\". The cost of leaving them SYMBOL had nowhere to live: a symbol is a bag of nodes with a frame space and
together was not untidiness. It was that a SYMBOL had nowhere to live: a library nothing else, so under the old shape it would have had to be a clip with seven
timeline is a bag of nodes with a frame space and nothing else, so under the old meaningless fields.
shape it would have had to be a clip with seven meaningless fields, or a second
structure with the same `:nodes` key that every walk had to be taught about.
Now there is one node-holding type — `arthur.domain.timeline` — and a clip holds Now there is one node-holding type — `arthur.domain.symbol` — and a clip holds
a MAP of them. A `:kind :symbol` instance names a timeline in `:timelines`, a MAP of them. A `:kind :instance` node places one symbol inside another, and
and the clip resolver gives each placement its own reading heads. the clip resolver gives each instance its own reading heads.
THE ROOT TIMELINE HAS A RESERVED ID, `:main`, rather than the clip carrying a WHAT IS NOT HERE: how nested symbols' frames and coordinates relate, and
pointer to it. A pointer is a field that can be wrong — it can name a timeline moving nodes between them, are `arthur.domain.nest`; bringing symbols in from
that is not there, and then every reader needs a fallback — where a reserved name another clip is `arthur.domain.bring`. This namespace is the document and the
can only be absent, which `problems` reports once. Flash reserves `_root` the operations that only need the document.
same way and for the same reason. Nothing else about `:main` is special: it is an
ordinary entry in the map, and a symbol is another one.
WHY :fps IS HERE AND :frames IS NOT. A rate is how fast the whole clip plays NO SYMBOL IS SPECIAL. There is no reserved root id: which symbol is on screen
against its audio, and a nested timeline cannot have one of its own — retiming an is the editor's state, not the document's, and every function here that needs
instance is `:rate` on its `:time` map, which is a factor and not a rate. A a symbol is told which. A new document has one symbol called `:main` because
frame COUNT is a property of a frame space, so every timeline has its own." it has to be called something, and that is all the name means — it can be
(:require [arthur.domain.feature :as feature] renamed or placed inside another symbol like any of them. What the document
does say is `:root`, which symbol it opens on: a pointer, not a kind of
symbol, the way a Flash file names its scene. See `opens-on` for why that
cannot be worked out instead.
:fps is the output grid. A symbol's optional :fps names the native grid its
frames were authored or measured on; absent means the document's grid."
(:refer-clojure :exclude [symbol])
(:require [arthur.domain.cadence :as cadence]
[arthur.domain.channel :as ch]
[arthur.domain.feature :as feature]
[arthur.domain.node :as node] [arthur.domain.node :as node]
[arthur.domain.palette :as pal] [arthur.domain.palette :as pal]
[arthur.domain.pose :as pose] [arthur.domain.pose :as pose]
[arthur.domain.timeline :as timeline])) [arthur.domain.symbol :as symbol]))
(def ^:const root-id
"The reserved id of the timeline a clip plays. See the namespace docstring."
:main)
(def clip-keys (def clip-keys
"Every top-level field of a clip, and the reason `arthur.domain.leaf` refuses "Every top-level field of a clip, and the reason `arthur.domain.leaf` refuses
@ -50,46 +52,287 @@
that loses something on every round trip, which is the one bug a persistence that loses something on every round trip, which is the one bug a persistence
layer must not be able to have. Add the field here and to `leaf/leaves` and layer must not be able to have. Add the field here and to `leaf/leaves` and
`leaf/clip` in the same commit." `leaf/clip` in the same commit."
#{:name :fps :analysis :subjects :features :groups :width :height :timelines}) #{:name :fps :analyses :subjects :features :groups :width :height :symbols
:palettes :default-palette :root})
(defn timeline (defn symbol
"One of the clip's timelines, by id." "One of the clip's symbols, by id."
[clip id] [clip sid]
(get-in clip [:timelines id])) (get-in clip [:symbols sid]))
(defn root (defn trace?
"The timeline the clip plays." "Is `sym` a tracing symbol: footage or a still to draw over, which is placed and
[clip] moved like any symbol and never drawn into the picture? See `trace-op`."
(timeline clip root-id)) [sym]
(= :trace (:type sym)))
(defn symbol-name
"What to call a symbol: its `:name`, or its id when it has none."
[clip sid]
(or (:name (symbol clip sid)) (name sid)))
(defn node-label
"What to call node `n` on screen.
A NAME A PERSON TYPED WINS, and for an instance that is the ONLY thing `:name`
now means: `place-symbol` deliberately does not copy the symbol's name onto the
node it makes. Two instances of one symbol are told apart by what somebody
called them — `8625 left` and `8625 right` of one `face` — and reading through
in front of that would collapse them to the same word.
OTHERWISE AN INSTANCE IS LABELLED BY WHAT IT PLACES, read through on every
render. A name copied at creation goes stale the moment the symbol is renamed,
and then the document shows one thing under two names: the symbol reads `bg` in
its tab while an instance of it still reads `symbol-18`, which is how a person
comes to paste a symbol into itself without being able to see that is what they
are doing. `problems` refuses that cycle; this is why it stops looking like a
reasonable thing to try.
An id is a uuid for a placement and a keyword for an authored node, and neither
reads as a name, so the last resort is a legible stand-in rather than `(str
id)` — `:face-1` keeps its colon and a uuid pushes a column open."
[clip id n]
(or (:name n)
(some->> (node/source n) (symbol-name clip))
(if (keyword? id) (subs (str id) 1) (subs (str id) 0 8))))
(defn frames (defn frames
"The clip's length, which is its root timeline's frame space and is not written "A symbol's length. Read off the symbol, never copied beside it."
down twice. Reading it off the root is what stops the two from disagreeing." [clip sid]
(:frames (symbol clip sid)))
(defn fps [clip sid] (or (:fps (symbol clip sid)) (:fps clip)))
(defn set-fps
"Change the output grid without rewriting authored content's frames.
A new document's empty symbol is the one exception: it has no native rate yet,
so it follows the project grid and its empty extent is rescaled to preserve its
duration. Once a symbol contains anything, changing the project rate records
the old effective rate on it before changing the output grid."
[clip rate]
(let [old (:fps clip)]
(-> clip
(update :symbols
#(into {}
(map (fn [[sid sym]]
[sid (cond
(:fps sym) sym
(empty? (:nodes sym))
(update sym :frames cadence/frames rate old)
:else (assoc sym :fps old))]))
%))
(assoc :fps rate))))
(defn output-frames [clip sid]
(cadence/frames (frames clip sid) (:fps clip) (fps clip sid)))
(defn grid-time [clip sid]
{:at 0 :rate (cadence/ratio (:fps clip) (fps clip sid))})
;; ---------------------------------------------------------------------------
;; the only crossing between the output grid and a symbol's own frames
;;
;; Everything authored is in a symbol's own frames and every editing gesture is
;; too — see `docs/one-grid-plan.md`. The output grid belongs to playback: the
;; clock, the audio mix, export, and the frame number the stage is drawing. These
;; two functions are the whole of the way between, so a caller that has a
;; playhead and needs a document coordinate says so in one visible call instead
;; of multiplying by a rate it had to know about.
(defn shown-frame
"Which of symbol `sid`'s own frames the output frame `f` shows."
[clip sid f]
(cadence/frame f (:fps clip) (fps clip sid)))
(defn first-output-frame
"The output frame that first shows symbol `sid`'s own frame `n`: what to seek
to to put the playhead on a mark of `sid`'s ruler."
[clip sid n]
(cadence/reader-frame n (:fps clip) (fps clip sid)))
(defn source-time
"Derived cel-to-content map. Frame-rate units never enter stored retimes."
[clip host n]
(when-let [t (node/source-time n)]
(let [r (cadence/ratio (fps clip host) (fps clip (node/source n)))]
(-> t (update :rate * r) (update :at / r)))))
(defn placed-frame [clip host n f]
(let [child (node/source n)
r (cadence/ratio (fps clip host) (fps clip child))]
(some-> (node/placed-frame (update-in n [:playback :speed] #(* (or % 1) r))
f (frames clip child))
(update :frame js/Math.floor))))
(defn stage
"A symbol's stage as `[width height]`: its own, or the clip's where it has none.
Absent rather than copied in at creation, so a symbol nobody has sized follows
the project's size when that changes."
[clip sid]
(let [sym (symbol clip sid)]
[(or (:width sym) (:width clip)) (or (:height sym) (:height clip))]))
(defn update-symbol
"Apply f to one symbol in place."
[clip sid f & args]
(apply update-in clip [:symbols sid] f args))
(defn places
"The ids of the symbols `sid` places, directly."
[clip sid]
(into #{} (mapcat node/sources) (vals (:nodes (symbol clip sid)))))
(defn contains-symbol?
"Whether `inner` is `outer` or is placed anywhere inside it. Placing `outer`
into `inner` when this is true is a cycle."
[clip outer inner]
(let [seen (volatile! #{})]
(letfn [(walk [sid]
(or (= sid inner)
(when-not (@seen sid)
(vswap! seen conj sid)
(some walk (places clip sid)))))]
(boolean (walk outer)))))
(defn unplaced
"The symbols no other symbol places, sorted by id. What to open when a
document is opened."
[clip] [clip]
(:frames (root clip))) (let [placed (into #{} (mapcat #(places clip %)) (keys (:symbols clip)))]
(vec (sort-by str (remove placed (keys (:symbols clip)))))))
(defn update-timeline (defn- longest-unplaced
"Apply f to one timeline in place." "The longest symbol nothing else places, ties broken by id, and never a
[clip id f & args] tracing symbol: that is footage nobody has placed yet, as long as its take."
(apply update-in clip [:timelines id] f args))
(defn update-root [clip f & args]
(apply update-timeline clip root-id f args))
(defn nodes
"The root timeline's nodes. A convenience for the many callers that mean the
root and would otherwise spell it out; anything that could mean a symbol says
which timeline instead."
[clip] [clip]
(:nodes (root clip))) (first (sort-by (fn [sid] [(- (or (frames clip sid) 0)) (str sid)])
(remove #(trace? (symbol clip %)) (unplaced clip)))))
(defn opens-on
"The symbol a document opens on: its `:root`.
IT IS STORED, NOT WORKED OUT. It used to be the longest unplaced symbol, on
the theory that the one containing everything else is always that. A take
disproves it: imported footage is a symbol a thousand frames long, and the
moment its instance is deleted, or the drop lands somewhere other than the
root, nothing places it and it outranks a 120-frame `:main`. The document then
opened on the take, and `set-root-fps` rewrote the take's rate to the
project's on the way in.
A document with no `:root` — a demo, a fixture — still gets the old answer."
[clip]
(let [root (:root clip)]
(if (contains? (:symbols clip) root) root (longest-unplaced clip))))
(defn pin-root
"Give a document saved before `:root` existed the root it was made with.
Every such document started as `blank`, whose root is `:main`, and ids never
change, so `:main` is the answer wherever it survives; the old rule is only
for documents that never had one."
[clip]
(cond-> clip
(not (:root clip))
(assoc :root (if (contains? (:symbols clip) :main) :main (longest-unplaced clip)))))
(defn set-root-fps
"Set the document/output rate and the root symbol's editing rate together.
Project FPS is the root timeline's clock. Nested symbols keep their own native
rates and are sampled when placed across that boundary; only the root changes
here. Frame numbers are authored positions, so changing the rate does not
rewrite them or silently move cuts and keys."
[clip rate]
(let [root (opens-on clip)]
(cond-> (assoc clip :fps rate)
root (assoc-in [:symbols root :fps] rate))))
(def ^:const blank-frames
"How long a new document is before anything says otherwise. Four seconds at 30,
which is long enough to key something into and short enough to scrub by hand."
120)
(defn blank
"A new, empty document: one symbol, and nothing in it.
A SYMBOL IS BORN EMPTY. It used to be born holding a lane, because a lane was
the only place temporal content could go; now the symbol itself is the
container — see `symbol/children` — so there is nothing to invent and the
first drop into a symbol is the same operation as the second.
The tracking maps are ABSENT rather than empty, because `leaf/leaves` writes no
leaf for an empty one and so cannot bring it back: a blank document that opened
as a different map than it saved from is exactly the round trip that namespace
promises not to have. Nothing drawn by hand has them either — the demo scene
and the swarm carry no `:subjects` — so every reader already reads absence as
none, and `clip-keys` says which fields MAY be here, not which must."
[]
{:name "untitled"
:fps 30
:width 320 :height 200
:palettes {pal/default-id pal/default-palette}
:default-palette pal/default-id
:root :main
;; No native fps yet: an untouched canvas follows the project grid. Imported
;; and generated symbols carry their own rate explicitly.
:symbols {:main {:id :main :frames blank-frames :nodes {}}}})
(defn- op-path [op] (let [n (:node op)] (if (vector? n) n [n])))
(defn- own-span
"Where in `ops` the symbol at row path `path` is drawn: the end of its layer
when it has one, and the first and last of its own ops."
[ops path]
(let [own? #(and (< (count path) (count (op-path %)))
(= path (subvec (op-path %) 0 (count path))))
begin (first (keep-indexed #(when (and (= :begin (:kind %2)) (= path (op-path %2))) %1) ops))
end (when begin
(reduce (fn [depth i]
(case (:kind (ops i))
:begin (inc depth)
:end (if (= 1 depth) (reduced i) (dec depth))
depth))
0 (range begin (count ops))))
mine (keep-indexed #(when (own? %2) %1) ops)]
{:end (when (integer? end) end) :from (first mine) :to (last mine)}))
(defn- insert-at [ops i & more] (into (into (subvec ops 0 i) more) (subvec ops i)))
(defn atop
"`ops` with `op` drawn on top of what the symbol at row path `path` draws —
in ITS stacking context, where a shape added to it would land, so what is
above that symbol stays above. At the end when it drew nothing."
[ops path op]
(let [ops (vec ops)
{:keys [end to]} (own-span ops path)]
(cond end (insert-at ops end op)
to (insert-at ops (inc to) op)
:else (conj ops op))))
(defn in-layer
"`ops` with `op` drawn last INSIDE the symbol at row path `path` — in its
layer, so a knockout clears only what that symbol drew, as one saved there
would. A symbol that has no layer yet is given one around its ops. `ops`
unchanged when that symbol drew nothing."
[ops path op]
(let [ops (vec ops)
{:keys [end from to]} (own-span ops path)]
(cond
end (insert-at ops end op)
from (-> ops
(insert-at (inc to) op {:kind :end :node path})
(insert-at from {:kind :begin :node path}))
:else ops)))
(defn- transform-op (defn- transform-op
"Put a symbol's already resolved mark into its instance's parent space." "Put a symbol's already resolved mark into its instance's parent space. Its
name becomes its path of instances down to it, the path its timeline row has."
[op m path] [op m path]
(let [at (fn [x y] [(+ (* (aget m 0) x) (* (aget m 2) y) (aget m 4)) (let [at (fn [x y] [(+ (* (aget m 0) x) (* (aget m 2) y) (aget m 4))
(+ (* (aget m 1) x) (* (aget m 3) y) (aget m 5))]) (+ (* (aget m 1) x) (* (aget m 3) y) (aget m 5))])
scale (node/mean-scale m) scale (node/mean-scale m)
op (assoc op :node (conj path (:node op)))] n (:node op)
op (assoc op :node (if (vector? n) (into path n) (conj path n)))]
(case (:kind op) (case (:kind op)
:poly (let [out (js/Float64Array. (.-length (:pts op)))] :poly (let [out (js/Float64Array. (.-length (:pts op)))]
(dotimes [i (:n op)] (dotimes [i (:n op)]
@ -102,60 +345,394 @@
(assoc op :cx x :cy y :r (* scale (:r op)))) (assoc op :cx x :cy y :r (* scale (:r op))))
:rect (let [[x y] (at (:cx op) (:cy op))] :rect (let [[x y] (at (:cx op) (:cy op))]
(assoc op :cx x :cy y :size (* scale (:size op)))) (assoc op :cx x :cy y :size (* scale (:size op))))
:trace (assoc op :m (node/mul! (node/mat) m (:m op)))
op))) op)))
(defn- trace-op
"The one op an instance of tracing symbol `sym` makes, showing its frame `frame`
at world `m`, from node `id` of symbol `sid`.
NOT A PICTURE OP. Nothing indexed can show a photo, so the raster refuses this
kind and the player hands it to `ui/tracing` instead; and the resolver makes one
only when asked with `:tracing?`, which only the stage does. An export, a
symbol's centre and a thumbnail never ask, so a reference cannot reach the
picture by any path that forgets to filter it.
`:layer` is what the on/off switch for one layer is keyed by: the symbol the
placement is in and its id, so a face's plate is one layer wherever the face
is placed."
[sym frame m sid id]
{:kind :trace
:node id
:layer [sid id]
:media (:media sym)
:frame frame
:size [(:width sym) (:height sym)]
:m (js/Float64Array.from m)})
(defprotocol IActivePalette
(active-palette [this]
"The palette selected by this resolver's most recently resolved frame."))
(defn- channel-value [store selection frame]
(cond
(nil? selection) nil
(and (map? selection) (contains? selection :animated?))
(ch/value-at selection frame store)
:else selection))
(defn palette-at
"The palette symbol `owner` draws in at `frame`, as the stage's root does: a
palette id, or `{:from :to :t}` while its palette lane blends from one to the
next. `inherited` is what an unset palette falls back to, then `default`."
[clip store default owner frame inherited]
;; A palette track has meaningful uncovered time. Ordinary held
;; channels clamp to their first key before it, but doing that
;; here would erase the gap before the first palette segment.
(let [channel-value (partial channel-value store)
fallback (or (channel-value (:palette owner) frame)
inherited default)
materialize (fn [choice]
(cond
(= pal/inherit choice) fallback
(map? choice) (-> choice
(update :from #(if (= pal/inherit %) fallback %))
(update :to #(if (= pal/inherit %) fallback %)))
:else choice))
track (:palette-channel owner)
track-value (if-let [ks (:keys track)]
(some->> (keys ks)
(filter #(<= % frame))
sort last
(get ks))
(channel-value track frame))]
(or (when-let [track-sid (and (keyword? (:palette-track owner))
(:palette-track owner))]
(let [track-symbol (symbol clip track-sid)
palette-clip (first
(filter (fn [n]
(let [[a b] (node/placed-span n)]
(and a (<= a frame) (< frame b))))
(symbol/children (:nodes track-symbol))))
palette-symbol (some-> palette-clip node/source
(#(symbol clip %)))
fallback (when (= :palette (:type palette-symbol))
(:palette-ref palette-symbol))
choice (get-in palette-clip [:channels [:palette]])
start (some-> palette-clip node/placed-span first)]
(or (when (and choice start)
(materialize (channel-value choice (- frame start))))
fallback)))
track-value
fallback)))
(defn resolver (defn resolver
"Resolve a clip, including each library timeline placed by a symbol instance. "Resolve an output frame, selecting native content at each symbol boundary.
Every instance owns its cursors and buffers. The IResolver queries return
Each instance owns its own timeline resolver, so two offsets never share a native node frames and world matrices for the last rendered output frame."
channel cursor or point buffer. The returned ops must be drawn before the next [clip sid store palette opts]
frame, as with timeline/resolver. (let [context? (and (map? palette) (:palettes palette) (:offsets palette))
active-palette-state (atom (:default palette))]
`root` is which timeline to resolve AS the root, and it defaults to the clip's. (letfn [(root-selection-at [owner frame inherited]
Passing a symbol's id is the whole of \"render that symbol\": a library timeline (palette-at clip store (:default palette) owner frame inherited))
and the clip's own are the same type, so a symbol resolves by being rooted (selection-at [owner frame inherited]
rather than by a second code path — which is the return on collapsing the two (let [selection (:palette owner)
into `domain/timeline`. Its frame space is its own `:frames`, and nested symbols chosen (cond
inside it still resolve, because this is the function that knows how to do that." (nil? selection) nil
([clip store] (resolver clip store pal/index-of root-id)) (and (map? selection) (contains? selection :animated?))
([clip store palette] (resolver clip store palette root-id)) (ch/value-at selection frame store)
([clip store palette root] (resolver clip store palette root nil)) :else selection)]
([clip store palette root {:keys [picture-fps] :as opts}] (or chosen inherited (:default palette))))
(letfn [(build [tid chain pose-tracks] (build [sid chain pose-tracks root?]
(when (some #{tid} chain) (when (some #{sid} chain)
(throw (ex-info "symbol timeline cycle" {:chain (conj chain tid)}))) (throw (ex-info "symbol cycle" {:chain (conj chain sid)})))
(let [tl (or (timeline clip tid) (let [sym (or (symbol clip sid)
(throw (ex-info "symbol names a missing timeline" {:timeline tid}))) (throw (ex-info "an instance names a missing symbol" {:symbol sid})))
nodes (:nodes tl) active (volatile! (when context? (:default palette)))
rank (timeline/draw-rank nodes (timeline/order nodes)) nodes (:nodes sym)
rank (symbol/draw-rank nodes (symbol/order nodes))
ids (sort-by rank (keys nodes)) ids (sort-by rank (keys nodes))
own (timeline/resolver tl store palette pose-tracks own (symbol/resolver sym store (if context?
(assoc opts :source-fps (:fps clip))) #(pal/render-index palette @active %)
palette)
(assoc opts :pose-tracks pose-tracks))
;; Each cel owns its source resolver and mutable buffers. A
;; tracing symbol has nothing to resolve: it is one op.
children (into {} children (into {}
(for [[id n] nodes :when (= :symbol (:kind n))] (for [[id n] nodes
[id (build (:of n) (conj chain tid) :when (= :instance (:kind n))
(get-in n [:playback :tracks]))]))] child (sort-by str (node/sources n))
(fn [f] :when (not (trace? (symbol clip child)))]
(let [by-id (into {} (map (juxt :node identity)) (own f))] [[id child] (build child (conj chain sid)
(into [] (get-in n [:playback :tracks]) false)]))
;; The instances that were on the last frame, and WHICH
;; drawing each was showing — a row path is read back through
;; the child that was actually resolved, not the only one
;; there used to be. Their resolvers still hold the frame
;; before whenever they were not on.
entered (volatile! {})
;; A symbol that knocks out is drawn into a layer of its own,
;; so what it clears is only ever its own.
layered? (symbol/knocks? nodes)
step (fn [f pre inherited forced]
(when context?
(vreset! active (or forced
(if root?
(root-selection-at sym (js/Math.floor f) inherited)
inherited)
(:default palette))))
;; The same decision that maps local drawing slots to
;; the active bank also names the clear colour.
(when root? (reset! active-palette-state @active))
(vreset! entered {})
(let [by-id (into {} (map (juxt :node identity))
(own (js/Math.floor f) (js/Math.floor pre)))]
(cond-> (into (if layered? [{:kind :begin :node []}] [])
(mapcat (mapcat
(fn [id] (fn [id]
(let [n (get nodes id)] (let [n (get nodes id)]
(if (= :symbol (:kind n)) (if (= :instance (:kind n))
(let [m (timeline/world-of own id) (let [m (symbol/world-of own id)
local (timeline/frame-of own id) local (symbol/frame-of own id)
target (timeline clip (:of n)) prior (symbol/pre-frame-of own id)
length (:frames target) length (frames clip (node/source n))
frame (when (and m (number? local)) shown (when (and m (number? local))
(if (get-in n [:time :loop?]) (placed-frame clip sid n local))
(mod local length) frame (:frame shown)]
local))] (cond
(if (and frame (<= 0 frame) (< frame length)) (not (and frame (<= 0 frame) (< frame length)))
(map #(transform-op % m [id]) ((get children id) frame)) []
[]))
(trace? (symbol clip (:symbol shown)))
(when (:tracing? opts)
[(trace-op (symbol clip (:symbol shown)) frame m sid id)])
:else
(do (vswap! entered assoc id (:symbol shown))
(map #(transform-op % m [id])
((get children [id (:symbol shown)])
frame
;; The slot's interval crosses the
;; boundary by being MAPPED, not
;; carried: the previous slot's
;; frame goes through the same
;; placement and retime as this
;; one, so the gap comes out in the
;; child's frames and at the child's
;; rate. An instance appearing for
;; the first time on this slot has no
;; previous frame, and so no gap.
(or (:frame (when (number? prior)
(placed-frame clip sid n prior)))
(dec frame))
@active
(when (and context? (:palette n))
(selection-at n frame nil)))))))
(when-let [op (get by-id id)] [op])))) (when-let [op (get by-id id)] [op]))))
ids))))))] ids))
(build root [] nil)))) layered? (conj {:kind :end :node []}))))]
(reify
IFn
(-invoke [_ f] (step f (dec f) nil nil))
(-invoke [_ f pre] (step f pre nil nil))
(-invoke [_ f pre inherited forced] (step f pre inherited forced))
symbol/IResolver
(world-of [_ [id & more]]
(if more
(when-let [w (and (contains? @entered id)
(symbol/world-of (get children [id (get @entered id)])
(vec more)))]
(node/mul! (node/mat) (symbol/world-of own id) w))
(symbol/world-of own id)))
(frame-of [_ [id & more]]
(if more
(when (contains? @entered id)
(symbol/frame-of (get children [id (get @entered id)]) (vec more)))
(symbol/frame-of own id)))
(pre-frame-of [_ [id & more]]
(if more
(when (contains? @entered id)
(symbol/pre-frame-of (get children [id (get @entered id)]) (vec more)))
(symbol/pre-frame-of own id))))))]
(let [r (build sid [] nil true)
grid (or (:grid-fps opts) (:fps clip))
native (fps clip sid)]
(reify
IFn
(-invoke [_ f]
(r (cadence/frame f grid native)
;; `d(k-1)`: the native frame the slot BEFORE this one selected, which
;; with `d(k)` is the interval `(d(k-1), d(k)]` the preserve-snap may
;; reach back into — the frames this slot is the first to cover, and so
;; the ones the grid would otherwise show to nobody. Slot 0 has no slot
;; before it, so its interval is its own frame alone.
;;
;; THE SNAP DOES NOT HAPPEN HERE, although the interval is born here and
;; nothing would need threading. One native frame per output frame means
;; the WHOLE PICTURE reading 13 instead of 15 — a head going two frames
;; stale, a 67ms hitch at 12fps, to fix one group's mouth. It is per
;; group, so it is seated where groups exist.
(if (pos? f) (cadence/frame (dec f) grid native) -1)
(when context? (:default palette))
nil))
symbol/IResolver
(world-of [_ path] (symbol/world-of r path))
(frame-of [_ path] (symbol/frame-of r path))
(pre-frame-of [_ path] (symbol/pre-frame-of r path))
IActivePalette
(active-palette [_] @active-palette-state))))))
(defn center
"The middle of everything symbol `sid` draws, over all its frames, in its own
coordinates. ALL frames rather than the first, so a symbol whose drawing
enters late, or travels, still has its middle where the drawing is. A symbol
that draws nothing gets the STAGE's middle, which is where a drawing made into
it will be, because drawings are made on the stage.
WHERE A DROP LANDS, AND WHAT THE INSTANCE TURNS ABOUT. `place-symbol` puts this
point under the pointer and stores it as the instance's `[:xform :pivot]`, and
`ui/drag`'s ghost draws the cross there so a drop lands where it was aimed —
one point, one meaning, three uses.
IT IS A DEFAULT AND NOT A CACHE, which is the distinction the stored anchor got
wrong. A pivot written here is a CHOICE made on the instance's behalf at the
moment it is placed, the same way `paint/centred` chooses a drawing's origin
when it is drawn; editing the symbol afterwards does not revise either, and the
cross is draggable so neither is a trap. What the anchor got wrong was being
invisible and unmovable, not being stored."
[clip store sid]
(let [resolve (resolver clip sid store pal/index-of {:grid-fps (fps clip sid)})
bounds (fn [[x0 y0 x1 y1 :as b] x y]
(if b [(min x0 x) (min y0 y) (max x1 x) (max y1 y)] [x y x y]))
[x0 y0 x1 y1]
(reduce
(fn [b {:keys [kind pts n cx cy r size]}]
(case kind
:poly (reduce (fn [b i] (bounds b (aget pts (* 2 i)) (aget pts (inc (* 2 i)))))
b (range n))
:disc (-> b (bounds (- cx r) (- cy r)) (bounds (+ cx r) (+ cy r)))
:rect (let [h (/ size 2)] (-> b (bounds (- cx h) (- cy h)) (bounds (+ cx h) (+ cy h))))
b))
nil
(mapcat resolve (range (frames clip sid))))]
(if x0
[(/ (+ x0 x1) 2) (/ (+ y0 y1) 2)]
(mapv #(/ % 2) (stage clip sid)))))
(defn place-symbol
"An instance of symbol `sid`, inside symbol `host`, at `frame` of `host`.
THE MIDDLE GOES UNDER THE POINTER, AND IS WHAT THE INSTANCE TURNS ABOUT.
`center` says where the symbol's drawing sits in its own coordinates; `pos` is
set so that point lands on `point`, a stage pixel — without one, a drop on the
timeline, the drawing stays where it was drawn — and the same point is stored as
the instance's `[:xform :pivot]`, so a turn or a scale happens about the middle
of the drawing rather than about the symbol's origin.
WITHOUT THAT PIVOT AN INSTANCE TURNS ABOUT THE CORNER OF THE STAGE. A symbol's
origin is the stage's, because that is where its contents were drawn, so the
middle of a drawing inside one is typically a hundred-odd pixels away from it on
a 320x200 stage. `node/local!` composes about the pivot, so this one stored
point is the difference between spinning in place and orbiting the top-left
corner. See `domain/gesture`.
THE UUID IS AN ARGUMENT. An instance's identity is the key it has in the node
map — it is what `:linked-to`, an export target and a saved leaf all name — so
generating one in here would make this function's result depend on when it was
called, and this namespace is the pure one.
The instance's own time starts where it was dropped: `:at frame` means frame 0
of the symbol plays on `frame` of `host`, which is what dragging something onto
a playhead is asking for. Its `:span` is in its OWN frames — the whole symbol,
0 to its length — wherever it was dropped; see `node/placed-span`.
Refused, returning the clip unchanged, when it would make a cycle: a symbol
cannot be placed inside itself or inside anything it places."
[clip store host sid frame uuid point]
(let [target (symbol clip sid)
end (frames clip host)]
(if (or (nil? target) (nil? end) (nil? frame) (neg? frame) (>= frame end)
(contains-symbol? clip sid host))
clip
(let [middle (center clip store sid)]
(update-symbol
clip host assoc-in [:nodes uuid]
{:id uuid
:kind :instance
:parent nil
;; Lexicographic draw order, as `domain/paint` does it: an instance made
;; later sits above one made earlier, and neither has to renumber.
:z (str "z" (js/Date.now) "-" (name sid))
:span [0 (cadence/frames (:frames target) (fps clip host) (fps clip sid))]
:time {:mode :map :at frame :rate 1}
:source {:symbol sid}
:playback {:in 0 :speed 1 :end :stop}
:channels {[:xform :pos] {:animated? false
:value (if point (mapv - point middle) [0 0])}
[:xform :pivot] {:animated? false :value middle}}})))))
(defn place-sound
"Place a sound at host frame `frame`. Its span uses `source`'s fps when
supplied, otherwise the host's. `rate` is a deliberate playback speed."
[clip host source label length rate frame uuid]
(let [end (frames clip host)]
(if (or (nil? end) (nil? frame) (neg? frame) (>= frame end))
clip
(update-symbol
clip host assoc-in [:nodes uuid]
{:id uuid
:name label
:kind :audio
:parent nil
:z (str "z" (js/Date.now) "-sound")
:source source
:span [0 (max 1 length)]
:time {:mode :map :at frame :rate rate}}))))
(defn fresh-id
"The first `:symbol-N` the clip does not already hold. Readable because an id
shows up in saved leaf paths, and deterministic because this namespace is pure."
[clip]
(first (remove (:symbols clip) (map #(keyword (str "symbol-" %)) (iterate inc 1)))))
(defn new-symbol
"A new, empty symbol `sid`, placed inside `host` at `frame` and running to the
end of it. Placed at the origin, so whatever is drawn into it lands where it was
drawn until the instance is moved."
[clip host sid frame uuid]
(let [end (frames clip host)]
(if (or (nil? end) (symbol clip sid) (nil? frame) (neg? frame) (>= frame end))
clip
(-> clip
(assoc-in [:symbols sid] {:id sid :name (name sid) :fps (fps clip host)
:frames (- end frame) :nodes {}})
(place-symbol nil host sid frame uuid nil)))))
(defn free-id
"`wanted`, or the first `wanted-2`, `wanted-3`… `taken?` does not claim.
Keeps the namespace, so `:sym/face` becomes `:sym/face-2`."
[taken? wanted]
(first (remove taken?
(cons wanted
(map #(keyword (namespace wanted) (str (name wanted) "-" %))
(iterate inc 2))))))
(defn conflicts
"Every hand correction in the document that its base has outgrown, as
`[{:symbol :node :channel :id :why}]`.
SEPARATE FROM `problems` on purpose. A conflict is a document a person still
has to make a decision about — a regeneration changed the topology under a
correction that was right when it was made — and not a reason the document
will not load. Nothing is dropped and nothing is misapplied meanwhile: the
layer stays where it is, the picture is the base, and this is the list a view
offers to resolve."
[clip]
(vec (for [[sid sym] (:symbols clip)
[id n] (:nodes sym)
[prop c] (:channels n)
{:keys [why] :as x} (ch/conflicts c)]
(assoc (select-keys x [:id]) :symbol sid :node id :channel prop :why why))))
(defn problems (defn problems
"Human-readable reasons this clip will not evaluate or save." "Human-readable reasons this clip will not evaluate or save."
@ -164,41 +741,80 @@
(concat (concat
(for [k (remove clip-keys (keys clip))] (for [k (remove clip-keys (keys clip))]
(str "clip has a field with no leaf to save it in: " (pr-str k))) (str "clip has a field with no leaf to save it in: " (pr-str k)))
(when-not (map? (:timelines clip)) (when-not (map? (:symbols clip))
[":timelines must be a map of id -> timeline"]) [":symbols must be a map of id -> symbol"])
(when (and (map? (:timelines clip)) (nil? (root clip))) (when (and (contains? clip :analyses) (not (map? (:analyses clip))))
[(str "no " (pr-str root-id) " timeline — a clip plays the one with the reserved id")]) [":analyses must be a map of analysis id -> analysis"])
(for [[id analysis] (:analyses clip)
:when (not= id (:id analysis))]
(str "analysis under key " (pr-str id) " has :id " (pr-str (:id analysis))))
(for [[id subject] (:subjects clip)
:when (not (contains? (:analyses clip) (:analysis subject)))]
(str "subject " (pr-str id) " names missing analysis "
(pr-str (:analysis subject))))
(for [[id subject] (:subjects clip)
:when (not (keyword? (:source-subject subject)))]
(str "subject " (pr-str id) " has no source subject"))
(when (and (contains? clip :palettes) (not (map? (:palettes clip))))
[":palettes must be a map of id -> palette"])
(when (and (contains? clip :default-palette) (map? (:palettes clip))
(not (contains? (:palettes clip) (:default-palette clip))))
[":default-palette must name a project palette"])
(when (and (contains? clip :root) (map? (:symbols clip))
(not (contains? (:symbols clip) (:root clip))))
[(str ":root names missing symbol " (pr-str (:root clip)))])
(for [[id p] (:palettes clip)
:when (or (not= id (:id p)) (not (pal/valid-palette? p)))]
(str "palette " (pr-str id) " is invalid or has a different :id"))
(when-not (or (nil? (:fps clip)) (and (number? (:fps clip)) (pos? (:fps clip)))) (when-not (or (nil? (:fps clip)) (and (number? (:fps clip)) (pos? (:fps clip))))
[(str ":fps is " (pr-str (:fps clip)) " — a rate is a positive number")]) [(str ":fps is " (pr-str (:fps clip)) " — a rate is a positive number")])
(for [[id tl] (:timelines clip) (for [[id sym] (:symbols clip)
:when (not= id (:id tl))] :when (not= id (:id sym))]
(str "timeline under key " (pr-str id) " has :id " (pr-str (:id tl)))) (str "symbol under key " (pr-str id) " has :id " (pr-str (:id sym))))
(for [[id tl] (:timelines clip) (for [[id sym] (:symbols clip)
p (timeline/problems tl)] p (symbol/problems sym)]
(str "timeline " (pr-str id) ": " p)) (str "symbol " (pr-str id) ": " p))
(for [[tid tl] (:timelines clip) (for [[sid sym] (:symbols clip)
[id n] (:nodes tl) [id n] (:nodes sym)
:when (and (= :symbol (:kind n)) :when (= :instance (:kind n))
(not (contains? (:timelines clip) (:of n))))] missing (remove (:symbols clip) (node/sources n))]
(str "timeline " (pr-str tid) " symbol " (pr-str id) (str "symbol " (pr-str sid) " instance " (pr-str id)
" names missing timeline " (pr-str (:of n)))) " names missing symbol " (pr-str missing)))
(for [[tid tl] (:timelines clip) ;; THE INVARIANT `place-symbol` AND `ui/drag` ALREADY ENFORCE, stated here so
[id n] (:nodes tl) ;; that every command is checked against it rather than the two that remember
:when (= :symbol (:kind n)) ;; to ask. A symbol placed inside itself, or inside anything it places, has no
:let [target (get-in clip [:timelines (:of n)]) ;; finite expansion: `build` above and `nest/audio-tracks` both walk instances
;; and both throw on the way round. Paste reached this function without it and
;; wrote a document that saved, loaded, and only then threw — which is the one
;; outcome `problems` exists to make impossible.
(for [[sid sym] (:symbols clip)
[id n] (:nodes sym)
:when (= :instance (:kind n))
src (node/sources n)
;; A source that does not exist is the rule above's to report, not this
;; one's, so it does not get named twice.
:when (and (contains? (:symbols clip) src)
(contains-symbol? clip src sid))]
(str "symbol " (pr-str sid) " instance " (pr-str id) " places "
(pr-str src) (if (= src sid) ", which is itself" ", which contains it")))
;; Pose tracks belong to this cel's single source symbol.
(for [[sid sym] (:symbols clip)
[id n] (:nodes sym)
:when (= :instance (:kind n))
:let [targets (keep #(get-in clip [:symbols %]) (node/sources n))
active (filter (fn [node] active (filter (fn [node]
(some :pose-sampled? (vals (:channels node)))) (some :pose-sampled? (vals (:channels node))))
(vals (:nodes target))) (mapcat #(vals (:nodes %)) targets))
groups (set (concat groups (set (concat
(map #(or (:pose-group %) (:id %)) active) (map #(or (:pose-group %) (:id %)) active)
(map #(vector :node (:id %)) active)))] (map #(vector :node (:id %)) active)))]
p (pose/problems (get-in n [:playback :tracks]) p (pose/problems (get-in n [:playback :tracks])
(:frames target) groups)] (apply max 0 (keep :frames targets)) groups)]
(str "timeline " (pr-str tid) " symbol " (pr-str id) ": " p)) (str "symbol " (pr-str sid) " instance " (pr-str id) ": " p))
(for [[tid tl] (:timelines clip) (for [[sid sym] (:symbols clip)
[id n] (:nodes tl) [id n] (:nodes sym)
:when (and (= :audio (:kind n)) (:linked-to n) :when (and (= :audio (:kind n)) (:linked-to n)
(not (contains? (:nodes tl) (:linked-to n))))] (not (contains? (:nodes sym) (:linked-to n))))]
(str "timeline " (pr-str tid) " audio " (pr-str id) (str "symbol " (pr-str sid) " audio " (pr-str id)
" links to missing node " (pr-str (:linked-to n)))) " links to missing node " (pr-str (:linked-to n))))
(feature/problems clip)))) (feature/problems clip))))

View file

@ -0,0 +1,321 @@
(ns arthur.domain.clipboard
"Pure multi-node clipboard commands.
A clipboard value is a detached forest of node maps. Normal copies keep symbol
references; `duplicate` with `:unique?` copies the complete referenced symbol
graph once for the whole forest. UI state, playhead conversion, and history
stay in events.ui. See docs/clipboard-plan.md."
(:require [arthur.domain.bring :as bring]
[arthur.domain.clip :as clip]
[arthur.domain.nest :as nest]
[arthur.domain.node :as node]
[arthur.domain.span :as span]
[arthur.domain.symbol :as symbol]))
(defn- prefix? [a b]
(and (<= (count a) (count b)) (= a (subvec b 0 (count a)))))
(defn- node-selections [clip selections]
(->> selections
(keep (fn [[kind sid id path :as address]]
(when (and (= :node kind) id (get-in clip [:symbols sid :nodes id]))
{:address address :sid sid :id id :path (vec (or path [id]))})))
;; One owned node reached through two shared occurrences is still one edit.
(reduce (fn [{:keys [seen out] :as acc} {:keys [sid id] :as x}]
(if (contains? seen [sid id]) acc
{:seen (conj seen [sid id]) :out (conj out x)}))
{:seen #{} :out []})
:out))
(defn canonical
"Valid selected node occurrences, with anything visibly below another selected
occurrence omitted. Order is selection order and therefore keeps the primary
member last in the ordinary case."
[clip selections]
(let [xs (node-selections clip selections)]
(filterv (fn [{p :path}]
(not-any? (fn [{q :path}]
(and (< (count q) (count p)) (prefix? q p)))
xs))
xs)))
(defn- subtree [nodes root]
(into {} (filter (fn [[id _]] (some #{root} (symbol/lineage nodes id)))) nodes))
(defn snapshot
"Snapshot the canonical selected forest, or `{:refused why}`."
[clip selections]
(let [roots (canonical clip selections)]
(if (empty? roots)
{:refused "select something to copy"}
{:clipboard
{:items
(mapv (fn [{:keys [sid id path]}]
(let [nodes (get-in clip [:symbols sid :nodes])]
{:sid sid :root id :path path :parent (:parent (get nodes id))
:nodes (subtree nodes id)}))
roots)}})))
(defn cut
"Delete a previously snapshotted forest. Snapshotting first is what makes cut
retain data even though its source nodes are gone."
[clip {:keys [items]}]
(let [after (reduce (fn [c {:keys [sid root]}] (nest/delete-node c sid root)) clip items)]
(if-let [why (first (clip/problems after))]
{:refused why}
{:clip after :selections []})))
(defn- fresh
[taken fresh-id]
(loop [id (fresh-id)]
(if (contains? taken id) (recur (fresh-id)) id)))
(defn- allocate
[clip items fresh-id]
(loop [pending (vec (mapcat (fn [[i item]] (map #(vector i %) (keys (:nodes item))))
(map-indexed vector items)))
taken (into #{} (mapcat (comp keys :nodes val) (:symbols clip)))
ids {}]
(if-let [k (first pending)]
(let [id (fresh taken fresh-id)]
(recur (subvec pending 1) (conj taken id) (assoc ids k id)))
ids)))
(defn- unique-content
[clip items unique?]
(let [roots (into #{} (comp (mapcat #(vals (:nodes %))) (keep node/source)) items)]
(if (and unique? (seq roots))
(let [{c :clip ids :ids} (bring/symbols clip clip roots {})]
[c ids])
[clip {}])))
(defn- materialize
[clip items fresh-id unique? paste?]
(let [[clip source-ids] (unique-content clip items unique?)
ids (allocate clip items fresh-id)
made
(mapv
(fn [[i {:keys [sid root path parent nodes]}]]
(let [id-of #(get ids [i %])
copied (into {}
(map (fn [[old n]]
(let [id (id-of old)]
[id (cond-> (assoc n :id id)
(:parent n) (assoc :parent (id-of (:parent n)))
(:stencil n) (assoc :stencil (id-of (:stencil n)))
(node/source n)
(assoc-in [:source :symbol]
(get source-ids (node/source n)
(node/source n))))])))
nodes)
new-root (id-of root)
;; Paste reparents roots into one explicit destination.
;; Duplicate leaves them beside their originals.
copied (assoc-in copied [new-root :parent]
(when-not paste? parent))]
{:source-sid sid :old-root root :path path :root new-root
:nodes copied}))
(map-indexed vector items))]
{:clip clip :items made}))
(defn- shifted [n delta]
(if (node/placed-span n)
(update-in n [:time :at] (fnil + 0) delta)
n))
(defn- top-zs [nodes n]
(let [base (or (last (sort (map #(or (:z %) "") (vals nodes)))) "")]
(map #(str base (apply str (repeat % "m"))) (range 1 (inc n)))))
(defn- add-composition
[clip sid items at]
(let [starts (keep #(some-> (get-in % [:nodes (:root %)]) node/placed-span first) items)
anchor (when (seq starts) (apply min starts))
delta (if anchor (- at anchor) 0)
existing (get-in clip [:symbols sid :nodes])
zs (top-zs existing (count items))
nodes (reduce (fn [nodes [{:keys [root] copied :nodes} z]]
(into nodes (assoc-in copied [root]
(-> (get copied root)
(shifted delta)
(assoc :z z)))))
existing (map vector items zs))]
(assoc-in clip [:symbols sid :nodes] nodes)))
(defn- add-lane
[clip sid items at fresh-id]
(let [roots (map #(get-in % [:nodes (:root %)]) items)
starts (map #(some-> % node/placed-span first) roots)
anchor (when (every? some? starts) (apply min starts))
intervals (when anchor
(sort-by first
(map (fn [n]
(let [[lo hi] (node/placed-span n)]
[(+ at (- lo anchor)) (+ at (- hi anchor))]))
roots)))
overlaps? (some (fn [[[a b] [c d]]] (and (< a d) (< c b)))
(partition 2 1 intervals))]
(if (some nil? starts)
{:refused "a lane accepts copied things only when they have a finite span"}
(if overlaps?
{:refused "overlapping copied things cannot be pasted into one lane"}
(reduce
(fn [result item]
(if (:refused result)
(reduced result)
(let [c (:clip result)
root (:root item)
n (get-in item [:nodes root])
desired (+ at (- (first (node/placed-span n)) anchor))
r (span/place-node c sid n desired
{:extent :grow-symbol
:remainder-id (fresh (into #{} (keys (get-in c [:symbols sid :nodes])))
fresh-id)})]
(if-let [made (:clip r)]
{:clip (update-in made [:symbols sid :nodes]
into (dissoc (:nodes item) root))}
r))))
{:clip clip} items)))))
(defn paste
"Paste `clipboard` into `sid`, anchoring its first finite start at `at`.
`fresh-id` is supplied by the event so this domain command remains testable."
[clip clipboard sid at {:keys [fresh-id] :or {fresh-id random-uuid}}]
(cond
(nil? (clip/symbol clip sid)) {:refused "the paste target no longer exists"}
(not (and (integer? at) (not (neg? at))))
{:refused "the playhead is not on one frame of the paste target"}
(empty? (:items clipboard)) {:refused "there is nothing to paste"}
:else
(let [{base :clip items :items} (materialize clip (:items clipboard) fresh-id false true)
r (if (symbol/lane? (clip/symbol base sid))
(add-lane base sid items at fresh-id)
{:clip (add-composition base sid items at)})
made (:clip r)
why (when made (first (clip/problems made)))]
(cond
(:refused r) r
why {:refused why}
:else {:clip made :roots (mapv :root items)}))))
(defn- add-duplicate-composition [clip sid items]
(let [existing (get-in clip [:symbols sid :nodes])
zs (top-zs existing (count items))]
(assoc-in clip [:symbols sid :nodes]
(reduce (fn [nodes [{:keys [root] copied :nodes} z]]
(into nodes (assoc-in copied [root :z] z)))
existing (map vector items zs)))))
(defn- add-duplicate-lane
[clip sid items fresh-id]
(let [old-roots (map #(get-in clip [:symbols sid :nodes (:old-root %)]) items)
spans (map node/placed-span old-roots)
start (apply min (map first spans))
end (apply max (map second spans))
duration (- end start)
selected (set (map :old-root items))
shifted-clip
(update-in clip [:symbols sid :nodes]
(fn [nodes]
(reduce (fn [ns n]
(let [lo (some-> (node/placed-span n) first)]
(if (and (nil? (:parent n))
(not (contains? selected (:id n)))
lo (>= lo end))
(update-in ns [(:id n) :time :at] (fnil + 0) duration)
ns)))
nodes (vals nodes))))]
(reduce
(fn [result item]
(if (:refused result)
(reduced result)
(let [c (:clip result)
root (:root item)
n (get-in item [:nodes root])
old (get-in clip [:symbols sid :nodes (:old-root item)])
desired (+ (first (node/placed-span old)) duration)
r (span/place-node c sid n desired
{:extent :grow-symbol
:remainder-id (fresh (into #{} (keys (get-in c [:symbols sid :nodes])))
fresh-id)})]
(if-let [made (:clip r)]
{:clip (update-in made [:symbols sid :nodes]
into (dissoc (:nodes item) root))}
r))))
{:clip shifted-clip}
(sort-by #(first (node/placed-span
(get-in clip [:symbols sid :nodes (:old-root %)]))) items))))
(defn duplicate
"Duplicate a snapshot beside its sources. Direct finite children of lane
symbols repeat forward and ripple later cels; everything else copies in place.
With `:unique?`, referenced symbol graphs are deep-copied once for the batch."
[clip clipboard {:keys [fresh-id unique?] :or {fresh-id random-uuid}}]
(if (empty? (:items clipboard))
{:refused "select something to duplicate"}
(let [{base :clip items :items}
(materialize clip (:items clipboard) fresh-id unique? false)
groups (vals (group-by :source-sid items))
result
(reduce
(fn [result group]
(if (:refused result)
(reduced result)
(let [c (:clip result)
sid (:source-sid (first group))
lane? (symbol/lane? (clip/symbol c sid))
[lane-items other]
((juxt filter remove)
#(let [old (get-in clip [:symbols sid :nodes (:old-root %)])]
(and lane? (nil? (:parent old)) (node/placed-span old)))
group)
c (if (seq other) (add-duplicate-composition c sid other) c)
r (if (seq lane-items)
(add-duplicate-lane c sid lane-items fresh-id)
{:clip c})]
r)))
{:clip base} groups)
made (:clip result)
why (when made (first (clip/problems made)))]
(cond
(:refused result) result
why {:refused why}
:else {:clip made
:roots (mapv (fn [{:keys [source-sid root path]}]
{:sid source-sid :id root
:path (conj (vec (butlast path)) root)})
items)}))))
(defn move-many
"Reparent the selected roots atomically, preserving their world transforms and
clocks. Optional delta places the forest later on the open ruler."
[document store open selections to frame delta]
(let [roots (canonical document selections)
under? (fn [path] (prefix? path (vec to)))]
(cond
(empty? roots) {:refused "select something to move"}
(some #(under? (:path %)) roots) {:refused "a selection cannot go inside itself"}
:else
(let [moved
(reduce (fn [result {:keys [path]}]
(if (:refused result) (reduced result)
(let [r (nest/move-node (:clip result) store open path to frame)]
(if (:refused r) (reduced r)
{:clip (:clip r)
:selections (conj (:selections result)
[:node (:sid r) (:id r) (conj (vec to) (:id r))])}))))
{:clip document :selections []} roots)]
(if (:refused moved) moved
(let [shifted (nest/slide-many (:clip moved) open
(mapv #(nth % 3) (:selections moved)) (or delta 0))
checked (if (:refused shifted) shifted
(reduce (fn [result sid]
(if (:refused result) (reduced result)
(span/finish (:clip result) sid
(get-in (:clip result) [:symbols sid :nodes])
nil :grow-symbol)))
shifted (distinct (map :sid roots))))]
(if (:refused checked) checked
(if-let [why (first (clip/problems (:clip checked)))]
{:refused why}
(assoc checked :selections (:selections moved))))))))))

View file

@ -0,0 +1,193 @@
(ns arthur.domain.correction
"Pure commands that author and resolve correction layers.
Evaluation belongs to `channel`; this namespace only constructs a layer,
places it on its owning node, and refuses a document that would not be valid."
(:require [arthur.domain.channel :as ch]
[arthur.domain.clip :as clip]
[arthur.domain.node :as node]))
(def ^:private supported-paths #{[:xform :rot] [:xform :pos]})
(def ^:private motions #{:constant :ramp :return})
(defn- finite? [x] (and (number? x) (js/Number.isFinite x)))
(defn- numeric-value? [v]
(or (finite? v)
(and (vector? v) (pos? (count v)) (every? finite? v))))
(defn- same-shape? [a b]
(or (and (number? a) (number? b))
(and (vector? a) (vector? b) (= (count a) (count b)))))
(defn- expected-value? [path v]
(case path
[:xform :rot] (finite? v)
[:xform :pos] (and (vector? v) (= 2 (count v)) (every? finite? v))
false))
(defn- values-channel
[{:keys [motion support delta start end peak peak-frame]}]
(let [[a b] support]
(case motion
:constant (ch/framed delta)
:ramp (ch/keyed {a start, (dec b) end} :linear)
:return (ch/keyed {a start, peak-frame peak, (dec b) start} :linear)
nil)))
(defn- invalid
[path {:keys [id support motion delta start end peak peak-frame]} existing]
(let [[a b] (when (and (vector? support) (= 2 (count support))) support)
samples (case motion :constant [delta] :ramp [start end]
:return [start peak] [])]
(cond
(nil? id) "a correction needs an ID"
(some #(= id (:id %)) existing) "the correction ID is already used on this channel"
(not (contains? supported-paths path)) "that property does not support correction authoring"
(not (contains? motions motion)) "choose constant, ramp, or return motion"
(not (and (integer? a) (integer? b) (< a b)))
"support must be an increasing [in out) of whole owner frames"
(not-every? numeric-value? samples) "correction values must be finite numbers"
(not-every? #(expected-value? path %) samples)
"correction values do not have the property's shape"
(and (= :ramp motion) (< (- b a) 2)) "a ramp needs at least two samples"
(and (= :return motion) (< (- b a) 3)) "return motion needs at least three samples"
(and (= :return motion)
(not (and (integer? peak-frame) (< a peak-frame (dec b)))))
"the return peak must be a whole owner frame inside both endpoints"
(and (#{:ramp :return} motion) (not (same-shape? start (if (= :ramp motion) end peak))))
"motion endpoints must have the same shape")))
(defn- finish [candidate selection]
(if-let [why (first (clip/problems candidate))]
{:refused why}
{:clip candidate :selection selection}))
(defn add
"Append one offset correction to a node channel.
Support and value keys are in the selected node's own frames. Defaults are
materialized through `node/channels`, so correcting an unkeyed transform does
not need a special representation."
[document sid node-id path spec]
(let [n (get-in document [:symbols sid :nodes node-id])
base (when n (get (node/channels n) path))
existing (:over base)
why (cond
(nil? (clip/symbol document sid)) "the owning symbol does not exist"
(nil? n) "the correction target does not exist"
(nil? base) "the correction target has no such channel"
:else (invalid path spec existing))]
(if why
{:refused why}
(let [layer (ch/layer (:id spec) (:support spec) :offset (values-channel spec))
corrected (update base :over (fnil conj []) layer)
candidate (assoc-in document [:symbols sid :nodes node-id :channels path] corrected)]
(finish candidate node-id)))))
(defn remove-layer
"Remove one named layer, refusing when a later layer depended on its shape."
[document sid node-id path layer-id]
(let [at [:symbols sid :nodes node-id :channels path]
c (get-in document at)
layers (:over c)]
(cond
(nil? c) {:refused "the correction channel does not exist"}
(not-any? #(= layer-id (:id %)) layers) {:refused "the correction does not exist"}
:else (finish (assoc-in document at
(assoc c :over (vec (remove #(= layer-id (:id %)) layers))))
node-id))))
(defn retry-layer
"Clear one recorded conflict when the complete resulting stack is valid."
[document sid node-id path layer-id]
(let [at [:symbols sid :nodes node-id :channels path]
c (get-in document at)
found (some #(when (= layer-id (:id %)) %) (:over c))]
(cond
(nil? found) {:refused "the correction does not exist"}
(nil? (:conflict found)) {:refused "the correction has no recorded conflict"}
:else
(finish (update-in document (conj at :over)
(fn [layers]
(mapv #(if (= layer-id (:id %)) (dissoc % :conflict) %) layers)))
node-id))))
(defn borrow-pose [document sid {:keys [from through donor head?] :as spec} store]
(let [frames (get-in document [:symbols sid :frames])
ids (into #{} (mapcat :nodes)
(filter #(= sid (:symbol %)) (vals (:features document))))
ids (cond-> ids head? (conj :head))
paths (for [id ids [path c] (get-in document [:symbols sid :nodes id :channels])]
[id path c])]
(cond
(not (and (integer? frames) (every? integer? [from through donor])
(<= 0 from through (dec frames)) (<= 0 donor (dec frames))))
{:refused "choose whole face frames within this symbol"}
(<= from donor through) {:refused "choose a clean donor outside the repair interval"}
(empty? paths) {:refused "this symbol has no tracked face features"}
(not-any? (fn [[_ path c]]
(and (= path [:geom :pts])
(not (ch/nothing? (ch/value-at (dissoc c :repairs :over) donor store)))))
paths)
{:refused "the donor has no face pose; choose another frame"}
:else
(finish (reduce (fn [doc [node path _]]
(update-in doc [:symbols sid :nodes node :channels path :repairs]
(fnil conj []) (select-keys spec [:id :from :through :donor])))
document paths) nil))))
(declare remove-eye-keys)
(defn remove-repair [document sid repair-id]
{:clip (reduce (fn [doc [id path]]
(update-in doc [:symbols sid :nodes id :channels path :repairs]
#(vec (remove (fn [r] (= repair-id (:id r))) %))))
(-> document
(remove-eye-keys sid repair-id :l) :clip
(remove-eye-keys sid repair-id :r) :clip)
(for [[id n] (get-in document [:symbols sid :nodes])
[path c] (:channels n) :when (:repairs c)] [id path]))})
(defn eye-key
"Key a procedural lid adjustment and gaze offset over one repair interval."
[document sid repair-id side frame {:keys [opening gaze-x gaze-y]}]
(let [outer (keyword (str "eye-" (name side)))
inner (keyword (str "eye-" (name side) "-in"))
iris (keyword (str "iris-" (name side)))
repair (some #(when (= repair-id (:id %)) %)
(get-in document [:symbols sid :nodes outer :channels [:geom :pts] :repairs]))
{:keys [from through]} repair
layer-id (str repair-id "/eye/" (name side))
edits [[outer [:geom :pts] :eye-opening opening 1]
[inner [:geom :pts] :eye-opening opening 1]
[iris [:xform :pos] :offset [gaze-x gaze-y] [0 0]]]]
(cond
(nil? repair) {:refused "this eye has no such repair interval"}
(not (and (integer? frame) (<= from frame through)))
{:refused "move the playhead inside this repair interval"}
(not (and (every? finite? [opening gaze-x gaze-y]) (<= 0 opening 3)))
{:refused "eye opening must be between 0 and 3; gaze offsets must be finite"}
:else
(finish
(reduce
(fn [doc [id path op value neutral]]
(update-in doc [:symbols sid :nodes id :channels path :over]
(fn [layers]
(let [existing (some #(when (= layer-id (:id %)) %) layers)
values (or (:values existing)
(ch/keyed {from neutral through neutral} :linear))
layer (ch/layer layer-id [from (inc through)] op
(assoc-in values [:keys frame] value))]
(conj (vec (remove #(= layer-id (:id %)) layers)) layer)))))
document edits)
nil))))
(defn remove-eye-keys [document sid repair-id side]
(let [layer-id (str repair-id "/eye/" (name side))]
{:clip (reduce (fn [doc [id path]]
(update-in doc [:symbols sid :nodes id :channels path :over]
#(vec (remove (fn [l] (= layer-id (:id l))) %))))
document
(for [[id n] (get-in document [:symbols sid :nodes])
[path c] (:channels n) :when (:over c)] [id path]))}))

View file

@ -0,0 +1,67 @@
(ns arthur.domain.creation
"Resolve where a new thing goes from the primary selection and playhead.
This namespace owns no editor state. A row address is a preference; walking
the occurrence path at one frame answers which preferred or enclosing symbol
is actually available. See `docs/creating-in.md`."
(:require [arthur.domain.clip :as clip]
[arthur.domain.nest :as nest]
[arthur.domain.node :as node]
[arthur.domain.symbol :as symbol]))
(defn- path-node
"The node at the end of an occurrence `path`, walked from `open`.
Selection also carries an owner sid and node id for commands that edit the
node directly. Those are deliberately not used here: the path is the address
of the row as seen from the open symbol, and is the only part that describes
every enclosing occurrence at arbitrary depth."
[document open path]
(loop [sid open [id & more] (seq path) found nil]
(if-not id
found
(when-let [n (get-in document [:symbols sid :nodes id])]
(if (seq more)
(when-let [inner (and (= :instance (:kind n)) (node/source n))]
(recur inner more n))
n)))))
(defn preferred-path
"The container path structurally implied by `selection`.
Selecting an instance means inside it. Selecting any other node means its
containing symbol. An empty or non-node selection means the open symbol."
[document open selection]
(let [[kind _sid id selected-path] selection
path (when (= :node kind)
(vec (or (seq selected-path) (when id [id]))))
selected (path-node document open path)]
(cond
(empty? path) []
(= :instance (:kind selected)) path
:else (vec (butlast path)))))
(defn target
"The nearest creation context available at `frame` of `open`.
Every non-empty candidate is validated by `nest/inside`, so every enclosing
symbol occurrence must be under the playhead in the current context. Walking
outward stops at the first valid symbol. A lane needs no cel at that frame,
but the occurrence chain that reaches the lane must still be valid. The empty
path always resolves to the open symbol.
A tracing symbol is never one: it holds a picture to draw over and no nodes,
so selecting a tracing layer creates beside it, as selecting a shape does.
Returns `{:kind :lane|:symbol :sid :path :frame :matrix :time}`."
[document store open selection frame]
(let [preferred (preferred-path document open selection)]
(some (fn [path]
(when-let [inside (nest/inside document store open path frame)]
(when-let [sid (and (:sid inside)
(not (clip/trace? (clip/symbol document (:sid inside))))
(:sid inside))]
(assoc inside :path path
:kind (if (symbol/lane? (clip/symbol document sid))
:lane :symbol)))))
(take (inc (count preferred)) (iterate pop preferred)))))

View file

@ -0,0 +1,88 @@
(ns arthur.domain.cut
"The eraser's cut: a shape's ring with a stroke taken out of it.
Illustrator's and Flash's eraser, not a raster one: what is left is the
SHAPE, with the cut edge made of its own points — points the pen moves, keys
and tweens like any others. A cut through the middle leaves two pieces, and
a cut inside it leaves a hole, which is bridged into the one ring as a brush
stroke's is (see `outline`).
In stage pixels, on both sides: the preview cuts the rings it is about to
draw, and the saved cut cuts the same rings and takes the answer back into
the shape's own coordinates — one function, so the preview is the result."
(:require ["polygon-clipping" :as clipping]
[arthur.domain.channel :as channel]
[arthur.domain.nest :as nest]
[arthur.domain.node :as node]
[arthur.domain.outline :as outline]
[arthur.domain.paint :as paint]))
(defn- area [ring]
(let [ps (vec (partition 2 ring)) n (count ps)]
(js/Math.abs (/ (reduce + (map (fn [i] (let [[ax ay] (ps i) [bx by] (ps (mod (inc i) n))]
(- (* ax by) (* bx ay))))
(range n)))
2))))
(defn cut
"Ring `ring` with `cutters` — each `[outer & holes]` — taken out of it: one
ring per piece left, biggest first, an empty vector when nothing is left, or
nil when the cut does not touch it."
[ring cutters]
(let [shape #js [(outline/->js ring)]
knife (into-array (map #(into-array (map outline/->js %)) cutters))]
(when (seq (array-seq (clipping/intersection shape knife)))
(->> (array-seq (clipping/difference shape knife))
(map (fn [^js poly] (mapv outline/->ring (array-seq poly))))
(sort-by (comp - area first))
(mapv outline/join)))))
(defn- through [m pts]
(let [out (js/Float64Array. 2)]
(into [] (mapcat (fn [[x y]] (node/apply-pt! out 0 m x y) [(aget out 0) (aget out 1)]))
(partition 2 pts))))
(defn erase
"`clip` with `cutters`, in the stage pixels of symbol `open` at frame `f`,
cut out of each shape at row path in `paths`, on the frame each is showing.
The shape keeps the biggest piece, on a key at that frame — made there if
the frame had none, so the keys either side keep their points. Every other
piece is a new shape of the same colour, its id the next of `ids`. A shape
cut away entirely is deleted.
TWO SPACES COME BACK OUT, and they are not the same one. The piece the shape
KEEPS is written into that shape's own geometry, so it comes back through
`world⁻¹`, the node's own coordinates. Every other piece becomes a NEW node
beside it, whose points are read against its own fresh transform, so those come
back through `parent⁻¹` — the space a node's `pos` lives in, which is what
`paint/new-shape` takes and centres.
Both were `world⁻¹` before, and that was wrong for the new pieces by exactly
the cut shape's own transform. It could not be seen while every drawing had an
identity transform, which was true of all of them for as long as a stroke was
stored exactly as drawn: the two spaces coincided, so erasing a shape nobody had
moved worked, and erasing one somebody had moved scattered the offcuts."
[clip store open f paths cutters ids]
(first
(reduce
(fn [[clip ids] path]
(let [{:keys [sid id frame world parent]} (nest/placement clip store open path f)
n (get-in clip [:symbols sid :nodes id])
geom (get-in n [:channels paint/geometry])
inv (when world (node/invert world))
up (when parent (node/invert parent))
left (when (and inv up geom)
(cut (through world (channel/value-at geom frame store)) cutters))]
(cond
(nil? left) [clip ids]
(empty? left) [(nest/delete-node clip sid id) ids]
:else
(let [[keep & more] left
colour (channel/value-at (get-in n [:channels [:style :color]]) frame store)
kept (-> (if (contains? (:keys geom) frame) clip (paint/add-key clip sid id frame))
(paint/set-points sid id frame (through inv keep)))]
[(reduce (fn [c [nid pts]] (paint/new-shape c sid nid frame (through up pts) colour))
kept (map vector ids more))
(drop (count more) ids)]))))
[clip ids] paths)))

View file

@ -1,6 +1,6 @@
(ns arthur.domain.feature (ns arthur.domain.feature
"Tracked subjects, feature ownership, and eye-pair settings. "Tracked subjects, feature ownership, and eye-pair settings.
Features name their timeline explicitly; node ids are local to that timeline." Features name their symbol explicitly; node ids are local to that symbol."
(:require [arthur.domain.params :as params])) (:require [arthur.domain.params :as params]))
(defn owned (defn owned
@ -44,14 +44,14 @@
clip)) clip))
(defn problems (defn problems
"Check tracked identities and timeline-local node ownership." "Check tracked identities and symbol-local node ownership."
[clip] [clip]
(let [subjects (:subjects clip) (let [subjects (:subjects clip)
features (:features clip) features (:features clip)
groups (:groups clip) groups (:groups clip)
memberships (mapcat (comp :members val) groups) memberships (mapcat (comp :members val) groups)
node-owners (for [[_ f] features n (:nodes f)] node-owners (for [[_ f] features n (:nodes f)]
[(:timeline f) n])] [(:symbol f) n])]
(vec (vec
(concat (concat
(for [[id s] subjects :when (not= id (:id s))] (for [[id s] subjects :when (not= id (:id s))]
@ -60,8 +60,8 @@
:when (not (params/valid-settings? :subject (or (:params s) {})))] :when (not (params/valid-settings? :subject (or (:params s) {})))]
(str "subject " (pr-str id) " has invalid settings")) (str "subject " (pr-str id) " has invalid settings"))
(for [[id _] subjects (for [[id _] subjects
:when (not (seq (get-in clip [:timelines id :nodes :head :measured])))] :when (not (seq (get-in clip [:symbols id :nodes :head :measured])))]
(str "subject " (pr-str id) " has no measured head in its timeline")) (str "subject " (pr-str id) " has no measured head in its symbol"))
(for [[id f] features :when (not= id (:id f))] (for [[id f] features :when (not= id (:id f))]
(str "feature " (pr-str id) " has a different :id")) (str "feature " (pr-str id) " has a different :id"))
(for [[id f] features :when (not (contains? subjects (:subject f)))] (for [[id f] features :when (not (contains? subjects (:subject f)))]
@ -72,10 +72,10 @@
:when (not (params/valid-settings? (:area f) (or (:params f) {})))] :when (not (params/valid-settings? (:area f) (or (:params f) {})))]
(str "feature " (pr-str id) " has invalid settings for " (pr-str (:area f)))) (str "feature " (pr-str id) " has invalid settings for " (pr-str (:area f))))
(for [[id f] features (for [[id f] features
:when (not (contains? (:timelines clip) (:timeline f)))] :when (not (contains? (:symbols clip) (:symbol f)))]
(str "feature " (pr-str id) " names a missing timeline")) (str "feature " (pr-str id) " names a missing symbol"))
(for [[id f] features node-id (:nodes f) (for [[id f] features node-id (:nodes f)
:let [owned-nodes (get-in clip [:timelines (:timeline f) :nodes])] :let [owned-nodes (get-in clip [:symbols (:symbol f) :nodes])]
:when (not (contains? owned-nodes node-id))] :when (not (contains? owned-nodes node-id))]
(str "feature " (pr-str id) " refers to missing node " (pr-str node-id))) (str "feature " (pr-str id) " refers to missing node " (pr-str node-id)))
(for [[id n] (frequencies node-owners) :when (> n 1)] (for [[id n] (frequencies node-owners) :when (> n 1)]

View file

@ -0,0 +1,384 @@
(ns arthur.domain.gesture
"Moving, turning and scaling a node by hand on the stage, as channel values.
A drag says where the pointer went in stage pixels; this says what that makes
the node's `[:xform :pos]`, `[:xform :rot]`, `[:xform :scale]` or
`[:xform :pivot]`, given its `nest/placement`. Whatever is above the node —
instances, parents, a `:pinv` — is in the placement's matrices, so a shape five
symbols down moves under the pointer like one on top.
A GESTURE TURNS AND SCALES ABOUT THE NODE'S OWN PIVOT, and nothing here solves
for a position to fake one with. `node/local!` composes the rotation and the
scale about `[:xform :pivot]`, so a turn is `rot` alone, ALWAYS — one channel,
one key, and the pivot held exactly on every frame between two keys because the
matrix is built about it on every frame. This is Toon Boom's layer pivot and
Flash's transformation point, and the reason both store one.
A PIVOT NOBODY HAS CHOSEN IS THE MIDDLE OF WHAT THE NODE DRAWS, and the first
turn or scale WRITES IT DOWN — `pivot` derives it from `pick/bounds-of` on the
frame the drag starts, and `with-pivot` turns that into a `[:xform :pivot]` and
the `[:xform :pos]` that leaves the picture exactly where it is. After that it
is an ordinary stored, keyable, draggable channel, and the derived value is
never consulted again: a pivot is a CHOICE, and re-deriving it per drag means
editing a symbol silently moves what its instances turn about.
WHAT THIS REPLACES, because it was a deletion that cost a feature. There used
to be no pivot in the decomposition at all: a drag derived the middle of the
drawing, and `about` solved for the `pos` that holds that point still under the
new angle. That solution is an ARC in the angle while `pos` interpolates along
the CHORD, so it is exact on the frame it is written and WRONG EVERYWHERE
BETWEEN TWO KEYS. A drawing escaped it, because `paint/centred` puts a shape's
origin on the middle of what it draws and the correction is then nil — but a
SYMBOL INSTANCE cannot: its origin is its symbol's, and a symbol is drawn on the
stage, so its origin is the stage's top-left corner. One keyed turn of an
instance therefore swung its drawing round the corner of the stage on an orbit
the size of the stage. The answer at the time was a peg, and a peg is a real
thing — it is how a pivot is SHARED, or put over a measured transform — but it
is not something anybody should have to make in order to spin a drawing.
`about` is still here and is still that equation, for the one gesture that
genuinely has a pivot belonging to no node: a MULTI-SELECTION scaling about the
middle of its shared box. Nobody keys that.
The normal keying rule is `node/set-channel`, the inspector's: a channel with
keys gets one on the node's own frame, and one without has its one value
changed. Auto-key deliberately replaces that rule with `set-keyed-channel`,
so touching an otherwise static transform starts its animation at this frame."
(:require [arthur.domain.channel :as ch]
[arthur.domain.node :as node]))
(defn values
"Node `n`'s transform on its own frame `f`, as vectors.
`store` IS NOT OPTIONAL, though `ch/value-at` would let it be. A measured
transform is a dense channel, and a dense channel read without the tier-2
store it names throws — so leaving it off read correctly for every hand-placed
node and crashed the stage the moment a selection landed on an iris, a brow or
a head. Those are not hard to land on: `pick/choose` keeps a selection at the
depth it already has, so once anything inside a face is selected, an ordinary
click beside it selects its neighbour — which near the eyes is an iris."
[n f store]
(let [at #(ch/value-at (get (node/channels n) [:xform %]) f store)
xy #(let [v (at %)] [(ch/component v 0) (ch/component v 1)])]
{:pos (xy :pos) :pivot (xy :pivot) :rot (at :rot)
:scale (xy :scale) :skew (xy :skew)
;; WHETHER THE NODE HAS A PIVOT OF ITS OWN, and not what it is: a node
;; with no `[:xform :pivot]` channel reads `[0 0]` off `node/defaults`,
;; which is a real pivot — a drawing's own middle — and also what a node
;; nobody has pivoted yet looks like. The two have to be told apart
;; exactly once, when a drag decides whether to write the derived middle
;; down; see `pivot` and `with-pivot`.
:chosen? (contains? (:channels n) [:xform :pivot])}))
(defn- measured-channel? [n path]
(let [c (get-in n [:channels path])]
(boolean (or (:dense c) (:generated c)))))
(defn refusal
"Why `kind` cannot transform node `n`, or nil. Measured position and rotation
accept authored correction layers; measured scale cannot yet be decomposed
safely. The one-argument form asks whether any stage gesture is possible."
([n] (when (node/measured? n)
"its transform is measured — use a correction, or put a peg over it"))
([n kind]
(when (and (= :scale kind) (measured-channel? n [:xform :scale]))
"its scale is measured — put a peg over it and scale that")))
(defn- through [m [x y]]
(let [out (js/Float64Array. 2)]
(node/apply-pt! out 0 m x y)
[(aget out 0) (aget out 1)]))
(defn linear
"The node's own linear part, `R(rot)·K(skew)·S(scale)`, as a 2x3 whose
translation is zero — so putting a point through it applies the rotation, skew
and scale and nothing else. `node/local!` with a zero `pos` rather than a
second closed form, so there is one place the decomposition is written out."
[{:keys [rot scale skew]}]
(node/local! (node/mat) [0 0] [0 0] rot scale skew))
(defn local-of
"The node's whole local transform, `T(pos)·T(piv)·R·K·S·T(-piv)` — what takes a
point in the node's own coordinates to its parent's."
[{:keys [pos pivot rot scale skew]}]
(node/local! (node/mat) pos pivot rot scale skew))
(defn about
"The `pos` that keeps parent-space point `c` still while the node's linear part
changes from `v`'s to `v'`'s. Nil when `v`'s is singular — a node scaled to
nothing has no point under `c` to hold.
FOR A PIVOT THAT IS NO NODE'S, which since the pivot went into the
decomposition is one gesture and only one: a multi-selection scaling about the
middle of its shared box, where every member has to move to keep the
arrangement and none of them owns the point. `turn` and `scale` do not call
this, and the namespace docstring says why — the position it solves for is an
ARC in the angle and `pos` tweens along the CHORD, so it is exact on the frame
it is written and wrong between two keys.
Local is `T(t)·M` with `t = pos + a − M·a`, the composed translation. The
material point sitting under `c` is `q = M⁻¹(c − t)`, holding it there under the
new `M'` wants `t' = c − M'·q`, and the `pos` that composes to that `t'` is
p' = t' − a + M'·a = c − M'·(q − a) − a
Checkable at both ends it has to be right at: with `c` the node's own pivot
point, `c = pos + a`, so `q = a` and `p' = c − a = pos` — turning about your own
pivot never moves you. And with no pivot at all, `a = 0`, it is the familiar
`p' = c − M'·M⁻¹(c − p)`."
[v v' c]
(let [m (linear v)
l (local-of v)
t [(aget l 4) (aget l 5)]
a (:pivot v)]
(when-let [inv (node/invert m)]
(let [q (through inv (mapv - c t))]
(mapv - (mapv - c (through (linear v') (mapv - q a))) a)))))
(defn middle
"The middle of `bounds` — what the node draws, in its own coordinates — through
its own transform, so a parent-space point; or its own origin when it draws
nothing. What a node nobody has pivoted yet turns about."
[v bounds]
(let [[x0 y0 x1 y1] bounds]
(through (local-of v)
(if bounds [(/ (+ x0 x1) 2) (/ (+ y0 y1) 2)] [0 0]))))
(defn pivot
"The parent-space point a drag on this node turns and scales about: ITS OWN
PIVOT, `pos + piv` — or, for a node nobody has pivoted yet, the middle of
`bounds`, what it draws in its own coordinates, through its own transform.
THE CHOSEN ONE WINS, and that is the whole of the rule. A pivot is where
somebody put it: it does not follow the drawing afterwards, any more than
Flash's transformation point or a Harmony layer's pivot does, because a turn
that quietly changes its centre when a symbol is edited is worse than one
sitting somewhere a hand can see and move it. `ui/stage` draws the cross here
and `::ui/repivot` drags it.
THE DERIVED ONE IS A DEFAULT AND NOT A BEHAVIOUR. `pick/bounds-of` on the same
frame is where the bounds come from, so the cross and the selection box start
out as one computation — and the first turn or scale WRITES IT DOWN, which is
`with-pivot`. A `:group` draws nothing, so `bounds` is nil and this is its own
origin, which is where a peg was put.
`middle` is the derived half on its own, for `centred` — which is the way back
to it once a pivot HAS been chosen."
[v bounds]
(if (:chosen? v) (mapv + (:pos v) (:pivot v)) (middle v bounds)))
(defn at-pivot?
"Is parent-space point `c` where the node already pivots?
WITHIN A MILLIONTH OF A PIXEL, because this asks a question about intent and
gets an answer in floating point: `paint/centred` subtracts the middle of a
ring from its own points, so re-deriving that middle from the result lands on
zero to within the rounding of the subtraction, a part in 1e14 of the
coordinates. A point that close to the pivot IS the pivot — there is no gesture
in which a millionth of a pixel is a pivot somewhere else — and the whole point
of asking is to leave a drawing's channels alone: a pivot of `[0 0]` written
onto a shape that already turns about its own middle is a key nobody asked for
on a value that was already right."
[v c]
(let [[dx dy] (mapv - c (mapv + (:pos v) (:pivot v)))]
(< (js/Math.hypot dx dy) 1e-6)))
(defn repivot
"The channel values that put the node's pivot on parent-space point `c` WITHOUT
MOVING THE PICTURE, or nil when its linear part is singular.
Local is `T(pos)·T(a)·M·T(-a)`, so the new pivot has to be the material point
that is under `c` now, and the new position has to put it there:
a' = a + M⁻¹(c − (pos + a)) the point under c, in the node's own space
p' = c − a' so that p' + a' = c
and then `p' + a' + M(x − a')` is `pos + a + M(x − a)` for EVERY x, which is
the \"moving nothing\" in the first line, exact and not to a tolerance.
WITH NO ROTATION OR SCALE IT DOES NOT TOUCH `pos` AT ALL — `M = I` gives
`a' = c − pos` and `p' = pos` — which is the ordinary case of setting a pivot up
before animating, and is why this can be done quietly inside a first turn
without starting a position channel nobody asked for.
EXACT ON THE FRAME IT IS WRITTEN. `M` is this frame's, so on a node whose
rotation or scale is already keyed, the compensation that holds the picture
still here is not the one that would hold it still three frames later. That is
not an artefact of the arithmetic: moving a pivot genuinely changes what the
keyed angles mean. Flash and Harmony both let you do it and both move the
in-betweens; the alternative is refusing to repivot anything already animated,
which is the node a pivot is most often wrong on."
[v c]
(when-let [inv (node/invert (linear v))]
(let [a' (mapv + (:pivot v) (through inv (mapv - c (mapv + (:pos v) (:pivot v)))))]
{[:xform :pivot] a'
[:xform :pos] (mapv - c a')})))
(defn centred
"The channel values that put the node's pivot back on the middle of what it
draws NOW, moving nothing. Nil when its linear part is singular.
THE WAY BACK, and the thing a stored pivot needs in order to be safe to store.
A pivot does not follow the drawing — that is the point of storing it, since a
keyed spin must not be re-aimed by someone drawing one more shape inside the
symbol — but a drawing does grow, and \"put it back in the middle of what is
there now\" is then an obvious thing to want and an unobvious thing to do by
hand. It is `repivot` at the point `pivot` would have derived, so the button
and the default cannot disagree: this is exactly where an untouched node's
cross already is."
[v bounds]
(repivot v (middle v bounds)))
(defn- with-pivot
"`[v vs]`: the node's transform with its pivot on parent-space `c`, and the
channel values that put it there — or `v` untouched and nil, when that is where
it pivots already or when it has a pivot of its own.
THE ONE PLACE A DERIVED PIVOT BECOMES A STORED ONE. `turn` and `scale` both
start here, so the first drag on a node nobody has pivoted writes the middle of
what it draws down with the gesture, in the same edit, and every drag after it
turns about the stored one. The returned `v` carries the new pivot, because the
gesture itself is measured about it: scaling about the pivot it is in the act of
choosing is one answer, not two."
[v c]
(if (and c (not (:chosen? v)) (not (at-pivot? v c)))
(if-let [vs (repivot v c)]
[(assoc v :pivot (get vs [:xform :pivot]) :pos (get vs [:xform :pos])) vs]
[v nil])
[v nil]))
(defn move
"The node's position with the drag carried from stage point `p0` to `p1`.
ONE CHANNEL, AND IT NEVER DISTURBS THE PIVOT: `[:xform :pivot]` is in the
node's own coordinates, so it travels with the node and a move is `pos` alone,
exactly as it was before there was a pivot at all."
[{:keys [parent]} {:keys [pos]} p0 p1]
(when-let [inv (node/invert parent)]
{[:xform :pos] (mapv + pos (mapv - (through inv p1) (through inv p0)))}))
(defn angle
"The angle of stage point `p` about parent-space pivot `c`, in the space the
node's rotation is in."
[{:keys [parent]} c p]
(when-let [inv (node/invert parent)]
(let [[x y] (mapv - (through inv p) c)]
(js/Math.atan2 y x))))
(defn turn
"The node turned by `da` radians about parent-space point `c`: the rotation, and
— only on the first turn of a node nobody has pivoted — the pivot and position
that put its pivot on `c` without moving it.
ONE CHANNEL, ONCE THE PIVOT IS ITS OWN, and that is the whole reason the pivot
is in `node/local!` rather than solved for here. `rot` is the only thing a turn
changes, so a keyed turn interpolates ONE number and the matrix is composed
about the pivot on every frame of it: the pivot is held exactly between two keys
and not merely at them. A `pos` written beside the rotation — which is what
`about` solves for, and what this used to do for anything whose origin was not
its middle — tweens along the chord of an arc it has no way to know about, and a
360° turn keyed that way leaves the stage in the middle and comes back.
`c` IS A DEFAULT, NOT A TARGET. A node with its own pivot ignores it and turns
about what it has, which is what makes a pivot something a hand can place and
rely on; `pivot` is where `c` comes from either way."
[v c da]
(let [[v vs] (with-pivot v c)]
(merge vs {[:xform :rot] (+ (:rot v) da)})))
(defn scale
"The node's scale with the point under stage `p0` taken to `p1`, about its own
pivot, along the node's own axes — or by the same factor on both when
`uniform?` — and, on the first scale of a node nobody has pivoted, the pivot and
position that put its pivot on `c` without moving it.
The factors are measured in the node's OWN coordinates, which is what makes a
corner drag track the pointer on a node that has been turned, and they are
measured FROM THE PIVOT `q`: scaling by `k` takes `q + d` to `q + k·d`, so the
factor a corner wants is the ratio of its offsets from the pivot before and
after. `with-pivot` runs first because a pivot being chosen by this very drag is
the pivot the drag has to be measured about."
[{:keys [world]} v c p0 p1 uniform?]
(let [[v vs] (with-pivot v c)]
(when-let [winv (node/invert world)]
(let [q (:pivot v)
a (mapv - (through winv p0) q)
b (mapv - (through winv p1) q)
k (fn [a b] (if (< (js/Math.abs a) 1e-6) 1 (/ b a)))
r (when uniform?
(let [aa (reduce + (map * a a))]
(if (< aa 1e-9) 1 (/ (reduce + (map * a b)) aa))))
s (:scale v)
s' (if r (mapv #(* r %) s) (mapv * s (map k a b)))]
(merge vs {[:xform :scale] s'})))))
(defn scale-by
"The node scaled by factor `k` on both axes about parent-space point `c`, and
the position that holds `c` still.
What a MULTI-SELECTION scales by: one factor for everything about the shared
box, so a group of shapes keeps its arrangement instead of each member solving
for its own factors. The point belongs to the box and not to any node in it, so
this is the one gesture that still writes a position to hold a pivot still —
`about`, with everything its docstring says that costs. `scale` is the
single-node form, where the factors come out of the pointer in the node's own
axes and the pivot is the node's own."
[v c k]
(let [v' (update v :scale #(mapv (partial * k) %))]
(cond-> {[:xform :scale] (:scale v')}
c (into (when-let [p (about v v' c)] {[:xform :pos] p})))))
(def ^:private manual-layer ::manual-transform)
(defn- plus [a b]
(if (number? a) (+ a b) (mapv + (vec a) (vec b))))
(defn- minus [a b]
(if (number? a) (- a b) (mapv - (vec a) (vec b))))
(defn- corrected-channel
"Set the effective value of a measured channel without replacing its base."
[channel f target store extent auto-key?]
(let [current (ch/value-at channel f store)
layers (vec (:over channel))
i (first (keep-indexed #(when (= manual-layer (:id %2)) %1) layers))
old (when i (ch/value-at (get-in layers [i :values]) f store))
zero (if (number? target) 0 (vec (repeat (count target) 0)))
value (plus (or old zero) (minus target current))
values (or (when i (get-in layers [i :values])) (ch/framed zero))
values ((if auto-key? node/set-keyed-channel node/set-channel)
{:kind :group :channels {[:xform :pos] values}}
[:xform :pos] f value)
values (get-in values [:channels [:xform :pos]])
layer (ch/layer manual-layer [0 extent] :offset values)]
(assoc channel :over (if i (assoc layers i layer) (conj layers layer)))))
(defn apply-values
"Clip with channel values `vs`, `{path value}`, written into node `id` of
symbol `sid` on the node's own frame `f`, by the keying rule above. The sixth
argument arms auto-key; the five-argument form retains the normal rule."
([clip sid id f vs] (apply-values clip sid id f vs false nil))
([clip sid id f vs auto-key?] (apply-values clip sid id f vs auto-key? nil))
([clip sid id f vs auto-key? store]
(let [put (if auto-key? node/set-keyed-channel node/set-channel)]
(update-in clip [:symbols sid :nodes id]
#(reduce-kv
(fn [n path v]
(if (measured-channel? n path)
(update-in n [:channels path] corrected-channel f v store
(get-in clip [:symbols sid :frames]) auto-key?)
(put n path f v)))
% vs)))))
(defn apply-take
"Apply a buffered performance take. `take` is keyed by `[symbol node]`, then
local frame, then channel path. It becomes ordinary authored keys in one
document edit rather than making the edit pipeline run for every sample."
([clip take] (apply-take clip take nil))
([clip take store]
(reduce-kv
(fn [c [sid id] frames]
(reduce-kv (fn [c f values]
(apply-values c sid id f values true store))
c frames))
clip take)))

View file

@ -0,0 +1,130 @@
(ns arthur.domain.history
"Undo, per person, as leaf writes. docs/architecture.md, \"Undo is per-user\".
A step is the leaves one edit changed: what they held before, and what they
held after. Undoing writes the befores back as an ordinary edit, which the
next save sends like any other — so undo needs nothing from the server, and
nothing about it is shared.
ONLY YOUR OWN CHANGES. A step undoes only if every leaf it touched still holds
what the step left there. Somebody else's write to one of them since — their
edit to the shape you made — refuses the step rather than taking their work
with it; it is dropped, and the next undo is the step before. With nobody else
in the document the values always match, and this is ordinary undo.
A nil value is an absent leaf: a step that made a node has nil befores for its
leaves, so undoing it removes them."
(:require [clojure.string :as str]))
(def gap-ms
"Edits to the same leaves closer together than this are one step: a drag
writes a vertex per pointermove, and is one thing to undo. Only when each
starts where the last left off — anything landing between them, a
collaborator's write included, makes the next edit a step of its own."
1000)
(def depth 200)
(defn- changes
"`[before after]`, restricted to the paths that differ."
[before after]
(reduce (fn [[b a :as acc] path]
(let [x (get before path)
y (get after path)]
(if (= x y) acc [(assoc b path x) (assoc a path y)])))
[{} {}]
(distinct (concat (keys before) (keys after)))))
(defn- node-name [leaves path]
(let [[_ _ _ sid _ nid] (str/split path #"/")
node (get leaves (str/join "/" ["clip" "u" "symbol" sid "node" nid]))]
(or (:name node) (str/replace nid "~" "/"))))
(defn- said
"What one changed leaf was, in words, and how much it outranks the others:
making or deleting a thing names the step before editing it does."
[before after path]
(let [[_ _ kind a b] (str/split path #"/")
leaves (merge before after)]
(case [kind b]
["symbol" nil] [1 (str "symbol " (or (:name (get leaves path)) (str/replace a "~" "/")))]
["symbol" "node"]
(cond (nil? (get before path)) [0 (str "add " (node-name leaves path))]
(nil? (get after path)) [0 (str "delete " (node-name leaves path))]
:else [1 (str "edit " (node-name leaves path))])
(if (#{"channel" "measured"} b)
[1 (str "edit " (node-name leaves path))]
[2 (case kind
("timing" "stage" "name") "project settings"
("subject" "feature" "group") "tracking settings"
kind)]))))
(defn label
"A step in words: \"add shape 3\", \"edit mouth, brow-l\"."
[before after paths]
(let [said (->> paths (map #(said before after %)) distinct sort)
top (first (first said))
words (distinct (map second (filter #(= top (first %)) said)))]
(str (str/join ", " (take 2 words)) (when (< 2 (count words)) " …"))))
(defn record
"History `h` with an edit from leaves `before` to `after` at time `now`."
[{:keys [done held?] :as h} before after now]
(let [[b a] (changes before after)
top (peek done)]
(cond
(empty? a) h
(and top (not (:closed? top)) (= b (:after top))
(or held? (< (- now (:at top)) gap-ms)))
(assoc h :done (conj (pop done) (assoc top :after a :at now)) :undone [])
:else
(assoc h
:done (conj (vec (take-last (dec depth) done))
{:before b :after a :at now :label (label before after (keys a))})
:undone []))))
(defn- close [{:keys [done] :as h}]
(cond-> h (seq done) (assoc :done (conj (pop done) (assoc (peek done) :closed? true)))))
(defn hold
"While a field has focus, everything typed into it is one step, however slowly
— the digits of 45 are seen as 4 and then 45, and undone as one. It starts a
step of its own rather than joining whatever came before."
[h]
(assoc (close h) :held? true))
(defn settle
"The field is done with: its step is finished, and nothing joins it."
[h]
(dissoc (close h) :held?))
(defn steps
"The labels, newest first: `:done` is what undo would take off, `:undone`
what redo would put back."
[h]
{:done (mapv :label (rseq (or (:done h) [])))
:undone (mapv :label (rseq (or (:undone h) [])))})
(defn- holds? [leaves m]
(every? (fn [[path v]] (= v (get leaves path))) m))
(defn- put-all [leaves m]
(reduce-kv (fn [ls path v] (if (nil? v) (dissoc ls path) (assoc ls path v))) leaves m))
(defn- move
"One step from `from` to `to`, if `leaves` still hold what it expects."
[h leaves from to expect write]
(when-let [step (peek (get h from))]
(let [h (update h from pop)]
(if (holds? leaves (expect step))
{:leaves (put-all leaves (write step)) :history (update h to (fnil conj []) step)}
{:blocked step :history h}))))
(defn undo
"`{:leaves :history}`, `{:blocked :history}` when somebody else has since
changed what the step touched, or nil with nothing to undo."
[h leaves]
(move h leaves :done :undone :after :before))
(defn redo [h leaves]
(move h leaves :undone :done :before :after))

View file

@ -0,0 +1,37 @@
(ns arthur.domain.keyframes
(:require [arthur.domain.channel :as ch]))
(defn identity-of [k] (select-keys k [:sid :id :channel :frame]))
(defn shifted [items delta]
(let [delta (max delta (reduce max js/Number.NEGATIVE_INFINITY (map #(max (- (:at %)) (- (* (:frame %) (:scale %)))) items)))]
(mapv (fn [k]
(let [f (max 0 (js/Math.round (+ (:frame k) (/ delta (:scale k)))))]
(assoc k :frame f :at (+ (:at k) (* (:scale k) (- f (:frame k))))))) items)))
(defn edit-keys [document items delta]
(let [targets-by-id (when (some? delta)
(into {} (map vector (map identity-of items) (shifted items delta))))]
(reduce
(fn [doc [[sid id channel] selected]]
(let [path [:symbols sid :nodes id :channels channel]
c (get-in doc path)
selected (filter #(contains? (:keys c) (:frame %)) selected)
targets (when (some? delta) (mapv #(get targets-by-id (identity-of %)) selected))
remaining (apply dissoc (:keys c) (map :frame selected))
ks (if targets
(reduce (fn [ks [old new]] (assoc ks (:frame new) (get (:keys c) (:frame old))))
remaining (map vector selected targets))
remaining)
segments (apply dissoc (:segments c) (concat (map :frame selected) (map :frame targets)))
segments (if targets
(reduce (fn [s [old new]]
(if (contains? (:segments c) (:frame old))
(assoc s (:frame new) (get (:segments c) (:frame old))) s))
segments (map vector selected targets)) segments)]
(if (empty? selected) doc
(assoc-in doc path
(if (seq ks)
(cond-> (assoc c :keys ks) (:segments c) (assoc :segments segments))
(ch/framed (get (:keys c) (:frame (last selected)))))))))
document (group-by (juxt :sid :id :channel) (vals (into {} (map (juxt identity-of identity) items)))))))

View file

@ -10,22 +10,24 @@
clip/<cid>/name a label clip/<cid>/name a label
clip/<cid>/timing fps clip/<cid>/timing fps
clip/<cid>/stage width, height clip/<cid>/stage width, height
clip/<cid>/palette/<pid> a named indexed palette asset
clip/<cid>/palette-default the project fallback palette id
clip/<cid>/root the symbol the document opens on
clip/<cid>/source the analysis record this came out of clip/<cid>/source the analysis record this came out of
clip/<cid>/subject/<sid> a tracked subject and its params clip/<cid>/subject/<subj> a tracked subject and its params
clip/<cid>/feature/<fid> one feature: area, nodes, params clip/<cid>/feature/<fid> one feature: area, nodes, params
clip/<cid>/group/<gid> an eye pair and its shared params clip/<cid>/group/<gid> an eye pair and its shared params
clip/<cid>/timeline/<tid> frames, and a palette one day clip/<cid>/symbol/<sid> native frames and fps, optional palette
clip/<cid>/timeline/<tid>/node/<nid> kind, parent, stencil, z, time clip/<cid>/symbol/<sid>/node/<nid> kind, parent, stencil, z, time
clip/<cid>/timeline/<tid>/channel/<nid>/<prop> clip/<cid>/symbol/<sid>/channel/<nid>/<prop>
clip/<cid>/timeline/<tid>/measured/<nid> the channels a re-freeze owns clip/<cid>/symbol/<sid>/measured/<nid> the channels a re-freeze owns
WHY NODES SIT UNDER A TIMELINE. A clip holds a library of timelines. Its root WHY NODES SIT UNDER A SYMBOL. A clip holds a library of symbols and each has
and each symbol have their own nodes, so the timeline id is a path segment. its own nodes, so the symbol id is a path segment. No symbol has a reserved
The root is `main`, and a symbol's nodes use the same path shape. segment: `main` in a path is an id like any other.
`:frames` MOVED OFF `timing` onto the timeline. A timeline is a frame space and a The timing leaf holds output fps. Each symbol leaf holds its native fps
clip is a rate, so `timing` holds `:fps` alone. Both used to be in one leaf, which and frame count, so changing the output grid leaves content untouched.
is how a nested timeline's length would have had nowhere to go.
WHY THESE BOUNDARIES. Last-writer-wins only clobbers when its unit is too big, WHY THESE BOUNDARIES. Last-writer-wins only clobbers when its unit is too big,
so the cut is chosen so that the things people do simultaneously land on so the cut is chosen so that the things people do simultaneously land on
@ -44,9 +46,10 @@
WHY `measured` IS ONE LEAF AND CHANNELS ARE NOT. `:head`'s measured channels are WHY `measured` IS ONE LEAF AND CHANNELS ARE NOT. `:head`'s measured channels are
not authored: they are written together by a freeze and replaced together by a not authored: they are written together by a freeze and replaced together by a
re-freeze, and `head-mode` exposes them through `:channels`. The optional re-freeze, and `head-mode` exposes them through `:channels`. The same is true
`:anchors` map on the head node chooses which measured frame those channels of a face's `:plate`, whose measured channels register its footage. Which
read. A leaf per measured measured frame they read is the node's `:reads` and `:time :holds`, which
save with the node. A leaf per measured
channel would offer a write nobody can make. The authored channels beside them channel would offer a write nobody can make. The authored channels beside them
are one leaf each, because a hand writes one at a time. are one leaf each, because a hand writes one at a time.
@ -56,7 +59,7 @@
it is one character rather than a scheme." it is one character rather than a scheme."
(:require [arthur.domain.clip :as clip] (:require [arthur.domain.clip :as clip]
[arthur.domain.sha256 :as sha] [arthur.domain.sha256 :as sha]
[arthur.domain.timeline :as timeline] [arthur.domain.symbol :as symbol]
[clojure.string :as str])) [clojure.string :as str]))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
@ -124,46 +127,51 @@
(when (seq unknown) (when (seq unknown)
(throw (ex-info "the clip has a field with no leaf to save it in; see arthur.domain.clip/clip-keys" (throw (ex-info "the clip has a field with no leaf to save it in; see arthur.domain.clip/clip-keys"
{:unknown (vec (sort-by str unknown))})))) {:unknown (vec (sort-by str unknown))}))))
(doseq [[id tl] (:timelines clip)] (doseq [[id sym] (:symbols clip)]
(let [unknown (remove timeline/timeline-keys (keys tl))] (let [unknown (remove symbol/symbol-keys (keys sym))]
(when (seq unknown) (when (seq unknown)
(throw (ex-info "a timeline has a field with no leaf to save it in; see arthur.domain.timeline/timeline-keys" (throw (ex-info "a symbol has a field with no leaf to save it in; see arthur.domain.symbol/symbol-keys"
{:timeline id :unknown (vec (sort-by str unknown))}))))) {:symbol id :unknown (vec (sort-by str unknown))})))))
(let [at (fn [& parts] (str/join "/" (into ["clip" (segment cid)] parts))) (let [at (fn [& parts] (str/join "/" (into ["clip" (segment cid)] parts)))
some-leaf (fn [path v] (when (seq v) {path v}))] some-leaf (fn [path v] (when (seq v) {path v}))]
(apply merge (apply merge
(some-leaf (at "name") (select-keys clip [:name])) (some-leaf (at "name") (select-keys clip [:name]))
(some-leaf (at "timing") (select-keys clip [:fps])) (some-leaf (at "timing") (select-keys clip [:fps]))
(some-leaf (at "stage") (select-keys clip [:width :height])) (some-leaf (at "stage") (select-keys clip [:width :height]))
(some-leaf (at "source") (:analysis clip)) (some-leaf (at "palette-default") (select-keys clip [:default-palette]))
(some-leaf (at "root") (select-keys clip [:root]))
(some-leaf (at "analyses") (:analyses clip))
(concat (concat
(for [[id v] (:subjects clip)] {(at "subject" (segment id)) v}) (for [[id v] (:subjects clip)] {(at "subject" (segment id)) v})
(for [[id v] (:features clip)] {(at "feature" (segment id)) v}) (for [[id v] (:features clip)] {(at "feature" (segment id)) v})
(for [[id v] (:groups clip)] {(at "group" (segment id)) v}) (for [[id v] (:groups clip)] {(at "group" (segment id)) v})
;; The timeline's own facts. `:id` is the path segment, so writing it (for [[id v] (:palettes clip)] {(at "palette" (segment id)) v})
;; The symbol's own facts. `:id` is the path segment, so writing it
;; into the value as well would be the one field a rename could ;; into the value as well would be the one field a rename could
;; disagree with itself about; `clip` puts it back. ;; disagree with itself about; `clip` puts it back.
(for [[tid tl] (:timelines clip)] (for [[sid sym] (:symbols clip)]
{(at "timeline" (segment tid)) {(at "symbol" (segment sid))
(select-keys tl [:frames :palette])}) (select-keys sym [:name :frames :fps :width :height :palette
(for [[tid tl] (:timelines clip) :palette-track :palette-channel :type :palette-ref :display
[id n] (:nodes tl)] :media])})
{(at "timeline" (segment tid) "node" (segment id)) (for [[sid sym] (:symbols clip)
[id n] (:nodes sym)]
{(at "symbol" (segment sid) "node" (segment id))
(apply dissoc n node-channel-keys)}) (apply dissoc n node-channel-keys)})
(for [[tid tl] (:timelines clip) (for [[sid sym] (:symbols clip)
[id n] (:nodes tl) [id n] (:nodes sym)
:when (seq (:measured n))] :when (seq (:measured n))]
{(at "timeline" (segment tid) "measured" (segment id)) (:measured n)}) {(at "symbol" (segment sid) "measured" (segment id)) (:measured n)})
(for [[tid tl] (:timelines clip) (for [[sid sym] (:symbols clip)
[id n] (:nodes tl) [id n] (:nodes sym)
[prop ch] (:channels n)] [prop ch] (:channels n)]
{(at "timeline" (segment tid) "channel" (segment id) (prop->path prop)) ch}))))) {(at "symbol" (segment sid) "channel" (segment id) (prop->path prop)) ch})))))
(defn clip (defn clip
"The inverse of `leaves`, for one clip. Paths belonging to another clip are "The inverse of `leaves`, for one clip. Paths belonging to another clip are
ignored, so a project's whole leaf map can be handed straight in. ignored, so a project's whole leaf map can be handed straight in.
A timeline's `:id` is restored from its path segment rather than read out of the A symbol's `:id` is restored from its path segment rather than read out of the
value, which is why `leaves` does not write it: a segment and a field that both value, which is why `leaves` does not write it: a segment and a field that both
claim to be the id are two places for one fact." claim to be the id are two places for one fact."
[cid leaves] [cid leaves]
@ -173,14 +181,16 @@
(let [[_ found kind a b c] (str/split path #"/")] (let [[_ found kind a b c] (str/split path #"/")]
(if-not (= want found) (if-not (= want found)
acc acc
(if (= "timeline" kind) (if (= "symbol" kind)
(let [tid (unsegment a) (let [sid (unsegment a)
acc (assoc-in acc [:timelines tid :id] tid)] acc (assoc-in acc [:symbols sid :id] sid)]
(case b (case b
nil (update-in acc [:timelines tid] merge v) ;; `:nodes` is there before any node leaf is: an empty
"node" (update-in acc [:timelines tid :nodes (unsegment c)] merge v) ;; symbol has none, and is still a symbol.
"measured" (assoc-in acc [:timelines tid :nodes (unsegment c) :measured] v) nil (update-in acc [:symbols sid] #(merge {:nodes {}} % v))
"channel" (assoc-in acc [:timelines tid :nodes (unsegment c) "node" (update-in acc [:symbols sid :nodes (unsegment c)] merge v)
"measured" (assoc-in acc [:symbols sid :nodes (unsegment c) :measured] v)
"channel" (assoc-in acc [:symbols sid :nodes (unsegment c)
:channels (path->prop (nth (str/split path #"/") 6))] :channels (path->prop (nth (str/split path #"/") 6))]
v) v)
(throw (ex-info "not a leaf path" {:path path})))) (throw (ex-info "not a leaf path" {:path path}))))
@ -188,7 +198,10 @@
"name" (merge acc v) "name" (merge acc v)
"timing" (merge acc v) "timing" (merge acc v)
"stage" (merge acc v) "stage" (merge acc v)
"source" (assoc acc :analysis v) "analyses" (assoc acc :analyses v)
"palette-default" (merge acc v)
"root" (merge acc v)
"palette" (assoc-in acc [:palettes (unsegment a)] v)
"subject" (assoc-in acc [:subjects (unsegment a)] v) "subject" (assoc-in acc [:subjects (unsegment a)] v)
"feature" (assoc-in acc [:features (unsegment a)] v) "feature" (assoc-in acc [:features (unsegment a)] v)
"group" (assoc-in acc [:groups (unsegment a)] v) "group" (assoc-in acc [:groups (unsegment a)] v)
@ -210,27 +223,27 @@
content-addressed is that it does not have to travel with tier 1 to be found." content-addressed is that it does not have to travel with tier 1 to be found."
[leaves] [leaves]
(let [parts (into {} (map (juxt identity #(vec (str/split % #"/")))) (keys leaves)) (let [parts (into {} (map (juxt identity #(vec (str/split % #"/")))) (keys leaves))
;; A node leaf, by (clip, timeline, node). Under a timeline id, because a ;; A node leaf, by (clip, symbol, node). Under a symbol id, because two
;; symbol and the root may both hold a `:mouth` and a channel of one is not ;; symbols may both hold a `:mouth` and a channel of one is not a channel
;; a channel of the other. ;; of the other.
nodes (into #{} (keep (fn [[_ p]] nodes (into #{} (keep (fn [[_ p]]
(when (and (= 6 (count p)) (= "timeline" (nth p 2)) (when (and (= 6 (count p)) (= "symbol" (nth p 2))
(= "node" (nth p 4))) (= "node" (nth p 4)))
[(nth p 1) (nth p 3) (nth p 5)]))) [(nth p 1) (nth p 3) (nth p 5)])))
parts) parts)
;; Which segment index holds the kind, and what shapes are legal. ;; Which segment index holds the kind, and what shapes are legal.
legal? (fn [p] legal? (fn [p]
(and (= "clip" (first p)) (second p) (and (= "clip" (first p)) (second p)
(if (= "timeline" (nth p 2 nil)) (if (= "symbol" (nth p 2 nil))
(case (count p) (case (count p)
4 true ; the timeline itself 4 true ; the symbol itself
6 (#{"node" "measured"} (nth p 4)) 6 (#{"node" "measured"} (nth p 4))
7 (= "channel" (nth p 4)) 7 (= "channel" (nth p 4))
false) false)
(case (count p) (case (count p)
;; The clip's own facts carry no id. ;; The clip's own facts carry no id.
3 (#{"name" "timing" "stage" "source"} (nth p 2)) 3 (#{"name" "timing" "stage" "analyses" "palette-default" "root"} (nth p 2))
4 (#{"subject" "feature" "group"} (nth p 2)) 4 (#{"subject" "feature" "group" "palette"} (nth p 2))
false))))] false))))]
(vec (vec
(concat (concat
@ -238,13 +251,13 @@
:when (not (legal? p))] :when (not (legal? p))]
(str (pr-str path) " is not a leaf path")) (str (pr-str path) " is not a leaf path"))
(for [[path p] (sort-by key parts) (for [[path p] (sort-by key parts)
:when (and (legal? p) (= "timeline" (nth p 2 nil)) (>= (count p) 6) :when (and (legal? p) (= "symbol" (nth p 2 nil)) (>= (count p) 6)
(#{"channel" "measured"} (nth p 4)) (#{"channel" "measured"} (nth p 4))
(not (contains? nodes [(nth p 1) (nth p 3) (nth p 5)])))] (not (contains? nodes [(nth p 1) (nth p 3) (nth p 5)])))]
(str (pr-str path) " addresses a node with no node leaf")) (str (pr-str path) " addresses a node with no node leaf"))
(for [[path p] (sort-by key parts) (for [[path p] (sort-by key parts)
:let [v (get leaves path)] :let [v (get leaves path)]
:when (and (legal? p) (= "timeline" (nth p 2 nil)) (= 7 (count p)) :when (and (legal? p) (= "symbol" (nth p 2 nil)) (= 7 (count p))
(:dense v) (not (sha/key? (:store (:dense v)))))] (:dense v) (not (sha/key? (:store (:dense v)))))]
(str (pr-str path) " names tier 2 as " (pr-str (:store (:dense v))) (str (pr-str path) " names tier 2 as " (pr-str (:store (:dense v)))
" — a dense channel in a saved document names a content address")))))) " — a dense channel in a saved document names a content address"))))))

View file

@ -0,0 +1,718 @@
(ns arthur.domain.nest
"How nested symbols relate, and moving things between them.
A row path — the ids from the open symbol down through instances, as the
timeline names a row — says where something is. Walking one answers three
questions at once, which is why there is one walk: what frame is showing down
there, what matrix takes its coordinates up to the open symbol's, and what
time map takes the open symbol's frames down to its own.
ONE NESTING, AS FAR AS A PERSON IS CONCERNED. Putting a node inside another
symbol is how things are grouped: the symbol is a shared timeline, and its
instance is the handle that moves, retimes and transforms everything in it
together. Parent pointers inside a symbol stay — the roto rig is built on them
— but they are not something the timeline hands out.
A MOVE CHANGES NEITHER THE PICTURE NOR THE TIMING. Every node has the same two
maps into its parent — the matrix of its transform, and `node/time-of` — and a
move keeps a node's world maps and re-expresses them under the new parent: the
matrix becomes a `:pinv`, Blender's parent-inverse, and the time a new `:at`
and `:rate`. Its channels, keys and span are untouched."
(:require [arthur.domain.channel :as ch]
[arthur.domain.clip :as clip]
[arthur.domain.gesture :as gesture]
[arthur.domain.node :as node]
[arthur.domain.palette :as pal]
[arthur.domain.pick :as pick]
[arthur.domain.span :as span]
[arthur.domain.symbol :as symbol]))
(defn- resolved
"Node `id` of symbol `sid`, resolved at `frame`: the resolver, which then
answers `symbol/world-of` and `symbol/frame-of` for it on that frame.
Only its lineage is resolved, because where a node is depends on its parents
and nothing else in the symbol — and the whole symbol costs more than a frame
of the stage, which the editor asks for on every frame."
[clip store sid frame id]
(let [sym (clip/symbol clip sid)
sym (update sym :nodes select-keys (symbol/lineage (:nodes sym) id))
r (symbol/resolver sym store pal/index-of nil)]
(r frame)
r))
(defn inside
"Walk row path `path` down from symbol `sid`, whose OWN frame `f` is showing,
into the node it ends at. Returns `{:sid :frame :matrix :time}`: the symbol that
node places (nil for one that places none), the frame of its own it is
showing, the matrix from its coordinates to `sid`'s, and the time map from
`sid`'s frames to its own — or nil when a node on the way is not on screen at
that frame, where there is no inside to be in.
`f` IS ONE OF `sid`'S OWN FRAMES, which is the coordinate everything authored
and every editing gesture is in — see `docs/one-grid-plan.md`. It used to be an
OUTPUT frame, multiplied into `sid`'s space here, which made the units of the
argument something each caller had to know without being told and put a
2.5-frame step between what the ruler offered and what the document could hold.
A caller holding the playhead converts with `clip/shown-frame`.
THE SAME STEP FOR EVERY NODE. Inside an instance is the symbol it places;
inside a shape is where its points and keys are. Either way it is the node's
own coordinates and frames, so a shape any depth down is edited through the
maps it is drawn with.
The frame and the matrix come from RESOLVING each level, so they are the ones
the stage draws with, floors included. The time map is the affine part, floors
aside, and is nil through a looping node, whose frames come round again and do
not map one to one."
[clip store sid path f]
(reduce (fn [{:keys [sid frame matrix time]} id]
(let [r (resolved clip store sid frame id)
nodes (:nodes (clip/symbol clip sid))
chain (map #(get nodes %) (rseq (symbol/lineage nodes id)))
m (symbol/world-of r id)
local (symbol/frame-of r id)
inst? (= :instance (:kind (get nodes id)))
;; WHICH symbol, and which frame of it, are both read off the
;; cel: inside a held cel is its drawing on the frame
;; the hold pins, not on `local`, and inside a playing insert
;; is its animation at its own in-point and speed.
shown (when (number? local)
(clip/placed-frame clip sid (get nodes id) local))
inner (:symbol shown)
lf (if shown (:frame shown) local)]
(if (and m (number? local)
;; A clip over a gap has no inside to be in.
(or (not inst?) shown)
(or (nil? inner) (< -1 lf (clip/frames clip inner))))
{:sid inner :frame (js/Math.floor lf)
:matrix (node/mul! (node/mat) matrix m)
;; Forward sampling above works for holds too. :time is the
;; invertible edit map; source in-points and speeds belong in it.
:time (when (and time (not-any? #(get-in % [:time :loop?]) chain)
(or (not inst?) (clip/source-time clip sid (get nodes id))))
(cond-> (reduce node/then-time time (map node/time-of chain))
inst? (node/then-time (clip/source-time clip sid (get nodes id)))))}
(reduced nil))))
{:sid sid :frame (js/Math.floor f)
:matrix (node/mat) :time node/same-time}
path))
(defn own-time
"The time map from symbol `sid`'s frames to the OWN frames of the node at row
path `path` from it, with `sid` showing frame `f`: the frames its keys and
holds are written in. Nil where no affine map exists — through a loop or a
floor above the node, or where it is not on screen.
It STOPS AT THE NODE, where `inside` goes into what the node places, and it
leaves out the node's own floors: a hold is added on a frame the node's holds
would otherwise floor away."
[clip store sid path f]
(when-let [{inner :sid t :time} (inside clip store sid (pop path) f)]
(let [nodes (:nodes (clip/symbol clip inner))
n (get nodes (peek path))
up (symbol/frame-map nodes (:parent n))]
(when (and n t up)
(-> t (node/then-time up) (node/then-time (node/time-of n)))))))
(defn placement
"Where the node at row path `path` is, from symbol `sid` showing frame `f`, as
a transform would change it: `{:sid :id :frame :parent :world}` — the symbol it
lives in, its own frame, the matrix from the space its `[:xform :pos]` is in to
`sid`'s, and the one from its own coordinates. Nil when it is not on screen.
`:parent` is everything above the node's own transform, `world = parent ·
local`: its symbol's way to the stage, its parents there, and its `:pinv`."
[clip store sid path f]
(when-let [{:keys [sid frame matrix]} (inside clip store sid (pop path) f)]
(let [id (peek path)
n (get-in clip [:symbols sid :nodes id])
r (when n (resolved clip store sid frame id))
w (when r (symbol/world-of r id))]
(when w
{:sid sid :id id
:frame (js/Math.floor (symbol/frame-of r id))
:parent (reduce #(node/mul! (node/mat) %1 %2) matrix
(keep identity [(some->> (:parent n) (symbol/world-of r))
(node/pinv n)]))
:world (node/mul! (node/mat) matrix w)}))))
(defn drawn-inside
"Flat points drawn on symbol `sid`'s stage at frame `f`, re-expressed inside the
symbol `path` leads to, so a shape added there lands exactly where it was drawn.
`{:sid :frame :pts}`, or nil where `inside` finds nothing to be inside."
[clip store sid path f pts]
(when-let [{:keys [matrix] :as at} (inside clip store sid path f)]
(when-let [inv (node/invert matrix)]
(let [out (js/Float64Array. 2)]
(assoc (select-keys at [:sid :frame])
:pts (into [] (mapcat (fn [[x y]]
(node/apply-pt! out 0 inv x y)
[(aget out 0) (aget out 1)]))
(partition 2 pts)))))))
(defn audio-tracks
"Flatten audible source intervals through cel and parent clocks.
A held visual source is silent. Every returned track carries a source offset,
an output interval, and automation mapped into the open symbol's time.
Automatically associated face audio can be reached more than once in a
multi-face take. Equal `:media-link`s at the same source and output interval
are one recording, not a louder mix; at different placements they remain
separate scheduled clips."
[clip sid]
(letfn [(to-local [m f] (* (:rate m) (- f (:at m))))
(to-outer [m f] (+ (:at m) (/ f (:rate m))))
(window [m span bounds]
(if span
[(max (first bounds) (to-outer m (first span)))
(min (second bounds) (to-outer m (second span)))]
bounds))
(channels [chs m]
(into {}
(map (fn [[p c]]
[p (cond-> c
(:keys c) (update :keys #(into {} (map (fn [[f v]] [(to-outer m f) v])) %))
(:segments c) (update :segments #(into {} (map (fn [[f v]] [(to-outer m f) v])) %))
(:dense c) (assoc :sample-time m))]))
chs))
(walk [sid outer bounds path seen]
(when (contains? seen sid)
(throw (ex-info "symbol cycle in audio" {:symbol sid})))
(let [sym (clip/symbol clip sid)
nodes (:nodes sym)
bounds (window outer [0 (:frames sym)] bounds)
positions (reduce
(fn [acc id]
(let [n (get nodes id)
parent (if-let [pid (:parent n)] (get acc pid)
{:time outer :bounds bounds})
m (node/then-time (:time parent) (node/time-of n))
m (if (= :audio (:kind n))
(update m :rate * (/ (or (get-in n [:source :fps]) (clip/fps clip sid))
(clip/fps clip sid))) m)]
(assoc acc id {:time m :bounds (window m (:span n) (:bounds parent))})))
{} (symbol/order nodes))]
(mapcat
(fn [[id n]]
(let [{m :time [lo hi] :bounds} (get positions id)]
(when (< lo hi)
(case (:kind n)
:audio [(-> n
(assoc :parent nil :path (conj path id) :owner sid
:fps (or (get-in n [:source :fps]) (clip/fps clip sid))
:span [(to-local m lo) (to-local m hi)]
:time (merge (:time n) {:mode :map :at (:at m) :rate (:rate m) :offset 0})
:channels (channels (:channels n) m)))]
:instance
(let [{:keys [in speed end]} (node/playback-of n)
child (node/source n)
length (clip/frames clip child)]
;; A visual freeze does not emit a sustained audio sample.
(when (and child length (pos? speed))
(let [source (node/then-time m (clip/source-time clip sid
(-> n
(assoc-in [:playback :end] :stop)
(update :time dissoc :loop?))))
loop? (or (= end :loop) (get-in n [:time :loop?]))
periods (if loop?
(range (js/Math.floor (/ (to-local source lo) length))
(js/Math.ceil (/ (to-local source hi) length)))
[0])]
(mapcat (fn [period]
(let [cycle (update source :at + (/ (* period length) (:rate source)))]
(walk child cycle [lo hi] (conj path id) (conj seen sid))))
periods))))
nil))))
(sort-by (comp str key) nodes))))]
(let [tracks (walk sid (clip/grid-time clip sid)
[0 (clip/output-frames clip sid)] [] #{})]
(second
(reduce (fn [[seen out] track]
(let [link (:media-link track)
k (when link [link (:source track) (:span track)
(select-keys (:time track) [:at :rate :offset])])]
(if (and k (contains? seen k))
[seen out]
[(cond-> seen k (conj k)) (conj out track)])))
[#{} []] tracks)))))
(defn- retime
"Node `n` with its own time map replaced by `m`, and nothing else touched: its
span and keys are in its own frames, which a move does not change."
[n {:keys [at rate]}]
(assoc n :time (merge (:time n) {:mode :map :at at :rate rate :offset 0})))
(defn- subtree
"`id` and every node whose parent chain reaches it."
[nodes id]
(into #{} (filter #(some #{id} (symbol/lineage nodes %))) (keys nodes)))
(defn delete-node
"Take node `id` out of symbol `sid`, with everything hanging off it."
[clip sid id]
(clip/update-symbol clip sid update :nodes #(apply dissoc % (subtree % id))))
(defn- transplant
"Move node `id` from symbol `host`, where frame `frame` is showing, into symbol
`target`, keeping where it is on screen and when. `carry` is the matrix from
`host`'s coordinates to `target`'s, and `back` the time map from `target`'s
frames to `host`'s.
THE ONE RULE, for space and time alike: the node's new map is its old one
under what it leaves — its parents here, and the way from here to there — so
the picture and the timing through it do not change. For space that is a
`:pinv`, Blender's parent-inverse; for time it is a new `:at` and `:rate`. Its
channels, keys and span are untouched, and its children keep their parent
pointers and come with it. `{:clip}` or `{:refused why}`."
[clip store host frame id target carry back]
(let [nodes (:nodes (clip/symbol clip host))
n (get nodes id)
moving (subtree nodes id)
;; Its parents in this symbol, outermost first.
chain (map #(get nodes %) (reverse (rest (symbol/lineage nodes id))))
parent (when-let [p (:parent n)]
(some-> (symbol/world-of (resolved clip store host frame p) p)
js/Float64Array.from))]
(cond
(= host target) {:refused "it is already there"}
(and (= :instance (:kind n))
(some #(clip/contains-symbol? clip % target) (node/sources n)))
{:refused "a symbol cannot go inside itself"}
(some (fn [m] (or (:measured (get nodes m))
(some #(or (:dense %) (:generated %)) (vals (:channels (get nodes m))))))
moving)
{:refused "generated parts stay with their take — move the instance that places it"}
(some (fn [[k m]] (and (:stencil m)
(not= (contains? moving k) (contains? moving (:stencil m)))))
nodes)
{:refused "a stencil and what it clips have to move together"}
(and (:parent n) (nil? parent))
{:refused "its parent is not on screen at this frame"}
(some #(get-in % [:time :loop?]) chain)
{:refused "a looping parent is in the way"}
:else
(let [taken (:nodes (clip/symbol clip target))
ids (into {} (map (fn [m] [m (clip/free-id #(contains? taken %) m)])) moving)
pinv (reduce #(node/mul! (node/mat) %1 %2) carry (keep identity [parent (node/pinv n)]))
;; back · parents · own: target frames to the node's own.
time (reduce node/then-time back (concat (map node/time-of chain)
[(node/time-of n)]))
moved (for [m moving
:let [x (get nodes m)]]
(cond-> (-> x
(assoc :id (ids m))
(update :parent #(get ids %)))
(:stencil x) (update :stencil ids)
(= m id) (-> (retime time)
(assoc :pinv (vec (array-seq pinv))
:z (str "z" (js/Date.now) "-" (ids m))))))]
{:id (ids id)
:sid target
:clip (-> clip
(clip/update-symbol host update :nodes #(apply dissoc % moving))
(clip/update-symbol target update :nodes (fnil into {})
(map (juxt :id identity)) moved))}))))
(defn move-refusal
"Why `move-node` would refuse to move `from` into `to` at frame `f`, or nil.
SAID BEFORE THE DROP, not after it. A drag that reparents has to tell the
person what it would do while they can still change their mind, and the only
honest source for that is the check the command itself makes. Hence one
function, asked by the gesture on the way past and by `move-node` on the way
in.
The one that surprises: BOTH have to be on screen at this frame, because the
move keeps the picture and there is no common frame to keep it at otherwise.
Two clips of one lane never overlap, so nesting one into another there can
never be done — it is a thing to do between symbols, with the playhead
somewhere both of them are showing.
The deeper refusals — generated parts, a stencil parted from what it clips —
belong to the transplant and are only known when it runs."
[clip store open from to f]
(let [here (inside clip store open (pop from) f)
there (inside clip store open to f)
n (get-in clip [:symbols (:sid here) :nodes (peek from)])]
(cond
(nil? n) "nothing to move"
(or (nil? here) (nil? there)) "both have to be on screen at this frame"
(nil? (:sid there)) "only a symbol can take it"
(clip/trace? (clip/symbol clip (:sid there)))
"a tracing layer is a picture to draw over — nothing goes inside it"
(span/placement-refusal clip (:sid there) n)
(span/placement-refusal clip (:sid there) n)
(not (and (:time here) (:time there)))
"a held or looping clip has no clock to move through"
(nil? (some-> there :matrix node/invert)) "the target is scaled to nothing"
(= (:sid here) (:sid there)) "it is already there"
(and (= :instance (:kind n))
(some #(clip/contains-symbol? clip % (:sid there)) (node/sources n)))
"a symbol cannot go inside itself")))
(defn move-node
"Move the node at row path `from` — its last id is the node, the rest the
instances down to where it lives — into the symbol placed by the instance at
row path `to`, or to the top of `open` when `to` is empty. Row paths start at
`open`, and `f` is its current frame, at which both have to be on screen.
`{:clip :sid :id}` — the symbol it landed in and its id there, renamed only if
that one was taken — or `{:refused why}`."
[clip store open from to f]
(let [here (inside clip store open (pop from) f)
there (inside clip store open to f)
a (:time here)
b (:time there)
inv (some-> there :matrix node/invert)]
(if-let [why (move-refusal clip store open from to f)]
{:refused why}
(transplant clip store (:sid here) (:frame here) (peek from) (:sid there)
(node/mul! (node/mat) inv (:matrix here))
(node/then-time (node/invert-time b) a)))))
(defn- down
"Walk row path `path` down from symbol `sid` by structure alone: `{:sid
:time}`, the symbol it leads to and the time map from `sid`'s frames to that
symbol's own, nil through a loop.
`inside` without the frame. Which symbol a row is in and how fast it runs
there are the same on every frame, so asking needs nothing to be on screen;
only a move that keeps the PICTURE needs a frame, for the matrix."
[clip sid path]
(let [;; Structurally, a row leads into a symbol only where it names one:
;; a clip does, and a group holding one does not, so the walk stops at
;; the group rather than picking the drawing showing now — which
;; would make where a row lives depend on the playhead.
only (fn [sid id]
(when sid (node/source (get-in clip [:symbols sid :nodes id]))))
sids (reductions only sid path)
;; Every node on the way, outermost first: each instance, after its
;; parents in the symbol it is in.
maps (mapcat (fn [sid id]
(let [nodes (:nodes (clip/symbol clip sid))
chain (map #(get nodes %) (rseq (symbol/lineage nodes id)))]
(mapcat (fn [n]
(if (get-in n [:time :loop?]) [nil]
(cond-> [(node/time-of n)]
(= :instance (:kind n)) (conj (clip/source-time clip sid n)))))
chain)))
sids path)]
{:sid (last sids)
:time (when (every? some? maps)
(reduce node/then-time node/same-time maps))}))
(defn- dragged
"Frame `from` of node `id`'s own space, dragged `df` frames of the open
symbol: ONE WHOLE FRAME of that space.
THE ONE PLACE A RULER GESTURE BECOMES A FRAME NUMBER, and the whole of it.
`df` counts the open symbol's own frames, which is what the ruler is drawn in,
so for a row of the open symbol itself this is `from + df` and nothing happens
here at all. It earns its keep for a row reached THROUGH a retimed instance,
where one frame of the ruler is a fraction of the node's own: a frame number is
the one thing in a document that cannot fall between two frames —
`span/resize-out`, `resize-in` and `roll` refuse a fractional edge outright,
and `slide` would have let it into `:time :at` and refused every later edge
edit on that clip for ever.
Rounding, not flooring: a drag is a gesture at a position, and the frame it
means is the nearer one in both directions. Nothing else belongs here."
[here nodes id from df]
(let [chain (map #(get nodes %) (reverse (rest (symbol/lineage nodes id))))
rate (:rate (reduce node/then-time (:time here) (map node/time-of chain)))]
(js/Math.round (+ from (* df rate)))))
(defn slide
"Move the node at row path `path` along its symbol's time by `df` frames of
`open`. `{:clip}` or `{:refused why}`.
ONE WRITE TO `:at`, for every node alike: its span, keys and children are in
its own frames and come with it. `df` is carried down into the frames `:at` is
in — the symbol's, through each instance on the way, and its parents' there."
[clip open path df]
(let [here (down clip open (pop path))
id (peek path)
nodes (:nodes (clip/symbol clip (:sid here)))]
(cond
(nil? (get nodes id)) {:refused "nothing to move"}
(nil? (:time here)) {:refused "a looping instance is in the way"}
:else
(let [n (get nodes id)
;; Measured from where the node STARTS, so what lands on a whole
;; frame is the thing you can see moving. A node with no span is
;; on screen throughout and only its `:at` moves.
from (or (first (node/placed-span n)) (get-in n [:time :at] 0))
d (- (dragged here nodes id from df) from)
;; MOVE THE MAP THAT IS THERE; write a fresh one only where there is
;; none. The test for "there is one" is `:time` itself, and whether
;; it is the affine kind is `node/mapped-time?` — which an absent
;; `:mode` satisfies, because `node/time-of` has always read it as
;; `:map`. Asking for the key instead said no to every clip
;; `span/held` makes, whose `:time` is `{:at f :rate 1}` and nothing
;; more, and the else branch then REPLACED that map with one built
;; from `d` alone: the first drag of a freshly drawn clip threw away
;; its `:at` and jumped it to the head of the lane.
shift (fn [n]
(if (and (:time n) (node/mapped-time? n))
(update-in n [:time :at] (fnil + 0) d)
(assoc n :time {:mode :map :at d :rate 1})))
moved (assoc nodes id (shift (get nodes id)))
;; An editorial audio link follows a moved picture. Moving or
;; trimming the audio itself remains independent.
moved (if (= :audio (:kind (get nodes id)))
moved
(reduce (fn [ns [audio-id n]]
(if (and (= :audio (:kind n)) (= id (:linked-to n)))
(assoc ns audio-id (shift n))
ns))
moved nodes))]
;; COMMITTED LIKE EVERY OTHER SPAN EDIT. This validated with
;; `symbol/problems` and wrote the nodes in itself, which made it a
;; SECOND commit path — and the one `span/finish`'s docstring says is
;; the only one: "no command can commit an overlap in lane mode, and
;; `symbol/overlaps` turning up anything is a bug in a command rather
;; than a state to design around". It was that bug. A body drag could
;; leave two clips of a lane on screen over the same frames, and from
;; there the lane stops behaving: `symbol/children` has two clips
;; claiming one frame, so which drawing a polygon lands in and whether
;; a boundary can be rolled depend on which of them `some` reaches
;; first. `finish` does the same `problems` check and the overlap one
;; too, so this is less code and one fewer invariant to remember.
(span/claim clip (:sid here) moved id :grow-symbol (random-uuid))))))
(defn slide-many
"Move a selection simultaneously. Lane collisions refuse the entire edit."
[document open paths df]
(let [paths (vec (distinct (filter seq paths)))
entries (mapv (fn [path]
(let [here (down document open (pop path))
id (peek path)
nodes (get-in document [:symbols (:sid here) :nodes])]
{:path path :here here :sid (:sid here) :id id
:nodes nodes :node (get nodes id)})) paths)
roots (remove
(fn [{:keys [path sid id nodes]}]
(some (fn [other]
(or (and (< (count (:path other)) (count path))
(= (:path other) (subvec path 0 (count (:path other)))))
(and (= sid (:sid other)) (not= id (:id other))
(some #{(:id other)} (rest (symbol/lineage nodes id))))))
entries)) entries)]
(if (some #(or (nil? (:node %)) (nil? (get-in % [:here :time]))) roots)
{:refused "selection includes a node without an editable clock"}
(let [changes
(reduce (fn [changes {:keys [here sid id nodes node]}]
(let [from (or (first (node/placed-span node)) (get-in node [:time :at] 0))
d (- (dragged here nodes id from df) from)
shift (fn [n] (update-in n [:time :at] (fnil + 0) d))
ids (cons id (when (not= :audio (:kind node))
(for [[aid n] nodes :when (= id (:linked-to n))] aid)))]
(reduce (fn [out nid] (assoc-in out [sid nid] (shift (get nodes nid)))) changes ids)))
{} roots)]
(reduce (fn [result [sid changed]]
(if (:refused result) (reduced result)
(span/finish (:clip result) sid
(merge (get-in (:clip result) [:symbols sid :nodes]) changed)
nil :grow-symbol)))
{:clip document} changes)))))
(defn resize-out
"Move the right edge of the node at `path` by `df` frames of `open`.
One command either way: in a symbol drawn as a lane `span/resize-out` claims
the time it grows into, and in a composition it is one write to one span. The
mode says which, and nothing here has to ask."
[clip open path df ripple?]
(let [here (down clip open (pop path))
sid (:sid here)
id (peek path)
nodes (:nodes (clip/symbol clip sid))
n (get nodes id)]
(cond
(nil? n) {:refused "nothing to resize"}
(nil? (:time here)) {:refused "a looping instance is in the way"}
:else
(span/resize-out clip sid id (dragged here nodes id (second (node/placed-span n)) df)
{:ripple? ripple? :extent :grow-symbol}))))
(defn resize-in
"Move the left edge of the node at `path` by `df` frames of `open`."
[clip open path df]
(let [here (down clip open (pop path))
sid (:sid here)
id (peek path)
nodes (:nodes (clip/symbol clip sid))
n (get nodes id)]
(cond
(nil? n) {:refused "nothing to resize"}
(nil? (:time here)) {:refused "a looping instance is in the way"}
:else
(span/resize-in clip sid id (dragged here nodes id (first (node/placed-span n)) df)))))
(defn roll
"Move the shared boundary at `right-path` and the adjacent `left-path`."
[clip open left-path right-path df]
(let [here (down clip open (pop right-path))
sid (:sid here)
left-id (peek left-path)
right-id (peek right-path)
nodes (:nodes (clip/symbol clip sid))
right (get nodes right-id)]
(cond
(or (not= (pop left-path) (pop right-path)) (nil? right))
{:refused "a rolling edit needs adjacent clips in one sequence"}
(nil? (:time here)) {:refused "a looping instance is in the way"}
:else
(span/roll clip sid left-id right-id
(dragged here nodes right-id (first (node/placed-span right)) df)))))
(defn restack
"Put the node at row path `from` just in front of the one at `to` when
`front?`, or just behind it — side by side in one symbol, as the timeline lists
them. `{:clip :sid :id}` or `{:refused why}`.
ONE WRITE TO `:z`, between the two it lands between, so nothing else is
renumbered. Among the nodes that share its parent, because that is what `:z`
orders; a roto part's parent is the rig, and it restacks within that."
[clip open from to front?]
(let [{sid :sid} (down clip open (pop to))
nodes (:nodes (clip/symbol clip sid))
n (get nodes (peek from))
t (get nodes (peek to))
z #(or (:z %) "")
zs (->> nodes
(keep (fn [[k m]] (when (and (= (:parent m) (:parent t)) (not= k (peek from)))
(z m))))
sort)]
(cond
(not= (pop from) (pop to)) {:refused "only things side by side can be restacked"}
(or (nil? n) (nil? t)) {:refused "nothing to restack"}
(not= (:parent n) (:parent t)) {:refused "they hang off different parents"}
:else
{:sid sid
:id (peek from)
:clip (clip/update-symbol
clip sid assoc-in [:nodes (peek from) :z]
(if front?
(symbol/z-between (z t) (first (filter #(pos? (compare % (z t))) zs)))
(symbol/z-between (last (filter #(neg? (compare % (z t))) zs)) (z t))))})))
(defn group
"Put the nodes at row paths `froms`, all side by side in one symbol, into a
NEW symbol `sid`, placed where they were by instance `uuid`. `{:clip}` or
`{:refused why}`.
The new symbol starts where the earliest of them starts and ends where the
last one ends, so its instance's bar on the timeline covers exactly theirs.
Its instance sits at the identity, so nothing moves. It stores no pivot: an
instance turns about the middle of what it draws at the moment it is dragged,
so this cannot leave one behind when the group's contents are edited later."
[clip store open froms sid uuid f]
(let [host-path (pop (first froms))
{host :sid frame :frame} (inside clip store open host-path f)
nodes (:nodes (clip/symbol clip host))]
(cond
(nil? host) {:refused "they have to be on screen at this frame"}
(not-every? #(= host-path (pop %)) froms) {:refused "only things side by side can be grouped"}
(some #(nil? (get nodes (peek %))) froms) {:refused "nothing to group"}
:else
(let [whole [0 (clip/frames clip host)]
spans (for [from froms
:let [n (get nodes (peek from))]]
(or (node/placed-span n)
(when (= :instance (:kind n))
(node/placed-span
(assoc n :span [0 (or (clip/frames clip (node/source n)) 0)])))
whole))
start (js/Math.floor (max 0 (apply min (map first spans))))
end (min (second whole) (apply max (map second spans)))
made (-> clip
(assoc-in [:symbols sid] {:id sid :name (name sid) :fps (clip/fps clip host)
:frames (max 1 (js/Math.ceil (- end start)))
:nodes {}})
(clip/place-symbol store host sid start uuid nil))
back (node/invert-time (node/time-of (get-in made [:symbols host :nodes uuid])))
moved (reduce (fn [acc from]
(let [r (transplant (:clip acc) store host frame (peek from) sid
(node/mat) back)]
(if (:refused r) (reduced r) r)))
{:clip made} froms)]
moved))))
;; ---------------------------------------------------------------------------
;; pegs
(defn peg
"Put a PEG over the node at row path `path`: a free `:group` between it and
whatever it hangs off now, sitting on the pivot the node has at frame `f`.
`{:clip :sid :id}` or `{:refused why}`.
WHAT A PEG IS FOR, NOW THAT A NODE HAS ITS OWN PIVOT. `[:xform :pivot]` is how
ONE node turns about a point of its own — that is `node/local!`, it needs no
parent, and it is right between two keys. A peg is for the three things that
are about MORE THAN ONE NODE, or about a node whose channels are not yours to
write:
A PIVOT SHARED BETWEEN NODES. An arm and a forearm turning about one shoulder
is one transform driving two drawings, and two pivots that have to agree
frame for frame are not that. The peg is the shoulder, and they hang off it.
A SECOND TRANSFORM ON ONE NODE. A drawing turning about its own middle while
the whole limb swings about the shoulder is two rotations, and a node has one
`rot`. Toon Boom stacks pegs for exactly this.
A HAND TRANSFORM OVER A MEASURED ONE. `gesture/refusal` turns a drag on a
measured node away because the next regenerate would discard it. A peg's
channels are its own, so the hand transform composes OUTSIDE the measured one
and the measurement stays regenerable. Resolve publishes a track and parents a
transform to it; Harmony puts a peg over the drawing. Same shape.
The node's own pivot is NOT one of them, and that is the correction: a peg used
to be the only pivot there was, so making one was the answer to \"this turns
about the wrong point\" — which is a thing to drag a cross for, not a node to
create. See `xform-paths`.
NOTHING MOVES, and that is `:pinv`'s whole job — Blender's parent-inverse, the
same field `transplant` writes for the same reason. The peg takes the node's
place in the hierarchy, inheriting its `:parent` and its `:pinv`; the node hangs
off the peg with `T(-c)` as its own, so
... · pinv · T(c) · T(-c) · local = ... · pinv · local
to the bit. The node's channels are untouched, which is what lets this work on a
measured node at all — and what lets the node keep its own pivot, which is in
its own coordinates and therefore says nothing about who its parent is.
THE PEG TAKES THE NODE'S `:z`, so draw order is unchanged: a parent's z path is
a prefix of its child's, so the node now sorts at `[… z z]` where it sorted at
`[… z]`, and against any sibling the comparison is decided at the same place it
was before. It takes no `:time` and no `:span`: an identity time map leaves the
node's own frames exactly as they were, and a peg with no span is simply always
there, so what is on screen when does not change either."
[clip store open path f uuid]
(let [{:keys [sid id frame]} (placement clip store open path f)
n (when sid (get-in clip [:symbols sid :nodes id]))]
(cond
(nil? n) {:refused "it is not on screen at this frame"}
(= :audio (:kind n)) {:refused "a sound has no transform to pivot"}
(contains? (:nodes (clip/symbol clip sid)) uuid) {:refused "that id is taken"}
:else
(let [c (gesture/pivot (gesture/values n frame store)
((pick/bounds-of clip store sid n) frame))]
{:sid sid
:id uuid
:clip (clip/update-symbol
clip sid update :nodes
(fn [nodes]
(-> nodes
(assoc uuid (cond-> {:id uuid :kind :group
:name (str (clip/node-label clip id n) " peg")
:parent (:parent n)
:z (:z n)
:channels {[:xform :pos] (ch/framed c)}}
(:pinv n) (assoc :pinv (:pinv n))))
(update id assoc
:parent uuid
:pinv (vec (array-seq
(node/local! (node/mat) (mapv - c) [0 0] 0 [1 1] [0 0])))))))}))))

View file

@ -21,18 +21,41 @@
"`:bitmap` is in the vocabulary and not implemented; it is "`:bitmap` is in the vocabulary and not implemented; it is
here so that a scene that names one fails as \"not implemented\" rather than as here so that a scene that names one fails as \"not implemented\" rather than as
\"not a kind\"." \"not a kind\"."
#{:poly :disc :rect :group :bitmap :symbol :audio}) #{:poly :disc :rect :group :bitmap :instance :audio})
(def implemented-kinds #{:poly :disc :rect :group :symbol :audio}) (def implemented-kinds #{:poly :disc :rect :group :instance :audio})
(def xform-paths (def xform-paths
"In composition order, which is also the order they have to be sampled in. "In composition order, which is also the order they have to be sampled in.
:skew and :anchor are in here although nothing drives either yet. A :skew is in here although nothing drives it yet. A decomposition is not
decomposition is not extensible after the fact: adding a component later means extensible after the fact: adding a component later means migrating every
migrating every stored transform, so both are in the shape and in the stored transform, so it is in the shape and in the composition order from the
composition order from the start." start.
[[:xform :pos] [:xform :rot] [:xform :scale] [:xform :skew] [:xform :anchor]])
:pivot IS THE POINT ROTATION AND SCALE HAPPEN ABOUT, in the node's own
coordinates, and it is in the decomposition because THERE IS NOWHERE ELSE IT
CAN BE. Toon Boom gives every layer and peg a pivot; Flash gives every instance
a transformation point; After Effects calls it the anchor point. All three store
it, and all three are right to, for one reason: a turn has to be a turn ON EVERY
FRAME, and the only way to keep a point still through an interpolated angle is
for the angle to be composed about that point. Solving for the `pos` that holds
a point still — `gesture/about` — is exact on the frame it is solved and wrong
between two keys, because the solution is an ARC in the angle and `pos` tweens
along the CHORD. See docs/animation-model.md, which has the measurements off the
document where this was found.
So `local!` is `T(pos)·T(piv)·R·K·S·T(-piv)`, the transform conjugated by a
translation — \"do M in a frame shifted by piv\" — and the node's pivot point
sits at `pos + piv` in its parent. A pivot of `[0 0]`, which is the default,
leaves that exactly `T(pos)·R·K·S`: a drawing needs no pivot, because
`paint/centred` already put its origin on the middle of what it draws.
A PEG IS STILL A PEG. `nest/peg` is how a pivot gets SHARED between nodes, or
put above a measured transform that a regenerate would overwrite; this is how
one node turns about its own middle. The two were conflated — the peg was made
to stand in for the pivot — and that is what cost a keyed turn its pivot."
[[:xform :pos] [:xform :pivot] [:xform :rot] [:xform :scale] [:xform :skew]])
(def valid-paths (def valid-paths
"The set of valid channel paths follows from the node's :kind, and that is a "The set of valid channel paths follows from the node's :kind, and that is a
@ -45,8 +68,13 @@
change to this spec silently change what gets drawn." change to this spec silently change what gets drawn."
(let [base (into #{[:vis]} xform-paths)] (let [base (into #{[:vis]} xform-paths)]
{:group base {:group base
:symbol base ;; A palette-track placement is still an instance. Its palette choice is
:audio (into base [[:audio :gain] [:audio :pan] [:audio :rate]]) ;; a parameter channel on that instance, so the generic span commands can
;; move and trim it without knowing what kind of lane owns it.
:instance (conj base [:palette])
;; A sound is not in the picture: no transform, no visibility. After
;; Effects' audio-only layer has no Transform group for the same reason.
:audio #{[:audio :gain] [:audio :pan] [:audio :rate]}
:poly (into base [[:geom :pts] [:style :color]]) :poly (into base [[:geom :pts] [:style :color]])
;; A disc's radius is framed in practice — iris size is a knob, not a ;; A disc's radius is framed in practice — iris size is a knob, not a
;; performance — but it is a channel like any other so it can be keyed. ;; performance — but it is a channel like any other so it can be keyed.
@ -57,29 +85,141 @@
(def defaults (def defaults
"The identity transform, as channels. A node's channel map is merged over this, "The identity transform, as channels. A node's channel map is merged over this,
so a hand-written scene says only what it means to say." so a hand-written scene says only what it means to say.
THE DEFAULT PIVOT IS THE NODE'S OWN ORIGIN, which is a real default and not a
missing value: a drawing's origin IS the middle of what it draws — `paint/centred`
put it there — so a shape turns about its own middle with no pivot stored, and
`local!` collapses to `T(pos)·R·K·S` for it. What needs a pivot of its own is a
node whose content is nowhere near its origin: a symbol instance, whose origin
is the symbol's, and a measured part, whose origin is the corner of the footage.
Both get one where they are made — `clip/place-symbol` — and `gesture/turn`
gives one to anything that turns without having been given one."
{[:xform :pos] (ch/framed [0.0 0.0]) {[:xform :pos] (ch/framed [0.0 0.0])
[:xform :pivot] (ch/framed [0.0 0.0])
[:xform :rot] (ch/framed 0.0) [:xform :rot] (ch/framed 0.0)
[:xform :scale] (ch/framed [1.0 1.0]) [:xform :scale] (ch/framed [1.0 1.0])
[:xform :skew] (ch/framed [0.0 0.0]) [:xform :skew] (ch/framed [0.0 0.0])
[:xform :anchor] (ch/framed [0.0 0.0])
[:vis] (ch/framed true)}) [:vis] (ch/framed true)})
(def audio-defaults
"Unity gain, centred, at its own speed."
{[:audio :gain] (ch/framed 1.0)
[:audio :pan] (ch/framed 0.0)
[:audio :rate] (ch/framed 1.0)})
(defn defaults-of [n]
(if (= :audio (:kind n)) audio-defaults defaults))
(defn channels (defn channels
"The node's channels with the transform defaults filled in." "The node's channels with its kind's defaults filled in."
[n] [n]
(merge defaults (:channels n))) (merge (defaults-of n) (:channels n)))
(defn measured?
"Is this node's transform regenerated from the footage rather than authored?
What it guards is that a hand edit to a measured transform is thrown away by
the next regenerate, which is what `gesture/refusal` refuses. The way to
transform one of these by hand is a PEG above it: the hand transform is then on
a node of its own and the measured channels underneath are left to be
regenerated. Nothing has to be written onto the measured node at all.
ITS PIVOT IS STILL ITS OWN, and that is the one part of its transform a hand
may write: `[:xform :pivot]` is authored on every node alike, never dense and
never regenerated, so a measured mouth can be told to turn about its own middle
without a peg and without anything a regenerate would discard. What the peg is
still for is the TRANSFORM over a measured one, which is this function's
business; where the pivot goes is not."
[n]
(boolean (some #(let [c (get-in n [:channels [:xform %]])]
(or (:dense c) (:generated c)))
[:pos :rot :scale])))
(defn hold-only?
"Channels whose values are choices, not quantities. Their keys may change at
a frame boundary but there is no meaningful value between two keys."
[path value]
(or (= path [:style :color])
(boolean? value)
(and (keyword? value) (not= path [:palette]))))
(defn- enforce-hold [path c]
(if (and (= path [:style :color]) (:keys c))
(-> c (assoc :interp :hold) (dissoc :segments))
c))
(defn set-channel
"Write `v` into channel `path`: a key on the node's own frame `f` when the
channel is keyed, its one value when it is not."
[n path f v]
(let [c (get (channels n) path)]
(assoc-in n [:channels path]
(enforce-hold
path
(if (:keys c)
(assoc-in c [:keys f] v)
(merge (select-keys c [:semantic]) (ch/framed v)))))))
(defn set-keyed-channel
"Write `v` as a key at `f`, starting an animated channel when needed. This is
the auto-key counterpart to `set-channel`; an existing channel keeps its
interpolation and segment choices."
[n path f v]
(let [c (get (channels n) path)]
(assoc-in n [:channels path]
(enforce-hold
path
(if (:keys c)
(assoc-in c [:keys f] v)
(merge (select-keys c [:semantic])
(ch/keyed {f v} (if (hold-only? path v) :hold :linear))))))))
(defn toggle-key
"Key channel `path` on the node's own frame `f` with the value it has there, or
take the key there off. The first key starts the channel animating and taking
the last one off leaves it that one value. A boolean holds; anything else tweens.
`store` because the value it keys is read out of the channel, and a measured
channel's values live in tier 2. Colour is always held even though current
documents store palette choices as numeric slot indices."
[n path f store]
(let [c (get (channels n) path)
v (ch/value-at c f store)
ks (dissoc (:keys c) f)]
(assoc-in n [:channels path]
(enforce-hold
path
(cond
(not (:keys c)) (merge (select-keys c [:semantic])
(ch/keyed {f v} (if (hold-only? path v) :hold :linear)))
(not (contains? (:keys c) f)) (assoc-in c [:keys f] v)
(seq ks) (cond-> (assoc c :keys ks)
(:segments c) (update :segments dissoc f))
:else (ch/framed v))))))
(defn set-segment-interp
"Choose how channel `path`'s key at `left` leads to the next one: `:hold` cuts
there, `:linear` tweens. The same for a drawing's points as for a transform —
only a gap that exists, between a key and a later one, can be chosen."
[n path left interp]
(let [ks (:keys (get (channels n) path))]
(if (and (contains? ks left) (some #(< left %) (keys ks))
(#{:hold :linear} interp)
(or (= :hold interp) (not (hold-only? path nil))))
(assoc-in n [:channels path :segments left] interp)
n)))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; time maps ;; time maps
;; ;;
;; Exposure, mouth lead and a symbol instance's timing are ONE mechanism, and ;; Cel, mouth lead and a symbol instance's timing are ONE mechanism, and
;; seeing that is what keeps them from being three implementations that disagree ;; seeing that is what keeps them from being three implementations that disagree
;; at the edges. ;; at the edges.
(defn expose (defn expose
"Hold a frame back onto an exposure grid: 1 = on 1s, 2 = on 2s. Frame 5 at "Hold a frame back onto an exposure grid: 1 = on 1s, 2 = on 2s. Frame 5 at
exposure 2 reads the pose from frame 4. cel 2 reads the pose from frame 4.
FLOOR, NEVER ROUND. Rounding would let an output frame read a pose from the FLOOR, NEVER ROUND. Rounding would let an output frame read a pose from the
FUTURE, which is a lead — a separate control, applied after this one, for a FUTURE, which is a lead — a separate control, applied after this one, for a
@ -87,53 +227,138 @@
[f n] [f n]
(if (and n (> n 1)) (* (js/Math.floor (/ f n)) n) f)) (if (and n (> n 1)) (* (js/Math.floor (/ f n)) n) f))
(defn sample-frame (def same-time
"Pick a source frame for a lower picture rate without changing clip time. "The identity time map: these frames ARE those frames. What a walk starts from
before it has composed anything, and what `time-of` gives a node with no time
of its own."
{:at 0 :rate 1})
The input is already an integer source frame from the audio clock. Its time is (defn time-of
f/source-fps. Quantise that time to the picture grid, then read the latest "A node's own time as the affine map it is: `{:at a :rate r}`, meaning a frame
source frame at or before it. The result is always an integer and never from `p` of its parent is frame `r·(p − a)` of its own. THE SAME FOR EVERY NODE. A
the future, including when the rates do not divide (30 source → 24 picture)." node with no time map is `{:at 0 :rate 1}`, reading its parent's frames as its
[f source-fps picture-fps] own; a mouth lead's `:offset` is folded into `:at`. Exposure is a floor, not
(if (and source-fps picture-fps part of the map, and is left out: this is the map a move preserves and a
(pos? source-fps) (pos? picture-fps) timeline row draws with, and `local-frame` is what reads a frame, the floor and
(< picture-fps source-fps)) the lead in their load-bearing order.
(min f (js/Math.floor
(* (js/Math.floor (/ (* f picture-fps) source-fps)) `:rate` is a RETIME SOMEBODY CHOSE and nothing else — half speed on an insert.
(/ source-fps picture-fps)))) Reconciling two frame rates is not a retime and does not belong here: it is a
selection, and it lives in `domain/cadence`."
[n]
(let [{:keys [mode at rate offset] :or {mode :map at 0 rate 1 offset 0}} (:time n)]
(if (= mode :map)
{:at (- at (/ offset rate)) :rate rate}
{:at 0 :rate 1})))
(defn mapped-time?
"Whether `n`'s own time is the affine map `time-of` reads, and therefore
whether moving it is a write to `:time :at`.
AN ABSENT `:mode` IS `:map`, which is what `time-of` has always defaulted it
to — and the default is the ordinary case, not an edge one: `span/held` writes
`{:at f :rate 1}` with no mode at all, so every clip a drawing creates is in
it. Code that asked `(= :map (get-in n [:time :mode]))` instead answered no
for those, and `nest/slide` acted on the answer by REPLACING the whole map
with a fresh one — so the first drag of a freshly drawn clip discarded its
`:at` and teleported it to the head of the lane."
[n]
(= :map (:mode (:time n) :map)))
(defn then-time
"`outer` then `inner`: the map from `outer`'s parent straight to `inner`'s own
frames. Time maps compose like matrices do, which is what makes a nesting of
any depth one map."
[{a1 :at r1 :rate} {a2 :at r2 :rate}]
{:at (+ a1 (/ a2 r1)) :rate (* r1 r2)})
(defn invert-time [{:keys [at rate]}]
{:at (- (* at rate)) :rate (/ 1 rate)})
(defn placed-span
"Where a node exists, as `[in out)` in its PARENT's frames, or nil for always.
A `:span` is in the node's OWN frames — which of its frames exist — for every
node alike, and its time map says where they land in the parent. For a node
with no time map the two are the same frames, so a shape's span reads as it
always did. Moving a node along its parent is then one write to `:at`, and the
span, which says what the node IS, does not change when it is moved."
[n]
(when-let [[in out] (:span n)]
(let [{:keys [at rate]} (time-of n)]
[(+ at (/ in rate)) (+ at (/ out rate))])))
;; ---------------------------------------------------------------------------
;; what an instance places
;;
(defn source
"The symbol used by this cel. Sequence groups arrange cels;
a row is a view of that group, not one row per source."
[n]
(when (= :instance (:kind n)) (get-in n [:source :symbol])))
(defn sources
"Structural references, including cels outside the playhead."
[n]
(if-let [sid (source n)] #{sid} #{}))
(defn playback-of [n]
(merge {:in 0 :speed 1 :end :stop} (:playback n)))
(defn source-time
"Invertible cel -> source map, or nil for holds and endpoint policies.
Forward sampling remains available through `placed-frame` in every case."
[n]
(let [{:keys [in speed end]} (playback-of n)]
(when (and (pos? speed) (= :stop end) (not (get-in n [:time :loop?])))
{:at (- (/ in speed)) :rate speed})))
(defn placed-frame
"Sample source time without changing the cel's property clock:
`{:symbol :frame}`, the symbol shown and which of its frames. `length` is that
symbol's frame count. Nil means no source contribution — this cel
places nothing, or its playback has run past what there is to show."
[n f length]
(when-let [sid (source n)]
(when (and (number? length) (pos? length))
(let [{:keys [in speed end]} (playback-of n)
raw (+ in (* speed f))
frame (case (if (get-in n [:time :loop?]) :loop end)
:loop (mod raw length)
:hold (max 0 (min (dec length) raw))
:stop raw)]
(when (and (<= 0 frame) (< frame length))
{:symbol sid :frame frame})))))
(defn finite-number? [v] (and (number? v) (js/Number.isFinite v)))
(defn hold
"Floor `f` onto the last of `holds` at or before it, and before the first onto
the first. Exposure on authored frames rather than on a grid: `[0 12 30]` shows
frame 12 from 12 until 30. What a tracing layer's held photos and a face's
trace keys are.
A `reduce` that stops at the first hold past `f`, because this runs per node
per frame and the list is sorted."
[f holds]
(if (seq holds)
(reduce (fn [held h] (if (<= h f) h (reduced held))) (first holds) holds)
f)) f))
(defn local-frame (defn local-frame
"Apply a node's time map to the frame it was handed by its parent. "Apply an artistic time map within one frame space: expose, hold, then offset.
Frame-rate selection happens at symbol boundaries in domain/clip.
ORDER IS LOAD-BEARING: expose first, then offset. Flooring onto a grid and Holds come after exposure and inherit the same way, strictly: they are a floor
shifting against the clock do not commute — shift first and the floor discards of this node's own frame, and its children are handed the floored frame."
it on most frames, so the lead slider reads as doing nothing at exposures above
1, which is indistinguishable from the slider being unwired.
Composed along the parent chain, outermost first, by timeline/eval-frame. Two
rules fall out and they are different rules: exposure INHERITS STRICTLY,
because a head cutting on odd frames against a mouth cutting on even ones reads
as two performances; offset is PER-NODE by design, because mouth lead applies
to performance nodes and not to the plate, which is the entire point of it."
[n f] [n f]
(let [{:keys [mode offset rate at in source-fps sample-fps] (let [{:keys [mode at rate offset holds] ex :expose :or {mode :map at 0 rate 1}} (:time n)]
ex :expose :or {mode :inherit}} (:time n)]
(if (= mode :inherit) (if (= mode :inherit)
f f
(do (cond-> (* rate (- f at))
(when (and (not (#{:symbol :audio} (:kind n))) rate (not= rate 1.0) (not= rate 1))
(throw (ex-info "time map :rate belongs to a symbol or audio instance"
{:node (:id n) :time (:time n)})))
(when (and sample-fps (not (and source-fps (pos? source-fps))))
(throw (ex-info "picture sampling needs a positive source fps"
{:node (:id n) :time (:time n)})))
(cond-> (if (#{:symbol :audio} (:kind n))
(+ (or in 0) (* (or rate 1) (- f (or at 0))))
f)
sample-fps (sample-frame source-fps sample-fps)
ex (expose ex) ex (expose ex)
offset (+ offset)))))) (seq holds) (hold holds)
offset (+ offset)))))
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; the transform ;; the transform
@ -164,11 +389,17 @@
dest)) dest))
(defn local! (defn local!
"dest := T(pos) · T(anchor) · R(rot) · K(skew) · S(scale) · T(-anchor) "dest := T(pos) · T(piv) · R(rot) · K(skew) · S(scale) · T(-piv)
Written out closed-form rather than as five matrix products, because this runs The transform CONJUGATED BY ITS PIVOT, which is to say: do the rotation, skew
per node per frame and the five products would each allocate. The derivation, and scale in a frame shifted to `piv`, so the point `piv` of the node's own
so the constants are checkable rather than trusted: coordinates does not move however they change. Toon Boom's layer pivot, Flash's
transformation point, After Effects' anchor point; `xform-paths` says why it is
in the decomposition rather than solved for per drag.
Written out closed-form rather than as six matrix products, because this runs
per node per frame and each product would allocate. The derivation, so the
constants are checkable rather than trusted:
R·K·S = | c -s | · | 1 kx | · | sx 0 | R·K·S = | c -s | · | 1 kx | · | sx 0 |
| s c | | ky 1 | | 0 sy | | s c | | ky 1 | | 0 sy |
@ -179,32 +410,36 @@
R·K·S = | sx(c - s·ky) sy(c·kx - s) | R·K·S = | sx(c - s·ky) sy(c·kx - s) |
| sx(s + c·ky) sy(s·kx + c) | | sx(s + c·ky) sy(s·kx + c) |
and the translation is anchor + pos - M·anchor, which is what makes rotation which is the linear part, UNTOUCHED BY THE PIVOT — a conjugation by a
and scale happen ABOUT the anchor. :anchor is Flash's registration point and translation cannot change it, which is why a pivot is free to move without
Blender's origin, and getting it wrong is why hand-placed parts swing rather reshaping anything. All of it lands in the translation:
than turn.
T(p)·T(a)·M·T(-a) = T(p + a - M·a) · M
so with `a = [0 0]` this is exactly `T(pos)·R·K·S` and a node with no pivot
composes as it always did, to the bit.
:skew is stored as shear FACTORS, not angles — kx is x gained per unit y — so :skew is stored as shear FACTORS, not angles — kx is x gained per unit y — so
that the identity is 0 and a decomposition round-trips without a tangent." that the identity is 0 and a decomposition round-trips without a tangent."
[^js dest pos rot scale skew anchor] [^js dest pos piv rot scale skew]
(let [c (js/Math.cos rot) (let [c (js/Math.cos rot)
s (js/Math.sin rot) s (js/Math.sin rot)
sx (ch/component scale 0) sx (ch/component scale 0)
sy (ch/component scale 1) sy (ch/component scale 1)
kx (ch/component skew 0) kx (ch/component skew 0)
ky (ch/component skew 1) ky (ch/component skew 1)
ax (ch/component anchor 0) ax (ch/component piv 0)
ay (ch/component anchor 1) ay (ch/component piv 1)
a (* sx (- c (* s ky))) a (* sx (- c (* s ky)))
b (* sx (+ s (* c ky))) b (* sx (+ s (* c ky)))
cc (* sy (- (* c kx) s)) c* (* sy (- (* c kx) s))
d (* sy (+ (* s kx) c))] d (* sy (+ (* s kx) c))]
(aset dest 0 a) (aset dest 0 a)
(aset dest 1 b) (aset dest 1 b)
(aset dest 2 cc) (aset dest 2 c*)
(aset dest 3 d) (aset dest 3 d)
(aset dest 4 (+ ax (ch/component pos 0) (- (+ (* a ax) (* cc ay))))) (aset dest 4 (+ (ch/component pos 0) ax (- (+ (* a ax) (* c* ay)))))
(aset dest 5 (+ ay (ch/component pos 1) (- (+ (* b ax) (* d ay))))) (aset dest 5 (+ (ch/component pos 1) ay (- (+ (* b ax) (* d ay)))))
dest)) dest))
(defn pinv (defn pinv
@ -228,6 +463,16 @@
pinv-m (mul! dest pinv-m local) pinv-m (mul! dest pinv-m local)
:else (doto dest (.set local)))) :else (doto dest (.set local))))
(defn invert
"The inverse of a 2x3 affine, or nil when it has none — a node scaled to
nothing has no inside to draw into."
[^js m]
(let [[a b c d e f] (array-seq m)
det (- (* a d) (* b c))]
(when-not (zero? det)
(js/Float64Array. #js [(/ d det) (/ (- b) det) (/ (- c) det) (/ a det)
(/ (- (* c f) (* d e)) det) (/ (- (* b e) (* a f)) det)]))))
(defn apply-pt! (defn apply-pt!
"out[2i], out[2i+1] := m · (x, y)." "out[2i], out[2i+1] := m · (x, y)."
[^js out i ^js m x y] [^js out i ^js m x y]
@ -265,19 +510,49 @@
(not (contains? implemented-kinds k))) (not (contains? implemented-kinds k)))
(conj (str ":kind " k " is in the vocabulary but not implemented")) (conj (str ":kind " k " is in the vocabulary but not implemented"))
(and (= k :symbol) (nil? (:of n))) (conj "a symbol instance needs :of") (and (= k :instance) (not (keyword? (source n))))
(and (= k :audio) (nil? (get-in n [:source :footage]))) (conj "an instance needs :source {:symbol <symbol-id>}")
(conj "an audio instance needs :source :footage") (and (= k :instance)
(and (#{:symbol :audio} k) (some? (get-in n [:time :rate])) (let [{:keys [in speed end]} (playback-of n)]
(not (pos? (get-in n [:time :rate])))) (not (and (finite-number? in) (<= 0 in)
(conj "an instance's :rate must be positive") (finite-number? speed) (<= 0 speed)
(#{:stop :hold :loop} end)))))
(conj "playback needs a nonnegative finite :in and :speed, and :end :stop, :hold or :loop")
(:layout n)
(conj ":layout is not a node field — a lane is how the timeline DRAWS a symbol, not a thing in the document")
(and (= k :audio) (not (some (:source n) [:footage :sound])))
(conj "an audio node needs a :source :footage or :sound")
(and (some? (get-in n [:time :rate]))
(not (and (finite-number? (get-in n [:time :rate]))
(pos? (get-in n [:time :rate])))))
(conj ":time :rate must be positive")
(contains? (:channels n) [:xform :anchor])
(conj (str ":anchor is now [:xform :pivot], and it means the same point "
"in the same coordinates — but the composition around it "
"changed from T(pos)·M·T(-a) to T(pos)·T(a)·M·T(-a), so a "
"turned or scaled node reading one as the other would move"))
(nil? (:z n)) (conj "no :z — draw order is authored per scene, not implied by the tree") (nil? (:z n)) (conj "no :z — draw order is authored per scene, not implied by the tree")
(and (:span n) (not= 2 (count (:span n)))) (and (:span n) (not (and (vector? (:span n)) (= 2 (count (:span n)))
(conj ":span must be [in out]")) (every? finite-number? (:span n))
(apply < (:span n)))))
(conj ":span must be a finite, increasing [in out]")
(some? (get-in n [:time :in]))
(conj ":time has an :in — an instance's first frame is the start of its own :span")
(let [hs (get-in n [:time :holds])]
(and (some? hs) (not (and (vector? hs) (every? finite-number? hs)
(or (empty? hs) (apply < hs))))))
(conj ":time :holds must be a vector of increasing frames"))
;; `[:xform :anchor]` is excluded because it has a NAMED refusal above.
;; It is the one invalid path a stored document is likely to carry — every
;; schema-6 drawing and placement had one, and schema 7 refused them all
;; rather than convert — so "not valid on a :poly node" would be the first
;; thing a person saw, and it says nothing about what to do. The precedent
;; is `symbol/problems`' refusal of `:trace`.
(into (when valid (into (when valid
(for [[path _] (:channels n) (for [[path _] (:channels n)
:when (not (contains? valid path))] :when (and (not (contains? valid path))
(not= path [:xform :anchor]))]
(str "channel " (pr-str path) " is not valid on a " k " node")))) (str "channel " (pr-str path) " is not valid on a " k " node"))))
(into (for [[path c] (:channels n) (into (for [[path c] (:channels n)

View file

@ -0,0 +1,20 @@
(ns arthur.domain.onion
"Editor-only ghosts of the picture a few frames either side of the playhead.
THE STAGE AT ANOTHER FRAME, nothing more. The resolver already draws any
frame, so a ghost is what it draws `n` frames back or ahead — whatever is
animating, by spans, keys or anything else, shows up without being looked
for.")
(def defaults {:on? false :before 1 :after 1 :opacity 0.25})
(defn frames
"The output frames to ghost around frame `f` of a `length`-frame transport:
`before` back and `after` ahead, each tagged with its direction, nearest
first, and none off either end."
[f length {:keys [before after]}]
(concat
(for [i (range 1 (inc before)) :let [g (- f i)] :when (<= 0 g)]
{:frame g :direction :before})
(for [i (range 1 (inc after)) :let [g (+ f i)] :when (< g length)]
{:frame g :direction :after})))

View file

@ -0,0 +1,303 @@
(ns arthur.domain.outline
"A brush stroke, as pixels and then as polygons.
Flash's brush: what you paint becomes filled shapes the moment you let go, and
the stroke is not kept. Here the stroke is first a MASK — a byte per stage
pixel, stamped with the brush's disc as the pointer moves, so what is on screen
while painting is exactly the pixels — and on letting go each piece of it is
traced round its pixel edges and simplified to as many points as are asked for,
as the mouth's ring is.
The trace walks pixel CORNERS, so before simplifying, a traced ring filled by
`raster/fill-poly-buf!` is the mask again exactly.
A HOLE STAYS A HOLE, and a shape is still one ring. A loop painted round a gap
is a piece with a hole in it; the hole is traced as well, and joined to the
outside by a HORIZONTAL bridge along a whole-pixel row — out and back along
the same line. The fill samples at pixel centres (y + 0.5) and an edge with no
height crosses no scanline, so the bridge draws nothing, the even-odd fill
leaves the hole empty, and nothing downstream has to know a ring can have
one. It is how earcut and the TrueType rasterisers take holes too.
`pieces` keeps the trace — outside and holes, at full resolution — so the
fit can change after the stroke, and `polygon` is the one place it is fitted
and bridged: the preview and the saved shape are the same function's output,
so what is on screen while painting is what is kept.
FITTED TO A TOLERANCE, not cut to a count: every point of the traced edge is
within so many pixels of the polygon (Douglas-Peucker), so points go where
the shape bends and none are spent on a straight run."
(:require ["polygon-clipping" :as clipping]
[arthur.domain.raster :as raster]))
(defn mask [w h] {:w w :h h :buf (js/Uint8Array. (* w h))})
(defn stamp!
"The brush, `size` pixels across, from `[ax ay]` to `[bx by]`: a disc every
half radius along the way, so a fast stroke is still one stroke."
[m [ax ay] [bx by] size]
(let [[ox oy] (or (:origin m) [0 0])
ax (- ax ox) ay (- ay oy) bx (- bx ox) by (- by oy)
r (max 0.5 (/ size 2))
steps (max 1 (js/Math.ceil (/ (js/Math.hypot (- bx ax) (- by ay)) (max 0.5 (/ r 2)))))]
(dotimes [i (inc steps)]
(let [t (/ i steps)]
(raster/fill-disc! m (+ ax (* t (- bx ax))) (+ ay (* t (- by ay))) r 1))))
m)
(defn- flood!
"Label the 4-connected region of pixels whose ink is `ink` from pixel `start`
with `id` in `lab`, inside the box `[x0 y0 x1 y1]`. Its size, and whether it
reaches the box's edge — a gap that does is outside, not a hole.
A typed stack and no allocation per pixel: this runs on every pointer move."
[^js lab ^js buf w [x0 y0 x1 y1] ink start id ^js stack]
(aset lab start id)
(aset stack 0 start)
(let [top (volatile! 1) size (volatile! 0) edge? (volatile! false)
visit! (fn [q] (when (and (zero? (aget lab q)) (== ink (aget buf q)))
(aset lab q id)
(aset stack @top q)
(vswap! top inc)))]
(while (pos? @top)
(let [p (aget stack (vswap! top dec))
x (mod p w) y (quot p w)]
(vswap! size inc)
(when (or (== x x0) (== x x1) (== y y0) (== y y1)) (vreset! edge? true))
(when (< x0 x) (visit! (dec p)))
(when (< x x1) (visit! (inc p)))
(when (< y0 y) (visit! (- p w)))
(when (< y y1) (visit! (+ p w)))))
{:size @size :edge? @edge?}))
(def ^:private dirs [[1 0] [0 1] [-1 0] [0 -1]])
(defn- ring
"The edge of the region `in?` that starts at pixel `start`, the first of it in
raster order, as flat corner points: walked with the region on the right and a
point only where the walk turns. Round a piece, that is its outside; round a
hole, the inside edge of the piece around it."
[w in? start]
(let [x0 (mod start w) y0 (quot start w)]
;; Heading east along the start pixel's top edge, the region is below: on the
;; right. At each corner the two pixels ahead decide: the one ahead on the
;; right empty turns right, the one ahead on the left full turns left, and
;; otherwise straight on. Turning right first keeps two regions that touch
;; only at a corner apart, as the 4-connected fill did.
;; The start corner is always a turn — the walk arrives at it heading
;; north up the start pixel's left edge — and is passed only once, because
;; the three pixels round it other than the start are all outside.
(loop [x x0 y y0 d 0 out [x0 y0]]
(let [[l r] (case d
0 [[x (dec y)] [x y]]
1 [[x y] [(dec x) y]]
2 [[(dec x) y] [(dec x) (dec y)]]
3 [[(dec x) (dec y)] [x (dec y)]])
nd (cond (not (apply in? r)) (mod (inc d) 4)
(apply in? l) (mod (+ d 3) 4)
:else d)
out (if (or (= nd d) (and (= x x0) (= y y0))) out (conj out x y))
[dx dy] (dirs nd)
nx (+ x dx) ny (+ y dy)]
(if (and (= nx x0) (= ny y0))
out
(recur nx ny nd out))))))
(defn- bounds
"`[x0 y0 x1 y1]` round what the mask holds, a pixel wider each way so a gap
open to the outside reaches the edge of it — or nil for an empty mask."
[{:keys [w h ^js buf]}]
(let [b #js [w h -1 -1]]
(dotimes [o (* w h)]
(when (== 1 (aget buf o))
(let [x (mod o w) y (quot o w)]
(aset b 0 (min (aget b 0) x)) (aset b 1 (min (aget b 1) y))
(aset b 2 (max (aget b 2) x)) (aset b 3 (max (aget b 3) y)))))
(when (<= 0 (aget b 2))
[(max 0 (dec (aget b 0))) (max 0 (dec (aget b 1)))
(min (dec w) (inc (aget b 2))) (min (dec h) (inc (aget b 3)))])))
(defn- buffer-pieces
"Each piece of the mask bigger than `smallest` pixels, biggest first, as
`{:outer ring :holes [ring …]}` at full resolution: a hole is a gap inside a
piece that does not reach the outside, of more than `smallest` pixels."
([m] (buffer-pieces m 2))
([{:keys [w h ^js buf] :as m} smallest]
(when-let [[x0 y0 x1 y1 :as box] (bounds m)]
(let [lab (js/Int32Array. (* w h))
stack (js/Int32Array. (* w h))
;; Pieces get positive labels and gaps negative ones, in raster
;; order, so each is labelled from its own first pixel.
found (let [acc (array)]
(doseq [y (range y0 (inc y1))]
(dotimes [i (inc (- x1 x0))]
(let [o (+ x0 i (* y w))]
(when (zero? (aget lab o))
(let [ink (aget buf o)
id (cond-> (inc (.-length acc)) (zero? ink) -)]
(.push acc (assoc (flood! lab buf w box ink o id stack)
:id id :start o)))))))
(vec acc))
in (fn [id] (fn [x y] (and (< -1 x w) (< -1 y h) (== id (aget lab (+ x (* y w)))))))
;; A hole's first pixel has a pixel of its piece directly above it:
;; anything else above would be the hole itself, and earlier.
holes (group-by #(aget lab (- (:start %) w))
(filter #(and (neg? (:id %)) (not (:edge? %)) (< smallest (:size %))) found))]
(->> found
(filter #(and (pos? (:id %)) (< smallest (:size %))))
(sort-by (comp - :size))
(mapv (fn [{:keys [id start]}]
{:outer (ring w (in id) start)
:holes (mapv #(ring w (in (:id %)) (:start %)) (get holes id))})))))))
(defn pieces
([m] (pieces m 2))
([m smallest]
(let [[ox oy] (or (:origin m) [0 0])
shift (fn [ring] (mapv (fn [i v] (+ v (if (even? i) ox oy))) (range) ring))]
(mapv (fn [p] (-> p (update :outer shift) (update :holes #(mapv shift %))))
(buffer-pieces m smallest)))))
(defn rings
"The outside of every piece of the mask, as flat corner points, biggest
first."
([m] (rings m 2))
([m smallest] (mapv :outer (pieces m smallest))))
(defn- area
"The area inside ring `pts`, by the shoelace."
[pts]
(let [ps (vec (partition 2 pts)) n (count ps)]
(js/Math.abs (/ (reduce + (map (fn [i] (let [[ax ay] (ps i) [bx by] (ps (mod (inc i) n))]
(- (* ax by) (* bx ay))))
(range n)))
2))))
(defn- bridge
"Ring `outer` with `hole` joined into it: from the hole's leftmost point,
along its row to the nearest edge of `outer` on the left, round the hole, and
back. Both are on a whole-pixel row, so the bridge is never filled. See the
namespace docstring."
[outer hole]
(let [hs (vec (partition 2 hole))
k (apply min-key (comp first hs) (range (count hs)))
[hx hy] (hs k)
os (vec (partition 2 outer))
n (count os)
;; The edge the row meets nearest on the left. A level edge is skipped:
;; running along one draws nothing either.
[i qx] (->> (range n)
(keep (fn [i]
(let [[ax ay] (os i) [bx by] (os (mod (inc i) n))]
(when (and (not= ay by) (<= (min ay by) hy (max ay by)))
(let [x (+ ax (* (/ (- hy ay) (- by ay)) (- bx ax)))]
(when (< x hx) [i x]))))))
(reduce (fn [best c] (if (or (nil? best) (< (second best) (second c))) c best))
nil))]
(if i
(vec (concat (apply concat (subvec os 0 (inc i)))
[qx hy]
(apply concat (subvec hs k)) (apply concat (subvec hs 0 k))
[hx hy qx hy]
(apply concat (subvec os (inc i)))))
outer)))
(defn- dp
"Douglas-Peucker over the open run of points `i`..`j` of `xs`/`ys`: the
indices kept so that no point between is further than `tol` from the line."
[^js xs ^js ys i j tol]
(let [ax (aget xs i) ay (aget ys i) bx (aget xs j) by (aget ys j)
dx (- bx ax) dy (- by ay) l (js/Math.hypot dx dy)
[k d] (reduce (fn [[_ best :as acc] k]
(let [px (- (aget xs k) ax) py (- (aget ys k) ay)
e (if (zero? l) (js/Math.hypot px py) (/ (js/Math.abs (- (* dx py) (* dy px))) l))]
(if (< best e) [k e] acc)))
[nil 0] (range (inc i) j))]
(if (and k (< tol d))
(into (dp xs ys i k tol) (rest (dp xs ys k j tol)))
[i j])))
(defn fit
"Ring `pts` with as few points as keep every point of it within `tol` pixels
of the result: Douglas-Peucker, closed by splitting at the point furthest from
the first. A pixel staircase within a pixel of a diagonal becomes the
diagonal, and a square corner stays — the points go where the shape bends."
[pts tol]
(let [c (quot (count pts) 2)]
(if (<= c 3)
(vec pts)
(let [xs (js/Float64Array. (take-nth 2 pts)) ys (js/Float64Array. (take-nth 2 (rest pts)))
far (apply max-key #(js/Math.hypot (- (aget xs %) (aget xs 0)) (- (aget ys %) (aget ys 0)))
(range c))
xs2 (js/Float64Array. (inc c)) ys2 (js/Float64Array. (inc c))
_ (dotimes [k c] (aset xs2 k (aget xs k)) (aset ys2 k (aget ys k)))
_ (do (aset xs2 c (aget xs 0)) (aset ys2 c (aget ys 0)))
keep (into (dp xs2 ys2 0 far tol) (rest (butlast (dp xs2 ys2 far c tol))))]
(if (< (count keep) 3)
(vec pts)
(into [] (mapcat (fn [k] [(aget xs k) (aget ys k)])) keep))))))
(defn- crosses?
"Does any edge of ring `a` cross any edge of ring `b`?"
[a b]
(let [edges (fn [r] (let [ps (vec (partition 2 r)) n (count ps)]
(map (fn [i] [(ps i) (ps (mod (inc i) n))]) (range n))))
side (fn [[ax ay] [bx by] [px py]] (- (* (- bx ax) (- py ay)) (* (- by ay) (- px ax))))]
(some (fn [[p q]]
(some (fn [[r s]]
(and (neg? (* (side p q r) (side p q s)))
(neg? (* (side r s p) (side r s q)))))
(edges b)))
(edges a))))
(defn- perimeter [pts]
(let [ps (vec (partition 2 pts)) n (count ps)]
(reduce + (map (fn [i] (let [[ax ay] (ps i) [bx by] (ps (mod (inc i) n))]
(js/Math.hypot (- bx ax) (- by ay))))
(range n)))))
(defn ->js
"Flat ring `ring` as `polygon-clipping` takes one."
[ring]
(into-array (map into-array (partition 2 ring))))
(defn ->ring
"A ring back from `polygon-clipping`, flat, without the point it repeats to
close."
[^js r]
(into [] (mapcat identity) (butlast (map vec (array-seq r)))))
(defn rings-of
"Piece `p` of `pieces` fitted to within `tol` pixels, outside first, holes
after.
EVERY HOLE IS KEPT, and kept simple: fitted like the outside, then clipped to
the inside of it and clear of the holes before it. Fitting each ring on its
own can push a hole's edge across the outside's, and a crossing is what makes
the even-odd fill cut through the body; clipping takes exactly that part off
and leaves the few points the fit chose. Fitting closer until nothing crossed
was tried, and spent hundreds of points on what is plainly a straight line."
[{:keys [outer holes]} tol]
(let [outer (fit outer tol)]
(reduce (fn [rs hole]
(let [h (fit hole tol)]
(if (not-any? #(crosses? h %) rs)
(conj rs h)
(let [inside (clipping/intersection #js [(->js h)] #js [(->js outer)])
clear (if (next rs)
(clipping/difference inside (into-array (map #(array (->js %)) (rest rs))))
inside)]
(into rs (comp (map #(->ring (aget % 0))) (filter #(< 2 (area %))))
(array-seq clear))))))
[outer]
(sort-by (comp - area) holes))))
(defn join
"Rings `[outer & holes]` as one ring, each hole bridged in, leftmost first."
[[outer & holes]]
(reduce bridge outer (sort-by #(apply min (take-nth 2 %)) holes)))
(defn polygon
"Piece `p` of `pieces` as the one ring a shape is made of. See `rings-of`."
[p tol]
(join (rings-of p tol)))

View file

@ -1,12 +1,57 @@
(ns arthur.domain.paint (ns arthur.domain.paint
"Small authored polygon operations. Paint nodes read timeline frames directly; "Small authored polygon operations, each on a named symbol. Paint nodes read
the roto root's exposure and picture sampling must not quantise a hand edit." their symbol's frames directly; a roto instance's exposure and picture sampling
must not quantise a hand edit.
A SHAPE'S ORIGIN IS THE MIDDLE OF WHAT IT DRAWS. Points arrive here in the
space the node's `[:xform :pos]` lives in — that is what `nest/drawn-inside`
hands over — and `centred` splits them into a ring about the origin and the
`pos` that puts it back where it was drawn. See `centred`: it is the invariant
the whole pivot story rests on, and this namespace is where it is established."
(:require [arthur.domain.channel :as channel])) (:require [arthur.domain.channel :as channel]))
(def geometry [:geom :pts]) (def geometry [:geom :pts])
(defn shapes [clip] (defn middle
(->> (get-in clip [:timelines :main :nodes]) "The middle of the box round flat points `pts`, in their own space."
[pts]
(let [xs (take-nth 2 pts)
ys (take-nth 2 (rest pts))]
[(/ (+ (apply min xs) (apply max xs)) 2)
(/ (+ (apply min ys) (apply max ys)) 2)]))
(defn centred
"Flat points `pts`, given in the space a node's `pos` lives in, as
`[ring pos]`: the same drawing about the origin, and the position that puts it
back exactly where it was.
THE ORIGIN OF A SHAPE IS THE MIDDLE OF WHAT IT DRAWS, so a shape needs no
pivot of its own: `[:xform :pivot]` defaults to the node's origin, and for a
drawing that IS the middle of the drawing. A stroke stored exactly as it was
drawn would have its origin at the SYMBOL's origin, which on the stage is the
top-left corner — 126 px away on a 320x200 stage, an orbit wider than the
stage — and that is what every drawing would turn about with a default pivot
and no centring here.
IT IS WORTH DOING ANYWAY, NOW THAT THERE IS A PIVOT, because this is the half
nobody has to choose. A pivot is a stored choice and a default has to be a good
one: with the origin on the content the default is already right, a turn writes
`rot` alone, and nothing is stored on the node for anybody to have to look at.
A symbol instance is the case that cannot do this — its origin is its symbol's,
and moving a symbol's origin would move every drawing inside it out from under
everything that reads them — so it gets a pivot written where it is placed;
see `clip/place-symbol`.
Hand-authored scenes have always been written this way — `demo/scene.edn`'s
card is `[-44 -30 44 -30 44 30 -44 30]` with its place in `pos` — so this is
the paint tool joining the convention rather than a new one."
[pts]
(let [[cx cy] (middle pts)]
[(into [] (map-indexed (fn [i v] (- v (if (even? i) cx cy)))) pts)
[cx cy]]))
(defn shapes [clip sid]
(->> (get-in clip [:symbols sid :nodes])
(filter (fn [[_ node]] (:paint? node))) (filter (fn [[_ node]] (:paint? node)))
(sort-by (comp :z val)) (sort-by (comp :z val))
vec)) vec))
@ -15,43 +60,116 @@
(let [frames (sort (keys (:keys ch)))] (let [frames (sort (keys (:keys ch)))]
(or (last (take-while #(<= % frame) frames)) (first frames)))) (or (last (take-while #(<= % frame) frames)) (first frames))))
(defn new-shape [clip id frame points color] (defn new-shape
(let [end (get-in clip [:timelines :main :frames]) "`clip` with a shape drawn at `points` — in the space the new node's `pos` will
live in, which is the symbol's own coordinates — on frame `frame` of `sid`.
THE RING IS CENTRED AND THE MIDDLE GOES IN `pos`, which is the whole of
`centred`: the shape's origin is what it draws, so it turns and scales about
itself with nothing stored and nothing solved. This is the one place every
drawing is born — the pen, the brush, and each piece the eraser leaves — so it
is the one place the invariant has to be established."
[clip sid id frame points color]
(let [end (get-in clip [:symbols sid :frames])
z (str "z" (js/Date.now) "-" (name id))] z (str "z" (js/Date.now) "-" (name id))]
(if (and (<= 0 frame) (< frame end) (>= (count points) 6) (if (and (<= 0 frame) (< frame end) (>= (count points) 6)
(even? (count points))) (even? (count points)))
(assoc-in clip [:timelines :main :nodes id] (let [[ring pos] (centred points)]
{:id id :name (str "shape " (inc (count (shapes clip)))) (assoc-in clip [:symbols sid :nodes id]
{:id id :name (str "shape " (inc (count (shapes clip sid))))
:kind :poly :paint? true :parent nil :z z :kind :poly :paint? true :parent nil :z z
:span [frame end] :span [frame end]
:channels {geometry (channel/keyed {frame points}) :channels {geometry (channel/keyed {frame ring} :hold)
[:style :color] (channel/framed color)}}) [:xform :pos] (channel/framed pos)
[:style :color] (channel/framed color)}}))
clip))) clip)))
(defn add-key [clip id frame] (defn add-key [clip sid id frame]
(let [path [:timelines :main :nodes id] (let [path [:symbols sid :nodes id]
node (get-in clip path) node (get-in clip path)
ch (get-in node [:channels geometry]) ch (get-in node [:channels geometry])
[start end] (:span node)] [start end] (:span node)]
(if (and (:paint? node) (<= start frame) (< frame end) ch) (if (and (:paint? node) (<= start frame) (< frame end) ch)
(assoc-in clip (into path [:channels geometry :keys frame]) (assoc-in clip (into path [:channels geometry :keys frame])
(vec (channel/value-at ch frame))) ;; A drawing is authored and keyed, never dense, so there is
;; no tier-2 store to read it out of.
(vec (channel/value-at ch frame nil)))
clip))) clip)))
(defn set-vertex [clip id key-frame vertex [x y]] (defn set-vertex [clip sid id key-frame vertex [x y]]
(let [path [:timelines :main :nodes id :channels geometry :keys key-frame] (let [path [:symbols sid :nodes id :channels geometry :keys key-frame]
points (get-in clip path) points (get-in clip path)
i (* 2 vertex)] i (* 2 vertex)]
(if (and points (< (inc i) (count points))) (if (and points (< (inc i) (count points)))
(assoc-in clip path (-> points (assoc i x) (assoc (inc i) y))) (assoc-in clip path (-> points (assoc i x) (assoc (inc i) y)))
clip))) clip)))
(defn set-segment-interp [clip id key-frame interp] (defn- every-key
(let [node (get-in clip [:timelines :main :nodes id]) "`f` over the points of every key of the shape's geometry."
keys (get-in node [:channels geometry :keys])] [clip sid id f]
(if (and (:paint? node) (contains? keys key-frame) (let [path [:symbols sid :nodes id :channels geometry :keys]]
(some #(< key-frame %) (clojure.core/keys keys)) (if (and (:paint? (get-in clip [:symbols sid :nodes id])) (map? (get-in clip path)))
(#{:hold :linear} interp)) (update-in clip path update-vals f)
(assoc-in clip [:timelines :main :nodes id :channels geometry clip)))
:segments key-frame] interp)
(defn insert-vertex
"A new point after point `i`, a fraction `t` of the way along the edge to the
next, on EVERY key: a point's index is what it is across keys, so a tween
between two keys only means anything while they have the same points. The
shape does not change on any key."
[clip sid id i t]
(every-key clip sid id
(fn [pts]
(let [n (quot (count pts) 2)
j (mod (inc i) n)
at #(+ (nth pts (+ (* 2 i) %))
(* t (- (nth pts (+ (* 2 j) %)) (nth pts (+ (* 2 i) %)))))]
(if (< i n)
(-> (subvec pts 0 (* 2 (inc i)))
(conj (at 0) (at 1))
(into (subvec pts (* 2 (inc i)))))
pts)))))
(defn delete-vertex
"Point `i` gone from every key, as `insert-vertex` adds one. A triangle keeps
its three."
[clip sid id i]
(every-key clip sid id
(fn [pts]
(if (and (< 6 (count pts)) (< (* 2 i) (count pts)))
(into (subvec pts 0 (* 2 i)) (subvec pts (* 2 (inc i))))
pts))))
(defn set-points
"Key `key-frame` of the shape is `points`, whatever it held, IN THE NODE'S OWN
COORDINATES: a cut writing back the piece it kept.
The node's origin is left where it is, which is why this is the form a cut
uses. Re-centring would have to move `pos` to compensate, and `pos` can be
keyed and the drawing can be keyed, so there is no one middle to move it to —
the only exact moment for that is while the transform is static, which is what
`place-points` is for."
[clip sid id key-frame points]
(let [path [:symbols sid :nodes id :channels geometry :keys key-frame]]
(if (get-in clip path)
(assoc-in clip path points)
clip)))
(defn place-points
"Key `key-frame` of the shape is `points`, GIVEN IN THE SPACE ITS `pos` LIVES
IN: centred as `new-shape` centres them, with `pos` moved to put them back.
What a re-fit writes. Adjusting a brush stroke's fit re-traces the same stroke
from the stage pixels it was painted in, so its points arrive in the same space
they did when the shape was made, and writing them as the node's own would move
the drawing by its own position. Only ever used on a shape a stroke has just
made, which is why moving `pos` is exact here: nothing is keyed yet."
[clip sid id key-frame points]
(let [path [:symbols sid :nodes id :channels geometry :keys key-frame]]
(if (get-in clip path)
(let [[ring pos] (centred points)]
(-> clip
(assoc-in path ring)
(assoc-in [:symbols sid :nodes id :channels [:xform :pos]]
(channel/framed pos))))
clip))) clip)))

View file

@ -1,15 +1,13 @@
(ns arthur.domain.palette (ns arthur.domain.palette
"The indexed palette. "Project palette assets and their render-time index banks.
THE RULE, and it is a rule rather than a default: a part carries a palette Drawing data stores a dumb LOCAL SLOT NUMBER. A symbol or instance supplies
INDEX, never a sampled RGB value. Sampling colour off the footage produces a the palette context. `compile` gives project palettes disjoint ranges in the
pixel-art filter, and it does so irrecoverably — once a shape holds a measured raster index space, so differently-paletted subtrees can coexist. The ranges
colour there is no way back to an authored one, because the information that it are derived, never persisted: adding a palette never rewrites drawing data.")
was ever a choice is gone. Every `[:style :color]` channel holds one of the
keywords below.
Entries are ordered, and the order IS the index the raster writes. Inserting in (def default-id :arthur/default)
the middle renumbers every stored index, so new tones append.") (def inherit :arthur.palette/inherit)
(def entries (def entries
[{:name :bg :hex "#12141c"} [{:name :bg :hex "#12141c"}
@ -48,3 +46,106 @@
(def rgb (def rgb
"Index -> [r g b], precomputed." "Index -> [r g b], precomputed."
(mapv hex->rgb hexes)) (mapv hex->rgb hexes))
(def default-palette
{:id default-id :name "Arthur"
:slots (into entries (repeat 7 {:hex "#000000"}))})
(defn palettes [clip]
(if (seq (:palettes clip)) (:palettes clip) {default-id default-palette}))
(defn default-palette-id [clip]
(let [ps (palettes clip)]
(or (:default-palette clip)
(when (contains? ps default-id) default-id)
(first (sort-by str (keys ps))))))
(defn- slot-index [p value]
(cond
(and (integer? value) (<= 0 value) (< value (count (:slots p)))) value
;; Compatibility for existing documents. New drawing data is numeric.
(keyword? value) (first (keep-indexed #(when (= value (:name %2)) %1) (:slots p)))
:else nil))
(defn compile
"Compile the project's palette assets into the one ramp used by the raster.
Index 255 remains the conspicuous bad-data sentinel."
[clip]
(let [selection-values (fn [x]
(cond
(nil? x) []
(and (map? x) (:keys x)) (vals (:keys x))
(and (map? x) (contains? x :value)) [(:value x)]
:else [x]))
used (into #{(default-palette-id clip)}
(mapcat selection-values)
(concat (map :palette (vals (:symbols clip)))
(map :palette-channel (vals (:symbols clip)))
(map (fn [sym]
(when (= :palette (:type sym)) (:palette-ref sym)))
(vals (:symbols clip)))
(for [sym (vals (:symbols clip))
n (vals (:nodes sym))]
(:palette n))
(for [sym (vals (:symbols clip))
n (vals (:nodes sym))]
(get-in n [:channels [:palette]]))))
ps (select-keys (palettes clip) used)
ordered (sort-by (comp str key) ps)
offsets (loop [xs ordered at 0 out {}]
(if-let [[id p] (first xs)]
(recur (next xs) (+ at (count (:slots p))) (assoc out id at))
out))
total (reduce + (map #(count (:slots (val %))) ordered))]
(when (> total 255)
(throw (ex-info "palettes visible in one render use more than 255 slots"
{:slots total :palettes (count ps)})))
{:palettes ps
:default (default-palette-id clip)
:offsets offsets
:ramp (vec (mapcat (fn [[_ p]] (map #(hex->rgb (:hex %)) (:slots p))) ordered))
:bg (get offsets (default-palette-id clip) 0)}))
(defn render-index
"A local slot (or a legacy tone keyword) in palette `id` -> raster index."
[{:keys [palettes offsets]} id value]
(let [id (if (map? id) (:from id) id)
p (get palettes id)
i (and p (slot-index p value))]
(if (some? i) (+ (get offsets id 0) i) 255)))
(defn background-index
"The raster index of slot zero in the resolver's active palette.
`index-of` remains a supported legacy palette map for domain callers; it has
no palette banks or active selection, so its established `:bg` index applies."
[palette active]
(if (and (:palettes palette) (:offsets palette))
(render-index palette active 0)
(get palette :bg 0)))
(defn effective-ramp
"The compiled ramp with the active palette blend applied. A hold returns the
original vector; a linear palette segment rewrites only the left palette's
bank, which is the bank the indexed raster was drawn into."
[{:keys [palettes offsets ramp]} active]
(if-let [{:keys [from to t]} (when (map? active) active)]
(let [a (get palettes from)
b (get palettes to)
at (get offsets from)
n (min (count (:slots a)) (count (:slots b)))]
(if (and a b (number? at) (number? t))
(reduce (fn [out i]
(let [x (hex->rgb (get-in a [:slots i :hex]))
y (hex->rgb (get-in b [:slots i :hex]))]
(assoc out (+ at i)
(mapv #(js/Math.round (+ %1 (* t (- %2 %1)))) x y))))
ramp (range n))
ramp))
ramp))
(defn valid-palette? [{:keys [id name slots]}]
(and id (string? name) (seq name) (vector? slots) (pos? (count slots))
(<= (count slots) 255)
(every? #(and (string? (:hex %))
(boolean (re-matches #"#[0-9a-fA-F]{6}" (:hex %)))) slots)))

View file

@ -0,0 +1,226 @@
(ns arthur.domain.pick
"What is under the pointer on the stage, and which row a click on it selects.
THE OPS ALREADY SAY. The stage draws a flat list of ops, and each op's `:node`
is the row path of what drew it — the instances down to it and its own id — so
hit-testing is a walk of the list, topmost first, and needs nothing resolved.
WHICH LEVEL a click selects is Figma's and Illustrator's rule, and Flash's
without its edit mode: a click selects the thing in the open symbol, a
double-click goes one level into what is selected, ⌥-click goes straight to the
shape itself. A click inside what is selected keeps it, so a deep selection can
be dragged; one elsewhere selects at the same depth, beside it."
(:require [arthur.domain.channel :as ch]
[arthur.domain.clip :as clip]
[arthur.domain.node :as node]
[arthur.domain.palette :as pal]))
(def ^:private slop
"Stage pixels a click may miss by. The shapes here are a few pixels across."
2)
(defn- path-of [op]
(let [n (:node op)] (if (vector? n) n [n])))
(defn- near-segment? [x y ax ay bx by]
(let [dx (- bx ax) dy (- by ay)
l2 (+ (* dx dx) (* dy dy))
t (if (zero? l2) 0 (-> (/ (+ (* (- x ax) dx) (* (- y ay) dy)) l2) (max 0) (min 1)))
ex (- x (+ ax (* t dx)))
ey (- y (+ ay (* t dy)))]
(<= (+ (* ex ex) (* ey ey)) (* slop slop))))
(defn- on-poly? [^js pts n x y]
(let [px #(aget pts (* 2 (mod % n)))
py #(aget pts (inc (* 2 (mod % n))))]
(or (odd? (count (filter (fn [i]
(let [ay (py i) by (py (inc i))]
(and (not= (> ay y) (> by y))
(< x (+ (px i) (/ (* (- y ay) (- (px (inc i)) (px i)))
(- by ay)))))))
(range n))))
(some #(near-segment? x y (px %) (py %) (px (inc %)) (py (inc %))) (range n)))))
(defn- trace-corners
"A trace op's image rectangle on the stage, as four [x y] corners."
[{[w h] :size m :m}]
(for [[x y] [[0 0] [w 0] [w h] [0 h]]]
[(+ (* (aget m 0) x) (* (aget m 2) y) (aget m 4))
(+ (* (aget m 1) x) (* (aget m 3) y) (aget m 5))]))
(defn- on? [{:keys [kind pts n cx cy r size m]} x y]
(case kind
:poly (on-poly? pts n x y)
:disc (<= (js/Math.hypot (- x cx) (- y cy)) (+ r slop))
:rect (let [h (+ slop (/ size 2))]
(and (<= (js/Math.abs (- x cx)) h) (<= (js/Math.abs (- y cy)) h)))
;; Into the image's own pixels, where the test is a rectangle however the
;; layer is turned or scaled.
:trace (when-let [inv (node/invert m)]
(let [[w h] size
u (+ (* (aget inv 0) x) (* (aget inv 2) y) (aget inv 4))
v (+ (* (aget inv 1) x) (* (aget inv 3) y) (aget inv 5))]
(and (<= 0 u) (< u w) (<= 0 v) (< v h))))
false))
;; A HOLE AND A LIGHT HAVE BOUNDS LIKE ANYTHING ELSE. A knockout and a remap
;; were both left out of this, which is the rule `hit-op` plays by — a click
;; goes through them to what shows under — carried over to a gesture it is wrong
;; for: a marquee asks what is INSIDE it, not what a point lands on, and leaving
;; them out meant the only way to select one anywhere was its timeline row.
(defn- op-bounds [{:keys [kind pts n cx cy r size] :as op}]
(case kind
:trace (let [cs (trace-corners op)]
[(apply min (map first cs)) (apply min (map second cs))
(apply max (map first cs)) (apply max (map second cs))])
:poly (reduce (fn [b i]
(let [x (aget pts (* 2 i)) y (aget pts (inc (* 2 i)))]
(if b (let [[x0 y0 x1 y1] b]
[(min x0 x) (min y0 y) (max x1 x) (max y1 y)])
[x y x y]))) nil (range n))
:disc [(- cx r) (- cy r) (+ cx r) (+ cy r)]
:rect (let [h (/ size 2)] [(- cx h) (- cy h) (+ cx h) (+ cy h)])
nil))
(defn in-rect
"Distinct row paths whose drawn bounds intersect `[x0 y0 x1 y1]`. `depth`
chooses objects at one hierarchy level, just as an ordinary stage click does."
[ops [ax ay bx by] depth]
(let [[rx0 rx1] [(min ax bx) (max ax bx)]
[ry0 ry1] [(min ay by) (max ay by)]]
(->> ops
(keep (fn [op]
(when-let [[x0 y0 x1 y1] (op-bounds op)]
(when (and (<= x0 rx1) (<= rx0 x1) (<= y0 ry1) (<= ry0 y1))
(let [path (path-of op)]
(subvec path 0 (min (count path) (max 1 depth))))))))
distinct vec)))
(defn- prefix? [a b]
(and (<= (count a) (count b)) (= a (subvec b 0 (count a)))))
(defn hit-op
"The topmost op in `ops`, in draw order, that SHOWS at stage point `[x y]`, or
nil. A knockout is never hit: it is a hole, and a click in it is a click on
whatever shows through — so it hides the ops beneath it in its own symbol, of
the colour it clears. A remap is the same kind of thing the other way up: it is
light on what is under it, so a click goes through it to what it lights.
EXCEPT WHAT IS ALREADY SELECTED, which `selected` is the row path of. Both of
those rules are about reaching PAST a shape, and neither has anything to say
about the shape you have in your hands: a selected remap or knockout answered
no point on the stage at all, so there was nothing to drag it by — the stage's
move gesture is this hit-test on the ops, and `handles` draws a box round
bounds that nothing could then grab. The first press on one started a marquee
instead, which selected nothing and so threw the selection away, and that is
the whole of why a palette swapper could be selected from a timeline row and
then neither moved nor resized. The ordinary depth rule — a click inside what
is selected keeps it, so a deep selection can be dragged — is `choose`'s, and
this is the same rule reaching one step further down."
([ops point] (hit-op ops point nil))
([ops [x y] selected]
(:op (reduce (fn [holes op]
(cond
(not (on? op x y)) holes
(and selected (prefix? selected (path-of op))) (reduced {:op op})
(:lut op) holes
(:knock op) (conj holes [(pop (path-of op)) (:knock op)])
(some (fn [[in k]] (and (prefix? in (path-of op))
(or (neg? k) (== k (:color op)))))
holes) holes
:else (reduced {:op op})))
[] (let [{traces true drawn false} (group-by #(= :trace (:kind %)) (rseq (vec ops)))]
(concat drawn traces))))))
(defn hit
"The row path of the topmost op that shows at stage point `[x y]`, or nil.
`selected` is what is selected now, which a light or a hole does not hide."
([ops point] (hit ops point nil))
([ops point selected] (some-> (hit-op ops point selected) path-of)))
(defn choose
"The row path a click on `hit` selects, with `selected` the one selected now."
[selected hit deep?]
(cond
(nil? hit) nil
deep? hit
(and selected (prefix? selected hit)) selected
(and selected (prefix? (pop selected) hit)) (subvec hit 0 (count selected))
:else [(first hit)]))
(defn deeper
"One level into `selected` towards `hit`, for a double-click."
[selected hit]
(if (and selected hit (prefix? selected hit) (< (count selected) (count hit)))
(subvec hit 0 (inc (count selected)))
selected))
(defn bounds-of
"A closure from a frame to `[x0 y0 x1 y1]` around what node `n` draws on it, in
its own coordinates — nil on a frame it draws nothing on.
A CLOSURE, as `clip/resolver` is, and for the same reason it is: inside an
instance is its whole symbol resolved at that frame, and a resolver costs the
symbol to BUILD and a lookup to RUN. Asking frame by frame through a fresh one
is a resolver per frame.
WHAT A SELECTION BOX IS DRAWN FROM, and what a pivot DEFAULTS to: a node
nobody has pivoted turns about the middle of these bounds, and the first turn
or scale writes that point down as its `[:xform :pivot]` — `gesture/pivot` and
`gesture/with-pivot`. So the box and the cross start out as one computation,
and ⌖ in the inspector brings a chosen pivot back to it (`gesture/centred`).
A DEFAULT, AND NOT WHERE THE PIVOT LIVES. Once a pivot is the node's own, this
is not consulted for it again: the pivot is a choice, and a choice that
silently followed the drawing would mean adding a shape to a symbol re-aimed
every keyed spin of every instance of it. See `domain/gesture`."
[document store sid n]
(let [grow (fn [[x0 y0 x1 y1 :as b] x y]
(if b [(min x0 x) (min y0 y) (max x1 x) (max y1 y)] [x y x y]))
at (fn [f p] (ch/value-at (get (node/channels n) p) f store))]
(case (if (some->> (node/source n) (clip/symbol document) clip/trace?) :trace (:kind n))
;; A tracing symbol draws nothing to resolve: what it covers is its media's
;; pixels, in its own coordinates, on every frame it shows one.
:trace
(let [{:keys [width height]} (clip/symbol document (node/source n))]
(fn [f] (when (clip/placed-frame document sid n f) [0 0 width height])))
:instance
;; ONE RESOLVER PER DRAWING THE LANE CAN SHOW, built once for the reason
;; the single one used to be: a resolver costs the symbol to build and a
;; lookup to run, and a lane asked frame by frame through a fresh one is a
;; resolver per frame.
(let [loop? (get-in n [:time :loop?])
resolvers (into {} (map (fn [child]
[child (clip/resolver document child store
pal/index-of {:grid-fps (clip/fps document child)})]))
(node/sources n))]
(fn [f0]
(let [shown (clip/placed-frame document sid n f0)
frames (when shown (clip/frames document (:symbol shown)))
f (when (number? frames)
(if loop? (mod (:frame shown) frames) (:frame shown)))
resolve (when shown (get resolvers (:symbol shown)))]
(when (and resolve (< -1 f frames))
(reduce (fn [b {:keys [kind pts n cx cy r size]}]
(case kind
:poly (reduce #(grow %1 (aget pts (* 2 %2)) (aget pts (inc (* 2 %2))))
b (range n))
:disc (-> b (grow (- cx r) (- cy r)) (grow (+ cx r) (+ cy r)))
:rect (let [h (/ size 2)]
(-> b (grow (- cx h) (- cy h)) (grow (+ cx h) (+ cy h))))
b))
nil
(resolve f))))))
:poly (fn [f]
(let [pts (at f [:geom :pts])]
(when-not (ch/nothing? pts)
(reduce (fn [b i] (grow b (ch/component pts (* 2 i)) (ch/component pts (inc (* 2 i)))))
nil (range (quot (if (vector? pts) (count pts) (.-length pts)) 2))))))
:disc (fn [f]
(let [r (at f [:geom :radius])]
(when-not (ch/nothing? r) [(- r) (- r) r r])))
:rect (fn [f]
(let [s (at f [:geom :size])]
(when-not (ch/nothing? s) (let [h (/ s 2)] [(- h) (- h) h h]))))
(constantly nil))))

View file

@ -88,7 +88,7 @@
(defn encoder (defn encoder
"(fn [raster ramp] -> promise of PNG bytes), for one stage size and one zoom. "(fn [raster ramp] -> promise of PNG bytes), for one stage size and one zoom.
Built once per export rather than per frame, in the shape `timeline/resolver` Built once per export rather than per frame, in the shape `symbol/resolver`
already uses: everything that does not change frame to frame is held here. What already uses: everything that does not change frame to frame is held here. What
that buys is the scanline scratch, which at zoom 6 is seven megabytes — a that buys is the scanline scratch, which at zoom 6 is seven megabytes — a
per-frame allocation of that size is the one thing that would make a long export per-frame allocation of that size is the one thing that would make a long export

View file

@ -2,16 +2,146 @@
"An instance's explicit, held choices of source pose for each shape group. "An instance's explicit, held choices of source pose for each shape group.
A track is {local-frame -> source-frame}. The key is when the cut happens; A track is {local-frame -> source-frame}. The key is when the cut happens;
the value is the frozen pose to read. Skipped source frames remain available.") the value is the frozen pose to read. Skipped source frames remain available.
AND THE PRESERVE-SNAP, which is the same question asked where nobody has
answered it by hand. A grid slot already picks a native frame — the latest one
at or before its time, see `domain/cadence` — and that pick has no opinion about
content, so at 12fps out of 30 it drops two frames in three and cannot know that
one of them is where the mouth shut. The snap gives it one: a slot reads the
latest MARKED frame inside its own gap. `marks` is where the marks come from and
`snapped-frame` is the rule; `domain/symbol` seats the rule as the DEFAULT pose,
so a hand cut still beats it without anything having to say so."
(:require [arthur.domain.channel :as ch]
[arthur.domain.node :as node]))
(defn prepare (defn prepare
"Sort exposure tracks once when building a resolver." "Sort pose tracks once when building a resolver."
[tracks] [tracks]
(into {} (into {}
(map (fn [[group entries]] (map (fn [[group entries]]
[group (vec (sort-by first entries))])) [group (vec (sort-by first entries))]))
tracks)) tracks))
;; ---------------------------------------------------------------------------
;; the preserve-snap: which frames are worth landing on
(def ^:private closure-cuts
"The `[:vis]` provenances that are a CLOSURE, and therefore a frame worth
landing on.
A SKIPPED FRAME, A HIDDEN FEATURE AND AN ABSENT MEASUREMENT ARE THREE
DIFFERENT FACTS, and marks being derived from a visibility cut is exactly what
makes that easy to blur. `flow/freeze` writes `[:vis]` for three different
reasons: the mouth's aperture against the take's peak, `condition/resolve-blink`
with its cut and dwell, and whether a teeth contour could be extracted. The
first two are a part CLOSING — a thing the picture should land on. The third is
a feature not being there to draw, which says nothing about a performance, and
nor does a `[:vis]` somebody keyed by hand to switch a part off.
So this is a whitelist of two and not a test for `:generated`: reading every
stored `[:vis]` would snap the grid onto the frames a part happened to be
missing on, which is the cadence being dragged about by an absence."
#{:roto/mouth-aperture :roto/blink})
(defn- shut-frames
"The frames channel `c` calls shut, or nil when it is not a closure cut.
Sampled through a CURSOR rather than `ch/value-at` because the frames are read
in order: `value-at` rebuilds the sorted key index per call and the two are
required to agree exactly, so the sequential reader is the cheap half of an
equality the model already guarantees.
`false?` and not falsiness. `ch/absent` is not a closure — a frame the subject
was not on has no mouth to be shut — and a dense `[:vis]` yields 0, which is
truthy in CLJS, so neither obvious test is right. `symbol/visible?` insists on
the same boolean for the same reason."
[c frames store]
(when (contains? closure-cuts (:by (:generated c)))
(let [cur (ch/cursor c store)]
(into [] (filter #(false? (ch/sample! cur %))) (range frames)))))
(defn marks
"Frames worth landing on, per pose group: `{group -> ascending frames}`, or nil
where nothing is marked.
PRECOMPUTED WHEN THE RESOLVER IS BUILT, beside `prepare`,
and this is the reason it is a function rather than a line inside the snap. A
`[:vis]` channel is keys and not dense, so reading one per node per frame would
be cheap — but the snap needs the marks SORTED for a backward lookup, and
building that per frame is what docs/animation-model.md forbids on the render
path.
PER POSE GROUP, because the thing being recovered is one group's closure. The
alternative — snapping the slot itself, where the grid becomes a native frame —
is one native frame for the whole picture, so a head would go two frames stale
for one output frame to fix a mouth. A group's related parts share one answer,
which is what `:pose-group` is already for: the mouth outline, its interior and
the teeth reading different frames is the bug grouping prevents.
No new signal and no new stored field: the cut `flow/freeze` already wrote is
read where it lies, with its thresholding and hysteresis already decided. A
group whose parts carry no closure cut gets no marks and no entry, which is
correct for brows — there is no extreme brow position worth protecting."
[nodes frames store]
(when (and (integer? frames) (pos? frames))
(not-empty
(into {}
(keep (fn [[group ns]]
(let [fs (into (sorted-set)
(mapcat #(shut-frames (get (:channels %) [:vis])
frames store))
ns)]
(when (seq fs) [group (vec fs)]))))
(group-by #(or (:pose-group %) (:id %)) (vals nodes))))))
(defn- latest-mark
"The largest mark at or before `f`, or nil. Binary search, as `held-frame` is:
the marks are sorted once and read in whatever order the transport asks for."
[ms f]
(loop [lo 0 hi (dec (count ms)) hit nil]
(if (> lo hi)
(when (some? hit) (nth ms hit))
(let [mid (bit-shift-right (+ lo hi) 1)]
(if (<= (nth ms mid) f)
(recur (inc mid) hi mid)
(recur lo (dec mid) hit))))))
(defn snapped-frame
"The native frame a grid slot reads: THE LATEST MARK IN `(lo, hi]`, AND `hi`
WHEN THERE IS NONE.
`hi` is the frame the slot defaults to — the latest native frame at or before
its time — and `lo` is the frame the slot BEFORE it defaulted to. So the
half-open interval is exactly the frames this slot is the first to cover, which
are exactly the ones the grid shows to nobody. That one sentence is the whole
mechanism: it recovers a dropped frame out of the slot's own gap, it can never
read a frame another slot already showed, and it cannot reach past `hi`.
BACKWARD ONLY, NEVER FORWARD, and note which direction that actually is because
it is the easy thing to get wrong. A closure at native 13 in a 12-from-30 output
is recovered by the slot whose default is 15, reading 13. It is NOT recovered by
the slot whose default is 12 reaching forward to 13: that slot's instant is
5/12s and native 13's is 13/30s, so it would show the closure 17ms before the
mouth shut. `cadence/frame`'s contract is at-or-before and `cadence_test` asserts
`selected <= f*native/grid` over every grid and native pair, so reaching forward
would break a tested invariant as well as the no-lead rule. Reading 13 two
native frames late is the lateness every hold already has.
An EMPTY interval snaps nothing. A slot that reads what the slot before it read
— a held exposure, or an output grid faster than the content — has no gap of its
own, and `(hi, hi]` contains nothing to find.
Takes no tolerance and will not grow one. The output rate or the exposure
setting has already chosen the sparseness; the only question left is WHICH
native frame an already-decided slot reads, and a second knob here would be a
control with nothing to control."
[marks group lo hi]
(if-let [ms (get marks group)]
(let [m (latest-mark ms hi)]
(if (and (some? m) (> m lo)) m hi))
hi))
(defn held-frame (defn held-frame
"Last value keyed at or before f, or default before the first key." "Last value keyed at or before f, or default before the first key."
[entries f default-frame] [entries f default-frame]
@ -32,16 +162,20 @@
default-frame)) default-frame))
(defn put-cut (defn put-cut
"Set one held pose on a symbol instance. Earlier motion stays untouched." "Set one held pose on an instance inside symbol `sid`. Earlier motion stays
[clip instance group at source] untouched."
(let [node (get-in clip [:timelines :main :nodes instance]) [clip sid instance group at source]
symbol (get-in clip [:timelines (:of node)]) (let [inst (get-in clip [:symbols sid :nodes instance])
length (:frames symbol) ;; A cut is checked against the ONE symbol this cel places. Which
;; drawing a lane shows is a question about the lane's other cels,
;; and each of them owns its own tracks — so there is nothing to union.
placed (get-in clip [:symbols (node/source inst)])
length (or (:frames placed) 0)
active (filter (fn [n] (some :pose-sampled? (vals (:channels n)))) active (filter (fn [n] (some :pose-sampled? (vals (:channels n))))
(vals (:nodes symbol))) (vals (:nodes placed)))
groups (set (map #(or (:pose-group %) (:id %)) active)) groups (set (map #(or (:pose-group %) (:id %)) active))
ids (set (map :id active))] ids (set (map :id active))]
(when-not (and (= :symbol (:kind node)) (when-not (and (= :instance (:kind inst))
(or (contains? groups group) (or (contains? groups group)
(and (vector? group) (= 2 (count group)) (and (vector? group) (= 2 (count group))
(= :node (first group)) (= :node (first group))
@ -50,22 +184,22 @@
(integer? source) (<= 0 source) (< source length)) (integer? source) (<= 0 source) (< source length))
(throw (ex-info "invalid stage pose cut" (throw (ex-info "invalid stage pose cut"
{:instance instance :group group :at at :source source}))) {:instance instance :group group :at at :source source})))
(update-in clip [:timelines :main :nodes instance :playback :tracks group] (update-in clip [:symbols sid :nodes instance :playback :tracks group]
#(assoc (or % {}) at source)))) #(assoc (or % {}) at source))))
(defn remove-cut (defn remove-cut
"Remove a cut; an empty track again follows the normal generated motion." "Remove a cut; an empty track again follows the normal generated motion."
[clip instance group at] [clip sid instance group at]
(let [path [:timelines :main :nodes instance :playback :tracks group]] (let [path [:symbols sid :nodes instance :playback :tracks group]]
(if-let [entries (get-in clip path)] (if-let [entries (get-in clip path)]
(if-let [remaining (not-empty (dissoc entries at))] (if-let [remaining (not-empty (dissoc entries at))]
(assoc-in clip path remaining) (assoc-in clip path remaining)
(update-in clip [:timelines :main :nodes instance :playback :tracks] (update-in clip [:symbols sid :nodes instance :playback :tracks]
dissoc group)) dissoc group))
clip))) clip)))
(defn problems (defn problems
"Errors in one symbol instance's exposure tracks." "Errors in one symbol instance's pose tracks."
[tracks source-frames groups] [tracks source-frames groups]
(cond (cond
(nil? tracks) [] (nil? tracks) []

View file

@ -29,6 +29,31 @@
(:require [arthur.domain.leaf :as leaf] (:require [arthur.domain.leaf :as leaf]
[arthur.domain.wire :as wire])) [arthur.domain.wire :as wire]))
(def schema-version
"The stored document format. A client refuses a project stored under a
different one — see `events/project` — so this is bumped by any change to what
a leaf may contain.
8 added `[:xform :pivot]`, the point a node turns and scales about, which 7 had
deleted as `[:xform :anchor]` on the theory that a peg could stand in for one.
It cannot: a peg is a node and a turn about one is still `pos` solved per
frame, so a KEYED turn of anything whose origin was not its own middle orbited
that origin. See `node/xform-paths` and docs/animation-model.md.
CONVERTED, AND THE ONLY VERSION SO FAR THAT IS, because this one is exact: a
schema-7 node has no pivot, an absent pivot reads as `[0 0]` off
`node/defaults`, and `T(pos)·T(0)·M·T(-0)` is `T(pos)·M` to the bit. So every
stored document composes to the same matrices it did, and the migration only
restamps the version — `clips/migrations/0017`. That is the difference from 7,
which could not convert an anchor it had deleted: dropping one moved anything
since turned or scaled by hand, and tier-2 positions cannot be rewritten at
all. Adding a component with an identity default takes nothing away.
A pivot a schema-7 document never got to choose is still unchosen, and the
first turn or scale of such a node writes one — `gesture/with-pivot` — so the
conversion does not have to guess where anybody wanted it."
8)
(defn block-keys (defn block-keys
"Every tier-2 key a leaf map names, in a stable order." "Every tier-2 key a leaf map names, in a stable order."
[leaves] [leaves]
@ -53,8 +78,8 @@
round-trip a clip through `JSON.parse(JSON.stringify(...))` and be running the round-trip a clip through `JSON.parse(JSON.stringify(...))` and be running the
same conversion the network runs, rather than a CLJS-shaped rehearsal of it. The same conversion the network runs, rather than a CLJS-shaped rehearsal of it. The
one thing a keywordising `js->clj` would quietly break is the leaf paths — one thing a keywordising `js->clj` would quietly break is the leaf paths —
`:clip/c1/timeline/main/node/mouth` is a keyword whose `name` is `:clip/c1/symbol/main/node/mouth` is a keyword whose `name` is
\"c1/timeline/main/node/mouth\", so the \"c1/symbol/main/node/mouth\", so the
\"clip/\" would be lost on the way back in. \"clip/\" would be lost on the way back in.
Refuses a document `domain/leaf` calls unaddressable, which is where a hand-made Refuses a document `domain/leaf` calls unaddressable, which is where a hand-made
@ -83,19 +108,27 @@
:state (when state (wire/base64 state))})) :state (when state (wire/base64 state))}))
(block-keys leaves)))}))) (block-keys leaves)))})))
(defn load (defn tier1
"The parsed response -> `{:clip :store}`, which is what `flow/freeze` returns "A response's leaves object -> `{path value}`, the shape `leaf/leaves` returns."
and therefore what the player already knows how to play." [^js leaves]
[cid ^js doc] (into {} (map (fn [path] [path (wire/decode-json (aget leaves path))]))
(let [leaves (.-leaves doc) (js-keys leaves)))
tier1 (into {} (map (fn [path] [path (wire/decode-json (aget leaves path))]))
(js-keys leaves))] (defn store
{:clip (leaf/clip cid tier1) "Fetched blocks -> the store `save` reads them back out of."
:store (into {} [blocks]
(into {}
(map (fn [^js b] (map (fn [^js b]
[(.-key b) [(.-key b)
(cond-> {:descriptor (.-descriptor b) (cond-> {:descriptor (.-descriptor b)
:data (wire/typed (block-type (.-descriptor b)) :data (wire/typed (block-type (.-descriptor b))
(.-data b))} (.-data b))}
(.-state b) (assoc :state (wire/bytes-of (.-state b))))])) (.-state b) (assoc :state (wire/bytes-of (.-state b))))]))
(array-seq (or (.-blocks doc) #js [])))})) (array-seq (or blocks #js []))))
(defn load
"The parsed response -> `{:clip :store}`, which is what `flow/freeze` returns
and therefore what the player already knows how to play."
[cid ^js doc]
{:clip (leaf/clip cid (tier1 (.-leaves doc)))
:store (store (.-blocks doc))})

View file

@ -24,19 +24,67 @@
(.fill buf index) (.fill buf index)
r) r)
;; ---------------------------------------------------------------------------
;; layers and knockouts
;;
;; A symbol holding a KNOCKOUT shape is drawn into a layer of its own first —
;; Flash's Erase blend inside a symbol set to Layer. A layer is a raster with a
;; `:cov` byte per pixel saying whether anything is there; a knockout clears
;; coverage, of every colour or of one, and the layer lands on the one under it
;; only where it is covered. The main raster has no `:cov` and is always
;; covered.
;;
;; THE INK is how a shape marks the pixels it covers, and there are three, as
;; Deluxe Paint and Animator Pro had inks:
;;
;; nil an ordinary fill: write the shape's index
;; a number a knockout: clear coverage — of every colour at -1, or of
;; that index only
;; a Uint8Array a REMAP, index -> index: what is already there is drawn
;; in another slot. A flashlight is a circle of this.
(defn- plot! [^js buf cov o index ink]
(cond
(nil? ink) (do (aset buf o index)
(when cov (aset cov o 1)))
(number? ink) (when (and cov (or (neg? ink) (== (aget buf o) ink)))
(aset cov o 0))
:else (aset buf o (aget ink (aget buf o)))))
(defn- shows?
"Does pixel `o` hold `over`? A stencil only matches what is really there, so
an uncovered pixel of a layer holds nothing."
[^js buf cov o over]
(or (nil? over)
(and (or (nil? cov) (== 1 (aget cov o))) (== (aget buf o) over))))
(defn layer [w h]
{:w w :h h :buf (js/Uint8Array. (* w h)) :cov (js/Uint8Array. (* w h))})
(defn composite!
"Land layer `l` on raster `r` wherever `l` is covered."
[{:keys [buf cov] :as r} l]
(let [^js lb (:buf l) ^js lc (:cov l)]
(dotimes [o (.-length lb)]
(when (== 1 (aget lc o))
(aset buf o (aget lb o))
(when cov (aset cov o 1)))))
r)
(defn fill-poly-buf! (defn fill-poly-buf!
"Even-odd scanline fill of a polygon held FLAT in `pts` as [x0 y0 x1 y1 …], "Even-odd scanline fill of a polygon held FLAT in `pts` as [x0 y0 x1 y1 …],
using the first `n` points. Samples at pixel centres (y + 0.5), so a polygon using the first `n` points. Samples at pixel centres (y + 0.5), so a polygon
edge landing exactly on a pixel boundary resolves consistently. edge landing exactly on a pixel boundary resolves consistently.
Flat and preallocated because this is the per-frame path: fixed topology means Flat and preallocated because this is the per-frame path: fixed topology means
a node's vertex count is known at freeze time, so timeline/resolver hands the same a node's vertex count is known at freeze time, so symbol/resolver hands the same
buffer back every frame and a frame allocates nothing. At 30fps per-frame buffer back every frame and a frame allocates nothing. At 30fps per-frame
allocation is the only thing that will make this stutter. allocation is the only thing that will make this stutter.
`pts` may be a CLJS vector or any typed array; scanline crossings are collected `pts` may be a CLJS vector or any typed array; scanline crossings are collected
into a plain JS array and sorted in place." into a plain JS array and sorted in place. `ink` is as `plot!`'s."
[{:keys [w h buf] :as r} pts n index] ([r pts n index] (fill-poly-buf! r pts n index nil))
([{:keys [w h buf cov] :as r} pts n index ink]
(when (>= n 3) (when (>= n 3)
(let [px (fn [i] (if (vector? pts) (-nth pts (* 2 i)) (aget pts (* 2 i)))) (let [px (fn [i] (if (vector? pts) (-nth pts (* 2 i)) (aget pts (* 2 i))))
py (fn [i] (if (vector? pts) (-nth pts (inc (* 2 i))) (aget pts (inc (* 2 i))))) py (fn [i] (if (vector? pts) (-nth pts (inc (* 2 i))) (aget pts (inc (* 2 i)))))
@ -78,8 +126,8 @@
x-to (min (dec w) (js/Math.floor (- xb 0.5))) x-to (min (dec w) (js/Math.floor (- xb 0.5)))
row (* y w)] row (* y w)]
(dotimes [dx (inc (- x-to x-from))] (dotimes [dx (inc (- x-to x-from))]
(aset buf (+ row x-from dx) index))))))))) (plot! buf cov (+ row x-from dx) index ink)))))))))
r) r))
(defn fill-poly! (defn fill-poly!
"`fill-poly-buf!` over a seq of {:x :y} points. "`fill-poly-buf!` over a seq of {:x :y} points.
@ -99,8 +147,9 @@
any radius, including mid-blink when the opening is a two-pixel sliver, so any radius, including mid-blink when the opening is a two-pixel sliver, so
the lid crops the iris for free instead of the gaze range needing a the lid crops the iris for free instead of the gaze range needing a
clamp that would flatten the performance at the extremes." clamp that would flatten the performance at the extremes."
([r cx cy rad index] (fill-disc! r cx cy rad index nil)) ([r cx cy rad index] (fill-disc! r cx cy rad index nil nil))
([{:keys [w h buf] :as r} cx cy rad index over] ([r cx cy rad index over] (fill-disc! r cx cy rad index over nil))
([{:keys [w h buf cov] :as r} cx cy rad index over ink]
(let [rr (* rad rad) (let [rr (* rad rad)
y0 (max 0 (js/Math.floor (- cy rad))) y0 (max 0 (js/Math.floor (- cy rad)))
y1 (min (dec h) (js/Math.ceil (+ cy rad))) y1 (min (dec h) (js/Math.ceil (+ cy rad)))
@ -114,8 +163,8 @@
dy (- (+ y 0.5) cy)] dy (- (+ y 0.5) cy)]
(when (<= (+ (* dx dx) (* dy dy)) rr) (when (<= (+ (* dx dx) (* dy dy)) rr)
(let [o (+ (* y w) x)] (let [o (+ (* y w) x)]
(when (or (nil? over) (= (aget buf o) over)) (when (shows? buf cov o over)
(aset buf o index))))))) (plot! buf cov o index ink)))))))
r))) r)))
(defn fill-rect! (defn fill-rect!
@ -132,8 +181,9 @@
on every frame. Round the extents instead and a fractional centre gives you on every frame. Round the extents instead and a fractional centre gives you
three pixels on one frame and four on the next, which reads as the pupil three pixels on one frame and four on the next, which reads as the pupil
breathing." breathing."
([r cx cy size index] (fill-rect! r cx cy size index nil)) ([r cx cy size index] (fill-rect! r cx cy size index nil nil))
([{:keys [w h buf] :as r} cx cy size index over] ([r cx cy size index over] (fill-rect! r cx cy size index over nil))
([{:keys [w h buf cov] :as r} cx cy size index over ink]
(let [size (js/Math.round size)] (let [size (js/Math.round size)]
(when (>= size 1) (when (>= size 1)
(let [x0 (js/Math.round (- cx (/ size 2))) (let [x0 (js/Math.round (- cx (/ size 2)))
@ -143,8 +193,8 @@
(dotimes [iy (- yb ya)] (dotimes [iy (- yb ya)]
(dotimes [ix (- xb xa)] (dotimes [ix (- xb xa)]
(let [o (+ (* (+ ya iy) w) xa ix)] (let [o (+ (* (+ ya iy) w) xa ix)]
(when (or (nil? over) (= (aget buf o) over)) (when (shows? buf cov o over)
(aset buf o index)))))))) (plot! buf cov o index ink))))))))
r)) r))
(def ^:private little-endian? (def ^:private little-endian?
@ -223,6 +273,27 @@
(aset d (+ o 3) 255)))))) (aset d (+ o 3) 255))))))
{:width W :height H :data d}))) {:width W :height H :data d})))
(defn- fill-mask!
"Every pixel set in `mask`, a byte per stage pixel: a brush stroke as it is
being painted, before it is a polygon."
[{:keys [buf cov] :as r} ^js mask index ink]
(dotimes [o (.-length mask)]
(when (== 1 (aget mask o))
(plot! buf cov o index ink)))
r)
;; One layer per depth of nesting, reused from frame to frame, as the resolver
;; reuses its point buffers.
(defonce ^:private layers (atom {}))
(defn- layer-at [depth w h]
(let [l (get @layers depth)]
(if (and l (= w (:w l)) (= h (:h l)))
l
(let [l (layer w h)]
(swap! layers assoc depth l)
l))))
(defn draw-ops! (defn draw-ops!
"Paint a list of draw ops, in the order given, into the raster. Stage 7. "Paint a list of draw ops, in the order given, into the raster. Stage 7.
@ -230,12 +301,26 @@
space points and a PALETTE INDEX, and the rasteriser knows nothing about nodes, space points and a PALETTE INDEX, and the rasteriser knows nothing about nodes,
channels, time maps or provenance. Everything above here can be rearranged channels, time maps or provenance. Everything above here can be rearranged
without touching a scanline, and a painted cel and a rotoscoped mouth arrive without touching a scanline, and a painted cel and a rotoscoped mouth arrive
here indistinguishable from each other, which is the point." here indistinguishable from each other, which is the point.
[r ops]
(doseq [{:keys [kind pts n color stencil cx cy size] :as op} ops] `:begin` and `:end` bracket the ops of a symbol drawn into a layer of its own;
`:knock` on an op makes it a knockout and `:lut` a remap. See `plot!`."
[{:keys [w h] :as r} ops]
(reduce
(fn [stack {:keys [kind pts n color stencil cx cy size] :as op}]
(let [top (peek stack)
ink (or (:knock op) (:lut op))]
(case kind (case kind
:poly (fill-poly-buf! r pts n color) :begin (let [l (layer-at (count stack) w h)]
:disc (fill-disc! r cx cy (:r op) color stencil) (.fill (:cov l) 0)
:rect (fill-rect! r cx cy size color stencil) (conj stack l))
(throw (ex-info "draw op kind is not rasterisable" {:op (dissoc op :pts)})))) :end (if (< 1 (count stack))
(let [s (pop stack)] (composite! (peek s) top) s)
stack)
:poly (do (fill-poly-buf! top pts n color ink) stack)
:disc (do (fill-disc! top cx cy (:r op) color stencil ink) stack)
:rect (do (fill-rect! top cx cy size color stencil ink) stack)
:mask (do (fill-mask! top (:mask op) color ink) stack)
(throw (ex-info "draw op kind is not rasterisable" {:op (dissoc op :pts)})))))
[r] ops)
r) r)

View file

@ -13,9 +13,15 @@
back to positions with indexOf would silently pick the wrong slot if a table back to positions with indexOf would silently pick the wrong slot if a table
ever repeated an id. ever repeated an id.
For even n this naturally lands on the cardinal positions (corners and lip Fixed indices, never adaptive decimation: the vertex at slot k means the same
centres) of a 20-point ring. Fixed indices, never adaptive decimation: the thing on every frame of the shot.
vertex at slot k means the same thing on every frame of the shot."
WHICH CARDINAL POSITIONS SURVIVE DEPENDS ON n, and not merely on n being even.
On a 20-point ring the corners at 0 and 10 are kept by every even n, but the lip
centres at 5 and 15 are kept only when n is a multiple of 4: n = 6, 10, 14 and
18 all drop them. So nothing may assume a named slot is still in the output —
the aperture pair least of all. A signal that needs those two landmarks reads
them from a measurement, not from a subsampled ring."
[len n] [len n]
(mapv (fn [k] (mod (js/Math.round (/ (* k len) n)) len)) (range n))) (mapv (fn [k] (mod (js/Math.round (/ (* k len) n)) len)) (range n)))

View file

@ -0,0 +1,191 @@
(ns arthur.domain.select
"Which frames out of a dense measurement are worth keeping.
A SELECTION IS A SET OF FRAMES CHOSEN OUT OF A DENSE MEASUREMENT, AND READ BY
HOLDING THE LATEST ONE AT OR BEFORE NOW. The holding half is `pose/held-frame`
and is already shared by tracing and by pose tracks; this is the choosing half,
which nothing did automatically before. One component, two sites: the plate
frames an artist has to draw a head on, and the frames a performance changes a
shape on.
`domain/cadence` answers the same question with no opinion about content, and
cannot know that the one frame where a mouth is fully shut is worth more than
its neighbours. That is the gap `:protect` closes.
THREE LAYERS, AND THE MIDDLE ONE IS DERIVED. A selection is a `:policy` (what
the proposer was asked for), a `:keep` and a `:drop` (what the hand insists on
and refuses). Only the hand's two sets are the document: the proposal is
recomputed whenever the policy or the signal changes, which is the whole reason
the hand's decisions are stored as their own sets rather than as the resulting
frame list. Re-suggesting at a new tolerance must never cost somebody their
pinned blink.
Pure. No store access and no clip access — each site reads its own dense data
and hands over a plain vector. Proposing is a command and not a subscription:
this allocates, which is fine for a button press over a few hundred frames and
would not be fine on the per-frame path.")
(defn- sample
"One sample as a flat vector of numbers, or nil where there is no measurement.
A plain number is a one-component sample, and nested points are flattened:
four transformed corners and the eight numbers in them are the same signal, so
neither site has to flatten on the way in. Nil is an ABSENT measurement — a
frame where the face was not found — which says nothing about the signal and is
not the same fact as a frame being skipped."
[s]
(cond
(nil? s) nil
(number? s) [s]
:else (not-empty (vec (flatten s)))))
(defn- distance
"How far apart two samples are: the largest absolute difference over their
components. A tolerance therefore means \"this far\" in the units the site
handed over, whether it handed over an aperture or a quad of corners, and
nobody has to say which.
Both samples are the same width, which `propose` has already insisted on rather
than comparing whatever prefix the two happen to share."
[a b]
(reduce (fn [m i] (max m (js/Math.abs (- (nth a i) (nth b i)))))
0 (range (count a))))
(defn- walk
"The greedy walk: keep frame 0 as the anchor, keep every frame in `forced`, and
keep every other frame that has moved further than `tolerance` from the anchor.
Each kept frame becomes the anchor.
A FORCED FRAME BECOMES THE ANCHOR LIKE ANY OTHER, which is why protection is
settled before the walk rather than unioned into its result. The anchor is what
is on screen; once a protected frame is kept the viewer is looking at it, so
measuring drift from a frame that is no longer displayed is simply wrong. Doing
it the other way holds a protected one-frame closure across two frames and
shows it twice as long as it was measured, which is the perceptual error the
protection was added to prevent.
Order-dependent and slightly suboptimal, and deliberately the first
implementation anyway — it is what the prototype shipped, and the dynamic
program that replaces it gets tested by being compared against it. Keeping the
anchor honest here leaves greed as the only difference between the two.
An absent sample is not a change: it cannot move the anchor and it is not worth
a frame. A measurement arriving where the anchor has none is, since a signal
coming back is a frame the picture has to show something on. A forced frame
with no measurement is still kept — it was asked for — and leaves the anchor
where it was, there being nothing there to measure from."
[n sig tolerance forced]
(loop [f 1 anchor (nth sig 0) kept (transient [0])]
(if (>= f n)
(persistent! kept)
(let [s (nth sig f)]
(if (or (contains? forced f)
(and s (or (nil? anchor) (> (distance anchor s) tolerance))))
(recur (inc f) (or s anchor) (conj! kept f))
(recur (inc f) anchor kept))))))
(defn- segments
"Maximal runs of consecutive measured frames. A gap is not something to compare
across: the samples either side of an absent measurement are not neighbours."
[n sig]
(->> (range n)
(partition-by #(some? (nth sig %)))
(remove #(nil? (nth sig (first %))))))
(defn- turns
"Frames in one measured segment that are a strict local extremum by more than
`tolerance` — a full mouth closure, a full blink. These are exactly the frames
a zero-order hold is most wrong about, and exactly what a cadence drops.
A run of equal samples is ONE extremum, reported at the frame it begins: the
hold reads from there, so naming any later frame of the run would show the
closure late. The ends of a segment are not extrema; they have only one side."
[sig tolerance frames]
(let [runs (vec (partition-by #(nth sig %) frames))
at #(first (nth sig %))]
(for [i (range 1 (dec (count runs)))
:let [f (first (nth runs i))
v (at f)
p (at (first (nth runs (dec i))))
q (at (first (nth runs (inc i))))]
:when (and (or (and (< v p) (< v q)) (and (> v p) (> v q)))
(> (min (js/Math.abs (- v p)) (js/Math.abs (- v q)))
tolerance))]
f)))
(defn propose
"Frames worth keeping out of `n`, given `signal`.
`signal` is a vector of n samples, each a number or a seq of numbers in a fixed
order. FRAME REMOVAL, NOT KEY EXTRACTION: every frame is a candidate and the
question is which can be dropped, which is why this walks and holds rather than
looking for peaks.
`:tolerance` is how far the signal may move before a frame has to be kept, in
whatever units the signal is in. `:protect` is the ONE input for frames that
have to be kept whatever the walk thought, and it carries two kinds of thing at
once so that it never has to become two parameters:
:extrema keep the signal's own strict local extrema as well
[12 30] keep these frames, whatever the walk thought
#{:extrema 12} both
The frames are how one selection constrains another — every kept plate frame is
a protected frame of the performance selection, so the drawing and the
performance can never cut against each other on neighbouring frames — and it is
the same mechanism that keeps a blink, which is why there is only one input.
`:extrema` is refused on a multi-component signal. The magnitude of a vector of
landmarks has local maxima that mean nothing; an aperture's closure is a real
extremum and a head has no such thing as an extreme position worth protecting.
Protection is settled BEFORE the walk, not unioned into its result, because a
protected frame anchors the walk like any other kept frame — see `walk`. The
result is therefore not `(walk) ∪ (protected)`: protected frames change what is
kept after them, which is the whole reason they are not applied afterwards.
Always contains frame 0 when there is a frame at all. The result is an ascending
vector of distinct frames, which is the shape both sites already store."
[n signal {:keys [tolerance protect]}]
(if-not (pos? n)
[]
(let [;; A slider that has been cleared reads as NaN, and every comparison
;; against NaN is false, which would propose frame 0 alone and quietly
;; collapse a whole performance to one pose. Nought keeps every frame
;; that changes, which is merely the cadence back again: wrong in a way
;; somebody can see and undo.
tol (if (js/Number.isFinite tolerance) tolerance 0)
sig (mapv #(sample (nth signal % nil)) (range n))
;; One width for the whole signal, so `distance` is never comparing the
;; prefix two samples happen to share. A reader that drops a component
;; on one frame is a bug with a loud version and a silent version, and
;; the silent version is a signal that is quietly the wrong shape.
seen (into #{} (map count) (remove nil? sig))
width (first seen)
p (cond (nil? protect) #{}
(keyword? protect) #{protect}
:else (set protect))
pins (into #{} (filter #(and (integer? %) (<= 0 %) (< % n))) p)]
(when (< 1 (count seen))
(throw (ex-info "signal samples are not all the same width"
{:widths (vec (sort seen))})))
(when (and (contains? p :extrema) width (< 1 width))
(throw (ex-info ":protect :extrema needs a single-component signal"
{:components width})))
(walk n sig tol (cond-> pins
(contains? p :extrema)
(into (mapcat #(turns sig tol %) (segments n sig))))))))
(defn effective
"`proposed` with the hand's decisions applied. Always contains 0: something has
to be on screen at the start.
Manual precedence is absolute — a drop beats a proposal, always — and it is the
proposal it beats, not the start of the take. Hand decisions apply whether or
not the proposer is switched on, which is why they are not a mode: turning
smart picking off must not throw away the frames somebody pinned."
[proposed keep drop]
(-> (reduce disj (into (set proposed) keep) drop)
(conj 0)
sort
vec))

View file

@ -0,0 +1,803 @@
(ns arthur.domain.span
"The commands over a node's place in time: split it, trim an edge, move it —
and, where the symbol holding it is drawn as a lane, re-span its siblings to
make room.
A `:span` is in the node's OWN frames and its `:time` says where those land in
its parent, and that is true of EVERY node. A clip in a sequence, a symbol
placed straight into a shot, a shape that exists for part of one: each is a
span in a parent's frame space, and a span in a parent's frame space is the
whole of what these commands touch.
THE COORDINATE IS ALWAYS THE PARENT'S. `host-frame` reads the frame space the
node is positioned in, whatever that is, so a caller holding a node does not
branch on what it sits in.
A GROUP IS REFUSED where one node is being divided. Dividing a group means
deciding what becomes of its children, and nothing in a span says: a span that
narrows past a child hides it without saying so.
WHY THE SEQUENCE COMMANDS ARE HERE TOO. They used to be `domain/lane`, gated on
a group with `:layout :sequence`, because a lane was the only thing anybody had
timed. There is no such group any more — the SYMBOL is the container and a lane
is how the timeline draws one, `symbol/lane?` — so their subject is a symbol and
its children, which is the same subject as everything else here: one write to
one node's span, with the siblings re-spanned around it. See
`docs/lane-is-a-view-plan.md`.
THE CLAIM-TIME RULE FOLLOWS THE MODE. In lane mode, placing, moving or growing
over occupied frames TRIMS what it lands on — trimming the incumbent, removing
one wholly covered, splitting one it lands inside — so the result has no overlap
because the operation that could have made one did not. Outside lane mode
nothing is enforced, because overlapping children are what compositing IS.
EVERY COMMAND IS ONE STEP AND ALL OF IT. Each returns `{:clip :selection}` or
`{:refused reason}` — never a half-applied edit, and never a document that
`clip/problems` would reject. A command that cannot say what the person meant
refuses and says why, rather than picking for them: the overflow policy is a
caller's `:extent`, and decoupling shared content is its own command instead of
something an ordinary edit does silently.
IDS COME FROM THE CALLER, because a clip's identity is a uuid and this namespace
is pure. Ids for new CONTENT are derived from the drawing being copied —
`clip/free-id` is pure too, and `drawing-a-2` says what it came from in a way
`symbol-7` does not.
`finish` lives here because every command in this namespace commits through it."
(:require [arthur.domain.bring :as bring]
[arthur.domain.clip :as clip]
[arthur.domain.node :as node]
[arthur.domain.symbol :as symbol]))
(defn- fit-lanes
"Make direct lane children cover `sid`'s authored window.
A lane has no independently authored extent: it is a view of its parent
symbol's timeline. Keep that invariant at the one commit point that can grow
a symbol, so neither the wrapper nor the lane symbol retains an old parent
length."
[clip sid]
(let [frames (clip/frames clip sid)]
(reduce
(fn [c [id n]]
(let [source (node/source n)]
(if (and (nil? (:parent n))
(symbol/lane? (clip/symbol c source)))
(-> c
(assoc-in [:symbols sid :nodes id :span] [0 frames])
(assoc-in [:symbols sid :nodes id :time :at] 0)
(assoc-in [:symbols source :frames] frames))
c)))
clip
(get-in clip [:symbols sid :nodes]))))
(defn finish
"Commit `nodes` as symbol `sid`'s, or refuse.
THE SHOT LENGTH IS AUTHORED. `:frames` is the symbol's window — how long the
shot IS — and where its clips reach is a different fact derived from them. A
command may GROW the window when the caller says `:grow-symbol`, and never
shrinks it: emptying the end of a shot leaves a shot with empty frames at the
end, which is a true statement about what somebody authored. Deriving the window
from the reach instead would make deleting the last drawing silently shorten the
film.
So there are two numbers and this function keeps them apart: `needed` is where
the clips reach, `:frames` is what was authored, and the only way the second
follows the first is a caller asking.
Only a symbol drawn AS A LANE is measured for reach. A node placed into a
composition may hang off the end of it — that is an ordinary thing to author and
the window is what crops it — whereas a lane's clips are a sequence whose length
is the thing being edited.
THE OVERLAP INVARIANT IS ENFORCED HERE, and here is the only place it needs to
be: this is the single commit path for every sequence command, it validates
before it returns, and it refuses rather than half-applying. So no command can
commit an overlap in lane mode, and `symbol/overlaps` turning up anything is a
bug in a command rather than a state to design around."
[clip sid nodes selection extent]
(let [sym (clip/symbol clip sid)
lane? (symbol/lane? sym)
after (assoc sym :nodes nodes)
needed (if-not lane?
0
(js/Math.ceil (apply max 0 (keep #(second (node/placed-span %))
(symbol/children nodes)))))
ps (symbol/problems after)
clashing (when lane? (symbol/overlaps after))]
(cond
(seq ps) {:refused (first ps)}
(seq clashing)
{:refused (str "that would put " (pr-str (ffirst clashing)) " and "
(pr-str (second (first clashing)))
" on screen over the same frames of a lane")}
(not (#{:keep :grow-symbol} extent)) {:refused "choose an explicit shot-length policy"}
(and (> needed (:frames sym)) (= :keep extent))
{:refused (str "the edit needs " needed " frames; extend the shot to continue")
:required-frames needed}
:else {:clip (fit-lanes
(cond-> (assoc-in clip [:symbols sid :nodes] nodes)
(> needed (:frames sym))
(assoc-in [:symbols sid :frames] needed))
sid)
:selection selection})))
;; ---------------------------------------------------------------------------
;; the geometry every edge edit is made of
;;
;; A `:span` is in the node's OWN frames and its `:time` says where those
;; land in the parent. So moving an edge is one write to `:span`, and `:time`
;; and `:playback` are untouched — which is why trimming the front of a playing
;; insert starts it later in its source instead of resetting it, and why the two
;; halves of a split go on meaning what the one node meant.
;; Trim, split and `blank` are all this one operation, applied differently.
(defn local
"Parent frame `f` as one of `n`'s own frames."
[n f]
(let [{:keys [at rate]} (node/time-of n)]
(* rate (- f at))))
(defn edged
"`n` with its `:in` or `:out` edge at parent frame `f`."
[n which f]
(assoc-in n [:span (case which :in 0 :out 1)] (local n f)))
(defn claim
"Commit proposed `nodes`, with `id` claiming its interval in lane mode.
This is the one difference between editing a lane and a composition. In a
composition it is exactly `finish`. In a lane, immediately before that same
commit, clips covered by the edited one are removed and clips crossing either
edge are trimmed. A clip crossing both edges is split and therefore needs a
caller-supplied `remainder-id`."
[clip sid nodes id extent remainder-id]
(let [n (get nodes id)
lane-child? (and (symbol/lane? (clip/symbol clip sid))
n (nil? (:parent n)) (node/placed-span n))]
(if-not lane-child?
(finish clip sid nodes id extent)
(let [[a b] (node/placed-span n)
others (remove #(= id (:id %)) (symbol/children nodes))
spanning (first (filter #(let [[lo hi] (node/placed-span %)]
(and (< lo a) (> hi b)))
others))]
(if (and spanning (or (nil? remainder-id) (contains? nodes remainder-id)))
{:refused "claiming time inside one clip needs a free ID for its remainder"}
(finish
clip sid
(reduce
(fn [ns other]
(let [oid (:id other)
[lo hi] (node/placed-span other)]
(cond
(or (<= hi a) (>= lo b)) ns
(and (< lo a) (> hi b))
(-> ns
(assoc oid (edged other :out a))
(assoc remainder-id
(assoc (edged other :in b) :id remainder-id
:z (str "a-" remainder-id))))
(and (>= lo a) (<= hi b)) (dissoc ns oid)
(< lo a) (assoc ns oid (edged other :out a))
:else (assoc ns oid (edged other :in b)))))
nodes others)
id extent))))))
(defn host-frame
"Symbol frame `f` as a frame of the space node `id` is POSITIONED in — its
parent's — which is the frame space every command here takes its coordinate
in. For a clip of a lane that is the symbol's own frames, because the symbol is
the container and the clip has no parent. Nil through a stepped or looping
ancestor, where one frame of the symbol is not one frame of the parent and there
is no single answer to give."
[clip sid id f]
(let [nodes (get-in clip [:symbols sid :nodes])]
(when-let [{:keys [at rate]} (symbol/frame-map nodes (:parent (get nodes id)))]
(* rate (- f at)))))
(defn- subject
"The node `id` names, as `{:node n}`, or `{:refused why}` where these commands
have nothing to act on. The one guard they share."
[nodes id]
(let [n (get nodes id)]
(cond
(nil? n) {:refused "select something with a place in time"}
(= :group (:kind n))
{:refused "a group is divided by its children, not by its span"}
(nil? (node/placed-span n))
{:refused "this is on screen for the whole shot, so it has no edges to cut"}
:else {:node n})))
(defn- siblings
"The other clips of `sid`'s sequence, or nil where `sid` is not drawn as a lane
and so has no sequence to re-span."
[clip sid nodes id]
(when (symbol/lane? (clip/symbol clip sid))
(remove #(= id (:id %)) (symbol/children nodes))))
(defn split
"Cut node `id` in two at parent frame `cut`. The left piece keeps its
identity; the right gets `new-id`.
NOTHING BUT `:span` DIFFERS between the two pieces. They keep one `:time`, so
the right piece's own frames carry on exactly where the left's stopped, and its
source clock, its keys and its corrections therefore go on meaning what they
meant before the cut — preserved by construction rather than by arithmetic on
in-points that could be wrong. A held drawing holds the same frame on both
sides; a playing insert plays on through the cut without a seam; a shape goes
on being the same shape over each half. That is what `:span` being in the
node's OWN coordinates buys, and it is why splitting needs no shot-length
policy: the pieces occupy the frames the one node occupied.
THE RIGHT PIECE KEEPS THE ORIGINAL'S `:z`. Two halves of one thing draw at
one depth; nothing orders them against each other, because they are never on
screen on the same frame. A lane's clips do not consult `:z` at all —
`symbol/children` sorts them by where they start.
The right piece is the selection, because it is the piece that was made."
[clip sid id cut new-id]
(let [nodes (get-in clip [:symbols sid :nodes])
{:keys [node refused]} (subject nodes id)
[lo hi] (when node (node/placed-span node))]
(cond
refused {:refused refused}
(not (integer? cut)) {:refused "a cut is a whole frame"}
(contains? nodes new-id) {:refused "the new ID is already used"}
(not (< lo cut hi)) {:refused (str "frame " cut " is not inside this")}
:else
(let [nodes (-> nodes
(assoc id (edged node :out cut))
(assoc new-id (assoc (edged node :in cut) :id new-id)))]
(finish clip sid nodes new-id :keep)))))
(defn trim
"Move one edge of node `id` to parent frame `to`, without disturbing anything
else at all.
TRIM NARROWS. Lengthening is `resize-out`, which carries a ripple policy and a
shot-length policy because it needs them; letting trim grow as well would give
one gesture two sets of rules and a way to overlap its neighbour. `edge` is
`:in` or `:out`.
The source clock is untouched, so trimming the front of a playing insert
starts it later INTO its animation rather than restarting it — which is the
difference between trimming and slipping, and why they are separate commands."
[clip sid id edge to]
(let [nodes (get-in clip [:symbols sid :nodes])
{:keys [node refused]} (subject nodes id)
[lo hi] (when node (node/placed-span node))]
(cond
refused {:refused refused}
(not (#{:in :out} edge)) {:refused "an edge is :in or :out"}
(not (integer? to)) {:refused "an edge goes to a whole frame"}
(not (< lo to hi))
{:refused (str "frame " to " is not inside this; trim narrows it")}
:else (finish clip sid (assoc nodes id (edged node edge to)) id :keep))))
(defn move
"Put node `id` at parent frame `to`, leaving its own length, source and
corrections alone — and, in a lane, every other clip.
One write to `:time :at`. A destination that would overlap a neighbour IN A
LANE is refused rather than rippled or overwritten: moving a drawing and
re-timing the ones around it are different intentions, and a move that
silently pushed the rest would be the second one wearing the first one's name.
Clear the room first — `blank` makes a gap, `trim` shortens a neighbour — or
say you meant to claim it, which is `adopt`, what a body drag does.
Outside lane mode there is no such rule to break: things placed in a
composition are allowed to be on screen together, so the move simply happens."
[clip sid id to]
(let [nodes (get-in clip [:symbols sid :nodes])
{:keys [node refused]} (subject nodes id)]
(cond
refused {:refused refused}
(not (integer? to)) {:refused "a move goes to a whole frame"}
:else
(let [moved (update-in node [:time :at] (fnil + 0) (- to (first (node/placed-span node))))]
(if (not= to (first (node/placed-span moved)))
{:refused "timing through a stepped or looping parent is not supported"}
(finish clip sid (assoc nodes id moved) id :keep))))))
;; ---------------------------------------------------------------------------
;; the sequence: one edge edit, with the siblings re-spanned around it
(defn- trimmed-into-a-sequence
"`nodes` with every clip's right edge pulled back to where the next one starts,
and any clip the next one wholly covers removed.
THE SAME CLAIM-TIME RULE, APPLIED ALL AT ONCE. Later claims from earlier
everywhere it has to, which is the rule every other command here follows one
edit at a time; doing it as a pass is only what makes the answer to \"make this
a lane\" one undo step."
[nodes]
(reduce
(fn [ns [earlier later]]
(let [[lo hi] (node/placed-span (get ns (:id earlier)))
[next-lo _] (node/placed-span later)]
(cond
(nil? lo) ns
(<= hi next-lo) ns
(<= next-lo lo) (dissoc ns (:id earlier))
:else (assoc ns (:id earlier) (edged (get ns (:id earlier)) :out next-lo)))))
nodes
(partition 2 1 (symbol/children nodes))))
(defn draw-as-lane
"Turn symbol `sid`'s lane mode on or off. `{:clip c}` or `{:refused why}`.
OFF IS ALWAYS POSSIBLE: a composition has no invariant to break, so dropping
the hint drops the rules with it and nothing in the document moves.
ON IS THE ONE PLACE A PERSON CAN ASK FOR THE IMPOSSIBLE. Every other command
maintains the sequence; this one asks a symbol whose clips may already be on
screen together to start being one, and there is no answer that does not throw
frames away. So it refuses and says how many clips it would have to trim,
carrying `:required-trim` for the retry the UI offers as one button — the same
shape as `finish`'s `:required-frames`. With `trim?` it does it: later claims
from earlier, which is the rule everything else here already follows."
[clip sid on? {:keys [trim?]}]
(let [sym (clip/symbol clip sid)]
(cond
(nil? sym) {:refused "there is no such symbol"}
(not on?) {:clip (update-in clip [:symbols sid] dissoc :display)}
:else
(let [clashing (symbol/overlaps sym)]
(cond
(empty? clashing) {:clip (assoc-in clip [:symbols sid :display] :lane)}
(not trim?)
{:refused (str "drawing this as a lane means trimming "
(count clashing) " clip"
(when (< 1 (count clashing)) "s")
" that overlap a neighbour")
:required-trim (count clashing)}
:else
(let [nodes (trimmed-into-a-sequence (:nodes sym))
after (assoc sym :nodes nodes :display :lane)]
(if (seq (symbol/overlaps after))
{:refused "these clips cannot be trimmed into a sequence"}
{:clip (assoc-in clip [:symbols sid] after)})))))))
(defn resize-out
"Put clip `id`'s right edge at parent frame `to`, allowing it to grow.
IN A LANE this claims time. Without `ripple?`, growing consumes the starts of
the clips it reaches: wholly covered clips disappear and the last partially
covered one is trimmed. Shrinking leaves a gap. With `ripple?`, every clip
beginning at or after the old edge moves by the same delta, in either
direction, so their contents are preserved. Either way there is no overlap,
because the operation that could have made one did not.
OUTSIDE A LANE it is one write to one span and nothing else moves, because
things placed in a composition are allowed to be on screen together. One
gesture, and the mode says which rule it plays by."
[clip sid id to {:keys [extent ripple?] :or {extent :keep ripple? false}}]
(let [nodes (get-in clip [:symbols sid :nodes])
{:keys [node refused]} (subject nodes id)
[lo old-out] (when node (node/placed-span node))
later (when (and node ripple?)
(filter #(>= (first (node/placed-span %)) old-out)
(siblings clip sid nodes id)))]
(cond
refused {:refused refused}
(not (integer? to)) {:refused "an edge goes to a whole frame"}
(not (< lo to)) {:refused "a clip must keep at least one frame"}
(= to old-out) {:clip clip :selection id}
:else
(let [delta (- to old-out)
resized (assoc nodes id (edged node :out to))
changed (reduce (fn [ns sibling]
(update-in ns [(:id sibling) :time :at] (fnil + 0) delta))
resized later)]
(claim clip sid changed id extent nil)))))
(defn resize-in
"Put clip `id`'s left edge at parent frame `to`. Shrinking leaves a gap;
in a lane, growing left consumes earlier clips symmetrically with `resize-out`."
[clip sid id to]
(let [nodes (get-in clip [:symbols sid :nodes])
{:keys [node refused]} (subject nodes id)
[old-in hi] (when node (node/placed-span node))]
(cond
refused {:refused refused}
(not (integer? to)) {:refused "an edge goes to a whole frame"}
(not (< to hi)) {:refused "a clip must keep at least one frame"}
(neg? to) {:refused "a clip cannot begin before the shot"}
(= to old-in) {:clip clip :selection id}
:else (claim clip sid (assoc nodes id (edged node :in to)) id :keep nil))))
(defn roll
"Move the shared boundary between adjacent clips `left-id` and `right-id`.
This is deliberately only the composition of the two ordinary edge edits."
[clip sid left-id right-id to]
(let [nodes (get-in clip [:symbols sid :nodes])
left (get nodes left-id)
right (get nodes right-id)
[llo lhi] (when left (node/placed-span left))
[rlo rhi] (when right (node/placed-span right))]
(cond
(not (symbol/lane? (clip/symbol clip sid)))
{:refused "a rolling edit needs a symbol drawn as a lane"}
(not (and llo rlo (nil? (:parent left)) (nil? (:parent right))))
{:refused "a rolling edit needs two clips of one sequence"}
(not= lhi rlo) {:refused "a rolling edit needs one shared boundary"}
(not (integer? to)) {:refused "a clip edge goes to a whole frame"}
(not (< llo to rhi)) {:refused "both clips must keep at least one frame"}
:else
(let [left-result (resize-out clip sid left-id to {})]
(if (:refused left-result)
left-result
(resize-in (:clip left-result) sid right-id to))))))
(defn blank
"Clear frames `[a b)` of symbol `sid`, leaving a GAP.
A gap is not a drawing. Nothing is invented to cover those frames and nothing
closes the hole — the clips after it stay where they are, because emptying
frames and re-timing a performance are different intentions.
What it does to each clip it meets is `edged`, applied three ways: one wholly
inside is removed, one overlapping an end is trimmed to it, and the one that
spans the whole range is split, which is the only case that needs `id`. Their
drawings stay in the library — a symbol does not own its content, and a drawing
whose last clip is gone is still a drawing somebody made."
[clip sid [a b] {:keys [id]}]
(let [nodes (get-in clip [:symbols sid :nodes])
lane? (symbol/lane? (clip/symbol clip sid))
members (when lane? (symbol/children nodes))
spanning (when members
(first (filter #(let [[lo hi] (node/placed-span %)] (and (< lo a) (> hi b)))
members)))]
(cond
(not lane?) {:refused "clearing a range of frames needs a symbol drawn as a lane"}
(not (and (integer? a) (integer? b) (< a b)))
{:refused "a range to blank is whole frames, and not empty"}
(and spanning (or (nil? id) (contains? nodes id)))
{:refused "blanking inside one clip splits it, which needs a free ID for the remainder"}
:else
(let [nodes (reduce
(fn [ns n]
(let [[lo hi] (node/placed-span n)]
(cond
(or (<= hi a) (>= lo b)) ns
(and (< lo a) (> hi b))
(-> ns
(assoc (:id n) (edged n :out a))
(assoc id (assoc (edged n :in b) :id id :z (str "a-" id))))
(and (>= lo a) (<= hi b)) (dissoc ns (:id n))
(< lo a) (assoc ns (:id n) (edged n :out a))
:else (assoc ns (:id n) (edged n :in b)))))
nodes members)]
;; NOTHING SENSIBLE IS SELECTED by emptying frames, so nothing is: a
;; split names its remainder, and otherwise the caller keeps whatever was
;; selected rather than being handed a clip it did not ask for.
(finish clip sid nodes (when spanning id) :keep)))))
(defn extend-hold
"Change one held clip's duration by `delta` frames and ripple its later
siblings. Keys, source clocks and the clips' own channels stay put.
Returns {:clip :selection} or {:refused :required-frames?}; never partially edits."
[clip sid id delta {:keys [extent] :or {extent :keep}}]
(let [nodes (get-in clip [:symbols sid :nodes])
n (get nodes id)
rate (:rate (node/time-of n))
span (:span n)]
(cond
(not (symbol/lane? (clip/symbol clip sid)))
{:refused "a hold is lengthened in a symbol drawn as a lane"}
(nil? (node/placed-span n)) {:refused "select a clip in the lane"}
(some? (:parent n)) {:refused "select a clip of the lane itself"}
(not (and (integer? delta) (not (zero? delta)))) {:refused "hold change must be a nonzero whole number of frames"}
(not (zero? (:speed (node/playback-of n)))) {:refused "hold length applies to a held drawing"}
(<= (+ (second span) (* rate delta)) (first span)) {:refused "a drawing must keep a positive exposure"}
:else
(let [[_ boundary] (node/placed-span n)
later (filter #(>= (first (node/placed-span %)) boundary)
(siblings clip sid nodes id))
nodes (assoc-in nodes [id :span 1] (+ (second span) (* rate delta)))
nodes (reduce (fn [ns sibling]
(update-in ns [(:id sibling) :time :at] (fnil + 0) delta))
nodes later)]
(finish clip sid nodes id extent)))))
(defn placement-refusal
"Why direct child `n` may not be placed in symbol `sid`, or nil.
This is the type-containment boundary for every placement path. Specialized
symbols are confined here rather than by drag affordances, so pool drops,
transfers, structural nesting and future commands cannot disagree."
[clip sid n]
(let [destination (clip/symbol clip sid)
source (when (= :instance (:kind n)) (clip/symbol clip (node/source n)))
destination-type (:type destination)
source-type (:type source)]
(cond
(= :palette destination-type) "a palette symbol cannot contain nodes"
(and (= :palette-track destination-type) (not= :palette source-type))
"a palette track accepts only palette symbols"
(and (= :palette source-type) (not= :palette-track destination-type))
"a palette symbol can be placed only in a palette track"
:else nil)))
(defn place-node
"Place the already-materialized, direct child `n` into symbol `sid` at `at`.
THIS IS THE PLACEMENT RULE. Materializing a symbol instance, an audio node, or
imported content is deliberately somebody else's job; once it is a node, its
origin no longer matters. The destination alone decides the edit:
- an ordinary symbol attaches it and permits overlap;
- a lane symbol first clears the interval it claims.
The caller hands this function a node not currently present in the destination.
Its duration, channels, source clock and identity are preserved; only its
parent-space start changes. Returns the ordinary command result shape."
[clip sid n at {:keys [extent remainder-id] :or {extent :keep}}]
(let [sym (clip/symbol clip sid)
nodes (:nodes sym)
id (:id n)
[lo hi] (when n (node/placed-span n))
duration (when (and lo hi) (- hi lo))
incompatible (when (and sym n) (placement-refusal clip sid n))]
(cond
(nil? sym) {:refused "there is no destination symbol"}
(nil? n) {:refused "there is no clip to place"}
incompatible {:refused incompatible}
(contains? nodes id) {:refused "the destination already uses that clip ID"}
(some? (:parent n)) {:refused "only a direct child can be placed in a symbol"}
(not (and (integer? at) (not (neg? at))))
{:refused "a position is a nonnegative whole frame"}
(not (pos? duration)) {:refused "the clip has no frames to place"}
(and remainder-id (contains? nodes remainder-id))
{:refused "the remainder clip needs a free ID"}
:else
(let [placed (update-in n [:time :at] (fnil + 0) (- at lo))]
(claim clip sid (assoc nodes id placed) id extent remainder-id)))))
(defn place-symbol
"Materialize an instance of `source-id`, then place it into `sid` at `at`.
IN A LANE the new clip claims its interval: existing clips under it are
trimmed, removed, or split, so the sequence stays a partition rather than
storing an overlap. Outside one it is simply placed. This is the generic
operation behind dropping a library symbol into the timeline; one-frame held
drawing creation remains a policy of `append-drawing`/`overwrite-drawing`,
not a different kind of container."
[clip store sid id source-id at
{:keys [extent point remainder-id] :or {extent :keep}}]
(let [nodes (get-in clip [:symbols sid :nodes])
seeded (clip/place-symbol clip store sid source-id 0 id point)
n (get-in seeded [:symbols sid :nodes id])]
(cond
(contains? nodes id) {:refused "the new clip ID is already used"}
(nil? (clip/symbol clip source-id)) {:refused "there is no such symbol to place"}
(nil? n) {:refused "a symbol cannot go inside itself"}
:else
(place-node clip sid n at {:extent extent :remainder-id remainder-id}))))
(defn adopt
"Move an existing clip of `sid` to frame `at`, CLAIMING the time it lands on.
Its source, span, transforms, corrections, and identity come with it; in a lane
the destination interval is claimed with the same trimming as a pool drop. This
is what a body drag does, and it is why `move` and this are two commands:
`move` refuses to disturb a neighbour, and a drag onto occupied time has
already said it means to."
[clip sid id at opts]
(let [nodes (get-in clip [:symbols sid :nodes])
n (get nodes id)]
(cond
(nil? n) {:refused "select a clip to move"}
(some? (:parent n)) {:refused "only a clip of the symbol itself is placed in its sequence"}
:else
(place-node (assoc-in clip [:symbols sid :nodes] (dissoc nodes id))
sid n at opts))))
(defn transfer
"Move direct child `id` from `from-sid` into `to-sid` at `at`.
Cross-symbol and same-symbol moves are deliberately the same composition:
detach, then `place-node`. The destination's mode—not the gesture, source, or
payload kind—decides whether occupied time is claimed."
[clip from-sid id to-sid at opts]
(let [from-nodes (get-in clip [:symbols from-sid :nodes])
n (get from-nodes id)]
(cond
(nil? n) {:refused "select a clip to move"}
(some? (:parent n)) {:refused "only a direct child can move between symbols"}
:else
(let [detached (assoc-in clip [:symbols from-sid :nodes] (dissoc from-nodes id))
result (place-node detached to-sid n at opts)]
(if-let [made (:clip result)]
(if-let [why (first (clip/problems made))]
{:refused why}
result)
result)))))
;; ---------------------------------------------------------------------------
;; putting drawings in a sequence
(defn- held
"A one-frame held clip of `drawing-id`, starting at frame `at`.
Held rather than playing, and one frame rather than the length of what it
places: a clip's duration is the sequence's business — `extend-hold` and
`resize-out` are how it changes — and reading it off the content would make
placing a ten-frame animation and holding its first drawing the same gesture.
NO PIVOT, unlike `clip/place-symbol`, and the difference is whether the content
EXISTS YET. Dropping a symbol onto the stage places a drawing somebody can see,
so the middle of it is known and is stored as the instance's pivot; a cel is
made to be drawn in, and the middle of an empty drawing is nothing to commit to.
So a cel's pivot stays unchosen until a turn or a scale chooses it, which is
`gesture/with-pivot`, from the bounds the drawing has by then."
[id drawing-id at]
{:id id :kind :instance :z (str "a-" id)
:span [0 1] :time {:at at :rate 1}
:source {:symbol drawing-id} :playback {:in 0 :speed 0 :end :stop}})
(defn- sequence-end
"Where `sid`'s occupied frames stop."
[nodes]
(apply max 0 (map #(second (node/placed-span %)) (symbol/children nodes))))
(defn- place
"Put a held clip of `drawing-id` into `sid` at frame `at`, and RIPPLE:
everything starting at or after it moves later by its duration.
There is one placement function and `:end` is a position like any other, so
appending is not a different operation from inserting — the end is just where
nothing has to move. Overwriting is the other policy and is NOT this: taking
frames away from the clip already there is trimming, which is its own command
and not something placing a drawing should do on the quiet.
`:frame` in the result is where it landed, in the symbol's own frames, for a
caller that wants to look at what it just made."
[clip sid id drawing-id at extent ripple?]
(let [nodes (get-in clip [:symbols sid :nodes])
at (if (= :end at) (sequence-end nodes) at)
n (held id drawing-id at)
[lo hi] (node/placed-span n)
;; RIPPLE FOLLOWS THE MODE. Pushing later siblings is what inserting
;; into a sequence means; in a composition there is no "later sibling"
;; to push, because being on screen together is the point.
later (when (and ripple? (symbol/lane? (clip/symbol clip sid)))
(filter #(>= (first (node/placed-span %)) lo) (symbol/children nodes)))
nodes (reduce (fn [ns sibling]
(update-in ns [(:id sibling) :time :at] (fnil + 0) (- hi lo)))
(assoc nodes id n) later)
result (finish clip sid nodes id extent)]
(cond-> result
(:clip result) (assoc :frame at))))
(defn- placeable
"Why a held clip cannot go into `sid` at `at`, or nil."
[clip sid id at]
(let [nodes (get-in clip [:symbols sid :nodes])
;; INSIDE a clip is not a position for another one, IN A LANE. Splitting
;; that clip is what makes it two, and doing it here would be one command
;; quietly performing two: the caller asks for `split` and then places.
;; In a composition landing inside something is not a collision at all.
inside (when (and (number? at) (symbol/lane? (clip/symbol clip sid)))
(some (fn [n] (let [[lo hi] (node/placed-span n)]
(when (< lo at hi) n)))
(symbol/children nodes)))]
(cond
(contains? nodes id) "the new clip ID is already used"
(not (or (= :end at) (and (integer? at) (not (neg? at)))))
"a position is :end or a whole frame"
inside (str "frame " at " is inside a clip; split it first"))))
(defn append-drawing
"Append fresh empty content and a held clip of it. IDs come from the
caller so a command is deterministic and replayable.
Fresh content, not a blank range: a sequence with no clip over a frame shows
nothing there already, and a drawing nobody has drawn in is a different thing
from a gap."
[clip sid id drawing-id {:keys [at extent] :or {extent :keep at :end}}]
(if-let [why (or (placeable clip sid id at)
(when (clip/symbol clip drawing-id) "the new drawing ID is already used"))]
{:refused why}
(place (assoc-in clip [:symbols drawing-id]
{:id drawing-id :name (name drawing-id) :fps (clip/fps clip sid) :frames 1 :nodes {}})
sid id drawing-id at extent true)))
(defn reuse-drawing
"Append a held clip of content the document ALREADY has, so the same
drawing is exposed twice and editing it changes both clips.
This is the command `make-unique` is the undo of, and the reason they are two
commands: reuse is a decision to share, and sharing is not something to
discover later when an edit turns up somewhere else."
[clip sid id drawing-id {:keys [at extent] :or {extent :keep at :end}}]
(if-let [why (or (placeable clip sid id at)
(when-not (clip/symbol clip drawing-id) "there is no such drawing to reuse")
;; Placing something that contains this symbol would close a
;; loop, and a sequence is no different from any other placement.
(when (clip/contains-symbol? clip drawing-id sid)
"a symbol cannot go inside itself"))]
{:refused why}
(place clip sid id drawing-id at extent true)))
(defn- copied
"A copy of symbol `from`, as `{:clip :id}`.
SHALLOW by default: its own nodes and channels are copied, and its references
to other symbols are kept, so a head built out of reusable eyes still uses
those eyes. `deep?` copies everything it places as well, with new ids
throughout, for a drawing that must share nothing — the distinction the
shallow copy cannot make on its own, and a promise of independence that only
the deep one keeps."
[clip from deep?]
(if deep?
(let [{c :clip ids :ids} (bring/symbols clip clip [from] {})]
{:clip c :id (ids from)})
(let [id (clip/free-id (:symbols clip) from)]
{:clip (assoc-in clip [:symbols id] (assoc (clip/symbol clip from) :id id))
:id id})))
(defn duplicate-drawing
"Append a held clip of a COPY of what clip `id` places, for when
the drawing on screen is the starting point for the next one.
The copy is of the content only. The new clip is a plain one-frame hold
rather than a copy of `id`'s own transform or corrections: those belong to
that clip, and carrying them over would make duplicating a drawing quietly
duplicate the treatment of one use of it."
[clip sid id new-id {:keys [at extent deep?] :or {extent :keep at :end}}]
(let [n (get-in clip [:symbols sid :nodes id])
from (node/source n)]
(if-let [why (or (when-not from "select a clip to duplicate")
(when-not (clip/symbol clip from) "the drawing it places is missing")
(placeable clip sid new-id at))]
{:refused why}
(let [{c :clip copy :id} (copied clip from deep?)]
(place c sid new-id copy at extent true)))))
(defn overwrite-drawing
"Put a fresh one-frame drawing at frame `at`, replacing whatever was there and
leaving every other clip where it was.
This is `blank` and placement composed in ONE command and therefore one undo
step. `remainder-id` is used only when clearing the frame cuts one clip into
two; ids still come from the caller because this namespace is pure."
[clip sid id drawing-id at {:keys [extent remainder-id] :or {extent :keep}}]
(let [nodes (get-in clip [:symbols sid :nodes])]
(if-let [why (cond
(not (and (integer? at) (not (neg? at))))
"a position is a nonnegative whole frame"
(contains? nodes id) "the new clip ID is already used"
(or (= id remainder-id) (contains? nodes remainder-id))
"the remainder clip needs a free ID different from the new clip"
(clip/symbol clip drawing-id) "the new drawing ID is already used")]
{:refused why}
(let [c (assoc-in clip [:symbols drawing-id]
{:id drawing-id :name (name drawing-id)
:fps (clip/fps clip sid) :frames 1 :nodes {}})
nodes (assoc nodes id (held id drawing-id at))]
(claim c sid nodes id extent remainder-id)))))
(defn make-unique
"Point clip `id` at a private copy of its content, leaving every other
clip of that drawing sharing the original.
Refused when nothing else uses it: a drawing with one clip is already
unique, and answering with a silent copy would leave a second identical symbol
in the library for no reason a person could see."
[clip sid id {:keys [deep?]}]
(let [n (get-in clip [:symbols sid :nodes id])
from (node/source n)
elsewhere (for [[osid osym] (:symbols clip)
[oid on] (:nodes osym)
:when (and (= from (node/source on)) (not= [sid id] [osid oid]))]
[osid oid])]
(if-let [why (or (when-not from "select a clip to make unique")
(when-not (clip/symbol clip from) "the drawing it places is missing")
(when (empty? elsewhere) "nothing else uses this drawing"))]
{:refused why}
(let [{c :clip copy :id} (copied clip from deep?)
c (assoc-in c [:symbols sid :nodes id :source :symbol] copy)
ps (clip/problems c)]
(if (seq ps) {:refused (first ps)} {:clip c :selection id})))))

View file

@ -0,0 +1,866 @@
(ns arthur.domain.symbol
"A SYMBOL: an ordered bag of nodes in its own frame space, and the two ways to
evaluate it at a frame.
{:id :main :frames 229 :nodes {id -> node} :palette nil}
That is the whole type, and EVERYTHING THAT HOLDS NODES IS ONE OF THESE. What
a document opens on is a symbol; what a `:kind :instance` node places is a
symbol; there is no second structure. An earlier arrangement had a root node
tree and a library entry as two structures with the same fields and never said
they were the same thing. Flash's `_root` is a MovieClip and After Effects'
pre-comp is just a layer; collapsing them is what makes nesting arbitrary and
free rather than a feature to be added.
The clip-level facts are in `arthur.domain.clip`. A symbol has a FRAME SPACE,
not a rate and not a size: `:fps` is the clip's, because a rate is a fact about
how fast the whole thing plays, and a nested symbol cannot have its own.
TWO AXES OF NESTING, and conflating them is why \"nested\" and \"flat with parent
pointers\" sound contradictory when they are not. Parent/child is transform
composition WITHIN one symbol and is stored flat with pointers. Instance is a
symbol inside another symbol and is stored by reference into the library.
Each symbol is flat; symbols nest. Every argument for flat storage —
addressability, one-field reparenting, structural sharing, per-node sync leaves —
is about the first axis and is untouched by the second.
Two ways to evaluate one at a frame:
(eval-frame sym f store palette opts)
THE SPECIFICATION. Allocating, order-free,
obviously correct. Use it in tests and for a
one-off render.
(resolver sym store palette opts)
-> (fn [f] ops). What playback uses. Caches the
topological order and the z paths, holds one
CURSOR per channel and one PREALLOCATED point
buffer per node, so a frame allocates the op
maps and nothing else.
Both run the same walk — `eval-into` below — parameterised by how a channel is
read and where points are written. That is deliberate: two independent
implementations of frame evaluation would drift, and the drift would look like
a rendering bug rather than like two functions disagreeing. What differs
between them is exactly the part that can be wrong, and symbol-test asserts
they agree frame for frame in forward, backward and random order.
The output is a list of DRAW OPS, and it is the boundary with the rasteriser:
ops carry palette indices and raster-space points, and the rasteriser knows
nothing about nodes, channels or time.
Geometry is stored FLAT — [x0 y0 x1 y1 …] — in authored channels as well as
dense ones. A dense block is a rectangular Int16Array and an authored ring is a
vector of numbers, and they read the same way, which is what makes freezing
fill in the same channel rather than convert into a second format."
(:require [arthur.domain.channel :as ch]
[arthur.domain.node :as node]
[arthur.domain.pose :as pose]
[arthur.domain.palette :as pal]))
;; ---------------------------------------------------------------------------
;; structure: depth, topological order, draw order
(defn lineage
"The node's id and every ancestor's, nearest first and root last.
One walk, shared by `depth` and `z-path`, which otherwise duplicate it.
A cycle is caught by LENGTH rather than by a `seen` set: a chain that does not
repeat cannot be longer than the number of nodes, so one step past that is
proof of a loop and needs no bookkeeping. Caught rather than hung — a cycle is
reachable from one bad `:node/set-parent`, and a hung tab is a far worse
diagnostic than a stack trace naming the nodes.
THE PROOF ONLY HOLDS FOR A NODE THIS SYMBOL HAS. An id that is not in `nodes`
contributes a step the count knows nothing about, and its lineage is just
itself; in an EMPTY symbol that one step used to be read as a loop, so asking
where a node of another symbol sits threw `parent cycle` instead of answering
that it sits nowhere."
[nodes id]
(let [up (fn [i]
(when-let [p (:parent (get nodes i))]
(if (contains? nodes p)
p
(throw (ex-info "node's :parent is not in the symbol"
{:node i :parent p})))))
chain (into [] (comp (take-while some?) (take (inc (count nodes))))
(iterate up id))]
(when (and (contains? nodes id) (> (count chain) (count nodes)))
(throw (ex-info "parent cycle in symbol" {:node id :chain chain})))
chain))
(defn depth
"Number of ancestors."
[nodes id]
(dec (count (lineage nodes id))))
(defn children
"The symbol's own clips — the nodes placed directly in it — in timeline order.
THE SYMBOL IS THE CONTAINER. There is no lane node to ask for its members: a
symbol drawn as a lane draws THESE, and the sequence commands re-span THESE.
See `docs/lane-is-a-view-plan.md`.
Sorted by where they START, not by `:z`: blocks in a sequence follow one
another in time, and two of them cannot be in the same place for `:z` to
decide between. Ties go to the id so the order is the same on every run.
A NODE WITH NO SPAN IS NOT IN THE SEQUENCE. A shape on screen for the whole
shot has no `[in out)` to follow anything else, so it is not something an edge
edit can trim or ripple, and a command that destructured its nil span would
fail on the most ordinary node there is."
[nodes]
(->> (vals nodes)
(filter #(and (nil? (:parent %)) (node/placed-span %)))
(sort-by (juxt #(first (node/placed-span %)) #(str (:id %))))
vec))
(defn frame-map
"Node `id`'s own frames as a map from the containing symbol's, inverted:
`{:at a :rate r}`, meaning symbol frame `p` is frame `r·(p − a)` of `id`.
Identity for `nil`, which is a node sitting directly in the symbol.
THE FRAME SPACE A COMMAND IS GIVEN ITS COORDINATE IN. A direct child reads
the symbol's own frames. A node grouped beneath another composes through that
parent chain. One rule either way, so a caller holding a node does not have to
ask what it is sitting in or how the timeline happens to draw the symbol.
Refuses floors and loops rather than pretend an affine map preserves them:
through either, one frame of the symbol is not one frame of `id` and a command
handed a single frame has no single answer to give."
[nodes id]
(loop [id id seen #{} chain []]
(if (nil? id)
(reduce node/then-time {:at 0 :rate 1} (map node/time-of (reverse chain)))
(let [n (get nodes id) t (:time n)]
(when (and n (not (contains? seen id))
(not (:loop? t))
(<= (or (:expose t) 1) 1)
(empty? (:holds t)))
(recur (:parent n) (conj seen id) (conj chain n)))))))
(defn lane?
"Whether symbol `sym` is EDITED AND DRAWN as a lane: its clips as blocks on
one row, following one another in time and claiming it from each other.
A DISPLAY HINT AND NOT A TYPE. Nothing in evaluation reads it, `problems` does
not check it, and a symbol carrying it behaves identically on the stage — it
says how the timeline draws the symbol and, because the editing rules follow
the mode, which rules an edge drag inside it plays by. It is on the symbol
rather than in editor state so that those rules are reproducible between two
people looking at one document, and it is a field rather than something derived
from \"the clips do not currently overlap\" because a symbol must not stop being
a lane the moment something overlaps — that is when the rules are needed.
Editing and presentation may read it; evaluation may not. See
`docs/lane-is-a-view-plan.md`."
[sym]
(= :lane (:display sym)))
(defn overlaps
"The pairs of `sym`'s clips that are on screen over the same frames, as
`[[a b] ...]` of ids. Empty for a symbol whose clips form a sequence.
A BUG REPORT, NOT A CONDITION TO DESIGN AROUND. In lane mode this cannot
happen: placing claims time, so anything placed, moved or grown over occupied
frames TRIMS what it lands on, and `span/finish` — the one commit path for
every sequence command — refuses rather than committing one. So an overlap that
appears anyway is a defect in a command.
Which is why it is NOT `clip/problems`, which means the document will not load:
a display hint must never be able to stop a document loading, and a document
that somehow arrives holding an overlap still opens and is drawn visibly wrong.
Nor `clip/conflicts`, which means a person has a decision to make. This is
neither."
[sym]
(let [spans (for [n (children (:nodes sym))
:let [[lo hi] (node/placed-span n)]
:when (and (node/finite-number? lo) (node/finite-number? hi))]
[lo hi (:id n)])]
(vec (for [[[_ b x] [c _ y]] (partition 2 1 (sort-by (juxt first second str) spans))
:when (> b c)]
[x y]))))
(defn order
"Node ids in topological order: every node after its parent.
Sorting by parent depth is enough — it does not need Kahn's algorithm, because
the only edge is parent, and a node's depth is by definition greater than its
parent's. Ties are broken by id so the order is deterministic across runs,
which matters because the draw-order sort below falls back on this position."
[nodes]
(vec (sort-by (juxt #(depth nodes %) str) (keys nodes))))
(defn z-path
"The node's z index and every ancestor's, root first.
Draw order is depth-first by sibling z, so the key that sorts it is the chain
of z values from the root. A parent's path is a PREFIX of its child's, which is
why a parent draws before its children without that being a special case.
`:z` values are fractional-index STRINGS (\"a1\", \"a3\") and compare
lexicographically, so a node can always be inserted between two siblings
without renumbering either."
[nodes id]
(mapv #(:z (get nodes %)) (rseq (lineage nodes id))))
(defn z-between
"A `:z` that sorts strictly between `a` and `b`, which must be in order; nil
for either is no bound on that side. What makes restacking one write.
The midpoint of the first character they differ in, when there is room. When
there is not, anything that starts with `a` and is longer sorts after it, and
before `b` too unless `a` is a prefix of `b` — and then the room is found one
character further into `b`. \"0\" is the floor, and nothing this makes ends
in it, so there is always a further character to go to; only an authored key
ending in \"0\" leaves none, and then one character less than it does."
[a b]
(let [a (or a "")]
(if (nil? b)
(str a "m")
(let [i (count (take-while true? (map = a b)))
hi (.charCodeAt b i)
lo (if (< i (count a)) (.charCodeAt a i) 48)
mid (quot (+ lo hi) 2)]
(cond
(> mid lo) (str (subs b 0 i) (char mid))
(< i (count a)) (str a "m")
(< (inc i) (count b)) (str (subs b 0 (inc i)) (z-between nil (subs b (inc i))))
:else (str (subs b 0 i) (char (dec hi)) "m"))))))
(defn- z-lex
"Lexicographic compare of two z paths, a prefix sorting first.
`compare` on vectors will not do: it compares COUNT first, so a deep
descendant of \"a1\" would sort after a shallow \"a2\" and a painted cel would
jump in front of the head that carries it.
`map` over two collections stops at the shorter and `first` short-circuits at
the first difference, so this walks no further than it has to."
[a b]
(or (first (remove zero? (map compare a b)))
(- (count a) (count b))))
(defn draw-rank
"id -> its position in draw order.
Computed ONCE. Draw order is a function of the z paths, which are structural —
they change when the symbol changes and never because the playhead moved — so
sorting ops by z on every frame was re-deriving a constant thirty times a
second. Here it is derived when the symbol is, and a frame sorts small integers.
`sort-by` is stable and `ord` is topological, so nodes sharing a z path keep
parent-before-child order without a tiebreak field on every op."
[nodes ord]
(let [paths (into {} (map (juxt identity #(z-path nodes %))) ord)]
(into {} (map-indexed (fn [i id] [id i])) (sort-by paths z-lex ord))))
;; ---------------------------------------------------------------------------
;; colour
(defn colour-index
"Tone keyword -> the index the raster writes, in a given palette.
`palette` is a map of tone -> index. It is a PARAMETER, not a global: a tone
names which mark this is, and which ramp it is read in belongs to the symbol
the node sits in, so resolution cannot reach for one ambient answer. Today
there is one palette and it is passed in anyway; when symbols carry a
`:palette` channel, the walk carries the palette in scope exactly as it already
carries the parent transform and the local frame.
An unknown tone resolves to 255, which the palette expansion renders MAGENTA.
Loud rather than fatal, and the same choice raster/->rgba already makes:
naming a colour the ramp does not have is a bug in authored data, and it should
be impossible to miss and should not take the frame down."
[palette k]
(cond
(fn? palette) (palette k)
(number? k) k
(nil? k) 255
:else (get palette k 255)))
(defn knockout?
"Is colour `c` a KNOCKOUT rather than a tone? `:clear` clears every colour
beneath it in its symbol, `[:clear tone]` clears that tone only. See
`raster/plot!`."
[c]
(or (= :clear c) (and (vector? c) (= :clear (first c)))))
(defn remap?
"Is colour `c` a REMAP: `[:remap {from to …}]`, slots of the palette in
scope? It draws nothing of its own; what is already under it is drawn in
other slots — a flashlight is a circle of this. See `raster/plot!`."
[c]
(and (vector? c) (= :remap (first c))))
(defn- knock-index [palette c]
(if (= :clear c) -1 (colour-index palette (second c))))
(defn lut
"Remap `[:remap m]` as a raster index -> index table in `palette`. Slots of
any other palette are left as they are."
[palette [_ m]]
(let [t (js/Uint8Array. 256)]
(dotimes [i 256] (aset t i i))
(doseq [[a b] m] (aset t (colour-index palette a) (colour-index palette b)))
t))
(defn knocks?
"Does any node in `nodes` knock out, on any frame? Such a symbol is drawn into
a layer of its own."
[nodes]
(some (fn [[_ n]]
(let [c (get-in n [:channels [:style :color]])]
(some knockout? (cons (:value c) (vals (:keys c))))))
nodes))
;; ---------------------------------------------------------------------------
;; the walk
(defn- in-span?
"`:span` is Lottie's ip/op and Flash's PlaceObject/RemoveObject: the range over
which the node EXISTS, tested in the PARENT's frame space and therefore before
the node's own time map runs — an instance's own-time span is mapped out by
`node/placed-span`. Distinct from `[:vis]`, which blinks an existing node on and
off. Half-open, so two adjacent spans do not both own a frame."
[n f]
(if-let [[in out] (node/placed-span n)]
(and (>= f in) (< f out))
true))
(defn- finish
"Resolve stencils, then sort into draw order.
A stencil is a COLOUR KEY, not a node reference: it is the take format's
`clip=`, and the indexed buffer being its own clip mask is what keeps the iris
inside the eye at any gaze and any radius without a per-part mask. So the
stencil node's own colour is looked up here, after the walk, because the
stencil may sit anywhere in the order. Two nodes sharing a palette entry share
a stencil, which is inherent to the technique rather than a defect in it.
A node stencilled by something that drew NOTHING is DROPPED, not drawn
unclipped: unclipped would be an iris floating over the cheek on exactly the
frames where the eye is missing."
[rank ops]
(let [by-id (into {} (map (juxt :node :color)) ops)]
(->> ops
(keep (fn [op]
(if-let [s (:stencil op)]
(when-let [idx (get by-id s)]
(assoc op :stencil idx))
op)))
(sort-by (comp rank :node))
vec)))
(defn- n-points
"Points in a flat [x0 y0 x1 y1 …] value, authored vector or dense view alike."
[pts]
(quot (if (vector? pts) (count pts) (.-length pts)) 2))
(defn- xform-at
"The five transform components at the node's local frame, or nil when any of
them has no value on it.
The pivot is read like the rest and is not special-cased to a default here:
`node/channels` has already filled in `[0 0]` for a node that stores none, and
a node whose pivot is absent on a frame it is otherwise on has the same nothing
to be drawn at as one whose position is."
[rd]
(let [pos (rd [:xform :pos])
piv (rd [:xform :pivot])
rot (rd [:xform :rot])
scl (rd [:xform :scale])
skw (rd [:xform :skew])]
(when-not (or (ch/nothing? pos) (ch/nothing? piv) (ch/nothing? rot)
(ch/nothing? scl) (ch/nothing? skw))
[pos piv rot scl skw])))
(defn- visible?
"Is the node switched on this frame?
`[:vis]` IS A BOOLEAN, and this insists on it rather than testing truthiness,
because the two obvious implementations are both wrong about a DENSE `[:vis]`.
A dense block yields 0 or 1, and 0 is TRUTHY in CLJS — so `(if v …)` shows a
hidden frame, and `(true? v)` hides every frame. Neither reads as an error.
docs/animation-model.md's parts table says `:mouth-in` carries `[:vis]` dense;
flow/freeze writes it KEYED, because a threshold crossing is a handful of
transitions and hold is the default, and because a human has to be able to fix
one frame of it. When something does want a dense one it will land here loudly
instead of blanking the symbol.
Absence is not a boolean and is not an error: a subject that is not on the
frame has nothing to show."
[id v]
(cond
(true? v) true
(false? v) false
(ch/nothing? v) false
:else (throw (ex-info "[:vis] must sample to a boolean"
{:node id :value v}))))
(defn- place
"Where a node sits this frame, as {:m world :f local-frame :rd reader}, or nil
when it is not on the frame at all.
Three gates, and nil from any of them removes the node's DESCENDANTS too,
which is why this is one answer rather than three flags: a node outside its
span does not exist, a switched-off feature takes its parts with it, and a node
with no transform gives its children nowhere to be.
A missing [:geom :pts] is deliberately NOT one of them — that is `emit`'s
business. An absent mouth outline has nothing to draw, but the head it hangs
off is still exactly where it was, and that asymmetry is the whole reason
presence is tracked per channel rather than per node."
[{:keys [read mat-for pinv-for scratch]} n parent f pre]
(let [pf (if parent (:f parent) f)
;; Where the PREVIOUS grid slot landed, carried down the same path as `f`
;; through the same time maps. `(:pre parent)` rather than a second walk,
;; so an exposure fold or a retime on an ancestor is in it already — which
;; is what makes a held exposure's interval come out right without the snap
;; knowing exposure exists.
ppf (if parent (:pre parent) pre)]
(when (in-span? n pf)
(let [id (:id n)
chs (node/channels n)
lf (node/local-frame n pf)
plf (node/local-frame n ppf)
rd (fn [path] (read id path (get chs path) lf plf))]
(when (visible? id (rd [:vis]))
(when-let [[pos piv rot scl skw] (xform-at rd)]
;; dest aliases `local` here, which mul! allows: it reads both
;; operands fully before writing either.
(let [m (node/local! (mat-for id) pos piv rot scl skw)]
{:m (node/world! m (:m parent) (pinv-for id) m scratch)
:f lf
:pre plf
:rd rd})))))))
(defn- emit
"Emit geometry in the symbol's space. Rect sizes stay fractional until
rasterization, so enclosing symbol transforms can still scale them."
[{:keys [palette buf-for]} n {:keys [m rd]} base]
;; Read only by the kinds that have a colour: a group or an instance has no
;; `[:style :color]` to read.
(let [paint (fn [op]
(let [c (rd [:style :color])]
(cond
(knockout? c) (assoc op :knock (knock-index palette c) :color 0)
(remap? c) (assoc op :lut (lut palette c) :color 0)
:else (assoc op :color (colour-index palette c)))))]
(case (:kind n)
:group nil
:instance nil
:poly
(let [pts (rd [:geom :pts])]
(when-not (ch/nothing? pts)
(let [np (n-points pts)
out (buf-for (:id n) np)]
(dotimes [k np]
(node/apply-pt! out k m
(ch/component pts (* 2 k))
(ch/component pts (inc (* 2 k)))))
(paint (assoc base :kind :poly :pts out :n np)))))
:disc
(let [rad (rd [:geom :radius])]
(when-not (ch/nothing? rad)
(paint (assoc base :kind :disc
:cx (aget m 4) :cy (aget m 5)
:r (* rad (node/mean-scale m))))))
:rect
(let [size (rd [:geom :size])]
(when-not (ch/nothing? size)
(paint (assoc base :kind :rect
:cx (aget m 4) :cy (aget m 5)
:size (* size (node/mean-scale m))))))
(throw (ex-info "node kind is not implemented"
{:node (:id n) :kind (:kind n)})))))
(defn- nodes-of
"The symbol's node map, REFUSING a map that has none.
A clip and a symbol both have an `:id` and both are maps, so handing a CLIP to
an evaluator is the one mistake this type split makes easy — and the result is
not an error, it is `(:nodes clip)` being nil and a frame resolving to no ops at
all. That reads as a black stage, or, in a benchmark, as \"0 nodes\" and a
flattering number. It happened once while the split was being made, which is why
this is a guard and not a comment.
Sounds are left out: they are heard, not drawn, and have no transform to
evaluate. `audio/mix` is what plays them."
[sym]
(let [nodes (:nodes sym)]
(when-not (map? nodes)
(throw (ex-info (str "not a symbol: :nodes is " (pr-str nodes)
" — a clip is not a symbol, its `:symbols` hold them")
{:keys (vec (sort-by str (keys sym)))})))
(into {} (remove #(= :audio (:kind (val %)))) nodes)))
(defn- base-channel-frame
"`:reads` holds the frames a node's own channels read; marked channels read
instance pose choices, and the preserve-snap where nobody has made one.
`lf` is this slot's local frame and `plf` the PREVIOUS slot's, both already
through placement and retime, so `(plf, lf]` is the interval this slot is the
first to cover — the frames the grid shows to nobody. That is the only thing the
snap needs from the grid, and it is why the pair is threaded this far down
instead of the snap happening where the grid becomes a native frame: the snap is
per pose group, and a group is a fact that only exists here."
[choices marks reads nodes id c lf plf]
(cond
(contains? reads id)
(node/hold (js/Math.floor lf) (get reads id))
(:pose-sampled? c)
(let [group (or (:pose-group (get nodes id)) id)]
(pose/source-frame choices
(if (contains? choices [:node id]) [:node id] group)
lf
;; THE SNAP IS THE DEFAULT POSE. `source-frame` reaches a
;; default only where the hand has said nothing, so seating
;; it here leaves an explicit cut beating a snap for free —
;; manual precedence is absolute — and costs no plumbing on
;; the pose side, per-group choices being threaded already.
(pose/snapped-frame marks group
(js/Math.floor plf)
(js/Math.floor lf))))
:else (js/Math.floor lf)))
(defn- holds-read
"The frames node `n` reads its OWN channels at, held — not its children's,
which is what makes this different from `:time :holds` — or nil for every
frame. `{:holds [0]}` is a face's head at its start, and `{:holds-of :plate}`
is the head holding wherever its footage holds, which is one list of frames
read by two nodes rather than two lists kept in step."
[nodes n]
(let [{:keys [holds holds-of]} (:reads n)
hs (if holds-of (get-in nodes [holds-of :time :holds]) holds)]
(when (seq hs) hs)))
(defn- prepared-reads [nodes]
(into {} (keep (fn [[id n]] (some->> (holds-read nodes n) (vector id)))) nodes))
(defn- eval-into
"One frame, as a fold over the nodes in topological order.
`ctx` carries how a channel is read and where its points are written:
:read (fn [id path channel local-frame] -> v)
:palette tone -> index, the ramp in scope
:mat-for (fn [id] -> Float64Array) the node's world transform
:pinv-for (fn [id] -> Float64Array|nil) its parent-inverse
:buf-for (fn [id n-points] -> Float64Array)
:scratch one spare 6-element matrix"
[ctx nodes ord rank f pre]
(-> (reduce
(fn [{:keys [placed ops] :as acc} id]
(let [n (get nodes id)
pid (:parent n)
parent (when pid (get placed pid))]
;; A node whose parent was dropped is dropped with it, and so is
;; everything under it. Topological order is what makes that one
;; lookup instead of a subtree walk.
(if (and pid (nil? parent))
acc
(if-let [p (place ctx n parent f pre)]
(let [_ (when-let [on-place (:on-place ctx)] (on-place id p))
op (emit ctx n p {:node id :stencil (:stencil n)})]
(cond-> (update acc :placed assoc id p)
op (update :ops conj op)))
acc))))
{:placed {} :ops []}
ord)
:ops
(->> (finish rank))))
;; ---------------------------------------------------------------------------
;; the specification
(defn eval-frame
"Symbol at frame f -> draw ops in z order. Pure, and allocates freely.
`f` is in THIS symbol's frame space. For the symbol on screen that is the
transport's frame; inside an instance it is the instance's own space, and the instance boundary is
the only place the space changes.
`pre` is where the grid slot BEFORE this one landed, which with `f` is the
interval the preserve-snap may reach back into — see `pose/snapped-frame`. The
shorter arity means one native frame per slot, so the interval is `f` alone and
nothing snaps: that is what a caller rendering a native frame directly is asking
for, and it is what keeps this evaluator and `resolver` the same answer.
This is the definition of what a frame means. `resolver` is what plays it."
([sym f store palette opts] (eval-frame sym f (dec f) store palette opts))
([sym f pre store palette {:keys [pose-tracks snap]}]
(let [nodes (nodes-of sym)
choices (pose/prepare pose-tracks)
marks (when (and snap (snap (:id sym)))
(pose/marks nodes (:frames sym) store))
reads (prepared-reads nodes)
ord (order nodes)]
(eval-into {:read (fn [id path c lf plf]
(ch/value-at c (base-channel-frame choices marks reads
nodes id c lf plf)
lf store))
:palette palette
:mat-for (fn [_id] (node/mat))
:pinv-for (fn [id] (node/pinv (get nodes id)))
:buf-for (fn [_id n] (js/Float64Array. (* 2 n)))
:scratch (node/mat)}
nodes ord (draw-rank nodes ord) f pre))))
;; ---------------------------------------------------------------------------
;; the playback path
(defn- point-capacity
"How many points the widest value of a [:geom :pts] channel holds.
FIXED TOPOLOGY is what makes this a number at all: every key of a part carries
the same vertex count with the same vertex meanings, so the buffer can be
allocated once. A variable vertex count would force a per-frame offset table
and a scan, which is why the aesthetic constraint is a performance asset rather
than a cost."
[c]
(quot (cond
(:dense c) (:stride (:dense c))
(:animated? c) (transduce (map #(if (vector? %) (count %) (.-length %)))
max 0 (vals (:keys c)))
:else (let [v (:value c)] (if (vector? v) (count v) (.-length v))))
2))
(defprotocol IResolver
(world-of [this id]
"The node's world transform AS OF THE LAST FRAME RESOLVED, or nil if it was
not placed on that frame.
The matrices are the ones evaluation mutates in place, so this is a read of
live state rather than a snapshot — which is exactly what the caller wants.
A registered photo underlay has to ride the same transform the vectors went
through or it is merely decorative, and it paints immediately after the frame
it belongs to, so \"as of the last frame\" is the only answer that can be
correct.")
(frame-of [this id] "The placed node's local frame on the last resolve.")
(pre-frame-of [this id]
"The same node's local frame for the grid slot BEFORE the last resolve.
Half of the preserve-snap's interval, and the half only the walk can answer:
the mapping from a grid slot to a node's local frame runs through every time
map between them, so the frame the previous slot landed on is what the walk
carried, not something a caller can recompute from the slot index."))
(defn resolver
"(fn [f] -> ops). Holds everything that does not change per frame.
The point buffers are REUSED between frames, so a caller must consume the ops
before asking for the next frame. That is the contract the rAF loop wants
anyway — it reads, blits, and dispatches nothing — and it is what makes a frame
cost a lookup and a blit rather than an allocation per vertex.
The op maps themselves are allocated fresh, and deliberately: there are a dozen
of them per frame against hundreds of points, so pooling them would buy
nothing and cost the ability to hand an op list around as plain data.
`store` and `palette` are POSITIONAL because neither is optional: a dense
channel cannot be read without the store it names, and every op carries a
colour index. `opts` is a map because the rest genuinely are optional, and
because a fifth of them later is then a key rather than a nil at every one of
these call sites — which is what the arity ladder that used to be here was
standing in for."
[sym store palette {:keys [pose-tracks snap]}]
(let [nodes (nodes-of sym)
choices (pose/prepare pose-tracks)
;; NO MARKS IS THE OFF STATE, and that it needs no second code path is
;; the reason `snapped-frame` falls back to the slot's own default rather
;; than being asked whether it should. Off is the cadence alone, which is
;; what ships today, so it is also the DEFAULT: an opts map that says
;; nothing gets the behaviour it got before the snap existed.
;;
;; `:snap` IS ASKED ABOUT THIS SYMBOL, not the resolver, because the marks
;; are the FACE'S: the closure cuts are stored on its own nodes, so every
;; placement of one face has the same ones and a per-placement answer would
;; be a setting with nothing in it. A set of symbol ids is the usual
;; argument, and any predicate on one will do.
;;
;; One option, and it is the snap's alone. The plate side will never want
;; one — its proposal is materialised into the plate's `:time :holds` rather than
;; computed on the render path — so this is not half of a pair.
marks (when (and snap (snap (:id sym)))
(pose/marks nodes (:frames sym) store))
reads (prepared-reads nodes)
ord (order nodes)
rank (draw-rank nodes ord)
cursors (into {}
(map (fn [id]
[id (into {} (map (fn [[p c]] [p (ch/cursor c store)]))
(node/channels (get nodes id)))]))
ord)
mats (into {} (map (fn [id] [id (node/mat)])) ord)
pinvs (into {} (keep (fn [id] (when-let [p (node/pinv (get nodes id))] [id p]))) ord)
bufs (into {}
(keep (fn [id]
(when-let [c (get-in nodes [id :channels [:geom :pts]])]
[id (js/Float64Array. (* 2 (point-capacity c)))])))
ord)
scratch (node/mat)
;; Every call to mat-for is a placement: eval-into reaches it only after
;; the span, visibility and transform gates have all passed. So wrapping
;; it is how the resolver learns which nodes exist this frame without
;; eval-into having to report it — and it covers groups, which are
;; placed but emit no op, and which are exactly what an underlay rides.
placed (volatile! {})
ctx {:read (fn [id path c lf plf]
(when-let [cursor (get-in cursors [id path])]
(ch/sample! cursor
(base-channel-frame choices marks reads
nodes id c lf plf)
lf)))
:palette palette
:mat-for (fn [id] (get mats id))
:on-place (fn [id p] (vswap! placed assoc id p))
:pinv-for (fn [id] (get pinvs id))
:buf-for (fn [id _n] (get bufs id))
:scratch scratch}
step (fn [f pre]
(vreset! placed {})
(eval-into ctx nodes ord rank f pre))]
(reify
IFn
;; One frame and no interval is one native frame per slot — see `eval-frame`.
(-invoke [_ f] (step f (dec f)))
(-invoke [_ f pre] (step f pre))
IResolver
(world-of [_ id] (:m (get @placed id)))
(frame-of [_ id] (:f (get @placed id)))
(pre-frame-of [_ id] (:pre (get @placed id))))))
;; ---------------------------------------------------------------------------
(def symbol-keys
"Every field a symbol may carry, and the reason `arthur.domain.leaf` refuses
one it does not know: a field added without a leaf to save it in is a field that
saves silently and comes back missing.
`:palette` is in the vocabulary and nothing writes one yet. A symbol is where
a ramp belongs — `domain/symbol` takes the palette as a PARAMETER rather than
reaching for a global precisely so that a nested symbol can carry its own —
and leaving the field out would make the first one a migration instead of a
write.
`:name` is what a person calls it, and is not its id: an id is what instances
and saved leaves point at, so renaming a symbol must not change it.
`:width` and `:height` are the symbol's own stage, and are absent until someone
sets them: a symbol without them uses the clip's — see `clip/stage`.
`:display` is how the TIMELINE draws the symbol — `:lane` for its clips as
blocks on one row — and is saved because two people editing one document must
play by the same editing rules. See `lane?`.
`:media` is what a `:type :trace` symbol shows — `{:footage id :range [in
out]}` or `{:image sha256}` — and such a symbol has no nodes: its frames are
the footage's, its `:fps` the footage's own and its `:width`/`:height` its
pixels. A still shows the same picture on every one of its frames, so how many
it has is only how long it was made to last, and trimming its placement is
how that changes. See `clip/trace-op`."
#{:id :name :frames :fps :width :height :nodes :palette :palette-track
:palette-channel :type :palette-ref :display :media})
(defn- media-problems
"Why `sym`, a tracing symbol, does not say what it shows. Empty when it does."
[{:keys [media frames width height nodes]}]
(let [{:keys [footage image range]} media]
(cond-> []
(seq nodes)
(conj "a tracing symbol cannot contain nodes")
(not (= 1 (count (filter some? [footage image]))))
(conj ":media must name exactly one of :footage or :image")
(and footage (not (and (vector? range) (= 2 (count range))
(every? integer? range) (apply < range)
(= frames (- (second range) (first range))))))
(conj "a footage :media needs a [in out) :range as long as the symbol's :frames")
(not (and (integer? frames) (pos? frames)))
(conj "a tracing symbol is a positive whole number of frames")
(not (and (pos? width) (pos? height)))
(conj "a tracing symbol needs its media's pixel :width and :height"))))
(defn problems
"Human-readable reasons this symbol will not evaluate. Empty means it will.
Node structure only. The tracking identities — subjects, features, groups — are
the CLIP's and are checked by `arthur.domain.clip/problems`, which is not a
layering nicety: a feature names nodes, and a library symbol's nodes are not
the ones a face was tracked into.
Total by construction — it reports a cycle rather than looping on one — because
its whole job is to be safe to run over authored data before that data is
trusted."
[sym]
(let [nodes (:nodes sym)]
(if-not (map? nodes)
[":nodes must be a map of id -> node"]
(-> []
(cond->
(and (= :palette (:type sym)) (seq nodes))
(conj "a palette symbol cannot contain nodes"))
(into (when (= :trace (:type sym)) (media-problems sym)))
(into (for [[id n] nodes
:when (not= id (:id n))]
(str "node under key " (pr-str id) " has :id " (pr-str (:id n)))))
(into (for [[id n] nodes
:when (and (:parent n) (not (contains? nodes (:parent n))))]
(str "node " (pr-str id) " has :parent " (pr-str (:parent n))
" which is not in the symbol")))
(into (for [[id n] nodes
:when (and (:stencil n) (not (contains? nodes (:stencil n))))]
(str "node " (pr-str id) " has :stencil " (pr-str (:stencil n))
" which is not in the symbol")))
(into (for [[id n] nodes
p (node/problems n)]
(str "node " (pr-str id) ": " p)))
(into (for [[id n] nodes
:when (some? (:trace n))]
(str "node " (pr-str id) ": :trace is gone — trace keys are the "
"footage placement's :time :holds, and a head follows them "
"with :reads {:holds-of …}; make the face again from its footage")))
;; `:reads` names its holds or another node's, the way `:stencil` names
;; a node: one that is here, and never one whose frame depends on the
;; reader's own channels being read first.
(into (for [[id n] nodes
:let [{:keys [holds holds-of] :as r} (:reads n)]
:when (some? r)
p (cond
(and holds holds-of)
[":reads names its own :holds or a node's, not both"]
holds-of
(cond
(not (contains? nodes holds-of))
[(str ":reads :holds-of " (pr-str holds-of) " is not in the symbol")]
(some #{holds-of} (lineage nodes id))
[":reads :holds-of names the node itself or one above it"])
:else
(when-not (and (vector? holds) (every? node/finite-number? holds)
(or (empty? holds) (apply < holds)))
[":reads :holds must be a vector of increasing frames"]))]
(str "node " (pr-str id) ": " p)))
(into (for [k (remove symbol-keys (keys sym))]
(str "symbol has a field with no leaf to save it in: " (pr-str k))))
(into (when-not (or (nil? (:frames sym)) (and (integer? (:frames sym)) (pos? (:frames sym))))
[(str ":frames is " (pr-str (:frames sym))
" — a symbol is a frame SPACE, so its length is a positive integer")]))
(into (when (and (some? (:fps sym))
(not (and (node/finite-number? (:fps sym)) (pos? (:fps sym)))))
[":fps must be a positive finite native rate"]))
(into (for [k [:width :height]
:let [v (get sym k)]
:when (and (some? v) (not (and (integer? v) (pos? v))))]
(str k " is " (pr-str v) " — a symbol stage dimension must be a positive integer")))
(into (try
(doall (map #(depth nodes %) (keys nodes)))
nil
(catch :default e [(ex-message e)])))))))

View file

@ -1,573 +0,0 @@
(ns arthur.domain.timeline
"A TIMELINE: an ordered bag of nodes in its own frame space, and the two ways to
evaluate it at a frame.
{:id :main :frames 229 :nodes {id -> node} :palette nil}
That is the whole type, and EVERYTHING THAT HOLDS NODES IS ONE OF THESE. A
clip's root timeline is one; a symbol in the library is one; a `:kind :symbol`
node is an INSTANCE of one. An earlier arrangement had the clip's node tree and
a library symbol as two structures with the same fields and never said they were
the same thing — the clip map carried `:fps`, `:width`, `:height`, `:analysis`
and the tracking identities alongside `:nodes`, so a symbol had nowhere to live
that was not a clip with seven meaningless fields. Flash's `_root` is a
MovieClip and After Effects' pre-comp is just a layer; collapsing them is what
makes nesting arbitrary and free rather than a feature to be added.
The clip-level facts are in `arthur.domain.clip`. A timeline has a FRAME SPACE,
not a rate and not a size: `:fps` is the clip's, because a rate is a fact about
how fast the whole thing plays, and a nested timeline cannot have its own.
TWO AXES OF NESTING, and conflating them is why \"nested\" and \"flat with parent
pointers\" sound contradictory when they are not. Parent/child is transform
composition WITHIN one timeline and is stored flat with pointers. Instance is a
timeline inside another timeline and is stored by reference into the library.
Each timeline is flat; timelines nest. Every argument for flat storage —
addressability, one-field reparenting, structural sharing, per-node sync leaves —
is about the first axis and is untouched by the second.
Two ways to evaluate one at a frame:
(eval-frame tl f store) THE SPECIFICATION. Allocating, order-free,
obviously correct. Use it in tests and for a
one-off render.
(resolver tl store) -> (fn [f] ops). What playback uses. Caches the
topological order and the z paths, holds one
CURSOR per channel and one PREALLOCATED point
buffer per node, so a frame allocates the op
maps and nothing else.
Both run the same walk — `eval-into` below — parameterised by how a channel is
read and where points are written. That is deliberate: two independent
implementations of frame evaluation would drift, and the drift would look like
a rendering bug rather than like two functions disagreeing. What differs
between them is exactly the part that can be wrong, and timeline-test asserts
they agree frame for frame in forward, backward and random order.
The output is a list of DRAW OPS, and it is the boundary with the rasteriser:
ops carry palette indices and raster-space points, and the rasteriser knows
nothing about nodes, channels or time.
Geometry is stored FLAT — [x0 y0 x1 y1 …] — in authored channels as well as
dense ones. A dense block is a rectangular Int16Array and an authored ring is a
vector of numbers, and they read the same way, which is what makes freezing
fill in the same channel rather than convert into a second format."
(:require [arthur.domain.channel :as ch]
[arthur.domain.node :as node]
[arthur.domain.pose :as pose]
[arthur.domain.palette :as pal]))
;; ---------------------------------------------------------------------------
;; structure: depth, topological order, draw order
(defn lineage
"The node's id and every ancestor's, nearest first and root last.
One walk, shared by `depth` and `z-path`, which otherwise duplicate it.
A cycle is caught by LENGTH rather than by a `seen` set: a chain that does not
repeat cannot be longer than the number of nodes, so one step past that is
proof of a loop and needs no bookkeeping. Caught rather than hung — a cycle is
reachable from one bad `:node/set-parent`, and a hung tab is a far worse
diagnostic than a stack trace naming the nodes."
[nodes id]
(let [up (fn [i]
(when-let [p (:parent (get nodes i))]
(if (contains? nodes p)
p
(throw (ex-info "node's :parent is not in the timeline"
{:node i :parent p})))))
chain (into [] (comp (take-while some?) (take (inc (count nodes))))
(iterate up id))]
(when (> (count chain) (count nodes))
(throw (ex-info "parent cycle in timeline" {:node id :chain chain})))
chain))
(defn depth
"Number of ancestors."
[nodes id]
(dec (count (lineage nodes id))))
(defn order
"Node ids in topological order: every node after its parent.
Sorting by parent depth is enough — it does not need Kahn's algorithm, because
the only edge is parent, and a node's depth is by definition greater than its
parent's. Ties are broken by id so the order is deterministic across runs,
which matters because the draw-order sort below falls back on this position."
[nodes]
(vec (sort-by (juxt #(depth nodes %) str) (keys nodes))))
(defn z-path
"The node's z index and every ancestor's, root first.
Draw order is depth-first by sibling z, so the key that sorts it is the chain
of z values from the root. A parent's path is a PREFIX of its child's, which is
why a parent draws before its children without that being a special case.
`:z` values are fractional-index STRINGS (\"a1\", \"a3\") and compare
lexicographically, so a node can always be inserted between two siblings
without renumbering either."
[nodes id]
(mapv #(:z (get nodes %)) (rseq (lineage nodes id))))
(defn- z-lex
"Lexicographic compare of two z paths, a prefix sorting first.
`compare` on vectors will not do: it compares COUNT first, so a deep
descendant of \"a1\" would sort after a shallow \"a2\" and a painted cel would
jump in front of the head that carries it.
`map` over two collections stops at the shorter and `first` short-circuits at
the first difference, so this walks no further than it has to."
[a b]
(or (first (remove zero? (map compare a b)))
(- (count a) (count b))))
(defn draw-rank
"id -> its position in draw order.
Computed ONCE. Draw order is a function of the z paths, which are structural —
they change when the timeline changes and never because the playhead moved — so
sorting ops by z on every frame was re-deriving a constant thirty times a
second. Here it is derived when the timeline is, and a frame sorts small integers.
`sort-by` is stable and `ord` is topological, so nodes sharing a z path keep
parent-before-child order without a tiebreak field on every op."
[nodes ord]
(let [paths (into {} (map (juxt identity #(z-path nodes %))) ord)]
(into {} (map-indexed (fn [i id] [id i])) (sort-by paths z-lex ord))))
;; ---------------------------------------------------------------------------
;; colour
(defn colour-index
"Tone keyword -> the index the raster writes, in a given palette.
`palette` is a map of tone -> index. It is a PARAMETER, not a global: a tone
names which mark this is, and which ramp it is read in belongs to the timeline
the node sits in, so resolution cannot reach for one ambient answer. Today
there is one palette and it is passed in anyway; when timelines carry a
`:palette` channel, the walk carries the palette in scope exactly as it already
carries the parent transform and the local frame.
An unknown tone resolves to 255, which the palette expansion renders MAGENTA.
Loud rather than fatal, and the same choice raster/->rgba already makes:
naming a colour the ramp does not have is a bug in authored data, and it should
be impossible to miss and should not take the frame down."
[palette k]
(cond
(number? k) k
(nil? k) 255
:else (get palette k 255)))
;; ---------------------------------------------------------------------------
;; the walk
(defn- in-span?
"`:span` is Lottie's ip/op and Flash's PlaceObject/RemoveObject: the range over
which the node EXISTS, tested in the PARENT's frame space and therefore before
the node's own time map runs. Distinct from `[:vis]`, which blinks an existing
node on and off. Half-open, so two adjacent spans do not both own a frame."
[n f]
(if-let [[in out] (:span n)]
(and (>= f in) (< f out))
true))
(defn- finish
"Resolve stencils, then sort into draw order.
A stencil is a COLOUR KEY, not a node reference: it is the take format's
`clip=`, and the indexed buffer being its own clip mask is what keeps the iris
inside the eye at any gaze and any radius without a per-part mask. So the
stencil node's own colour is looked up here, after the walk, because the
stencil may sit anywhere in the order. Two nodes sharing a palette entry share
a stencil, which is inherent to the technique rather than a defect in it.
A node stencilled by something that drew NOTHING is DROPPED, not drawn
unclipped: unclipped would be an iris floating over the cheek on exactly the
frames where the eye is missing."
[rank ops]
(let [by-id (into {} (map (juxt :node :color)) ops)]
(->> ops
(keep (fn [op]
(if-let [s (:stencil op)]
(when-let [idx (get by-id s)]
(assoc op :stencil idx))
op)))
(sort-by (comp rank :node))
vec)))
(defn- n-points
"Points in a flat [x0 y0 x1 y1 …] value, authored vector or dense view alike."
[pts]
(quot (if (vector? pts) (count pts) (.-length pts)) 2))
(defn- xform-at
"The five transform components at the node's local frame, or nil when any of
them has no value on it."
[rd]
(let [pos (rd [:xform :pos])
rot (rd [:xform :rot])
scl (rd [:xform :scale])
skw (rd [:xform :skew])
anc (rd [:xform :anchor])]
(when-not (or (ch/nothing? pos) (ch/nothing? rot) (ch/nothing? scl)
(ch/nothing? skw) (ch/nothing? anc))
[pos rot scl skw anc])))
(defn- visible?
"Is the node switched on this frame?
`[:vis]` IS A BOOLEAN, and this insists on it rather than testing truthiness,
because the two obvious implementations are both wrong about a DENSE `[:vis]`.
A dense block yields 0 or 1, and 0 is TRUTHY in CLJS — so `(if v …)` shows a
hidden frame, and `(true? v)` hides every frame. Neither reads as an error.
docs/animation-model.md's parts table says `:mouth-in` carries `[:vis]` dense;
flow/freeze writes it KEYED, because a threshold crossing is a handful of
transitions and hold is the default, and because a human has to be able to fix
one frame of it. When something does want a dense one it will land here loudly
instead of blanking the timeline.
Absence is not a boolean and is not an error: a subject that is not on the
frame has nothing to show."
[id v]
(cond
(true? v) true
(false? v) false
(ch/nothing? v) false
:else (throw (ex-info "[:vis] must sample to a boolean"
{:node id :value v}))))
(defn- place
"Where a node sits this frame, as {:m world :f local-frame :rd reader}, or nil
when it is not on the frame at all.
Three gates, and nil from any of them removes the node's DESCENDANTS too,
which is why this is one answer rather than three flags: a node outside its
span does not exist, a switched-off feature takes its parts with it, and a node
with no transform gives its children nowhere to be.
A missing [:geom :pts] is deliberately NOT one of them — that is `emit`'s
business. An absent mouth outline has nothing to draw, but the head it hangs
off is still exactly where it was, and that asymmetry is the whole reason
presence is tracked per channel rather than per node."
[{:keys [read mat-for pinv-for scratch]} n parent f]
(let [pf (if parent (:f parent) f)]
(when (in-span? n pf)
(let [id (:id n)
chs (node/channels n)
lf (node/local-frame n pf)
rd (fn [path] (read id path (get chs path) lf))]
(when (visible? id (rd [:vis]))
(when-let [[pos rot scl skw anc] (xform-at rd)]
;; dest aliases `local` here, which mul! allows: it reads both
;; operands fully before writing either.
(let [m (node/local! (mat-for id) pos rot scl skw anc)]
{:m (node/world! m (:m parent) (pinv-for id) m scratch)
:f lf
:rd rd})))))))
(defn- emit
"Emit geometry in the timeline's space. Rect sizes stay fractional until
rasterization, so enclosing symbol transforms can still scale them."
[{:keys [palette buf-for]} n {:keys [m rd]} base]
(let [colour #(colour-index palette (rd [:style :color]))]
(case (:kind n)
:group nil
:symbol nil
:audio nil
:poly
(let [pts (rd [:geom :pts])]
(when-not (ch/nothing? pts)
(let [np (n-points pts)
out (buf-for (:id n) np)]
(dotimes [k np]
(node/apply-pt! out k m
(ch/component pts (* 2 k))
(ch/component pts (inc (* 2 k)))))
(assoc base :kind :poly :pts out :n np :color (colour)))))
:disc
(let [rad (rd [:geom :radius])]
(when-not (ch/nothing? rad)
(assoc base :kind :disc
:cx (aget m 4) :cy (aget m 5)
:r (* rad (node/mean-scale m))
:color (colour))))
:rect
(let [size (rd [:geom :size])]
(when-not (ch/nothing? size)
(assoc base :kind :rect
:cx (aget m 4) :cy (aget m 5)
:size (* size (node/mean-scale m))
:color (colour))))
(throw (ex-info "node kind is not implemented"
{:node (:id n) :kind (:kind n)})))))
(defn- nodes-of
"The timeline's node map, REFUSING a map that has none.
A clip and a timeline both have an `:id` and both are maps, so handing a CLIP to
an evaluator is the one mistake this type split makes easy — and the result is
not an error, it is `(:nodes clip)` being nil and a frame resolving to no ops at
all. That reads as a black stage, or, in a benchmark, as \"0 nodes\" and a
flattering number. It happened once while the split was being made, which is why
this is a guard and not a comment."
[tl]
(let [nodes (:nodes tl)]
(when-not (map? nodes)
(throw (ex-info (str "not a timeline: :nodes is " (pr-str nodes)
" — a clip is not a timeline, its `:timelines` hold them")
{:keys (vec (sort-by str (keys tl)))})))
nodes))
(defn- channel-frame
"Anchors select measured frames; marked channels read instance pose choices."
[choices anchors nodes source-fps picture-fps id c lf]
(cond
(contains? anchors id)
(pose/held-frame (get anchors id) lf lf)
(:pose-sampled? c)
(pose/source-frame choices
(if (contains? choices [:node id])
[:node id]
(or (:pose-group (get nodes id)) id))
lf
(node/sample-frame lf source-fps picture-fps))
:else lf))
(defn- prepared-anchors [nodes]
(into {}
(for [[id n] nodes :when (seq (:anchors n))]
[id (vec (sort-by first (:anchors n)))])))
(defn- eval-into
"One frame, as a fold over the nodes in topological order.
`ctx` carries how a channel is read and where its points are written:
:read (fn [id path channel local-frame] -> v)
:palette tone -> index, the ramp in scope
:mat-for (fn [id] -> Float64Array) the node's world transform
:pinv-for (fn [id] -> Float64Array|nil) its parent-inverse
:buf-for (fn [id n-points] -> Float64Array)
:scratch one spare 6-element matrix"
[ctx nodes ord rank f]
(-> (reduce
(fn [{:keys [placed ops] :as acc} id]
(let [n (get nodes id)
pid (:parent n)
parent (when pid (get placed pid))]
;; A node whose parent was dropped is dropped with it, and so is
;; everything under it. Topological order is what makes that one
;; lookup instead of a subtree walk.
(if (and pid (nil? parent))
acc
(if-let [p (place ctx n parent f)]
(let [_ (when-let [on-place (:on-place ctx)] (on-place id p))
op (emit ctx n p {:node id :stencil (:stencil n)})]
(cond-> (update acc :placed assoc id p)
op (update :ops conj op)))
acc))))
{:placed {} :ops []}
ord)
:ops
(->> (finish rank))))
;; ---------------------------------------------------------------------------
;; the specification
(defn eval-frame
"Timeline at frame f -> draw ops in z order. Pure, and allocates freely.
`f` is in THIS timeline's frame space. At the clip's root that is clip frames;
inside an instance it is the instance's own space, and the instance boundary is
the only place the space changes.
This is the definition of what a frame means. `resolver` is what plays it."
([tl f] (eval-frame tl f nil pal/index-of))
([tl f store] (eval-frame tl f store pal/index-of))
([tl f store palette] (eval-frame tl f store palette nil nil))
([tl f store palette pose-tracks opts]
(let [nodes (nodes-of tl)
choices (pose/prepare pose-tracks)
anchors (prepared-anchors nodes)
{:keys [source-fps picture-fps]} opts
ord (order nodes)]
(eval-into {:read (fn [id path c lf]
(ch/value-at c (channel-frame choices anchors nodes
source-fps picture-fps id c lf)
store))
:palette palette
:mat-for (fn [_id] (node/mat))
:pinv-for (fn [id] (node/pinv (get nodes id)))
:buf-for (fn [_id n] (js/Float64Array. (* 2 n)))
:scratch (node/mat)}
nodes ord (draw-rank nodes ord) f))))
;; ---------------------------------------------------------------------------
;; the playback path
(defn- point-capacity
"How many points the widest value of a [:geom :pts] channel holds.
FIXED TOPOLOGY is what makes this a number at all: every key of a part carries
the same vertex count with the same vertex meanings, so the buffer can be
allocated once. A variable vertex count would force a per-frame offset table
and a scan, which is why the aesthetic constraint is a performance asset rather
than a cost."
[c]
(quot (cond
(:dense c) (:stride (:dense c))
(:animated? c) (transduce (map #(if (vector? %) (count %) (.-length %)))
max 0 (vals (:keys c)))
:else (let [v (:value c)] (if (vector? v) (count v) (.-length v))))
2))
(defprotocol IResolver
(world-of [this id]
"The node's world transform AS OF THE LAST FRAME RESOLVED, or nil if it was
not placed on that frame.
The matrices are the ones evaluation mutates in place, so this is a read of
live state rather than a snapshot — which is exactly what the caller wants.
A registered photo underlay has to ride the same transform the vectors went
through or it is merely decorative, and it paints immediately after the frame
it belongs to, so \"as of the last frame\" is the only answer that can be
correct.")
(frame-of [this id] "The placed node's local frame on the last resolve."))
(defn resolver
"(fn [f] -> ops). Holds everything that does not change per frame.
The point buffers are REUSED between frames, so a caller must consume the ops
before asking for the next frame. That is the contract the rAF loop wants
anyway — it reads, blits, and dispatches nothing — and it is what makes a frame
cost a lookup and a blit rather than an allocation per vertex.
The op maps themselves are allocated fresh, and deliberately: there are a dozen
of them per frame against hundreds of points, so pooling them would buy
nothing and cost the ability to hand an op list around as plain data."
([tl] (resolver tl nil pal/index-of nil nil))
([tl store] (resolver tl store pal/index-of nil nil))
([tl store palette] (resolver tl store palette nil nil))
([tl store palette pose-tracks] (resolver tl store palette pose-tracks nil))
([tl store palette pose-tracks {:keys [source-fps picture-fps]}]
(let [nodes (nodes-of tl)
choices (pose/prepare pose-tracks)
anchors (prepared-anchors nodes)
ord (order nodes)
rank (draw-rank nodes ord)
cursors (into {}
(map (fn [id]
[id (into {} (map (fn [[p c]] [p (ch/cursor c store)]))
(node/channels (get nodes id)))]))
ord)
mats (into {} (map (fn [id] [id (node/mat)])) ord)
pinvs (into {} (keep (fn [id] (when-let [p (node/pinv (get nodes id))] [id p]))) ord)
bufs (into {}
(keep (fn [id]
(when-let [c (get-in nodes [id :channels [:geom :pts]])]
[id (js/Float64Array. (* 2 (point-capacity c)))])))
ord)
scratch (node/mat)
;; Every call to mat-for is a placement: eval-into reaches it only after
;; the span, visibility and transform gates have all passed. So wrapping
;; it is how the resolver learns which nodes exist this frame without
;; eval-into having to report it — and it covers groups, which are
;; placed but emit no op, and which are exactly what an underlay rides.
placed (volatile! {})
ctx {:read (fn [id path c lf]
(ch/sample! (get-in cursors [id path])
(channel-frame choices anchors nodes
source-fps picture-fps id c lf)))
:palette palette
:mat-for (fn [id] (get mats id))
:on-place (fn [id p] (vswap! placed assoc id p))
:pinv-for (fn [id] (get pinvs id))
:buf-for (fn [id _n] (get bufs id))
:scratch scratch}
step (fn [f]
(vreset! placed {})
(eval-into ctx nodes ord rank f))]
(reify
IFn
(-invoke [_ f] (step f))
IResolver
(world-of [_ id] (:m (get @placed id)))
(frame-of [_ id] (:f (get @placed id)))))))
;; ---------------------------------------------------------------------------
(def timeline-keys
"Every field a timeline may carry, and the reason `arthur.domain.leaf` refuses
one it does not know: a field added without a leaf to save it in is a field that
saves silently and comes back missing.
`:palette` is in the vocabulary and nothing writes one yet. A timeline is where
a ramp belongs — `domain/timeline` takes the palette as a PARAMETER rather than
reaching for a global precisely so that a nested timeline can carry its own —
and leaving the field out would make the first one a migration instead of a
write."
#{:id :frames :nodes :palette})
(defn problems
"Human-readable reasons this timeline will not evaluate. Empty means it will.
Node structure only. The tracking identities — subjects, features, groups — are
the CLIP's and are checked by `arthur.domain.clip/problems`, which is not a
layering nicety: a feature names nodes, and a library symbol's nodes are not
the ones a face was tracked into.
Total by construction — it reports a cycle rather than looping on one — because
its whole job is to be safe to run over authored data before that data is
trusted."
[tl]
(let [nodes (:nodes tl)]
(if-not (map? nodes)
[":nodes must be a map of id -> node"]
(-> []
(into (for [[id n] nodes
:when (not= id (:id n))]
(str "node under key " (pr-str id) " has :id " (pr-str (:id n)))))
(into (for [[id n] nodes
:when (and (:parent n) (not (contains? nodes (:parent n))))]
(str "node " (pr-str id) " has :parent " (pr-str (:parent n))
" which is not in the timeline")))
(into (for [[id n] nodes
:when (and (:stencil n) (not (contains? nodes (:stencil n))))]
(str "node " (pr-str id) " has :stencil " (pr-str (:stencil n))
" which is not in the timeline")))
(into (for [[id n] nodes
p (node/problems n)]
(str "node " (pr-str id) ": " p)))
;; Anchors re-address the node's measurement, regardless of its name.
(into (for [[id n] nodes
:let [anchors (:anchors n)]
:when (some? anchors)
:when (not (and (map? anchors) (contains? anchors 0)
(integer? (:frames tl))
(every? #(and (integer? %) (<= 0 %)
(< % (:frames tl)))
(concat (keys anchors) (vals anchors)))
(seq (:measured n))
(= (:channels n) (:measured n))))]
(str "node " (pr-str id)
": :anchors must start at frame 0, name valid measured frames, and read that node's own measured channels")))
(into (for [k (remove timeline-keys (keys tl))]
(str "timeline has a field with no leaf to save it in: " (pr-str k))))
(into (when-not (or (nil? (:frames tl)) (and (integer? (:frames tl)) (pos? (:frames tl))))
[(str ":frames is " (pr-str (:frames tl))
" — a timeline is a frame SPACE, so its length is a positive integer")]))
(into (try
(doall (map #(depth nodes %) (keys nodes)))
nil
(catch :default e [(ex-message e)])))))))

View file

@ -0,0 +1,438 @@
(ns arthur.events.collab
"Everything that makes a document somewhere other people are: its address, who
you are, who else is in it, and their writes arriving while you work.
docs/architecture.md, Collaboration, and tl's model with the four additions it
asks for. Writes stay on HTTP; the socket carries presence and the deltas the
server broadcasts after a write commits.
ONE RULE FOR THE ADDRESS AND THE ROOM. They follow `[:project :id]`, whatever
event changed it — open, save, new, a copy — through one interceptor, so no
event that loads a document has to remember to join its room.
THE OUTBOX RULE, without an outbox. A remote leaf lands unless we have a change
to that leaf the server has not seen — a leaf whose local value differs from the
last value we synced. Otherwise their write would snap our unsaved edit back.
The next save sends ours, and if theirs moved since, it answers 409 and we catch
up, and the save after that is ours."
(:require [arthur.domain.leaf :as leaf]
[arthur.domain.project :as project]
[arthur.events.edit :as edit]
[arthur.events.playback :as pb]
[arthur.events.project :as events.project]
[arthur.footage.store :as store]
[arthur.fx.http :as http]
[clojure.string :as str]
[re-frame.core :as rf]))
;; ---------------------------------------------------------------------------
;; the address
(defn- path-id
"The project a path names: `/p/<uuid>/<slug>`. The slug is for people; the
id is what finds it."
[path]
(second (re-matches #"/p/([0-9a-fA-F-]{36})(?:/.*)?" path)))
(defn slug [name]
(or (not-empty (-> (str/lower-case (or name ""))
(str/replace #"[^a-z0-9]+" "-")
(str/replace #"^-+|-+$" "")))
"untitled"))
(defn project-path [id name] (str "/p/" id "/" (slug name)))
(defn- route! []
(rf/dispatch [::routed (path-id (.. js/window -location -pathname))]))
(defn navigate! [path]
(.pushState js/history nil "" path)
(route!))
(rf/reg-event-fx
::routed
;; `/` is the index of your projects; a project is only ever at its address.
(fn [{:keys [db]} [_ id]]
(cond
(nil? id) {:db (assoc db :route :index)
:dispatch [::events.project/list]}
(= id (get-in db [:project :id])) {:db (assoc db :route [:project id])}
:else {:db (assoc db :route [:project id])
:dispatch [::events.project/open id]})))
(rf/reg-sub ::route (fn [db _] (:route db)))
(rf/reg-fx
::create!
(fn [name]
(-> (http/POST "/api/projects" #js {:name name})
(.then (fn [^js made] (navigate! (project-path (.-id made) (.-name made)))))
(.catch #(rf/dispatch [::refused (ex-message %)])))))
(rf/reg-event-fx ::create (fn [_ [_ name]] {::create! (or name "untitled")}))
;; ---------------------------------------------------------------------------
;; the socket
(defonce ^:private socket (atom nil))
(defonce ^:private conn (atom {:id nil :tries 0 :timer nil}))
(defn- ws-url [id]
(str (if (= "https:" (.. js/window -location -protocol)) "wss://" "ws://")
(.. js/window -location -host) "/ws/projects/" id))
(declare open!)
(defn- retry-later! [id]
(let [tries (:tries @conn)
delay (min 30000 (* 500 (js/Math.pow 2 tries)))]
(swap! conn assoc :tries (inc tries)
:timer (js/setTimeout #(when (= id (:id @conn)) (open! id)) delay))))
(defn- open! [id]
(let [s (js/WebSocket. (ws-url id))]
(reset! socket s)
(set! (.-onopen s) (fn [_] (swap! conn assoc :tries 0)))
(set! (.-onmessage s) (fn [e] (rf/dispatch [::message (js/JSON.parse (.-data e))])))
;; Only the CURRENT socket clears the roster and retries: closing the last
;; project's on a switch must not wipe the new one's.
(set! (.-onclose s) (fn [_]
(when (identical? s @socket)
(reset! socket nil)
(rf/dispatch [::peers-reset])
(retry-later! id))))))
(defn- connect! [id]
(some-> (:timer @conn) js/clearTimeout)
(when-let [s @socket] (set! (.-onclose s) nil) (.close s))
(reset! socket nil)
(reset! conn {:id id :tries 0 :timer nil})
(rf/dispatch [::peers-reset])
(when id (open! id)))
(rf/reg-fx
::follow!
(fn [{:keys [id name]}]
(when id
(let [here (.. js/window -location -pathname)
path (project-path id name)]
(cond
(= path here) nil
;; Renamed: the same page, a new slug, and no new history entry.
(= id (path-id here)) (.replaceState js/history nil "" path)
:else (.pushState js/history nil "" path))))
(set! (.-title js/document) (if name (str name " — arthur") "arthur"))
(when (not= id (:id @conn))
(connect! id))))
(rf/reg-fx ::reconnect! (fn [_] (connect! (:id @conn))))
(def autosave?
"Every edit saves. Off only for tests that need an edit held unsaved."
true)
(def ^:private follow
"The address, the title and the room follow the open project, and the
document saves itself on every edit — `:paint/revision` is what moves when
the document does. A save with nothing to send sends nothing, and one made
while another is in flight goes when it lands.
THERE IS NO BARE PROJECT. A document with no id on screen at a project's
address — a built-in example, opened from the menu — is saved at once, and
becomes a project with an address of its own."
(rf/->interceptor
:id ::follow
:after (fn [ctx]
(let [db (get-in ctx [:effects :db] (get-in ctx [:coeffects :db]))
before (get-in ctx [:coeffects :db :project])
after (:project db)]
(cond-> ctx
(not= (select-keys before [:id :name]) (select-keys after [:id :name]))
(update-in [:effects :fx] (fnil conj [])
[::follow! (select-keys after [:id :name])])
(and autosave? (:id after) (vector? (:route db))
(not= (:paint/revision db) (get-in ctx [:coeffects :db :paint/revision])))
(update-in [:effects :fx] (fnil conj [])
[:dispatch [::events.project/save {:auto? true}]])
(and (nil? (:id after)) (vector? (:route db))
(or (:id before) (not= (:cid before) (:cid after))))
(update-in [:effects :fx] (fnil conj [])
[:dispatch [::events.project/save]]))))))
;; ---------------------------------------------------------------------------
;; presence
(rf/reg-event-db ::peers-reset (fn [db _] (assoc db :peers {})))
(defn- peer [^js m] {:cid (.-cid m) :user (.-user m)})
(rf/reg-event-fx
::message
(fn [{:keys [db]} [_ ^js m]]
(case (.-kind m)
"welcome" {:db (assoc db :peers {} :peer-cid (.-cid m))
;; Anything written between our GET and our joining the room
;; was broadcast to a room we were not in yet.
:dispatch [::catch-up]}
"roster" {:db (update db :peers into (map (fn [^js p] [(.-cid p) (peer p)]))
(array-seq (.-peers m)))}
("join" "state") {:db (assoc-in db [:peers (.-cid m)] (peer m))}
"leave" {:db (update db :peers dissoc (.-cid m))}
"delta" {:dispatch [::delta m]}
"access" {:dispatch [::catch-up]}
{})))
(rf/reg-sub
::peers
(fn [db _]
(->> (vals (:peers db))
(remove #(= (:cid %) (:peer-cid db)))
(sort-by (juxt (comp nil? :user) :user)))))
;; ---------------------------------------------------------------------------
;; their writes
(defn- put [m path v] (if (nil? v) (dissoc m path) (assoc m path v)))
(defn- landed
"Their change laid over ours, as `[local synced behind]`; a nil value is a
removal.
A leaf we have changed and not saved keeps our value, and theirs waits in
`behind` rather than in `synced`: `synced` is what we have SEEN, and putting
theirs there would let our next save overwrite it without a word. `take?` is
the first write winning — theirs was, so it goes on screen over ours."
[local synced behind theirs take?]
(let [pending? #(not= (get local %) (get synced %))]
(reduce-kv (fn [[now seen behind] path v]
(cond
(not (pending? path)) [(put now path v) (put seen path v) behind]
take? [(put now path v) (put seen path v) (dissoc behind path)]
:else [now seen (assoc behind path v)]))
[local synced behind] theirs)))
(rf/reg-fx
::fetch-blocks!
(fn [{:keys [keys then]}]
(-> (js/Promise.all (into-array (map #(http/GET (str "/api/blocks/" %)) keys)))
(.then #(rf/dispatch (conj then (project/store %))))
(.catch #(rf/dispatch [::events.project/failed (ex-message %)])))))
(rf/reg-event-fx
::remote
;; `written` and `removed` against what we last synced; `blocks` is the store
;; of any the new leaves name that we do not hold, once fetched.
(fn [{:keys [db]} [_ {:keys [by written removed take?] at :seq :as change} blocks]]
(let [cid (get-in db [:project :cid])
entry (store/entry (:clip/current db))
local (leaf/leaves cid (:clip entry))
theirs (merge written (zipmap removed (repeat nil)))
lost (if take?
(count (filter #(not= (get local %) (get (:synced entry) %)) (keys theirs)))
0)
[now synced behind] (landed local (:synced entry) (:behind entry) theirs take?)
have (merge (:store entry) blocks)
lack (remove #(contains? have %) (project/block-keys now))
status (fn [now synced]
(if (pos? lost)
(str lost (if (= 1 lost) " change" " changes")
" of yours lost to someone else's at the same moment — in your undo list")
(str (or by "someone") " saved r" at
(when (not= now synced) " · yours unsaved"))))]
(cond
(seq lack)
{::fetch-blocks! {:keys lack :then [::remote change]}}
(= now local)
{:db (-> db
(update :clip/current
#(or (store/edit-entry! % (fn [e] (assoc e :synced synced
:behind behind)))
%))
(assoc-in [:project :seq] at)
;; Our own write, back from the room, changes nothing to say.
(cond-> (or take? (not= by (get-in db [:me :username])))
(assoc-in [:project :status] (status now synced))))}
:else
(let [clip (leaf/clip cid now)
;; Theirs, so not a step of ours to undo.
db' (-> (edit/replace-entry db #(-> %
(assoc :clip clip :synced synced
:behind behind)
(update :store merge blocks)))
(edit/transport clip)
(update :project merge
{:seq at :status (status now synced)}))]
(cond-> {:db db'}
(not= (:fps clip) (get-in db [:clip :fps]))
(assoc ::pb/seek! [(:fps clip) (pb/frames db') (get-in db [:playback :frame])])))))))
(defn- ours
"The clip in a delta or a document that is the one open here."
[db clips]
(let [cid (get-in db [:project :cid])]
(first (filter #(= cid (.-cid ^js %)) (array-seq clips)))))
(rf/reg-event-fx
::delta
(fn [{:keys [db]} [_ ^js m]]
(let [local (get-in db [:project :seq])
seq (.-seq m)]
(cond
(or (nil? local) (<= seq local)) {}
;; A missed delta is a stale document forever, unless it is noticed.
(> seq (inc local)) {:dispatch [::catch-up]}
:else
(let [^js c (ours db (.-clips m))]
(cond-> {:db (cond-> (assoc-in db [:project :seq] seq)
(.-name m) (assoc-in [:project :name] (.-name m)))}
c (assoc :dispatch [::remote {:seq seq :by (.-by m)
:written (project/tier1 (.-leaves c))
:removed (vec (.-removed c))}])))))))
(rf/reg-fx
::catch-up!
(fn [[id take?]]
(-> (http/GET (str "/api/projects/" id))
(.then #(rf/dispatch [::caught-up % take?]))
(.catch #(js/console.warn "catching up failed" %)))))
(rf/reg-event-fx
::catch-up
(fn [{:keys [db]} [_ take?]]
(if-let [id (get-in db [:project :id])]
{::catch-up! [id take?]}
{})))
(rf/reg-event-fx
::caught-up
;; The whole document, diffed against what we last synced: which leaves they
;; wrote, and which they deleted.
(fn [{:keys [db]} [_ ^js loaded take?]]
(let [^js c (ours db (.-clips loaded))
synced (:synced (store/entry (:clip/current db)))
theirs (when c (project/tier1 (.-leaves c)))
access {:owner (.-owner loaded) :editors (vec (.-editors loaded))
:can-edit? (.-can_edit loaded)}]
(cond-> {:db (update db :project merge access)}
(and c (or take? (not= (.-seq loaded) (get-in db [:project :seq]))))
(assoc :dispatch [::remote {:seq (.-seq loaded) :by nil :take? take?
:written (into {} (remove (fn [[p v]] (= v (get synced p))))
theirs)
:removed (remove #(contains? theirs %) (keys synced))}])))))
;; ---------------------------------------------------------------------------
;; who you are, and who else may write
(rf/reg-fx
::request!
(fn [{:keys [method url body then]}]
(-> (http/request! method url body)
(.then #(rf/dispatch (conj then %)))
(.catch #(rf/dispatch [::refused (ex-message %)])))))
(rf/reg-event-fx ::who (fn [_ _] {::request! {:method "GET" :url "/api/me" :then [::signed]}}))
(rf/reg-event-fx
::sign-in
(fn [_ [_ mode username password]]
{::request! {:method "POST" :url (str "/api/" (name mode))
:body #js {:username username :password password}
:then [::signed]}}))
(rf/reg-event-fx
::sign-out
(fn [_ _] {::request! {:method "POST" :url "/api/logout" :then [::signed]}}))
(rf/reg-event-fx
::signed
;; Who you are changes what you may write and what the room calls you.
(fn [{:keys [db]} [_ ^js who]]
(let [username (.-username who)
changed? (not= username (get-in db [:me :username]))]
(cond-> {:db (assoc db :me {:username username})}
(and changed? (contains? db :me)) (assoc ::reconnect! nil
:fx [[:dispatch [::catch-up]]
[:dispatch [::events.project/list]]])))))
(rf/reg-event-db ::refused (fn [db [_ message]] (assoc-in db [:me :error] message)))
(rf/reg-sub ::me (fn [db _] (:me db)))
(rf/reg-event-fx
::add-editor
(fn [{:keys [db]} [_ username]]
{::request! {:method "POST" :url (str "/api/projects/" (get-in db [:project :id]) "/editors")
:body #js {:username username} :then [::editors]}}))
(rf/reg-event-fx
::remove-editor
(fn [{:keys [db]} [_ username]]
{::request! {:method "DELETE"
:url (str "/api/projects/" (get-in db [:project :id]) "/editors/"
(js/encodeURIComponent username))
:then [::editors]}}))
(rf/reg-event-db
::editors
(fn [db [_ ^js answer]]
(-> (assoc-in db [:project :editors] (vec (.-editors answer)))
(update :me dissoc :error))))
;; ---------------------------------------------------------------------------
;; snapshots: named versions, now that every edit saves itself
(defn- snapshots-url [db] (str "/api/projects/" (get-in db [:project :id]) "/revisions"))
(rf/reg-event-fx
::snapshots
(fn [{:keys [db]} _]
{::request! {:method "GET" :url (snapshots-url db) :then [::snapshots-listed]}}))
(rf/reg-event-db
::snapshots-listed
(fn [db [_ ^js answer]]
(assoc db :snapshots
(mapv (fn [^js r] {:id (.-id r) :name (.-summary r) :author (.-author r)
:seq (.-seq r) :created (.-created r)})
(array-seq (.-revisions answer))))))
(rf/reg-sub ::snapshot-list (fn [db _] (:snapshots db)))
(rf/reg-event-fx
::snapshot
(fn [{:keys [db]} [_ name]]
{::request! {:method "POST" :url (snapshots-url db) :body #js {:summary name}
:then [::snapshotted name]}}))
(rf/reg-event-fx
::snapshotted
(fn [{:keys [db]} [_ name _]]
{:db (assoc-in db [:project :status] (str "snapshot \"" name "\" taken"))
:dispatch [::snapshots]}))
(rf/reg-event-fx
::restore
;; An ordinary write on the server, which comes back to every open tab —
;; this one included — as a delta.
(fn [{:keys [db]} [_ {:keys [id name]}]]
{::request! {:method "POST" :url (str (snapshots-url db) "/" id "/restore")
:then [::restored name]}}))
(rf/reg-event-db
::restored
(fn [db [_ name _]] (assoc-in db [:project :status] (str "restored \"" name "\""))))
;; ---------------------------------------------------------------------------
(defn start!
"Follow the open project from now on, show what the address names — the
index, or a project — and answer the back button."
[]
(rf/reg-global-interceptor follow)
(rf/dispatch [::who])
(route!)
(.addEventListener js/window "popstate" route!))

View file

@ -0,0 +1,83 @@
(ns arthur.events.edit
"The one way an event changes the loaded document.
Three things have to happen together and the bug is any one of them being
forgotten: the clip in `footage/store` is edited, the id app-db refers to it by
is updated — `edit-clip!` may INSTALL A COPY, because a built-in clip is a
delayed value that must stay reusable — and `:paint/revision` is bumped so the
layer-3 subs downstream of `::render/clip` recompute. The revision exists
because the clip itself is behind a handle: app-db holds an id, the id does not
change when the document does, and a sub keyed only on the id would never see
the edit.
It started life private inside `events/paint`, which was right while polygons
were the only thing anyone could edit. They are not.
It is also where UNDO is recorded, for the same reason: being the one way a
person changes the document, it is the one place that sees every change they
make — and nothing else. A collaborator's write and an undo itself go through
`replace-entry`, which is this without the recording."
(:require [arthur.domain.history :as history]
[arthur.domain.leaf :as leaf]
[arthur.footage.store :as store]))
(defn leaves
"The clip as leaves, which is what a history step is made of; nil for a clip
that has no leaf form."
[clip]
(try (leaf/leaves "u" clip) (catch :default _ nil)))
(defn- recorded [f]
(fn [entry]
(let [after (f entry)
b (when-not (identical? (:clip entry) (:clip after)) (leaves (:clip entry)))
a (when b (leaves (:clip after)))]
(cond-> after
a (assoc :history (history/record (:history entry) b a (js/Date.now)))))))
(defn replace-entry
"Apply `f` to the loaded entry without recording it as a step of yours."
[db f]
(let [id (store/edit-entry! (:clip/current db) f)]
(if id
(-> db
(assoc :clip/current id)
(update :paint/revision (fnil inc 0))
(update :project merge {:status "edited · unsaved"}))
db)))
(defn edit-entry
"Apply `f` to the loaded ENTRY — the document and the blocks, footage and
source tracks beside it — and return the new db. For an edit that brings tier-2
data in with it, which a document edit alone cannot."
[db f]
(replace-entry db (recorded f)))
(defn transport
"App-db's copy of what the transport reads off the clip, after the clip was
replaced under it — as `::events.project/project-setting` writes it."
[db clip]
(cond-> (update db :clip merge (select-keys clip [:width :height]))
(not= (:fps clip) (get-in db [:clip :fps]))
(update :clip merge {:fps (:fps clip)})))
(defn history
"Apply `f` to the loaded entry's undo history, which is not an edit: nothing
is redrawn and nothing becomes unsaved."
[db f]
(if-let [id (store/edit-entry! (:clip/current db) #(update % :history f))]
(assoc db :clip/current id)
db))
(defn edit
"Apply `f` to the loaded clip and return the new db."
[db f]
(edit-entry db #(update % :clip f)))
(defn transaction
"One command is one undo step, independent of neighboring edits or timing."
[db f]
(-> db
(history history/hold)
(edit f)
(history history/settle)))

View file

@ -2,7 +2,7 @@
"Export, as intents and one effect. "Export, as intents and one effect.
The walk is not an event and must not become one: it is a promise chain that The walk is not an event and must not become one: it is a promise chain that
runs for as long as the timeline is long, and re-frame events are the wrong unit runs for as long as the symbol is long, and re-frame events are the wrong unit
for something with a middle. So `::start` collects what the render needs out of for something with a middle. So `::start` collects what the render needs out of
the db and hands it to an fx, and the fx dispatches progress back — the same the db and hands it to an fx, and the fx dispatches progress back — the same
arrangement `events/project`'s save uses, and for the same reason. arrangement `events/project`'s save uses, and for the same reason.
@ -10,7 +10,8 @@
WHAT GOES IN THE DB IS THE REQUEST AND THE PROGRESS, never the frames. A WHAT GOES IN THE DB IS THE REQUEST AND THE PROGRESS, never the frames. A
megabyte of PNG in app-db would be compared by every mounted subscription on megabyte of PNG in app-db would be compared by every mounted subscription on
every tick." every tick."
(:require [arthur.domain.palette :as pal] (:require [arthur.domain.clip :as clip]
[arthur.domain.palette :as pal]
[arthur.export :as export] [arthur.export :as export]
[arthur.export.frames :as frames] [arthur.export.frames :as frames]
[arthur.footage.store :as store] [arthur.footage.store :as store]
@ -44,84 +45,87 @@
(defn target-value (defn target-value
"An export target as a `<select>` option value. "An export target as a `<select>` option value.
Two kinds, told apart by a leading letter: `t:<timeline>` is a whole timeline, Two kinds, told apart by a leading letter: `s:<symbol>` is a whole symbol,
`n:<timeline>:<node>` is one placement inside one. The parts are joined with `:` `n:<symbol>:<node>` is one instance inside one. The parts are joined with `:`
because neither a timeline id nor a uuid contains one. because neither a symbol id nor a uuid contains one.
IT CARRIES THE NAMESPACE. `(name :sym/face-8625)` is \"face-8625\", and a value IT CARRIES THE NAMESPACE. `(name :sym/face-8625)` is \"face-8625\", and a value
written that way cannot be read back: `keyword` on it gives `:face-8625`, which written that way cannot be read back: `keyword` on it gives `:face-8625`, which
is not a key in `:timelines`, so the plan silently becomes nil and the export is not a key in `:symbols`, so the plan silently becomes nil and the export
throws \"there is no such timeline\" from inside re-frame's `:do-fx`. That throws \"there is no such symbol\" from inside re-frame's `:do-fx`. That
presented as the tab locking up rather than as an error — see `::run!` below for presented as the tab locking up rather than as an error — see `::run!` below for
the other half of why — and it is the reason this is a named pair of functions the other half of why — and it is the reason this is a named pair of functions
with a test rather than `name` and `keyword` at the two ends of a select." with a test rather than `name` and `keyword` at the two ends of a select."
[{:keys [timeline isolate]}] [{sid :symbol isolate :isolate}]
(let [tl (subs (str (or timeline :main)) 1)] (let [s (subs (str sid) 1)]
(if isolate (str "n:" tl ":" isolate) (str "t:" tl)))) (if isolate (str "n:" s ":" isolate) (str "s:" s))))
(defn target-id (defn target-id
"The inverse of `target-value`. `keyword` splits on the `/` itself, so a "The inverse of `target-value`. `keyword` splits on the `/` itself, so a
namespaced timeline id survives; a placement comes back a uuid, which is what namespaced symbol id survives; an instance comes back a uuid, which is what
the node map is keyed by." the node map is keyed by."
[v] [v]
(let [[kind tl node] (str/split v #":")] (let [[kind s node] (str/split v #":")]
(cond-> {:timeline (keyword tl)} (cond-> {:symbol (keyword s)}
(= "n" kind) (assoc :isolate (uuid node))))) (= "n" kind) (assoc :isolate (uuid node)))))
(defn targets (defn targets
"Everything an export can be pointed at, in the order the picker lists them. "Everything an export can be pointed at, in the order the picker lists them.
THREE KINDS, and the distinction is the point. `:main` is the clip. A symbol TWO KINDS, and the distinction is the point. A symbol is the DRAWING — one file
timeline is the DRAWING — one file however many times it is placed, in its own however many times it is placed, in its own frame space. An instance is that
frame space. A placement is that drawing WHERE IT SITS: the stage's length and drawing WHERE IT SITS in the open symbol: that symbol's length and rate, with
rate, with the other placements removed, which is why seven instances of one the other instances removed, which is why seven instances of one symbol are
symbol are seven different exports rather than seven copies of one. seven different exports rather than seven copies of one.
Placements are ordered and labelled by `:name`, never by id: a uuid sorts at Instances are ordered and labelled by `clip/node-label`, never by id: a uuid
random and means nothing to read." sorts at random and means nothing to read."
[clip] [clip open]
(let [libs (cons :main (sort-by str (remove #{:main} (keys (:timelines clip))))) (let [label #(clip/node-label clip %1 %2)
placements (->> (get-in clip [:timelines :main :nodes]) instances (->> (get-in clip [:symbols open :nodes])
(filter (comp #{:symbol} :kind val)) (filter (comp #{:instance} :kind val))
(sort-by (fn [[id n]] [(or (:name n) "") (str id)])))] (sort-by (fn [[id n]] [(label id n) (str id)])))]
(into (mapv (fn [tid] (into (mapv (fn [sid] {:symbol sid :label (name sid)})
{:timeline tid (sort-by str (keys (:symbols clip))))
:label (if (= :main tid) "main (the clip)" (name tid))})
libs)
(mapv (fn [[id n]] (mapv (fn [[id n]]
{:timeline :main :isolate id {:symbol open :isolate id :label (label id n)})
:label (or (:name n) (str id))}) instances))))
placements))))
(defn target
"What the export is pointed at. A nil symbol is whichever one is open, so a
new document exports what is on screen without anyone choosing."
[db]
(let [{sid :symbol isolate :isolate} (:export db)]
{:symbol (or sid (get-in db [:ui :open])) :isolate isolate}))
(defn- label-of (defn- label-of
"The label of the target `db` currently points at, for the filename." "The label of the target `db` currently points at, for the filename."
[clip {:keys [timeline isolate]}] [clip open {sid :symbol isolate :isolate}]
(:label (or (first (filter #(and (= timeline (:timeline %)) (:label (or (first (filter #(and (= sid (:symbol %)) (= isolate (:isolate %)))
(= isolate (:isolate %))) (targets clip open)))
(targets clip))) {:label (some-> sid name)})))
{:label (some-> timeline name)})))
(rf/reg-sub ::state (fn [db _] (:export db))) (rf/reg-sub ::state (fn [db _] (assoc (:export db) :target (target db))))
(rf/reg-sub (rf/reg-sub
::targets ::targets
(fn [db _] (fn [db _]
(targets (:clip (store/entry (:clip/current db)))))) (targets (:clip (store/entry (:clip/current db))) (get-in db [:ui :open]))))
(rf/reg-sub (rf/reg-sub
::plan ::plan
(fn [db _] (fn [db _]
(let [{:keys [clip]} (store/entry (:clip/current db)) (let [{:keys [clip]} (store/entry (:clip/current db))
{:keys [timeline zoom isolate]} (:export db)] {sid :symbol isolate :isolate} (target db)]
(export/plan {:clip clip :timeline timeline :zoom zoom :isolate isolate (export/plan {:clip clip :symbol sid :zoom (get-in db [:export :zoom])
:picture-fps (get-in db [:clip :display-fps])})))) :isolate isolate}))))
(rf/reg-event-db (rf/reg-event-db
::set-target ::set-target
;; Both keys always, so switching from a placement back to a whole timeline ;; Both keys always, so switching from an instance back to a whole symbol
;; clears the isolate rather than leaving it to filter the new target. ;; clears the isolate rather than leaving it to filter the new target.
(fn [db [_ {:keys [timeline isolate]}]] (fn [db [_ {sid :symbol isolate :isolate}]]
(update db :export merge {:timeline (or timeline :main) :isolate isolate}))) (update db :export merge {:symbol sid :isolate isolate})))
(rf/reg-event-db (rf/reg-event-db
::set-zoom ::set-zoom
@ -134,30 +138,29 @@
{} {}
(let [id (:clip/current db) (let [id (:clip/current db)
entry (store/entry id) entry (store/entry id)
{:keys [timeline zoom isolate]} (:export db)] {sid :symbol isolate :isolate} (target db)
zoom (get-in db [:export :zoom])]
{:db (update db :export merge {:busy? true :done 0 {:db (update db :export merge {:busy? true :done 0
:total (:frames (export/plan :total (:frames (export/plan
{:clip (:clip entry) {:clip (:clip entry)
:timeline timeline :symbol sid
:isolate isolate :isolate isolate
:zoom zoom})) :zoom zoom}))
:status "rendering…"}) :status "rendering…"})
::run! {:clip (:clip entry) ::run! {:clip (:clip entry)
:timeline timeline :symbol sid
:isolate isolate :isolate isolate
:store (:store entry) :store (:store entry)
;; The same palette and ramp the preview resolves and blits ;; The same palette and ramp the preview resolves and blits
;; through. Read here rather than in the fx so that the effect ;; through. Read here rather than in the fx so that the effect
;; takes data and nothing else. ;; takes data and nothing else.
:palette (get {:arthur/default pal/index-of} :palette (pal/compile (:clip entry))
(:palette db) pal/index-of) :ramp (:ramp (pal/compile (:clip entry)))
:ramp (get {:arthur/default pal/rgb} (:palette db) pal/rgb)
:zoom zoom :zoom zoom
:picture-fps (get-in db [:clip :display-fps])
:audio-url (:audio entry) :audio-url (:audio entry)
:name (stem (:label entry) :name (stem (:label entry)
(label-of (:clip entry) (label-of (:clip entry) (get-in db [:ui :open])
{:timeline timeline :isolate isolate}))}})))) {:symbol sid :isolate isolate}))}}))))
(rf/reg-event-db (rf/reg-event-db
::progress ::progress
@ -197,7 +200,7 @@
::run! ::run!
(fn [spec] (fn [spec]
;; THE CALL IS GUARDED because `export/run!` validates its request BEFORE it ;; THE CALL IS GUARDED because `export/run!` validates its request BEFORE it
;; returns a promise, so a bad timeline id throws synchronously — here, inside ;; returns a promise, so a bad symbol id throws synchronously — here, inside
;; re-frame's `:do-fx` interceptor. An uncaught throw there never reaches the ;; re-frame's `:do-fx` interceptor. An uncaught throw there never reaches the
;; `.catch` below, so `::failed` never dispatches and `:busy?` stays true: the ;; `.catch` below, so `::failed` never dispatches and `:busy?` stays true: the
;; button sits disabled on \"rendering…\" and the readout on \"frame 0 /\" ;; button sits disabled on \"rendering…\" and the readout on \"frame 0 /\"

View file

@ -4,7 +4,11 @@
The frames come from the server by URL since step 9 — see `flow/ingest` — and the The frames come from the server by URL since step 9 — see `flow/ingest` — and the
detector's identity comes from the server too, because it goes into the content detector's identity comes from the server too, because it goes into the content
address of every block this produces." address of every block this produces."
(:require [arthur.domain.clip :as clip] (:require [arthur.domain.bring :as bring]
[arthur.domain.clip :as clip]
[arthur.domain.span :as span]
[arthur.events.edit :as edit]
[arthur.events.ui :as ui]
[arthur.events.playback :as pb] [arthur.events.playback :as pb]
[arthur.flow.detect :as detect] [arthur.flow.detect :as detect]
[arthur.flow.ingest :as ingest] [arthur.flow.ingest :as ingest]
@ -14,6 +18,7 @@
[arthur.footage.store :as store] [arthur.footage.store :as store]
[arthur.domain.landmarks :as lm] [arthur.domain.landmarks :as lm]
[arthur.fx.http :as http] [arthur.fx.http :as http]
[clojure.string :as string]
[re-frame.core :as rf])) [re-frame.core :as rf]))
(defonce ^:private clock (atom 0)) (defonce ^:private clock (atom 0))
@ -44,8 +49,12 @@
The work happens inside `decode!`'s callback, and the promise it returns is the The work happens inside `decode!`'s callback, and the promise it returns is the
backpressure: the decoder does not run ahead of the detector, so a 900-frame backpressure: the decoder does not run ahead of the detector, so a 900-frame
take does not hold 900 decoded frames at 1440x1920 in memory." take does not hold 900 decoded frames at 1440x1920 in memory.
[manifest model]
Only source frames `[start end)` are measured. Decoding still begins at frame
0, because every frame after the first is coded against the ones before it,
and it stops at `end`; frames before `start` are decoded and dropped unread."
[manifest model [start end]]
(let [[w h] [(:width manifest) (:height manifest)] (let [[w h] [(:width manifest) (:height manifest)]
canvas (.createElement js/document "canvas") canvas (.createElement js/document "canvas")
ctx (.getContext canvas "2d" #js {:willReadFrequently true}) ctx (.getContext canvas "2d" #js {:willReadFrequently true})
@ -53,22 +62,23 @@
raw (atom []) raw (atom [])
crops (atom []) crops (atom [])
inner (atom []) inner (atom [])
total (:frames manifest)] total end]
(set! (.-width canvas) w) (set! (.-width canvas) w)
(set! (.-height canvas) h) (set! (.-height canvas) h)
(rf/dispatch [::progress "loading the video…"]) (rf/dispatch [::progress "loading the video…"])
(-> (ingest/stream! (ingest/stream-url manifest) total) (-> (ingest/stream! (ingest/stream-url manifest) (:frames manifest))
(.then (.then
(fn [stream] (fn [stream]
(ingest/decode! (ingest/decode!
stream fps w h (update stream :units subvec 0 end) fps w h
(fn [i frame] (fn [i frame]
(when (>= i start)
(.drawImage ctx frame 0 0) (.drawImage ctx frame 0 0)
;; EVERY FACE ON THIS FRAME, each with its own mouth crop taken ;; EVERY FACE ON THIS FRAME, each with its own mouth crop taken
;; while the frame's pixels are still on the canvas. Which of these ;; while the frame's pixels are still on the canvas. Which of
;; detections belongs to which subject is not decided here — the ;; these detections belongs to which subject is not decided here
;; answer needs the whole take — so all three vectors stay in ;; — the answer needs the whole take — so all three vectors stay
;; DETECTION ORDER and `detect/tracks` re-keys them afterwards. ;; in DETECTION ORDER and `detect/tracks` re-keys them afterwards.
(let [faces (detect/detect! model canvas (ingest/frame-ms fps i)) (let [faces (detect/detect! model canvas (ingest/frame-ms fps i))
boxes (mapv (fn [face] boxes (mapv (fn [face]
(interior/crop (mapv #(nth face %) lm/LIPS-INNER) (interior/crop (mapv #(nth face %) lm/LIPS-INNER)
@ -89,9 +99,11 @@
;; frame counter frozen on its last value — which reads as the ;; frame counter frozen on its last value — which reads as the
;; decoder hanging, and was diagnosed as that twice. ;; decoder hanging, and was diagnosed as that twice.
(swap! inner conj (mapv #(source/measure-crop take/knobs %) (swap! inner conj (mapv #(source/measure-crop take/knobs %)
frame-crops))) frame-crops))))
(when (or (zero? i) (zero? (mod (inc i) 4)) (= (inc i) total)) (when (or (zero? i) (zero? (mod (inc i) 4)) (= (inc i) total))
(rf/dispatch [::progress (str "detecting " (inc i) "/" total)])) (rf/dispatch [::progress (if (< i start)
(str "seeking " (inc i) "/" start)
(str "detecting " (- (inc i) start) "/" (- end start)))]))
;; Yield, so the status and the transport paint between synchronous ;; Yield, so the status and the transport paint between synchronous
;; MediaPipe calls. `decode!` waits on this before feeding more. ;; MediaPipe calls. `decode!` waits on this before feeding more.
(js/Promise. (fn [done] (js/setTimeout done 0))))))) (js/Promise. (fn [done] (js/setTimeout done 0)))))))
@ -139,11 +151,11 @@
:detector detector}) :detector detector})
_ (mark! "build-clip: freeze") _ (mark! "build-clip: freeze")
built (:clip frozen) built (:clip frozen)
source-blocks (source/pack-subjects (:id (:analysis built)) subjects) analysis-id (-> built :analyses keys first)
source-blocks (source/pack-subjects analysis-id subjects)
_ (mark! "build-clip: pack source blocks")] _ (mark! "build-clip: pack source blocks")]
(assoc (select-keys built [:fps :width :height]) (assoc (select-keys built [:fps :width :height])
:frames (clip/frames built)
:display-fps (:fps built)
:clip built :store (:store frozen) :clip built :store (:store frozen)
:source-blocks source-blocks :source-blocks source-blocks
:source-inputs (assoc source-inputs :subjects with-presence) :source-inputs (assoc source-inputs :subjects with-presence)
@ -252,11 +264,15 @@
(js/Promise.resolve track) (js/Promise.resolve track)
(:subjects track))) (:subjects track)))
(rf/reg-fx (defn- analyse!
::begin! "Promise of source frames `[start end)` of footage `footage-id`, detected,
(fn [footage-id] measured and frozen — `{:clip :store ...}` as `build-clip` makes it, with the
footage reading as though it were only those frames. A saved analysis of
exactly that range is reused instead of detecting again."
[footage-id [start end]]
(-> (js/Promise.all #js [(ingest/manifest! footage-id) (ingest/detector!)]) (-> (js/Promise.all #js [(ingest/manifest! footage-id) (ingest/detector!)])
(.then (fn [[manifest detector]] (.then (fn [[full detector]]
(let [manifest (ingest/slice full start end)]
(reset! clock (js/Date.now)) (reset! clock (js/Date.now))
(rf/dispatch [::progress "looking for saved analysis…"]) (rf/dispatch [::progress "looking for saved analysis…"])
(-> (cached-source! manifest detector) (-> (cached-source! manifest detector)
@ -270,20 +286,22 @@
(= done total)) (= done total))
(rf/dispatch (rf/dispatch
[::progress (str "measuring " done "/" total)])))) [::progress (str "measuring " done "/" total)]))))
(.then (fn [measured] (.then #(build-clip manifest detector %))))
(build-clip manifest detector measured)))))
(do (rf/dispatch [::progress "loading MediaPipe…"]) (do (rf/dispatch [::progress "loading MediaPipe…"])
(-> (detect/landmarker!) (-> (detect/landmarker!)
(.then (fn [model] (.then (fn [model]
(mark! "MediaPipe ready") (mark! "MediaPipe ready")
(rf/dispatch [::progress "opening the video…"]) (rf/dispatch [::progress "opening the video…"])
(-> (detect-frames! manifest model) (detect-frames! full model [start end])))
(.then (fn [fresh] (.then #(build-clip manifest detector %)))))))))))))
(build-clip manifest detector fresh))))))))))))))
(.then (fn [entry] (rf/reg-fx
::convert!
(fn [{:keys [footage-id range] :as request}]
(-> (analyse! footage-id range)
(.then (fn [built]
(mark! "build-clip: done") (mark! "build-clip: done")
(let [id (store/install! entry)] (rf/dispatch [::converted request built])))
(rf/dispatch [::loaded id (:summary entry)]))))
(.catch (fn [error] (.catch (fn [error]
(js/console.error error) (js/console.error error)
;; A run that ended badly may have ended on a MediaPipe graph ;; A run that ended badly may have ended on a MediaPipe graph
@ -295,8 +313,12 @@
(rf/reg-fx (rf/reg-fx
::list! ::list!
(fn [_] (fn [_]
(-> (ingest/available!) (-> (js/Promise.all #js [(ingest/available!)
(.then (fn [footage] (rf/dispatch [::listed footage]))) (.then (http/GET "/api/sounds")
#(:sounds (js->clj % :keywordize-keys true)))
(.then (http/GET "/api/images")
#(:images (js->clj % :keywordize-keys true)))])
(.then (fn [[footage sounds images]] (rf/dispatch [::listed footage sounds images])))
(.catch (fn [error] (.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))])))))) (rf/dispatch [::failed (or (ex-message error) (str error))]))))))
@ -312,12 +334,32 @@
(.catch (fn [error] (.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))]))))) (rf/dispatch [::failed (or (ex-message error) (str error))])))))
(defn- sending
"A progress callback that puts the whole percentage sent in the status line.
ONLY WHEN THE PERCENTAGE MOVES. The browser fires upload progress as often as
it pleases and a status line has something new to say a hundred times at most,
and each dispatch here re-renders the pane.
Worth having at all because the transfer is the longest part of an import on
anything but a local server, and it was the part with no number on it:
`uploading video…` sat unchanged from the first byte to the last, however many
there were, and only the extraction that followed it ever counted. See
`arthur.fx.http/POST-form`."
[label]
(let [reported (atom -1)]
(fn [fraction]
(let [percent (js/Math.round (* 100 fraction))]
(when (not= percent @reported)
(reset! reported percent)
(rf/dispatch [::progress (str label " " percent "%")]))))))
(rf/reg-fx (rf/reg-fx
::upload! ::upload!
(fn [file] (fn [file]
(let [form (js/FormData.)] (let [form (js/FormData.)]
(.append form "file" file) (.append form "file" file)
(-> (http/POST-form "/api/sources" form) (-> (http/POST-form "/api/sources" form (sending "uploading video…"))
(.then (fn [^js source] (.then (fn [^js source]
(rf/dispatch [::progress "queued for extraction…"]) (rf/dispatch [::progress "queued for extraction…"])
(http/POST "/api/extractions" #js {:source (.-id source) (http/POST "/api/extractions" #js {:source (.-id source)
@ -326,19 +368,55 @@
(.catch (fn [error] (.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))) (rf/dispatch [::failed (or (ex-message error) (str error))])))))))
(rf/reg-fx
::upload-sound!
(fn [file]
(let [form (js/FormData.)]
(.append form "file" file)
(-> (http/POST-form "/api/sounds" form (sending "uploading sound…"))
(.then (fn [^js sound] (rf/dispatch [::uploaded (.-id sound) "sound imported"])))
(.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))])))))))
(rf/reg-fx
::upload-image!
(fn [file]
(let [form (js/FormData.)]
(.append form "file" file)
(-> (http/POST-form "/api/images" form (sending "uploading image…"))
(.then (fn [^js image] (rf/dispatch [::uploaded (.-id image) "image imported"])))
(.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))])))))))
(rf/reg-event-fx (rf/reg-event-fx
::upload ::upload
(fn [{:keys [db]} [_ file]] (fn [{:keys [db]} [_ ^js file]]
;; By type, and by name for a browser that leaves the type empty.
(let [kind (when file
(cond
(or (string/starts-with? (.-type file) "audio/")
(re-find #"(?i)\.(mp3|wav|aiff?|flac|ogg|m4a|aac)$" (.-name file)))
:sound
(or (string/starts-with? (.-type file) "image/")
(re-find #"(?i)\.(png|jpe?g|gif|webp)$" (.-name file)))
:image
:else :video))]
(if (or (nil? file) (get-in db [:footage :loading?])) (if (or (nil? file) (get-in db [:footage :loading?]))
{} {}
{:db (update db :footage merge {:loading? true :status "uploading video…"}) {:db (update db :footage merge {:loading? true
::upload! file}))) :status (str "uploading " (name kind) "…")})
(case kind :sound ::upload-sound! :image ::upload-image! ::upload!) file}))))
(rf/reg-event-fx (rf/reg-event-fx
::uploaded ::uploaded
(fn [{:keys [db]} [_ footage-id]] (fn [{:keys [db]} [_ id status]]
{:db (update db :footage merge {:loading? false :chosen footage-id ;; Into the pool, and no further. An upload is media for this project; turning
:status "video extracted — load frames to analyze"}) ;; it into a symbol or a sound is a separate decision — which frames, what
;; name, where — made by dropping it where it should go.
{:db (update db :footage #(-> %
(merge {:loading? false :status (or status "video extracted")})
(cond-> (not status) (assoc :chosen id))
(update :uploaded (fnil conj #{}) id)))
:dispatch [::refresh]})) :dispatch [::refresh]}))
(rf/reg-event-fx (rf/reg-event-fx
@ -347,28 +425,72 @@
(rf/reg-event-db (rf/reg-event-db
::listed ::listed
(fn [db [_ footage]] (fn [db [_ footage sounds images]]
(update db :footage merge (update db :footage merge
(cond-> {:available (vec footage) (cond-> {:available (vec footage)
:sounds (vec sounds)
:images (vec images)
:chosen (or (:chosen (:footage db)) (:id (first footage)))} :chosen (or (:chosen (:footage db)) (:id (first footage)))}
(empty? footage) (assoc :status "upload a video to begin"))))) (empty? footage) (assoc :status "upload a video to begin")))))
(rf/reg-event-db (rf/reg-event-db
::choose ::choose
(fn [db [_ id]] (assoc-in db [:footage :chosen] id))) (fn [db [_ id]]
(-> db
(assoc-in [:footage :chosen] id)
(ui/selected [:footage id]))))
;; ---------------------------------------------------------------------------
;; renaming an asset
;;
;; A SERVER WRITE, NOT A DOCUMENT EDIT, and so not on the undo list. Footage and
;; sounds live beside projects rather than inside one — the pool's ALL ASSETS
;; folder is exactly that — so a label is shared by every project that uses the
;; row, and undoing an edit to this document must not reach out and rename
;; something another one is showing.
;;
;; Written through optimistically. The lists in app-db are what the pool draws
;; from; waiting for the round trip would leave the old name under the cursor for
;; as long as the request takes, and the failure is visible and recoverable —
;; `::failed` says so, and `::refresh` puts back whatever the server actually
;; holds.
(defn- relabelled
"Replace one row's `:label` in a list held by id."
[rows id label]
(mapv #(cond-> % (= id (:id %)) (assoc :label label)) rows))
(rf/reg-fx
::relabel!
(fn [{:keys [url label]}]
(-> (http/PATCH url #js {:label label})
(.then (fn [_]
;; Only a CLEARED label needs the answer. The server's fallback
;; is the name the file was uploaded under, which this client
;; cannot reconstruct — footage falls back to its source and a
;; sound to its filename — so the one case the optimistic write
;; cannot guess is the one case that re-lists.
(when (empty? label) (rf/dispatch [::refresh]))))
(.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))])
(rf/dispatch [::refresh]))))))
(rf/reg-event-fx (rf/reg-event-fx
::load ::relabel
(fn [{:keys [db]} _] (fn [{:keys [db]} [_ kind id value]]
(let [chosen (get-in db [:footage :chosen])] ;; `kind` is `:footage`, `:sound` or `:image`: resources with one field between
(cond ;; them, and one event rather than two that differ by a path and a URL.
(get-in db [:footage :loading?]) {} (let [label (string/trim (str value))
(nil? chosen) [key url] (case kind
{:db (assoc-in db [:footage :status] "upload a video to begin")} :footage [:available (str "/api/footage/" id)]
:else :sound [:sounds (str "/api/sounds/" id)]
{:db (update db :footage merge {:loading? true :status "reading the manifest…"}) :image [:images (str "/api/images/" id)]
::pb/pause! nil [nil nil])]
::begin! chosen})))) (if (or (nil? id) (nil? key))
{}
{:db (cond-> db
(seq label) (update-in [:footage key] relabelled id label))
::relabel! {:url url :label label}}))))
(rf/reg-event-db (rf/reg-event-db
::progress ::progress
@ -380,15 +502,116 @@
(assoc db :footage (assoc (:footage db) (assoc db :footage (assoc (:footage db)
:loading? false :status (str "footage failed: " message))))) :loading? false :status (str "footage failed: " message)))))
;; ---------------------------------------------------------------------------
;; footage -> a symbol
;;
;; Dropping a video asks first. `[:ui :convert]` is the question — which footage,
;; which of its frames, what to call the result — and where the answer will be
;; placed: `:host`, `:frame` and `:point` are the drop's, captured when it happened
;; so that switching tabs while detection runs does not move where it lands.
(rf/reg-event-db
::ask-convert
(fn [db [_ {:keys [frames label] :as footage} frame point target]]
(assoc-in db [:ui :convert]
(merge (select-keys footage [:id :label :frames :fps :video :width :height])
{:range [0 frames]
:name (string/replace (str label) #"\.[^.]*$" "")
:host (get-in db [:ui :open]) :frame frame :point point
:target target}))))
(rf/reg-event-db
::convert-set
(fn [db [_ k v]] (assoc-in db [:ui :convert k] v)))
(rf/reg-event-db
::convert-cancel
(fn [db _]
(if (get-in db [:footage :loading?]) db (update db :ui dissoc :convert))))
(rf/reg-event-fx (rf/reg-event-fx
::loaded ::convert
(fn [{:keys [db]} [_ id summary]] (fn [{:keys [db]} _]
(let [clip (store/entry id)] (let [{:keys [id range] :as request} (get-in db [:ui :convert])]
(if (or (nil? request) (get-in db [:footage :loading?]))
{}
{:db (update db :footage merge {:loading? true :status "starting…"})
::pb/pause! nil
::convert! {:footage-id id :range range :request request}}))))
(rf/reg-event-fx
::convert-tracing
;; The same question answered the other way: the chosen frames become a tracing
;; layer to draw over, with nothing detected and nothing measured. Its frames
;; are the footage's own and its rate the footage's, so it plays at the speed it
;; was filmed in a project at any rate.
(fn [{:keys [db]} _]
(let [{:keys [id range fps width height name frame point target] :as request}
(get-in db [:ui :convert])]
(if (or (nil? request) (get-in db [:footage :loading?]))
{}
{:db (update db :ui dissoc :convert)
:dispatch [::ui/drop-tracing
{:name name :type :trace
:media {:footage id :range range}
:frames (- (second range) (first range)) :fps fps
:width width :height height :nodes {}}
frame point target]}))))
(rf/reg-event-fx
::converted
;; A TAKE IS A CLIP LIKE ANY OTHER, placed into whichever symbol the drop names
;; — see `ui/drop-destination` — rather than into a container invented for it.
(fn [{:keys [db]} [_ {{:keys [name frame point range target]} :request footage-id :footage-id}
built]]
(let [entry (store/entry (:clip/current db))
uuid (random-uuid)
fps (get-in db [:clip :fps])
{:keys [clip sid subject-ids]}
(bring/take (:clip entry) (:clip built) name footage-id range)
st (merge (:store entry) (:store built))
imported-frames (clip/output-frames clip sid)
source-fps (get-in built [:clip :fps])
where (if point
(ui/creation-destination db clip st frame)
(ui/drop-destination db clip st frame target))
point (ui/destination-point where point)
result (if (:refused where)
where
(span/place-symbol (:clip where) st (:sid where)
uuid sid (:at where)
{:extent :grow-symbol :point point
:remainder-id (random-uuid)}))]
(if-let [why (or (:refused where) (:refused result))]
{:db (-> db {:db (-> db
(assoc :clip/current id (update :ui dissoc :convert)
:clip (select-keys clip [:fps :frames :width :height :audio :display-fps]) (update :footage merge {:loading? false :status why}))}
:footage (assoc (:footage db) :id id :label (:label clip) (let [db (edit/edit-entry
:loading? false :status summary)) db
(assoc-in [:playback :frame] 0) #(let [analysis-id (-> built :clip :analyses keys first)
(assoc-in [:playback :playing?] false)) remap-subjects
::pb/pause! nil}))) (fn [inputs]
(update inputs :subjects
(fn [subjects]
(into {} (map (fn [[old new]] [new (get subjects old)]))
subject-ids))))]
(-> (assoc % :clip (:clip result) :store st)
(update-in [:sources analysis-id]
(fn [source]
{:footage-id (:footage-id built)
:source-blocks (:source-blocks built)
:source-inputs
(update (remap-subjects (:source-inputs built))
:subjects merge
(get-in source [:source-inputs :subjects]))})))))]
{:db (-> db
(update :ui dissoc :convert)
(ui/selected
[:node (:sid where) uuid (conj (vec (:path where)) uuid)])
(update :footage merge
{:loading? false
:status (str "made " name " · " imported-frames " frames at " fps " fps"
(when (not= fps source-fps)
(str " · sampled from " source-fps " fps")))}))
:dispatch [::pb/refresh-clock]})))))

View file

@ -0,0 +1,98 @@
(ns arthur.events.history
"Undo and redo: `domain/history` against the open document, and the keys.
An undone step is an ordinary unsaved edit afterwards, and the next save sends
it — so undo reaches a collaborator the way any change of yours does."
(:require [arthur.domain.history :as history]
[arthur.domain.leaf :as leaf]
[arthur.events.edit :as edit]
[arthur.events.playback :as pb]
[arthur.events.ui :as ui]
[arthur.footage.store :as store]
[re-frame.core :as rf]))
(defn- step
"One step of `move` on `db`: `{:db :fps :ok?}`, `:ok?` false when there was
nothing to do or the step was refused."
[db move done]
(let [entry (store/entry (:clip/current db))
leaves (edit/leaves (:clip entry))
r (when leaves (move (:history entry) leaves))]
(cond
(nil? r)
{:db (assoc-in db [:project :status] (str "nothing to " (subs done 0 4))) :ok? false}
(:blocked r)
{:db (-> (edit/replace-entry db #(assoc % :history (:history r)))
(assoc-in [:project :status]
(str "not " done ": " (:label (:blocked r))
" — someone else has changed it since")))
:ok? false}
:else
(let [clip (leaf/clip "u" (:leaves r))
valid? (fn [[kind host node]]
(or (not= :node kind)
(nil? node)
(get-in clip [:symbols host :nodes node])))
selections (vec (filter valid? (get-in db [:ui :selections])))
primary (get-in db [:ui :selection])
primary (if (valid? primary) primary (peek selections))
label (:label (peek (get (:history r) (if (= done "undone") :undone :done))))]
{:ok? true
:db (-> (edit/replace-entry db #(assoc % :clip clip :history (:history r)))
(edit/transport clip)
(assoc-in [:ui :selections] selections)
(assoc-in [:ui :selection] primary)
(assoc-in [:project :status] (str done " " label " · unsaved")))}))))
(defn- steps
"`n` steps, stopping at the first that cannot be taken."
[db move done n]
(let [fps (get-in db [:clip :fps])
db (loop [db db n n]
(let [r (step db move done)]
(if (and (:ok? r) (< 1 n)) (recur (:db r) (dec n)) (:db r))))]
(cond-> {:db db}
(not= fps (get-in db [:clip :fps]))
(assoc ::pb/seek! [(get-in db [:clip :fps]) (pb/frames db) (get-in db [:playback :frame])]))))
(rf/reg-event-fx ::undo (fn [{:keys [db]} [_ n]] (steps db history/undo "undone" (or n 1))))
(rf/reg-event-fx ::redo (fn [{:keys [db]} [_ n]] (steps db history/redo "redone" (or n 1))))
(rf/reg-event-db ::hold (fn [db _] (edit/history db history/hold)))
(rf/reg-event-db ::settle (fn [db _] (edit/history db history/settle)))
(rf/reg-sub
::steps
;; The history is on the entry, outside app-db; the revision is what moves
;; when the entry does, undo and redo included.
(fn [db _]
(:paint/revision db)
(history/steps (:history (store/entry (:clip/current db))))))
(defn- typing? [^js target]
(or (#{"INPUT" "TEXTAREA" "SELECT"} (.-tagName target)) (.-isContentEditable target)))
(defn install-keys!
"The document edit keys. Not while typing in a field, where the browser's own
clipboard and undo stack are the ones wanted."
[]
(.addEventListener
js/window "keydown"
(fn [^js e]
(when-not (typing? (.-target e))
(let [k (.toLowerCase (.-key e))]
(when-let [ev (if (and (or (.-metaKey e) (.-ctrlKey e))
(not (.-altKey e)))
(cond (and (= k "z") (.-shiftKey e)) ::redo
(= k "z") ::undo
(= k "y") ::redo
(= k "c") ::ui/copy
(= k "x") ::ui/cut
(= k "v") ::ui/paste
(and (= k "d") (.-shiftKey e)) ::ui/duplicate-unique
(= k "d") ::ui/duplicate)
(when (#{"delete" "backspace"} k) ::ui/delete-selected))]
(.preventDefault e)
(rf/dispatch [ev])))))))

View file

@ -1,33 +1,25 @@
(ns arthur.events.paint (ns arthur.events.paint
"Polygon edits. Every one of them is `edit/edit` plus a pure `domain/paint`
function, which is the shape every document edit in this app should have."
(:require [arthur.domain.paint :as paint] (:require [arthur.domain.paint :as paint]
[arthur.footage.store :as store] [arthur.events.edit :as edit]
[re-frame.core :as rf])) [re-frame.core :as rf]))
(defn- edit [db f]
(let [id (store/edit-clip! (:clip/current db) f)]
(if id
(-> db
(assoc :clip/current id)
(update :paint/revision (fnil inc 0))
(update :project merge {:status "paint edited · unsaved"}))
db)))
(rf/reg-event-db (rf/reg-event-db
::new-shape ::new-shape
(fn [db [_ id points color]] ;; `frame` is the symbol's own: a shape drawn into an instance starts on the
(edit db #(paint/new-shape % id (get-in db [:playback :frame]) points color)))) ;; frame that instance is showing, not the transport's.
(fn [db [_ sid id frame points color]]
(edit/edit db #(paint/new-shape % sid id frame points color))))
(rf/reg-event-db (rf/reg-event-db
::add-key ::add-key
(fn [db [_ id]] ;; `frame` is the shape's own, which is the transport's only for a shape in the
(edit db #(paint/add-key % id (get-in db [:playback :frame]))))) ;; open symbol with no time map of its own.
(fn [db [_ sid id frame]]
(edit/edit db #(paint/add-key % sid id frame))))
(rf/reg-event-db (rf/reg-event-db
::set-vertex ::set-vertex
(fn [db [_ id key-frame vertex point]] (fn [db [_ sid id key-frame vertex point]]
(edit db #(paint/set-vertex % id key-frame vertex point)))) (edit/edit db #(paint/set-vertex % sid id key-frame vertex point))))
(rf/reg-event-db
::set-segment-interp
(fn [db [_ id key-frame interp]]
(edit db #(paint/set-segment-interp % id key-frame interp))))

View file

@ -10,29 +10,64 @@
traversals a second, which is the one genuinely expensive thing you can do to traversals a second, which is the one genuinely expensive thing you can do to
a small app-db. If global interceptors are added later they are added to a a small app-db. If global interceptors are added later they are added to a
chain these events are excluded from, not to `reg-global-interceptor`." chain these events are excluded from, not to `reg-global-interceptor`."
(:require [arthur.clock :as clock] (:require [arthur.audio.mix :as mix]
[arthur.clock :as clock]
[arthur.domain.clip :as clip]
[arthur.db :as app-db]
[arthur.footage.store :as footage] [arthur.footage.store :as footage]
[re-frame.core :as rf])) [re-frame.core :as rf]))
(defn- fps [db] (get-in db [:clip :fps])) (defn- fps [db] (get-in db [:clip :fps]))
(defn- frames [db] (get-in db [:clip :frames]))
(rf/reg-event-db (defn frames
"The open symbol's length. Derived from the document every time, never kept in
the db beside it, because an edit can change it."
[db]
(or (some-> (footage/entry (:clip/current db)) :clip
(clip/output-frames (get-in db [:ui :open])))
1))
(defn show
"Put loaded clip `id` on screen, open on the symbol it opens on, with the
playhead home. What every way of loading a document ends in, so that none of
them can forget which symbol is open."
[db id]
(let [entry (footage/entry id)
sid (clip/opens-on (:clip entry))]
(-> db
(assoc :clip/current id
:clip (select-keys entry [:fps :width :height :audio]))
(update :ui merge
{:open sid :tabs (if sid [sid] [])
;; The layers switched off were another document's, and their
;; ids mean nothing in this one. Whether tracing shows at all,
;; and how strongly, is the person's and carries over.
:tracing (assoc (get-in db [:ui :tracing] app-db/tracing) :hidden #{})})
;; Occurrence addresses belong to the document being left. Creation is
;; derived from the primary selection, so carrying one across documents
;; could otherwise make a coincidentally equal id a nested destination.
(update :ui dissoc :selection :selections :points)
(assoc-in [:playback :frame] 0)
(assoc-in [:playback :playing?] false))))
(rf/reg-event-fx
::tick ::tick
(fn [db [_ f]] (fn [{:keys [db]} [_ f]]
;; Written from the rAF loop when the DERIVED frame changes — not every ;; Written from the rAF loop when the DERIVED frame changes — not every
;; animation frame, and never as the thing the blit waits on. The picture is ;; animation frame, and never as the thing the blit waits on. The picture is
;; painted from the clock directly; this only brings the document's idea of ;; painted from the clock directly; this only brings the document's idea of
;; the playhead up to date so the readout and the scrubber agree with it. ;; the playhead up to date so the readout and the scrubber agree with it.
(if (= f (get-in db [:playback :frame])) (if (= f (get-in db [:playback :frame]))
db {:db db}
(assoc-in db [:playback :frame] f)))) (cond-> {:db (assoc-in db [:playback :frame] f)}
(get-in db [:ui :gesture :auto-key?])
(assoc :dispatch [:arthur.events.ui/record-gesture f])))))
(rf/reg-event-fx (rf/reg-event-fx
::play ::play
(fn [{:keys [db]} _] (fn [{:keys [db]} _]
{:db (assoc-in db [:playback :playing?] true) {:db (assoc-in db [:playback :playing?] true)
::play! nil})) ::play-from! [(fps db) (frames db) (get-in db [:playback :frame] 0)]}))
(rf/reg-event-fx (rf/reg-event-fx
::pause ::pause
@ -45,7 +80,8 @@
(fn [{:keys [db]} _] (fn [{:keys [db]} _]
(if (get-in db [:playback :playing?]) (if (get-in db [:playback :playing?])
{:db (assoc-in db [:playback :playing?] false) ::pause! nil} {:db (assoc-in db [:playback :playing?] false) ::pause! nil}
{:db (assoc-in db [:playback :playing?] true) ::play! nil}))) {:db (assoc-in db [:playback :playing?] true)
::play-from! [(fps db) (frames db) (get-in db [:playback :frame] 0)]})))
(rf/reg-event-fx (rf/reg-event-fx
::seek ::seek
@ -65,16 +101,16 @@
{:db (assoc-in db [:playback :rate] r) {:db (assoc-in db [:playback :rate] r)
::rate! r})) ::rate! r}))
(rf/reg-event-db
::set-picture-fps
(fn [db [_ target]]
(if (and (number? target) (pos? target) (<= target (fps db)))
(assoc-in db [:clip :display-fps] target)
db)))
;; --- effects: every DOM touch on the audio element is one of these --- ;; --- effects: every DOM touch on the audio element is one of these ---
(rf/reg-fx ::play! (fn [_] (clock/play!))) (rf/reg-fx ::play! (fn [_] (clock/play!)))
(rf/reg-fx ::play-from!
(fn [[fps frames f]]
;; The transport is the audio clock; app-db's frame is the visible
;; playhead. Re-anchor before starting so Play always honors the frame
;; currently shown, including after the clock has reached the clip end.
(clock/seek! fps frames f)
(clock/play!)))
(rf/reg-fx ::pause! (fn [_] (clock/pause!))) (rf/reg-fx ::pause! (fn [_] (clock/pause!)))
(rf/reg-fx ::rate! (fn [r] (clock/set-rate! r))) (rf/reg-fx ::rate! (fn [r] (clock/set-rate! r)))
(rf/reg-fx ::seek! (fn [[fps frames f]] (clock/seek! fps frames f))) (rf/reg-fx ::seek! (fn [[fps frames f]] (clock/seek! fps frames f)))
@ -99,14 +135,120 @@
;; Changing the clip changes the resolver, the frame count and the rate all ;; Changing the clip changes the resolver, the frame count and the rate all
;; at once, so the playhead goes home rather than being left pointing at a ;; at once, so the playhead goes home rather than being left pointing at a
;; frame the new clip may not have. ;; frame the new clip may not have.
(let [{:keys [fps frames] :as clip} (footage/entry id)] (let [{:keys [label cid]} (footage/entry id)
db (-> (show db id)
;; The document's identity goes with it. A built-in scene has no
;; project on the server, so this CLEARS the id rather than
;; keeping the last one — saving a fixture must create a
;; document of its own, not overwrite whatever was open before.
(assoc :project {:id nil :cid cid :name label
:seq nil :busy? false
:status "built-in example · not a saved project"}))]
{:db db
::pause! nil
::seek! [(fps db) (frames db) 0]
::clock! {:id id :sid (get-in db [:ui :open])}})))
;; ---------------------------------------------------------------------------
;; tabs
;;
;; A tab is an open symbol. `:open` is the one on screen and `:tabs` the ones
;; beside it, in the order they were opened. Switching is `:open` and a clock:
;; the stage, the timeline rows, the transport's length and a new shape's home
;; all follow `:open` through their subscriptions without being told.
(defonce ^:private clock-url (atom nil))
(rf/reg-fx
::clock!
;; WHAT THE OPEN SYMBOL SOUNDS LIKE, fetched once per thing that can change it:
;; a document opening, a tab switch, an edit to a track. The graph backend is
;; handed the buffer; the element backend needs a URL, which means a WAV, which
;; is the whole of what this used to cost.
(fn [{:keys [id sid]}]
(let [{:keys [clip audio store]} (footage/entry id)]
(-> (if (clock/graph?)
(mix/clock-source! clip sid audio store)
(.then (mix/clock! clip sid audio store) (fn [u] {:url u})))
(.then #(rf/dispatch [::clock-ready id sid %]))
(.catch #(js/console.error %))))))
(rf/reg-event-fx
::clock-ready
(fn [{:keys [db]} [_ id sid {:keys [url buffer seconds]}]]
(if-not (and (= id (:clip/current db)) (= sid (get-in db [:ui :open])))
{}
(if (clock/graph?)
(do
(clock/attach-buffer! buffer seconds)
;; RATE, LOOP AND MUTE ARE RE-APPLIED. They are transport state and
;; outlive the clock they were set on, but a fresh graph starts at its
;; defaults — so without this a tab switch would quietly drop the take
;; back to 1x, unlooped and unmuted. The element kept them because they
;; were properties of a node that survived its own `src` changing.
{:fx [[::rate! (get-in db [:playback :rate] 1.0)]
[::loop! (boolean (get-in db [:playback :loop?]))]
[::mute! (boolean (get-in db [:playback :muted?]))]
[::seek! [(fps db) (frames db) (get-in db [:playback :frame] 0)]]
;; An edit under a playing take resumes it. The element stopped
;; dead here, which made adjusting a fade while listening to it
;; a thing you could only do once.
(when (get-in db [:playback :playing?]) [::play! nil])]})
(do
;; A blob URL made for the last tab is released when the next one lands,
;; and never the document's own file.
(when-let [old @clock-url]
(when (not= old url) (js/URL.revokeObjectURL old)))
(reset! clock-url (when (and (not= url (:audio (footage/entry id)))
(.startsWith url "blob:"))
url))
{:db (assoc-in db [:clip :audio] url)})))))
(rf/reg-event-db
::paint-failed
(fn [db [_ message]]
(update db :project merge {:status (str "cannot draw: " message)})))
(rf/reg-event-fx
::refresh-clock
;; After an edit that changes what the open symbol sounds like.
(fn [{:keys [db]} _]
{::clock! {:id (:clip/current db) :sid (get-in db [:ui :open])}}))
(rf/reg-event-fx
::open-symbol
(fn [{:keys [db]} [_ sid]]
(let [clip (:clip (footage/entry (:clip/current db)))]
;; A tracing symbol has no inside to open: it is footage, which is seen by
;; placing it.
(if (or (nil? (clip/symbol clip sid)) (= sid (get-in db [:ui :open]))
(clip/trace? (clip/symbol clip sid)))
{}
{:db (-> db {:db (-> db
(assoc :clip/current id) (update-in [:ui :tabs] #(if (some #{sid} %) % (conj (vec %) sid)))
;; The stage travels with the clip: two clips may be different (assoc-in [:ui :open] sid)
;; sizes, and the raster the loop paints into is the clip's, not ;; A node address is relative to the tab being left. Opening a
;; the app's. ;; symbol therefore arrives at its root; a pool `[:symbol _]`
(assoc :clip (select-keys clip [:fps :frames :width :height :audio :display-fps])) ;; selection is not an occurrence address and may survive.
(update :ui (fn [ui]
(cond-> ui
(= :node (first (:selection ui)))
(dissoc :selection :selections))))
(assoc-in [:playback :frame] 0) (assoc-in [:playback :frame] 0)
(assoc-in [:playback :playing?] false)) (assoc-in [:playback :playing?] false))
::pause! nil ::pause! nil
::seek! [fps frames 0]}))) ::clock! {:id (:clip/current db) :sid sid}}))))
(rf/reg-event-fx
::close-tab
(fn [{:keys [db]} [_ sid]]
;; The last tab cannot be closed: something is always on screen, because the
;; transport and the stage have no meaning without a symbol.
(let [tabs (get-in db [:ui :tabs])
left (vec (remove #{sid} tabs))]
(cond
(empty? left) {}
(not= sid (get-in db [:ui :open])) {:db (assoc-in db [:ui :tabs] left)}
:else (let [i (.indexOf tabs sid)]
{:db (assoc-in db [:ui :tabs] left)
:dispatch [::open-symbol (get left (min i (dec (count left))))]})))))

View file

@ -21,14 +21,22 @@
Nothing here touches app-db except through events. The promise chain lives in an Nothing here touches app-db except through events. The promise chain lives in an
fx, which is the only thing in this namespace that is not pure." fx, which is the only thing in this namespace that is not pure."
(:require [arthur.domain.clip :as clip] (:require [arthur.db :as db]
[arthur.audio.mix :as mix] [arthur.domain.bring :as bring]
[arthur.domain.channel :as ch]
[arthur.domain.clip :as clip]
[arthur.domain.span :as span]
[arthur.domain.leaf :as leaf]
[arthur.domain.node :as node]
[arthur.domain.palette :as pal]
[arthur.events.edit :as edit]
[arthur.demo.stage :as stage] [arthur.demo.stage :as stage]
[arthur.domain.feature :as feature] [arthur.domain.feature :as feature]
[arthur.domain.project :as project] [arthur.domain.project :as project]
[arthur.domain.wire :as wire] [arthur.domain.wire :as wire]
[arthur.events.footage :as footage] [arthur.events.footage :as footage]
[arthur.events.playback :as pb] [arthur.events.playback :as pb]
[arthur.events.ui :as ui]
[arthur.footage.store :as store] [arthur.footage.store :as store]
[arthur.flow.address :as address] [arthur.flow.address :as address]
[arthur.flow.ingest :as ingest] [arthur.flow.ingest :as ingest]
@ -37,6 +45,7 @@
[arthur.flow.take :as take] [arthur.flow.take :as take]
[arthur.fx.http :as http] [arthur.fx.http :as http]
[arthur.synth :as synth] [arthur.synth :as synth]
[clojure.string :as str]
[re-frame.core :as rf])) [re-frame.core :as rf]))
(defn- analysis-payload [analysis] (defn- analysis-payload [analysis]
@ -92,73 +101,270 @@
(.then (fn [^js created] (.-id created)))))) (.then (fn [^js created] (.-id created))))))
(defn- opened-entry! [^js clip-json] (defn- opened-entry! [^js clip-json]
(let [footage-id (.-footage clip-json)]
(-> (js/Promise.all (-> (js/Promise.all
#js [(js/Promise.all
(into-array (map #(http/GET (str "/api/blocks/" %)) (into-array (map #(http/GET (str "/api/blocks/" %))
(array-seq (.-blocks clip-json))))) (array-seq (.-blocks clip-json)))))
(if footage-id (.then (fn [blocks]
(http/GET (str "/api/footage/" footage-id))
(js/Promise.resolve nil))])
(.then (fn [[blocks ^js footage]]
(let [cid (.-cid clip-json) (let [cid (.-cid clip-json)
loaded (project/load loaded (project/load
cid #js {:leaves (.-leaves clip-json) cid #js {:leaves (.-leaves clip-json)
:blocks blocks}) :blocks blocks})
built (:clip loaded)] ;; Project FPS and the root timeline are one clock. This
;; also normalizes documents saved by the earlier model,
;; where changing project FPS left the root on its old
;; editing grid.
built (-> (:clip loaded)
clip/pin-root
(clip/set-root-fps (:fps (:clip loaded))))]
(let [entry (merge (select-keys built [:fps :width :height]) (let [entry (merge (select-keys built [:fps :width :height])
{:label (str (or (.-name clip-json) cid) " (saved)") {:label (str (or (.-name clip-json) cid) " (saved)")
:cid cid :frames (clip/frames built) :cid cid
:display-fps (:fps built) ;; What the server holds, as of the seq
;; this was opened at: a save sends what
;; differs from it, and a collaborator's
;; write lands on what does not.
:synced (project/tier1 (.-leaves clip-json))
:clip built :store (:store loaded) :clip built :store (:store loaded)
:footage-id footage-id ;; The document's OWN file, unmixed. What
:audio (if footage (.-audio footage) ;; the symbol actually sounds like is the
"/static/arthur/audio.wav")})] ;; clock's business and is fetched once,
(-> (mix/mix! built (:audio entry) (:store entry)) ;; by `::pb/clock!`, when it goes on
(.then (fn [audio] (assoc entry :audio audio))))))))))) ;; screen — this used to mix it here as
;; well, and the two encodes of the same
;; audio were most of a project open.
:audio "/static/arthur/audio.wav"})]
entry))))))
(defn- saved-clip!
"Promise of clip `cid` of saved project `pid`, as `{:clip :store}`: its
document and the blocks it names, and nothing played or mixed."
[pid cid]
(-> (http/GET (str "/api/projects/" pid))
(.then (fn [^js doc]
(let [^js c (or (first (filter #(= cid (.-cid ^js %)) (array-seq (.-clips doc))))
(throw (ex-info "that project no longer has that clip" {:cid cid})))]
(-> (js/Promise.all (into-array (map #(http/GET (str "/api/blocks/" %))
(array-seq (.-blocks c)))))
(.then (fn [blocks]
(project/load cid #js {:leaves (.-leaves c) :blocks blocks})))))))))
(rf/reg-fx
::import!
(fn [{:keys [project cid] :as request}]
(-> (saved-clip! project cid)
(.then #(rf/dispatch [::imported request %]))
(.catch (fn [error]
(js/console.error error)
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))
(rf/reg-event-fx
::import
;; A symbol out of another saved project, dropped at `frame` of the open symbol
;; and, from the stage, with its middle on `point`.
(fn [{:keys [db]} [_ carried frame point target]]
{:db (update db :project merge {:status (str "fetching " (:label carried) "…")})
::import! (assoc (select-keys carried [:project :cid :symbol :label])
:host (get-in db [:ui :open]) :frame frame :point point
:target target)}))
(rf/reg-event-fx
::import-palette
(fn [{:keys [db]} [_ {:keys [project cid palette name]}]]
{:db (update db :project merge {:status (str "fetching " name "…")})
::import-palette! {:project project :cid cid :palette palette}}))
(rf/reg-fx
::import-palette!
(fn [{:keys [project cid palette]}]
(-> (saved-clip! project cid)
(.then (fn [{other :clip}]
(let [source (leaf/unsegment palette)
p (get-in other [:palettes source])]
(if p
(rf/dispatch [::palette-imported p])
(rf/dispatch [::failed "that palette no longer exists"])))))
(.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))
(rf/reg-event-db
::palette-imported
(fn [db [_ palette]]
(let [id (random-uuid)]
(-> (edit/edit db #(assoc-in % [:palettes id]
(assoc palette :id id :name (str (:name palette) " copy"))))
(assoc-in [:ui :palette] id)
(assoc-in [:project :status] (str "imported " (:name palette)))))))
(rf/reg-event-fx
::imported
;; As drawing: its tracking stays with the analysis that measured it. See
;; `arthur.domain.bring`.
(fn [{:keys [db]} [_ {:keys [symbol frame point label target]} other]]
(let [entry (store/entry (:clip/current db))
sid (leaf/unsegment symbol)
uuid (random-uuid)
{:keys [clip ids]} (bring/symbols (:clip entry) (:clip other) [sid] {})
st (merge (:store entry) (:store other))
;; A symbol from another project arrives as an ordinary clip, the same
;; as one from this project's pool.
where (if point
(ui/creation-destination db clip st frame)
(ui/drop-destination db clip st frame target))
point (ui/destination-point where point)
result (if (:refused where)
where
(span/place-symbol (:clip where) st (:sid where)
uuid (ids sid) (:at where)
{:extent :grow-symbol :point point
:remainder-id (random-uuid)}))]
(if-let [why (or (:refused where) (:refused result))]
{:db (update db :project merge {:status why})}
{:db (-> (edit/edit-entry db #(assoc % :clip (:clip result) :store st))
(ui/selected
[:node (:sid where) uuid (conj (vec (:path where)) uuid)])
(update :project merge {:status (str "brought in " label)}))
:dispatch [::pb/refresh-clock]}))))
(defn- clip-payload
"One clip of a save. With `base` — the seq the open document last caught up
to — only the leaves that differ from what the server held then, and the ones
since deleted: a collaborator's leaves are not ours to write back. Without it,
the whole clip."
[^js doc base local synced]
(if base
(let [all (.-leaves doc)
out (js-obj)]
(doseq [[path v] local :when (not= v (get synced path))]
(aset out path (aget all path)))
#js {:leaves out
:removed (into-array (remove #(contains? local %) (keys synced)))})
#js {:leaves (.-leaves doc)}))
(defonce ^:private uploaded-blocks
;; Block keys this page has already put on the server. Content addressed, so
;; once there they are there: a save of a moved vertex asks for none again.
(atom #{}))
(defonce ^:private linked-analyses (atom #{}))
(defn- upload-new! [^js doc]
(let [keys (array-seq (block-keys doc))]
(if (every? @uploaded-blocks keys)
(js/Promise.resolve 0)
(.then (upload-missing! doc) (fn [n] (swap! uploaded-blocks into keys) n)))))
(defn- analyses-of [entry]
(vals (get-in entry [:clip :analyses])))
(defn- upload-sources! [sources]
(reduce
(fn [chain [analysis-id {:keys [source-blocks]}]]
(.then chain
(fn [_]
(when (and (seq source-blocks)
(not (@linked-analyses analysis-id)))
(-> (upload-missing! #js {:blocks (source/upload-blocks source-blocks)})
(.then #(http/PUT
(str "/api/analyses/" analysis-id)
#js {:source_blocks
(into-array (source/block-keys source-blocks))}))
(.then (fn [answer]
(swap! linked-analyses conj analysis-id)
answer)))))))
(js/Promise.resolve nil)
sources))
(rf/reg-fx (rf/reg-fx
::save! ::save!
(fn [{:keys [id cid label clip]}] (fn [{:keys [id cid label clip base]}]
(let [analysis (:analysis (:clip clip)) (let [entry clip
doc (project/save cid clip) analyses (analyses-of entry)
source-blocks (:source-blocks clip)] doc (project/save cid entry)
local (leaf/leaves cid (:clip entry))
base (when (and id (:synced entry)) base)
sources (:sources entry)]
(-> (ensure-project! id label) (-> (ensure-project! id label)
(.then (fn [pid] (.then (fn [pid]
(-> (if analysis (-> (reduce (fn [chain one]
(http/POST "/api/analyses" (analysis-payload analysis)) (.then chain
(js/Promise.resolve nil)) #(http/POST "/api/analyses"
(analysis-payload one))))
(js/Promise.resolve nil)
analyses)
(.then (fn [_] (.then (fn [_]
(when (seq source-blocks) (upload-sources! sources)))
(-> (upload-missing!
#js {:blocks (source/upload-blocks source-blocks)})
(.then (fn [_] (.then (fn [_]
(http/PUT (upload-new! doc)))
(str "/api/analyses/" (:id analysis))
;; One set per tracked subject, in
;; the order `source/unpack` does
;; not depend on.
#js {:source_blocks
(into-array
(source/block-keys source-blocks))})))))))
(.then (fn [_] (upload-missing! doc)))
(.then (fn [uploaded] (.then (fn [uploaded]
(-> (http/PUT (str "/api/projects/" pid) (-> (http/PUT (str "/api/projects/" pid)
#js {:name label #js {:name label
:clips #js [#js {:cid cid :base base
:clips #js [(js/Object.assign
#js {:cid cid
:name label :name label
:analysis (:id analysis) :analyses (into-array
:footage (:footage-id clip) (keys (get-in entry [:clip :analyses])))
:leaves (.-leaves doc) :blocks (block-keys doc)}
:blocks (block-keys doc)}]}) (clip-payload doc base local
(:synced entry)))]})
(.then (fn [^js saved] (.then (fn [^js saved]
(rf/dispatch [::saved pid cid label (rf/dispatch [::saved pid cid label
(.-seq saved) (.-seq saved)
(count (array-seq (.-written saved))) (count (array-seq (.-written saved)))
uploaded]))))))))) uploaded
{:synced local :base base}])))))))))
(.catch (fn [error] (.catch (fn [error]
(js/console.error error) (js/console.error error)
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))) (if-let [conflicts (get-in (ex-data error) [:body :conflicts])]
(rf/dispatch [::conflicted (count conflicts)])
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))))
(rf/reg-fx
::list!
(fn [_]
(-> (http/GET "/api/projects")
(.then (fn [^js listed]
(rf/dispatch [::listed
(mapv (fn [^js row]
{:id (.-id row) :name (.-name row)
:owner (.-owner row)
:seq (.-seq row) :updated (.-updated row)})
(array-seq (.-projects listed)))])))
(.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))
(rf/reg-fx
::list-symbols!
(fn [_]
(-> (http/GET "/api/symbols")
(.then (fn [^js listed]
(rf/dispatch [::symbols-listed
(mapv (fn [^js r]
{:project (.-project r) :project-name (.-project_name r)
:cid (.-cid r) :symbol (.-symbol r)
:name (.-name r) :frames (.-frames r)})
(array-seq (.-symbols listed)))
(mapv (fn [^js r]
{:project (.-project r) :project-name (.-project_name r)
:cid (.-cid r) :palette (.-palette r) :name (.-name r)})
(array-seq (.-palettes listed)))])))
(.catch (fn [error]
(rf/dispatch [::failed (or (ex-message error) (str error))]))))))
(rf/reg-event-fx
::list-symbols
(fn [{:keys [db]} _]
{:db (assoc-in db [:assets :loading?] true)
::list-symbols! nil}))
(rf/reg-event-db
::symbols-listed
(fn [db [_ rows palettes]]
(assoc db :assets {:symbols rows :palettes palettes :loading? false})))
(rf/reg-sub ::assets (fn [db _] (:assets db)))
(declare blank-entry)
(rf/reg-fx (rf/reg-fx
::open! ::open!
@ -174,15 +380,26 @@
(.then (fn [^js row] (http/GET (str "/api/projects/" (.-id row))))) (.then (fn [^js row] (http/GET (str "/api/projects/" (.-id row)))))
(.then (fn [^js loaded] (.then (fn [^js loaded]
(let [^js clip-json (first (array-seq (.-clips loaded)))] (let [^js clip-json (first (array-seq (.-clips loaded)))]
(when-not clip-json (when (not= project/schema-version (.-schema_version loaded))
(throw (ex-info "that project has no clips" {}))) (throw (ex-info (str "that project is stored as schema "
(-> (opened-entry! clip-json) (.-schema_version loaded) " and this client reads "
project/schema-version)
{})))
;; A project made from the index has nothing in it yet: it
;; opens on a blank document, of which the server has seen
;; nothing, so the first save sends all of it.
(-> (if clip-json
(opened-entry! clip-json)
(js/Promise.resolve (assoc (blank-entry) :synced {})))
(.then (fn [entry] (.then (fn [entry]
(rf/dispatch [::opened (rf/dispatch [::opened
(store/install! entry "project") (store/install! entry "project")
(.-id loaded) (.-id loaded)
(.-name loaded) (.-name loaded)
(.-seq loaded)]))))))) (.-seq loaded)
{:owner (.-owner loaded)
:editors (vec (.-editors loaded))
:can-edit? (.-can_edit loaded)}])))))))
(.catch (fn [error] (.catch (fn [error]
(js/console.error error) (js/console.error error)
(rf/dispatch [::failed (or (ex-message error) (str error))])))))) (rf/dispatch [::failed (or (ex-message error) (str error))]))))))
@ -199,13 +416,9 @@
(.then (fn [entry] (.then (fn [entry]
(let [built (stage/compose (:clip entry)) (let [built (stage/compose (:clip entry))
entry (assoc entry :clip built :label (:name built) entry (assoc entry :clip built :label (:name built)
:cid "stage-8625" :frames (clip/frames built) :cid "stage-8625"
:width (:width built) :height (:height built))] :width (:width built) :height (:height built))]
(-> (mix/mix! built (:audio entry) (:store entry)) (rf/dispatch [::stage-opened (store/install! entry "stage")]))))
(.then (fn [audio]
(rf/dispatch
[::stage-opened
(store/install! (assoc entry :audio audio) "stage")])))))))
(.catch (fn [error] (.catch (fn [error]
(js/console.error error) (js/console.error error)
(rf/dispatch [::failed (or (ex-message error) (str error))])))))) (rf/dispatch [::failed (or (ex-message error) (str error))]))))))
@ -236,39 +449,38 @@
(assoc measured :interior-key block-key)))))))) (assoc measured :interior-key block-key))))))))
(throw error))))))) (throw error)))))))
(defn- source-for! [entry] (defn- source-for! [entry edit]
(if-let [inputs (:source-inputs entry)] (let [subject (:subject (regenerate/plan (:clip entry) edit))
(js/Promise.resolve inputs) subject-record (get-in entry [:clip :subjects subject])
(let [analysis (get-in entry [:clip :analysis])] analysis-id (:analysis subject-record)
(if (= (:id analysis) (:id @retained-source)) analysis (get-in entry [:clip :analyses analysis-id])
source-subject (:source-subject subject-record)
local (get-in entry [:sources analysis-id :source-inputs :subjects subject])]
(if local
(js/Promise.resolve {:subjects {subject local}})
(if (= [analysis-id subject] (:key @retained-source))
(:promise @retained-source) (:promise @retained-source)
(let [promise (let [promise
(if (= "synth" (:detector analysis)) (if (= "synth" (:detector analysis))
;; The synthetic take tracks one face and regenerating it reads
;; that face's landmarks, so it arrives in the same shape real
;; footage does rather than in a flat one only this branch uses.
(js/Promise.resolve (js/Promise.resolve
{:subjects {:subjects
{:face-1 {:dense (synth/synth-dense (:frames analysis) {subject {:dense (synth/synth-dense (:frames analysis)
{:seed (:seed analysis)})}}}) {:seed (:seed analysis)})}}})
(-> (ingest/manifest! (:footage-id entry)) (.then (ingest/manifest! (:footage subject-record))
(.then (fn [manifest] (fn [manifest]
(-> (footage/saved-source! (:id analysis) (.then (footage/saved-source!
[(:width manifest) analysis-id [(:width manifest) (:height manifest)])
(:height manifest)]) (fn [inputs]
(.then (fn [inputs]
(when-not inputs (when-not inputs
(throw (ex-info "saved analysis has no source blocks" {}))) (throw (ex-info "saved analysis has no source blocks" {})))
(update inputs :subjects {:subjects
(fn [subjects] {subject
(into {} (assoc (get-in inputs [:subjects source-subject])
(map (fn [[id one]] :presence
[id (assoc one :presence (footage/presence-for manifest
(footage/presence-for source-subject))}})))))]
manifest id))]))
subjects))))))))))]
(do (do
(reset! retained-source {:id (:id analysis) :promise promise}) (reset! retained-source {:key [analysis-id subject] :promise promise})
promise)))))) promise))))))
(defn- inputs-for-edit! (defn- inputs-for-edit!
@ -281,18 +493,19 @@
one (get-in inputs [:subjects subject])] one (get-in inputs [:subjects subject])]
(if (and teeth (:crops one)) (if (and teeth (:crops one))
(let [settings (merge take/knobs (feature/effective-params clip teeth)) (let [settings (merge take/knobs (feature/effective-params clip teeth))
analysis (get-in clip [:analysis :id]) analysis (get-in clip [:subjects subject :analysis])
source-subject (get-in clip [:subjects subject :source-subject])
frames (count (:crops one)) frames (count (:crops one))
key (source/interior-key analysis subject settings frames) key (source/interior-key analysis source-subject settings frames)
done (fn [measured] (assoc-in inputs [:subjects subject] measured))] done (fn [measured] (assoc-in inputs [:subjects subject] measured))]
(if (and (:interior one) (if (and (:interior one)
(or (= (:interior-key one) key) (or (= (:interior-key one) key)
(and (nil? (:interior-key one)) (and (nil? (:interior-key one))
(= key (source/interior-key analysis subject take/knobs frames))))) (= key (source/interior-key analysis source-subject take/knobs frames)))))
(js/Promise.resolve inputs) (js/Promise.resolve inputs)
(.then (if (= key (:key @retained-interior)) (.then (if (= key (:key @retained-interior))
(:promise @retained-interior) (:promise @retained-interior)
(let [promise (retained-interior! analysis subject settings one)] (let [promise (retained-interior! analysis source-subject settings one)]
(reset! retained-interior {:key key :promise promise}) (reset! retained-interior {:key key :promise promise})
promise)) promise))
done))) done)))
@ -301,7 +514,7 @@
(rf/reg-fx (rf/reg-fx
::preview-settings! ::preview-settings!
(fn [{:keys [id entry edit request]}] (fn [{:keys [id entry edit request]}]
(-> (source-for! entry) (-> (source-for! entry edit)
(.then (fn [inputs] (inputs-for-edit! entry edit inputs))) (.then (fn [inputs] (inputs-for-edit! entry edit inputs)))
(.then (fn [inputs] (.then (fn [inputs]
(regenerate/change (assoc entry :source-inputs inputs) edit))) (regenerate/change (assoc entry :source-inputs inputs) edit)))
@ -316,7 +529,7 @@
(fn [{:keys [db]} [_ edit]] (fn [{:keys [db]} [_ edit]]
(let [id (:clip/current db) (let [id (:clip/current db)
entry (store/entry id)] entry (store/entry id)]
(if (or (get-in db [:project :busy?]) (nil? (:analysis (:clip entry)))) (if (or (get-in db [:project :busy?]) (empty? (:analyses (:clip entry))))
{} {}
(let [plan (regenerate/plan (:clip entry) edit) (let [plan (regenerate/plan (:clip entry) edit)
report (select-keys plan [:features :roles]) report (select-keys plan [:features :roles])
@ -330,6 +543,7 @@
:request request}}))))) :request request}})))))
(rf/reg-sub ::regeneration (fn [db _] (:regeneration db))) (rf/reg-sub ::regeneration (fn [db _] (:regeneration db)))
(rf/reg-sub ::listing (fn [db _] (:projects db)))
(rf/reg-event-fx (rf/reg-event-fx
::settings-previewed ::settings-previewed
@ -345,27 +559,326 @@
;; --------------------------------------------------------------------------- ;; ---------------------------------------------------------------------------
;; events ;; events
(def blank-audio
"The audio a new document opens on, until it gets one of its own.
It is no longer load-bearing. The frame is derived from a position, and the
graph backend can hold one for a symbol with no sound at all — see
`mix/clock-source!` — so a silent stage has time and `play` works. This stays
because a new document borrowing the synthetic take's soundtrack is a
convenience worth keeping, not because the clock would stop without it."
"/static/arthur/audio.wav")
(defn blank-entry
"A blank clip, in the shape `footage/store` and the transport expect."
[]
(let [c (clip/blank)]
{:label "untitled" :clip c :store nil
:audio blank-audio
;; A content id of its own from the start: `::save!` addresses the clip by
;; it, and two untitled documents saved from two tabs are two documents.
:cid (str (random-uuid))
:fps (:fps c) :width (:width c) :height (:height c)}))
(rf/reg-event-fx
::new
(fn [{:keys [db]} _]
;; A document with no id on the server, so the next `save` creates one. This
;; is also what the app opens on: nothing is loaded until something is asked
;; for, and the built-in scenes are rows in the media pool like anything else.
(let [entry (blank-entry)
id (store/install! entry "new")]
{:db (-> (assoc db :ui (:ui db/default))
(pb/show id)
(assoc :project {:id nil :cid (:cid entry) :name nil :seq nil
:busy? false :status "new document"}))
;; The readout goes home with the document; so must the clock, or play
;; picks up wherever the last document's audio had got to.
::pb/seek! [(:fps entry) (clip/output-frames (:clip entry) (clip/opens-on (:clip entry))) 0]
::pb/pause! nil
::pb/clock! {:id id :sid (clip/opens-on (:clip entry))}})))
(rf/reg-event-fx (rf/reg-event-fx
::save ::save
(fn [{:keys [db]} _] ;; `auto?` is the save an edit schedules (see `arthur.events.collab`): it does
;; not lock the controls the way a save you asked for does, and it never saves
;; somebody else's project as a copy. Either kind waits its turn behind one in
;; flight, and writes nothing when nothing changed. `force?` saves what is not a leaf —
;; the name.
(fn [{:keys [db]} [_ {:keys [auto? force?] :as how}]]
(let [id (:clip/current db) (let [id (:clip/current db)
clip (store/entry id)] clip (store/entry id)
(if (or (:busy? (:project db)) (nil? clip)) {pid :id :keys [busy? saving? can-edit?]} (:project db)
cid (or (:cid clip) (name id))]
(cond
(nil? clip)
{} {}
{:db (update db :project merge {:busy? true :status "saving…"})
::save! {:id (:id (:project db)) ;; Behind the one in flight, never instead of it: an edit made while a
:cid (or (:cid clip) (name id)) ;; save is on the wire is not in that save. One request at a time, and
:label (or (:label clip) (name id)) ;; the next carries everything that changed meanwhile — so a drag goes
;; out as fast as the round trip allows, and no faster.
(or busy? saving?)
{:db (assoc-in db [:project :again] (or how {}))}
(and auto? (false? can-edit?))
{}
(and pid (:synced clip) (not force?) (empty? (:behind clip))
(= (:synced clip) (try (leaf/leaves cid (:clip clip)) (catch :default _ nil))))
{}
;; Somebody else wrote leaves we had changed too, first. The first
;; write wins: theirs goes on screen over ours, which stays in our undo
;; list. See `arthur.events.collab/landed`.
(seq (:behind clip))
{:dispatch [:arthur.events.collab/remote
{:seq (get-in db [:project :seq]) :take? true
:written (into {} (remove (comp nil? val)) (:behind clip))
:removed (keep (fn [[p v]] (when (nil? v) p)) (:behind clip))}]}
:else
;; Somebody else's project, which we may look at and not write, saves as
;; a copy of our own.
{:db (update db :project merge {(if auto? :saving? :busy?) true :status "saving…"})
::save! {:id (when-not (false? can-edit?) pid)
:base (get-in db [:project :seq])
:cid cid
:label (or (not-empty (get-in db [:project :name]))
(:label clip) (name id))
:clip clip}})))) :clip clip}}))))
(rf/reg-event-fx (rf/reg-event-fx
::open ::rename
(fn [{:keys [db]} [_ value]]
{:db (assoc-in db [:project :name] (not-empty (str/trim value)))
:dispatch [::save {:auto? true :force? true}]}))
(rf/reg-event-fx
::project-setting
(fn [{:keys [db]} [_ key value]]
(if (or (not (#{:fps :width :height} key))
(not (and (integer? value) (pos? value))))
{}
(let [db' (edit/edit db #(if (= key :fps) (clip/set-root-fps % value) (assoc % key value)))
db' (assoc-in db' [:clip key] value)
frame (min (dec (pb/frames db'))
(js/Math.floor (* (get-in db [:playback :frame])
(/ value (get-in db [:clip :fps])))))]
(cond-> {:db db'}
(= key :fps) (assoc :db (assoc-in db' [:playback :frame] frame)
::pb/seek! [value (pb/frames db') frame]
:dispatch [::pb/refresh-clock]))))))
(rf/reg-event-db
::rename-symbol
(fn [db [_ sid value]]
;; A transaction, so one rename is one undo step: `edit/edit` alone would let
;; a rename coalesce with whatever edit happened next.
;;
;; BLANK REMOVES THE NAME rather than storing an empty one. `clip/symbol-name`
;; falls back to the id, so a symbol cleared of its name reads as `main`
;; again instead of as a row with nothing on it — and the document carries no
;; field it did not need.
(let [value (not-empty (str/trim (str value)))]
(if-not (clip/symbol (:clip (store/entry (:clip/current db))) sid)
db
(edit/transaction db #(if value
(assoc-in % [:symbols sid :name] value)
(update-in % [:symbols sid] dissoc :name)))))))
(rf/reg-event-db
::symbol-setting
(fn [db [_ sid key value]]
(if-not (and (#{:frames :fps :width :height} key)
(or (nil? value) (and (integer? value) (pos? value))))
db
(edit/edit db
(fn [c]
(if (nil? value)
(update-in c [:symbols sid] dissoc key)
(assoc-in c [:symbols sid key] value)))))))
(rf/reg-event-db
::new-palette
(fn [db _]
(let [id (random-uuid)
p {:id id :name "Untitled palette"
:slots (mapv #(select-keys % [:name :hex]) (:slots pal/default-palette))}]
(-> (edit/edit db #(assoc-in % [:palettes id] p))
(assoc-in [:ui :palette] id)))))
(rf/reg-event-db
::duplicate-palette
;; A copy of an existing palette, selected so the next colour edit lands on the
;; copy rather than the original. The point is a variant: start from a palette
;; that works and change two tones, instead of 16 colour pickers from black.
;; `pal/palettes` is the source so the implicit default - a project that has
;; never had a palette asset of its own - can be duplicated like any other.
(fn [db [_ id]]
(let [clip (:clip (store/entry (:clip/current db)))
p (get (pal/palettes clip) id)]
(if-not p
db
(let [new-id (random-uuid)]
(-> (edit/edit db #(assoc-in % [:palettes new-id]
(assoc p :id new-id
:name (str (:name p) " copy"))))
(assoc-in [:ui :palette] new-id)))))))
(rf/reg-event-db
::palette-name
(fn [db [_ id value]]
(let [value (str/trim (str value))]
(if (str/blank? value) db
(edit/edit db #(assoc-in % [:palettes id :name] value))))))
(rf/reg-event-db
::palette-color
(fn [db [_ id slot hex]]
(if-not (re-matches #"#[0-9a-fA-F]{6}" (str hex))
db
(edit/edit db #(assoc-in % [:palettes id :slots slot :hex] (str/lower-case hex))))))
(rf/reg-event-db
::default-palette
(fn [db [_ id]]
(edit/edit db #(if (get-in % [:palettes id]) (assoc % :default-palette id) %))))
(rf/reg-event-db
::symbol-palette
(fn [db [_ sid id]]
(edit/edit db #(if id
(assoc-in % [:symbols sid :palette] id)
(update-in % [:symbols sid] dissoc :palette)))))
(rf/reg-event-db
::set-channel
;; `frame` is the node's own, as for a drawing key.
(fn [db [_ sid id path frame value]]
(let [put (if (get-in db [:ui :auto-key?]) node/set-keyed-channel node/set-channel)]
(edit/edit db #(update-in % [:symbols sid :nodes id] put path frame value)))))
(rf/reg-event-db
::instance-playback
(fn [db [_ sid id field value]]
(let [valid? (case field
:mode (#{:once :loop :frame} value)
(:in :speed) (and (node/finite-number? value) (<= 0 value))
false)]
(if-not valid?
db
(edit/edit
db
(fn [document]
(let [path [:symbols sid :nodes id]
n (get-in document path)]
(if-not (= :instance (:kind n))
document
(let [{:keys [speed]} (node/playback-of n)
updated (if (= :mode field)
(-> n
(update :time dissoc :loop?)
(assoc-in [:playback :end] (if (= :loop value) :loop :stop))
(assoc-in [:playback :speed]
(if (= :frame value) 0 (if (pos? speed) speed 1))))
(assoc-in n [:playback field] value))]
(assoc-in document path updated))))))))))
(defn- seed-palette-choice [document sid id]
(if (get-in document [:symbols sid :nodes id :channels [:palette]])
document
(let [source (get-in document [:symbols sid :nodes id :source :symbol])
palette-id (or (get-in document [:symbols source :palette-ref]) pal/inherit)]
(assoc-in document [:symbols sid :nodes id :channels [:palette]]
(assoc (ch/framed palette-id) :semantic :palette)))))
(rf/reg-event-db
::set-palette-choice
(fn [db [_ sid id frame palette-id]]
(let [put (if (get-in db [:ui :auto-key?]) node/set-keyed-channel node/set-channel)]
(edit/edit db
(fn [document]
(update-in (seed-palette-choice document sid id)
[:symbols sid :nodes id]
put [:palette] frame palette-id))))))
(rf/reg-event-db
::toggle-palette-key
(fn [db [_ sid id frame]]
(let [st (:store (store/entry (:clip/current db)))]
(edit/edit db
(fn [document]
(update-in (seed-palette-choice document sid id)
[:symbols sid :nodes id]
node/toggle-key [:palette] frame st))))))
(rf/reg-event-db
::set-channels
;; One property assignment over a stage selection is one document edit.
(fn [db [_ edits]]
(let [put (if (get-in db [:ui :auto-key?]) node/set-keyed-channel node/set-channel)]
(edit/edit db
#(reduce (fn [c {:keys [sid id path frame value]}]
(update-in c [:symbols sid :nodes id] put path frame value))
% edits)))))
(rf/reg-event-db
::toggle-key
(fn [db [_ sid id path frame]]
(let [st (:store (store/entry (:clip/current db)))]
(edit/edit db #(update-in % [:symbols sid :nodes id] node/toggle-key path frame st)))))
;; A hold on node `id`'s own frame `frame`, or none there if there was one: the
;; frames a tracing layer holds its picture on — a face's trace keys are its
;; plate's — and, for any node, `node/hold`. Taking the last one off removes the
;; floor rather than leaving an empty list behind.
(rf/reg-event-db
::toggle-hold
(fn [db [_ sid id frame]]
(edit/edit db #(update-in % [:symbols sid :nodes id :time]
(fn [t]
(let [hs (get t :holds [])
hs (vec (sort (if (some #{frame} hs)
(remove #{frame} hs)
(conj hs frame))))]
(if (seq hs) (assoc t :holds hs) (dissoc t :holds))))))))
;; How a face's head moves between its footage's holds — `symbol`'s `:reads`.
;; Nil reads every frame.
(rf/reg-event-db
::set-reads
(fn [db [_ sid reads]]
(edit/edit db #(update-in % [:symbols sid :nodes :head]
(fn [n] (if reads (assoc n :reads reads) (dissoc n :reads)))))))
(rf/reg-event-db
::set-segment-interp
(fn [db [_ sid id path left interp]]
(edit/edit db #(update-in % [:symbols sid :nodes id] node/set-segment-interp path left interp))))
(rf/reg-event-fx
::list
(fn [{:keys [db]} _] (fn [{:keys [db]} _]
{:db (assoc-in db [:projects :loading?] true)
::list! nil}))
(rf/reg-event-db
::listed
(fn [db [_ rows]] (assoc db :projects {:items rows :loading? false})))
(rf/reg-event-fx
::open
(fn [{:keys [db]} [_ id]]
;; `id` names which project. Without one it is the open document's own id, and
;; without that the most recently updated — which is what "open" meant when
;; there was no list to pick from.
(if (:busy? (:project db)) (if (:busy? (:project db))
{} {}
{:db (update db :project merge {:busy? true :status "opening…"}) {:db (update db :project merge {:busy? true :status "opening…"})
::pb/pause! nil ::pb/pause! nil
::open! (:id (:project db))}))) ::open! (or id (:id (:project db)))})))
(rf/reg-event-fx (rf/reg-event-fx
::load-stage ::load-stage
@ -379,41 +892,70 @@
(rf/reg-event-fx (rf/reg-event-fx
::stage-opened ::stage-opened
(fn [{:keys [db]} [_ clip-id]] (fn [{:keys [db]} [_ clip-id]]
(let [entry (store/entry clip-id)] (let [db (-> (pb/show db clip-id)
{:db (-> db
(assoc :clip/current clip-id
:clip (select-keys entry [:fps :frames :width :height :audio :display-fps]))
(assoc :project {:id nil :cid nil :name nil :seq nil (assoc :project {:id nil :cid nil :name nil :seq nil
:busy? false :status "loaded 8625 stage study"}) :busy? false :status "loaded 8625 stage study"}))]
(assoc-in [:playback :frame] 0) {:db db
(assoc-in [:playback :playing?] false)) ::pb/seek! [(get-in db [:clip :fps]) (pb/frames db) 0]
::pb/seek! [(:fps entry) (:frames entry) 0]}))) ::pb/clock! {:id clip-id :sid (get-in db [:ui :open])}})))
(rf/reg-event-db (rf/reg-event-fx
::saved ::saved
(fn [db [_ id cid label seq written uploaded]] (fn [{:keys [db]} [_ id cid label seq written uploaded {:keys [synced base]}]]
(update db :project merge (let [fresh? (not= id (get-in db [:project :id]))]
{:id id :cid cid :name label :seq seq :busy? false {:db (-> db
(update :clip/current #(or (store/edit-entry! % (fn [e] (-> (assoc e :synced synced)
(dissoc :behind))))
%))
(update :project dissoc :again)
(update :project merge
{:id id :cid cid :name label :seq seq :busy? false :saving? false
:status (str "saved r" seq " · " written :status (str "saved r" seq " · " written
(if (= 1 written) " leaf" " leaves") (if (= 1 written) " leaf" " leaves")
" · " uploaded (if (= 1 uploaded) " block" " blocks"))}))) " · " uploaded (if (= 1 uploaded) " block" " blocks"))}
(when fresh?
{:owner (get-in db [:me :username]) :editors [] :can-edit? true})))
:fx [;; The all-assets folder lists saved symbols, so a save can add rows to it.
[:dispatch [::list-symbols]]
;; Somebody wrote between what we last saw and this save. Their
;; deltas may still be on the wire, and a seq we have jumped past
;; would drop them, so ask for the document instead.
(when (and base (not= seq (inc base)))
[:dispatch [:arthur.events.collab/catch-up]])
(when-let [how (get-in db [:project :again])]
[:dispatch [::save how]])]})))
(rf/reg-event-fx
::conflicted
;; Nothing was written: somebody else's write to the same leaves got there
;; first. Catching up TAKES theirs, over ours — then the rest of ours saves.
(fn [{:keys [db]} [_ n]]
{:db (-> db
(update :project dissoc :again)
(update :project merge {:busy? false :saving? false}))
:fx [[:dispatch [:arthur.events.collab/catch-up true]]
[:dispatch [::save {:auto? true}]]]}))
(rf/reg-event-fx (rf/reg-event-fx
::opened ::opened
(fn [{:keys [db]} [_ clip-id project-id name seq]] (fn [{:keys [db]} [_ clip-id project-id name seq access]]
(let [clip (store/entry clip-id)] {:db (-> (pb/show db clip-id)
{:db (-> db (update :project merge access
(assoc :clip/current clip-id {:id project-id :name name :seq seq
:clip (select-keys clip [:fps :frames :width :height :audio :display-fps])) :cid (:cid (store/entry clip-id))
(update :project merge
{:id project-id :name name :seq seq :cid (:cid clip)
:busy? false :busy? false
:status (str "opened " name " r" seq)}) :status (str "opened " name " r" seq)}))
(assoc-in [:playback :frame] 0) ::pb/pause! nil
(assoc-in [:playback :playing?] false)) ::pb/seek! (let [{c :clip fps :fps} (store/entry clip-id)]
::pb/pause! nil}))) [fps (clip/output-frames c (clip/opens-on c)) 0])
::pb/clock! {:id clip-id
:sid (clip/opens-on (:clip (store/entry clip-id)))}}))
(rf/reg-event-db (rf/reg-event-fx
::failed ::failed
(fn [db [_ message]] (fn [{:keys [db]} [_ message]]
(update db :project merge {:busy? false :status (str "failed: " message)}))) (cond-> {:db (-> db
(update :project dissoc :again)
(update :project merge {:busy? false :saving? false
:status (str "failed: " message)}))}
(get-in db [:project :again]) (assoc :dispatch [::save (get-in db [:project :again])]))))

File diff suppressed because it is too large Load diff

View file

@ -12,7 +12,7 @@
reachable and neither should be the other's special case. reachable and neither should be the other's special case.
What is genuinely shared is everything above the sink, and it is most of the What is genuinely shared is everything above the sink, and it is most of the
work: rooting the resolver at the chosen timeline, generated-channel picture work: rooting the resolver at the chosen symbol, generated-channel picture
sampling, sampling,
the raster, the frame loop, the audio mix, the progress reporting and the the raster, the frame loop, the audio mix, the progress reporting and the
yielding that lets the page paint. So `run!` owns all of that and calls three yielding that lets the page paint. So `run!` owns all of that and calls three
@ -20,7 +20,7 @@
TWO RULES THE WALK ENFORCES, both about sync: TWO RULES THE WALK ENFORCES, both about sync:
Every frame of the timeline's frame space is emitted, at the CLIP's rate. A Every frame of the symbol's frame space is emitted, at the CLIP's rate. A
lower picture rate holds a pose across several frames — it never drops them — lower picture rate holds a pose across several frames — it never drops them —
so the exported duration matches the audio no matter what the picture rate is. so the exported duration matches the audio no matter what the picture rate is.
Decimating instead is how an export silently runs short and the sound slides Decimating instead is how an export silently runs short and the sound slides
@ -34,17 +34,18 @@
(:refer-clojure :exclude [run!]) (:refer-clojure :exclude [run!])
(:require [arthur.audio.mix :as mix] (:require [arthur.audio.mix :as mix]
[arthur.domain.clip :as clip] [arthur.domain.clip :as clip]
[arthur.domain.palette :as pal]
[arthur.domain.raster :as raster])) [arthur.domain.raster :as raster]))
(defprotocol Exporter (defprotocol Exporter
"A sink for a rendered timeline. Implementations live under `arthur.export.*`. "A sink for a rendered symbol. Implementations live under `arthur.export.*`.
Called in this order, once, per export: `begin!`, then `frame!` for every frame Called in this order, once, per export: `begin!`, then `frame!` for every frame
in order from 0, then `finish!`. Any of them may return a promise and the walk in order from 0, then `finish!`. Any of them may return a promise and the walk
waits for it, which is what keeps a slow encoder from being fed faster than it waits for it, which is what keeps a slow encoder from being fed faster than it
drains and what gives the page a chance to paint between frames. drains and what gives the page a chance to paint between frames.
`Exporter` rather than `IExporter`, which is what `domain/timeline`'s `Exporter` rather than `IExporter`, which is what `domain/symbol`'s
`IResolver` would suggest, because it names a role a thing plays rather than a `IResolver` would suggest, because it names a role a thing plays rather than a
capability a value has." capability a value has."
@ -60,7 +61,7 @@
:fps frames per second of the finished file — the CLIP's rate :fps frames per second of the finished file — the CLIP's rate
:frames how many frames will arrive :frames how many frames will arrive
:ramp index -> [r g b], the palette to expand through :ramp index -> [r g b], the palette to expand through
:audio an AudioBuffer, or nil when the timeline has no sound :audio an AudioBuffer, or nil when the symbol has no sound
The ramp and the audio are here rather than on `frame!` because neither The ramp and the audio are here rather than on `frame!` because neither
changes across an export, and a muxer has to declare its tracks before it changes across an export, and a muxer has to declare its tracks before it
@ -71,7 +72,7 @@
THE RASTER IS REUSED and must be consumed before this returns (or before the THE RASTER IS REUSED and must be consumed before this returns (or before the
promise it returns settles). The walk hands back the same buffer every frame, promise it returns settles). The walk hands back the same buffer every frame,
for the same reason `timeline/resolver` reuses its point buffers: a 900-frame for the same reason `symbol/resolver` reuses its point buffers: a 900-frame
export that allocates a stage per frame is a tab that swaps. A sink that wants export that allocates a stage per frame is a tab that swaps. A sink that wants
to keep pixels has to copy or encode them here.") to keep pixels has to copy or encode them here.")
@ -84,15 +85,15 @@
Four things, and each for its own reason: Four things, and each for its own reason:
the node itself; the node itself;
everything ABOVE it, because a placement's transform is relative to its everything ABOVE it, because an instance's transform is relative to its
parent and dropping the chain would move the thing being isolated; parent and dropping the chain would move the thing being isolated;
everything BELOW it, because a group instance is its children; everything BELOW it, because a group instance is its children;
any audio track `:linked-to` it, because the link is the statement that this any audio track `:linked-to` it, because the link is the statement that this
sound belongs to that placement, and a face exported without its voice is sound belongs to that instance, and a face exported without its voice is
not the thing that was asked for. not the thing that was asked for.
Siblings go. That is the whole point: what comes out is one placement, where it Siblings go. That is the whole point: what comes out is one instance, where it
sits, on the timeline it sits on." sits, in the symbol it sits in."
[nodes id] [nodes id]
(let [up (loop [i id acc #{}] (let [up (loop [i id acc #{}]
(if (or (nil? i) (contains? acc i)) (if (or (nil? i) (contains? acc i))
@ -114,18 +115,18 @@
k)))) k))))
(defn isolate (defn isolate
"The timeline with only `id` and its kin kept. `nil` leaves it alone. "The symbol with only `id` and its kin kept. `nil` leaves it alone.
The FRAME SPACE IS UNTOUCHED, which is what makes this different from exporting The FRAME SPACE IS UNTOUCHED, which is what makes this different from exporting
the symbol a placement plays. Rooting at `:sym/face-8625` renders the drawing in the symbol an instance plays. Rooting at `:sym/face-8625` renders the drawing in
its own time, identically for all seven placements. Isolating one placement its own time, identically for all seven instances. Isolating one instance
renders the STAGE — its length, its rate, the placement's span, drift and scale renders the STAGE — its length, its rate, the instance's span, drift and scale
— with the other six removed. The first is the drawing; the second is that face — with the other six removed. The first is the drawing; the second is that face
on the stage, and they are different deliverables." on the stage, and they are different deliverables."
[tl id] [sym id]
(if (and id (get-in tl [:nodes id])) (if (and id (get-in sym [:nodes id]))
(update tl :nodes select-keys (kin (:nodes tl) id)) (update sym :nodes select-keys (kin (:nodes sym) id))
tl)) sym))
(defn- yield! (defn- yield!
"Hand the event loop a turn between frames. "Hand the event loop a turn between frames.
@ -140,68 +141,63 @@
(defn audio! (defn audio!
"Promise of the AudioBuffer to export alongside the picture, or nil. "Promise of the AudioBuffer to export alongside the picture, or nil.
A timeline's own placed audio tracks win. Failing that, the ROOT timeline — and A symbol's own placed audio tracks win. Failing that, the symbol the document
only the root — falls back to the clip's audio file, which is where a take's OPENS ON — and only that one — falls back to the clip's audio file, which is
sound lives before anyone has placed a track. A symbol exports silence rather where a take's sound lives before anyone has placed a track. Any other symbol
than the whole clip's soundtrack, because a symbol's frame space is its own and exports silence rather than the whole clip's soundtrack, because its frame
the clip's audio is not a fact about it." space is its own and the clip's audio is not a fact about it."
[clip-doc tid store fallback-url] [clip-doc sid store fallback-url]
(-> (mix/buffer! clip-doc tid store) (-> (mix/buffer! clip-doc sid store)
(.then (fn [buffer] (.then (fn [buffer]
(cond (cond
buffer buffer buffer buffer
(and (= tid clip/root-id) fallback-url) (mix/decode! fallback-url) (and (= sid (clip/opens-on clip-doc)) fallback-url) (mix/decode! fallback-url)
:else nil))))) :else nil)))))
(defn plan (defn plan
"What an export of `tid` will produce, without producing any of it. "What an export of `sid` will produce, without producing any of it.
Separate from `run!` so the UI can show the size and length it is about to Separate from `run!` so the UI can show the size and length it is about to
commit to, and so the arithmetic is assertable without a sink." commit to, and so the arithmetic is assertable without a sink."
[{:keys [clip timeline zoom picture-fps] isolate-id :isolate}] [{:keys [clip zoom] sid :symbol isolate-id :isolate}]
(let [tl (some-> (clip/timeline clip timeline) (isolate isolate-id)) (let [sym (some-> (clip/symbol clip sid) (isolate isolate-id))
[width height] (clip/stage clip sid)
zoom (max 1 (js/Math.floor (or zoom 1)))] zoom (max 1 (js/Math.floor (or zoom 1)))]
(when tl (when sym
{:frames (:frames tl) {:frames (clip/output-frames clip sid)
:fps (:fps clip) :fps (:fps clip)
:zoom zoom :zoom zoom
:width (* (:width clip) zoom) :width (* width zoom)
:height (* (:height clip) zoom) :height (* height zoom)
:seconds (/ (:frames tl) (:fps clip)) :seconds (/ (clip/output-frames clip sid) (:fps clip))})))
;; The unedited picture-grid count. A per-instance pose track can add or
;; remove changes, so this is only the grid's nominal count.
:poses (if (and picture-fps (< picture-fps (:fps clip)))
(js/Math.ceil (* (/ (:frames tl) (:fps clip)) picture-fps))
(:frames tl))})))
(defn run! (defn run!
"Render `timeline` into `exporter`. Promise of `{:filename :blob}`. "Render symbol `sid` into `exporter`. Promise of `{:filename :blob}`.
`on-progress` is called with `[done total]` as frames complete, and is where a `on-progress` is called with `[done total]` as frames complete, and is where a
UI hangs its readout." UI hangs its readout."
[{:keys [clip timeline store palette ramp zoom picture-fps name audio-url] [{:keys [clip store palette ramp zoom name audio-url]
isolate-id :isolate} sid :symbol isolate-id :isolate}
exporter on-progress] exporter on-progress]
(let [tl (some-> (clip/timeline clip timeline) (isolate isolate-id))] (let [sym (some-> (clip/symbol clip sid) (isolate isolate-id))]
(when-not tl (when-not sym
(throw (ex-info "there is no such timeline to export" (throw (ex-info "there is no such symbol to export"
{:timeline timeline {:symbol sid
:timelines (vec (sort-by str (keys (:timelines clip))))}))) :symbols (vec (sort-by str (keys (:symbols clip))))})))
(let [{:keys [frames fps zoom]} (plan {:clip clip :timeline timeline :zoom zoom (let [{:keys [frames fps zoom]} (plan {:clip clip :symbol sid :zoom zoom
:isolate isolate-id}) :isolate isolate-id})
;; Rooted at the chosen timeline, so exporting a symbol is exporting a [width height] (clip/stage clip sid)
;; clip whose root that symbol is. Nested symbols inside it still ;; Rooted at the chosen symbol, as the stage is. Nested instances
;; resolve — clip/resolver is the function that knows how. ;; inside it still resolve — clip/resolver is the function that knows
doc (assoc-in clip [:timelines timeline] tl) ;; how.
resolve-frame (clip/resolver doc store palette timeline doc (assoc-in clip [:symbols sid] sym)
{:picture-fps picture-fps}) resolve-frame (clip/resolver doc sid store palette nil)
ras (raster/make (:width clip) (:height clip)) ras (raster/make width height)]
bg (get palette :bg 0)] (-> (audio! doc sid store audio-url)
(-> (audio! doc timeline store audio-url)
(.then (fn [audio] (.then (fn [audio]
(js/Promise.resolve (js/Promise.resolve
(begin! exporter {:name name :width (:width clip) (begin! exporter {:name name :width width
:height (:height clip) :zoom zoom :height height :zoom zoom
:fps fps :frames frames :ramp ramp :fps fps :frames frames :ramp ramp
:audio audio})))) :audio audio}))))
(.then (fn [_] (.then (fn [_]
@ -214,14 +210,19 @@
(fn [chain i] (fn [chain i]
(.then chain (.then chain
(fn [_] (fn [_]
(let [ops (resolve-frame i)
active (clip/active-palette resolve-frame)]
(-> ras (-> ras
(raster/clear! bg) (raster/clear! (pal/background-index palette active))
(raster/draw-ops! (resolve-frame i))) (raster/draw-ops! ops))
(-> (js/Promise.resolve (frame! exporter i ras)) (-> (js/Promise.resolve
(frame! exporter i
(assoc ras :palette-ramp
(pal/effective-ramp palette active))))
(.then (fn [_] (.then (fn [_]
(when on-progress (when on-progress
(on-progress (inc i) frames)) (on-progress (inc i) frames))
(yield!))))))) (yield!))))))))
(js/Promise.resolve) (js/Promise.resolve)
(range frames)))) (range frames))))
(.then (fn [_] (finish! exporter))))))) (.then (fn [_] (finish! exporter)))))))

View file

@ -63,7 +63,7 @@
;; raster: keeping a reference to it and encoding later would encode ;; raster: keeping a reference to it and encoding later would encode
;; the last frame N times, and every frame would be a valid PNG of the ;; the last frame N times, and every frame would be a valid PNG of the
;; wrong picture. ;; wrong picture.
(-> (encode raster ramp) (-> (encode raster (or (:palette-ramp raster) ramp))
(.then (fn [bytes] (.then (fn [bytes]
(swap! state update :entries conj (swap! state update :entries conj
{:name (str name "/" (pad (inc i) digits) ".png") {:name (str name "/" (pad (inc i) digits) ".png")

View file

@ -67,8 +67,15 @@
over the same frames and the same model they disagree by up to 0.013 of frame over the same frames and the same model they disagree by up to 0.013 of frame
width, which is a visible difference on a mouth. Optional, because the synthetic width, which is a visible difference on a mouth. Optional, because the synthetic
take has no running mode to declare and an absent field is how the other take has no running mode to declare and an absent field is how the other
optional inputs already say \"not applicable\"." optional inputs already say \"not applicable\".
[{:keys [detector version source footage frames fps aspect seed mode tracking]}]
`:range` is `[first end)` source frames when only part of the footage was
analysed, and absent for all of it — so every analysis made before ranges
existed keeps its address. It is in here because a partial analysis is a
different artifact: without it, detecting frames 40–90 would be saved under the
same key as the whole take, and the next conversion of the whole take would be
handed fifty frames."
[{:keys [detector version source footage frames fps aspect seed mode tracking range]}]
(when-not (and (string? detector) (seq detector) (string? version) (seq version)) (when-not (and (string? detector) (seq detector) (string? version) (seq version))
(throw (ex-info "an analysis names its detector and the detector's VERSION: an upgrade that silently reuses old landmarks is the failure content addressing exists to prevent" (throw (ex-info "an analysis names its detector and the detector's VERSION: an upgrade that silently reuses old landmarks is the failure content addressing exists to prevent"
{:detector detector :version version}))) {:detector detector :version version})))
@ -82,7 +89,8 @@
footage (assoc :footage footage) footage (assoc :footage footage)
seed (assoc :seed seed) seed (assoc :seed seed)
mode (assoc :mode mode) mode (assoc :mode mode)
tracking (assoc :tracking tracking)))) tracking (assoc :tracking tracking)
range (assoc :range range))))
(defn analysis (defn analysis
"An analysis record with its `:id` filled in. The record is tier 1 — it says "An analysis record with its `:id` filled in. The record is tier 1 — it says
@ -150,6 +158,12 @@
"head-pos" [:anchor-avg] "head-pos" [:anchor-avg]
"head-rot" [:anchor-avg] "head-rot" [:anchor-avg]
"head-scale" [:anchor-avg] "head-scale" [:anchor-avg]
;; The head's fit itself, registering the footage under it — see
;; `freeze/plate-part`. The image height it divides by is the footage's, which
;; the analysis already names.
"plate-pos" [:anchor-avg]
"plate-rot" [:anchor-avg]
"plate-scale" [:anchor-avg]
"eyes" [:anchor-avg :contour-avg :eye-verts :lash-weight] "eyes" [:anchor-avg :contour-avg :eye-verts :lash-weight]
"iris-pos" [:anchor-avg :contour-avg :gaze-gain :gaze-step] "iris-pos" [:anchor-avg :contour-avg :gaze-gain :gaze-step]
"brows" [:anchor-avg :contour-avg :brow-verts :brow-gain :brow-step :brow-weight] "brows" [:anchor-avg :contour-avg :brow-verts :brow-gain :brow-step :brow-weight]

Some files were not shown because too many files have changed in this diff Show more