diff --git a/docs/animation-model.md b/docs/animation-model.md index 33a6ab5..f9c7e8a 100644 --- a/docs/animation-model.md +++ b/docs/animation-model.md @@ -14,7 +14,7 @@ primitive, scalar — each with its own source, vocabulary and interpolation. Th table is a good description of **where data comes from** and a bad description of **what data is**, and the current code follows it too literally: eyes, brows, teeth and mouth each get their own build function, their own key shape and their -own path through `take.js`. +own path through the prototype's writer. They are all one thing. A part is a **node** with **channels**, and the five kinds collapse into differences of which channels exist and who filled them in. @@ -164,6 +164,43 @@ This is what `docs/design.md` means by an override layer, and it is why re-freezing is safe: the base is regenerated, the layers are untouched. It is Blender's NLA blending and AE's effect stack at one property. +Layers are what "set it by hand" means for anything measured, and the measured +channel does not need to know. A hand-set gaze is an `:over` on +`[:xform :pos]` of the iris; a hand-set mouth shape is an `:over` on +`[:geom :pts]`. Turning the gaze-step or gaze-dwell knob regenerates the base and +leaves the correction alone, which is the entire reason a correction is stored as +a layer rather than written into the track. + +**A `:replace` layer overrides absence, an `:offset` layer does not.** Sampling a +channel is: read the base, then apply the layers — and the base coming back +`absent` does not short-circuit that. `:replace` is explicitly for the frame +where detection failed, so it has to be able to supply a value where there is +none; `:offset` is a delta, and there is nothing to nudge, so an offset over an +absent base stays absent. Implemented the obvious way — bail out on absence +before reaching the layers — the one case the feature exists for is the one case +it would not cover. + +### One signal, two nodes + +Gaze is deliberately **one measurement shared by both eyes**: at this size the +per-eye difference is noise, and independent noise reads as wall-eyed +immediately, which is the most expensive artefact on a face. But it is stored as +`[:xform :pos]` on `:iris-r` and on `:iris-l`, which are two channels on two +nodes with two different parents — so the invariant lives in `measure` and +nothing in the document enforces it. + +That matters as soon as either one can be overridden by hand, because an `:over` +on one iris alone reproduces exactly the artefact the shared measurement exists +to prevent. Until drivers exist, **the override is on both or on neither**, and +that is a rule the UI has to keep rather than one the data can. + +This is the case that will eventually justify **drivers** — one value, evaluated +once, feeding several channels — which is why gaze is named in Deferred as the +obvious first one. Nothing here forecloses it: a driver needs a place in the +document and a `:driven-by` on a channel, both of which are additive, and an +absent key means "not driven". So it stays deferred, and the shape does not have +to change to allow it. + ## Transform: decomposed, never a matrix ```clojure @@ -194,6 +231,101 @@ of thing that makes a parenting feature feel broken. `[:xform :pos]` with no conversion. The analysis output and the animation model meet without an adapter, which is a sign the decomposition is the right one. +## What space geometry is in + +**`[:geom :pts]` is always in the node's own local space, and the transform +chain says what that means.** There is no global geometry space and no decision +to make about one. + +| Node | Its local space | Why that one | +| --- | --- | --- | +| a rotoscoped feature | head-local, isotropic, unit = one image height | what the anchor fit already produces; the `xform` to raster is not applied and not stored | +| a painted cel | the stage, in pixels, grid-snapped | the artist is placing pixels, so the pixel grid is the thing being authored | +| a primitive under a feature | its parent's | the iris is positioned on the lid ring, not on the stage | + +This looks like a small clarification and it removes a whole class of argument. +The prototype bakes the framing into the numbers: `toRasterRing` applies +`makeXform`, which centres on the face oval's bounding box and zooms until the +face is 80% of the raster height, so **every stored vertex carries a cropping +decision** that was made once, at analysis time, from one frame's landmarks. +Dropping that step is a deletion, not a feature, and after it the framing is +simply a transform on a node. + +Grid snapping belongs to the cel and not to the roto, for the same reason: a cel +is authored on the grid and a traced contour is not. So it is a property of a +node's space rather than a rule about all geometry, and the tension between +"integer polygons" and "arbitrary placement" was never real. + +Each dense block therefore carries its own **fixed-point scale** in its header, +because a block in image-height units and a block in stage pixels need different +ones to fill an `Int16` usefully. + +### There is no camera node + +A camera is a global transform over everything, and nothing here wants one. +Placing the face on the stage is a transform on a node, which already exists; +what is not on the stage hangs off the edges and the canvas clips it. Every fill +in `domain/raster` already clamps rather than assuming it is inside, so drawing +past the edge is not a feature to add. + +Project dimensions are therefore **independent of the footage**. A 1440x1920 +portrait clip composited onto a 320x200 stage is not a problem to solve — the +head is placed and scaled where it belongs and the rest of the frame is simply +not on stage. The full frame stays *available* for tracing without being +*visible*, and those are different requirements. + +## The anchor: stabilisation is a channel, not a mode + +`stabilize` produces `{s, θ, tx, ty}` per frame, which is exactly +`[:xform :scale]`, `[:xform :rot]` and `[:xform :pos]`. So removing the head's +motion is not a pipeline setting — it is a question of **which node holds that +motion**, and the answer is one channel definition: + +```clojure +;; locked: the head sits still, for tracing and for judging articulation +[:xform :pos] {:animated? false :value [0.0 0.0]} + +;; as filmed: the head moves around the stage +[:xform :pos] {:animated? true :interp :hold + :dense {:store "sha256:…" :stride 2 :frames 600} + :generated {:by :anchor/similarity}} + +;; per plate: the head snaps at each selected frame and holds +[:xform :pos] {:animated? true :interp :hold :keys {0 […], 12 […], 23 […]}} +``` + +The three modes are the three channel shapes, on one channel, on one node. The +third is the one a plate strip wants — the head pose is stable for exactly as +long as a drawing is on screen — and it costs nothing because `:keys` already +exists. Its frame set is the kept-frame set, which is `suggestPlateFrames` in the +prototype and belongs to painting rather than to measurement. + +**Always measure, always store factored, toggle the parent.** The fit is computed +and the geometry is stored head-local in every mode, and only the parent's +channel changes. Two things downstream require it, and both would be lost by +making this an analysis-time switch: + +- *Smoothing.* "Smooth the transform, never the contour" only means anything + while the two are separate. +- *Key selection.* A velocity minimum is "articulation paused" in head-local + space and "the head happened to be still" in image space. + +It also makes the toggle an edit to the document rather than a reason to +re-analyse: tier 1, undoable, syncable, and instant. + +### Two nodes, because two different things want that transform + +``` +:face group — AUTHORED. where the face sits on the stage, and how big. + :head group — MEASURED. the head's motion, or identity. + :mouth :mouth-in :teeth :lid-r :lid-l :brow-r :brow-l … +``` + +Switching modes rewrites `:head` and never touches `:face`, so it cannot move +something that was placed by hand. A group node is free, and keeping the authored +and the measured transform apart is the whole reason the transform is decomposed +in the first place. + ## Time maps — exposure, lead and symbol timing are one thing Every node may map the frame it is evaluated at: @@ -221,27 +353,79 @@ different rules: *not* to the plate, which is the whole point of it — so the offset genuinely belongs at the node, not the clip. -## Symbols +## Timelines, and why a scene is one -The Flash idea, kept: +A **timeline** is an ordered bag of nodes in its own frame space: ```clojure -:library -{:sym/head {:kind :poly :channels {...}} - :sym/blink {:kind :timeline :frames 3 :nodes [...]}} +{:frames 91 + :palette {...} ; see Palettes + :nodes {id -> node}} ``` -A node with `:symbol :sym/blink` is an **instance**. Its own channels compose -*over* the symbol's, so one definition can be placed many times and tinted, -offset or retimed at each placement. A `:timeline` symbol has its own frame space, -which the instance's `:time` maps into — that is Flash's MovieClip, Lottie's -precomp and AE's pre-comp, and it is how a three-frame blink gets reused at -frames 40, 88 and 200 without copying it. +That is the whole type, and **everything that holds nodes is one of these**: + +- a clip's **scene** is its root timeline, +- a **symbol** in the library is a timeline, +- a node with `:kind :symbol` is an **instance** of one. + +An earlier draft of this document had a scene and a `:kind :timeline` symbol as +two structures with the same fields and never said they were the same thing. +They are. Flash's `_root` is a MovieClip; After Effects' "a pre-comp is just a +layer" is already in the prior-art table above. Collapsing them is what makes +nesting arbitrary and free, rather than a feature to be added. + +### Two axes of nesting, and they are different + +This is the distinction the flat-storage rule is about, and conflating the two is +why "nested" and "flat with parent pointers" sound contradictory when they are +not: + +| Axis | What nests | How it is stored | +| --- | --- | --- | +| **parent / child** | transform composition within one timeline | **flat, with parent pointers** — never nested maps | +| **instance** | a timeline inside another timeline | by reference into the library | + +Each timeline is flat. Timelines nest. Every argument for flat storage — +addressability, one-field reparenting, structural sharing, per-node sync leaves — +is about the first axis and is untouched by the second. + +The instance boundary is also **the only place the frame space changes.** Within +a timeline, `:time` is exposure and lead: a shift inside one space. At an +instance it is `(f - at)·rate + in`, into a different one. That is why `:rate` is +meaningless on an ordinary node and why sampling one must fail loudly rather than +be ignored. + +### What is scoped to a timeline + +Three fields on a node only have meaning relative to a timeline, and the answer +for all three is the same — **their own**: + +- **`:z`** orders among siblings; a node cannot interleave with nodes inside a + nested instance. The instance occupies one position in its parent's order and + its contents sort beneath it, which the z path gives for free by being a + vector. +- **`:stencil`** names a node in the same timeline. A colour key does not + naturally respect a boundary — it is just pixels — so this is a rule rather + than a consequence, and it is Flash's rule for masks. +- **`:span`** is in the parent node's frame space. + +### Instances + +A node with `:kind :symbol` and `:of :sym/blink` places one. Its own channels +compose *over* the symbol's, so one definition is placed many times and tinted, +offset or retimed at each placement — that is how a three-frame blink is reused +at frames 40, 88 and 200 without copying it. This is also where `docs/design.md`'s "closed vocabulary is right for the head" -lands: a plate library is a set of `:sym/head-*` definitions, and the strip -chooses which instance is placed on which frame. The take format's `plate=` field -becomes an instance reference. +lands: a plate library is a set of `:sym/head-*` timelines, and the strip chooses +which is instanced on which frame. + +**Cursors and point buffers are per-instance, not per-node.** Two instances of +one symbol sit at different frames in their own space, so they cannot share a +reading head over the same channel. The resolver keys its caches by the instance +path, not by node id — which is a detail of `Making it fast` below, and the one +place symbol nesting is not free. ## Evaluating a frame @@ -266,6 +450,26 @@ becomes an instance reference. The op list is the boundary with stage 7 in `docs/architecture.md`: the rasteriser takes ops and knows nothing about nodes, channels or time. +**A photographic underlay is not an op.** The registered source frame that an +animator traces over is a reference, not output, and it may not enter the indexed +buffer — the same rule `docs/architecture.md` already sets for handles and +vertex boxes. It is a `drawImage` at an affine on a separate canvas, which clips +at the canvas edge for free, and the only thing it needs from the model is the +world transform of the node it rides: + +```clojure +(world-of resolver :head) ;; -> Float64Array[6] +``` + +Composed with image-pixels-to-local — **both axes divided by `imgH`**, never by +their own dimension — the photo is registered with the shapes by construction, +and an unregistered underlay is merely decorative. Which frame it shows, the +current one or the held plate frame, is a UI choice and not a stored one. + +A photo that has to sit *between* two drawn layers is the case that would make it +a `:bitmap` node with an op of its own. Nothing wants that yet: a reference is +either under everything or over everything at low alpha. + ### Making it fast in CLJS Three things, and only these three matter: @@ -278,8 +482,16 @@ Three things, and only these three matter: on a seek. This is the difference between a `rsubseq` allocation per channel per frame and none. - **Preallocated point buffers per node.** Fixed topology means the size is known - at freeze time, so a frame allocates nothing. At 30fps, per-frame allocation is - the only thing that will make this stutter. + at freeze time, so the vertices — the overwhelming majority of the per-frame + bytes — are written into a buffer the node already owns. A frame still + allocates its op maps and the sorted op vector; that is a dozen small objects + against hundreds of points, and pooling them would buy nothing and cost the + ability to pass an op list around as plain data. At 30fps, per-vertex + allocation is the thing that will make this stutter. + + Because the buffers are reused, **ops must be consumed before the next frame is + asked for.** That is the contract the rAF loop wants anyway: it reads, blits, + and dispatches nothing. ### What is in app-db, and what is not @@ -309,6 +521,9 @@ Proof that it covers what exists, not just what is wanted: | square pupil | node `:pupil-r`, `:kind :rect`, parent `:iris-r`, stencil `:iris-r` | | brow ring + quantised raise | node `:brow-r`, `[:geom :pts]` dense (the traced ring with height removed), `[:xform :pos]` dense (the quantised raise). **The decomposition design.md insists on is two channels.** | | head plate, kept frames | node `:head`, `:symbol` per instance, keys on `[:symbol]` at kept frames | +| `makeXform` face-oval crop | **gone.** Placement is `[:xform :*]` on `:face`; the stage clips | +| `stabilize` transforms | `[:xform :*]` on `:head` — framed identity, dense, or keyed at kept frames | +| registered underlay | not data — a UI layer riding `(world-of resolver :head)` | | painted background cel | node per layer, `[:geom :pts]` **framed**, `[:style :color]` framed | | `mouth lead` | `:time {:offset k}` on performance nodes only | | `exposure` | `:time {:expose n}` on the clip root, inherited | @@ -320,21 +535,106 @@ measured *out* and put *back* or the brow moves twice. In this model that is not an argument to remember — it is two channels on one node, and getting it wrong would mean writing the height into both. +## Palettes + +Three levels, and keeping them apart is what makes a palette swap a +**reinterpretation** rather than an edit: + +| Level | Holds | Lives on | +| --- | --- | --- | +| **tone** | which mark this is — `:skin-dark` | `[:style :color]`, a channel on the node | +| **ramp** | what that tone looks like *here* | `:palette`, a channel on the timeline | +| **the ramps** | every named palette | the project | + +A node names a **tone**, never a colour and never a ramp. Which ramp the tone is +read in is decided by the timeline the node is in. So the same drawing reads day +or night without one stored value changing — which is the entire payoff of +indexed colour, and is why `docs/design.md` forbids sampled RGB: once a shape +holds a measured colour there is nothing left to reinterpret. + +Named palettes are **variants over one tone vocabulary**, not arbitrary colour +lists. `:day` and `:night` both define `:skin-dark`; that is what keeps a swap +total and keeps `docs/design.md`'s closed vocabulary closed. A tone the ramp in +scope does not define resolves to the loud magenta, like any other missing index. + +### The scope rule + +`:palette` on a timeline is a channel like any other: + +```clojure +{:frames 91 + :palette {:animated? true :interp :hold :keys {0 :day, 48 :dusk, 72 :night}} + :nodes {...}} +``` + +**Absent means inherit** from the instancing context. **Present means this +timeline's content is read in that ramp, and it travels with the timeline** — a +symbol authored against `:night` stays night wherever it is placed. That is +lexical scope, and deliberately: a character with their own palette is a +character, not a decoration of whichever scene they were dropped into. + +Composition is the same walk as `:time` — down the instance chain, **innermost +set palette wins**. An enclosing timeline's palette therefore applies to +everything inside it that does not set its own, which is adjustment-layer +behaviour with no adjustment layer in it. It is just scope. + +And because it is an ordinary channel, a project switches palette over time with +keys on the root timeline, a child timeline switches on its own, and neither +knows about the other. + +### One index space, partitioned by palette + +A raster is one `Uint8Array` and an index means one colour in it, so two ramps in +one frame cannot both own index 2. The resolution: **the output index space is +the concatenation of the named palettes**, and a tone resolves to +`palette-base + tone-index`. + +Everything downstream is then unchanged — one buffer, one flat table for +`->rgba`, no per-frame palette construction, and an index does not change meaning +between frames, so bakes and thumbnails stay valid. + +Two consequences worth stating rather than discovering: + +- **The limit is real and reachable.** 256 indices over a nine-tone vocabulary is + twenty-eight palettes. Detect it and say so; do not let it arrive as wrapped + colour. +- **It makes the stencil sharper.** A stencil is a colour key, so two nodes + sharing a tone share a stencil — a genuine weakness of the technique. + Partitioning the index space by palette means two nodes in *different* palettes + no longer collide at all, and the resolved stencil picks up whichever index the + stencil node actually drew in. + +### Where it is resolved + +At the op boundary, and nowhere else. `[:style :color]` holds a keyword all the +way through evaluation; the walk carries the palette in scope the same way it +carries the parent transform and the local frame; the op carries a resolved +index. The rasteriser never sees a tone name and the node never sees an index. + +This also means the palette is a **parameter of evaluation**, not a global. The +resolver takes it alongside the store. + ## Format on disk and on the wire Tier 1 is EDN/transit: the node tree, channel definitions, framed values, keys, layers, library. Kilobytes, human-readable, diffable, and leaf-addressable for sync. -Dense blocks are separate content-addressed binaries — `Int16Array` for raster +Dense blocks are separate content-addressed binaries — `Int16Array` for geometry, `Float32Array` for transforms — with a small header naming the channel -path, frame count and stride. +path, frame count, stride, and the **fixed-point scale** of the node-local space +the block is in. Geometry is stored in the node's own space, not in raster space; +see "What space geometry is in". **Not Lottie internally**, despite the property shape being borrowed from it. Lottie has no palette-indexed colour, its shapes are bezier with in/out tangents where these are integer polygons, and its interpolation defaults are the opposite -of what is wanted. It is a good **export** target later, next to the `.take` -writer, and a bad internal format. +of what is wanted. It is a fine thing to write out one day and a bad thing to +store. + +Output is deliberately not specified here. The target is encoding video in the +browser, which touches the op list and nothing above it — a writer consumes +frames, and frames are what stage 7 already produces. ## Deferred @@ -348,4 +648,6 @@ writer, and a bad internal format. - **Instance channel overrides on symbols.** Compose-over is specified; only colour and transform need it at first. - **Constraints and drivers.** Blender's other half. A gaze that aims at a null - object is the obvious first one, and it is a long way off. + object is the obvious first one, and it is a long way off. Until then the one + gaze shared by two iris nodes is a UI rule, not a stored relationship — see + "One signal, two nodes". diff --git a/docs/architecture.md b/docs/architecture.md index e872ce5..a992210 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -14,7 +14,7 @@ decision that is currently made in two places at once. ## What is actually wrong with the current shape The pure modules are fine. `pipeline.js`, `mathutil.js`, `landmarks.js`, -`interior.js`, `raster.js` and `take.js` are functions over data with the +`interior.js` and `raster.js` are functions over data with the reasoning written down next to them, and they port nearly verbatim. `app.js` is the whole problem, and not because it is long. It is long because @@ -26,7 +26,7 @@ six unrelated jobs are braided together in it: - resolving what is on screen at a frame (`plateIndex`, `perfIndex`, `leadIndex`), - rasterising (`renderFrame`, `compositeRender`), - driving the clock (`tick`), -- persisting (`saveCels`, `loadCels`, `exportTake`). +- persisting (`saveCels`, `loadCels`). Every one of those is a different layer, and `rebuild` recomputes all of them whenever any knob moves. That is affordable at one clip and 72 frames and is @@ -90,7 +90,7 @@ Renaming this later costs a day. ## The node, decomposed -`take.js` today gives a part a `parent` and a `clip`, and `parent` is doing +The prototype today gives a part a `parent` and a `clip`, and `parent` is doing nothing except documenting intent — `mouth_in` is already in the same head-local raster space as `mouth`, so composing its transform would be composing identity. The moment a painted cel is attached to a head plate that @@ -100,7 +100,7 @@ two fields have to come apart: | Field | What it does | Wrong to conflate because | | --- | --- | --- | | `:parent` | Transform composition. Child geometry is in parent's local space. | A node can be drawn over its parent without inheriting its motion. | -| `:stencil` | Colour-key clip, the `clip=` of the take format. | The iris is stencilled by the sclera and parented to the lid ring; those are different nodes. | +| `:stencil` | Colour-key clip: write only where the buffer already holds that index. | The iris is stencilled by the sclera and parented to the lid ring; those are different nodes. | | `:z` | Draw order. | Order is authored per scene, not implied by the tree. | A node is then: @@ -125,7 +125,7 @@ data shape that keeps the promise. Each node also gets an optional local transform channel — translate, rotate, scale, squash. That is the thing the current tool cannot express and that the -`slot_mouth` / `scale` / `rot` / `squash` fields in `take.js` are reaching for. +`slot_mouth` / `scale` / `rot` / `squash` fields in the prototype are reaching for. ## The flow, in seven stages @@ -165,7 +165,6 @@ dragging that slider does not re-run the interior extraction. | `toRasterRing`, iris placement at ring slots, `offsetRing` | 6 resolve | | `IndexedRaster`, `drawCel` | 7 palette | | `drawRegistered`, `posterizeInto` | reference only, off to the side | -| `writeTake` | serialization, off the end | Three things in `app.js` currently straddle a boundary, and each straddle is a bug waiting for a bigger project: @@ -212,7 +211,6 @@ src/arthur/ clip.cljs clip entity; the frame-space conversions cel.cljs painted vector layers timeline.cljs sequence: clip placement, qf <-> cf - take.cljs take-file writer (port take.js) flow/ ingest.cljs detect.cljs the MediaPipe boundary, and the only one @@ -237,7 +235,7 @@ src/arthur/ mode/ one ns per tool mode panel/ strip, worksheet, readouts, palette, layers canvas.cljs the one imperative sink - fx/ mediapipe, files, audio, persistence, export + fx/ mediapipe, files, audio, persistence ``` Two rules about this tree. `domain/` may not require `flow/`, and neither may diff --git a/docs/port-plan.md b/docs/port-plan.md index d2c8576..ec27b04 100644 --- a/docs/port-plan.md +++ b/docs/port-plan.md @@ -265,13 +265,30 @@ Port `stabilize` and the lip rings out of `pipeline.js`. Split **condition** (`smoothTransforms`, `smoothContours`) into its own stage so the two smoothing knobs do not re-run measurement. -**Done:** measured numbers match the JS on the synthetic track. +**Done:** measured numbers match the JS on the synthetic track. Note that +parity here is on `stabilize`'s output, not on `toRasterRing`'s — the framing +step is being deleted, not ported. ### 5 — freeze The new module, and the heart of this work: measurements → channels. A dense -`[:geom :pts]` block per node — `Int16Array[frames × verts × 2]`, raster space, -grid-snapped — with `:generated` attached. Fixed topology is what makes this a -rectangular array with no per-frame header. +`[:geom :pts]` block per node — `Int16Array[frames × verts × 2]` in the node's +own local space, with the block's fixed-point scale in its header — plus +`:generated`. Fixed topology is what makes this a rectangular array with no +per-frame header. + +**Do not port `makeXform`.** The prototype bakes the framing into the stored +numbers: it centres on the face oval's bbox and zooms until the face is 80% of +the raster height, so every vertex carries a cropping decision made once from one +frame's landmarks. Dropping it is a deletion. Placement becomes `[:xform :*]` on +an authored `:face` node, the stage clips whatever hangs off, and project +dimensions stop being tied to the footage. See "What space geometry is in" in +`docs/animation-model.md`. + +The anchor transform freezes onto `:head`, one level under `:face`, and the +normalise on/off/per-plate toggle is which of the three channel shapes that node +carries. Always measure and always store factored, whatever the toggle says: +smoothing and velocity-minimum key selection both require the split to exist in +storage. **Done:** the synthetic take plays back as a moving mouth. Full vertical slice. @@ -299,10 +316,15 @@ The parameter UI, as leaf-addressed params in app-db (`clip/:cid/params/:subject/:area` — see below), so the sync layer added later has nothing to retrofit. -### 9 — backend and take export +### 9 — backend Django project, the `clips` app, models for Project/Clip/Footage/Analysis/Leaf, -and project load/save. Port `take.js` — sixty-five lines, and it is the proof the -model serialises. +and project load/save. Round-tripping a project through the server is the proof +the model serialises. + +**Output is not in this plan.** The `.take` writer in `js/take.js` was for +driving an Animator Pro render script and it is not where this is going: the +target is encoding video in the browser, and that is a separate piece of design +nobody should pre-empt by porting the old one. ## Two things to not foreclose diff --git a/frontend/README.md b/frontend/README.md index b4fd966..c2d6cfd 100644 --- a/frontend/README.md +++ b/frontend/README.md @@ -32,6 +32,13 @@ node out/node-tests.js Run them separately if a compile error is in the way. The oracle JSON is generated, not committed. +**Run them through `mise`**, or make sure `mise`'s node is first on PATH. The +oracle imports `js/*.js` directly, and those are ES modules in a directory with +no `package.json`, so node needs the module detection that became default in +20.19. On an older node 20 — an nvm install shadowing the pinned one is the easy +way to get there — every import fails with "Named export not found ... is a +CommonJS module", which reads like the oracle being broken and is not. + ## The app ```sh @@ -42,9 +49,18 @@ Then open **** — with the `/index.html`, not bare `/`. This shadow-cljs does no directory-index resolution, so `/` is a 404 whatever the roots are. -Through step 1 the page is a placeholder on purpose: there is nothing to render -until the data model exists, and a shell built before the model is a shell built -around a guess. Something moves on screen at step 2. +The page is the hand-written scene from step 2, scrubbed by hand. There is +deliberately no clock: the audio clock, the rAF loop, the `::resolver` +subscription and the transport are step 3, and the question this step answers is +whether the data model evaluates correctly, not whether it evaluates at 30fps. +A scrubber answers the first and nothing else, which is what makes a failure +here unambiguous. + +The scene itself is `src/arthur/demo/scene.edn`, and it is not a face. It is the +smallest scene that exercises every mechanism the model claims to have — +inherited exposure, a sparse held `[:xform :pos]`, composition through a group, +rotation about an anchor, a stencil chain, a keyed `[:vis]`, a `:span`, and +fractional z among siblings — chosen so that each one is visible when it breaks. Port 8778 is deliberately not 8777. `python3 serve.py` from the repo root still runs the old JS tool on 8777, and the two are meant to run side by side — that is @@ -69,8 +85,32 @@ the prototype's mistakes into the rewrite and make them permanent. ## Layout ``` -src/arthur/domain/ pure. No re-frame, no DOM, no flow/. +src/arthur/domain/ pure. No re-frame, no DOM, no flow/. +src/arthur/demo.cljs the hand-written scene, read from demo/scene.edn +src/arthur/ui/canvas.cljs the one imperative sink — the only DOM canvas call test/arthur/synth.cljs the synthetic track — test infrastructure, not src test/parity/ the JS oracle harness. Deletable at step 5. public/index.html dev host page. Django replaces it at step 9. ``` + +## Two evaluators, on purpose + +`domain/scene` has both `eval-frame` and `resolver`, and they are not +alternatives: + +- **`(eval-frame scene f store)`** is the specification. Allocating, order-free, + obviously correct. Tests and one-off renders use it. +- **`(resolver scene store)` -> `(fn [f] ops)`** is what playback uses. It caches + the topological order and the z paths, holds a cursor per channel and reuses + one point buffer per node, so a frame allocates the op maps and nothing else. + +Both run the same walk, parameterised by how a channel is read and where its +points are written — two independent implementations of frame evaluation would +drift, and the drift would look like a rendering bug rather than like two +functions disagreeing. What differs between them is exactly the part that can be +wrong, and `scene-test` asserts they agree frame for frame in forward, backward +and random order. + +Because the resolver reuses its buffers, **ops must be rasterised before the +next frame is asked for.** That is the contract the rAF loop wants anyway: it +reads, blits, and dispatches nothing. diff --git a/frontend/public/index.html b/frontend/public/index.html index 3465e6d..b822379 100644 --- a/frontend/public/index.html +++ b/frontend/public/index.html @@ -15,6 +15,22 @@ /* The preview is nearest-neighbour everywhere. A browser that smooths the upscale would misrepresent the look the tool exists to judge. */ canvas { image-rendering: pixelated; } + h1 { font-size: 14px; font-weight: normal; opacity: .5; margin: 0 0 12px; } + .stage { display: block; background: #12141c; } + audio { display: none; } + .transport { margin-top: 12px; width: 640px; } + .transport .row { display: flex; gap: 6px; align-items: center; } + .transport .gap { flex: 1; } + button { + font: inherit; color: var(--fg); background: #1c1f2b; + border: 1px solid #2b3040; padding: 3px 10px; cursor: pointer; + } + button:hover { background: #242836; } + button.on { background: #3a4258; border-color: #556080; } + .scrub { width: 100%; margin: 10px 0 6px; } + .readout { display: flex; gap: 18px; opacity: .55; font-size: 12px; } + .readout .warn { color: #d98f5a; opacity: 1; } + .note { opacity: .35; font-size: 12px; max-width: 640px; } diff --git a/frontend/src/arthur/clock.cljs b/frontend/src/arthur/clock.cljs new file mode 100644 index 0000000..e5a849f --- /dev/null +++ b/frontend/src/arthur/clock.cljs @@ -0,0 +1,99 @@ +(ns arthur.clock + "The audio clock. Lives OUTSIDE app-db, deliberately. + + THE FRAME IS DERIVED FROM THE AUDIO, never counted: + + frame = ⌊currentTime · fps⌋ + + A loop that counted frames and hoped to keep up would drift, and drift against + a voice is the one artefact that cannot be fixed downstream — a lip-sync tool + whose sync wanders is not a lip-sync tool. Deriving instead means a slow frame + DROPS the frames it missed and the next one lands where the audio already is. + The failure mode becomes a visible stutter rather than an invisible slide, and + those are very different bugs to own. + + ½× and ¼× are `playbackRate` and nothing else. The audio slows, `currentTime` + advances proportionally, and the derived frame follows — so slow motion cannot + desync by construction. Implementing rate as a multiplier on a counted frame + would give the picture a rate and the sound another. + + It is outside app-db because the audio element is the source of truth and + copying it into the db every frame would make the db a lagging mirror of + something authoritative elsewhere. What DOES belong in the db is the playhead + as a piece of document state — see events/playback — and that is written from + here, not read by here." + (:require [arthur.domain.node :as node])) + +(defonce ^:private el (atom nil)) + +(defn attach! + "Hand the clock its audio element. Idempotent." + [audio-el] + (reset! el audio-el)) + +(defn element [] @el) + +(defn- clamp [f frames] + (-> f (max 0) (min (dec frames)))) + +(defn frame + "The clip frame the audio is currently on." + [fps frames] + (if-let [a @el] + (clamp (js/Math.floor (* (.-currentTime a) fps)) frames) + 0)) + +(defn playing? [] + (boolean (when-let [a @el] (and (not (.-paused a)) (not (.-ended a)))))) + +(defn rate [] + (if-let [a @el] (.-playbackRate a) 1.0)) + +(defn set-rate! [r] + (when-let [a @el] (set! (.-playbackRate a) r))) + +(defn play! [] + (when-let [a @el] + ;; Returns a promise that rejects if the browser has not seen a gesture yet. + ;; Swallowed: the transport button IS the gesture, so this can only fire on a + ;; programmatic play, where a console error is noise rather than news. + (some-> (.play a) (.catch (fn [_]))))) + +(defn pause! [] + (when-let [a @el] (.pause a))) + +(defn seek! + "Put the audio at the start of frame f. Seeking to the frame's start rather + than its middle keeps `frame` idempotent: seek to f, read back f." + [fps frames f] + (when-let [a @el] + (set! (.-currentTime a) (/ (clamp f frames) fps)))) + +(defn set-loop! + "Wrap at the end instead of stopping. The frame stays derived — `currentTime` + simply returns to zero — so nothing about the sync changes, which is the point + of not counting frames. + + It earns its place at 2x and 4x, where the whole clip is gone in under four + seconds and a profile wants more than that to look at." + [on?] + (when-let [a @el] (set! (.-loop a) (boolean on?)))) + +(defn set-muted! [on?] + (when-let [a @el] (set! (.-muted a) (boolean on?)))) + +(defn duration-frames + "How many frames the audio actually covers, which need not be the clip's + length. Reported rather than assumed: a clip longer than its audio is a + legitimate thing to be told about, not a thing to silently truncate." + [fps] + (when-let [a @el] + (let [d (.-duration a)] + (when (and d (js/isFinite d)) (js/Math.ceil (* d fps)))))) + +(defn exposed-frame + "The frame a clip-level exposure grid holds `f` back onto. The player shows it + as a readout so that `exposure 2` is visibly doing something at the transport + rather than only inside the scene." + [f expose] + (node/expose f expose)) diff --git a/frontend/src/arthur/core.cljs b/frontend/src/arthur/core.cljs index 73d4330..92a8041 100644 --- a/frontend/src/arthur/core.cljs +++ b/frontend/src/arthur/core.cljs @@ -1,19 +1,30 @@ (ns arthur.core - "The app's one entry point. Deliberately almost empty until port-plan step 2: - there is nothing to render until the data model exists, and a shell built - before the model would be a shell built around a guess." - (:require [reagent.dom.client :as rdc])) + "The app's entry point. + + port-plan step 3: the hand-written scene plays at 30fps against audio, scrubs, + and runs at ½× and ¼×." + (:require [arthur.db :as db] + [arthur.events.playback] + [arthur.subs.playback] + [arthur.subs.render] + [arthur.ui.player :as player] + [arthur.ui.shell :as shell] + [re-frame.core :as rf] + [reagent.dom.client :as rdc])) (defonce root (atom nil)) -(defn shell [] - [:main - [:h1 "arthur"] - [:p "Scaffold only. The scene renderer arrives with domain/scene."]]) +(rf/reg-event-db ::init (fn [_ _] db/default)) (defn ^:dev/after-load mount [] - (rdc/render @root [shell])) + ;; A hot reload changes the scene or the rasteriser and not the playhead, so + ;; the loop would otherwise sit on an unchanged frame number and never redraw. + (rf/clear-subscription-cache!) + (player/refresh-subs!) + (rdc/render @root [shell/view])) (defn init [] + (rf/dispatch-sync [::init]) (reset! root (rdc/create-root (js/document.getElementById "app"))) - (mount)) + (mount) + (player/start!)) diff --git a/frontend/src/arthur/db.cljs b/frontend/src/arthur/db.cljs new file mode 100644 index 0000000..bb92312 --- /dev/null +++ b/frontend/src/arthur/db.cljs @@ -0,0 +1,67 @@ +(ns arthur.db + "app-db: authored data and ids. Nothing derived, and nothing large. + + That sounds like hygiene and it is the precondition for two things that are + otherwise unbuildable — spec validation on every event, which is only + affordable over authored data, and cheap writes, since every edit `assoc`es + into this map and every mounted layer-2 sub compares the result. + + So the scene is here (it is a document — a human placed every node) and dense + channel blocks are not; they live behind a handle in `store`. Today the demo + scene has no dense blocks and the store is empty, which is why it is a map and + not yet a namespace." + (:require [arthur.demo :as demo] + [arthur.demo.swarm :as swarm])) + +(def scenes + "Two hand-made clips, selectable from the transport. + + `:swarm` holds its geometry in DENSE blocks behind store handles, which is the + shape freeze produces at step 5 — so the fun one is also the load test." + {:demo {:label "demo" :scene demo/scene :store nil + :fps demo/fps :frames demo/frames} + :swarm {:label "swarm" :scene @swarm/scene :store @swarm/store + :fps swarm/fps :frames swarm/frames}}) + +(def default + {;; --- the document --- + :scene/current :demo + :palette :arthur/default ; a NAME; the ramp itself is project data + + ;; --- the clip --- + :clip {:fps demo/fps + :frames demo/frames} + + ;; --- transport --- + ;; + ;; The playhead is in app-db like everything else. An earlier draft of + ;; docs/architecture.md put it in a standalone ratom to dodge an invalidation + ;; storm that does not happen: with layer-2 extractors and layer-3 + ;; computations, a tick re-runs one cheap extractor per mounted sub, each + ;; returning the same value for every subtree the tick did not touch, and + ;; therefore notifying nobody. ::resolver does not re-run. + ;; + ;; Two reasons it belongs here rather than outside: a seek in the event log is + ;; how scrubbing becomes inspectable in re-frame-10x, and a collaborator's + ;; playhead is a feature — putting it outside app-db puts it outside the + ;; machinery that would share it. + :playback {:frame 0 + :playing? false + :rate 1.0 + ;; Both for profiling: loop so a run at 4x lasts longer than the + ;; clip, mute so sitting in one does not require enduring it. + :loop? false + :muted? false}}) + +(def rates + "The transport's rates — all of them `playbackRate` on the audio element, so + the picture cannot drift from the sound at any of them. + + 2x and 4x are there to be profiled at rather than watched. A 30fps clip at 2x + wants sixty clip frames a second against a 60Hz display, so every animation + frame has to paint a new one: it is the point where the loop stops having + slack. Past that the clock starts dropping frames rather than falling behind, + which is the whole reason the frame is derived from the audio instead of + counted — and the transport reports the drop rate so that it is visible rather + than merely survivable." + [0.25 0.5 1.0 2.0 4.0]) diff --git a/frontend/src/arthur/demo.cljs b/frontend/src/arthur/demo.cljs new file mode 100644 index 0000000..8181d62 --- /dev/null +++ b/frontend/src/arthur/demo.cljs @@ -0,0 +1,26 @@ +(ns arthur.demo + "The hand-written scene from port-plan step 2, and nothing else. + + The EDN is a resource rather than a literal in this file so that the test and + the page read the SAME bytes. If the scene were written twice, the one the test + validates would not be the one that renders, and the model would be validated + against a scene nobody ever looked at." + (:require [arthur.domain.scene :as scene] + [cljs.reader :as reader] + [shadow.resource :as rc])) + +(def source (rc/inline "arthur/demo/scene.edn")) + +(def scene (reader/read-string source)) + +(def width 320) +(def height 200) + +(def fps (:fps scene)) +(def frames (:frames scene)) + +(defn ops-at + "Draw ops for one frame, via the specification path. The page uses + `scene/resolver` instead; this is here for the REPL." + [f] + (scene/eval-frame scene f)) diff --git a/frontend/src/arthur/demo/scene.edn b/frontend/src/arthur/demo/scene.edn new file mode 100644 index 0000000..f358f13 --- /dev/null +++ b/frontend/src/arthur/demo/scene.edn @@ -0,0 +1,93 @@ +;; A scene written by hand, before any analysis exists. +;; +;; port-plan step 2 is deliberately ahead of measurement: the data model has +;; never been validated, and it is worth finding out here, with fifty lines to +;; throw away, rather than after nine hundred lines of measurement have been +;; ported into a shape that does not work. +;; +;; So this is not a demo of a face. It is the smallest scene that exercises every +;; mechanism the model claims to have, chosen so that each one is visible when it +;; breaks: +;; +;; exposure inherited from the clip root the motion steps on 2s +;; a keyed [:xform :pos], sparse, held the card jumps between 4 poses +;; transform composition through a group the eye rides the card +;; rotation about an anchor the card turns, it does not swing +;; a stencil as a colour key the iris cannot leave the card +;; a stencil chain nor can the pupil +;; a keyed [:vis] the bar blinks off and back +;; a :span the bar does not exist at either end +;; fractional z among siblings the bar is behind, the pupil in front +;; +;; Everything is in 320x200 raster space, which is what [:geom :pts] holds. +{:name "step-2 demo" + ;; 229 frames at 30fps is 7.63s, which covers audio.wav (7.601s) with a frame to + ;; spare. fps belongs to the CLIP rather than to the timeline — a timeline has a + ;; frame space, not a rate — and it is here only because there is one clip. + :frames 229 + :fps 30 + + :nodes + {;; The clip root. EXPOSURE LIVES HERE and is inherited, because + ;; docs/design.md is emphatic that everything rides one grid: a head cutting on + ;; odd frames against a mouth cutting on even ones reads as two performances. + ;; Setting it lower on a child is possible and is meant to feel deliberate. + :root + {:id :root :name "clip" :kind :group :parent nil :z "a1" + :time {:mode :map :expose 2}} + + ;; A bar, behind everything, purely to assert that :vis and :span are different + ;; questions. It stops existing outside [6 66) — nothing to hide, nothing to + ;; hold — and inside that range it is switched off between frames 76 and 153. + :bar + {:id :bar :name "bar" :kind :poly :parent :root :z "a0" + :span [19 210] + :channels + {[:vis] {:animated? true :interp :hold :keys {0 true, 76 false, 153 true} :over []} + [:geom :pts] {:animated? false :value [20 168 300 168 300 176 20 176]} + [:style :color] {:animated? false :value :brow}}} + + ;; The group the plan asks for: four sparse keys on [:xform :pos], held. At + ;; exposure 2 the card reads its pose from an even frame, so a key landing on + ;; an odd frame would be seen on the even frame after it — which is the whole + ;; reason exposure is applied before anything else and not folded into keys. + :swing + {:id :swing :name "swing" :kind :group :parent :root :z "a1" + :channels + {[:xform :pos] {:animated? true :interp :hold + :keys {0 [90.0 100.0], 57 [200.0 70.0], 114 [230.0 140.0], 171 [110.0 150.0]} + :over []} + [:xform :rot] {:animated? true :interp :hold + :keys {0 0.0, 57 0.35, 114 0.0, 171 -0.35} + :over []}}} + + ;; The rectangle. Its points are centred on the origin and its :anchor is the + ;; origin too, so :swing's rotation TURNS it rather than swinging it round a + ;; corner — which is the failure mode :anchor exists to prevent. + :card + {:id :card :name "card" :kind :poly :parent :swing :z "a1" + :channels + {[:geom :pts] {:animated? false :value [-44 -30 44 -30 44 30 -44 30]} + [:style :color] {:animated? false :value :skin-base}}} + + ;; A disc stencilled by the card. The stencil is a COLOUR KEY, not a node + ;; reference — it is the take format's clip= — so the iris is written only over + ;; pixels that currently hold the card's index. Push the radius up and it is + ;; cropped by the card's edge rather than spilling, at any position, with no + ;; clamp anywhere. + :iris + {:id :iris :name "iris" :kind :disc :parent :card :stencil :card :z "a2" + :channels + {[:xform :pos] {:animated? false :value [14.0 -6.0]} + [:geom :radius] {:animated? false :value 13.0} + [:style :color] {:animated? false :value :iris}}} + + ;; The pupil is a SQUARE, and three pixels of it. A circle of radius 1.5 is not + ;; a circle, it is a plus sign with the corners gnawed off, and it changes shape + ;; as it moves. Stencilled by the iris, which is itself already cropped by the + ;; card, so the clip composes without the chain being expressed anywhere. + :pupil + {:id :pupil :name "pupil" :kind :rect :parent :iris :stencil :iris :z "a3" + :channels + {[:geom :size] {:animated? false :value 5.0} + [:style :color] {:animated? false :value :pupil}}}}} diff --git a/frontend/src/arthur/demo/swarm.cljs b/frontend/src/arthur/demo/swarm.cljs new file mode 100644 index 0000000..23db5b5 --- /dev/null +++ b/frontend/src/arthur/demo/swarm.cljs @@ -0,0 +1,163 @@ +(ns arthur.demo.swarm + "A hundred and twenty shapes, orbiting, spinning, pulsing and blinking. + + Not useful. It is here because it is the first thing to exercise the DENSE + channel path end to end — typed-array blocks behind a store handle, one value + per frame, read through a cursor — which until now had tests and no traffic. + Step 5 writes exactly this shape out of the freeze module, so it is worth + knowing the resolver can carry it at rate before anything depends on that. + + Everything is generated from deterministic trigonometry rather than from a + random seed: the same scene every load, so a stutter or a wrong pose is + reproducible instead of being a thing that happened once. + + Layout of each block is the rectangular one freeze produces — node-major, + frame-minor, no per-frame header and no indirection: + + offset(node i) = i · frames · stride + value(i, f) = data[offset(i) + f · stride]" + (:require [arthur.domain.channel :as ch] + [arthur.domain.palette :as pal])) + +(def frames 229) +(def fps 30) +(def n-orbits 6) +(def n-shapes 120) + +(def ^:private TAU (* 2 js/Math.PI)) + +;; Every tone except the background, so the swarm uses the whole ramp. +(def ^:private tones + (vec (remove #{:bg} (map :name pal/entries)))) + +(defn- regular-poly + "A closed n-gon about the origin, flat in [x0 y0 x1 y1 …] — the same layout a + dense block holds, which is the point of geometry being flat everywhere." + [n radius phase] + (vec (mapcat (fn [k] + (let [a (+ phase (/ (* TAU k) n))] + [(* radius (js/Math.cos a)) + (* radius (js/Math.sin a))])) + (range n)))) + +;; --------------------------------------------------------------------------- +;; the dense blocks + +(defn- fill-block! + "Write one node's frames into a node-major block." + [^js data i stride f->vals] + (let [base (* i frames stride)] + (dotimes [f frames] + (let [vs (f->vals f) + o (+ base (* f stride))] + (dotimes [k stride] + (aset data (+ o k) (nth vs k))))))) + +(defn- orbit-blocks [] + (let [pos (js/Float32Array. (* n-orbits frames 2)) + rot (js/Float32Array. (* n-orbits frames 1))] + (dotimes [i n-orbits] + (let [ph (/ (* TAU i) n-orbits) + ;; Lissajous, so the six orbits drift in and out of phase with each + ;; other instead of marching in step. + wx (+ 0.011 (* 0.004 (mod i 3))) + wy (+ 0.017 (* 0.003 (mod i 4))) + spin (* 0.008 (if (even? i) 1 -1) (inc (mod i 3)))] + (fill-block! pos i 2 + (fn [f] [(+ 160 (* 104 (js/Math.sin (+ (* f wx) ph)))) + (+ 100 (* 64 (js/Math.sin (+ (* f wy) (* 1.7 ph)))))])) + (fill-block! rot i 1 (fn [f] [(* f spin)])))) + {"swarm/orbit-pos" {:data pos :state nil} + "swarm/orbit-rot" {:data rot :state nil}})) + +(defn- shape-blocks [] + (let [pos (js/Float32Array. (* n-shapes frames 2)) + rot (js/Float32Array. (* n-shapes frames 1)) + scale (js/Float32Array. (* n-shapes frames 2)) + ;; The state mask: a handful of shapes wink out entirely for a stretch. + ;; ABSENT, not hidden — this is the mask meaning "there is no value on + ;; this frame", which is what an occluded subject will mean at step 6. + state (js/Uint8Array. (* n-shapes frames))] + (dotimes [i n-shapes] + (let [ph (/ (* TAU i) n-shapes) + ring (+ 18 (* 26 (js/Math.abs (js/Math.sin (* 2.3 ph))))) + wob (+ 0.03 (* 0.02 (mod i 5))) + spin (* (if (zero? (mod i 3)) -1 1) (+ 0.02 (* 0.011 (mod i 7)))) + pulse (+ 0.05 (* 0.013 (mod i 6)))] + (fill-block! pos i 2 + (fn [f] + ;; Orbit position plus a small independent wobble, so no + ;; two neighbours trace the same path. + (let [a (+ ph (* f 0.014 (if (even? i) 1 -1)))] + [(+ (* ring (js/Math.cos a)) (* 5 (js/Math.sin (* f wob)))) + (+ (* ring (js/Math.sin a)) (* 5 (js/Math.cos (+ 1.1 (* f wob)))))]))) + (fill-block! rot i 1 (fn [f] [(+ ph (* f spin))])) + (fill-block! scale i 2 + (fn [f] + (let [s (+ 1.0 (* 0.45 (js/Math.sin (+ ph (* f pulse)))))] + [s s]))) + ;; Every eleventh shape is absent for a window that moves with i. + (when (zero? (mod i 11)) + (let [from (mod (* i 9) frames) + to (min frames (+ from 34))] + (doseq [f (range from to)] + (aset state (+ (* i frames) f) ch/absent-bit)))))) + {"swarm/pos" {:data pos :state state} + "swarm/rot" {:data rot :state nil} + "swarm/scale" {:data scale :state nil}})) + +;; --------------------------------------------------------------------------- +;; the nodes + +(defn- dense [store i stride] + {:animated? true :interp :hold + :dense {:store store :offset (* i frames stride) :stride stride :frames frames} + ;; Provenance, which nothing in the renderer reads. Here it is honest about + ;; where these numbers came from, the same way :roto/lips-outer will be. + :generated {:by :demo/swarm} + :over []}) + +(defn- orbit-node [i] + {:id (keyword (str "orbit-" i)) :kind :group :parent :root + :z (str "b" i) + :channels {[:xform :pos] (dense "swarm/orbit-pos" i 2) + [:xform :rot] (dense "swarm/orbit-rot" i 1)}}) + +(defn- shape-node [i] + (let [orbit (keyword (str "orbit-" (mod i n-orbits))) + tone (nth tones (mod i (count tones))) + kind (case (mod i 7) 5 :disc 6 :rect :poly) + verts (+ 3 (mod i 10)) + size (+ 3.5 (* 0.9 (mod i 8))) + base {:id (keyword (str "s-" i)) :kind kind :parent orbit + ;; Fractional index among siblings. Zero-padded so the strings + ;; sort the way the numbers do — "c9" would otherwise land after + ;; "c10", which is the classic way a z order goes subtly wrong. + :z (str "c" (.padStart (str i) 4 "0")) + :channels {[:xform :pos] (dense "swarm/pos" i 2) + [:xform :rot] (dense "swarm/rot" i 1) + [:xform :scale] (dense "swarm/scale" i 2) + [:style :color] (ch/framed tone)}}] + (update base :channels merge + (case kind + :poly {[:geom :pts] (ch/framed (regular-poly verts size (* 0.3 i)))} + :disc {[:geom :radius] (ch/framed (* 0.75 size))} + :rect {[:geom :size] (ch/framed (js/Math.round size))})))) + +(def store + (delay (merge (orbit-blocks) (shape-blocks)))) + +(def scene + (delay + {:name "swarm" + :frames frames + :fps fps + :nodes + (into {:root {:id :root :kind :group :parent nil :z "a1" + ;; On 2s, like everything else. A hundred and twenty shapes + ;; cutting on one grid reads as animation; the same shapes on + ;; their own grids read as a screensaver, which is the whole + ;; argument for exposure inheriting strictly. + :time {:mode :map :expose 2}}} + (concat (map (juxt :id identity) (map orbit-node (range n-orbits))) + (map (juxt :id identity) (map shape-node (range n-shapes)))))})) diff --git a/frontend/src/arthur/domain/channel.cljs b/frontend/src/arthur/domain/channel.cljs new file mode 100644 index 0000000..4b0bf03 --- /dev/null +++ b/frontend/src/arthur/domain/channel.cljs @@ -0,0 +1,288 @@ +(ns arthur.domain.channel + "A channel is one animatable property, sampled at a frame. + + Three shapes, and the uniformity across them is the entire point of the model + — analysis does not produce a different kind of data, it produces keys densely + on the same channels a hand fills in sparsely: + + FRAMED {:animated? false :value v} + A thing that simply exists. A painted background cel is this. + + KEYED {:animated? true :interp :hold :keys {0 v, 4 v, 12 v}} + Sparse, authored, in the document. Undoable and syncable. + + DENSE {:animated? true :interp :hold + :dense {:store \"sha256:…\" :offset 0 :stride 40 :frames 600} + :generated {...}} + Generated, one value per frame, in a typed array outside app-db. + + `value-at` reads all three and is the specification. `cursor`/`sample!` is the + fast path for playback and must agree with it exactly; scene-test asserts that + across forward, backward and random frame order, because a cursor that drifts + is a bug you would see as the wrong pose rather than as an error. + + KEYS ARE A MAP BY FRAME, NEVER A VECTOR, and the map stored in the document is + a PLAIN map — transit and JSON both lose sortedness, so the sorted index is + built here at read time and never persisted. + + :generated is provenance. NOTHING IN HERE READS IT, and nothing downstream may: + it exists so the UI can offer a parameter panel instead of raw keys. It lives + on the channel rather than the node because a mouth wants a rotoscoped + [:geom :pts] and a hand-animated [:xform :pos] at the same time, and putting + the flag on the node would forbid the most useful thing in the model." + (:require [clojure.string :as str])) + +;; --------------------------------------------------------------------------- +;; the state mask +;; +;; PRESENCE IS NOT VISIBILITY, and the distinction is free now and expensive to +;; retrofit. A part that is hidden EXISTS and is not drawn, which is `[:vis]`, a +;; channel like any other. A subject that is occluded has NO VALUE on that frame +;; — there is nothing to hide and nothing to fall back on — and that is this. +;; +;; The mask carries ABSENCE ONLY. An earlier draft gave it a hidden bit as well, +;; per docs/architecture.md's "hidden flag + palette index", and that bit was +;; simply a dense `[:vis]` wearing a different hat: two mechanisms for one +;; question, which is how you end up with a part that is hidden by one and shown +;; by the other. Hiding is a channel; absence is a state. Remaining bits are +;; reserved. + +(def ^:const present 0) +(def ^:const absent-bit 1) + +(def absent + "Sampled value for a frame the subject was not on. + + Distinct from a part being switched off, which is `[:vis]` being false, and + distinct from a part having no keys. The identity tracker, when it arrives, + needs somewhere to say \"not on screen\" without inventing a pose." + ::absent) + +(defn nothing? + "True when there is no value to draw with." + [v] + (identical? v absent)) + +;; --------------------------------------------------------------------------- +;; constructors, for hand-written scenes and tests + +(defn framed [v] {:animated? false :value v}) + +(defn keyed + ([ks] (keyed ks :hold)) + ([ks interp] {:animated? true :interp interp :keys ks :over []})) + +;; --------------------------------------------------------------------------- + +(defn component + "Component i of a multi-component channel value. + + An authored value is a CLJS vector; a value read out of a dense block is a + typed-array view over the block, because copying it would allocate per node + per frame. Both have to read the same way here or every consumer downstream + grows the same two-way branch." + [v i] + (if (vector? v) (-nth v i) (aget v i))) + +(defn frames + "Sorted vector of the frames a keyed channel has keys on, or nil. Built here + and cached by `cursor`; `value-at` rebuilds it, which is why `value-at` is the + specification and not the playback path." + [ch] + (when-let [ks (:keys ch)] + (vec (sort (keys ks))))) + +(defn- check-unimplemented! + "An override layer or a retimed symbol instance must fail LOUDLY rather than + be ignored. + + Both are specified in docs/animation-model.md and neither is built yet + (port-plan scope: \"leave the :over field present and empty; leave :symbol out + entirely\"). Silently dropping an :over layer would present as a hand + correction that did not take — a correction the user made once, watched fail, + and has no reason to trust again. Nothing can produce one yet, so this can + only fire on a data shape that has run ahead of the code." + [ch] + (when (seq (:over ch)) + (throw (ex-info "channel has :over layers and the override layer is not built (port-plan step 2 scope)" + {:over (:over ch) :channel (dissoc ch :dense)})))) + +;; --------------------------------------------------------------------------- +;; dense blocks + +(defn- dense-state + [state f] + (if (nil? state) + present + (aget state f))) + +(defn dense-at + "Read frame f out of a dense block. + + `store` is {store-key -> {:data :state }}, + tier 2, behind a handle and never in app-db. + + The frame is CLAMPED into the block. A time map with an offset deliberately + reads the future — mouth lead is the whole reason `:offset` exists — so the + last frame of a leading track is asked for a frame past the end on every one + of the last `lead` frames. Clamping there is what the JS `shiftIndex` does and + it is the right answer: the track holds its final pose. Returning nothing + instead would blank the mouth at the end of every take. + + stride 1 yields a number; anything wider yields a SUBARRAY VIEW over the + block, not a copy. Fixed topology is what makes that possible — the frame's + data is a rectangular slice at a known offset with no per-frame header." + [{:keys [store offset stride] nf :frames} f st] + (let [{:keys [data state]} (get st store)] + (when (nil? data) + (throw (ex-info "dense channel's store key is not in the store" + {:store store :have (vec (sort (map str (keys st))))}))) + (let [f (-> f (max 0) (min (dec nf))) + sm (dense-state state f)] + (cond + (pos? (bit-and sm absent-bit)) absent + :else + (let [o (+ offset (* f stride))] + (if (= 1 stride) + (aget data o) + (.subarray data o (+ o stride)))))))) + +;; --------------------------------------------------------------------------- +;; the specification + +(defn- keyed-at + "The most recent key at or before f, CLAMPED to the first key below it. + + Hold is the default and clamping at the low end is the JS `activeKey`'s + behaviour, kept: a channel's first key is the pose the part starts in, so a + frame before it reads that pose rather than having no value. This is not the + same question as presence — a part with no value at all is `absent`, which is + a state bit, not an empty key map." + [ks f] + (let [fr (sort (keys ks))] + (if-let [hit (last (take-while #(<= % f) fr))] + (get ks hit) + (get ks (first fr))))) + +(defn value-at + "Sample a channel at frame f. THE SPECIFICATION — correct, allocating, and + O(n) in the keys. `cursor`/`sample!` is what playback uses." + ([ch f] (value-at ch f nil)) + ([ch f store] + (check-unimplemented! ch) + (cond + (not (:animated? ch)) (:value ch) + (:dense ch) (dense-at (:dense ch) f store) + (:keys ch) (let [ks (:keys ch)] + (if (empty? ks) absent (keyed-at ks f))) + :else + (throw (ex-info "animated channel has neither :keys nor :dense" {:channel ch}))))) + +;; --------------------------------------------------------------------------- +;; the playback path +;; +;; Playback is SEQUENTIAL, so "the most recent key at or before f" is an advance +;; of a saved index rather than a search. The difference at 30fps is a `sort` and +;; a `take-while` allocation per channel per frame against none, which is the +;; difference between the model being usable and being a demo. + +(defn- bsearch + "Largest index i with ks[i] <= f, or 0 when f precedes every key (hold clamps + low, see keyed-at)." + [ks f] + (loop [lo 0, hi (dec (count ks)), best 0] + (if (> lo hi) + best + (let [mid (bit-shift-right (+ lo hi) 1)] + (if (<= (nth ks mid) f) + (recur (inc mid) hi mid) + (recur lo (dec mid) best)))))) + +(deftype Cursor [ch ks store ^:mutable i] + Object + (toString [_] (str "#Cursor{" (pr-str (if ks :keyed (if (:dense ch) :dense :framed))) " i=" i "}"))) + +(defn cursor + "A reading head on one channel. Build once per channel per resolver, then + `sample!` it per frame. Holds the sorted key index, which is why the index is + built here and not in the document." + ([ch] (cursor ch nil)) + ([ch store] + (check-unimplemented! ch) + (->Cursor ch (when (and (:animated? ch) (not (:dense ch)) (seq (:keys ch))) + (frames ch)) + store 0))) + +(defn sample! + "Value of the cursor's channel at f. O(1) when f is at or one key past where + the cursor already sits — the playback case — and O(log n) otherwise, which is + a seek. Advancing and seeking are deliberately different costs: a scrub can + afford a binary search and a frame cannot." + [^Cursor cur f] + (let [ch (.-ch cur) + ks (.-ks cur)] + (cond + (not (:animated? ch)) (:value ch) + (:dense ch) (dense-at (:dense ch) f (.-store cur)) + (nil? ks) absent ; animated with an empty key map + :else + (let [n (count ks) + i (.-i cur) + last (dec n) + i' (cond + ;; still inside the key the cursor sits on + (and (<= (nth ks i) f) + (or (= i last) (> (nth ks (inc i)) f))) + i + ;; the next one — one frame of playback crossed one key + (and (< i last) + (<= (nth ks (inc i)) f) + (or (= (inc i) last) (> (nth ks (+ i 2)) f))) + (inc i) + + :else (bsearch ks f))] + (set! (.-i cur) i') + (get (:keys ch) (nth ks i')))))) + +;; --------------------------------------------------------------------------- + +(defn describe + "Which of the three shapes, for error messages and the parameter panel." + [ch] + (cond + (not (:animated? ch)) :framed + (:dense ch) :dense + :else :keyed)) + +(defn problems + "Human-readable reasons this map is not a channel. Empty means it is one." + [ch] + (cond-> [] + (not (map? ch)) + (conj "not a map") + + (and (map? ch) (not (contains? ch :animated?))) + (conj ":animated? is required — the flag is what makes framed and keyed one type") + + (and (map? ch) (:animated? ch) (not (or (:keys ch) (:dense ch)))) + (conj "animated but has neither :keys nor :dense") + + (and (map? ch) (:animated? ch) (:keys ch) (:dense ch)) + (conj "has both :keys and :dense; a channel is one shape at a time") + + (and (map? ch) (:keys ch) (not (map? (:keys ch)))) + (conj (str ":keys is a " (if (vector? (:keys ch)) "vector" "non-map") + " — keys are a MAP by frame, so a merge can be per-key")) + + (and (map? ch) (:keys ch) (map? (:keys ch)) (not (every? number? (keys (:keys ch))))) + (conj ":keys has a non-numeric frame") + + (and (map? ch) (:animated? ch) (not (#{:hold nil} (:interp ch)))) + (conj (str ":interp " (:interp ch) " — only :hold is implemented; docs/design.md" + " requires hold of every cut part and tweening reads as puppet software")) + + (and (map? ch) (seq (:over ch))) + (conj ":over layers are not implemented (port-plan step 2 scope)"))) + +(defn problems-str [ch] + (str/join "; " (problems ch))) diff --git a/frontend/src/arthur/domain/node.cljs b/frontend/src/arthur/domain/node.cljs new file mode 100644 index 0000000..b338329 --- /dev/null +++ b/frontend/src/arthur/domain/node.cljs @@ -0,0 +1,257 @@ +(ns arthur.domain.node + "A node is an instance in the scene: what kind of mark it is, who it hangs off, + what clips it, where it sits in draw order, and a bag of channels. + + The tree is stored FLAT, WITH PARENT POINTERS, never as nested maps. Four + reasons that all point the same way: any node is addressable without a walk; + reparenting is a one-field write rather than a subtree move; an edit to a leaf + does not change the identity of its ancestors, so re-frame's structural sharing + keeps ancestor subs from invalidating; and it is what lets every node be its own + sync leaf. Flash, Blender and After Effects all store it this way. + + Transforms are DECOMPOSED for storage and FLAT AND MUTABLE for evaluation, and + the two forms are allowed to differ. Decomposed because each component has to + be independently keyframable — that is what channels are for — and because + interpolating matrix entries is meaningless: a rotation tweened through its + matrix shears on the way. Flat Float64Array for evaluation because at 30fps + per-frame allocation is the only thing that will make this stutter." + (:require [arthur.domain.channel :as ch] + [clojure.string :as str])) + +(def kinds + "`:symbol` and `:bitmap` are in the vocabulary and not implemented; they are + here so that a scene that names one fails as \"not implemented\" rather than as + \"not a kind\"." + #{:poly :disc :rect :group :bitmap :symbol}) + +(def implemented-kinds #{:poly :disc :rect :group}) + +(def xform-paths + "In composition order, which is also the order they have to be sampled in. + + :skew and :anchor are in here although nothing drives either yet. A + decomposition is not extensible after the fact: adding a component later means + migrating every stored transform, so both are in the shape and in the + composition order from the start." + [[:xform :pos] [:xform :rot] [:xform :scale] [:xform :skew] [:xform :anchor]]) + +(def valid-paths + "The set of valid channel paths follows from the node's :kind, and that is a + SPEC rather than a schema migration — a node does not grow or lose fields, it + simply has no `[:geom :radius]` unless it is a disc." + (let [base (into #{[:vis]} xform-paths)] + {:group base + :poly (into base [[:geom :pts] [:style :color]]) + ;; A disc's radius is framed in practice (iris size is a knob, not a + ;; performance) but it is a channel like any other so that it can be keyed. + :disc (into base [[:geom :radius] [:style :color]]) + ;; :size, not a radius: the pupil is a SQUARE, and an exactly size x size + ;; block. See raster/fill-rect!. + :rect (into base [[:geom :size] [:style :color]])})) + +(def defaults + "The identity transform, as channels. A node's channel map is merged over this, + so a hand-written scene says only what it means to say." + {[:xform :pos] (ch/framed [0.0 0.0]) + [:xform :rot] (ch/framed 0.0) + [:xform :scale] (ch/framed [1.0 1.0]) + [:xform :skew] (ch/framed [0.0 0.0]) + [:xform :anchor] (ch/framed [0.0 0.0]) + [:vis] (ch/framed true)}) + +(defn channels + "The node's channels with the transform defaults filled in." + [n] + (merge defaults (:channels n))) + +;; --------------------------------------------------------------------------- +;; time maps +;; +;; Exposure, mouth lead and a symbol instance's timing are ONE mechanism, and +;; seeing that is what keeps them from being three implementations that disagree +;; at the edges. + +(defn expose + "Hold a frame back onto an exposure grid: 1 = on 1s, 2 = on 2s. Frame 5 at + exposure 2 reads the pose from frame 4. + + FLOOR, NEVER ROUND. Rounding would let an output frame read a pose from the + FUTURE, which is a lead — a separate control, applied after this one, for a + separate reason." + [f n] + (if (and n (> n 1)) (* (js/Math.floor (/ f n)) n) f)) + +(defn local-frame + "Apply a node's time map to the frame it was handed by its parent. + + ORDER IS LOAD-BEARING: expose first, then offset. Flooring onto a grid and + shifting against the clock do not commute — shift first and the floor discards + it on most frames, so the lead slider reads as doing nothing at exposures above + 1, which is indistinguishable from the slider being unwired. + + Composed along the nesting chain, outermost first, by scene/eval-frame. Two + rules fall out and they are different rules: exposure INHERITS STRICTLY, + because a head cutting on odd frames against a mouth cutting on even ones reads + as two performances; offset is PER-NODE by design, because mouth lead applies + to performance nodes and not to the plate, which is the entire point of it." + [n f] + (let [{:keys [mode offset rate] ex :expose :or {mode :inherit}} (:time n)] + (if (= mode :inherit) + f + (do + ;; A symbol instance's (f - at)·rate + in is the third face of this + ;; mechanism and symbols are out of scope. Loud rather than ignored: a + ;; silently dropped rate is a retimed blink playing at the wrong speed, + ;; which looks like a bad blink and not like a missing feature. + (when (and rate (not= rate 1.0) (not= rate 1)) + (throw (ex-info "time map :rate is symbol timing and symbols are not built (port-plan step 2 scope)" + {:node (:id n) :time (:time n)}))) + (cond-> f + ex (expose ex) + offset (+ offset)))))) + +;; --------------------------------------------------------------------------- +;; the transform +;; +;; A 2x3 affine as a 6-element Float64Array [a b c d e f], the canvas convention: +;; +;; | a c e | x' = a·x + c·y + e +;; | b d f | y' = b·x + d·y + f +;; | 0 0 1 | + +(defn mat [] (js/Float64Array. #js [1 0 0 1 0 0])) + +(defn set-identity! [^js m] + (aset m 0 1) (aset m 1 0) (aset m 2 0) (aset m 3 1) (aset m 4 0) (aset m 5 0) + m) + +(defn mul! + "dest := m · n. Reads both fully before writing, so dest may alias either." + [^js dest ^js m ^js n] + (let [a (+ (* (aget m 0) (aget n 0)) (* (aget m 2) (aget n 1))) + b (+ (* (aget m 1) (aget n 0)) (* (aget m 3) (aget n 1))) + c (+ (* (aget m 0) (aget n 2)) (* (aget m 2) (aget n 3))) + d (+ (* (aget m 1) (aget n 2)) (* (aget m 3) (aget n 3))) + e (+ (* (aget m 0) (aget n 4)) (* (aget m 2) (aget n 5)) (aget m 4)) + f (+ (* (aget m 1) (aget n 4)) (* (aget m 3) (aget n 5)) (aget m 5))] + (aset dest 0 a) (aset dest 1 b) (aset dest 2 c) + (aset dest 3 d) (aset dest 4 e) (aset dest 5 f) + dest)) + +(defn local! + "dest := T(pos) · T(anchor) · R(rot) · K(skew) · S(scale) · T(-anchor) + + Written out closed-form rather than as five matrix products, because this runs + per node per frame and the five products would each allocate. The derivation, + so the constants are checkable rather than trusted: + + R·K·S = | c -s | · | 1 kx | · | sx 0 | + | s c | | ky 1 | | 0 sy | + + K·S = | sx kx·sy | + | ky·sx sy | + + R·K·S = | sx(c - s·ky) sy(c·kx - s) | + | sx(s + c·ky) sy(s·kx + c) | + + and the translation is anchor + pos - M·anchor, which is what makes rotation + and scale happen ABOUT the anchor. :anchor is Flash's registration point and + Blender's origin, and getting it wrong is why hand-placed parts swing rather + than turn. + + :skew is stored as shear FACTORS, not angles — kx is x gained per unit y — so + that the identity is 0 and a decomposition round-trips without a tangent." + [^js dest pos rot scale skew anchor] + (let [c (js/Math.cos rot) + s (js/Math.sin rot) + sx (ch/component scale 0) + sy (ch/component scale 1) + kx (ch/component skew 0) + ky (ch/component skew 1) + ax (ch/component anchor 0) + ay (ch/component anchor 1) + a (* sx (- c (* s ky))) + b (* sx (+ s (* c ky))) + cc (* sy (- (* c kx) s)) + d (* sy (+ (* s kx) c))] + (aset dest 0 a) + (aset dest 1 b) + (aset dest 2 cc) + (aset dest 3 d) + (aset dest 4 (+ ax (ch/component pos 0) (- (+ (* a ax) (* cc ay))))) + (aset dest 5 (+ ay (ch/component pos 1) (- (+ (* b ax) (* d ay))))) + dest)) + +(defn pinv + "The parent-inverse, captured at the moment of parenting so the child does not + jump when it acquires a parent. Blender's `parent_inverse`. Small, and its + absence is the kind of thing that makes a parenting feature feel broken." + [n] + (if-let [p (:pinv n)] + (js/Float64Array.from (clj->js p)) + nil)) + +(defn world! + "dest := parent · pinv · local. `parent` is nil at the root, `pinv-m` nil until + something is reparented. `scratch` is a 6-element Float64Array the caller owns; + it is an argument rather than an allocation because this runs per node per + frame." + [^js dest ^js parent ^js pinv-m ^js local ^js scratch] + (cond + (and parent pinv-m) (mul! dest parent (mul! scratch pinv-m local)) + parent (mul! dest parent local) + pinv-m (mul! dest pinv-m local) + :else (doto dest (.set local)))) + +(defn apply-pt! + "out[2i], out[2i+1] := m · (x, y)." + [^js out i ^js m x y] + (aset out (* 2 i) (+ (* (aget m 0) x) (* (aget m 2) y) (aget m 4))) + (aset out (inc (* 2 i)) (+ (* (aget m 1) x) (* (aget m 3) y) (aget m 5))) + out) + +(defn mean-scale + "The geometric-mean scale of a transform, sqrt|det|. + + A disc under a non-uniform transform is an ellipse and this rasteriser has no + ellipse — the iris is a disc because at 320x200 it is a few pixels across, and + a few-pixel ellipse is not a shape, it is a stair. So a disc's radius takes the + mean scale. For a similarity, which is the only transform the anchor fit + produces, this is exact." + [^js m] + (js/Math.sqrt (js/Math.abs (- (* (aget m 0) (aget m 3)) + (* (aget m 1) (aget m 2)))))) + +;; --------------------------------------------------------------------------- + +(defn problems + "Human-readable reasons this map is not a usable node. Empty means it is one. + + Worth having at all because app-db holds only authored data, which is what + makes validating every event affordable; this is the per-node half of that." + [n] + (let [k (:kind n) + valid (get valid-paths k)] + (-> [] + (cond-> + (nil? (:id n)) (conj "no :id") + (not (contains? kinds k)) (conj (str ":kind " (pr-str k) " is not one of " (pr-str kinds))) + (and (contains? kinds k) + (not (contains? implemented-kinds k))) + (conj (str ":kind " k " is in the vocabulary but not implemented")) + + (nil? (:z n)) (conj "no :z — draw order is authored per scene, not implied by the tree") + (and (:span n) (not= 2 (count (:span n)))) + (conj ":span must be [in out]")) + + (into (when valid + (for [[path _] (:channels n) + :when (not (contains? valid path))] + (str "channel " (pr-str path) " is not valid on a " k " node")))) + + (into (for [[path c] (:channels n) + p (ch/problems c)] + (str "channel " (pr-str path) ": " p)))))) + +(defn problems-str [n] + (str/join "; " (problems n))) diff --git a/frontend/src/arthur/domain/raster.cljs b/frontend/src/arthur/domain/raster.cljs index 8a13e07..9cd8381 100644 --- a/frontend/src/arthur/domain/raster.cljs +++ b/frontend/src/arthur/domain/raster.cljs @@ -24,42 +24,75 @@ (.fill buf index) r) +(defn fill-poly-buf! + "Even-odd scanline fill of a polygon held FLAT in `pts` as [x0 y0 x1 y1 …], + using the first `n` points. Samples at pixel centres (y + 0.5), so a polygon + edge landing exactly on a pixel boundary resolves consistently. + + Flat and preallocated because this is the per-frame path: fixed topology means + a node's vertex count is known at freeze time, so scene/resolver hands the same + buffer back every frame and a frame allocates nothing. At 30fps per-frame + allocation is the only thing that will make this stutter. + + `pts` may be a CLJS vector or any typed array; scanline crossings are collected + into a plain JS array and sorted in place." + [{:keys [w h buf] :as r} pts n index] + (when (>= n 3) + (let [px (fn [i] (if (vector? pts) (-nth pts (* 2 i)) (aget pts (* 2 i)))) + py (fn [i] (if (vector? pts) (-nth pts (inc (* 2 i))) (aget pts (inc (* 2 i))))) + xs (array)] + (let [ymin (loop [i 1, acc (py 0)] (if (< i n) (recur (inc i) (min acc (py i))) acc)) + ymax (loop [i 1, acc (py 0)] (if (< i n) (recur (inc i) (max acc (py i))) acc)) + y0 (max 0 (js/Math.ceil (- ymin 0.5))) + y1 (min (dec h) (inc (js/Math.floor (- ymax 0.5))))] + (loop [y y0] + (when (<= y y1) + (let [sy (+ y 0.5)] + (set! (.-length xs) 0) + (dotimes [i n] + (let [j (mod (inc i) n) + ay (py i) by (py j)] + ;; A horizontal edge contributes no crossing, and dividing by + ;; its zero height would emit Infinity. + (when (not= ay by) + (let [lo (min ay by) hi (max ay by)] + ;; Half-open in y: >= lo and < hi. A vertex shared by two + ;; edges is counted exactly once, so the parity cannot flip + ;; at a corner and leak a whole scanline. + (when (and (>= sy lo) (< sy hi)) + (.push xs (+ (px i) (* (/ (- sy ay) (- by ay)) + (- (px j) (px i)))))))))) + (when (>= (.-length xs) 2) + (.sort xs (fn [a b] (- a b))) + (loop [k 0] + (when (< (inc k) (.-length xs)) + (let [x-from (max 0 (js/Math.ceil (- (aget xs k) 0.5))) + x-to (min (dec w) (js/Math.floor (- (aget xs (inc k)) 0.5))) + row (* y w)] + (loop [x x-from] + (when (<= x x-to) + (aset buf (+ row x) index) + (recur (inc x))))) + (recur (+ k 2)))))) + (recur (inc y))))))) + r) + (defn fill-poly! - "Even-odd scanline fill. Samples at pixel centres (y + 0.5), so a polygon - edge landing exactly on a pixel boundary resolves consistently." - [{:keys [w h buf] :as r} pts index] - (let [n (count pts)] - (when (>= n 3) - (let [ys (map :y pts) - y0 (max 0 (js/Math.ceil (- (apply min ys) 0.5))) - y1 (min (dec h) (inc (js/Math.floor (- (apply max ys) 0.5))))] - (doseq [y (range y0 (inc y1))] - (let [sy (+ y 0.5) - xs (sort - (for [i (range n) - :let [a (nth pts i) - b (nth pts (mod (inc i) n))] - ;; A horizontal edge contributes no crossing, and - ;; dividing by its zero height would emit Infinity. - :when (not= (:y a) (:y b)) - :let [lo (min (:y a) (:y b)) - hi (max (:y a) (:y b))] - ;; Half-open in y: >= lo and < hi. A vertex shared by - ;; two edges is counted exactly once, so the parity - ;; cannot flip at a corner and leak a whole scanline. - :when (and (>= sy lo) (< sy hi))] - (+ (:x a) (* (/ (- sy (:y a)) (- (:y b) (:y a))) - (- (:x b) (:x a))))))] - (when (>= (count xs) 2) - (doseq [[xa xb] (partition 2 xs)] - (let [x-from (max 0 (js/Math.ceil (- xa 0.5))) - x-to (min (dec w) (js/Math.floor (- xb 0.5))) - row (* y w)] - (loop [x x-from] - (when (<= x x-to) - (aset buf (+ row x) index) - (recur (inc x))))))))))) - r)) + "`fill-poly-buf!` over a seq of {:x :y} points. + + The map form is what the analysis stages and the paint tool speak, and what the + JS oracle is diffed against; the flat form is what evaluation produces. ONE + scanline implementation serves both, because two would drift and the drift + would read as a rendering bug rather than as two functions disagreeing." + [r pts index] + (let [n (count pts) + a (js/Float64Array. (* 2 n))] + (loop [i 0, ps (seq pts)] + (when ps + (aset a (* 2 i) (:x (first ps))) + (aset a (inc (* 2 i)) (:y (first ps))) + (recur (inc i) (next ps)))) + (fill-poly-buf! r a n index))) (defn fill-disc! "`over` is an optional stencil: when given, only pixels that currently hold @@ -127,12 +160,18 @@ Returns {:width :height :data} with :data a Uint8ClampedArray, ready to hand to an ImageData. An index with no palette entry comes out magenta rather than transparent or black: writing an index the palette does not have is a bug, and - it should be impossible to miss." - ([r palette-rgb] (->rgba r palette-rgb 1)) - ([{:keys [w h buf]} palette-rgb zoom] + it should be impossible to miss. + + `dest` is an optional Uint8ClampedArray to write into instead of allocating + one. At 320x200 the buffer is 256KB, and allocating and discarding that thirty + times a second is exactly the per-frame allocation the model is arranged to + avoid; ui/canvas passes the live ImageData's own array." + ([r palette-rgb] (->rgba r palette-rgb 1 nil)) + ([r palette-rgb zoom] (->rgba r palette-rgb zoom nil)) + ([{:keys [w h buf]} palette-rgb zoom dest] (let [W (* w zoom) H (* h zoom) - d (js/Uint8ClampedArray. (* W H 4))] + d (or dest (js/Uint8ClampedArray. (* W H 4)))] (loop [y 0] (when (< y H) (let [srow (* (js/Math.floor (/ y zoom)) w)] @@ -148,3 +187,20 @@ (recur (inc x))))) (recur (inc y)))) {:width W :height H :data d}))) + +(defn draw-ops! + "Paint a list of draw ops, in the order given, into the raster. Stage 7. + + This is the boundary the whole model is arranged around: an op carries raster + space points and a PALETTE INDEX, and the rasteriser knows nothing about nodes, + channels, time maps or provenance. Everything above here can be rearranged + without touching a scanline, and a painted cel and a rotoscoped mouth arrive + here indistinguishable from each other, which is the point." + [r ops] + (doseq [{:keys [kind pts n color stencil cx cy size] :as op} ops] + (case kind + :poly (fill-poly-buf! r pts n color) + :disc (fill-disc! r cx cy (:r op) color stencil) + :rect (fill-rect! r cx cy size color stencil) + (throw (ex-info "draw op kind is not rasterisable" {:op (dissoc op :pts)})))) + r) diff --git a/frontend/src/arthur/domain/scene.cljs b/frontend/src/arthur/domain/scene.cljs new file mode 100644 index 0000000..4af5a11 --- /dev/null +++ b/frontend/src/arthur/domain/scene.cljs @@ -0,0 +1,395 @@ +(ns arthur.domain.scene + "The scene: a flat map of id -> node, and the two ways to evaluate it at a + frame. + + (eval-frame scene f store) THE SPECIFICATION. Allocating, order-free, + obviously correct. Use it in tests and for a + one-off render. + + (resolver scene store) -> (fn [f] ops). What playback uses. Caches the + topological order and the z paths, holds one + CURSOR per channel and one PREALLOCATED point + buffer per node, so a frame allocates the op + maps and nothing else. + + Both run the same walk — `eval-into` below — parameterised by how a channel is + read and where points are written. That is deliberate: two independent + implementations of frame evaluation would drift, and the drift would look like + a rendering bug rather than like two functions disagreeing. What differs + between them is exactly the part that can be wrong, and scene-test asserts they + agree frame for frame in forward, backward and random order. + + The output is a list of DRAW OPS, and it is the boundary with the rasteriser: + ops carry palette indices and raster-space points, and the rasteriser knows + nothing about nodes, channels or time. + + Geometry is stored FLAT — [x0 y0 x1 y1 …] — in authored channels as well as + dense ones. A dense block is a rectangular Int16Array and an authored ring is a + vector of numbers, and they read the same way, which is what makes freezing + fill in the same channel rather than convert into a second format." + (:require [arthur.domain.channel :as ch] + [arthur.domain.node :as node] + [arthur.domain.palette :as pal] + [clojure.string :as str])) + +;; --------------------------------------------------------------------------- +;; structure: depth, topological order, draw order + +(defn depth + "Number of ancestors. Throws on a parent cycle rather than looping forever — a + cycle is reachable from one bad `:node/set-parent`, and a hung tab is a much + worse diagnostic than a stack trace naming the two nodes." + [nodes id] + (loop [id id, d 0, seen #{}] + (let [p (:parent (get nodes id))] + (cond + (nil? p) d + (contains? seen p) + (throw (ex-info "parent cycle in scene" {:node id :cycle (conj seen p)})) + (nil? (get nodes p)) + (throw (ex-info "node's :parent is not in the scene" {:node id :parent p})) + :else (recur p (inc d) (conj seen p)))))) + +(defn order + "Node ids in topological order: every node after its parent. + + Sorting by parent depth is enough — it does not need Kahn's algorithm, because + the only edge is parent, and a node's depth is by definition greater than its + parent's. Ties are broken by id so the order is deterministic across runs, + which matters because the draw-order sort below falls back on this position." + [nodes] + (vec (sort-by (juxt #(depth nodes %) #(str %)) (keys nodes)))) + +(defn z-path + "The node's z index and every ancestor's, root first. + + Draw order is depth-first by sibling z, so the key that sorts it is the chain + of z values from the root. A parent's path is a PREFIX of its child's, which is + why a parent draws before its children without that being a special case. + + `:z` values are fractional-index STRINGS (\"a1\", \"a3\") and compare + lexicographically, so a node can always be inserted between two siblings + without renumbering either." + [nodes id] + (loop [id id, acc ()] + (if (nil? id) + (vec acc) + (let [n (get nodes id)] + (recur (:parent n) (conj acc (:z n))))))) + +(defn- z-lex + "Lexicographic compare of two z paths, a prefix sorting first. + + `compare` on vectors will not do: it compares COUNT first, so a deep + descendant of \"a1\" would sort after a shallow \"a2\" and a painted cel would + jump in front of the head that carries it." + [a b] + (let [na (count a), nb (count b)] + (loop [i 0] + (if (or (= i na) (= i nb)) + (- na nb) + (let [c (compare (nth a i) (nth b i))] + (if (zero? c) (recur (inc i)) c)))))) + +(defn- op-compare [x y] + (let [c (z-lex (:z-path x) (:z-path y))] + (if (zero? c) (- (:i x) (:i y)) c))) + +;; --------------------------------------------------------------------------- +;; colour + +(defn colour-index + "Tone keyword -> the index the raster writes, in a given palette. + + `palette` is a map of tone -> index. It is a PARAMETER, not a global: a tone + names which mark this is, and which ramp it is read in belongs to the timeline + the node sits in, so resolution cannot reach for one ambient answer. Today + there is one palette and it is passed in anyway; when timelines carry a + `:palette` channel, the walk carries the palette in scope exactly as it already + carries the parent transform and the local frame. + + An unknown tone resolves to 255, which the palette expansion renders MAGENTA. + Loud rather than fatal, and the same choice raster/->rgba already makes: + naming a colour the ramp does not have is a bug in authored data, and it should + be impossible to miss and should not take the frame down." + [palette k] + (cond + (number? k) k + (nil? k) 255 + :else (get palette k 255))) + +;; --------------------------------------------------------------------------- +;; the walk + +(defn- in-span? + "`:span` is Lottie's ip/op and Flash's PlaceObject/RemoveObject: the range over + which the node EXISTS, tested in the PARENT's frame space and therefore before + the node's own time map runs. Distinct from `[:vis]`, which blinks an existing + node on and off. Half-open, so two adjacent spans do not both own a frame." + [n f] + (if-let [[in out] (:span n)] + (and (>= f in) (< f out)) + true)) + +(defn- finish + "Resolve stencils, then sort into draw order. + + A stencil is a COLOUR KEY, not a node reference: it is the take format's + `clip=`, and the indexed buffer being its own clip mask is what keeps the iris + inside the eye at any gaze and any radius without a per-part mask. So the + stencil node's own colour is looked up here, after the walk, because the + stencil may sit anywhere in the order. Two nodes sharing a palette entry share + a stencil, which is inherent to the technique rather than a defect in it. + + A node stencilled by something that drew NOTHING is DROPPED, not drawn + unclipped: unclipped would be an iris floating over the cheek on exactly the + frames where the eye is missing." + [ops] + (let [by-id (into {} (map (juxt :node :color)) ops)] + (->> ops + (keep (fn [op] + (if-let [s (:stencil op)] + (when-let [idx (get by-id s)] + (assoc op :stencil idx)) + op))) + (sort op-compare) + vec))) + +(defn- eval-into + "The one frame evaluation, parameterised by how a channel is read and where its + points are written. + + read (fn [id path channel local-frame] -> v) + palette tone -> index, the ramp in scope + mat-for (fn [id] -> Float64Array) the node's world transform + pinv-for (fn [id] -> Float64Array|nil) its parent-inverse + buf-for (fn [id n-points] -> Float64Array) + scratch one spare 6-element matrix + + Returns ops in z order." + [nodes ord zpaths read palette mat-for pinv-for buf-for scratch f] + (let [cnt (count ord)] + (loop [i 0, placed {}, ops []] + (if (= i cnt) + (finish ops) + (let [id (nth ord i) + n (get nodes id) + pid (:parent n) + parent (when pid (get placed pid))] + ;; A node whose parent was dropped is dropped with it, and so is + ;; everything under it. Topological order is what makes that one + ;; lookup instead of a subtree walk. + (if (and pid (nil? parent)) + (recur (inc i) placed ops) + (let [pf (if parent (:f parent) f)] + (if-not (in-span? n pf) + (recur (inc i) placed ops) + (let [chs (node/channels n) + lf (node/local-frame n pf) + rd (fn [path] (read id path (get chs path) lf)) + vis (rd [:vis]) + pos (rd [:xform :pos]) + rot (rd [:xform :rot]) + scl (rd [:xform :scale]) + skw (rd [:xform :skew]) + anc (rd [:xform :anchor])] + ;; [:vis] and the transform gate the DESCENDANTS as well as the + ;; node: a switched-off feature takes its parts with it, and a + ;; node with no transform gives its children nowhere to be. + ;; + ;; A missing [:geom :pts] does NOT gate descendants. An absent + ;; mouth outline has nothing to draw, but the head it hangs off + ;; is still exactly where it was. That asymmetry is the whole + ;; reason presence is tracked per channel rather than per node. + (if-not (and (true? vis) + (not (ch/nothing? pos)) (not (ch/nothing? rot)) + (not (ch/nothing? scl)) (not (ch/nothing? skw)) + (not (ch/nothing? anc))) + (recur (inc i) placed ops) + ;; dest aliases `local` here, which mul! allows: it reads both + ;; operands fully before writing either. + (let [m (node/local! (mat-for id) pos rot scl skw anc) + m (node/world! m (:m parent) (pinv-for id) m scratch) + base {:i i :z-path (get zpaths id) :node id + :stencil (:stencil n)} + op (case (:kind n) + :group nil + + :poly + (let [pts (rd [:geom :pts])] + (when-not (ch/nothing? pts) + (let [np (quot (if (vector? pts) (count pts) (.-length pts)) 2) + out (buf-for id np)] + (dotimes [k np] + (node/apply-pt! out k m + (ch/component pts (* 2 k)) + (ch/component pts (inc (* 2 k))))) + (assoc base :kind :poly :pts out :n np + :color (colour-index palette (rd [:style :color])))))) + + :disc + (let [rad (rd [:geom :radius])] + (when-not (ch/nothing? rad) + (assoc base :kind :disc + :cx (aget m 4) :cy (aget m 5) + :r (* rad (node/mean-scale m)) + :color (colour-index palette (rd [:style :color]))))) + + :rect + (let [size (rd [:geom :size])] + (when-not (ch/nothing? size) + (assoc base :kind :rect + :cx (aget m 4) :cy (aget m 5) + ;; ROUNDED, because :size is a pixel + ;; count: a scaled square 3.4px wide + ;; would be 3px on one frame and 4 on + ;; the next, which reads as the pupil + ;; breathing. See raster/fill-rect!. + :size (js/Math.round (* size (node/mean-scale m))) + :color (colour-index palette (rd [:style :color]))))) + + (throw (ex-info "node kind is not implemented" + {:node id :kind (:kind n)})))] + (recur (inc i) + (assoc placed id {:m m :f lf}) + (cond-> ops op (conj op)))))))))))))) + +;; --------------------------------------------------------------------------- +;; the specification + +(defn eval-frame + "Scene at clip frame f -> draw ops in z order. Pure, and allocates freely. + + This is the definition of what a frame means. `resolver` is what plays it." + ([scene f] (eval-frame scene f nil pal/index-of)) + ([scene f store] (eval-frame scene f store pal/index-of)) + ([scene f store palette] + (let [nodes (:nodes scene) + ord (order nodes) + zpaths (into {} (map (fn [id] [id (z-path nodes id)])) ord)] + (eval-into nodes ord zpaths + (fn [_id _path c lf] (ch/value-at c lf store)) + palette + (fn [_id] (node/mat)) + (fn [id] (node/pinv (get nodes id))) + (fn [_id n] (js/Float64Array. (* 2 n))) + (node/mat) + f)))) + +;; --------------------------------------------------------------------------- +;; the playback path + +(defn- point-capacity + "How many points the widest value of a [:geom :pts] channel holds. + + FIXED TOPOLOGY is what makes this a number at all: every key of a part carries + the same vertex count with the same vertex meanings, so the buffer can be + allocated once. A variable vertex count would force a per-frame offset table + and a scan, which is why the aesthetic constraint is a performance asset rather + than a cost." + [c] + (quot (cond + (:dense c) (:stride (:dense c)) + (:animated? c) (transduce (map #(if (vector? %) (count %) (.-length %))) + max 0 (vals (:keys c))) + :else (let [v (:value c)] (if (vector? v) (count v) (.-length v)))) + 2)) + +(defprotocol IResolver + (world-of [this id] + "The node's world transform AS OF THE LAST FRAME RESOLVED, or nil if it was + not placed on that frame. + + The matrices are the ones evaluation mutates in place, so this is a read of + live state rather than a snapshot — which is exactly what the caller wants. + A registered photo underlay has to ride the same transform the vectors went + through or it is merely decorative, and it paints immediately after the frame + it belongs to, so \"as of the last frame\" is the only answer that can be + correct.")) + +(defn resolver + "(fn [f] -> ops). Holds everything that does not change per frame. + + The point buffers are REUSED between frames, so a caller must consume the ops + before asking for the next frame. That is the contract the rAF loop wants + anyway — it reads, blits, and dispatches nothing — and it is what makes a frame + cost a lookup and a blit rather than an allocation per vertex. + + The op maps themselves are allocated fresh, and deliberately: there are a dozen + of them per frame against hundreds of points, so pooling them would buy + nothing and cost the ability to hand an op list around as plain data." + ([scene] (resolver scene nil pal/index-of)) + ([scene store] (resolver scene store pal/index-of)) + ([scene store palette] + (let [nodes (:nodes scene) + ord (order nodes) + zpaths (into {} (map (fn [id] [id (z-path nodes id)])) ord) + cursors (into {} + (map (fn [id] + [id (into {} (map (fn [[p c]] [p (ch/cursor c store)])) + (node/channels (get nodes id)))])) + ord) + mats (into {} (map (fn [id] [id (node/mat)])) ord) + pinvs (into {} (keep (fn [id] (when-let [p (node/pinv (get nodes id))] [id p]))) ord) + bufs (into {} + (keep (fn [id] + (when-let [c (get-in nodes [id :channels [:geom :pts]])] + [id (js/Float64Array. (* 2 (point-capacity c)))]))) + ord) + scratch (node/mat) + ;; Every call to mat-for is a placement: eval-into reaches it only after + ;; the span, visibility and transform gates have all passed. So wrapping + ;; it is how the resolver learns which nodes exist this frame without + ;; eval-into having to report it — and it covers groups, which are + ;; placed but emit no op, and which are exactly what an underlay rides. + placed (volatile! #{}) + step (fn [f] + (vreset! placed #{}) + (eval-into nodes ord zpaths + (fn [id path _c lf] (ch/sample! (get-in cursors [id path]) lf)) + palette + (fn [id] (vswap! placed conj id) (get mats id)) + (fn [id] (get pinvs id)) + (fn [id _n] (get bufs id)) + scratch + f))] + (reify + IFn + (-invoke [_ f] (step f)) + IResolver + (world-of [_ id] (when (contains? @placed id) (get mats id))))))) + +;; --------------------------------------------------------------------------- + +(defn problems + "Human-readable reasons this scene will not evaluate. Empty means it will. + + Total by construction — it reports a cycle rather than looping on one — because + its whole job is to be safe to run over authored data before that data is + trusted." + [scene] + (let [nodes (:nodes scene)] + (if-not (map? nodes) + [":nodes must be a map of id -> node"] + (-> [] + (into (for [[id n] nodes + :when (not= id (:id n))] + (str "node under key " (pr-str id) " has :id " (pr-str (:id n))))) + (into (for [[id n] nodes + :when (and (:parent n) (not (contains? nodes (:parent n))))] + (str "node " (pr-str id) " has :parent " (pr-str (:parent n)) + " which is not in the scene"))) + (into (for [[id n] nodes + :when (and (:stencil n) (not (contains? nodes (:stencil n))))] + (str "node " (pr-str id) " has :stencil " (pr-str (:stencil n)) + " which is not in the scene"))) + (into (for [[id n] nodes + p (node/problems n)] + (str "node " (pr-str id) ": " p))) + (into (try + (doall (map #(depth nodes %) (keys nodes))) + nil + (catch :default e [(ex-message e)]))))))) + +(defn problems-str [scene] + (str/join "; " (problems scene))) diff --git a/frontend/src/arthur/events/playback.cljs b/frontend/src/arthur/events/playback.cljs new file mode 100644 index 0000000..6e5e921 --- /dev/null +++ b/frontend/src/arthur/events/playback.cljs @@ -0,0 +1,100 @@ +(ns arthur.events.playback + "Transport events. + + Named for intent rather than for the field they happen to set: `::toggle` is + not `set-playing?`, because what the button means is \"start or stop\", and the + resulting boolean is a consequence. + + NO GLOBAL INTERCEPTORS ON ::tick. At 30fps a spec-validating `after` or + `std-interceptors/debug`'s `clojure.data/diff` would be thirty full-db + traversals a second, which is the one genuinely expensive thing you can do to + a small app-db. If global interceptors are added later they are added to a + chain these events are excluded from, not to `reg-global-interceptor`." + (:require [arthur.clock :as clock] + [arthur.db :as db] + [re-frame.core :as rf])) + +(defn- fps [db] (get-in db [:clip :fps])) +(defn- frames [db] (get-in db [:clip :frames])) + +(rf/reg-event-db + ::tick + (fn [db [_ f]] + ;; Written from the rAF loop when the DERIVED frame changes — not every + ;; animation frame, and never as the thing the blit waits on. The picture is + ;; painted from the clock directly; this only brings the document's idea of + ;; the playhead up to date so the readout and the scrubber agree with it. + (if (= f (get-in db [:playback :frame])) + db + (assoc-in db [:playback :frame] f)))) + +(rf/reg-event-fx + ::play + (fn [{:keys [db]} _] + {:db (assoc-in db [:playback :playing?] true) + ::play! nil})) + +(rf/reg-event-fx + ::pause + (fn [{:keys [db]} _] + {:db (assoc-in db [:playback :playing?] false) + ::pause! nil})) + +(rf/reg-event-fx + ::toggle + (fn [{:keys [db]} _] + (if (get-in db [:playback :playing?]) + {:db (assoc-in db [:playback :playing?] false) ::pause! nil} + {:db (assoc-in db [:playback :playing?] true) ::play! nil}))) + +(rf/reg-event-fx + ::seek + (fn [{:keys [db]} [_ f]] + (let [f (-> f (max 0) (min (dec (frames db))))] + {:db (assoc-in db [:playback :frame] f) + ::seek! [(fps db) (frames db) f]}))) + +(rf/reg-event-fx + ::step + (fn [{:keys [db]} [_ delta]] + {:fx [[:dispatch [::seek (+ (get-in db [:playback :frame]) delta)]]]})) + +(rf/reg-event-fx + ::set-rate + (fn [{:keys [db]} [_ r]] + {:db (assoc-in db [:playback :rate] r) + ::rate! r})) + +;; --- effects: every DOM touch on the audio element is one of these --- + +(rf/reg-fx ::play! (fn [_] (clock/play!))) +(rf/reg-fx ::pause! (fn [_] (clock/pause!))) +(rf/reg-fx ::rate! (fn [r] (clock/set-rate! r))) +(rf/reg-fx ::seek! (fn [[fps frames f]] (clock/seek! fps frames f))) +(rf/reg-fx ::loop! (fn [on?] (clock/set-loop! on?))) +(rf/reg-fx ::mute! (fn [on?] (clock/set-muted! on?))) + +(rf/reg-event-fx + ::toggle-loop + (fn [{:keys [db]} _] + (let [on? (not (get-in db [:playback :loop?]))] + {:db (assoc-in db [:playback :loop?] on?) ::loop! on?}))) + +(rf/reg-event-fx + ::toggle-mute + (fn [{:keys [db]} _] + (let [on? (not (get-in db [:playback :muted?]))] + {:db (assoc-in db [:playback :muted?] on?) ::mute! on?}))) + +(rf/reg-event-fx + ::select-scene + (fn [{:keys [db]} [_ id]] + ;; Changing the clip changes the resolver, the frame count and the rate all + ;; at once, so the playhead goes home rather than being left pointing at a + ;; frame the new clip may not have. + (let [{:keys [fps frames]} (get db/scenes id)] + {:db (-> db + (assoc :scene/current id) + (assoc :clip {:fps fps :frames frames}) + (assoc-in [:playback :frame] 0)) + ::seek! [fps frames 0]}))) diff --git a/frontend/src/arthur/subs/playback.cljs b/frontend/src/arthur/subs/playback.cljs new file mode 100644 index 0000000..2f0c344 --- /dev/null +++ b/frontend/src/arthur/subs/playback.cljs @@ -0,0 +1,13 @@ +(ns arthur.subs.playback + "Layer-2 extractors over the transport. Cheap by construction: each one reads a + path and returns a value, so a tick that changes only `:frame` notifies only + the things that asked for `:frame`." + (:require [re-frame.core :as rf])) + +(rf/reg-sub ::frame (fn [db _] (get-in db [:playback :frame]))) +(rf/reg-sub ::playing? (fn [db _] (get-in db [:playback :playing?]))) +(rf/reg-sub ::rate (fn [db _] (get-in db [:playback :rate]))) +(rf/reg-sub ::loop? (fn [db _] (get-in db [:playback :loop?]))) +(rf/reg-sub ::muted? (fn [db _] (get-in db [:playback :muted?]))) +(rf/reg-sub ::fps (fn [db _] (get-in db [:clip :fps]))) +(rf/reg-sub ::frames (fn [db _] (get-in db [:clip :frames]))) diff --git a/frontend/src/arthur/subs/render.cljs b/frontend/src/arthur/subs/render.cljs new file mode 100644 index 0000000..0c143d3 --- /dev/null +++ b/frontend/src/arthur/subs/render.cljs @@ -0,0 +1,60 @@ +(ns arthur.subs.render + "Stage 6. The subscription graph IS the staged dataflow — each stage is a + layer-3 sub over the previous one plus the parameters for its stage only, + which is what makes changing one knob recompute one thing. + + `::resolver` is the shape that matters. It does not yield geometry; it yields a + CLOSURE that produces geometry at a frame. So a scene edit costs one + recomputation here and a frame costs a lookup and a blit — and, crucially, the + playhead is not an input, so moving it cannot invalidate this." + (:require [arthur.db :as db] + [arthur.domain.palette :as pal] + [arthur.domain.scene :as scene] + [re-frame.core :as rf])) + +(rf/reg-sub ::scene-id (fn [db _] (:scene/current db))) + +(rf/reg-sub ::scene (fn [db _] (get-in db/scenes [(:scene/current db) :scene]))) + +(rf/reg-sub + ::exposure + :<- [::scene] + (fn [scene _] + ;; Exposure lives on the clip root and is INHERITED, so reading it there is + ;; reading it everywhere. The transport shows it so that `exposure 2` is + ;; visibly doing something at the transport rather than only inside the scene. + (or (get-in scene [:nodes :root :time :expose]) 1))) + +(rf/reg-sub + ::palette + (fn [db _] + ;; A NAME resolves to a ramp. One today; when timelines carry a `:palette` + ;; channel this becomes the project's table and the walk carries the ramp in + ;; scope, which is why domain/scene takes the palette as a parameter rather + ;; than reaching for a global. + (get {:arthur/default pal/index-of} (:palette db) pal/index-of))) + +(rf/reg-sub + ::ramp + (fn [db _] + ;; index -> [r g b]. The other half of the palette: `::palette` says which + ;; INDEX a tone resolves to, this says what that index LOOKS LIKE. Two subs + ;; because two different consumers — evaluation needs the first, the blit + ;; needs the second, and neither wants the other's map. + (get {:arthur/default pal/rgb} (:palette db) pal/rgb))) + +(rf/reg-sub + ::store + (fn [db _] + ;; Tier 2, behind a handle, and never in app-db itself — what is in the db is + ;; the id of the clip whose blocks these are. The hand-written demo has none; + ;; the swarm is entirely dense. + (get-in db/scenes [(:scene/current db) :store]))) + +(rf/reg-sub + ::resolver + :<- [::scene] + :<- [::store] + :<- [::palette] + (fn [[scene store palette] _] + (scene/resolver scene store palette))) diff --git a/frontend/src/arthur/ui/canvas.cljs b/frontend/src/arthur/ui/canvas.cljs new file mode 100644 index 0000000..681c1b2 --- /dev/null +++ b/frontend/src/arthur/ui/canvas.cljs @@ -0,0 +1,45 @@ +(ns arthur.ui.canvas + "The one imperative sink. + + Everything below here is pure and returns bytes; this is where bytes become + pixels, and it is the only namespace allowed to touch a canvas. domain/raster + deliberately stops at `->rgba` returning plain bytes — an ImageData is a DOM + type, and keeping it out of domain/ is what lets every rasteriser assertion run + under node. + + THE CANVAS IS THE RASTER'S OWN SIZE and is scaled up by CSS with + `image-rendering: pixelated`, rather than by expanding in `->rgba`. Two reasons: + a 2x expansion in JS is four times the bytes to write per frame for a result + the GPU gives away, and nearest-neighbour is then the browser's guarantee rather + than something this code has to keep being right about. A browser that smoothed + the upscale would misrepresent the exact thing the preview exists to judge, so + the CSS rule is load-bearing and lives in the host page next to the canvas." + (:require [arthur.domain.palette :as pal] + [arthur.domain.raster :as raster])) + +(defonce ^:private cache + ;; canvas element -> its ImageData, so a frame writes into the array the + ;; canvas already owns instead of allocating a quarter of a megabyte. + (atom {})) + +(defn- image-data-for [^js ctx ^js el w h] + (let [have (get @cache el)] + (if (and have (= w (.-width have)) (= h (.-height have))) + have + (let [img (.createImageData ctx w h)] + (swap! cache assoc el img) + img)))) + +(defn blit! + "Expand an indexed raster through the palette and put it on the canvas." + ([el r] (blit! el r pal/rgb)) + ([^js el {:keys [w h] :as r} palette-rgb] + (when el + ;; Guarded: assigning width reallocates the backing store, so doing it + ;; unconditionally would throw a canvas away thirty times a second. + (when (not= w (.-width el)) (set! (.-width el) w)) + (when (not= h (.-height el)) (set! (.-height el) h)) + (let [ctx (.getContext el "2d") + img (image-data-for ctx el w h)] + (raster/->rgba r palette-rgb 1 (.-data img)) + (.putImageData ctx img 0 0))))) diff --git a/frontend/src/arthur/ui/player.cljs b/frontend/src/arthur/ui/player.cljs new file mode 100644 index 0000000..309d299 --- /dev/null +++ b/frontend/src/arthur/ui/player.cljs @@ -0,0 +1,189 @@ +(ns arthur.ui.player + "The rAF loop. Stage 6 and 7, driven. + + THE LOOP READS AND BLITS. It does not compute geometry, it does not build a + resolver, and it does not wait on the event queue to paint. Per frame it reads + the audio clock, applies a closure it already has, and writes bytes — a lookup + and a blit, with no allocation beyond the op maps. + + It is not a Reagent component and must not become one. Stage 7 writes into a + canvas from an animation frame; it is a sink, not a view that re-renders, and + the actual content of the folklore about re-frame and canvas is that expensive + work must not live in a layer-2 sub. The resolver is a layer-3 sub over the + scene, so the playhead moving cannot invalidate it. + + The one dispatch is `::playback/tick`, and it is deliberately NOT what the + picture waits on: the frame is painted from the clock directly, and the tick + only brings the document's playhead up to date so the readout and the scrubber + agree with what is on screen. It fires when the derived frame CHANGES — thirty + times a second at a 30fps clip on a 60Hz display, not sixty — and it carries + no global interceptors." + (:require [arthur.clock :as clock] + [arthur.domain.raster :as raster] + [arthur.events.playback :as pb] + [arthur.subs.playback :as sub] + [arthur.subs.render :as render] + [arthur.ui.canvas :as canvas] + [re-frame.core :as rf] + [reagent.ratom :as ratom])) + +(defonce ^:private state + (atom {:raf nil :canvas nil :raster nil :last -1})) + +;; --------------------------------------------------------------------------- +;; the snapshot +;; +;; A PLAIN MAP that re-frame pushes into, which the loop reads without doing any +;; reactive work at all. +;; +;; This is not an optimisation, it is a correctness fix. A Reagent reaction +;; caches its value only while it has a watcher; deref it from outside a +;; reactive context — an rAF callback, say — and it RE-RUNS on every deref. So +;; `@(rf/subscribe [::render/resolver])` in the loop was rebuilding the resolver +;; sixty times a second: the topological order, the z paths, a cursor per +;; channel and a preallocated buffer per node, all of it, per frame. On the +;; five-node demo scene that was invisible. On a hundred-and-twenty-node scene +;; it was 3.6fps against a 170fps ceiling, and it presented as "the renderer is +;; slow" rather than as "the caching you assumed is not happening". +;; +;; `ratom/run!` keeps an always-active reaction, so the subscriptions it derefs +;; have a watcher and therefore cache, and the loop reads a plain atom. + +;; Measured paint rate and mean clip-frames advanced per paint. A Reagent atom, +;; so the transport re-renders on it. +;; +;; `:drop` is the informative one. At 1.0 the loop is painting every frame the +;; clock asks for. Above it the clock is moving faster than the painting and +;; frames are being DROPPED — which is the designed failure rather than a bug, +;; but it is a thing to see rather than to infer. +(defonce meter (ratom/atom {:fps 0 :drop 0})) + +(defonce ^:private meter-state + (atom {:t0 nil :paints 0 :samples 0 :advanced 0 :prev nil})) + +(def ^:private ^:const max-credible-advance + ;; A jump bigger than this is a seek or a loop wrap, not a dropped frame. Both + ;; move the playhead by an arbitrary amount in one tick, and counting either as + ;; a drop makes the readout say the renderer is failing whenever the clip + ;; restarts — which is exactly when a profile is running. + 16) + +(defn- meter! [f] + (let [now (js/performance.now) + {:keys [t0 paints samples advanced prev]} @meter-state] + (if (nil? t0) + (reset! meter-state {:t0 now :paints 0 :samples 0 :advanced 0 :prev f}) + (let [d (when prev (- f prev)) + credible (and d (pos? d) (<= d max-credible-advance)) + paints (inc paints) + samples (if credible (inc samples) samples) + advanced (if credible (+ advanced d) advanced) + dt (- now t0)] + (if (>= dt 500) + (do (reset! meter {:fps (/ (* 1000 paints) dt) + :drop (if (pos? samples) (/ advanced samples) 0)}) + (reset! meter-state {:t0 now :paints 0 :samples 0 :advanced 0 :prev f})) + (reset! meter-state {:t0 t0 :paints paints :samples samples + :advanced advanced :prev f})))))) + +(defonce ^:private snapshot (atom {})) +(defonce ^:private tracker (atom nil)) + +(defn- repaint! [] + (swap! state assoc :last -1)) + +(defn refresh-subs! + "Build (or rebuild) the tracking reaction. Rebuilt on hot reload, because + clearing the subscription cache orphans the reactions this holds." + [] + (some-> @tracker ratom/dispose!) + (reset! tracker + (ratom/run! + (let [was (:resolver @snapshot) + now @(rf/subscribe [::render/resolver])] + (reset! snapshot + {:resolver now + :palette @(rf/subscribe [::render/palette]) + :ramp @(rf/subscribe [::render/ramp]) + :fps @(rf/subscribe [::sub/fps]) + :frames @(rf/subscribe [::sub/frames]) + :frame @(rf/subscribe [::sub/frame]) + :playing? @(rf/subscribe [::sub/playing?])}) + ;; A new resolver means a new scene or a new palette, and neither + ;; moves the playhead — so nothing else would ask for a redraw. + (when-not (identical? was now) (repaint!)))))) + +(defn set-canvas! [el] + (swap! state assoc :canvas el) + ;; The :ref fires AFTER the loop has started, so the first tick or two run + ;; with nowhere to draw. Without this the loop would record frame 0 as + ;; painted, find it unchanged on every subsequent tick, and never draw at all + ;; — a blank canvas under a transport reading perfectly correct. + (when el (repaint!))) + +(defn- raster-for [w h] + (let [{:keys [raster]} @state] + (if (and raster (= w (:w raster)) (= h (:h raster))) + raster + (:raster (swap! state assoc :raster (raster/make w h)))))) + +(defn paint! + "Resolve `f` and put it on the canvas. `ops` are consumed here and only here — + the resolver reuses its point buffers between frames, so they have to be + rasterised before the next frame is asked for." + [f] + (let [{:keys [canvas]} @state + {:keys [resolver palette ramp]} @snapshot] + (when (and canvas resolver) + ;; User Timing, so a profile in the DevTools performance panel has named + ;; spans in the Timings track instead of a wall of anonymous frames. Three + ;; lines, and the difference between reading a profile and guessing at one. + (js/performance.mark "arthur/paint:start") + (let [ras (raster-for 320 200)] + (-> ras + (raster/clear! (get palette :bg 0)) + (raster/draw-ops! (resolver f))) + (js/performance.mark "arthur/blit:start") + (canvas/blit! canvas ras ramp)) + (js/performance.measure "arthur/resolve+draw" "arthur/paint:start" "arthur/blit:start") + (js/performance.measure "arthur/paint" "arthur/paint:start")))) + +(defonce ^:private watch + ;; A scene swap changes the resolver and not the frame number, so the loop + ;; would sit on an unchanged playhead and never redraw. Watching the reaction + ;; is cheaper than making every event that can touch the scene remember to. + (atom nil)) + +(defn- tick! [] + (let [{:keys [fps frames frame playing?]} @snapshot + live? (clock/playing?) + ready? (some? (:canvas @state)) + ;; DERIVED, never counted. A slow frame drops the frames it missed and + ;; the next one lands where the audio already is, so the failure mode is + ;; a visible stutter rather than an invisible slide out of sync. + f (if live? (clock/frame fps frames) frame)] + ;; `ready?` gates the bookkeeping as well as the draw: recording a frame as + ;; painted when it was not is how the canvas stays empty forever. + (when (and ready? f (not= f (:last @state))) + (swap! state assoc :last f) + (paint! f) + (meter! f) + (when live? (rf/dispatch [::pb/tick f]))) + ;; The audio ending is the authority on playback having stopped; nothing + ;; counts frames to notice it. + (when (and (not live?) playing?) + (rf/dispatch [::pb/pause])))) + +(defn- frame-loop [] + (tick!) + (swap! state assoc :raf (js/requestAnimationFrame frame-loop))) + +(defn start! [] + (refresh-subs!) + (when-not (:raf @state) + (swap! state assoc :raf (js/requestAnimationFrame frame-loop)))) + +(defn stop! [] + (when-let [id (:raf @state)] + (js/cancelAnimationFrame id) + (swap! state assoc :raf nil))) diff --git a/frontend/src/arthur/ui/shell.cljs b/frontend/src/arthur/ui/shell.cljs new file mode 100644 index 0000000..b64bb0c --- /dev/null +++ b/frontend/src/arthur/ui/shell.cljs @@ -0,0 +1,95 @@ +(ns arthur.ui.shell + "The page. Transport, canvas, readouts. + + Nothing here computes anything about a frame: it dispatches intents and reads + extractors. The picture is put on the canvas by ui/player's loop, not by this + component re-rendering — which is why the canvas has no reactive content and + why scrubbing at speed does not re-render the page." + (:require [arthur.clock :as clock] + [arthur.db :as db] + [arthur.demo :as demo] + [arthur.events.playback :as pb] + [arthur.subs.playback :as sub] + [arthur.subs.render :as render] + [arthur.ui.player :as player] + [re-frame.core :as rf])) + +(def ^:private zoom 2) + +(defn- audio [] + [:audio + {:ref #(when % (clock/attach! %)) + :src "/audio.wav" + :preload "auto" + ;; Transport state follows the ELEMENT, not the other way round: the audio + ;; is the clock, so anything that can change its state — the end of the + ;; file, the OS media keys, a browser autoplay block — has to be able to + ;; correct the document rather than be contradicted by it. + :on-play #(rf/dispatch [::pb/play]) + :on-pause #(rf/dispatch [::pb/pause])}]) + +(defn- transport [] + (let [playing? @(rf/subscribe [::sub/playing?]) + rate @(rf/subscribe [::sub/rate]) + frame @(rf/subscribe [::sub/frame]) + frames @(rf/subscribe [::sub/frames]) + fps @(rf/subscribe [::sub/fps]) + expose @(rf/subscribe [::render/exposure])] + [:div.transport + [:div.row + [:button {:on-click #(rf/dispatch [::pb/toggle])} + (if playing? "pause" "play")] + [:button {:on-click #(rf/dispatch [::pb/seek 0])} "|<"] + [:button {:on-click #(rf/dispatch [::pb/step -1])} "-1"] + [:button {:on-click #(rf/dispatch [::pb/step 1])} "+1"] + [:button {:class (when @(rf/subscribe [::sub/loop?]) "on") + :on-click #(rf/dispatch [::pb/toggle-loop])} "loop"] + [:button {:class (when @(rf/subscribe [::sub/muted?]) "on") + :on-click #(rf/dispatch [::pb/toggle-mute])} "mute"] + [:span.gap] + (doall + (for [[id {:keys [label]}] db/scenes] + ^{:key id} + [:button {:class (when (= id @(rf/subscribe [::render/scene-id])) "on") + :on-click #(rf/dispatch [::pb/select-scene id])} + label])) + [:span.gap] + (doall + (for [r db/rates] + ^{:key r} + [:button {:class (when (== r rate) "on") + ;; playbackRate and nothing else: the audio slows, currentTime + ;; advances proportionally, and the derived frame follows. Slow + ;; motion cannot desync by construction. + :on-click #(rf/dispatch [::pb/set-rate r])} + (case r 1.0 "1x" 0.5 "1/2x" 0.25 "1/4x" 2.0 "2x" 4.0 "4x" (str r))]))] + [:input.scrub + {:type "range" :min 0 :max (dec frames) :step 1 :value frame + :on-change #(rf/dispatch [::pb/seek (js/parseInt (.. % -target -value) 10)])}] + [:div.readout + [:span (str "frame " frame " / " frames)] + [:span (str fps " fps")] + [:span (str "exposure " expose " → holds " (clock/exposed-frame frame expose))] + [:span (str (js/Math.round (* 100 rate)) "%")] + ;; Measured in the loop, not derived from the clock: the whole question + ;; while profiling is whether the painting keeps up with the clock, so a + ;; number computed FROM the clock would answer itself. + (let [{:keys [fps drop]} @player/meter] + [:span {:class (when (and drop (> drop 1.35)) "warn")} + (str (.toFixed (or fps 0) 1) " paint/s" + (when (and drop (pos? drop)) + (str " · " (.toFixed drop 2) " frames/paint")))])]])) + +(defn view [] + [:main + [:h1 "arthur"] + [:canvas.stage + {:ref #(player/set-canvas! %) + :width demo/width :height demo/height + :style {:width (str (* zoom demo/width) "px") + :height (str (* zoom demo/height) "px")}}] + [audio] + [transport] + [:p.note + "Audio-clocked: the frame is ⌊currentTime · fps⌋, so a slow loop drops " + "frames instead of drifting. ½× and ¼× are playbackRate."]]) diff --git a/frontend/test/arthur/bench_test.cljs b/frontend/test/arthur/bench_test.cljs new file mode 100644 index 0000000..87c6b5f --- /dev/null +++ b/frontend/test/arthur/bench_test.cljs @@ -0,0 +1,40 @@ +(ns arthur.bench-test + "Not a correctness test — a floor. + + It exists because a performance problem in this pipeline does not announce + itself: the picture is still right, it just arrives late, and \"the renderer + feels sluggish\" is indistinguishable from \"the machine is busy\" without a + number. The swarm is the only scene big enough for a regression to show up in, + so it is the one measured. The assertion is deliberately loose — it catches an + order-of-magnitude regression, not a ten percent one, because a tight bound + here would fail on a loaded CI box and teach everyone to ignore it." + (:require [cljs.test :refer [deftest is]] + [arthur.demo.swarm :as swarm] + [arthur.domain.palette :as pal] + [arthur.domain.raster :as raster] + [arthur.domain.scene :as scene])) + +(defn- ms [label n f] + (let [t0 (js/Date.now)] + (dotimes [i n] (f i)) + (let [dt (- (js/Date.now) t0)] + (println (str " " label ": " dt "ms / " n " = " + (.toFixed (/ dt n) 2) "ms per frame")) + (/ dt n)))) + +(deftest bench + (let [res (scene/resolver @swarm/scene @swarm/store pal/index-of) + ras (raster/make 320 200) + dest (js/Uint8ClampedArray. (* 320 200 4)) + n 120] + (println "\nswarm:" (count (:nodes @swarm/scene)) "nodes") + (let [a (ms "resolve " n (fn [i] (res (mod i 229)))) + b (ms "resolve+draw " n (fn [i] + (raster/clear! ras 0) + (raster/draw-ops! ras (res (mod i 229))))) + c (ms "->rgba " n (fn [_] (raster/->rgba ras pal/rgb 1 dest)))] + (println (str " => draw alone ~" (.toFixed (- b a) 2) + "ms, total ~" (.toFixed (+ b c) 2) "ms (" + (.toFixed (/ 1000 (+ b c)) 1) " fps ceiling)\n")) + (is (< (+ b c) 40) + (str "a frame costs " (.toFixed (+ b c) 2) "ms; 40ms would be below 25fps"))))) diff --git a/frontend/test/arthur/clock_test.cljs b/frontend/test/arthur/clock_test.cljs new file mode 100644 index 0000000..489b95f --- /dev/null +++ b/frontend/test/arthur/clock_test.cljs @@ -0,0 +1,108 @@ +(ns arthur.clock-test + "The clock is four lines of arithmetic and the whole take's sync depends on + them, so they are asserted rather than eyeballed. A drift bug is invisible for + the first second and unmistakable by the tenth, which is the worst possible + shape for a bug to have." + (:require [cljs.test :refer [deftest is testing]] + [arthur.clock :as clock])) + +(defn- fake + "Stands in for the audio element. The clock only ever reads `currentTime`, + `paused`, `ended` and `playbackRate` and writes `currentTime`, which is the + entire coupling — so the whole of it can be asserted in node." + [& {:keys [t paused? ended? rate duration] + :or {t 0 paused? true ended? false rate 1.0 duration 7.6}}] + #js {:currentTime t :paused paused? :ended ended? + :playbackRate rate :duration duration}) + +(deftest the-frame-is-derived-from-the-audio-not-counted + ;; frame = ⌊currentTime · fps⌋. The whole point: nothing accumulates, so + ;; nothing can drift. + (clock/attach! (fake :t 0.0)) + (is (= 0 (clock/frame 30 229))) + (doseq [[t want] [[0.0 0] [0.033 0] [0.034 1] [1.0 30] [1.999 59] [2.0 60]]] + (clock/attach! (fake :t t)) + (is (= want (clock/frame 30 229)) (str t "s at 30fps")))) + +(deftest a-dropped-frame-lands-where-the-audio-already-is + ;; THE property the derivation buys. A loop that stalled for a third of a + ;; second resumes at the frame the audio reached, not a third of a second + ;; behind it — the failure is a visible stutter rather than an invisible slide + ;; out of sync, and those are very different bugs to own. + (clock/attach! (fake :t 1.0)) + (let [before (clock/frame 30 229)] + (clock/attach! (fake :t 1.5)) ; fifteen frames' worth of stall + (is (= 45 (clock/frame 30 229))) + (is (= 30 before) "and nothing was counted in between"))) + +(deftest the-frame-is-clamped-into-the-clip + ;; Audio is 7.601s and the clip is 229 frames at 30fps = 7.633s, so the last + ;; fraction of a second has no audio and the end of the audio has no frame + ;; past the last. Neither may produce an out-of-range index. + (clock/attach! (fake :t 99.0)) + (is (= 228 (clock/frame 30 229))) + (clock/attach! (fake :t -1.0)) + (is (= 0 (clock/frame 30 229)))) + +(deftest a-seek-round-trips + ;; Seeking to the START of a frame rather than its middle is what makes this + ;; idempotent: seek to f, read back f, at every f. + (let [a (fake)] + (clock/attach! a) + (doseq [f [0 1 57 114 171 228]] + (clock/seek! 30 229 f) + (is (= f (clock/frame 30 229)) (str "seek to " f))))) + +(deftest a-seek-past-either-end-is-clamped-before-it-reaches-the-element + (let [a (fake)] + (clock/attach! a) + (clock/seek! 30 229 9999) + (is (= 228 (clock/frame 30 229))) + (clock/seek! 30 229 -5) + (is (= 0 (clock/frame 30 229))) + (is (>= (.-currentTime a) 0) "and currentTime is never negative"))) + +(deftest rate-does-not-enter-the-frame-calculation + ;; ½× and ¼× are playbackRate and nothing else. If rate appeared here as well + ;; it would be applied twice — the picture would have one rate and the sound + ;; another, which is precisely the desync the audio clock exists to prevent. + (doseq [r [1.0 0.5 0.25]] + (clock/attach! (fake :t 2.0 :rate r)) + (is (= 60 (clock/frame 30 229)) (str "at " r "x, 2.0s is still frame 60")))) + +(deftest playing-follows-the-element + (clock/attach! (fake :paused? true)) + (is (not (clock/playing?))) + (clock/attach! (fake :paused? false)) + (is (clock/playing?)) + (testing "and a finished file is not playing, whatever `paused` says" + (clock/attach! (fake :paused? false :ended? true)) + (is (not (clock/playing?))))) + +(deftest the-audio-length-is-reported-rather-than-assumed + ;; A clip longer than its audio is a legitimate thing to be told about and not + ;; a thing to silently truncate. + (clock/attach! (fake :duration 7.601)) + (is (= 229 (clock/duration-frames 30))) + (is (= 92 (clock/duration-frames 12))) + (testing "and an unloaded element has no opinion" + (clock/attach! (fake :duration js/NaN)) + (is (nil? (clock/duration-frames 30))))) + +(deftest exposure-is-applied-to-the-derived-frame + ;; The transport shows which frame the grid holds the playhead back onto, so + ;; that `exposure 2` is visibly doing something rather than only inside the + ;; scene. + (is (= [0 0 2 2 4 4] (mapv #(clock/exposed-frame % 2) (range 6)))) + (is (= [0 1 2 3 4 5] (mapv #(clock/exposed-frame % 1) (range 6))))) + +(deftest with-no-element-attached-nothing-explodes + ;; The loop starts before the :ref has fired, so every reader has to be safe + ;; on a clock that has not been handed its element yet. + (clock/attach! nil) + (is (= 0 (clock/frame 30 229))) + (is (not (clock/playing?))) + (is (= 1.0 (clock/rate))) + (is (nil? (clock/duration-frames 30))) + (is (nil? (clock/seek! 30 229 5))) + (is (nil? (clock/pause!)))) diff --git a/frontend/test/arthur/domain/channel_test.cljs b/frontend/test/arthur/domain/channel_test.cljs new file mode 100644 index 0000000..b423b52 --- /dev/null +++ b/frontend/test/arthur/domain/channel_test.cljs @@ -0,0 +1,185 @@ +(ns arthur.domain.channel-test + "The channel is the load-bearing claim of the whole model: analysis and a hand + produce the SAME data, and the only difference between them is a flag nothing + in the renderer reads. These assert the parts of that claim that could silently + stop being true." + (:require [cljs.test :refer [deftest is testing]] + [arthur.domain.channel :as ch])) + +;; ---- the three shapes read the same way ---- + +(deftest framed-is-the-same-value-at-every-frame + (let [c (ch/framed :skin-dark)] + (is (= :framed (ch/describe c))) + (is (every? #(= :skin-dark (ch/value-at c %)) (range -5 20))))) + +(deftest keyed-holds-until-the-next-key + ;; Hold is the DEFAULT, not a special case: docs/design.md requires it of every + ;; cut part, and a tweened mouth reads as puppet software. + (let [c (ch/keyed {0 :a, 4 :b, 12 :c})] + (is (= :keyed (ch/describe c))) + (is (= [:a :a :a :a :b :b :b :b :b :b :b :b :c :c] + (mapv #(ch/value-at c %) (range 0 14)))))) + +(deftest a-frame-before-the-first-key-reads-the-first-key + ;; The JS activeKey clamps low, and that is kept: a channel's first key is the + ;; pose the part starts in. Having NO value is a different question — it is a + ;; state bit, not an empty region of the key map. + (let [c (ch/keyed {10 :a, 20 :b})] + (is (= :a (ch/value-at c 0))) + (is (= :a (ch/value-at c 9))) + (is (= :b (ch/value-at c 999)) "and clamps high by holding the last key"))) + +(deftest keys-are-a-map-so-frame-order-in-the-literal-cannot-matter + ;; Transit and JSON both lose sortedness, so the sorted index is built at read + ;; time. A resolver that trusted insertion order would work in the REPL and + ;; fail after a round trip through the server, which is the worst possible way + ;; to find out. + (let [forward (ch/keyed (array-map 0 :a, 4 :b, 12 :c)) + backward (ch/keyed (array-map 12 :c, 4 :b, 0 :a)) + shuffled (ch/keyed (array-map 4 :b, 12 :c, 0 :a))] + (doseq [c [backward shuffled]] + (is (= (mapv #(ch/value-at forward %) (range 0 16)) + (mapv #(ch/value-at c %) (range 0 16))))))) + +(deftest dense-reads-one-value-per-frame-out-of-a-typed-array + (let [store {"blk" {:data (js/Int16Array. #js [0 0, 10 20, 30 40, 50 60]) :state nil}} + c {:animated? true :interp :hold + :dense {:store "blk" :offset 0 :stride 2 :frames 4} + :generated {:by :roto/lips-outer :analysis "sha256:test"}}] + (is (= :dense (ch/describe c))) + (is (= [[0 0] [10 20] [30 40] [50 60]] + (mapv (fn [f] (let [v (ch/value-at c f store)] + [(ch/component v 0) (ch/component v 1)])) + (range 4)))))) + +(deftest a-dense-value-is-a-view-not-a-copy + ;; Fixed topology is what makes this possible — the frame's data is a + ;; rectangular slice at a known offset — and a copy per node per frame is + ;; exactly the allocation the model exists to avoid. + (let [data (js/Int16Array. #js [1 2 3 4]) + store {"blk" {:data data :state nil}} + c {:animated? true :dense {:store "blk" :offset 0 :stride 2 :frames 2}} + v (ch/value-at c 1 store)] + (is (= (.-buffer data) (.-buffer v)) "shares the block's buffer"))) + +(deftest a-dense-read-clamps-rather-than-running-off-the-end + ;; A time map with an offset deliberately reads the future — mouth lead is the + ;; entire reason :offset exists — so the last frames of a leading track ask for + ;; frames past the end on every take. Clamping holds the final pose; the + ;; alternative blanks the mouth at the end of every clip. + (let [store {"blk" {:data (js/Int16Array. #js [7 8 9]) :state nil}} + c {:animated? true :dense {:store "blk" :offset 0 :stride 1 :frames 3}}] + (is (= 7 (ch/value-at c -4 store))) + (is (= 9 (ch/value-at c 99 store))))) + +(deftest a-dense-channel-whose-store-is-missing-says-so + (let [c {:animated? true :dense {:store "gone" :offset 0 :stride 1 :frames 1}}] + (is (thrown-with-msg? ExceptionInfo #"store key is not in the store" + (ch/value-at c 0 {}))))) + +;; ---- presence is not visibility ---- + +(deftest the-mask-carries-absence-and-vis-carries-hiding + ;; An occluded subject has NO VALUE on a frame. A part being switched off is a + ;; different question and it is `[:vis]`, a channel like any other. Two + ;; mechanisms for one question is how you get a part hidden by one and shown by + ;; the other, so the mask carries absence only. + (let [state (js/Uint8Array. #js [ch/present ch/absent-bit ch/present]) + store {"blk" {:data (js/Int16Array. #js [1 2 3]) :state state}} + c {:animated? true :dense {:store "blk" :offset 0 :stride 1 :frames 3}}] + (is (= 1 (ch/value-at c 0 store))) + (is (= ch/absent (ch/value-at c 1 store))) + (is (= 3 (ch/value-at c 2 store))) + (is (ch/nothing? ch/absent)) + (is (not (ch/nothing? 0)) "zero is a value, not an absence") + (is (not (ch/nothing? false)) "and so is false"))) + +(deftest a-block-with-no-mask-is-present-throughout + ;; The mask is optional: a generator that cannot fail to detect has nothing to + ;; say, and allocating a zeroed byte per frame to say it would be noise. + (let [store {"blk" {:data (js/Int16Array. #js [1 2 3]) :state nil}} + c {:animated? true :dense {:store "blk" :offset 0 :stride 1 :frames 3}}] + (is (= [1 2 3] (mapv #(ch/value-at c % store) (range 3)))))) + +;; ---- the cursor is the playback path and must agree exactly ---- + +(defn- via-cursor + "Sample one cursor at each of `fs` in the order given, which is the point: a + cursor carries state between calls." + [c fs] + (let [cur (ch/cursor c)] + (mapv #(ch/sample! cur %) fs))) + +(deftest the-cursor-agrees-with-the-specification-in-any-frame-order + ;; This is the assertion the cursor exists for. A cursor that drifts produces + ;; the WRONG POSE rather than an error, so nothing would report it: the mouth + ;; would simply be a beat behind on some frames and not others, which reads as + ;; a bad take. + (doseq [[label c] [["sparse" (ch/keyed {0 :a, 4 :b, 12 :c, 13 :d, 40 :e})] + ["one key" (ch/keyed {7 :only})] + ["dense-ish" (ch/keyed (into {} (map (juxt identity #(* 10 %))) (range 40)))] + ["framed" (ch/framed :static)]]] + (let [spec #(ch/value-at c %) + forward (range 0 45) + back (reverse forward) + jumpy [0 44 1 43 12 12 13 3 40 7 0 22 22 21 44]] + (testing label + (doseq [[order-name fs] [["forward" forward] ["backward" back] ["random access" jumpy]]] + (is (= (mapv spec fs) (via-cursor c fs)) + (str label " / " order-name))))))) + +(deftest the-cursor-reads-a-dense-block-too + (let [store {"blk" {:data (js/Float32Array. #js [1 2 3 4 5]) :state nil}} + c {:animated? true :dense {:store "blk" :offset 0 :stride 1 :frames 5}} + cur (ch/cursor c store)] + (is (= [1 2 3 4 5] (mapv #(ch/sample! cur %) (range 5)))) + (is (= [5 1] (mapv #(ch/sample! cur %) [4 0])) "and seeks"))) + +;; ---- what is deliberately not built has to fail loudly ---- + +(deftest an-override-layer-is-refused-rather-than-ignored + ;; :over is specified in docs/animation-model.md and out of scope for this + ;; step. Dropping one silently would present as a hand correction that did not + ;; take — a correction the user made once, watched fail, and has no reason to + ;; trust again. + (let [c (assoc (ch/keyed {0 [0 0]}) :over [{:blend :offset :keys {0 [2 0]}}])] + (is (thrown-with-msg? ExceptionInfo #":over" (ch/value-at c 0))) + (is (thrown-with-msg? ExceptionInfo #":over" (ch/cursor c))) + (is (seq (ch/problems c))))) + +(deftest an-empty-over-is-fine-and-is-what-scenes-carry + (is (empty? (ch/problems (ch/keyed {0 1})))) + (is (= 1 (ch/value-at (ch/keyed {0 1}) 0)))) + +;; ---- shape validation ---- + +(deftest problems-names-the-ways-a-channel-is-malformed + (is (empty? (ch/problems (ch/framed 1)))) + (is (empty? (ch/problems (ch/keyed {0 1})))) + (testing "keys as a vector is the mistake most worth catching" + (is (seq (ch/problems {:animated? true :keys [[0 1]]})))) + (is (seq (ch/problems {:value 1})) "no :animated?") + (is (seq (ch/problems {:animated? true})) "animated with nothing to read") + (is (seq (ch/problems {:animated? true :keys {0 1} + :dense {:store "x" :offset 0 :stride 1 :frames 1}})) + "one shape at a time") + (is (seq (ch/problems {:animated? true :interp :linear :keys {0 1}})) + "only :hold is implemented")) + +(deftest component-reads-vectors-and-typed-arrays-the-same-way + (is (= 3 (ch/component [3 4] 0))) + (is (= 4 (ch/component [3 4] 1))) + (is (= 3 (ch/component (js/Int16Array. #js [3 4]) 0))) + (is (= 4 (ch/component (js/Int16Array. #js [3 4]) 1)))) + +(deftest generated-is-provenance-and-nothing-reads-it + ;; The flag lives on the CHANNEL, not the node, because a mouth wants a + ;; rotoscoped [:geom :pts] and a hand-animated [:xform :pos] at the same time. + ;; What this asserts is that sampling does not depend on it: strip :generated + ;; and every frame is identical. + (let [store {"blk" {:data (js/Int16Array. #js [1 2 3]) :state nil}} + base {:animated? true :dense {:store "blk" :offset 0 :stride 1 :frames 3}} + with (assoc base :generated {:by :roto/lips-outer :params {:verts 8}})] + (is (= (mapv #(ch/value-at base % store) (range 3)) + (mapv #(ch/value-at with % store) (range 3)))))) diff --git a/frontend/test/arthur/domain/node_test.cljs b/frontend/test/arthur/domain/node_test.cljs new file mode 100644 index 0000000..114681a --- /dev/null +++ b/frontend/test/arthur/domain/node_test.cljs @@ -0,0 +1,185 @@ +(ns arthur.domain.node-test + "The transform and the time map. Both are places where a wrong answer looks + like a plausible different answer, which is why they are asserted numerically + rather than looked at." + (:require [cljs.test :refer [deftest is testing]] + [arthur.domain.channel :as ch] + [arthur.domain.node :as node])) + +(defn- pt [m x y] + (let [out (js/Float64Array. 2)] + (node/apply-pt! out 0 m x y) + [(aget out 0) (aget out 1)])) + +(defn- close? [a b] (< (js/Math.abs (- a b)) 1e-12)) +(defn- close-pt? [[ax ay] [bx by]] (and (close? ax bx) (close? ay by))) + +(defn- local [& {:keys [pos rot scale skew anchor] + :or {pos [0 0] rot 0 scale [1 1] skew [0 0] anchor [0 0]}}] + (node/local! (node/mat) pos rot scale skew anchor)) + +;; ---- the transform, component by component ---- + +(deftest the-identity-transform-moves-nothing + (is (= [5 7] (pt (local) 5 7)))) + +(deftest translation-rotation-and-scale-each-do-their-own-job + (is (= [15 27] (pt (local :pos [10 20]) 5 7))) + (is (close-pt? [0 10] (pt (local :rot (/ js/Math.PI 2)) 10 0))) + (is (= [20 21] (pt (local :scale [2 3]) 10 7)))) + +(deftest rotation-and-scale-happen-about-the-anchor + ;; :anchor is Flash's registration point and Blender's origin. Getting it wrong + ;; is why hand-placed parts SWING rather than turn, and a swing looks like a + ;; parenting bug rather than like a wrong pivot. + (let [m (local :rot (/ js/Math.PI 2) :anchor [10 0])] + (is (close-pt? [10 0] (pt m 10 0)) "the anchor itself is a fixed point") + (is (close-pt? [10 10] (pt m 20 0)) "and the rest turns about it")) + (let [m (local :scale [2 2] :anchor [10 10])] + (is (close-pt? [10 10] (pt m 10 10))) + (is (close-pt? [30 30] (pt m 20 20))))) + +(deftest skew-is-shear-factors-so-the-identity-is-zero + ;; Stored as factors rather than angles: kx is x gained per unit y, so a + ;; decomposition round-trips without a tangent, and 0 means "none" rather than + ;; needing atan of something. + (is (= [5 7] (pt (local :skew [0 0]) 5 7))) + (is (= [12 7] (pt (local :skew [1 0]) 5 7)) "kx adds y into x") + (is (= [5 12] (pt (local :skew [0 1]) 5 7)) "ky adds x into y")) + +(deftest the-composition-order-is-the-one-the-model-specifies + ;; local = T(pos) · T(anchor) · R(rot) · K(skew) · S(scale) · T(-anchor) + ;; + ;; Asserted against the product of the five matrices built separately, so the + ;; closed form in node/local! is checked rather than trusted. Every other order + ;; produces a transform that is right at the origin and wrong everywhere else, + ;; which is exactly the kind of wrong that survives inspection. + (let [pos [3 -4] rot 0.7 scale [1.5 0.5] skew [0.25 -0.1] anchor [11 -6] + T (fn [x y] (js/Float64Array. #js [1 0 0 1 x y])) + R (fn [t] (js/Float64Array. #js [(js/Math.cos t) (js/Math.sin t) + (- (js/Math.sin t)) (js/Math.cos t) 0 0])) + K (fn [[kx ky]] (js/Float64Array. #js [1 ky kx 1 0 0])) + S (fn [[sx sy]] (js/Float64Array. #js [sx 0 0 sy 0 0])) + step (fn [acc m] (node/mul! (node/mat) acc m)) + want (reduce step (T (nth pos 0) (nth pos 1)) + [(T (nth anchor 0) (nth anchor 1)) + (R rot) (K skew) (S scale) + (T (- (nth anchor 0)) (- (nth anchor 1)))]) + got (local :pos pos :rot rot :scale scale :skew skew :anchor anchor)] + (is (every? (fn [i] (close? (aget want i) (aget got i))) (range 6)) + (str (vec (array-seq want)) " vs " (vec (array-seq got)))))) + +(deftest mul-may-write-into-either-operand + ;; Evaluation composes world := parent · local with dest aliasing local, so + ;; that a node's world transform needs no scratch. If mul! wrote before reading, + ;; the bug would be a node in the right place whose CHILDREN are wrong. + (let [a (js/Float64Array. #js [2 0 0 3 5 7]) + b (js/Float64Array. #js [1 0.5 -0.5 1 -2 4]) + want (node/mul! (node/mat) a b) + into-b (node/mul! b a b)] + (is (= (vec (array-seq want)) (vec (array-seq into-b)))))) + +(deftest world-composes-through-the-parent + (let [parent (local :pos [100 50] :scale [2 2]) + child (local :pos [10 0]) + w (node/world! (node/mat) parent nil child (node/mat))] + (is (close-pt? [120 50] (pt w 0 0))))) + +(deftest pinv-keeps-a-child-from-jumping-when-it-acquires-a-parent + ;; Blender's parent_inverse. Nothing produces one yet; what is asserted is that + ;; the field is in the composition, because its absence is the kind of thing + ;; that makes a parenting feature feel broken and the fix is a migration. + (let [parent (local :pos [100 50]) + child (local :pos [10 0]) + before (pt child 0 0) + ;; the inverse of the parent at the moment of parenting + pinv (js/Float64Array. #js [1 0 0 1 -100 -50]) + after (pt (node/world! (node/mat) parent pinv child (node/mat)) 0 0)] + (is (close-pt? before after)))) + +(deftest mean-scale-is-exact-for-a-similarity + ;; A disc under a non-uniform transform is an ellipse and the rasteriser has no + ;; ellipse, so a disc's radius takes sqrt|det|. For the similarity the anchor + ;; fit produces — the only transform that reaches a disc today — that is exact. + (is (close? 1.0 (node/mean-scale (local)))) + (is (close? 3.0 (node/mean-scale (local :scale [3 3])))) + (is (close? 3.0 (node/mean-scale (local :scale [3 3] :rot 1.234))) + "and rotation does not change it")) + +;; ---- time maps ---- + +(deftest exposure-floors-and-never-rounds + ;; Rounding would let an output frame read a pose from the FUTURE, which is a + ;; lead — a separate control, applied after this one, for a separate reason. + (is (= [0 1 2 3 4 5] (mapv #(node/expose % 1) (range 6)))) + (is (= [0 0 2 2 4 4] (mapv #(node/expose % 2) (range 6)))) + (is (= [0 0 0 3 3 3] (mapv #(node/expose % 3) (range 6)))) + (is (= 4 (node/expose 5 2)) "frame 5 at exposure 2 reads frame 4, not 6")) + +(deftest exposure-comes-before-offset-and-the-order-is-visible + ;; THE INVARIANT: flooring onto a grid and shifting against the clock do not + ;; commute. Shift first and the floor discards it on most frames, so the lead + ;; slider appears to do nothing at any exposure above 1 — which is + ;; indistinguishable from the slider being unwired. + (let [n {:id :m :time {:mode :map :expose 2 :offset 1}} + got (mapv #(node/local-frame n %) (range 8)) + wrong-way (mapv #(node/expose (+ % 1) 2) (range 8))] + (is (= [1 1 3 3 5 5 7 7] got)) + (is (= [0 2 2 4 4 6 6 8] wrong-way)) + (is (not= got wrong-way) "and the two orders really do differ"))) + +(deftest inherit-is-the-default-and-changes-nothing + (is (= (vec (range 8)) (mapv #(node/local-frame {:id :x} %) (range 8)))) + (is (= (vec (range 8)) (mapv #(node/local-frame {:id :x :time {:mode :inherit}} %) (range 8))))) + +(deftest a-retimed-instance-is-refused-rather-than-ignored + ;; :rate is a symbol instance's timing and symbols are out of scope. Silently + ;; dropping it would be a retimed blink playing at the wrong speed, which looks + ;; like a bad blink and not like a missing feature. + (is (thrown-with-msg? ExceptionInfo #":rate" + (node/local-frame {:id :x :time {:mode :map :rate 0.5}} 0))) + (is (= 4 (node/local-frame {:id :x :time {:mode :map :expose 2 :rate 1.0}} 5)) + "rate 1.0 is the identity and is allowed, because it appears in the spec's example")) + +;; ---- the shape ---- + +(deftest transform-channels-default-to-the-identity + (let [chs (node/channels {:id :x :kind :group})] + (is (= [0.0 0.0] (ch/value-at (get chs [:xform :pos]) 0))) + (is (= [1.0 1.0] (ch/value-at (get chs [:xform :scale]) 0))) + (is (= true (ch/value-at (get chs [:vis]) 0)))) + (testing "and a node's own channels win" + (let [chs (node/channels {:id :x :kind :group + :channels {[:xform :pos] (ch/framed [5 5])}})] + (is (= [5 5] (ch/value-at (get chs [:xform :pos]) 0))) + (is (= [1.0 1.0] (ch/value-at (get chs [:xform :scale]) 0)))))) + +(deftest skew-and-anchor-are-in-the-shape-although-nothing-drives-them + ;; A decomposition is not extensible after the fact: adding a component later + ;; means migrating every stored transform. So both are present from the start, + ;; on every kind. + (doseq [k node/implemented-kinds] + (is (contains? (get node/valid-paths k) [:xform :skew]) (str k)) + (is (contains? (get node/valid-paths k) [:xform :anchor]) (str k)))) + +(deftest valid-paths-follow-from-the-kind + (is (contains? (:poly node/valid-paths) [:geom :pts])) + (is (not (contains? (:poly node/valid-paths) [:geom :radius]))) + (is (contains? (:disc node/valid-paths) [:geom :radius])) + (is (contains? (:rect node/valid-paths) [:geom :size])) + (is (not (contains? (:group node/valid-paths) [:geom :pts])) + "a group is a pure transform node")) + +(deftest problems-names-the-ways-a-node-is-malformed + (is (empty? (node/problems {:id :x :kind :group :z "a1"}))) + (is (seq (node/problems {:kind :group :z "a1"})) "no :id") + (is (seq (node/problems {:id :x :kind :blob :z "a1"})) "not a kind") + (is (seq (node/problems {:id :x :kind :symbol :z "a1"})) "a kind that is not built") + (is (seq (node/problems {:id :x :kind :group})) "no :z") + (is (seq (node/problems {:id :x :kind :group :z "a1" :span [3]})) "a malformed span") + (is (seq (node/problems {:id :x :kind :group :z "a1" + :channels {[:geom :pts] (ch/framed [0 0 1 0 1 1])}})) + "a channel that is not valid on this kind") + (is (seq (node/problems {:id :x :kind :poly :z "a1" + :channels {[:geom :pts] {:animated? true}}})) + "and a channel that is malformed in itself")) diff --git a/frontend/test/arthur/domain/raster_test.cljs b/frontend/test/arthur/domain/raster_test.cljs index b3d8a97..a4f78f5 100644 --- a/frontend/test/arthur/domain/raster_test.cljs +++ b/frontend/test/arthur/domain/raster_test.cljs @@ -131,3 +131,61 @@ (is (= [18 20 28] (pal/hex->rgb "#12141c"))) (testing "index-of round-trips against the ordered vector" (is (every? (fn [[k i]] (= k (:name (nth pal/entries i)))) pal/index-of)))) + +;; ---- the flat buffer path is the same scanline fill ---- + +(deftest the-flat-and-map-polygon-forms-fill-identically + ;; fill-poly! is a thin wrapper over fill-poly-buf! rather than a second + ;; implementation. ONE scanline fill serves the analysis stages, which speak + ;; {:x :y}, and frame evaluation, which hands over a preallocated flat buffer. + ;; Two would drift, and the drift would read as a rendering bug rather than as + ;; two functions disagreeing. + (doseq [[label pts] [["axis-aligned rect" [{:x 8 :y 8} {:x 56 :y 8} {:x 56 :y 40} {:x 8 :y 40}]] + ["fractional" [{:x 3.5 :y 2.25} {:x 40.75 :y 5.5} + {:x 30.5 :y 44.5} {:x 6.25 :y 20.5}]] + ["concave" [{:x 4 :y 4} {:x 60 :y 4} {:x 60 :y 44} + {:x 32 :y 20} {:x 4 :y 44}]] + ["off the edges" [{:x -20 :y -10} {:x 80 :y 4} {:x 70 :y 60} + {:x -5 :y 50}]] + ["degenerate" [{:x 5 :y 5} {:x 20 :y 5}]]]] + (let [flat (js/Float64Array. (mapcat (juxt :x :y) pts)) + a (-> (r/make 64 48) (r/clear! 0) (r/fill-poly! pts 3)) + b (-> (r/make 64 48) (r/clear! 0) (r/fill-poly-buf! flat (count pts) 3))] + (is (= (vec (array-seq (:buf a))) (vec (array-seq (:buf b)))) label)))) + +(deftest fill-poly-buf-uses-only-the-first-n-points + ;; The buffer is preallocated at the node's vertex capacity, so a channel + ;; carrying fewer points than the buffer holds must not read the stale tail of + ;; the previous frame. + (let [big (js/Float64Array. 32)] + (doseq [[i v] (map-indexed vector [4 4, 20 4, 20 20, 4 20])] + (aset big i v)) + ;; leave garbage past the triangle + (aset big 8 999) (aset big 9 999) + (let [tri (-> (r/make 32 32) (r/clear! 0) (r/fill-poly-buf! big 3 1))] + (is (pos? (count-index tri 1))) + (is (zero? (aget (:buf tri) (+ (* 31 32) 31))) "and nothing leaked to the far corner")))) + +(deftest rgba-can-write-into-a-buffer-it-was-given + ;; At 320x200 the expanded buffer is 256KB; allocating and discarding that + ;; thirty times a second is exactly the per-frame allocation the model is + ;; arranged to avoid, so ui/canvas hands over the live ImageData's own array. + (let [ras (-> (r/make 4 4) (r/clear! 2)) + dest (js/Uint8ClampedArray. (* 4 4 4)) + {:keys [data]} (r/->rgba ras pal/rgb 1 dest)] + (is (identical? dest data)) + (is (= (pal/rgb 2) [(aget dest 0) (aget dest 1) (aget dest 2)])))) + +(deftest draw-ops-is-the-boundary-with-the-model + ;; An op carries raster-space points and a PALETTE INDEX, and the rasteriser + ;; knows nothing about nodes, channels, time maps or provenance. So a + ;; hand-written op list rasterises, and an op kind it cannot draw says so. + (let [ras (-> (r/make 40 40) (r/clear! 0))] + (r/draw-ops! ras + [{:kind :poly :pts (js/Float64Array. #js [10 10 30 10 30 20 10 20]) :n 4 + :color 1 :stencil nil} + {:kind :disc :cx 28 :cy 15 :r 9 :color 2 :stencil 1} + {:kind :rect :cx 28 :cy 15 :size 5 :color 4 :stencil 2}]) + (is (= #{0 1 2 4} (set (array-seq (:buf ras))))) + (is (thrown-with-msg? ExceptionInfo #"not rasterisable" + (r/draw-ops! ras [{:kind :bitmap}]))))) diff --git a/frontend/test/arthur/domain/scene_test.cljs b/frontend/test/arthur/domain/scene_test.cljs new file mode 100644 index 0000000..2404465 --- /dev/null +++ b/frontend/test/arthur/domain/scene_test.cljs @@ -0,0 +1,394 @@ +(ns arthur.domain.scene-test + "Frame evaluation, and the hand-written scene. + + port-plan step 2 exists to find out whether the data model works BEFORE nine + hundred lines of measurement are ported into it, so these assertions are about + the model's claims rather than about a look: that structure is flat and + addressable, that draw order is authored, that time maps compose, that + presence and visibility are different questions, and that the fast path and the + specification give the same frame." + (:require [cljs.test :refer [deftest is testing]] + [arthur.demo :as demo] + [arthur.domain.channel :as ch] + [arthur.domain.node :as node] + [arthur.domain.palette :as pal] + [arthur.domain.raster :as raster] + [arthur.domain.scene :as scene])) + +(defn- poly [id parent z pts color & [extra]] + (merge {:id id :kind :poly :parent parent :z z + :channels {[:geom :pts] (ch/framed pts) + [:style :color] (ch/framed color)}} + extra)) + +(defn- sc [& nodes] + {:nodes (into {} (map (juxt :id identity)) nodes)}) + +(defn- ids-at [scene f] + (mapv :node (scene/eval-frame scene f))) + +(defn- pts-of [op] + (mapv (fn [i] [(aget (:pts op) (* 2 i)) (aget (:pts op) (inc (* 2 i)))]) + (range (:n op)))) + +;; ---- structure ---- + +(deftest depth-order-puts-every-node-after-its-parent + (let [s (sc {:id :a :kind :group :z "a1"} + {:id :b :kind :group :parent :a :z "a1"} + {:id :c :kind :group :parent :b :z "a1"} + {:id :d :kind :group :parent :a :z "a2"}) + ord (scene/order (:nodes s))] + (is (= 0 (scene/depth (:nodes s) :a))) + (is (= 2 (scene/depth (:nodes s) :c))) + (let [pos (into {} (map-indexed (fn [i id] [id i])) ord)] + (doseq [[id p] [[:b :a] [:c :b] [:d :a]]] + (is (< (get pos p) (get pos id)) (str p " must come before " id)))))) + +(deftest a-parent-cycle-throws-instead-of-hanging + ;; Reachable from one bad :node/set-parent, and a hung tab is a far worse + ;; diagnostic than a stack trace naming the nodes. + (let [s (sc {:id :a :kind :group :parent :b :z "a1"} + {:id :b :kind :group :parent :a :z "a1"})] + (is (thrown-with-msg? ExceptionInfo #"cycle" (scene/order (:nodes s)))) + (is (seq (scene/problems s))))) + +(deftest a-missing-parent-is-named-rather-than-silently-orphaning + (let [s (sc {:id :a :kind :group :parent :nope :z "a1"})] + (is (seq (scene/problems s))))) + +(deftest reparenting-is-one-field-and-does-not-move-a-subtree + ;; The flat-with-pointers claim, asserted as the thing it buys: a reparent is an + ;; assoc-in at one node, and nothing else in the map changes identity — which is + ;; what keeps re-frame's ancestor subs from invalidating. + (let [s (sc {:id :a :kind :group :z "a1"} + {:id :b :kind :group :z "a2" :channels {[:xform :pos] (ch/framed [100 0])}} + (poly :c :a "a1" [0 0 10 0 10 10] :brow)) + s' (assoc-in s [:nodes :c :parent] :b)] + (is (identical? (get-in s [:nodes :a]) (get-in s' [:nodes :a])) + "the old parent is the same object") + (is (identical? (get-in s [:nodes :b]) (get-in s' [:nodes :b])) + "and so is the new one") + (is (= [[0 0] [10 0] [10 10]] + (pts-of (first (filter #(= :c (:node %)) (scene/eval-frame s 0)))))) + (is (= [[100 0] [110 0] [110 10]] + (pts-of (first (filter #(= :c (:node %)) (scene/eval-frame s' 0)))))))) + +;; ---- draw order ---- + +(deftest draw-order-is-depth-first-by-sibling-z + ;; z is a fractional index among siblings, so the sort key is the chain of z + ;; values from the root. A parent's chain is a PREFIX of its child's, which is + ;; why a parent draws before its children without that being a special case. + (let [s (sc {:id :root :kind :group :z "a1"} + (poly :under :root "a0" [0 0 1 0 1 1] :bg) + {:id :mid :kind :group :parent :root :z "a1"} + (poly :deep :mid "a5" [0 0 1 0 1 1] :brow) + (poly :over :root "a2" [0 0 1 0 1 1] :teeth))] + (is (= [:under :deep :over] (ids-at s 0))))) + +(deftest a-deep-child-of-an-early-sibling-still-draws-before-a-later-sibling + ;; The failure this guards: comparing z paths with `compare` would compare + ;; COUNT first, so a painted cel three levels under "a1" would jump in front of + ;; a bare "a2". It reads as a layer order that mostly works. + (let [s (sc {:id :root :kind :group :z "a1"} + {:id :g1 :kind :group :parent :root :z "a1"} + {:id :g2 :kind :group :parent :g1 :z "a1"} + (poly :deep :g2 "a1" [0 0 1 0 1 1] :brow) + (poly :shallow :root "a2" [0 0 1 0 1 1] :teeth))] + (is (= [:deep :shallow] (ids-at s 0))))) + +(deftest a-fractional-index-inserts-between-two-siblings-without-renumbering + (let [base (sc {:id :root :kind :group :z "a1"} + (poly :a :root "a1" [0 0 1 0 1 1] :bg) + (poly :c :root "a3" [0 0 1 0 1 1] :teeth)) + with (assoc-in base [:nodes :b] (poly :b :root "a2" [0 0 1 0 1 1] :brow))] + (is (= [:a :c] (ids-at base 0))) + (is (= [:a :b :c] (ids-at with 0))) + (is (= (get-in base [:nodes :a]) (get-in with [:nodes :a])) "and :a is untouched"))) + +;; ---- transform composition through the tree ---- + +(deftest geometry-lands-in-the-parents-space + (let [s (sc {:id :g :kind :group :z "a1" + :channels {[:xform :pos] (ch/framed [100 50]) + [:xform :scale] (ch/framed [2 2])}} + (poly :p :g "a1" [0 0 10 0 10 10 0 10] :skin-base)) + op (first (scene/eval-frame s 0))] + (is (= [[100 50] [120 50] [120 70] [100 70]] (pts-of op))))) + +(deftest a-keyed-group-position-moves-its-children-and-holds-between-keys + ;; This is the scene the plan asks for, minimally: a rectangle parented to a + ;; group whose [:xform :pos] is keyed on four frames. + (let [s (sc {:id :g :kind :group :z "a1" + :channels {[:xform :pos] + (ch/keyed {0 [0 0], 4 [10 0], 8 [10 10], 12 [0 10]})}} + (poly :p :g "a1" [0 0 2 0 2 2] :skin-base)) + at #(first (pts-of (first (scene/eval-frame s %))))] + (is (= [0 0] (at 0))) + (is (= [0 0] (at 3)) "held") + (is (= [10 0] (at 4))) + (is (= [10 10] (at 8))) + (is (= [0 10] (at 12))) + (is (= [0 10] (at 99)) "and holds the last key"))) + +;; ---- time maps compose along the chain ---- + +(deftest exposure-on-the-root-is-inherited-by-everything-under-it + ;; docs/design.md is emphatic that everything rides ONE grid: a head cutting on + ;; odd frames against a mouth cutting on even ones reads as two performances. + (let [s (sc {:id :root :kind :group :z "a1" :time {:mode :map :expose 3}} + {:id :g :kind :group :parent :root :z "a1" + :channels {[:xform :pos] (ch/keyed (into {} (map (juxt identity #(vector % 0))) (range 12)))}} + (poly :p :g "a1" [0 0 1 0 1 1] :skin-base)) + x-at #(first (first (pts-of (first (scene/eval-frame s %)))))] + (is (= [0 0 0 3 3 3 6 6 6 9 9 9] (mapv x-at (range 12))))) + + (testing "and a node may set its own grid, which the model permits deliberately" + (let [s (sc {:id :root :kind :group :z "a1" :time {:mode :map :expose 2}} + {:id :g :kind :group :parent :root :z "a1" :time {:mode :map :expose 4} + :channels {[:xform :pos] (ch/keyed (into {} (map (juxt identity #(vector % 0))) (range 12)))}} + (poly :p :g "a1" [0 0 1 0 1 1] :skin-base)) + x-at #(first (first (pts-of (first (scene/eval-frame s %)))))] + (is (= [0 0 0 0 4 4 4 4 8 8 8 8] (mapv x-at (range 12))))))) + +(deftest offset-is-per-node-which-is-the-entire-point-of-mouth-lead + ;; Lead applies to performance nodes and NOT to the plate. If it were a clip + ;; property the mouth would drag the whole head forward with it. + (let [keys (into {} (map (juxt identity #(vector % 0))) (range 12)) + s (sc {:id :root :kind :group :z "a1"} + {:id :plate :kind :group :parent :root :z "a1" + :channels {[:xform :pos] (ch/keyed keys)}} + (poly :plate-p :plate "a1" [0 0 1 0 1 1] :skin-base) + {:id :mouth :kind :group :parent :root :z "a2" :time {:mode :map :offset 2} + :channels {[:xform :pos] (ch/keyed keys)}} + (poly :mouth-p :mouth "a1" [0 0 1 0 1 1] :mouth-dark)) + x-of (fn [f id] (->> (scene/eval-frame s f) + (filter #(= id (:node %))) first pts-of first first))] + (is (= [0 1 2 3] (mapv #(x-of % :plate-p) (range 4)))) + (is (= [2 3 4 5] (mapv #(x-of % :mouth-p) (range 4))) "the mouth reads ahead"))) + +;; ---- span and visibility are different questions ---- + +(deftest span-removes-a-node-and-vis-switches-it-off + ;; :span is Lottie's ip/op and Flash's PlaceObject/RemoveObject: the range over + ;; which the node EXISTS. [:vis] blinks an existing node on and off. Conflating + ;; them is how you end up with a part that holds a stale pose outside its range. + (let [s (sc {:id :root :kind :group :z "a1"} + (poly :p :root "a1" [0 0 1 0 1 1] :brow + {:span [2 5] + :channels {[:geom :pts] (ch/framed [0 0 1 0 1 1]) + [:style :color] (ch/framed :brow) + [:vis] (ch/keyed {0 true, 3 false, 4 true})}}))] + (is (= [[] [] [:p] [] [:p] [] []] (mapv #(ids-at s %) (range 7)))))) + +(deftest a-hidden-group-takes-its-children-with-it + (let [s (sc {:id :g :kind :group :z "a1" + :channels {[:vis] (ch/keyed {0 true, 2 false})}} + (poly :p :g "a1" [0 0 1 0 1 1] :brow))] + (is (= [:p] (ids-at s 0))) + (is (= [] (ids-at s 2))))) + +(deftest an-absent-transform-drops-the-subtree-and-an-absent-geometry-does-not + ;; The asymmetry is the whole reason presence is tracked per CHANNEL rather than + ;; per node. An absent mouth outline has nothing to draw, but the head it hangs + ;; off is still exactly where it was. + (let [state (js/Uint8Array. #js [ch/present ch/absent-bit]) + store {"pos" {:data (js/Float32Array. #js [0 0, 0 0]) :state state} + "pts" {:data (js/Int16Array. #js [0 0 1 0 1 1, 0 0 1 0 1 1]) :state state}} + absent-pos (sc {:id :g :kind :group :z "a1" + :channels {[:xform :pos] {:animated? true + :dense {:store "pos" :offset 0 :stride 2 :frames 2}}}} + (poly :child :g "a1" [0 0 1 0 1 1] :brow)) + absent-pts (sc {:id :g :kind :group :z "a1"} + {:id :m :kind :poly :parent :g :z "a1" + :channels {[:geom :pts] {:animated? true + :dense {:store "pts" :offset 0 :stride 6 :frames 2}} + [:style :color] (ch/framed :mouth-dark)}} + (poly :teeth :m "a2" [0 0 1 0 1 1] :teeth))] + (is (= [:child] (mapv :node (scene/eval-frame absent-pos 0 store)))) + (is (= [] (mapv :node (scene/eval-frame absent-pos 1 store))) + "an absent transform gives the children nowhere to be") + (is (= [:m :teeth] (mapv :node (scene/eval-frame absent-pts 0 store)))) + (is (= [:teeth] (mapv :node (scene/eval-frame absent-pts 1 store))) + "an absent outline removes only itself"))) + +;; ---- stencils ---- + +(deftest a-stencil-resolves-to-the-stencil-nodes-palette-index + ;; A stencil is a COLOUR KEY, not a node reference — the take format's clip= — + ;; and the indexed buffer being its own clip mask is what keeps the iris inside + ;; the eye at any gaze and any radius with no clamp anywhere. + (let [s (sc {:id :root :kind :group :z "a1"} + (poly :sclera :root "a1" [0 0 10 0 10 10] :eye-white) + {:id :iris :kind :disc :parent :root :stencil :sclera :z "a2" + :channels {[:geom :radius] (ch/framed 4) + [:style :color] (ch/framed :iris)}}) + ops (scene/eval-frame s 0)] + (is (= [:sclera :iris] (mapv :node ops))) + (is (= (:eye-white pal/index-of) (:stencil (second ops)))))) + +(deftest a-node-stencilled-by-something-that-drew-nothing-is-dropped + ;; Unclipped would be an iris floating over the cheek on exactly the frames + ;; where the eye is missing, which is worse than a missing iris. + (let [s (sc {:id :root :kind :group :z "a1"} + (poly :sclera :root "a1" [0 0 10 0 10 10] :eye-white + {:channels {[:geom :pts] (ch/framed [0 0 10 0 10 10]) + [:style :color] (ch/framed :eye-white) + [:vis] (ch/keyed {0 true, 1 false})}}) + {:id :iris :kind :disc :parent :root :stencil :sclera :z "a2" + :channels {[:geom :radius] (ch/framed 4) + [:style :color] (ch/framed :iris)}})] + (is (= [:sclera :iris] (ids-at s 0))) + (is (= [] (ids-at s 1))))) + +;; ---- discs and rects ---- + +(deftest a-discs-radius-takes-the-mean-scale-and-a-rects-size-is-rounded + (let [s (sc {:id :g :kind :group :z "a1" + :channels {[:xform :pos] (ch/framed [50 60]) [:xform :scale] (ch/framed [2 2])}} + {:id :d :kind :disc :parent :g :z "a1" + :channels {[:geom :radius] (ch/framed 3) [:style :color] (ch/framed :iris)}} + {:id :r :kind :rect :parent :g :z "a2" + :channels {[:geom :size] (ch/framed 1.7) [:style :color] (ch/framed :pupil)}}) + [d r] (scene/eval-frame s 0)] + (is (= [50 60 6] [(:cx d) (:cy d) (:r d)])) + ;; 1.7 x 2 is 3.4, and a block 3.4px wide would be 3px on one frame and 4 on + ;; the next, which reads as the pupil breathing. + (is (= 3 (:size r))))) + +;; ---- the fast path and the specification agree ---- + +(deftest the-resolver-agrees-with-eval-frame-in-any-frame-order + ;; THE assertion of this step. The resolver caches the topological order and the + ;; z paths, holds a cursor per channel and reuses one point buffer per node, and + ;; every one of those is a way to be subtly wrong on some frames and not others + ;; — which presents as a bad take rather than as an error. + (let [s demo/scene + res (scene/resolver s) + n (:frames s) + snapshot (fn [ops] + (mapv (fn [op] + (cond-> (dissoc op :pts :i) + (:pts op) (assoc :points (pts-of op)))) + ops))] + (doseq [[label fs] [["forward" (range n)] + ["backward" (reverse (range n))] + ["random access" [0 71 5 5 40 6 70 1 23 24 25 24 23 0 47 48]] + ["every third" (range 0 n 3)]]] + (testing label + (doseq [f fs] + (is (= (snapshot (scene/eval-frame s f)) (snapshot (res f))) + (str label " at frame " f))))))) + +(deftest the-resolver-reuses-one-buffer-per-node + ;; At 30fps per-frame allocation is the only thing that will make this stutter, + ;; and fixed topology is what makes the buffer size knowable at all. + (let [res (scene/resolver demo/scene) + buf-of (fn [f id] (->> (res f) (filter #(= id (:node %))) first :pts))] + (is (identical? (buf-of 0 :card) (buf-of 30 :card))))) + +;; ---- the hand-written scene, end to end ---- + +(deftest the-hand-written-scene-is-valid + (is (= "" (scene/problems-str demo/scene))) + (is (pos? (:frames demo/scene)))) + +(deftest the-hand-written-scene-renders-and-moves + ;; port-plan step 2's done condition, as an assertion rather than a look: the + ;; scene rasterises, it writes only palette indices, and the pixels are not the + ;; same on every frame. + (let [res (scene/resolver demo/scene) + render (fn [f] + (let [r (raster/make demo/width demo/height)] + (raster/clear! r (:bg pal/index-of)) + (raster/draw-ops! r (res f)) + r)) + frames (mapv render (range 0 (:frames demo/scene) 6)) + sig (fn [r] (vec (array-seq (:buf r))))] + (is (every? (fn [r] (every? #(< % (count pal/rgb)) (array-seq (:buf r)))) frames) + "every byte written is a real palette index") + (is (> (count (distinct (map sig frames))) 1) "something moves") + (testing "the mark actually covers pixels" + (is (pos? (count (remove zero? (sig (first frames))))))))) + +(deftest the-hand-written-scene-steps-on-the-exposure-grid + ;; Exposure 2 on the clip root, inherited, so odd frames are identical to the + ;; even frame before them. If this fails, exposure is being applied somewhere + ;; other than the frame the channels are sampled at. + (let [res (scene/resolver demo/scene) + render (fn [f] + (let [r (raster/make demo/width demo/height)] + (raster/clear! r (:bg pal/index-of)) + (raster/draw-ops! r (res f)) + (vec (array-seq (:buf r)))))] + (doseq [f (range 0 (:frames demo/scene) 2)] + (is (= (render f) (render (inc f))) (str "frame " (inc f) " must hold frame " f))) + ;; Two grid slots that straddle a key, not two adjacent ones: between keys + ;; nothing changes, because that is what hold MEANS. The scene's second key + ;; is at 57, and exposure 2 floors that onto 58 — which is itself the + ;; expose-before-anything-else rule showing up in pixels. + (is (not= (render 56) (render 58)) "and a key on the grid is seen"))) + +(deftest the-hand-written-scene-keeps-the-iris-and-pupil-inside-the-card + ;; The stencil chain, on real pixels: the iris is clipped by the card and the + ;; pupil by the iris, and neither is expressed anywhere as a chain. + (let [res (scene/resolver demo/scene)] + (doseq [f (range 0 (:frames demo/scene) 4)] + (let [before (raster/make demo/width demo/height) + after (raster/make demo/width demo/height) + ops (res f) + card? (fn [op] (= :card (:node op)))] + (raster/clear! before (:bg pal/index-of)) + (raster/draw-ops! before (filter card? ops)) + (raster/clear! after (:bg pal/index-of)) + (raster/draw-ops! after ops) + (let [ci (:skin-base pal/index-of) + card (set (for [i (range (alength (:buf before))) + :when (= ci (aget (:buf before) i))] + i)) + eye (set (for [i (range (alength (:buf after))) + :when (#{(:iris pal/index-of) (:pupil pal/index-of)} + (aget (:buf after) i))] + i))] + (is (pos? (count eye)) (str "frame " f ": the iris drew something")) + (is (empty? (remove card eye)) + (str "frame " f ": " (count (remove card eye)) " pixels outside the card"))))))) + +;; ---- the palette is a parameter, not a global ---- + +(deftest the-same-scene-resolves-differently-under-a-different-ramp + ;; A node names a TONE; which ramp that tone is read in belongs to the timeline + ;; it sits in. So resolution must not reach for one ambient answer — the same + ;; drawing has to read day or night without a stored value changing, which is + ;; the entire payoff of indexed colour. + (let [s (sc {:id :root :kind :group :z "a1"} + (poly :p :root "a1" [0 0 10 0 10 10] :skin-base)) + day {:skin-base 1} + night {:skin-base 17}] + (is (= 1 (:color (first (scene/eval-frame s 0 nil day))))) + (is (= 17 (:color (first (scene/eval-frame s 0 nil night))))) + (is (= 17 (:color (first ((scene/resolver s nil night) 0)))) + "and the playback path agrees"))) + +(deftest a-tone-the-ramp-does-not-define-is-loudly-wrong + ;; 255 renders magenta. Naming a colour the ramp has no entry for is a bug in + ;; authored data and should be impossible to miss. + (let [s (sc {:id :root :kind :group :z "a1"} + (poly :p :root "a1" [0 0 10 0 10 10] :skin-base))] + (is (= 255 (:color (first (scene/eval-frame s 0 nil {}))))))) + +(deftest partitioning-the-index-space-stops-two-palettes-colliding-on-a-stencil + ;; A stencil is a colour key, so two nodes sharing a tone share a stencil — + ;; a real weakness of the technique. Concatenating the named palettes into one + ;; index space means two nodes in DIFFERENT palettes cannot collide at all. + (let [s (sc {:id :root :kind :group :z "a1"} + (poly :sclera :root "a1" [0 0 20 0 20 20] :eye-white) + {:id :iris :kind :disc :parent :root :stencil :sclera :z "a2" + :channels {[:geom :radius] (ch/framed 4) + [:style :color] (ch/framed :iris)}}) + ;; :night's tones sit above :day's in one concatenated space + night {:eye-white 14 :iris 15} + ops (scene/eval-frame s 0 nil night)] + (is (= 14 (:stencil (second ops))) + "the stencil resolves to the index the stencil node actually drew in")))