Represent tracing media as symbols and add pool thumbnails
This commit is contained in:
parent
551d572347
commit
17e1b4f403
49 changed files with 1777 additions and 864 deletions
|
|
@ -605,28 +605,26 @@ dense geometry. A topology setting cannot be treated as a per-frame gain curve.
|
|||
The op list is the boundary with stage 7 in `docs/architecture.md`: the
|
||||
rasteriser takes ops and knows nothing about nodes, channels or time.
|
||||
|
||||
**A photographic underlay is not an op.** The registered source frame that an
|
||||
animator traces over is a reference, not output, and it may not enter the indexed
|
||||
buffer — the same rule `docs/architecture.md` already sets for handles and
|
||||
vertex boxes. It is a `drawImage` at an affine on a separate canvas, which clips
|
||||
at the canvas edge for free, and the only thing it needs from the model is the
|
||||
world transform of the node it rides:
|
||||
**A tracing layer is an op that never reaches the raster.** Footage or a still
|
||||
to draw over is a symbol with `:type :trace` and a `:media`, placed by an ordinary
|
||||
instance — so it is moved, scaled, trimmed, held and put in a lane like anything
|
||||
else — and it resolves to one `:trace` op: `{:kind :trace :node :layer :media
|
||||
:frame :size :m}`. The raster refuses that kind, the player hands it to a
|
||||
`drawImage` on a separate canvas over the picture, and `clip/resolver` makes one
|
||||
only when asked with `:tracing?`, which only the stage does. An export, a
|
||||
symbol's centre and a thumbnail never ask, so a reference cannot reach the
|
||||
picture by any path that forgets to filter it. See `docs/tracing-symbol-plan.md`.
|
||||
|
||||
```clojure
|
||||
(world-of resolver :head) ;; -> Float64Array[6]
|
||||
```
|
||||
A face's footage is one of these, placed as `:plate` under `:head` with the
|
||||
anchor fit itself as its measured transform — the inverse of the head's, over
|
||||
image height. Its world is `head · fit · 1/H`, so on a frame where the head and
|
||||
the plate read the same measured frame the two cancel and the photo sits where
|
||||
the face was filmed; on any other frame it rides the head. Registration is the
|
||||
ordinary walk, not a matrix built beside it.
|
||||
|
||||
Composed with image-pixels-to-local — **both axes divided by `imgH`**, never by
|
||||
their own dimension — the photo is registered with the shapes by construction,
|
||||
and an unregistered underlay is merely decorative. The tracing editor chooses
|
||||
which source frame to show under a cel. That reference choice is independent of
|
||||
the finished picture fps and does not change the dense analysis track. A cel can
|
||||
therefore use any useful source frame as its drawing reference, even when that
|
||||
frame is not one of the displayed picture poses.
|
||||
|
||||
A photo that has to sit *between* two drawn layers is the case that would make it
|
||||
a `:bitmap` node with an op of its own. Nothing wants that yet: a reference is
|
||||
either under everything or over everything at low alpha.
|
||||
Which frame it shows is the placement's: `:time {:holds [...]}` holds it on
|
||||
chosen frames, and a head with `:reads {:holds-of :plate}` jumps to the same
|
||||
ones. Whether it is showing at all is the editor's, `[:ui :tracing]`.
|
||||
|
||||
### Making it fast in CLJS
|
||||
|
||||
|
|
@ -680,8 +678,8 @@ Proof that it covers what exists, not just what is wanted:
|
|||
| brow ring + quantised raise | node `:brow-r`, `[:geom :pts]` dense (the traced ring with height removed), `[:xform :pos]` dense (the quantised raise). **The decomposition design.md insists on is two channels.** |
|
||||
| head plate, kept frames | node `:head`, `:symbol` per instance, keys on `[:symbol]` at kept frames |
|
||||
| `makeXform` face-oval crop | **gone.** Placement is `[:xform :*]` on the face's own `:place`; the stage clips |
|
||||
| `stabilize` transforms | dense `[:xform :*]` on `:head`, read through its optional `:anchors` map |
|
||||
| registered underlay | not data — a UI layer riding `(world-of resolver :head)` |
|
||||
| `stabilize` transforms | dense `[:xform :*]` on `:head` (the inverse fit) and on its `:plate` (the fit), read where `:reads` and `:time :holds` say |
|
||||
| registered underlay | the face's `:plate`, an instance of the footage's tracing symbol under `:head`; a `:trace` op the raster never sees |
|
||||
| painted background cel | node per layer, `[:geom :pts]` **framed**, `[:style :color]` framed |
|
||||
| `mouth lead` | `:time {:offset k}` on performance nodes only |
|
||||
| `exposure` | `:time {:expose n}` on the clip root, inherited |
|
||||
|
|
|
|||
|
|
@ -1,5 +1,11 @@
|
|||
# Frame selection
|
||||
|
||||
> Since `docs/tracing-symbol-plan.md`: `domain/trace` is gone. Trace keys are
|
||||
> the face's `:plate` placement's `:time :holds`, the origin is the head's
|
||||
> `:reads`, and the photo's registration is the plate's own measured channels.
|
||||
> Where this document names `trace/measured-local` or `trace/prepare`, read the
|
||||
> plate's or the head's measured channels and `node/hold`.
|
||||
|
||||
Two mechanisms. One vocabulary. An earlier draft of this document claimed they
|
||||
were one component used twice — because `suggestPlateFrames` in the old
|
||||
`js/pipeline.js` and the never-built "performance poses" of
|
||||
|
|
@ -21,7 +27,7 @@ This document is how the picture gets an opinion.
|
|||
holding the latest one at or before now.**
|
||||
|
||||
The holding half already exists and is already shared: `pose/held-frame` is called
|
||||
by `trace/held-frame` and by `pose/source-frame`, which is the two sites agreeing
|
||||
by `node/hold` (a placement's `:time :holds`) and by `pose/source-frame`, which is the two sites agreeing
|
||||
about reading. The hand half is shared too — see *Three layers* below. Choosing is
|
||||
what differs.
|
||||
|
||||
|
|
@ -32,9 +38,9 @@ what differs.
|
|||
| signal | the measured head's motion | — none; a stored cut |
|
||||
| the baseline it improves on | drawing on 2s | the cadence, or the exposure grid |
|
||||
| the shape of the answer | a non-uniform set out of dense | the same grid, nudged |
|
||||
| lives on | the face's `:head` `:trace` | the instance's `:playback :tracks` |
|
||||
| hand edit today | `trace/toggle-frame` | `pose/put-cut` / `pose/remove-cut` |
|
||||
| UI today | `params/trace-keys` | **none** |
|
||||
| lives on | the face's `:plate` `:time :holds` | the instance's `:playback :tracks` |
|
||||
| hand edit today | `::project/toggle-hold` | `pose/put-cut` / `pose/remove-cut` |
|
||||
| UI today | `params/layer-section` | **none** |
|
||||
| proposes today | **nothing** | **nothing** |
|
||||
|
||||
## Why they are not one function
|
||||
|
|
@ -97,7 +103,7 @@ implementation for both. A preserve mark *is* a keep.
|
|||
### Materialise the result, do not derive it on the render path
|
||||
|
||||
The effective set is written back to where each site already reads it —
|
||||
`:trace :frames`, or the pose track — so that every existing reader is untouched
|
||||
the plate's `:time :holds`, or the pose track — so that every existing reader is untouched
|
||||
and nothing on the per-frame path has to open a dense block. Proposing is a
|
||||
command, not a subscription. `ch/value-at` allocates per call and says so; that
|
||||
is fine for a button press over a few hundred frames and would not be fine at
|
||||
|
|
@ -349,7 +355,7 @@ against anything.
|
|||
|
||||
### Between the kept frames is a third shared field
|
||||
|
||||
`:trace :origin` is not a tracing setting. It is the answer to *what happens
|
||||
The head's `:reads` (once `:trace :origin`) is not a tracing setting. It is the answer to *what happens
|
||||
between kept frames*, and the plate selection has to answer it:
|
||||
|
||||
- `:continuous` — ignore the selection for this purpose and read the frame you
|
||||
|
|
@ -430,7 +436,7 @@ Render it in two sections:
|
|||
invariant from *The upgrade path* here, since this is the first place a real
|
||||
signal exists to assert it over.
|
||||
4. **Storage for the plate selection.** `:policy`/`:keep`/`:drop` beside
|
||||
`:trace :frames`. Extend `leaf/leaves` and the key whitelists in the same
|
||||
the plate's `:time :holds`. Extend `leaf/leaves` and the key whitelists in the same
|
||||
commit — a field without a leaf saves silently and comes back missing, which
|
||||
is the one bug persistence must not be able to have. Round-trip test.
|
||||
5. **Re-suggest preserves hand decisions.** Propose at one tolerance, pin a frame,
|
||||
|
|
|
|||
|
|
@ -84,8 +84,8 @@ candidate poses so stage cuts can still select any of them.
|
|||
| --- | --- | --- |
|
||||
| Source frames and timestamps | Footage/analysis | Constant-rate frame indexing exists; variable timestamps remain future work |
|
||||
| Head anchor map | `:head` node | Implemented, stored with the node |
|
||||
| Trace frames (photo address) and origin | Face symbol's `:head` `:trace`, written through `:anchors` | Implemented, see `domain/trace` |
|
||||
| Showing the tracing photo, and its opacity | Face instance's `:underlay` | Implemented; a drawing aid, not keyed |
|
||||
| Trace frames (photo address) and origin | The face's `:plate` `:time :holds`, and its `:head` `:reads` | Implemented, see `docs/tracing-symbol-plan.md` |
|
||||
| Showing a tracing layer, and its opacity | Editor state, `[:ui :tracing]` | Implemented; a drawing aid, never saved or keyed |
|
||||
| Generated picture-rate proposal and closure protection | Roto clip/symbol | Generated-only picture sampling exists; closure protection remains future work |
|
||||
| Stage pose cuts | Symbol instance | Implemented, stored with the instance |
|
||||
|
||||
|
|
|
|||
411
docs/tracing-symbol-plan.md
Normal file
411
docs/tracing-symbol-plan.md
Normal file
|
|
@ -0,0 +1,411 @@
|
|||
# Plan: tracing is a symbol
|
||||
|
||||
Status: built, 2026-10-03, on branch `worktree-tracing-layers`. Where the build
|
||||
differs from the plan below, the build wins:
|
||||
|
||||
- The head's field is `:reads` (`{:holds [...]}` or `{:holds-of :plate}`), not
|
||||
`:follow`.
|
||||
- A still is not one frame. It gets as many frames as remain in the symbol it is
|
||||
dropped into, at that symbol's rate, and is trimmed like any clip.
|
||||
- An image's `:media` is `{:image <blob sha256>}`, served at `/blob/<sha>`. The
|
||||
server's `Image` row exists for the pool's list and labels, not for identity.
|
||||
- Dropping media that a tracing symbol in the document already shows reuses that
|
||||
symbol.
|
||||
- Nothing can be created or dropped inside a tracing symbol (`creation/target`,
|
||||
`drop-destination-at`, `nest/move-refusal`), and one cannot be opened in a tab.
|
||||
- Schema 6. Every project is marked 6. One that still carries `:trace` on a
|
||||
node is refused when opened, with what to do about it, rather than all old
|
||||
projects being refused.
|
||||
|
||||
No backward compatibility (see `lane-model.md`, "Goal and compatibility policy").
|
||||
|
||||
## What is wrong today
|
||||
|
||||
Tracing is not a thing in the document. It is three mechanisms that each know
|
||||
about faces:
|
||||
|
||||
- **`:trace {:frames :origin}` on a face's `:head`.** It decides which measured
|
||||
frame the head reads (`symbol/base-channel-frame` → `trace/held-frame`), and
|
||||
the underlay also reads it to decide which photo to show (`trace/photo-frame`).
|
||||
- **`ui/underlay`**, a painter that walks `trace/shown` → `trace/faces` (a
|
||||
separate instance walk), looks up each face's subject → analysis → footage,
|
||||
asks the resolver where `[...path :head]` went, and builds the photo's matrix
|
||||
by hand: `world(head) · M(p)⁻¹ · 1/imageH` (`trace/photo-matrix`).
|
||||
- **`[:ui :trace {:faces #{} :opacity}]`**, a per-face switch in editor state,
|
||||
plus the special "opening a face shows its footage, opening a take does not"
|
||||
rule (`trace/showing-for`).
|
||||
|
||||
What you cannot do: place footage or a still as a reference where you like, move
|
||||
it, scale it, turn it, trim it, hold it, put it in a lane, or trace something
|
||||
that is not a tracked face. The photo is not selectable and has no row. The one
|
||||
case that works (a face over its own footage) runs on code that nothing else
|
||||
uses.
|
||||
|
||||
## The model in one paragraph
|
||||
|
||||
A **tracing symbol** is a symbol with `:type :trace`. It has no nodes. It names
|
||||
media (a footage range or a still image) and has that media's frame count, fps and
|
||||
pixel size. It is **placed by an ordinary instance**, so lanes, spans, trim,
|
||||
split, move, playback (`:in :speed :end`), transform, nesting, selection, picking
|
||||
and gestures all work on it with no new code. It is evaluated to a single
|
||||
**`:trace` op**, which never reaches the raster or an export. A face's footage is
|
||||
not a special case. It is one of these instances, placed inside the face as a
|
||||
child of `:head` and carrying the measured registration transform. The trace keys
|
||||
become **holds on that instance's time**. The head's "origin" becomes the head
|
||||
saying **which node's frames it follows**.
|
||||
|
||||
This follows the precedent of `:type :palette` symbols (no nodes, they name an
|
||||
asset, and they are placed by instances in a lane), so the uniformity rule holds
|
||||
(see the `model-uniformity` memory). Nothing new holds nodes, and no special
|
||||
instance kind is added.
|
||||
|
||||
## Data model
|
||||
|
||||
### The tracing symbol
|
||||
|
||||
```clojure
|
||||
:footage-8625 ; a symbol id like any other
|
||||
{:id :footage-8625 :name "8625.mov"
|
||||
:type :trace
|
||||
:media {:footage #uuid "f8ca…" :range [12 241]} ; or {:image "sha256:…"}
|
||||
:frames 229 ; the range's length; 1 for a still
|
||||
:fps 30 ; the footage's own rate, so cadence handles 30→12 for free
|
||||
:width 1440 :height 1920 ; pixel size = the symbol's own stage
|
||||
:audio {:footage #uuid "f8ca…"} ; optional, the existing "a symbol says what it sounds like"
|
||||
:nodes {}}
|
||||
```
|
||||
|
||||
- `:media` is the one new symbol key. Add it to `symbol/symbol-keys` and to the
|
||||
symbol leaf's `select-keys` in `leaf/leaves`.
|
||||
- The symbol's local space is **pixels of the media**: `[0 w) × [0 h)`.
|
||||
`clip/center` of a node-less symbol already returns its stage middle, which here
|
||||
is `[w/2 h/2]`. So `place-symbol` puts the anchor at the image centre with no
|
||||
new code.
|
||||
- `symbol/problems`: `:type :trace` needs `:media` with exactly one of
|
||||
`:footage`/`:image`, needs `(empty? nodes)`, and for footage needs
|
||||
`(= frames (- end start))`.
|
||||
- **One tracing symbol per footage range, shared.** Two faces from one take
|
||||
place the same symbol, and so does a hand-placed reference. Like any symbol it
|
||||
is reused by reference, and `bring/symbols` copies it like any other.
|
||||
|
||||
### Placing it
|
||||
|
||||
It is an ordinary `:kind :instance` with `:source {:symbol :footage-8625}`:
|
||||
|
||||
- **Start and end** are `:span` (in its own frames) and `:time :at`. You get
|
||||
these from the existing trim, split, move and roll.
|
||||
- **Which frame shows** comes from `:playback {:in :speed :end}`. Footage plays
|
||||
with speed 1, a frozen frame has speed 0, and a still is 1 frame with `:end :hold`.
|
||||
- **Placement in space** is `[:xform …]`, the same as every node: gestures,
|
||||
inspector and keys.
|
||||
- **In a lane, or on its own**: it is a parent-less spanned node, so a `:display
|
||||
:lane` symbol draws it as a block like any cel. It can also sit as a free node
|
||||
or under a group. To have a reference with its own tab, wrap it in an ordinary
|
||||
symbol.
|
||||
|
||||
**Default on creation** (in the drop event, not in `place-symbol`): scale so the
|
||||
image's height fits the stage height, centred on the drop point. This is a
|
||||
creation default like "use center anchor when dropping", and nothing updates it
|
||||
afterwards.
|
||||
|
||||
### Holds: the one new time feature
|
||||
|
||||
`:time {:holds [0 12 30]}` floors a node's local frame to the last hold at or
|
||||
before it. Before the first hold, the first hold applies. This is `:expose`
|
||||
generalized from a regular grid to authored frames. It is applied in
|
||||
`node/local-frame` at the same point as `:expose`, and is inherited in the same
|
||||
way. It is in the node's **own** frames. On a tracing instance with
|
||||
`:in 0 :speed 1`, own frames are source frames, so the hold list is the set of
|
||||
traced frames.
|
||||
|
||||
It is general on purpose. Holding a playing symbol on chosen drawings is the
|
||||
same feature. It costs about three lines in `local-frame`, one `problems` clause
|
||||
(sorted, distinct, finite), and the timeline drawing hold frames as marks on the
|
||||
row.
|
||||
|
||||
`symbol/frame-map` refuses floors (`:expose > 1`). It must also refuse a
|
||||
non-empty `:holds`, for the same reason.
|
||||
|
||||
## The face: "a child symbol that represents the trace"
|
||||
|
||||
The face symbol after a freeze:
|
||||
|
||||
```
|
||||
:place group authored source→stage mapping (image heights → stage px)
|
||||
:head group measured M(p) :reads — see below
|
||||
:mouth … parts generated, every frame
|
||||
:plate instance of :footage-8625 ← the trace
|
||||
measured channels: fit(p) = M(p)⁻¹ · S(1/imageH)
|
||||
:time {:holds [0 12 30]} ← the trace keys
|
||||
```
|
||||
|
||||
### Registration comes from the parent
|
||||
|
||||
The plate is a child of `:head`. Its own measured channels are the
|
||||
**stabilizing fit** at frame p: the inverse of the head's measured transform,
|
||||
with pixels → image heights folded into the scale (it stays a similarity, so it
|
||||
decomposes into `pos`/`rot`/`scale`). Freeze already computes the fit; the
|
||||
head's measured channels are its inverse (`freeze/invert`). The plate gets a
|
||||
second dense block, which is three small channels.
|
||||
|
||||
Because holds are a **time** floor, the plate's channels and the frame its
|
||||
content shows are read at the **same** held frame q. Its world transform is:
|
||||
|
||||
```
|
||||
world(plate) = place · M(p_head) · M(q)⁻¹ · S(1/H)
|
||||
```
|
||||
|
||||
That is exactly `trace/photo-matrix`, but now it falls out of the ordinary walk.
|
||||
It is registered to whatever the head is doing, by construction:
|
||||
|
||||
| head reads (p_head) | plate shows (q) | result |
|
||||
| --- | --- | --- |
|
||||
| f (continuous) | f (no holds) | footage where filmed |
|
||||
| f (continuous) | held key | held photo rides the moving head (today's behaviour) |
|
||||
| held key (same as plate) | held key | `M(q)·M(q)⁻¹ = I`: photo sits where filmed |
|
||||
| 0 (start) | f or held | stabilized footage under a still head |
|
||||
|
||||
If a frame has no measurement, the plate's dense channel has nothing there, so
|
||||
`xform-at` gives nil, so the plate is not placed and no photo shows. That is what
|
||||
happens today too.
|
||||
|
||||
### Trace keys versus origin: who owns what
|
||||
|
||||
The two decisions are separate and stay separate:
|
||||
|
||||
- **Trace keys** are which footage frames get drawn over (the "plate drawings"
|
||||
in `frame-selection.md`). They belong to the **plate**, as `:time :holds`. A
|
||||
hand-placed tracing layer with holds is the *same thing*: the face's plate is
|
||||
an ordinary tracing placement and nothing more.
|
||||
- **Origin** is how the head moves between kept frames. `frame-selection.md`
|
||||
already says `:origin` "is not a tracing setting" but a performance one, so it
|
||||
belongs to the **head**:
|
||||
|
||||
```clojure
|
||||
:head {…} ; continuous — reads its own frame
|
||||
:head {… :reads {:holds [0]}} ; start
|
||||
:head {… :reads {:holds-of :plate}} ; at keys — reads its measured channels at
|
||||
; the frames :plate's holds select
|
||||
```
|
||||
|
||||
`{:holds [...]}` is also what a face with no footage uses for "at keys", since it
|
||||
has no plate to follow.
|
||||
|
||||
`:reads` changes only the **head's own channel reads**, not its children's
|
||||
frames. The parts must keep running every frame, which is why this cannot be a
|
||||
`:time` hold on the head. It is today's `traces` branch of
|
||||
`base-channel-frame` with the hold list read from the named node. A node
|
||||
reference has precedent (`:stencil`, `:pose-group`). It points from the follower
|
||||
to the thing followed, so there is still one stored list of frames and nothing to
|
||||
keep in sync. Validate it in `symbol/problems` the way `:stencil` is validated:
|
||||
the target exists in the symbol, and it is not the head itself or an ancestor of
|
||||
the head.
|
||||
|
||||
Rejected alternatives, so they are not re-proposed:
|
||||
|
||||
- **Keys on the head, and the plate reads them.** This is today's direction. It
|
||||
makes the plate special: a free tracing layer could not have keys that a face's
|
||||
plate also understands.
|
||||
- **Hold `:time` on `:head`.** Exposure inherits strictly, so the mouth and eyes
|
||||
would freeze along with the head.
|
||||
- **A wrapper "registered footage" symbol holding the fit.** It is correct but
|
||||
adds a symbol per face. The time-floor holds already put the fit read and the
|
||||
content read on the same frame, so the wrapper buys nothing.
|
||||
- **Plate as a sibling of `:head` at identity.** It is only registered when the
|
||||
head and the photo read the same frame, so it breaks the continuous-plus-holds
|
||||
and start rows above.
|
||||
|
||||
## Evaluation: an op that is never rendered
|
||||
|
||||
- **`clip/resolver`**, in the instance branch: when the source symbol has
|
||||
`:type :trace`, it does not recurse. If `(:tracing? opts)` is set, it emits one op:
|
||||
|
||||
```clojure
|
||||
{:kind :trace :node [id] :m <copy of world> :media … :frame shown-frame :size [w h]}
|
||||
```
|
||||
|
||||
The frame comes from the same `placed-frame` path as any instance, so playback,
|
||||
holds and the fps cadence apply. Do not build a child resolver for a trace
|
||||
symbol.
|
||||
- **`transform-op`** gets a `:trace` case that composes the matrix. Row paths,
|
||||
solo filtering and nesting at any depth then work for free.
|
||||
- **The output guarantee is structural.** `:tracing?` defaults to false. Only the
|
||||
stage's `::render/resolver` passes true. Export, `clip/center` (a big photo
|
||||
must not pull a symbol's pivot), thumbnails and the bench never ask for trace
|
||||
ops. `raster/draw-ops!` keeps throwing on unknown kinds, so a leak fails loudly.
|
||||
- **`ui/player`** sends picture ops to the raster and `:trace` ops to the
|
||||
painter.
|
||||
- **`ui/underlay` becomes `ui/tracing`.** It paints `:trace` ops in draw order
|
||||
with `drawImage` at `op.m` and the global opacity. The media URL is the footage
|
||||
manifest's `urls[range-start + frame]`, or the image blob URL. The three
|
||||
steadiness fixes stay: LRU cache, hold the last still per op `:node`, and read
|
||||
ahead while playing. Everything that walked faces is deleted.
|
||||
- **`pick`**: a `:trace` op is hit when the point, mapped through `m⁻¹`, falls in
|
||||
`[0 w) × [0 h)`. Picture ops are tested first and trace ops only if nothing
|
||||
drawn is under the pointer, so a full-frame photo does not steal every click.
|
||||
When the global switch is off, traces are neither painted nor picked.
|
||||
- **Gestures**: no change. The face's plate is measured, so `gesture/refusal`
|
||||
already says "place the instance it is in". A hand-placed layer is authored and
|
||||
moves, turns and scales like anything else.
|
||||
|
||||
## On and off
|
||||
|
||||
All of it is EDITOR STATE, as ed88c5e decided for the per-face switch: showing
|
||||
a reference is a way of looking at the stage, so it is not an undo step, does not
|
||||
travel to collaborators, and cannot reach an export.
|
||||
|
||||
```clojure
|
||||
[:ui :tracing {:on? true :opacity 0.5 :hidden #{[sid node-id] …}}]
|
||||
```
|
||||
|
||||
- **One layer** is an entry in `:hidden`, keyed by the symbol the tracing
|
||||
instance is in and its node id. The symbol id is needed because every face's
|
||||
plate is called `:plate`. Keyed this way, hiding a face's plate hides it in
|
||||
every placement of that face, which is what the per-face switch did. Shown is
|
||||
the default, so opening a face shows its footage with no setup.
|
||||
- **How it's applied**: `clip/resolver` already knows which symbol and node a
|
||||
trace op comes from, so the op carries `:layer [sid id]`. The painter and
|
||||
`pick` skip ops whose layer is hidden. The resolver does not change when you
|
||||
toggle a layer, so nothing is rebuilt.
|
||||
- **Global**: `:on?` and `:opacity`, a toggle plus an opacity slider in
|
||||
`ui/palette/bar` next to the palette controls. When it is off, traces are
|
||||
neither painted nor picked.
|
||||
- **Switching one layer on also switches the global setting on.** This applies
|
||||
from the tracing instance's inspector, from its timeline row, and from the
|
||||
face/roto section. A layer you just enabled must not stay invisible behind a
|
||||
switch you forgot. Turning one layer off never touches the global switch. It is
|
||||
one event (`::ui/show-trace layer on?`) that updates `:hidden` and, when
|
||||
turning on, sets `:on? true` in the same handler, so the three places cannot
|
||||
behave differently.
|
||||
- **`[:vis]`** still works on a tracing instance like on any node: key it to
|
||||
show a reference only over part of the shot. It is not the on/off switch.
|
||||
- **Deleted**: `[:ui :trace :faces]`, `::trace-face`, `::trace-faces`,
|
||||
`trace/showing-for`, `traceable-faces`, `shown`, `faces`, and the "a take shows
|
||||
nothing by default" rule.
|
||||
|
||||
## UI
|
||||
|
||||
- **Timeline.** A tracing instance is an ordinary row or clip block, styled to
|
||||
read as reference-only (hatched block, an eye icon in place of the colour
|
||||
chip). Hold frames are marks on the row. The row's eye button dispatches
|
||||
`::ui/show-trace`, which replaces the face row's `tl-trace` button.
|
||||
- **Inspector, tracing instance.** Show the media (footage name and range, or
|
||||
image), an on/off control (which goes through `::ui/show-trace`), and the hold
|
||||
list ("hold here" / "remove hold" plus seek buttons, which is `trace-keys`
|
||||
re-aimed at `:time :holds`). Playback, transform and span use the existing
|
||||
sections.
|
||||
- **Inspector, face/roto section.** The same hold controls, aimed at the face's
|
||||
`:plate`. Origin buttons write `:head :reads`. The section is found by "this
|
||||
face has a `:plate`", not by `trace/traceable?`.
|
||||
- **Palette bar.** The global toggle and opacity slider.
|
||||
- **Making one.**
|
||||
- Footage: the convert dialog gets a choice between "animate faces" (today's
|
||||
flow) and "tracing layer" (no detection). The tracing-layer choice makes the
|
||||
`:type :trace` symbol for the chosen range and places it where the video was
|
||||
dropped.
|
||||
- Footage can also be dragged from the pool with a modifier or as a second
|
||||
drag kind.
|
||||
- A still image: drop it on the pool and it uploads; drop it on the stage or
|
||||
timeline and it is placed with `:end :hold` and a span of the host's
|
||||
remaining frames.
|
||||
|
||||
## Freeze, bring, regenerate
|
||||
|
||||
- **`freeze/subject-part`** emits the `:plate` instance under `:head`, with fit
|
||||
measured channels and `:time {:holds []}`, and emits one `:type :trace` symbol
|
||||
for the analysed footage range. `bring/take` already copies every symbol the
|
||||
take reaches and rewrites `:source :symbol` references, so the tracing symbol
|
||||
comes along. It gets the footage's `:audio` too, so a dropped tracing layer
|
||||
can bring its sound through the existing link.
|
||||
- **`head-mode`** writes `:reads` on the head and `:holds` on the plate, in
|
||||
place of `:trace`. The `frame-selection.md` plate proposal materializes into the
|
||||
plate's `:time :holds` in place of `:trace :frames`.
|
||||
- **Regenerate** replaces both measured blocks (head and plate) and keeps
|
||||
`:holds` and `:reads`. Check that `regenerate-head`'s measured/authored
|
||||
comparison handles a second measured node.
|
||||
|
||||
## Server
|
||||
|
||||
- `Image` endpoints mirroring `Sound`: `POST /api/images` stores a `Blob` and
|
||||
returns `{id, width, height, url}`, and `GET /api/images` lists them. The
|
||||
migration is a model only. Bump `schema_version` and refuse older documents
|
||||
clearly. Do not convert them.
|
||||
|
||||
## Deleted outright
|
||||
|
||||
`trace/of`, `prepare`, `held-frame`, `problems`, `toggle-frame`, `photo-frame`,
|
||||
`measured-local`, `photo-matrix`, `traceable?`, `faces`, `traceable-faces`,
|
||||
`showing-for`, `shown`, `opacity-default`. `domain/trace.cljs` probably goes
|
||||
away entirely; if anything is left, it is a hold helper that belongs in `node`.
|
||||
|
||||
Also deleted: `symbol/prepared-traces` and the `traces` arm of
|
||||
`base-channel-frame`, which becomes the `:reads` lookup. `:trace` on nodes (and
|
||||
`symbol/problems` reports it as removed). `::project/set-trace`.
|
||||
`::render/underlay` and `::render/tracing` become one `::render/tracing` that
|
||||
returns `[:ui :tracing]`. The face lookup in `params/view`.
|
||||
|
||||
Expected effect on code size: net negative. One time-floor clause, one op kind,
|
||||
one resolver branch, one pick case and a `:reads` lookup replace the face walk,
|
||||
the hand-built photo matrix, per-face showing state and the underlay's face
|
||||
bookkeeping. Measure the change and report the number honestly (see the
|
||||
`cljs-style` memory).
|
||||
|
||||
## Tests
|
||||
|
||||
**Domain (node).**
|
||||
|
||||
- `:holds` in `local-frame`, with eval-frame and resolver agreeing forward,
|
||||
backward and in random order.
|
||||
- A trace op appears only with `:tracing?`, and `export/run!` output contains no
|
||||
`:trace` op.
|
||||
- `transform-op` composes the matrix through two nesting levels.
|
||||
- `clip/center` ignores traces, and is `[w/2 h/2]` for a tracing symbol.
|
||||
- `pick` returns picture ops before traces and inverse-maps through a rotated
|
||||
layer.
|
||||
- `symbol/problems` covers `:type :trace`, `:reads` targets and cycles, and
|
||||
`:holds` shape.
|
||||
- Registration: port `trace_test`'s photo-matrix assertions onto the walk.
|
||||
For every row of the registration table, the plate's world transform equals
|
||||
the expected matrix. At a held key with `:reads {:holds-of :plate}`, it equals
|
||||
`world(:place) · S(1/H)`.
|
||||
- Freeze emits the plate and the tracing symbol. Regeneration keeps the holds.
|
||||
- The `::ui/show-trace` event sets the global switch on when turning a layer on
|
||||
and leaves it alone when turning one off.
|
||||
|
||||
**Browser** (CDP, see the `arthur-verify-dont-guess` memory):
|
||||
|
||||
- Drop a still, then drag, turn and scale it.
|
||||
- Toggle a layer on with the global switch off, and confirm both are on and the
|
||||
photo paints.
|
||||
- Export a frame and check it has no photo pixels.
|
||||
- Open a face: the plate is registered at a hold with origin "at keys", and
|
||||
stabilized with origin "start".
|
||||
|
||||
## Order of work, each step green
|
||||
|
||||
1. `:time :holds` in `node/local-frame`, `problems`, `frame-map`, and timeline
|
||||
marks.
|
||||
2. `:type :trace` symbol, `:media`, the `:trace` op behind `:tracing?`,
|
||||
`transform-op`, the player split, `ui/tracing` painter and `pick`. Footage
|
||||
media only, placed by hand from a REPL or test document.
|
||||
3. The face: freeze and bring emit the plate and tracing symbol, `:reads`
|
||||
replaces `:trace`, delete the old trace and underlay paths, and re-aim the
|
||||
inspector's face section.
|
||||
4. On/off: the `:hidden` set and `:layer` on trace ops, the row eye, the
|
||||
`::ui/show-trace` rule, the global toggle and opacity in the palette bar, and
|
||||
delete `[:ui :trace :faces]`.
|
||||
5. Creation: "tracing layer" in the convert dialog, pool drag, then the image
|
||||
endpoint, pool images and image drops.
|
||||
6. Docs: `animation-model.md` "A photographic underlay is not an op" becomes "a
|
||||
trace is an op that never reaches the raster". Update `frame-selection.md`
|
||||
(where plate keys live) and `lane-model.md` (tracing clips in lanes).
|
||||
|
||||
## Open questions
|
||||
|
||||
1. **"A take shows its faces' footage."** Dropping the old "only in a face's own
|
||||
tab" default means opening a take shows every face's plate if the global
|
||||
switch is on. The recommendation is to accept that, since the switch is one
|
||||
click.
|
||||
2. **Lane-model audio rule.** `lane-problems` requires visual cels. A tracing cel
|
||||
is visual for this purpose, but say so explicitly when a lane may hold both
|
||||
tracing and drawing cels.
|
||||
Loading…
Add table
Add a link
Reference in a new issue