Everything could only be added to the end, because `append` computed its own
position — the max end of the lane — and so had no opinion to state. Insert is
not a new command; it is the argument that function was missing. `:at` takes a
lane frame or `:end`, `:end` is the position where nothing has to move, and
appending stops being a separate operation from inserting. New, reused and
duplicated drawings all take it, because there was only ever one placement rule.
Placing ripples: occurrences at or after the position move later by the new
exposure's duration, and `:keep` against `:grow-symbol` still decides what
happens at the shot's end. OVERWRITE is deliberately not a policy argument yet.
Taking frames away from the occurrence already there is TRIMMING, and an
argument whose second value is unimplemented is worse than an argument that is
not there. A position strictly inside an existing exposure refuses and names
`split`, rather than splitting on the quiet: one command performing two is how
a command stops being predictable.
Then split, which turned out to cost almost nothing, and that is the
interesting part. The two pieces keep ONE `:time` and differ only in `:span`.
The right piece's own frames therefore carry on exactly where the left's
stopped, so its source clock, its keys and its corrections go on meaning what
they meant: a held drawing holds the same frame either side of the cut, and a
playing insert plays through it without a seam. There is no arithmetic on
in-points to get wrong, and no shot-length question, since the pieces occupy
the frames the one exposure occupied. The test samples every frame before and
after and asserts the picture is identical — for a hold, for an exposure with a
correction of its own, and for a playing insert.
That is not a clever split. It is `:span` being in the node's OWN coordinates,
which was decided long before there were lanes, paying for something it was not
designed for. The same property is why extending a hold leaves lane keys alone.
Both new commands act at the playhead, which needed `lane-frame` — the symbol's
frame as a frame of the lane's own time, nil through a stepped or looping lane
where one is not the other. Nil refuses; it does not snap to a nearby frame.
Two smaller things found while doing it. `placeable` promised "a whole lane
frame" in its refusal and then accepted 2.5, so both it and `split` now require
an integer, as `extend-hold` already did for its delta. And `lane-end` is
private: `:end` is the only way to ask for it.
401 tests, 5,612 assertions. The browser flow now splits an exposure at the
playhead and puts a drawing in the gap, and checks that six exposures are still
one row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The model's whole claim is that content and its occurrences are different
things, and until now nothing in the editor could tell them apart: you could
make a drawing and time it, but not expose one drawing twice, and so never find
out whether an edit arrives in two places. That is the first proof obligation in
the lane model and it was the one the commands could not reach.
Three commands, and the distinctions between them are the point:
reuse another occurrence of the same drawing. A decision to share,
made on purpose, because sharing discovered later — when an edit
turns up somewhere you did not expect — is the bad version.
duplicate a copy of the drawing, appended, for when what is on screen is
the starting point for the next one.
make unique this occurrence gets a private copy; the others keep sharing.
The undo of reuse, and refused when nothing else uses the
drawing: a copy nobody asked for is a second identical symbol in
the library for no reason a person could see.
Duplicate copies the CONTENT and not the exposure. Its new occurrence is a plain
one-frame hold, not a copy of the source occurrence's transform or corrections,
because those belong to that use of the drawing — carrying them over would make
duplicating a drawing quietly duplicate the treatment of one exposure of it.
A copy is SHALLOW by default and keeps its references to other symbols, so a
head built out of reusable eyes still uses those eyes. `:deep? true` copies
everything it places with new ids throughout. The lane model asks for both and
says why: never promise decoupling while leaving the edited object shared, and
only the deep copy can keep that promise. `bring/symbols` already did the
reachability walk and the id remapping, so the deep copy is that function
pointed at its own clip.
`node/sources` was still being read as a SET at five call sites, each with a
comment about a lane that cuts between several drawings — the keyed source that
no longer exists. An occurrence names one symbol, so they now ask `node/source`,
and `placed-frame` answers with `:symbol` rather than `:of`, which was the last
echo of the retired field name.
To let the commands use `clip/free-id` and the copy machinery, the lane's own
validation moved from `domain/sequence` to `domain/symbol`, which is where it
belonged anyway: a sequence is the one composition rule a node map carries, and
it now sits beside the parent and stencil checks rather than in the namespace
that happens to build lanes. That also breaks the cycle — sequence can require
clip and bring, and nothing below it requires sequence. Preconditions still
check only the LANE's shape: refusing an exposure edit over an unrelated defect
elsewhere in the symbol would be this command answering for a part of the
document it never touches.
The cel strip gains reuse, duplicate and make unique, the last shown only where
the selected exposure actually shares its drawing. Drawing on twos is also now
under test: exposure length is the cadence, the lane's transform has its own
clock, and it still moves on every frame — stepping it would be the cel cadence
leaking into continuous motion.
397 tests, 5,561 assertions. `test/browser/sequence.mjs` drives the three new
commands through the real editor and checks that three exposures are still one
row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A lane's drawings were going to be one instance whose source was a KEYED
channel: frame 0 says `:drawing-a`, frame 4 says `:drawing-b`, and the cels of
a row are that channel's keys. Two things followed from it, and both were
wrong.
The first is that playback meant whichever shape the channel happened to have.
A framed source played its symbol; a keyed source froze the selected frame.
So `node/placed-at` read animation out of storage, and adding an ordinary key
to a still turned it into an animation — the last-key bug, which was not a bug
in the code so much as the rule working as written. But WHICH drawing is used
and HOW time runs inside it are independent questions, and all four combinations
are ordinary: hold one drawing, play one animation, cut between held drawings,
cut between playing ones.
So an occurrence names one symbol in `:source {:symbol ...}` and says how its
source time advances in `:playback {:in :speed :end}` — `source = in + speed *
f`, a hold being speed 0, with `:stop`, `:hold` or `:loop` at the end named
rather than guessed. `node/placed-frame` samples it forwards, which works for
holds too, and `node/source-time` is the separate, invertible edit map, nil
where inversion is meaningless. The two were one function before, and a hold
had to lie about one of them.
The second is that a keyed source only looked necessary because an occurrence
was assumed to need a ROW. It does not. A lane is a group with `:layout
:sequence`, its occurrences are ordinary instances in the same flat node map,
and `timeline/rows` draws them as cel blocks on the lane's own row: twelve
exposures, one row, each cel still separately selectable and addressable. The
vertical growth that justified the keyed source is a presentation question, and
it is answered in the view.
`arthur.domain.sequence` holds the first commands over that shape — add lane,
append drawing, extend hold — each one history step, each refusing rather than
half-applying. Extending a hold leaves the lane's keys at their authored times,
because you are adjusting drawings underneath timed motion; a correction owned
by an occurrence travels with it. Ownership does that work, so no key needs a
flag saying what it follows. Ripple past the symbol's end is refused with the
frame count it would need, and `:extent :grow-symbol` is the caller saying yes.
`clip/blank` no longer carries `:subjects {} :features {} :groups {}`. Empty
maps write no leaf, so a blank document could not survive its own round trip —
`leaf/leaves` promises exactness and was the only honest side of that.
Documents are schema 3. A version 2 document is not read; nothing here converts
one. `docs/lane-model.md` is the design, and says which of its parts are built.
392 tests, 5,525 assertions, and `test/browser/sequence.mjs` drives the editor
through create, hold, explicit overflow and undo.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Finishing the last commit, which traded a simpler definition for noisier call
sites: deleting the arity ladders left `(symbol/resolver sym st pal/index-of
nil nil)` at twenty-odd places, and two trailing nils tell a reader nothing
except to go and count positions.
The ladder was a symptom. The disease is five positional parameters, and the
split that matters is which of them are OPTIONAL:
store, palette positional, because neither is optional. A dense channel
cannot be read without the store it names — that is the
crash two commits ago — and every op carries a colour.
pose-tracks one call site, in `clip/resolver`'s own recursion
source/picture-fps two call sites
So the last three become one `opts` map, and the common call loses a nil. The
point is not the nil: it is that the sixth option, whenever it arrives, is a
key at one call site rather than a nil at fifty.
`clip/resolver` also had `sid` FOURTH, behind two arguments that say nothing
about which symbol is being resolved. It is second now, beside the clip it is
in: `(clip/resolver c :main store pal/index-of nil)`.
62 call sites rewritten by parsing the forms rather than by regex, because
`clip/resolver`'s arguments move past each other and a regex cannot see that.
An earlier attempt at this dropped `palette` on the floor and still compiled
at 62 sites — it only surfaced as an arity error, so if that had been a
same-arity mistake the tests would have been the last line of defence.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pre-alpha. Nothing here is owed a call shape it used to have.
Twelve convenience arities deleted, and the only reason to single any of
them out is that one of them was a live bug: `channel/value-at`'s `([ch f])`
filled in a nil tier-2 store, so a caller could omit it, read correctly for
every channel that happened not to be dense, and throw the first time one
was. That is the iris crash, and threading the store through `gesture/values`
last commit fixed the symptom while leaving the trapdoor open. Deleting the
arity found `node/toggle-key` standing on it too — the inspector's stopwatch
on a measured channel, the same throw, never reported.
Gone, and what the compiler then made explicit at each site:
channel/value-at, cursor, dense-at the store, and `nil` where a caller
genuinely has none and means it
channel/keyed `:hold`, which is a cut rather than a
tween and not a thing to leave implied
symbol/resolver (4), eval-frame (3) store, palette, pose-tracks, opts
clip/resolver opts
mix/buffer!, store/install! dead: no caller used the short form
`pick/local-bounds` goes the same way — it was `bounds-of` with the closure
thrown away, so callers build the closure and call it.
Every site was found by shadow-cljs `:fn-arity` rather than by grep, which is
the argument for the change: 90-odd call sites, and the compiler listed all of
them. BUILD BOTH TARGETS — the last three only appear in `:app`, since `:test`
compiles what the tests reach and the inspector, the pool drag and the vertex
overlay are not that.
Left alone, because an argument with a default is not the same thing as a
shim: genuine optionality like `fx/http`'s body, `geom`'s iteration count,
`zip`'s injected clock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three faults, one gesture. None of them was in `gesture/scale`, whose
`s' = s·b/a` about the pivot was right all along.
THE PIVOT. `clip/place-symbol` writes an instance's anchor, `paint/new-shape`
a drawing's, `nest/group` a new symbol's and `face-placement` the source
placement's — and `flow/freeze` wrote none for the parts underneath, so a
traced mouth turned and scaled about its own coordinate ORIGIN, which for
head-local geometry is the top-left corner of the footage. On a 320x200 stage
that put the mouth's pivot at (-234, -395), so dragging a corner outward slid
the shape about and shrank it. `pick/pivot` is `clip/center`'s rule for a node
rather than a symbol; `freeze/pivoted` applies it to every node the freeze
makes, skipping `node/measured?` — the predicate `gesture/refusal` already
refuses a hand edit by, so a pivot is written exactly where a hand can use
one. That also keeps it off `:head`, whose scale is not 1 and where an anchor
would NOT cancel out of `local!`; skipping it for drawing nothing would have
been true only by accident. Asserted in pixels: the pass moves nothing.
THE JUMP. `ui/stage`'s overlay dereferenced the document while RENDERING and
used it when the pointer went down. The store is a mutable handle behind an id
that does not change when the document does — `:paint/revision` says that, and
the overlay subscribes to neither it nor `::render/clip` — so after any edit
the next drag began from the transform the node had before the last one: still
under the press, jumping on the first pointermove. `ctx` now carries the id and
`loaded` reads at pointer-down.
THE CRASH. `gesture/values` read its channels without the tier-2 store, which
is fine until a selection lands on a dense transform — an iris follows the
gaze, a brow the raise, a head the similarity — and then `dense-at` throws and
takes the stage down, in `begin!` and again in `handles`. It takes the store
now. Kept in this commit because it is the same two functions.
`pick/bounds-of` yields a closure, as `clip/resolver` does, so an instance's
resolver is built once for a walk instead of once per frame — which is what
let `pivot` be the one walk it is rather than a separate path for instances.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The photo blinked out for a frame or two and sometimes never arrived, from
three causes that each present as the same bug. The still cache emptied
itself on the frame it filled, so every still on screen had to be fetched
and decoded again — every 48 frames of a scrub, and with two traced faces a
permanent flicker, each one's still evicting the other's; it drops the least
recently used now, a still being touched on every frame it is drawn. A still
cannot decode in the animation frame that asks for it, and a face whose next
one had not arrived drew nothing, so it now keeps the still it was showing
until the new one is there. And nothing was read ahead, so continuous
playback was always a frame behind its own footage; the next few frames'
stills are asked for while the playhead is moving on its own, and only then,
because a scrub asks for a different few at every step.
Whether the footage shows is no longer the document's. It was an :underlay
on an instance, inherited down the row path, nearest wins, and it went
through edit — so showing a reference photo was an undo step that travelled
to collaborators. It is [:ui :trace] now, the faces switched on and one
opacity, like solo, and there is nothing left to inherit: a face is the same
face wherever it is placed, so one switch covers every placement of it. The
paint loop asks for its own redraw when that changes, since the resolver no
longer does it for them.
Opening a face shows its footage, because a symbol has measured footage
behind it only because it was traced from that; a take does not, because a
take is the picture. The switch is on the bar above the stage, with the tone
and the tool, and on each face's timeline row beside solo — not a section
that appeared once the right row had been found. A face open in its own tab
could not show its footage at all before, shown having walked instances, and
that is the one place tracing matters most. The inspector keeps the face's
own keyed facts, its trace keys and its origin, and says why it cannot key a
frame rather than greying out the only button in the section.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A face's :head carries :trace {:frames :origin}: the frames its photo holds
on, and whether the head reads every frame, jumps to the trace frames, or
holds frame 0. It replaces :anchors, so which measured frame a head reads is
one stored fact. An instance's :underlay shows the tracing stills over every
face at or below it, registered through each face's own head, at an opacity,
unkeyed. The clip resolver answers where a row path went on its last frame,
so the paint loop reads the photo's matrix instead of resolving again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A click selects the thing in the open symbol, a double-click goes one
level in, ⌘-click goes to the shape itself and Esc comes back out —
Figma's rule — and a click inside the selection keeps it, so a shape
several symbols down can be dragged. The selection is the one a
timeline row makes, so the inspector shows it and its row opens and
scrolls into view.
A drag writes what the inspector writes: a key on the node's own frame
where the channel has keys, its one value where it has none. It is
previewed like a bar being slid and let go as one edit, so one undo
step. Measured transforms refuse. A shape's points are edited by
double-clicking it, and new shapes turn about their middle.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A nested mark is now named by the flat path its row has, [a b mark] rather
than [a [b mark]], so a soloed row is a prefix of what it shows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The boxes wrote nothing until blur, so a spinner click showed nothing on
screen. Now every keystroke and step is an edit, and history holds the
step open from focus to blur, so typing 4 then 5 is seen as 4, then 45,
and undone once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The title is a button that becomes a field, saved on Enter or blur. Export's
target and scale move out of the bar into a dialog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The inspector edits the project's width, height and fps, and a symbol's length
and, optionally, its own width and height. A symbol without them follows the
project's (`clip/stage`); the stage, export and placement all read it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The new symbol gets project-rate frames covering the same wall-clock length,
and its root and sound map them back onto the source's frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delete or Backspace, or the × on the selected row, takes the node out of its
symbol along with every node hanging off it. One edit, so undo brings it back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each of vis, anchor, pos, rot, scale and skew is a row: ◆ keys the channel on
the node's own frame, or takes the key there off, and a typed value is written
on blur or Enter — a key on a keyed channel, the one value on one that is not.
Rotation reads in degrees. Dense and other channels keep their readout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A node's bar drags along the timeline: one write to its :at, for every node
alike, carried down through the instances above it. The stage and rows show
the slide live through ::render/clip, and it lands as one edit on release.
A row dropped on another's top or bottom edge goes in front of it or behind:
one write to :z, between its new neighbours (symbol/z-between). Onto the edge
of a row in another symbol, it moves there first. Neither needs anything on
screen — nest/down walks a row path by structure, without a frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`nest/inside` walked a row path only through instances. Every node has
the same two maps to its parent, so the walk now steps into any node:
inside an instance is the symbol it places, inside a shape is where its
points and keys are. The selected shape at any depth is `inside` over
its full path, which gives the stage editor its handles (through the
matrix, drags back through the inverse) and the inspector the shape's
own frame to key at, and the time map back for jumping to a key.
`inside` resolves only the node's lineage: where a node is depends on
its parents and nothing else, and resolving the whole symbol cost more
than a stage frame (7.9ms against 3ms on the swarm; now 0.09ms).
Checked equal to the whole-symbol resolve on every node and frame of
the swarm, two instances deep.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project is only ever at /p/<id>/<slug>; / is the index of the projects
you own or edit. Every project has an owner, who can name editors;
anyone with the link can view. Every edit saves itself, one request in
flight at a time, as a patch of the leaves that changed, and a websocket
(channels + daphne) carries presence and each committed write to
everyone else in the project. The first write to a leaf wins, and the
loser is told.
Undo is per person: a step undoes only if the leaves it touched still
hold what it left, so it never takes a collaborator's work with it.
Named snapshots replace saving, and restore as an ordinary write.
An empty symbol now survives the leaf round trip with `:nodes {}`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
domain/clip is the document and what only needs the document: its symbol
table, placing, the resolver, problems. domain/nest is how nested symbols
relate — one walk down a row path gives the frame, the matrix and the time
map, which merges inside and time-down — and moving and grouping between
them, and nested sound. domain/bring is copying symbols in from another
clip, a take from footage, and one placed that merges the store and places
the instance, which conversion and import both now call instead of each
doing it in their own words. Tests follow the same split.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dropping a symbol out of the pool's all-assets folder fetches that project's
clip and the blocks it names, copies the symbol and everything it places in
through clip/adopt — renamed where ids collide — and places it where it was
dropped. It comes in as drawing: its tracking stays with the analysis that
measured it. New and open now seek the audio clock to 0 with the readout,
where play used to pick up wherever the last document's audio had got to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A node's row can be dragged onto an instance's row, to go inside the symbol
it places; onto any other node's row, to be grouped with it into a new
symbol; or onto empty label space, to come back to the top of the open
symbol. All three keep the picture and the timing as they are, and a refused
move says why in the status line. Pool drop targets now only take things out
of the pool, and row drags set a move effect so the browser delivers the drop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every node now has the same time map into its parent, local = rate·(parent
− at), with its span and keys in its own frames: what instances had, made the
rule. A shape without a time map reads as it always did, so no data changes.
The per-kind branches, the span-start term and the rate refusal are gone;
node/time-of, then-time and invert-time compose it like the matrix.
clip/move-node puts a node into another symbol without changing the picture
or the timing — its matrix becomes a :pinv, its time a new :at and :rate, and
its channels, keys and span are untouched — and clip/group makes a new symbol
around side-by-side nodes. Generated parts, split stencils, cycles and
looping instances are refused with the reason. Nested sounds use the same
map.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With an instance selected, a finished polygon goes into the symbol it places,
its points and frame carried in through clip/inside — which resolves each
level, so the frame and matrix are the ones the stage draws with — so the
shape lands exactly where it was drawn however the instance is moved, turned,
scaled or retimed. Beside any other selected node, or at the top of the open
symbol with nothing selected. clip/inside also replaces frame-inside for new
symbols, so there is one account of 'which frame is it in there'.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
place-symbol sets a new instance's anchor to clip/center — the middle of the
bounds of everything the symbol draws over all its frames, or the stage's
middle for a symbol that draws nothing — and a stage drop puts that middle
under the pointer. The anchor is set once and never follows the symbol, as
Flash's transformation point and After Effects' anchor point do, so a symbol
that grows later moves nothing on screen. The drop preview marks the pivot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dropping footage on the stage or the timeline asks which frames and what name:
a dialog plays the video with start and end handles that seek it. Detection
runs on that range only (decoding stops at the end, frames before the start
are skipped) and the range is part of the analysis address, so a partial
analysis is never served as a whole one. The frozen take comes in as one
named symbol, carrying its sound as an audio node, placed where it was
dropped; tracking and tuning come with it when the document has no analysis
of its own.
clip/adopt copies symbols between documents, renaming ids that collide, and
clip/audio-tracks carries sounds out of nested instances so a placed symbol
is heard where it is placed.
Fixes the stage going blank after a conversion: ::store recomputed only when
the clip id changed, so blocks merged into the loaded entry were invisible to
the resolver. And the paint loop now schedules its next frame before
painting and reports a frame it cannot draw instead of stopping.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A symbol dragged out of the pool shows where it would land: on the stage, a
dashed outline of its first frame centred on the pointer, and on the timeline
a preview row of its own length. A stage drop places it at the playhead under
the pointer; a timeline drop places it at the frame under the pointer, in its
own coordinates. A drop that would make a cycle is refused while hovering.
Pool thumbnails are capped at 40x30.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This project lists every symbol in the document and the video it uses or was
given this session, each with a frame from its proxy as a thumbnail. All
assets lists every upload on the server and every other saved project's
symbols, from a new /api/symbols that reads symbol leaf paths. Every row is a
drag source (symbol:, import:, footage:). Uploading no longer runs straight
into detection; it lands in the project's media.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Double-clicking a symbol in the pool, or an instance's row in the timeline,
opens it as a tab above the stage; the stage, the rows, the transport and new
shapes follow the open tab. Each tab gets a clock as long as it is: its own
mixed tracks, the document's audio for the symbol it opens on, or silence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shape's :span stays in its parent's frames, but an instance's or a sound's
is now in its own: dropping a symbol at frame 97 gives it span 0 … length and
:at 97, so moving it along its parent is one write to :at. :time :in is gone
(it was the span's start written twice); node/placed-span maps an own-time
span out to the parent for playback, the mixer and the timeline rows, and
node/problems reports a stale :in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
'+ symbol' in the timeline makes an empty symbol and places it at the
playhead: inside the selected instance, beside any other selected node, or in
the open symbol. Timeline selections carry their row path so a symbol placed
twice nests into the row that was clicked. Symbols gain a :name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Everything that holds nodes is a symbol (domain/timeline -> domain/symbol,
:timelines -> :symbols) and a node that places one is :kind :instance. The
reserved :main root is gone: which symbol is on screen is editor state
([:ui :open]), every domain function that needs a symbol is told which, and
a document opens on the longest symbol nothing else places.
Saved projects move to schema 2 through migration 0007, which rewrites leaf
paths, instance kinds and the feature :symbol key; the client refuses a
schema it does not read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"the video never presented frame 1; it offered 2" was this function waiting
for a frame it had not asked for a second time. The walk discarded a wrong
frame and re-armed the callback WITHOUT seeking again, and nothing further is
ever presented to a paused element that has not been asked to move — so one
wrong answer starved until the timeout and reported it as the browser refusing.
Underneath that, the wrong answer was not wrong. A container can carry an edit
list, and `currentTime` then counts from the start of the edited presentation
while a frame's `mediaTime` counts from the start of the media. The two differ
by a constant, so the frame at currentTime 0 can honestly report a mediaTime
two frames in. Seeking cannot correct for it in the positive direction: source
frame 0 would have to be found before the start of the video, every attempt
clamps at zero, and the walk offers frame 2 forever.
So the constant is measured once and subtracted. `calibrate!` takes whatever
the browser calls the first frame it shows and makes that the origin, which is
the honest definition anyway — it is what a viewer sees at time zero, and the
audio clock this take plays against starts in the same place.
Calibration also leaves the element on frame 0, so the walk starts holding it.
`frame!` returns immediately for a frame already on screen rather than seeking
to where it already is, which presents nothing and would hang.
Residual error still re-seeks, corrected by exactly the measured miss, and
still fails loudly with every frame that was offered and where it was asked
from, so a next failure is diagnosable in one shot rather than four.
The proxy also drops B-frames now. That removes the edit list at the source
rather than only coping with it, and makes decode order presentation order
should this ever be fed to WebCodecs. 14% larger, and extraction refuses a
proxy whose timeline is shifted so it cannot come back silently. Honest note:
this was my first diagnosis and it did NOT reproduce the failure — both
proxies walk correctly here on hardware decode — so it is hardening, not the
fix.
Verified by forcing the fault: a harness that offsets the reported timeline by
-2, -1, 0, +1, +2 and +5 frames failed on every positive offset before and
recovers on all six now, first attempt. Real app in headed Chrome with Metal
hardware decode: 280/280. 41 backend and 234 frontend tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Detection now walks a browser-seekable H.264 proxy in MediaPipe's VIDEO
running mode. The PNG sequence it replaces was 112MB for 7.6 seconds at
1440x1920 and 1.1GB at the 900-frame limit; the proxy is 6MB, and landmarks
detected off decoded H.264 rather than off the PNGs moved at most 0.0033 of
frame width.
Three things had to be true for video mode to work, and each was measured
against the same footage decoded to PNGs:
/blob/<digest> answers byte ranges. Django's FileResponse does no Range
handling, and a media element handed 200 with no Accept-Ranges reports an
empty `seekable`, no-ops every currentTime write, and detects frame one
ninety times without raising.
A seek aims at the MIDDLE of its frame. Aiming at i/fps sits on a frame
boundary and landed one frame early 31 times in 91; (i + 0.5)/fps was exact
on all 91.
Timestamps are strictly increasing footage milliseconds. Video mode is a
tracker: a repeat leaves the graph in an error state every later call
re-throws, so the landmarker is discarded on failure, and passing the frame
index instead of i*1000/fps moved landmarks six times further from the
per-frame answer.
Frames are verified rather than trusted. requestVideoFrameCallback states
which frame it handed over, the walker discards any other and fails loudly
if the one it asked for never arrives — a stale presentation from the tail
of a previous seek is what produced "asked for frame 1 and it presented
frame 2" on a video whose seeks were in fact exact.
The proxy is re-encoded even when the upload is already H.264: HEVC is not
decodable everywhere, and footage identity is the proxy's digest. The JPEG
stills beside it are tracing references, outside the footage digest because
re-rendering them at another size is not different footage.
Verified end to end in a real browser against real footage: 228/228 frames
detected, a drawn roto face, 37 backend and 234 frontend tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
