Give features stable identity, eye pairs and per-feature presence
Step 8's data model, ahead of its controls. Nothing here is a UI. domain/params holds every knob's definition once — default, applicable area, value constraints and the areas a change would force to regenerate. flow/take's literal knob map becomes a view of it, so the take's defaults and the future parameter panel cannot drift apart. domain/feature adds subjects, features and groups as document data the renderer never reads. A feature ID is stable for the whole clip, across occlusion: a run of visible frames is not a new identity. An eye pair is an explicit group of one or two eyes of the same subject, so a profile view with one identified eye needs no invented partner. Settings resolve area -> subject -> group -> feature, and dropping an eye from a pair materialises its effective values first so playback does not jump. scene/problems now validates all of it. Presence becomes per-feature rather than per-subject. freeze's :absent predicate takes a track as well as a frame, so one occluded eye can be absent while its partner still has a value; a full-face miss still marks everything absent. A manifest may annotate known gaps as one-based inclusive intervals, which ingest expands into observation tracks before measurement. An unobserved eye then gets no vote in the iris pairing and cannot steer the shared gaze — gaze falls back to whichever eye is visible. Temporal filters still see a sample on every frame, held from the last observed one, because the numbers are a rectangular buffer; the state mask, not the buffer, is what says the frame has no value. js/app.js gets the same occlusion lesson: leading nulls from a face that starts occluded used to throw away the whole take, and the neutral frame could be chosen from a held duplicate pose. Parameter editing, scoped regeneration and a feature-level detector remain. Until one exists, footage without annotations falls back to the full-face mask rather than claiming occlusions it cannot see. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B87NVmiU36qQmN9gmFYnJ9
This commit is contained in:
parent
ccca93e233
commit
35ef150b48
18 changed files with 631 additions and 86 deletions
|
|
@ -7,8 +7,11 @@ Step 6 reads extracted footage from the manifest, detects landmarks with local
|
|||
MediaPipe assets at full source cadence, and runs the same freeze path as the
|
||||
synthetic take. The scene time map can sample the frozen roto at a lower picture
|
||||
fps without changing source analysis, duration or audio. Step 7 adds dense
|
||||
eyelids, shared gaze, brows and pixel-derived teeth. Step 8 is next: authored
|
||||
parameter controls and their scoped recomputation.
|
||||
eyelids, shared gaze, brows and pixel-derived teeth. Step 8's data model now
|
||||
has stable feature identity, feature-level presence, explicit eye pairs and
|
||||
shared parameter definitions. A manifest can now supply known feature absence
|
||||
intervals through measurement and freeze. Automatic per-feature detection,
|
||||
parameter editing and scoped regeneration remain.
|
||||
|
||||
## What arthur is
|
||||
|
||||
|
|
@ -326,9 +329,28 @@ minus paint. The fixed pixel thresholds remain provisional; step 8 exposes their
|
|||
parameters for tuning without changing the source track or picture timing.
|
||||
|
||||
### 8 — knobs
|
||||
The parameter UI, as leaf-addressed params in app-db
|
||||
(`clip/:cid/params/:subject/:area` — see below), so the sync layer added later has
|
||||
nothing to retrofit.
|
||||
Build the parameter model before its UI. Define each parameter once with its
|
||||
default, validation, applicable area and regeneration dependencies. Store values
|
||||
by stable subject and feature ID. Represent an eye pair as one group with one or
|
||||
two eye member IDs from the same subject; a profile view with one identified eye
|
||||
needs no invented partner. Each eye may override a pair value. Removing an eye
|
||||
from a pair materialises its effective values so playback does not change. Keep the
|
||||
existing frozen channels as renderer input; settings and provenance do not enter
|
||||
the render path.
|
||||
|
||||
Carry feature-level presence through freeze and dense channel state. The same
|
||||
feature ID covers every observed run across occlusion; a missing measurement
|
||||
has no channel value on that frame. Full-face detection is the fallback mask
|
||||
until there is a feature-level detector or authored presence data. A shared gaze
|
||||
measurement may still feed two independently identified eyes. A small manifest
|
||||
annotation can supply feature absence intervals now: the loader expands them
|
||||
before measurement, so invalid eye landmarks are ignored and gaze uses the
|
||||
visible eye. This is an input format, not a control UI or an automatic detector.
|
||||
|
||||
Use leaf-addressable settings under the clip, subject, feature and optional
|
||||
group. Retain source measurements so a setting change can regenerate affected
|
||||
channels without re-detecting footage. Time-varying parameter values and all
|
||||
parameter controls are deferred to the UI pass.
|
||||
|
||||
### 9 — backend
|
||||
Django project, the `clips` app, models for Project/Clip/Footage/Analysis/Leaf,
|
||||
|
|
@ -342,16 +364,14 @@ nobody should pre-empt by porting the old one.
|
|||
|
||||
## Two things to not foreclose
|
||||
|
||||
The feature controls will later be rethought to handle more than one face,
|
||||
periodic occlusion, stable identity across frames, and feature groups with their
|
||||
own parameters. That design can wait; two decisions here are free now and
|
||||
annoying to reverse:
|
||||
Feature controls will later handle more than one face and editing presence.
|
||||
The underlying identity, occlusion and group association model begins in step 8:
|
||||
|
||||
- **Presence is not visibility.** An occluded subject has *no value* on a frame,
|
||||
which is different from a part being hidden. Give every dense block a
|
||||
`Uint8Array` state mask per frame and let it mean *absent* as well as hidden.
|
||||
- **Params carry a subject segment.** `clip/:cid/params/:subject/:area`, with one
|
||||
subject today. Adding a path segment later touches every read and write.
|
||||
- **Presence is not visibility.** An occluded feature has *no value* on a frame,
|
||||
which is different from a part being hidden. Dense blocks carry a per-track,
|
||||
per-frame absence mask; `[:vis]` remains the sole hiding mechanism.
|
||||
- **Params carry stable identity.** A subject and its features keep their IDs
|
||||
across observation gaps. A run of visible frames is not a new identity.
|
||||
|
||||
The identity tracker, when it comes, should use the same pattern the iris and brow
|
||||
correspondences already use: vote across every frame rather than trusting one.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue