e497e9ac4d
Three parts per eye, stacked the way the mouth is - dark lash ring, sclera inside it, iris inside that, square pupil in the iris. A blink then costs nothing: when the lid shuts the traced ring goes flat and the lash line collapses to a lens, which is a closed eye, drawn correctly, for free. Lids are a FEATURE, rotoscoped like the mouth: head-local, a key on every frame, the same contour avg knob. The iris is a PRIMITIVE - a disc at a quantised position - and that is where the stylisation lives. Line of sight. Gaze is the iris centre relative to the midpoint of the eye's two corners, in units of corner distance. Both corners are in RIGID, so the origin and the scale are immune to the performance being measured; against the lid ring's centroid instead, every blink would drag the origin down and fake a glance at the floor on exactly the frames where the eye is most visible. Both eyes share one gaze - at this size the difference between the two measurements is noise, not vergence, and independent per-eye noise reads as wall-eyed immediately. Openness stays per-eye so a wink survives. Gaze is then quantised to a pixel grid with a dwell, which is not a stylisation imposed on the truth: real eyes move in saccades, and the smooth drift left in the measurement is tracker noise plus head-compensation error. Snapping to a grid removes the noise and recovers the saccade in one operation. The iris is placed in the frame of the already-smoothed, already-subsampled lid ring - slots 0 and 8 of a 16-slot ring are the corners, and subsampling to any even budget keeps them at 0 and n/2 - so it cannot drift relative to its own eye. Size is authored from the take mean, never remeasured per frame: a radius that breathes by a fraction of a pixel flickers a pixel on and off around the whole silhouette. iris anchor toggles steady/free/locked, because how much the eye wanders turns out to be an aesthetic choice and not only a correctness one. Blinking gets hysteresis and a dwell like the teeth, plus one knob they do not have: blink hold. A blink is one frame at 12fps and a single frame of closed eye reads as a dropped frame, so once the eye shuts it stays shut long enough to be legible. Detection accuracy is not the problem; legibility is. The pupil is a square because at three pixels a circle is a plus sign with the corners gnawed off, and it changes shape as it moves. Drawn from a rounded centre shared with the iris so it is exactly its nominal size on every frame. Iris/pupil clip by colour key against the indexed buffer, the way Animator Pro would: the lid crops the iris at extreme gaze for free, so nothing has to clamp the gaze, which would flatten the performance at the extremes that carry it. Which iris block belongs to which eye is RESOLVED from geometry, not declared. A swap looks almost right - each eye still has a disc roughly where it belongs - so it survives an eyeball and then reads as a subtly wall-eyed character forever. Voted across every frame; the test feeds a deliberately swapped track. Also: exposure. Aesthetic sparseness was set by the extraction rate, which made the timing a property of a directory of PNGs - auditioning 12 against 24 meant re-ripping and re-detecting the whole clip. It is now a render-time grid, on 1s/2s/3s/4s, so the dense track keeps everything and the audio clock is untouched. The take format already carried an exposure field; it was never driven. Everything rides the same grid, because a head cutting on the odd frames while the mouth cuts on the even ones reads as two performances laid over each other. 41 -> 91 assertions. The load-bearing new ones: the iris pairing follows a swapped track, a blink does not fake a change of gaze, a stencilled disc cannot spill past its clip, a 3px pupil is 3x3 at every sub-pixel centre, and exposure never reads a pose from the future. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
97 lines
4.3 KiB
JavaScript
97 lines
4.3 KiB
JavaScript
// MediaPipe FaceLandmarker index tables.
|
|
// Ring arrays are ORDERED traversals, not raw connection sets: vertex position
|
|
// within a ring is the vertex's identity, and every downstream stage depends on
|
|
// that ordering being stable. See docs/design.md, "Fixed topology".
|
|
|
|
// Rigid landmarks for the similarity fit. Eye corners, nose bridge, nose tip.
|
|
// Nothing here may be a feature that moves under performance: including the
|
|
// mouth or brows bleeds performance into the stabilization.
|
|
export const RIGID = [33, 133, 362, 263, 168, 6, 1];
|
|
|
|
// Outer lip ring, clockwise from the right corner over the top.
|
|
// index 0 = right corner, 5 = top centre, 10 = left corner, 15 = bottom centre.
|
|
export const LIPS_OUTER = [
|
|
61, 185, 40, 39, 37, 0, 267, 269, 270, 409,
|
|
291, 375, 321, 405, 314, 17, 84, 181, 91, 146,
|
|
];
|
|
|
|
// Inner lip ring, same orientation and the same four cardinal positions.
|
|
export const LIPS_INNER = [
|
|
78, 191, 80, 81, 82, 13, 312, 311, 310, 415,
|
|
308, 324, 318, 402, 317, 14, 87, 178, 88, 95,
|
|
];
|
|
|
|
// Inner upper / lower lip centres. Their separation is the aperture signal that
|
|
// decides whether the mouth interior is present at all.
|
|
export const APERTURE = [13, 14];
|
|
|
|
// Face oval, used only to derive the placeholder plate in v1.
|
|
export const FACE_OVAL = [
|
|
10, 338, 297, 332, 284, 251, 389, 356, 454, 323, 361, 288,
|
|
397, 365, 379, 378, 400, 377, 152, 148, 176, 149, 150, 136,
|
|
172, 58, 132, 93, 234, 127, 162, 21, 54, 103, 67, 109,
|
|
];
|
|
|
|
// Eye corners, for the calibration box and for reporting fit residual.
|
|
export const EYE_INNER = [133, 362];
|
|
|
|
// Pick `n` slots from a ring of `len` by even spacing. Returns RING POSITIONS,
|
|
// not landmark ids: positions are the vertex identity downstream, and mapping ids
|
|
// back to positions with indexOf would silently pick the wrong slot if a table
|
|
// ever repeated an id.
|
|
//
|
|
// For even n this naturally lands on the cardinal positions (corners and lip
|
|
// centres) of a 20-point ring. Fixed indices, never adaptive decimation: the
|
|
// vertex at slot k means the same thing on every frame of the shot.
|
|
export function subsampleSlots(len, n) {
|
|
const out = [];
|
|
for (let k = 0; k < n; k++) out.push(Math.round((k * len) / n) % len);
|
|
return out;
|
|
}
|
|
|
|
export function subsampleRing(ring, n) {
|
|
return subsampleSlots(ring.length, n).map((s) => ring[s]);
|
|
}
|
|
|
|
// ---- eyes ----
|
|
//
|
|
// Eyelid rings, under the same contract as the lip rings: ORDERED traversals
|
|
// where slot position IS vertex identity. Both eyes start at the OUTER corner
|
|
// and go over the UPPER lid first, so slot k means the same anatomy on both
|
|
// sides. On a 16-slot ring that puts the four cardinals exactly on the four
|
|
// quarter slots - 0 outer corner, 4 upper lid centre, 8 inner corner, 12 lower
|
|
// lid centre - so every even vertex budget lands on real landmarks.
|
|
//
|
|
// The two rings traverse opposite directions on screen, because they are
|
|
// mirrored anatomy described the same way. Nothing downstream cares: an
|
|
// even-odd fill has no winding, and ring SIMPLICITY is what is asserted.
|
|
export const EYE_R_RING = [
|
|
33, 246, 161, 160, 159, 158, 157, 173,
|
|
133, 155, 154, 153, 145, 144, 163, 7,
|
|
];
|
|
export const EYE_L_RING = [
|
|
263, 466, 388, 387, 386, 385, 384, 398,
|
|
362, 382, 381, 380, 374, 373, 390, 249,
|
|
];
|
|
|
|
// Outer, inner corner per eye. All four are also in RIGID, and that is the
|
|
// point: the eye's reference frame is built only from landmarks that do not
|
|
// move under performance, so a blink cannot be mistaken for a change of gaze.
|
|
export const EYE_R_CORNERS = [33, 133];
|
|
export const EYE_L_CORNERS = [263, 362];
|
|
|
|
// Upper and lower lid centres. Their separation over the corner distance is the
|
|
// openness signal that decides whether the eye is shut - the same shape of
|
|
// measurement as APERTURE is for the mouth, but normalised, so one threshold
|
|
// carries across takes and faces.
|
|
export const EYE_R_LIDS = [159, 145];
|
|
export const EYE_L_LIDS = [386, 374];
|
|
|
|
// The two iris blocks the refined mesh appends: centre first, then four ring
|
|
// points. WHICH BLOCK BELONGS TO WHICH EYE IS NOT DECLARED HERE - MediaPipe's
|
|
// own "left"/"right" is viewer-relative in some docs and subject-relative in
|
|
// others, and a swap looks almost right, so it would survive an eyeball and
|
|
// then read as a permanently wall-eyed character. pipeline.js resolves it from
|
|
// the geometry instead.
|
|
export const IRIS_A = [468, 469, 470, 471, 472];
|
|
export const IRIS_B = [473, 474, 475, 476, 477];
|