Brows: traced ring, quantised raise
A brow at 320x200 is fourteen pixels wide and three tall. Its shape carries almost nothing at that size; its height above the eye carries the expression, and a brow raise is the most legible beat on a face. So the ring is traced and the height is quantised - the split the eyes already got, where the lid is a traced feature and the iris a quantised primitive. The decomposition is the point. The traced ring already contains the real height, so adding a quantised raise on top would move the brow twice. The height is measured OUT of the ring, quantised, and put back, so the shape that renders is his at a height that snaps between a few levels and holds. Measured at both ends rather than as one number, because raise and tilt are different expressions out of one mechanism: both ends up is surprise, inner up alone is worry, inner down is anger. They share a dwell - the gaze quantiser, renamed quantizeSnap now that it has two callers - so the brow hits its pose in one frame instead of crawling into it with one end arriving before the other. Measured against the eye's corner midpoint, never its lid. Same trap the gaze origin has and worth avoiding twice: brows and lids move together constantly, so a brow that jumped on every blink would read as a tic. Rest pose from the take median rather than the neutral frame, for the reason gaze learned the hard way - that frame is picked by minimum mouth aperture and says nothing about the brows. Two correspondences resolved from geometry, not declared: which ring is which brow, and which end is the outer one. The second matters more - backwards, the tilt mirrors and worry renders as its own opposite, which reads as a directed performance choice rather than a bug and would never be questioned. Which EDGE is upper is deliberately left unresolved: it traverses the same ring the other way, an even-odd fill has no winding, and both ends still land on fixed slots. Also fixes a bug from the exposure work: the live render applied exposure to the plate and the mouth but not to the eyes, so on 2s the preview and the export disagreed. A preview that disagrees with the export is the one bug this tool cannot afford. perfIndex now exists as a named thing so the two paths cannot drift apart again. 91 -> 105 assertions. Ground truth on all four synthetic brow poses, tilt separating worry from anger by sign, a blink not faking a raise, and a shared dwell never emitting a half-raised brow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
2e4406d579
commit
03df81fa0b
8 changed files with 419 additions and 26 deletions
|
|
@ -4,7 +4,8 @@
|
|||
|
||||
import { RIGID, LIPS_OUTER, LIPS_INNER, APERTURE, FACE_OVAL, EYE_INNER,
|
||||
EYE_R_RING, EYE_L_RING, EYE_R_CORNERS, EYE_L_CORNERS,
|
||||
EYE_R_LIDS, EYE_L_LIDS, IRIS_A, IRIS_B, subsampleSlots } from './landmarks.js';
|
||||
EYE_R_LIDS, EYE_L_LIDS, IRIS_A, IRIS_B,
|
||||
BROW_A_RING, BROW_B_RING, BROW_END_0, BROW_END_1, subsampleSlots } from './landmarks.js';
|
||||
import { fitSimilarity, applySimAll, applySim, fitResidual, procrustesMean, smoothTransforms, movingAverage } from './mathutil.js';
|
||||
|
||||
// MediaPipe normalises x by image WIDTH and y by image HEIGHT, so its normalised
|
||||
|
|
@ -60,9 +61,81 @@ export function stabilize(dense, smoothRadius, aspect = 1) {
|
|||
lidsR: map(EYE_R_LIDS), lidsL: map(EYE_L_LIDS),
|
||||
irisA: hasIris ? map(IRIS_A) : null,
|
||||
irisB: hasIris ? map(IRIS_B) : null,
|
||||
browA: map(BROW_A_RING), browB: map(BROW_B_RING),
|
||||
};
|
||||
}
|
||||
|
||||
/* ---------- brows ---------- */
|
||||
|
||||
// Two correspondences resolved from geometry, for the same reason the iris
|
||||
// pairing is: a wrong guess here is survivable enough to escape notice.
|
||||
//
|
||||
// Which ring is which brow follows MediaPipe's left/right naming, which is the
|
||||
// naming that would have put the irises on the wrong eyes. Which END of a ring
|
||||
// is the OUTER one matters more: get it backwards and the tilt mirrors, so
|
||||
// inner-up "worried" renders as outer-up, which is a different expression
|
||||
// rather than a broken one. It would read as a directed performance choice and
|
||||
// never be questioned.
|
||||
//
|
||||
// Both are decided by voting across every frame against landmarks already known
|
||||
// to be rigid, so one bad detection cannot swing them.
|
||||
export function pairBrows(stab) {
|
||||
const N = stab.browA.length;
|
||||
const cen = (ring) => {
|
||||
let x = 0;
|
||||
for (const p of ring) x += p.x;
|
||||
return x / ring.length;
|
||||
};
|
||||
let side = 0, ends = 0;
|
||||
for (let f = 0; f < N; f++) {
|
||||
const cR = mid(stab.cornersR[f][0], stab.cornersR[f][1]).x;
|
||||
const cL = mid(stab.cornersL[f][0], stab.cornersL[f][1]).x;
|
||||
side += Math.abs(cen(stab.browA[f]) - cR) < Math.abs(cen(stab.browA[f]) - cL) ? 1 : -1;
|
||||
|
||||
// EYE_R_CORNERS is [outer, inner], so this asks whether slot 0 of the ring
|
||||
// sits nearer the eye's outer corner than its inner one.
|
||||
const ring = side > 0 ? stab.browA[f] : stab.browB[f];
|
||||
const co = side > 0 ? stab.cornersR[f] : stab.cornersL[f];
|
||||
const s0 = ring[BROW_END_0[0]];
|
||||
ends += Math.abs(s0.x - co[0].x) < Math.abs(s0.x - co[1].x) ? 1 : -1;
|
||||
}
|
||||
return {
|
||||
right: side > 0 ? 'browA' : 'browB',
|
||||
left: side > 0 ? 'browB' : 'browA',
|
||||
outerAtSlot0: ends > 0,
|
||||
};
|
||||
}
|
||||
|
||||
// Brow height above its own eye, at each end, in eye widths.
|
||||
//
|
||||
// Measured against the eye's CORNER MIDPOINT, not the lid: the corners are
|
||||
// rigid, so a blink cannot read as a brow raise. That is the same trap the gaze
|
||||
// origin has and it is worth avoiding twice - brows and lids move together
|
||||
// constantly, and a brow that jumped on every blink would look like a tic.
|
||||
//
|
||||
// Two ends rather than one height, because raise and tilt are different
|
||||
// expressions built from the same measurement: both ends up is surprise, inner
|
||||
// up alone is worry, inner down is anger. One number could not tell them apart.
|
||||
export function browSignals(stab) {
|
||||
const N = stab.browA.length;
|
||||
const pairing = pairBrows(stab);
|
||||
const endOuter = pairing.outerAtSlot0 ? BROW_END_0 : BROW_END_1;
|
||||
const endInner = pairing.outerAtSlot0 ? BROW_END_1 : BROW_END_0;
|
||||
const out = { R: [], L: [], pairing };
|
||||
|
||||
for (let f = 0; f < N; f++) {
|
||||
for (const [side, corners] of [['R', stab.cornersR], ['L', stab.cornersL]]) {
|
||||
const ring = stab[pairing[side === 'R' ? 'right' : 'left']][f];
|
||||
const c = mid(corners[f][0], corners[f][1]);
|
||||
const w = dist(corners[f][0], corners[f][1]);
|
||||
const at = (pair) => (ring[pair[0]].y + ring[pair[1]].y) / 2;
|
||||
// y grows downward, so a brow ABOVE the eye gives a positive raise.
|
||||
out[side].push({ x: (c.y - at(endOuter)) / w, y: (c.y - at(endInner)) / w });
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/* ---------- eyes ---------- */
|
||||
|
||||
const mid = (a, b) => ({ x: (a.x + b.x) / 2, y: (a.y + b.y) / 2 });
|
||||
|
|
@ -200,7 +273,10 @@ export function gazeOrigin(gazeRaw, mode = 'median', neutral = 0, radius = 2) {
|
|||
return { x: mid1(gazeRaw.map((g) => g.x)), y: mid1(gazeRaw.map((g) => g.y)) };
|
||||
}
|
||||
|
||||
// Snap gaze onto a grid, then require a new cell to hold before it takes.
|
||||
// Snap a two-channel track onto a grid, then require a new cell to hold before
|
||||
// it takes. Gaze uses it for (x, y); brows use it for (outer raise, inner raise),
|
||||
// where sharing the dwell is the point - a brow whose inner end arrived a frame
|
||||
// before its outer end would crawl instead of snapping.
|
||||
//
|
||||
// This is the "Primitive - quantised" row of the part table in docs/design.md,
|
||||
// and it is not a stylisation imposed on the truth: real eyes move in saccades,
|
||||
|
|
@ -212,9 +288,9 @@ export function gazeOrigin(gazeRaw, mode = 'median', neutral = 0, radius = 2) {
|
|||
// The dwell is what stops a gaze parked on a cell boundary from chattering
|
||||
// between two cells forever. It is meaningless without a grid, because
|
||||
// continuous values never repeat, so step 0 short-circuits both.
|
||||
export function quantizeGaze(gaze, step, dwell) {
|
||||
if (!(step > 0)) return gaze.map((g) => ({ x: g.x, y: g.y }));
|
||||
const q = gaze.map((g) => ({
|
||||
export function quantizeSnap(track, step, dwell) {
|
||||
if (!(step > 0)) return track.map((g) => ({ x: g.x, y: g.y }));
|
||||
const q = track.map((g) => ({
|
||||
x: Math.round(g.x / step) * step,
|
||||
y: Math.round(g.y / step) * step,
|
||||
}));
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue