noe/arthur
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Teeth from mouth-interior image content

Browse source 
MediaPipe has no landmarks inside the lips, so teeth have to come from pixels.
Tracing the bright blob would give a new contour every frame with no vertex
correspondence - the exact boil docs/roto-puppet.md warns about.

So the measurement yields a scalar, not a shape: Otsu within the cavity, scanned
from the top for where the bright run stops, giving one line height per frame.
The teeth polygon is the inner lip ring clipped to that line, so the silhouette
is always the mouth's own shape and cannot disagree with the lips around it,
and the only per-frame variable is a single number that smooths trivially.

Presence gets hysteresis and minimum dwell, as plate selection does: a teeth
block blinking on and off for single frames is worse than one simply absent.

Tongue is not implemented. The same scalar approach would apply, gated on
redness rather than brightness, but it is not visible in the test footage - the
cavity reads dark with a bright upper-teeth band and nothing else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Your Name 2026-09-24 15:12:09 -04:00
parent 29b0bd6690
commit 941022b69f
5 changed files with 242 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

23
README.md
View file

@ -62,6 +62,29 @@ is hand-drawn head plates, which this tool does not yet do.
| closed-mouth cut | Aperture below which the mouth interior is emitted as `hidden`. |
| suggest tolerance | Max head movement before a new plate drawing is required. Affects **Suggest** only. |
## Teeth
MediaPipe has no landmarks inside the lips: the inner ring bounds the cavity and
everything within it is just pixels. So teeth come from the image — but tracing
the bright blob would produce a new contour every frame with no vertex
correspondence, which is precisely the boil the design exists to avoid.
So the extraction yields a **scalar, not a shape**. The teeth polygon is the inner
lip ring clipped to a horizontal line, and only that line's height is measured
(Otsu threshold within the cavity, scanned from the top). The silhouette is
therefore always the mouth's own shape — stable by construction — and the only
per-frame variable is one number, which smooths trivially. It is also how the
shape gets drawn by hand: a band bounded by the lip.
Presence uses hysteresis plus a minimum dwell, the same treatment plate selection
gets, because a teeth block that blinks on and off for single frames is worse
than one that is simply absent.
**Tongue** would work the same way — a shape filling the lower cavity, gated on a
redness rather than a brightness statistic. Not implemented, because it is not
visible in the test footage: the cavity reads as dark with a bright upper-teeth
band and nothing else.
## The plate is reference, not art
The plate layer has several representations because its job changes. Cycle with